[petsc-dev] Writing rich state

[Jed Brown]

On Tue, 23 Feb 2010 13:44:55 -0600, Dmitry Karpeev <karpeev at mcs.anl.gov> wrote:
> Yes, but what about using Spotlight programmatically (e.g., from
> PETSc) to store rich state, checkpointing, etc?  For example, I want
> to store a Vec.  How do I label it?  There maybe various user contexts
> that share it, so I'd like to label it with all of them.

Right, so I think SQL is one way to formalize this.  Ad-hoc indexing is
great for interactive use, but this system needs to be deterministic and
have somewhat more precise semantics.  This is not to say that a generic
indexer could not be used, but I think it would end up being difficult
to maintain certain invariants since the schema would end up being
encoded in conventions.  Other database paradigms may also be fine, but
the point of NoSQL is typically to weaken the supported queries and
guarantees about concurrent modification in exchange for improved
throughput/scalability.

> In a way, I don't to have to look at my home directory (or any
> directory) at all.  I just want to extract files based on a given (set
> of) label(s).

"Labels", in the gmail sense are difficult to maintain, and I mostly use
them as aliases for more sophisticated searches (by writing filters).
Keeping labels distinct from the filters that define them is really just
rubbing a particular caching scheme in everyone's face (manually applied
labels are useful for workflow).  But I get your point, the hierarchy is
just selecting one organizational scheme as special, and hard/symlinks
are band-aids to permit sharing and one-way relations existing outside
the hierarchy.  Good tools recognize this (web search, distributed SCMs,
code navigation, gmail/notmuch, filesystem indexers).

Jed