Maybe that's a good opportunity to gauge interest in something I've been working on for my own codes, but which could be used in PETSc. It's not quite ready even in its separate form, but starting to work okay on my side.<br>
<br>It's basically an object model for C, and in many senses quite similar to PetscObj. (I haven't looked at all the intricacies of PetscObj, but what I have is quite flexible, so I'm pretty sure things could be made to work). There are "classes", think for Vec, Mat. So a Vec object is an instance of the Vec class, and has its own C type (a pointer to a struct). Classes can have subclasses a.k.a. types, that is they can at run-time internally be switched between different implementations, think a dense or a sparse matrix, etc. There's create(), destroy() and ref counting.<br>
<br>Okay, so that sounds exactly like petsc (it pretty much is). But there's more: There are descriptors for classes and subclasses (types), which for the most part of course are just structs -- but those descriptors list certain members of the struct, for example parameters. So after parameters are registered, the object model knows about them and can do certain things without the need for additional repetitive code, e.g. parameter parsing from the command line and a "view" method that will print out the values. In fact, these descriptors help in saving and restoring the state of an object to a file, too. To give you some idea, the whole thing looks somewhat like (not too useful, since a class normally wants its own methods in addition to the standard ones)<br>
<br>// in the header<br><br><span style="font-family: courier new,monospace;">extern struct mrc_class mrc_class_mrc_io;</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"></span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">MRC_OBJ_DEFINE_STANDARD_METHODS(mrc_io, struct mrc_io);</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">// class specific methods go here</span><br>
<br>// private header<br><br><span style="font-family: courier new,monospace;">struct mrc_io {</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> struct mrc_obj obj;</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> char *outdir;</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">
char *basename;</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">};</span><br style="font-family: courier new,monospace;"><br>// implementation<br><br><span style="font-family: courier new,monospace;">static void</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">mrc_io_init()</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">{</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">#ifdef HAVE_HDF5_H</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> libmrc_io_register_xdmf();</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">#endif</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> libmrc_io_register_ascii();</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> libmrc_io_register_combined();</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">}</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">#define VAR(x) (void *)offsetof(struct mrc_io, x)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">static struct param mrc_io_descr[] = {</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> { "outdir" , VAR(outdir) , PARAM_STRING(".") },</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> { "basename" , VAR(basename) , PARAM_STRING("run") },</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> {},</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">};</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">#undef VAR</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">struct mrc_class mrc_class_mrc_io = {</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> .name = "mrc_io",</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> .size = sizeof(struct mrc_io),</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> .param_descr = mrc_io_descr,</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> .init = mrc_io_init,</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> .setup = _mrc_io_setup,</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">};</span><br style="font-family: courier new,monospace;"><br>which is basically all to provide you with a mrc_io class, though not too useful until you add in your own methods. The class will have standard methods create(), destroy(), view(), set_type(), set_from_options(), setup() etc. setup() in the example above is actually overridden, though not shown. As it is in C, there's some ugliness in casting types involved when implementing / overriding methods, but not too bad (IMO).<br>
<br>Users now can set an option with 'mrc_io_set_param_string(io, "basename", "myname");' Of course that's not as fast as a a direct assignment (which requires knowledge of the internal struct though), but setting parameters is not normally performance critical in the first place. Of course a given class or subclass can implement their own methods which bypass "pass the option name as string" if needed for some reason. "--mrc_io_basename myname" will also automatically work (this was the primary goal, the ..._set_param_<type>() methods are just a side benefit which should alleviate the need to write custom code for every integer / float / choice / ... parameter. I realize there is sometimes the need to customize what happens when a certain parameter is set rather than just assigning the value, that wouldn't be a problem to add. The descriptor table would also be the place to add the help texts.<br>
<br>Not all of the following is finished, but there's more: Objects can have a name assigned (and should have a name assigned, maybe that should even be mandatory). The name is automatically used for e.g. parsing the command line, so the option above then might become "--my_io_name_basename ...". Objects can be organized in a hierarchy (but they don't have to be). If a PC has a KSP as its parent, the option name will automatically be "--ksp_pc_...", or "--<ksp_name>_<pc_name>_..." if the objects have been named. Objects can also have references to other objects (a Vec may have a reference to a DA, but it wouldn't want to be its parent). This kind of state can be written to disk (currently in HDF5, but the interface is more generic). Standard state is written and restored automatically, e.g., parameters, children and parent objs as well as references to other objects. For a given class or subclass, one only needs to add in the code which writes the remaining specific state. There's certainly a caveat here about initialization/setup order when rereading such a thing from disk, which can be tricky and not happen as automatically as one would like.<br>
<br>Everything can be overridden, so I'm quite sure the model can deal with everything that is currently done in petsc, but it of course only makes sense to go there if much of petsc follows a standard model, which I think it does. Then a lot of redundant code can be removed and replaced by a somewhat self-describing object type which generalizes general housekeeping and improves consistency, while allowing to keep the fast paths untouched.<br>
<br>--Kai<br><br><br><div class="gmail_quote">On Fri, Jan 7, 2011 at 10:29 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:petsc-maint@mcs.anl.gov">petsc-maint@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
I have bitten the HDF5 bullet. From Python, it is great since you can use<div>PyTables, which is an excellent package. I am still getting the PETSc</div><div>support there, but the Vec output is fine.</div><div><br></div>
<div>
Matt<div><div></div><div class="h5"><br><br><div class="gmail_quote">On Fri, Jan 7, 2011 at 3:48 AM, Jed Brown <span dir="ltr"><<a href="mailto:jed@59a2.org" target="_blank">jed@59a2.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div><div></div><div>On Thu, Jan 6, 2011 at 16:04, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
<br>
> > I guess one could store some kind of a catalog in the .info file that<br>
> would allow reloading object by name in any order. Do you have a feeling<br>
> that anybody other than me could be interested?<br>
><br>
<br>
In the long term, we need something better. PETSc binary formats can't<br>
carry enough metadata to be usefully plottable without user interaction. I<br>
think I'm still in favor of a SQLite backend (for simplicity and<br>
versatility) indexing binary files written using MPI-IO. Most visualization<br>
and metadata tasks should be answerable in 1-line SQL queries instead of<br>
custom page-long graph traversals.<br>
<br>
>>> On a different but not unrelated topic, Imtiaz who had started looking<br>
> at the vtk IO had to leave LSU after our international services office<br>
> screwed up his immigration paperwork... I am going to work with another<br>
> student, Matt Kemp, on this project. Matt happens to also work part time for<br>
> ANL and will be at MCS next week. I'll ask him to stop by and introduce<br>
> himself to the group. He is also a good python / web programer so I was<br>
> thinking of asking him to have a look at item 3 of the proposed project<br>
> list: "Converting PetscLogViewPython() to generate JSON instead and<br>
> developing Python parsers for quickly generating nice tables of performance<br>
> details from runs or groups of runs." Is this still open?<br>
><br>
<br>
Yes, but have a look at petscplot (<br>
<a href="https://github.com/jedbrown/petscplot/wiki/PETSc-Plot" target="_blank">https://github.com/jedbrown/petscplot/wiki/PETSc-Plot</a>) for some ideas on<br>
plotting. (petscplot parses plain ASCII output and creates a few plot<br>
styles, see <a href="https://github.com/jedbrown/tme-ice/blob/master/make.sh#L5" target="_blank">https://github.com/jedbrown/tme-ice/blob/master/make.sh#L5</a> for<br>
more example invokations.)<br>
<br>
</div></div><br><div class="gmail_quote">On Thu, Jan 6, 2011 at 16:04, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div>> I guess one could store some kind of a catalog in the .info file that would allow reloading object by name in any order. Do you have a feeling that anybody other than me could be interested?<br></div></blockquote>
<div><br></div><div>In the long term, we need something better. PETSc binary formats can't carry enough metadata to be usefully plottable without user interaction. I think I'm still in favor of a SQLite backend (for simplicity and versatility) indexing binary files written using MPI-IO. Most visualization and metadata tasks should be answerable in 1-line SQL queries instead of custom page-long graph traversals.</div>
<div><br></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div>
>>> On a different but not unrelated topic, Imtiaz who had started looking at the vtk IO had to leave LSU after our international services office screwed up his immigration paperwork... I am going to work with another student, Matt Kemp, on this project. Matt happens to also work part time for ANL and will be at MCS next week. I'll ask him to stop by and introduce himself to the group. He is also a good python / web programer so I was thinking of asking him to have a look at item 3 of the proposed project list: "Converting PetscLogViewPython() to generate JSON instead and developing Python parsers for quickly generating nice tables of performance details from runs or groups of runs." Is this still open?</div>
</blockquote></div><br><div>Yes, but have a look at petscplot (<a href="https://github.com/jedbrown/petscplot/wiki/PETSc-Plot" target="_blank">https://github.com/jedbrown/petscplot/wiki/PETSc-Plot</a>) for some ideas on plotting. (petscplot parses plain ASCII output and creates a few plot styles, see <a href="https://github.com/jedbrown/tme-ice/blob/master/make.sh#L5" target="_blank">https://github.com/jedbrown/tme-ice/blob/master/make.sh#L5</a> for more example invokations.)</div>
<br></blockquote></div><br><br clear="all"><br></div></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</div>
</blockquote></div><br><br clear="all"><br>-- <br>Kai Germaschewski<br>Assistant Professor, Dept of Physics / Space Science Center<br>University of New Hampshire, Durham, NH 03824<br>office: Morse Hall 245E<br>phone: +1-603-862-2912<br>
fax: +1-603-862-2771<br><br>