On Fri, Dec 4, 2009 at 8:52 AM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im"><br>
On Dec 4, 2009, at 2:58 AM, Jed Brown wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
On Thu, 3 Dec 2009 16:40:10 -0600, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
The vector has a pointer to the DM so the VecView() for that<br>
derived vector class has access to the DM information. The same<br>
viewer object can be used with a bunch of different sized Vecs since<br>
it gets the DM information out of each vector. With your model when<br>
the DM info is attached to the viewer you need to pass in a different<br>
viewer for different vectors (of different sizes). This is just plan<br>
nuts.<br>
</blockquote>
<br>
The DM info is not attached to the viewer, I have plain Vecs with a DM<br>
composed so it can call DMView as in your snippet below. The viewer<br>
holds no state with regard to the DM. But VecView_MyViewer_DM needs to<br>
get called somehow,<br>
</blockquote>
<br></div>
Agreed.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
it will in turn call DMView and use an internal<br>
Viewer API so that the file has a connection from the Vec to the DM.<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I say this is nonsense, it ain't more closely coupled to the viewer.<br>
</blockquote>
<br>
VecGetArray is public, all vectors have to implement it in order to<br>
function in PETSc.<br>
</blockquote>
<br></div>
This is not accurate. The SAMRAI vector class does not implement it. Yes, this means the SAMRAI vector class cannot use any PETSc built in matrix classes, but that is ok it provides its own. (note also that PETSc allows incomplete implementations of abstract classes where a new class implementaiton simply does not provide some methods and errors are generated if one tries to use them; I think this is a great design feature because it allows many-method base abstract classes, but I realize most of the computing world would be shocked and disgusted by this design feature).<div>
<div></div><div class="h5"><br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
The internal Viewer API to write the bytes of a Vec<br>
with some connection to a DM is not public in the sense that users never<br>
call it directly and no other viewers implement it. You can substitute<br>
any other Vec and this code will still write it correctly because your<br>
Vec would be broken if it didn't implement VecGetArray [1].<br>
<br>
DM currently monkey-patches Vec to set ops->view and<br>
ops->loadintovector, but it doesn't rely on the implementation of Vec<br>
(the definition of Vec_Seq/MPI is never used in src/dm) and in principle<br>
DACreateGlobalVector could call VecSetFromOptions so that it's actually<br>
operating with any Vec type (e.g. the Vec could implement it's<br>
operations on a GPU, only bringing the values to the CPU for<br>
VecGetArray). Many viewers (formats) have no special representation of<br>
vectors that come from a DM [2] which is the reason for having<br>
ops->viewnative and ops->loadintovectornative.<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
But somehow, calling VecView needs to be able to call the viewer<br>
with enough context for the viewer to wire up the metadata. Note<br>
that the coordinates of the nodes in the DM are part of the vector<br>
(it's an ALE formulation) so the metadata is nontrivial.<br>
</blockquote>
<br>
Each vector has a pointer to its DM (DMComposite etc) so its<br>
viewer can dump/read that info when needed. In fact, when writing a<br>
vector it could do something like<br>
<br>
myvecview(Vec v,Viewer viewer)<br>
{<br>
DM dm = getreferencetoDMfrom(v);<br>
DMView(dm,viewer);<br>
... now dump my vector with writes or whatever<br>
</blockquote>
<br>
My code looks exactly like this, my point is that the elided part<br>
involves only the public Vec API, but a private Viewer API. I can put<br>
this chunk of code anywhere, but it needs changes when I make private<br>
changes to the viewer, it doesn't need anything when I make private<br>
changes to the DM or Vec (it doesn't even need to know the types of<br>
those objects). It is currently sitting with my DM, mostly because I<br>
wrote the DMView first.<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
You have to admit that it would be crazy to have the DM viewer method<br>
stored inside a viewer object which seems to be what your model is.<br>
</blockquote>
<br>
I wish I could get the dispatch with a symmetric relationship, but in<br>
lieu of that, I don't especially care where the function pointer is,<br>
what matters is that there is a way to get DMView_MyViewer_YourDM<br>
called [3]. As long as dispatch is happening via PetscTypeCompare against<br>
all the known types, we are restricted in that we either have no way to<br>
use a third-party Viewer, or no way to view a third-party DM. The<br>
current choice is the former; I don't think it's inherently superior,<br>
it's just a choice of which component can be extended without modifying<br>
every object it touches.<br>
<br>
If I was distributing a binary-only viewer plugin, I would have no<br>
choice but to put all these methods in with the viewer (but I would<br>
currently have no way to get them called). Note that a viewer can be<br>
very functional without ever using a private API or looking underneath<br>
obj->data. With all other PETSc objects (DM not so much because nobody<br>
works with the generic DM interface), I can distribute a binary-only<br>
implementation which is first class after dynamic loading.<br>
<br>
My DMView is packaged up with my DM because it needs to know many<br>
internal details of the DM, but the higher level logic about how to<br>
write a time-dependent file with association between Vecs and DMs and<br>
possibly other objects is currently with my viewer. Now I need to write<br>
DMView_MyViewer_DMComposite, but I won't be able to get it called<br>
without modifying DMComposite (not okay since my viewer is not part of<br>
PETSc).<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
You know my answer. The Viewers are abstractions of file IO, ;<br>
except instead of having only methods like read()/write() bytes it has<br>
somewhat "higher level" operations. Note: these methods can include<br>
graphics operations for example that are still essentially like<br>
writes(). YOU are making the viewers into these strange "all knowing"<br>
beasts which I think is wrong.<br>
</blockquote>
<br>
Current PETSc viewers put minimal metadata and associations into a file,<br>
the application is mostly responsible for "knowing" what was in the file<br>
and wiring everything up correctly. I'm trying to get more of this in<br>
the file, partly to go with a VisIt plugin that is not tied to a<br>
specific model. I don't want to maintain a bunch of manual calls to<br>
HDF5 to walk this file in the VisIt plugin especially when this<br>
functionality is useful in other contexts [4]. I also hate C++. ;-)<br>
<br>
So I've been putting stuff like walking the header and attributes to<br>
find out what's there, selecting a time step, etc., into my viewer. The<br>
viewer is able to offer up information like which fields are available<br>
on which meshes, the plugin is a wrapper that uses this, DMLoadIntoDM<br>
(but not this elegant), VecLoadIntoVector, etc., and stuffs it into<br>
avt/vtk data structures. Note that this plugin makes no direct calls to<br>
HDF5 and uses only the public API of the Viewer, DM, and Vec.<br>
</blockquote>
<br></div></div>
Ok so you are deriving a new Viewer that implements much higher level operations that are particular to PETSc (since it knows about Vecs and DMs). This is perhaps reasonable. But the vecload/view methods still belong in the Vec object :-)<div class="im">
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Jed<br>
<br>
<br>
[1] In my view, a Vec is just an element of some finite dimensional linear<br>
space. If you want to use a special format, or stuff extra info in<br>
there to make function evaluation easier or some operation faster,<br>
that's fine, but it's still just an element of this linear space and it<br>
better satisfy the appropriate axioms. The linear space that this<br>
vector lives in is defined by the DM and we need this to do anything<br>
useful (like visualization) with the vector.<br>
</blockquote>
<br>
<br></div>
To the algebraic solvers I agree a " Vec is just an element of some finite dimensional ....." (since that is all the solvers need to know about them). But to a specific application a Vec can be thought of as the linear algebra beast PLUS all kinds of grid information etc (like the DM). So I think of a (application) Vec in the application as basically derived off of two classes: the linear algebra beast PLUS a DM (it is implemented as a isA(linear algebra beast) and hasA(DM beast)). This is why I feel so strongly that the VecView class that also reads/writes stuff related specifically to the Vec belongs with the Vec, not with the Viewer.<br>
</blockquote><div><br>I would like to point out here that the notion of a Vec as a member of a vector space is very often in adequate. What we really want is a section of a fiber bundle. This<br>is much more appropriate for most computational settings. That is why we want grid information to tag along with the Vec.<br>
<br> Matt<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Regarding where code needs to be changed/where : The reason for the "native/super" support for the VecLoadIntoVector() (which I admit is terribly crude and ugly) is exactly so that one can implement a new loader/writer directly for a specific Vec/DM combination IN THE APPLICATION CODE or another library. One does not have to add it to src/vec/vec/impls<br>
Do you have suggestions on how one could improve the "native/super" thing to satisfy you?<br>
<br>
As I said before a Viewer in PETSc is just an abstraction/generalization of a file pointer with read/write methods on the file pointer. So it disturbs me that a method in a Viewer would even know what a Vec. It leads to these loops of knowledge that make code much more complicated. I would do anything to prevent these loops of knowledge I consider them so troublesome.<br>
<font color="#888888">
<br>
Barry</font><div><div></div><div class="h5"><br>
<br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
[2] Or the Vec doesn't use information about the DM when viewing on that<br>
format if you like, I think this relationship is symmetric and<br>
conceptually owned by neither.<br>
<br>
[3] I think the location of the source for DMView_MyViewer_YourDM is<br>
mostly a packaging and distribution matter rather than a design matter.<br>
I don't see a problem with it being distributed independently of either<br>
MyViewer or YourDM (except for the technical issue of how it announces<br>
it's existence).<br>
<br>
[4] It's all slightly complicated by the fact that for visualization,<br>
it's important to be able to read chunks of the domain independently,<br>
whereas the analysis code would prefer to only define proper collective<br>
operations.<br>
</blockquote>
<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>