[petsc-dev] questions on new include organization

[Matthew Knepley]

On Wed, Feb 13, 2013 at 12:45 AM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

>
> On Tue, Feb 12, 2013 at 11:34 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>>  Is it? Maybe a more general definition of PetscSection would be better
>> as map from (int) -> IS   Matt's current usage is very restricted to only
>> contiguous blocks of elements in a group. Need that be it? Should that be
>> it?
>
>
> There is some hackery to ascribe "field" structure to the output of the
> map, but I'm skeptical of whether it belongs at this level.
>
> The granularity is normally very small so we certainly don't want to pay
> the memory or performance overhead of an IS for each output. Although I'm
> sure we can think up scenarios where that flexibility would be useful, I
> think generality in the outputs here would add significant complexity with
> limited practical return.
>
> Note that PetscSection is currently accessed using function calls in inner
> loops. This already accounts for a noticeable amount of time for a CFD
> application. So we either need a coarser grained access API or we need a
> non-polymorphic accessor for use in hot access sites. (This is a rare case
> where indirect function call overhead matters.)
>

I think its helpful to look at the one universal use case right now:

  You would like to describe a function over a mesh, so
  associate functions with each part of the mesh and
  store coefficients of these functions.
  This happens for:
    mesh geometry
    PDE solutions and auxiliary fields (like viscosity)
    the mesh itself (the adjacency is a field over the points)

The inner loops Jed is referring to are structured like this

VecGetArray(sol, &solArray);
Loop over pieces (like cell, face, etc.)
   Get solution on this piece:
     PetscSectionGetDof(solSection, piece, &dof),
PetscSectionGetOffset(solSection, piece, &off)
     pieceArray = &solArray[off];
     Jed condenses this to DMPlexPointLocalRead(solDM, piece, solArray,
&pieceArray)

Thus this adds structure to the Vec sol by breaking it into irregular
pieces. The lower level
function Dof/Offset() are definitely necessary so you can reason about the
operations. I think
you could get away with only the array offsetting being fast, but I don't
see how you make that
fast and those slow.

Barry: > Is it? Maybe a more general definition of PetscSection would be
better as map from (int) -> IS   Matt's current usage is very restricted to
only contiguous blocks of elements in a group. Need that be it? Should that
be it?

I think you are missing the point here. I use this all over the place. This
is PetscSection+IS. You put your
non-contiguous indices in the IS and index into it using PetscSection.
PetscSection is the composable part
of this indexing-by-groups. It can be combined with IS or Vec or anything.
I think it is the right building block.

   Matt
-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130213/9a194c64/attachment.html>