[petsc-dev] Ghost values in sieve

[Chris Eldred]

Thanks- that helps a lot. If I need stencils that are larger than
closure(p) U star(p) (for a higher-order finite difference method, for
example), I assume that I need to create my own PetscSF's that
describe which points need to be ghosted?

Is there some documentation or example code that explains the theory
behind star forests?

On Sat, Aug 18, 2012 at 3:49 PM, Matthew Knepley <knepley at gmail.com> wrote:
> On Sat, Aug 18, 2012 at 4:11 PM, Chris Eldred <chris.eldred at gmail.com>
> wrote:
>>
>> Yes I think that makes sense. How do you access the associated PetscSF
>> for the DMComplex? Is it possible to define multiple PetscSF's for a
>> given DMComplex, each associated with different PetscSections?
>
>
> 1) DMGetDefaultSF() for the one for the default Section
>
> 2) Yes, but you need to create them yourself using DMCreateDefaultSF()
>
>>
>> My current understanding of using DMComplex is (for the serial case):
>>
>> 1) Create a DMComplex object, set its cones/supports, etc.
>
>
> A good guide is to look at complexcreate.c, where I create a few default
> meshes.
>
>>
>> 2) Create PetscSections that describe the layout of data across the
>> DMComplex
>> 3) Set a PetscSection as the default section
>> 4) Use DMCreateLocalVector to create the Vec objects needed for the
>> computation
>>
>> Is this correct?
>
>
> Yes
>
>>
>> In the parallel case, what else needs to be done?
>
>
> In parallel,
>
>   a) I call DMComplexDistribute(). The alternative is to create your mesh
> directly in parallel,
>       but this does not work with any mesh generator I know, and you would
> have to create
>       the PetscSF for points that this function makes automatically.
>
>   b) Now the section you create is local
>
>   c) You can use the local section and pointSF to create a global section,
> which knows about overlap
>
>   d) You can use the local and global section to make the dofSF
>
> c) and d) are done automatically for the default section (data layout).
>
>    Matt
>
>>
>>
>> -Chris
>>
>> On Fri, Aug 17, 2012 at 1:24 PM, Matthew Knepley <knepley at gmail.com>
>> wrote:
>> > On Fri, Aug 17, 2012 at 1:21 PM, Chris Eldred <chris.eldred at gmail.com>
>> > wrote:
>> >>
>> >> I was wondering how ghost values were handled when creating a
>> >> distributed (global) vector from a specific PetscSection over a
>> >> DMComplex object- ie what defines the stencils used to figure out
>> >> which values need to be ghosted? What about local vs. global indexes?
>> >
>> >
>> > This answer will have to be long because the question is mixing levels
>> > in
>> > the interface.
>> >
>> > A PetscSection is only an offset structure, so it has no designated
>> > fields
>> > for overlap information. What I do for DM is to create 2 sections: one
>> > describing the layout of the local vector, and one describing the global
>> > vector and ghosts. The global section can be created automatically given
>> > the local section and a PetscSF (successor to VecScatter).
>> >
>> > The PetscSF describes the parallel overlap, and also has scatters and
>> > reductions.
>> > This is created by the DMComplex instead of a scatter since it can
>> > handle
>> > arbitrary types, and has a better query interface. DMComplex creates a
>> > (non-overlapping) partition of some stratum of points, usually cells for
>> > FEM.
>> > Then the closure(p) U star(p) for each point is added to that partition,
>> > and
>> > the dofs on any shared point are ghosted.
>> >
>> > This is all now done using the vanilla DM interface, so
>> > DMGlobalToLocal()
>> > and LocalToGlobal() and cousins work as normal.
>> >
>> > Does that make sense?
>> >
>> >     Matt
>> >
>> >>
>> >>
>> >> -Chris
>> >>
>> >> --
>> >> Chris Eldred
>> >> DOE Computational Science Graduate Fellow
>> >> Graduate Student, Atmospheric Science, Colorado State University
>> >> B.S. Applied Computational Physics, Carnegie Mellon University, 2009
>> >> chris.eldred at gmail.com
>> >
>> >
>> >
>> >
>> > --
>> > What most experimenters take for granted before they begin their
>> > experiments
>> > is infinitely more interesting than any results to which their
>> > experiments
>> > lead.
>> > -- Norbert Wiener
>>
>>
>>
>> --
>> Chris Eldred
>> DOE Computational Science Graduate Fellow
>> Graduate Student, Atmospheric Science, Colorado State University
>> B.S. Applied Computational Physics, Carnegie Mellon University, 2009
>> chris.eldred at gmail.com
>
>
>
>
> --
> What most experimenters take for granted before they begin their experiments
> is infinitely more interesting than any results to which their experiments
> lead.
> -- Norbert Wiener



-- 
Chris Eldred
DOE Computational Science Graduate Fellow
Graduate Student, Atmospheric Science, Colorado State University
B.S. Applied Computational Physics, Carnegie Mellon University, 2009
chris.eldred at gmail.com