[petsc-dev] Need mechanism to set LocalToGlobal mapping in a DMShell

[Jed Brown]

On Thu, Feb 21, 2013 at 1:23 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> On Thu, Feb 21, 2013 at 1:10 PM, Matthew Knepley <knepley at gmail.com>wrote:
>
>> Yes, he needs the local to global scatters. Rich, I think the easiest way
>> to do this now may be
>> to define your data layout as a PetscSection (these are all local) using
>> DMSetDefaultSection(),
>> and record who owns which point using a PetscSF with DMSetPointSF().
>>
>> Using this information, the DM will automatically construct a scatter
>> between local and global,
>> and the CreateLocal/GlobalVector() functions will also work right. This
>> is what its is designed for.
>>
>> Does this make sense?
>>
>
>
> Rich, this will work fine if you don't mind having PETSc do the
> communication. It will be done using PetscSFBcastBegin/End, which should
> have similar performance to VecScatterBegin/End. (I should do a performance
> test of that.) If you want to use your own internal communication routines,
> then we need to add the DMShell functions to get your functions called.
>

It occurs to me that the description above might be pretty confusing. The
idea is that we start with your local space and lump it together into
(possibly variable-size) chunks, usually one per topological point in the
mesh. Matt would describe this chunking with a PetscSection where the
chunks are indexed by "point" (an integer).

The next component is that we have to describe how the points fit together,
so we make a PetscSF where the "roots" are the global points, referred to
by (rank, offset) pairs or global indices, and the "leaves" are the local
space. You use PetscSFSetGraph() to create this, which Matt calls the
"Point SF".

If you tell the DM about these two things by using DMSetDefaultSection()
and DMSetPointSF(), then DM will create the SF to communicate the Vec data
(using simple composition). You could also just use DMSetDefaultSF()
directly, in which case you don't ever need to touch a PetscSection.


The alternative to use VecScatter is also fine, but sometimes it is easier
to use PetscSF. The Point SF, in particular, is very useful for handling
integer data in the setup phase. Its API is richer than VecScatter, in
addition to supporting more data types.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130221/3f3a4af6/attachment.html>