[petsc-users] ISLocalToGlobalMapping + VecCreateGhost

[Matthew Knepley]

On Wed, Oct 2, 2013 at 12:27 AM, Mohammad Mirzadeh <mirzadeh at gmail.com>wrote:

> Hi guys,
>
> I just did something by pure guessing which seems to work and I want
> to make sure its the right thing!
>
> I have a specific layout for my vectors that look like this
>
>  --------------------------------------------------------------
> | ghost values | local values | ghost values |
>  --------------------------------------------------------------
> 0, 1, 2, ...        m, m+1, ...   m+n, m+n+1 ... N
>
> which means all nodes with index [0,m) and [m+n, N) are to be treated
> as ghost and all intermediate ones as local. Since PETSc stores the
> ghost values at the end of ghosted vec, so far I have been manually
> mapping back and forth between petsc and my application numbering.
> After profiling my code, it turned out that about 15% of run-time was
> just doing this mapping which is absolutely ridiculous!
>
> Anyway, to fix this now I set up an ISLocalToGlobalMapping object
> using my own application original local2global relation shown above.
> Then I create the petsc vec like this:
>
> // Create PetscVec
> // We do not care about the actual global index of ghost nodes at this
> point
> std::vector<PetscInt> ghost_nodes(N - n -1, 0);
> ierr = VecCreateGhost(mpicomm, n+1, num_global, ghost_nodes.size(),
> (const PetscInt*)&ghost_nodes[0], &v); CHKERRQ(ierr);
>
>
> // Apply mapping
> ierr = VecSetLocalToGlobalMapping(v, mapping); CHKERRQ(ierr);
>
> After this point I do the usual VecGetArray on the vec and set the
> values, but this time without any extra mapping ... my very simple
> test seems to be ok. Is this the correct way of using
> ISLocalToGlobalMapping?
>

Yes, this is the intention. And at a higher level, PETSc now tends to use a
DM to
organize this process. You tell the DM about your data layout (somewhat
like giving
the L2G mapping) and then you can DMGetLocalVector(), DMGetGlobalVector(),
and
DMLocalToGlobal().

  Thanks,

     Matt


> I guess I'm suspicious because VecCreateGhost is supposed to
> internally set the mapping and it is supposed to position ghost nodes
> at the end of the array which I don't want it to do...
>
> Thanks and sorry about the long email!
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20131002/aa9b88cb/attachment.html>