[petsc-dev] Ghost update in VecAssembly?

[Matthew Knepley]

On Tue, Mar 1, 2016 at 9:50 AM, Jed Brown <jed at jedbrown.org> wrote:

> Matthew Knepley <knepley at gmail.com> writes:
> > I am trying to understand what you are saying here. I have a vector on 2
> > procs, divided as
> >
> > p0  |----interior---|--ghost--|
> >
> > p1  |----interior---|--ghost--|
>
> Remember that the global Vec consists of just the interiors.  The local
> form is a sequential Vec that includes ghosts.
>
> > Now I calculate some values in the interior on both procs using
> > VecGetArray(),
> >
> > p0  |xxxxinteriorxxx|--ghost--|
> >
> > p1  |xxxxinteriorxxx|--ghost--|
>
> If you do it this way, then the global Vec is complete without needing
> to communicate anything.
>
> > What is wrong with calling VecGhostUpdate() here to fill in the
> > ghosts?
>
> The ghost values aren't part of the global Vec, so it's just needless
> communication.  But more insidiously, it encourages an interface where
> the input Vec is assumed to have updated ghost values (a "hidden"
> property).
>
> > Is your contention that I should have also calculated the ghosts?
>
> No.  But if you have a FE discretization then you're presumably using
> ADD_VALUES.  With VecGhost, the natural thing is to put your entries in
> the local form and use VecGhostUpdate(x,ADD_VALUES,SCATTER_REVERSE).
>

Okay, this shows that we always need to specify exactly which vector, local
or global,
we are talking about (and also why I don't use VecGhost).

I was confusing the VecGhostUpdate() above which puts local values in the
global vector
with the update which would refresh the local portion. The former update
should be called
automatically during assembly, which already happens in the
FormFunctionLocals, and
would make sense to have in VecAssembly() for VecGhost I think. However, my
data
arrows, and those of the questioner, were loading data into the ghost
portion, which does
not make sense for VecAssembly().

So I think, on balance, we should not do this because the relevant assembly
is done by
a better, more transparent mechanism already.

  Matt


> Note that you would need a subsequent
> VecGhostUpdate(x,INSERT_VALUES,SCATTER_FORWARD) if you wanted the local
> form to be consistent with the now-assembled global form.  But the
> algebraic solvers don't need that, so it's only relevant if user code
> will be called on that Vec prior to modification of any type by the
> algebraic solver, and having user code assume the ghost values are
> current is a dangerous assumption that will make your code fragile and
> disgusting.
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20160301/c9d062f8/attachment.html>