[petsc-users] Any overheads of VecGetArray/VecRestoreArray?

[Jed Brown]

On Tue, Mar 26, 2013 at 12:32 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
>    Dominik,
>
>      There is no copy of vector entries on the calls to
> VecGetArray/RestoreArray() so the overhead is minimal. Likely you would
> barely be able to measure a difference.
>

Calling these inside the loop requires caution, however, in case some
process may have zero elements. (This often happens on coarse grids, for
example.)

The problem is that VecRestoreArray() invalidates any cached norms. So if
you run:

VecNorm(X,NORM_2,&xnorm_before);
for (each local element) {
  VecGetArray(X,&x);
  VecRestoreArray(X,&x);
}
VecNorm(X,NORM_2,&xnorm_after);

if some process has no local entries, it will think that X has not been
modified and will not enter the MPI_Allreduce() to compute 'xnorm_after',
which is deadlock.

To avoid stuff like this, we recommend placing the VecGetArray() outside
the loop, which is consistent with its designation as "logically
collective".

You can use VecGetArrayRead() without worrying about this problem.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130326/10fea706/attachment.html>