[petsc-users] difference between local and global vectors

[Dave May]

On 19 August 2015 at 03:38, TAY wee-beng <zonexo at gmail.com> wrote:

> Hi,
>
> I am using DA. For e.g.
>
> DM da_u
>
> call
> DMDACreate3d(MPI_COMM_WORLD,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DMDA_STENCIL_STAR,size_x,size_y,&
>
>
> size_z,1,PETSC_DECIDE,PETSC_DECIDE,1,stencil_width,lx,PETSC_NULL_INTEGER,PETSC_NULL_INTEGER,da_u,ierr)
>
> call DMCreateGlobalVector(da_u,u_global,ierr)
>
> call DMCreateLocalVector(da_u,u_local,ierr)
>
> To update the ghost values, I use:
>
> call DMLocalToLocalBegin(da_u,u_local,INSERT_VALUES,u_local,ierr)
>
> call DMLocalToLocalEnd(da_u,u_local,INSERT_VALUES,u_local,ierr)
>


This is incorrect.
The manpage for DMLocalToLocal clearly says "Maps from a local vector
(including ghost points that contain irrelevant values) to another local
vector where the ghost points in the second are set correctly."
To update ghost values from a global vector (e.g. to perform the scatter)
you need to use DMGlobalToLocalBegin() , DMGlobalToLocalEnd().



>
> It seems that I don't need to use global vector at all.
>
> So what's the difference between local and global vector?
>


* Local vectors contain ghost values from any neighbouring MPI processes.
They are always defined over PETSC_COMM_SELF.
* Global vectors store the DOFs assigned to each sub-domain. These will
parallel vectors defined over the same communicator as your DM

Thus, you use local vectors to compute things like the sub-domain
contribution to (i) a non-linear residual evaluation or (ii) a
sparse-matric vector product.
You use global vectors together with linear and non-linear solvers as these
vectors.

If your stencil width was zero (in your DMDACreate3d() function call), then
the would be no ghost values to communicate between neighbouring MPI
processes. Hence, the entries in the following two arrays LA_u_local[],
LA_u[] would be identical
  VecGetArrayRead(u_local,&LA_u_local);
and
  VecGetArrayRead(u,&LA_u);

That said, u_local would still be of type VECSEQ, where as u would be of
type VECMPI.




>
> When will I need to use?:
>
> call DMGlobalToLocalBegin(da_u,u_global,INSERT_VALUES,u_local,ierr)
>
> call DMGlobalToLocalEnd(da_u,u_global,INSERT_VALUES,u_local,ierr)
>

See points (i) and (ii) above from common use cases.

Thanks,
  Dave





>
> --
> Thank you
>
> Yours sincerely,
>
> TAY wee-beng
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150819/70e70e01/attachment.html>