[petsc-users] difference between local and global vectors

[TAY wee-beng]

On 19/8/2015 1:17 PM, Dave May wrote:
>
>
> On 19 August 2015 at 03:38, TAY wee-beng <zonexo at gmail.com 
> <mailto:zonexo at gmail.com>> wrote:
>
>     Hi,
>
>     I am using DA. For e.g.
>
>     DM da_u
>
>     call
>     DMDACreate3d(MPI_COMM_WORLD,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DMDA_STENCIL_STAR,size_x,size_y,&
>
>     size_z,1,PETSC_DECIDE,PETSC_DECIDE,1,stencil_width,lx,PETSC_NULL_INTEGER,PETSC_NULL_INTEGER,da_u,ierr)
>
>     call DMCreateGlobalVector(da_u,u_global,ierr)
>
>     call DMCreateLocalVector(da_u,u_local,ierr)
>
>     To update the ghost values, I use:
>
>     call DMLocalToLocalBegin(da_u,u_local,INSERT_VALUES,u_local,ierr)
>
>     call DMLocalToLocalEnd(da_u,u_local,INSERT_VALUES,u_local,ierr)
>
>
>
> This is incorrect.
> The manpage for DMLocalToLocal clearly says "Maps from a local vector 
> (including ghost points that contain irrelevant values) to another 
> local vector where the ghost points in the second are set correctly."
> To update ghost values from a global vector (e.g. to perform the 
> scatter) you need to use DMGlobalToLocalBegin() , DMGlobalToLocalEnd().

Hi Dave,

Thanks for the clarification although I'm still confused. Supposed I 
have a 1D vector da_u, It has size 8, so it's like da_u_array(8), with 
stencil width 1

So for 2 procs,

there will be 2 da_u_array - da_u_array(1:5) and da_u_array(4:8)

After performing some operations on each procs's da_u_array, I need to 
update 1st procs's da_u_array(5) and 2nd procs's da_u_array(4) from the 
2nd and 1st procs respectively. I simply call:

call DMLocalToLocalBegin(da_u,u_local,INSERT_VALUES,u_local,ierr)

call DMLocalToLocalEnd(da_u,u_local,INSERT_VALUES,u_local,ierr)

and it seems to be enough. I check the ghost values and they have been 
updated. So if I am not using the linear solvers, I do not need the 
global vector,is that so?
>
>
>
>     It seems that I don't need to use global vector at all.
>
>     So what's the difference between local and global vector?
>
>
>
> * Local vectors contain ghost values from any neighbouring MPI 
> processes. They are always defined over PETSC_COMM_SELF.
> * Global vectors store the DOFs assigned to each sub-domain. These 
> will parallel vectors defined over the same communicator as your DM
>
> Thus, you use local vectors to compute things like the sub-domain 
> contribution to (i) a non-linear residual evaluation or (ii) a 
> sparse-matric vector product.
> You use global vectors together with linear and non-linear solvers as 
> these vectors.
>
> If your stencil width was zero (in your DMDACreate3d() function call), 
> then the would be no ghost values to communicate between neighbouring 
> MPI processes. Hence, the entries in the following two arrays 
> LA_u_local[], LA_u[] would be identical
>   VecGetArrayRead(u_local,&LA_u_local);
> and
>   VecGetArrayRead(u,&LA_u);
>
> That said, u_local would still be of type VECSEQ, where as u would be 
> of type VECMPI.
>
>
>
>     When will I need to use?:
>
>     call DMGlobalToLocalBegin(da_u,u_global,INSERT_VALUES,u_local,ierr)
>
>     call DMGlobalToLocalEnd(da_u,u_global,INSERT_VALUES,u_local,ierr)
>
>
> See points (i) and (ii) above from common use cases.
>
> Thanks,
>   Dave
>
>
>
>
>     -- 
>     Thank you
>
>     Yours sincerely,
>
>     TAY wee-beng
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150819/f3099a8b/attachment.html>