[petsc-users] local Vec to global Vec and global 3-D array

[Barry Smith]

  All processes have a "subblock" of the entire array; with the local Vec the subblock has the ghost values, with the global Vec it does not have the ghost points. No process has the entire array. BUT the indexing INTO the array on each process is using the GLOBAL I,J,K indices so in some sense all processes have the entire array "virtually" but they can ONLY access their chunk, if they try to access outside of their chunk it will fail.

  Barry

On May 16, 2013, at 10:34 AM, Roc Wang <pengxwang at hotmail.com> wrote:

> Thanks.  One more question. Sorry, but I am really confused here. 
> 
>> Subject: Re: [petsc-users] local Vec to global Vec and global 3-D array
>> From: bsmith at mcs.anl.gov
>> Date: Wed, 15 May 2013 15:30:07 -0500
>> CC: petsc-users at mcs.anl.gov
>> To: pengxwang at hotmail.com
>> 
>> 
>> On May 15, 2013, at 2:25 PM, Roc Wang <pengxwang at hotmail.com> wrote:
>> 
>>> Thanks, I will write the stand alone code for it. One more question about the global and local vector below.
>>> 
>>>>>>> ierr = DMCreateGlobalVector(da,&gsol3d);CHKERRQ(ierr);
>>>>>>> ierr = DMLocalToGlobalBegin(da,x,INSERT_VALUES,gsol3d);CHKERRQ(ierr);
>>>>>>> ierr = DMLocalToGlobalEnd(da,x,INSERT_VALUES,gsol3d);CHKERRQ(ierr);
>>>>>> 
>> 
>> In the above you are trying to scatter a local vector x to a global vector gsol3d. But x is a global vector. 
>> 
>>> 
>>> !*******************************
>>> The x from KSPGetSolution() IS a global vector, it is not a "local" vector? (It should be a typo :) )
>> 
>> It is a global vector.
>> 
>>> So Can I convert the global x to a 3d array directly? Thank. 
>> 
>> You can convert either a global vector or a local vector to 3d array access BUT the converted global vector has NO ghost values accessible while the converted local vector has ghost points available (after the call to DMGlobalToLocalBegin/End()).
>> 
>    if x is a global vector with da, then after the following procedure, 
> 
>    PetscScalar ***globalArray3d;
>    ierr = DMDAVecGetArray(da, x, &globalArray3d ); CHKERRQ(ierr);
> 
>    All processes have the same globalArray3d with size of global or local? For example, if M,N,P is the global dimension in each direction of the 3-d array, then the size of  globalArray3d is [0,M-1], [0, N-1] and [0,P-1] in each process?