<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">On 19/8/2015 1:17 PM, Dave May wrote:<br>
</div>
<blockquote
cite="mid:CAJ98EDpFpaq-fee9VuOykN-2Bk_geO4=PeoseyuB-DbtKOwqSQ@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 19 August 2015 at 03:38, TAY
wee-beng <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:zonexo@gmail.com" target="_blank">zonexo@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
I am using DA. For e.g.<br>
<br>
DM da_u<br>
<br>
call
DMDACreate3d(MPI_COMM_WORLD,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DM_BOUNDARY_NONE,DMDA_STENCIL_STAR,size_x,size_y,&<br>
<br>
size_z,1,PETSC_DECIDE,PETSC_DECIDE,1,stencil_width,lx,PETSC_NULL_INTEGER,PETSC_NULL_INTEGER,da_u,ierr)<br>
<br>
call DMCreateGlobalVector(da_u,u_global,ierr)<br>
<br>
call DMCreateLocalVector(da_u,u_local,ierr)<br>
<br>
To update the ghost values, I use:<br>
<br>
call
DMLocalToLocalBegin(da_u,u_local,INSERT_VALUES,u_local,ierr)<br>
<br>
call
DMLocalToLocalEnd(da_u,u_local,INSERT_VALUES,u_local,ierr)<br>
</blockquote>
<div><br>
<br>
</div>
<div>This is incorrect. <br>
The manpage for DMLocalToLocal clearly says "Maps from a
local vector (including ghost points that contain
irrelevant values) to another local vector where the ghost
points in the second are set correctly."</div>
<div>To update ghost values from a global vector (e.g. to
perform the scatter) you need to use
DMGlobalToLocalBegin() , DMGlobalToLocalEnd().<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<div>Hi Dave, </div>
<div><br>
</div>
<div>Thanks for the clarification although I'm still confused.
Supposed I have a 1D vector da_u, It has size 8, so it's like
da_u_array(8), with stencil width 1</div>
<div><br>
</div>
<div>So for 2 procs,</div>
<div><br>
</div>
<div>there will be 2 da_u_array - da_u_array(1:5) and
da_u_array(4:8)</div>
<div><br>
</div>
<div>After performing some operations on each procs's da_u_array, I
need to update 1st procs's da_u_array(5) and 2nd procs's
da_u_array(4) from the 2nd and 1st procs respectively. I simply
call:</div>
<div><br>
</div>
<div>call
DMLocalToLocalBegin(da_u,u_local,INSERT_VALUES,u_local,ierr)<br>
<br>
call DMLocalToLocalEnd(da_u,u_local,INSERT_VALUES,u_local,ierr)<br>
</div>
<div><br>
</div>
<div>and it seems to be enough. I check the ghost values and they
have been updated. So if I am not using the linear solvers, I do
not need the global vector,is that so?</div>
<blockquote
cite="mid:CAJ98EDpFpaq-fee9VuOykN-2Bk_geO4=PeoseyuB-DbtKOwqSQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
It seems that I don't need to use global vector at all.<br>
<br>
So what's the difference between local and global vector?<br>
</blockquote>
<div><br>
<br>
<div>
<div>
<div>
<div>
<div>
<div>* Local vectors contain ghost values from
any neighbouring MPI processes. They are
always defined over PETSC_COMM_SELF.<br>
</div>
* Global vectors store the DOFs assigned to each
sub-domain. These will parallel vectors defined
over the same communicator as your DM<br>
</div>
<br>
Thus, you use local vectors to compute things like
the sub-domain contribution to (i) a non-linear
residual evaluation or (ii) a sparse-matric vector
product.<br>
</div>
You use global vectors together with linear and
non-linear solvers as these vectors.<br>
<br>
</div>
If your stencil width was zero (in your DMDACreate3d()
function call), then the would be no ghost values to
communicate between neighbouring MPI processes. Hence,
the entries in the following two arrays LA_u_local[],
LA_u[] would be identical<br>
VecGetArrayRead(u_local,&LA_u_local);<br>
</div>
and <br>
</div>
VecGetArrayRead(u,&LA_u);<br>
<div>
<div>
<div>
<div>
<div><br>
</div>
<div>That said, u_local would still be of type
VECSEQ, where as u would be of type VECMPI.<br>
<br>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
When will I need to use?:<br>
<br>
call
DMGlobalToLocalBegin(da_u,u_global,INSERT_VALUES,u_local,ierr)<br>
<br>
call
DMGlobalToLocalEnd(da_u,u_global,INSERT_VALUES,u_local,ierr)<span
class="HOEnZb"><font color="#888888"><br>
</font></span></blockquote>
<div><br>
</div>
<div>See points (i) and (ii) above from common use cases.<br>
<br>
<div>Thanks,<br>
</div>
<div> Dave<br>
</div>
<div><br>
</div>
<br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><span
class="HOEnZb"><font color="#888888">
<br>
-- <br>
Thank you<br>
<br>
Yours sincerely,<br>
<br>
TAY wee-beng<br>
<br>
</font></span></blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>