<div dir="ltr">You also need calls to VecGhostUpdateBegin()/VecGhostUpdateEnd() functions to update the ghost values if you change them in the global representation. See Petsc Manual 3.2 pp 55-56<div><br></div><div>Mohammad<br>
<br><div class="gmail_quote">On Thu, Feb 23, 2012 at 3:18 PM, Bojan Niceno <span dir="ltr"><<a href="mailto:bojan.niceno@psi.ch">bojan.niceno@psi.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div class="im">
On 2/23/2012 11:49 PM, Mohammad Mirzadeh wrote:
<blockquote type="cite">
<div dir="ltr">based on,
<div><br>
</div>
<div><span>VecCreateGhost(PETSC_COMM_</span><span>WORLD, n, PETSC_DECIDE, nghost, &ghosts[0],
&x);</span></div>
<div><font color="#222222" face="monospace" size="1"><br>
</font></div>
<div><font color="#222222" face="monospace" size="1"><br>
</font></div>
<div>it seems to me that x is your actual ghosted vector. If
this is true, then you need to get its "local" form via <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecGhostGetLocalForm.html#VecGhostGetLocalForm" target="_blank">VecGhostGetLocalForm</a>().
Once you have done, you should be able to access the ghosted
nodes. Are you calling this function anywhere?</div>
</div>
</blockquote>
<br></div>
I tried that before. I did:<br>
<br>
Vec lx;<br>
VecGhostGetLocalForm(x, &lx)<br>
<br>
then I copied "lx" to my variable, like<br>
<br>
for(Int i=1; i<N; i++) /* N includes buffer cells */<br>
unk[i] = lx[i]<br>
<br>
but ghost values were also zero. I am thinking that PETSc somehow
clears the ghost values after a call to KSP. Is it the case?<br>
<br>
<br>
Kind regards,<br>
<br>
<br>
Bojan<div><div class="h5"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><br>
<div class="gmail_quote">On Thu, Feb 23, 2012 at 2:32 PM,
Bojan Niceno <span dir="ltr"><<a href="mailto:bojan.niceno@psi.ch" target="_blank">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Dear Mohammad,<br>
<br>
<br>
it doesn't help me, or I did not understand your
explanation.<br>
<br>
If I do this:<br>
<br>
<tt> /* copy internal values (THIS WORKS, BUT COPIES NO
BUFFER VALUES) */
<div><br>
for(Int i=0; i<n; i++) {<br>
Int gi = mesh.nodes[i].global_number;<br>
VecGetValues(x, 1, &gi, &unk[i]);<br>
}<br>
<br>
</div>
/* copy ghost values (CREATES MANY WARNINGS */
<div><br>
for(Int i=n; i<N; i++) {<br>
</div>
VecGetValues(x, 1, &i, &unk[i]);<br>
}</tt><br>
<br>
I get arnings are like this.<br>
<br>
<tt>
<div>[0]PETSC ERROR: ---------------------
Error Message ------------------------------------<br>
[0]PETSC ERROR: Argument out of range!<br>
</div>
[0]PETSC ERROR: Can only get local values, trying
3518!<br>
[3]PETSC ERROR: --------------------- Error Message
------------------------------------<br>
[3]PETSC ERROR: Argument out of range!<br>
[3]PETSC ERROR: Can only get local values, trying
3511!<br>
[3]PETSC ERROR:
------------------------------------------------------------------------<br>
</tt><br>
What am I doing wrong here?<br>
<br>
<br>
Cheers,<br>
<br>
<br>
Bojan
<div>
<div><br>
<br>
<br>
On 2/23/2012 11:23 PM, Mohammad Mirzadeh wrote:
<blockquote type="cite">
<p>just index x with the local numberings. if you
have 'n' local nodes and 'g' ghost nodes in x,
ghost nodes indecies run from 'n' to 'n+g-1'</p>
<div class="gmail_quote">On Feb 23, 2012 1:16 PM,
"Bojan Niceno" <<a href="mailto:bojan.niceno@psi.ch" target="_blank">bojan.niceno@psi.ch</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Dear
Jed,<br>
<br>
thanks.<br>
<br>
Now I have the following:<br>
<br>
- Array unk, which should hold values inside
the partition and in ghost cells. It is big
enough to hold both<br>
- Vec x, created by command VecCreateGhost,
with proper padding for ghost cells<br>
- Successful call to linear solver in
parallel.<br>
<br>
But I need to copy ghost values from x to my
array unk. <br>
<br>
How can I do it?<br>
<br>
<br>
Kind regards,<br>
<br>
<br>
Bojan<br>
<br>
On 2/23/2012 10:10 PM, Jed Brown wrote:
<blockquote type="cite">
<div class="gmail_quote">On Thu, Feb 23,
2012 at 15:05, Bojan Niceno <span dir="ltr"><<a href="mailto:bojan.niceno@psi.ch" target="_blank">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">No,
I use global.<br>
</div>
</blockquote>
<div><br>
</div>
<div>The local form is just a local
vector. It doesn't even know that a
global problem exists. You can't index
into it using global indices. (In
general, there is no efficient way to
look up information in the local
vector (includes ghost points) using
global indices.)</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<br>
<tt> for(Int i=0; i<n; i++) {<br>
Int gi =
mesh.nodes[i].global_number;<br>
VecGetValues(x, 1, &gi,
&unk[i]);<br>
}<br>
</tt><br>
"n" is defined as the number of
cells inside, i.e. without buffers.
"unk" is my external array. If I
try to access buffer values, I use:<br>
<br>
<tt> for(Int i=0; i<N; i++) {<br>
Int gi =
mesh.nodes[i].global_number;<br>
VecGetValues(x, 1, &gi,
&unk[i]);<br>
}<br>
</tt><br>
But then I end up with tons of
warnings, presumably because I am
going beyond "n". Vector x was
created with VecCreateGhost.</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
<br>
<div>-- <br>
<img src="cid:part1.05050700.08080101@psi.ch" border="0"></div>
</div>
</blockquote>
</div>
</blockquote>
<br>
<br>
</div>
</div>
<span><font color="#888888">
<div>-- <br>
<img src="cid:part2.02030501.08090504@psi.ch" border="0"></div>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888"><div>-- <br>
<img src="cid:part3.01070207.08050808@psi.ch" border="0"></div>
</font></span></div>
</blockquote></div><br></div></div>