<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
No, I use global.<br>
<br>
<tt> for(Int i=0; i<n; i++) {<br>
Int gi = mesh.nodes[i].global_number;<br>
VecGetValues(x, 1, &gi, &unk[i]);<br>
}<br>
</tt><br>
"n" is defined as the number of cells inside, i.e. without buffers.
"unk" is my external array. If I try to access buffer values, I
use:<br>
<br>
<tt> for(Int i=0; i<N; i++) {<br>
Int gi = mesh.nodes[i].global_number;<br>
VecGetValues(x, 1, &gi, &unk[i]);<br>
}<br>
</tt><br>
But then I end up with tons of warnings, presumably because I am
going beyond "n". Vector x was created with VecCreateGhost.<br>
<br>
On 2/23/2012 10:00 PM, Mohammad Mirzadeh wrote:
<blockquote
cite="mid:CAKsPKwm+kjohxzFS2PE2N1XOH1ORWyXu1o0vS+Gq9bcfpE8rZQ@mail.gmail.com"
type="cite">
<div dir="ltr">Are you using local numbering when accessing the
local part of ghost nodes?<br>
<br>
<div class="gmail_quote">On Thu, Feb 23, 2012 at 12:43 PM, Bojan
Niceno <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:bojan.niceno@psi.ch">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Dear Matt,<br>
<br>
<br>
I have a new insight, although is not the full
resolution. If I change my code in PETScSolver.cpp from:<br>
<br>
<tt><br>
</tt><tt>/*-------------------------------------------------+<br>
| Make necessary PETSc intializations for vetors |<br>
+-------------------------------------------------*/<br>
Int nghost = N - n;<br>
Int * ghosts = new Int(nghost);<br>
for(Int n=0; n<M.mesh.nodes.size(); n++) {<br>
assert( M.mesh.nodes[n].global_number >= 0);<br>
assert( M.mesh.nodes[n].global_number < 14065);<br>
}<br>
for(Int i=n; i<N; i++) {<br>
assert( M.mesh.nodes[i].global_number >= 0);<br>
assert( M.mesh.nodes[i].global_number < 14065);<br>
assert( ! (M.mesh.nodes[i].global_number >=
n_start &&<br>
M.mesh.nodes[i].global_number <
n_end) );<br>
ghosts[i] = M.mesh.nodes[i].global_number;<br>
}<br>
<br>
VecCreateGhost(PETSC_COMM_WORLD, n, PETSC_DECIDE,
nghost, &ghosts[0], &x);<br>
</tt><br>
to:<br>
<br>
<tt> /*-------------------------------------------------+<br>
| Make necessary PETSc intializations for vetors |<br>
+-------------------------------------------------*/<br>
Int nghost = N - n;<br>
Indices
ghosts; //
<---= NEW!<br>
for(Int n=0; n<M.mesh.nodes.size(); n++) {<br>
assert( M.mesh.nodes[n].global_number >= 0);<br>
assert( M.mesh.nodes[n].global_number < 14065);<br>
}<br>
for(Int i=n; i<N; i++) {<br>
assert( M.mesh.nodes[i].global_number >= 0);<br>
assert( M.mesh.nodes[i].global_number < 14065);<br>
assert( ! (M.mesh.nodes[i].global_number >=
n_start &&<br>
M.mesh.nodes[i].global_number <
n_end) );<br>
ghosts.push_back( M.mesh.nodes[i].global_number );</tt><tt>
// <---= NEW!</tt><tt> <br>
}<br>
assert( ghosts.size() == nghost
); </tt><tt>// <---= NEW!</tt><br>
<tt><br>
VecCreateGhost(PETSC_COMM_WORLD, n, PETSC_DECIDE,
nghost, &ghosts[0], &x);</tt><br>
<br>
I pass the VecCreateGhost phase. "Indices" is an STL
container of integers. It seems it works better than
classical C array for this case.<br>
<br>
<br>
However, I still do not see the ghost values, i.e. I get
the following error:<br>
<br>
<tt>[1]PETSC ERROR: --------------------- Error Message
------------------------------------<br>
[1]PETSC ERROR: [0]PETSC ERROR: ---------------------
Error Message ------------------------------------<br>
[0]PETSC ERROR: Argument out of range!<br>
[0]PETSC ERROR: Can only get local values, trying 3529!<br>
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR: Argument out of range!<br>
[1]PETSC ERROR: Can only get local values, trying 22!<br>
[1]PETSC ERROR:
------------------------------------------------------------------------<br>
[1]PETSC ERROR: Petsc Release Version 3.2.0, Patch 6,
Wed Jan 11 09:28:45 CST 2012<br>
[1]PETSC ERROR: See docs/changes/index.html for recent
updates.<br>
[1]PETSC ERROR: See docs/faq.html for hints about
trouble shooting.<br>
[1]PETSC ERROR: See docs/index.html for manual pages.<br>
[1]PETSC ERROR: [2]PETSC ERROR: ---------------------
Error Message ------------------------------------<br>
[2]PETSC ERROR: Argument out of range!<br>
[2]PETSC ERROR: Can only get local values, trying 86!<br>
[2]PETSC ERROR:
------------------------------------------------------------------------<br>
[2]PETSC ERROR: Petsc Release Version 3.2.0, Patch 6,
Wed Jan 11 09:28:45 CST 2012</tt><br>
<br>
when I am trying to access values in ghost cells. What do
I have to use to see them ghosts? I reckon
VecGhostGetLocalForm should be used, right?<br>
<br>
<br>
Kind regards,<br>
<br>
<br>
Bojan
<div>
<div class="h5"><br>
<br>
<br>
On 2/23/2012 8:36 PM, Matthew Knepley wrote:
<blockquote type="cite">On Thu, Feb 23, 2012 at 1:33
PM, Bojan Niceno <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bojan.niceno@psi.ch"
target="_blank">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Dear
Matt,<br>
<br>
<br>
I sent the code as an attached tarball. I
sent it with case I run, so is 2 MB big. It
is now in the cue for moderator's approval. <br>
</div>
</blockquote>
<div><br>
</div>
<div>No, you HAVE to send it to <a
moz-do-not-send="true"
href="mailto:petsc-maint@mcs.anl.gov"
target="_blank">petsc-maint@mcs.anl.gov</a>,
as I said last time, for exactly this reason.</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Thanks.<br>
<br>
<br>
Kind regards,<br>
<br>
<br>
Bojan<br>
<br>
<br>
On 2/23/2012 8:04 PM, Matthew Knepley wrote:
<blockquote type="cite">On Thu, Feb 23, 2012
at 12:51 PM, Bojan Niceno <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bojan.niceno@psi.ch"
target="_blank">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<div class="gmail_quote">
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Dear Matt, <br>
<br>
<br>
are you sure? It is almost 4000 lines
long! Shall I send only the function
which bother me?<br>
<br>
If the entire code is what you need,
shall I make a tarball and attach it?<br>
</div>
</blockquote>
<div><br>
</div>
<div>Send something the builds and runs.
Don't care how long it is.</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Kind regards,<br>
<br>
<br>
Bojan<br>
<br>
On 2/23/2012 7:44 PM, Matthew Knepley
wrote:
<blockquote type="cite">On Thu, Feb
23, 2012 at 12:28 PM, Bojan Niceno <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bojan.niceno@psi.ch"
target="_blank">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<div class="gmail_quote">
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000"> On 2/23/2012
7:24 PM, Matthew Knepley
wrote:
<blockquote type="cite">On
Thu, Feb 23, 2012 at 12:05
PM, Bojan Niceno <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bojan.niceno@psi.ch"
target="_blank">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<div class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000"> Dear
Matthew,<br>
<br>
<br>
thank you for your
response. When I use
VecCreateGhost, I get
the following:<br>
</div>
</blockquote>
<div><br>
</div>
<div>It appears that you
passed a bad
communicator. Did you
not initialize a 'comm'
variable?</div>
</div>
</blockquote>
<br>
I pass PETSC_COMM_WORLD to
VecCreateGhost. <br>
<br>
I don't know what you mean by
'comm' variable :-( I called
all the routines to initialize
PETSc.</div>
</blockquote>
<div><br>
</div>
<div>Send your code to <a
moz-do-not-send="true"
href="mailto:petsc-maint@mcs.anl.gov"
target="_blank">petsc-maint@mcs.anl.gov</a>.</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000"> <br>
Cheers,<br>
<br>
<br>
Bojan<br>
<br>
<blockquote type="cite">
<div class="gmail_quote">
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000">
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR: Caught
signal number 11 SEGV:
Segmentation
Violation, probably
memory access out of
range<br>
[0]PETSC ERROR: Try
option
-start_in_debugger or
-on_error_attach_debugger<br>
[0]PETSC ERROR: or see
<a
moz-do-not-send="true"
href="http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind"
target="_blank">http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind</a>[0]PETSC
ERROR: or try <a
moz-do-not-send="true"
href="http://valgrind.org" target="_blank">http://valgrind.org</a> on
GNU/linux and Apple
Mac OS X to find
memory corruption
errors<br>
[0]PETSC ERROR: likely
location of problem
given in stack below<br>
[0]PETSC ERROR:
---------------------
Stack Frames
------------------------------------<br>
[0]PETSC ERROR: Note:
The EXACT line numbers
in the stack are not
available,<br>
[0]PETSC ERROR:
INSTEAD the line
number of the start of
the function<br>
[0]PETSC ERROR:
is given.<br>
[0]PETSC ERROR: [0]
PetscCommDuplicate
line 140
src/sys/objects/tagm.c<br>
[0]PETSC ERROR: [0]
PetscHeaderCreate_Private
line 30
src/sys/objects/inherit.c<br>
[0]PETSC ERROR: [0]
VecCreate line 32
src/vec/vec/interface/veccreate.c<br>
[0]PETSC ERROR: [0]
VecCreateGhostWithArray
line 567
src/vec/vec/impls/mpi/pbvec.c<br>
[0]PETSC ERROR: [0]
VecCreateGhost line
647
src/vec/vec/impls/mpi/pbvec.c<br>
[0]PETSC ERROR:
---------------------
Error Message
------------------------------------<br>
[0]PETSC ERROR: Signal
received!<br>
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR: Petsc
Release Version 3.2.0,
Patch 6, Wed Jan 11
09:28:45 CST 2012<br>
[0]PETSC ERROR: See
docs/changes/index.html
for recent updates.<br>
[0]PETSC ERROR: See
docs/faq.html for
hints about trouble
shooting.<br>
[0]PETSC ERROR: See
docs/index.html for
manual pages.<br>
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR:
./PSI-Flow on a
arch-linu named
lccfd06 by niceno Thu
Feb 23 19:02:45 2012<br>
[0]PETSC ERROR:
Libraries linked from
/homecfd/niceno/PETSc-3.2-p6/arch-linux2-c-debug/lib<br>
[0]PETSC ERROR:
Configure run at Fri
Feb 10 10:24:13 2012<br>
[0]PETSC ERROR:
Configure options<br>
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR: User
provided function()
line 0 in unknown
directory unknown file<br>
<br>
I don't understand
what could be causing
it. I took very good
care to match the
global numbers of
ghost cells when
calling VecCreateGhost<br>
<br>
<br>
Kind regards,<br>
<br>
<br>
Bojan<br>
<br>
<br>
On 2/23/2012 5:53 PM,
Matthew Knepley wrote:
<blockquote
type="cite">On Thu,
Feb 23, 2012 at
10:46 AM, Bojan
Niceno <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bojan.niceno@psi.ch" target="_blank">bojan.niceno@psi.ch</a>></span>
wrote:<br>
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
Hi all,<br>
<br>
I've never used
a mailing list
before, so I
hope this
message will
reach PETSc
users and
experts and
someone might be
willing to help
me. I am also
novice in PETSc.<br>
<br>
I have developed
an unstructured
finite volume
solver on top of
PETSc libraries.
In sequential,
it works like a
charm. For the
parallel
version, I do
domain
decomposition
externally with
Metis, and work
out local and
global
numberings, as
well as
communication
patterns between
processor. (The
latter don't
seem to be
needed for
PETSc, though.)
When I run my
program in
parallel, it
also works, but
I miss values in
vectors' ghost
points.<br>
<br>
I create vectors
with command:
VecCreate(PETSC_COMM_WORLD,
&x);<br>
<br>
Is it possible
to get the ghost
values if a
vector is
created like
this?<br>
</blockquote>
<div><br>
</div>
<div>I do not
understand this
question. By
definition,
"ghost values"
are those not
stored in the
global vector.</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
I have tried to
use
VecCreateGhost,
but for some
reason which is
beyond my
comprehension,
PETSc goes
berserk when it
reaches the
command:
VecCreateGhost(PETSC_COMM_WORLD,
n, PETSC_DECIDE,
nghost, ifrom,
&x)<br>
</blockquote>
<div><br>
</div>
<div>I think you
can understand
that "berserk"
tells me
absolutely
nothing. Error
message? Stack
trace? Did you
try to run an</div>
<div>example which
uses VecGhost?</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
Can anyone help
me? Either how
to reach ghost
values for
vector created
by VecCreate, or
how to use
VecCreateGhost
properly?<br>
<br>
<br>
Kind regards,<br>
<br>
Bojan<br>
</blockquote>
</div>
<br>
<br clear="all">
<span><font
color="#888888">
<div><br>
</div>
-- <br>
What most
experimenters
take for granted
before they
begin their
experiments is
infinitely more
interesting than
any results to
which their
experiments
lead.<br>
-- Norbert
Wiener<br>
</font></span></blockquote>
<span><font
color="#888888"> <br>
<br>
<div>-- <br>
<img
src="cid:part1.01070109.07010904@psi.ch"
border="0"></div>
</font></span></div>
</blockquote>
</div>
<br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
What most experimenters
take for granted before
they begin their
experiments is
infinitely more
interesting than any
results to which their
experiments lead.<br>
-- Norbert Wiener<br>
</font></span></blockquote>
<span><font color="#888888"> <br>
<br>
<div>-- <br>
<img
src="cid:part2.02030305.00030608@psi.ch"
border="0"></div>
</font></span></div>
</blockquote>
</div>
<br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
What most experimenters take for
granted before they begin their
experiments is infinitely more
interesting than any results to
which their experiments lead.<br>
-- Norbert Wiener<br>
</font></span></blockquote>
<span><font color="#888888"> <br>
<br>
<div>-- <br>
<img
src="cid:part3.01010507.00050706@psi.ch"
border="0"></div>
</font></span></div>
</blockquote>
</div>
<br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
What most experimenters take for granted
before they begin their experiments is
infinitely more interesting than any
results to which their experiments lead.<br>
-- Norbert Wiener<br>
</font></span></blockquote>
<span><font color="#888888"> <br>
<br>
<div>-- <br>
<img
src="cid:part4.00020303.02000002@psi.ch"
border="0"></div>
</font></span></div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
What most experimenters take for granted before they
begin their experiments is infinitely more
interesting than any results to which their
experiments lead.<br>
-- Norbert Wiener<br>
</blockquote>
<br>
<br>
</div>
</div>
<span class="HOEnZb"><font color="#888888">
<div>-- <br>
<img src="cid:part5.09070504.00070808@psi.ch"
border="0"></div>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<div class="moz-signature">-- <br>
<img src="cid:part6.05070600.00070501@psi.ch" border="0"></div>
</body>
</html>