<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi Matthew,</p>
<p>Thanks for pointing out. The problem is that vec_natural is
always zero after calling DMPlexGlobalToNaturalEnd. I am afraid
something is missing in my code.</p>
<p>Thanks,</p>
<p>Danyang<br>
</p>
On 18-11-28 07:34 PM, Matthew Knepley wrote:<br>
<blockquote
cite="mid:CAMYG4GmNBPJi5WT+tbmLm0G8wdp+axgkjs3ZXx2CXW4J4Q5fPA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Wed, Nov 28, 2018 at 8:58 PM Danyang Su via
petsc-users <<a moz-do-not-send="true"
href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Dear All,<br>
<br>
I got the following error when using DMPlexGlobalToNatural
function <br>
using 1 processor.<br>
</blockquote>
<div><br>
</div>
<div>We do not create that mapping on 1 proc because the
orderings are the same. Reordering happens</div>
<div>when we redistribute.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
[0]PETSC ERROR: --------------------- Error Message <br>
--------------------------------------------------------------<br>
[0]PETSC ERROR: Object is in wrong state<br>
[0]PETSC ERROR: DM global to natural SF was not created.<br>
You must call DMSetUseNatural() before DMPlexDistribute().<br>
<br>
[0]PETSC ERROR: See <a moz-do-not-send="true"
href="http://www.mcs.anl.gov/petsc/documentation/faq.html"
rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a>
<br>
for trouble shooting.<br>
[0]PETSC ERROR: Petsc Release Version 3.10.2, Oct, 09, 2018<br>
<br>
The same code does not return error when using more than 2
processors, <br>
however, the vec_natural is always zero after calling <br>
DMPlexGlobalToNaturalEnd.<br>
<br>
<br>
DMSetUseNatural() has already been used before calling
DMPlexDistribute. <br>
The code section looks like below<br>
<br>
if (rank == 0) then<br>
<br>
call
DMPlexCreateFromCellList(Petsc_Comm_World,ndim,0,0,
&<br>
num_nodes_per_cell,
&<br>
Petsc_False,dmplex_cells,ndim, &
!use Petsc_True <br>
to create intermediate mesh entities (faces, edges),<br>
dmplex_verts,dmda_flow%da,ierr)<br>
CHKERRQ(ierr)<br>
<br>
end if<br>
<br>
!c Set the flag for creating a mapping to the natural
order on <br>
distribution<br>
call DMSetUseNatural(dmda_flow%da,PETSC_TRUE,ierr)<br>
CHKERRQ(ierr)<br>
<br>
!c distribute mesh over processes<br>
call
DMPlexDistribute(dmda_flow%da,stencil_width,
&<br>
PETSC_NULL_SF,
distributedMesh,ierr)<br>
<br>
CHKERRQ(ierr)<br>
<br>
!c destroy original global mesh after distribution<br>
if (distributedMesh /= PETSC_NULL_DM) then<br>
call DMDestroy(dmda_flow%da,ierr)<br>
CHKERRQ(ierr)<br>
!c set the global mesh as distributed mesh<br>
dmda_flow%da = distributedMesh<br>
end if<br>
<br>
...<br>
<br>
call
DMPlexCreateSection(dmda_flow%da,dmda_flow%dim,
&<br>
numFields,pNumComp,pNumDof, &<br>
numBC,pBcField, &<br>
pBcCompIS,pBcPointIS, &<br>
PETSC_NULL_IS, &<br>
section,ierr)<br>
CHKERRQ(ierr)<br>
<br>
call
PetscSectionSetFieldName(section,0,'flow',ierr)<br>
CHKERRQ(ierr)<br>
<br>
<br>
call DMSetSection(dmda_flow%da,section,ierr)<br>
CHKERRQ(ierr)<br>
<br>
call PetscSectionDestroy(section,ierr)<br>
CHKERRQ(ierr)<br>
<br>
call DMSetUp(dmda_flow%da,ierr)<br>
CHKERRQ(ierr)<br>
<br>
...<br>
<br>
<br>
!c global - natural order<br>
<br>
call DMCreateLocalVector(dmda_flow%da,vec_loc,ierr)<br>
CHKERRQ(ierr)<br>
<br>
call
DMCreateGlobalVector(dmda_flow%da,vec_global,ierr)<br>
CHKERRQ(ierr)<br>
<br>
call
DMCreateGlobalVector(dmda_flow%da,vec_natural,ierr)<br>
CHKERRQ(ierr)<br>
<br>
!c zero entries<br>
call VecZeroEntries(vec_loc,ierr)<br>
CHKERRQ(ierr)<br>
<br>
!Get a pointer to vector data when you need access to
the array<br>
call VecGetArrayF90(vec_loc,vecpointer,ierr)<br>
CHKERRQ(ierr)<br>
<br>
do inode = 1, num_nodes<br>
vecpointer(inode) = node_idx_lg2pg(inode) !vector
value using <br>
PETSc global order, negative ghost index has been reversed<br>
end do<br>
<br>
!Restore the vector when you no longer need access to
the array<br>
call VecRestoreArrayF90(vec_loc,vecpointer,ierr)<br>
CHKERRQ(ierr)<br>
<br>
!Insert values into global vector<br>
call
DMLocalToGlobalBegin(dmda_flow%da,vec_loc,INSERT_VALUES,
&<br>
vec_global,ierr)<br>
CHKERRQ(ierr)<br>
<br>
call
DMLocalToGlobalEnd(dmda_flow%da,vec_loc,INSERT_VALUES,
&<br>
vec_global,ierr)<br>
CHKERRQ(ierr)<br>
<br>
<br>
!c global to natural ordering<br>
call
DMPlexGlobalToNaturalBegin(dmda_flow%da,vec_global,
&<br>
vec_natural,ierr)<br>
CHKERRQ(ierr)<br>
<br>
call
DMPlexGlobalToNaturalEnd(dmda_flow%da,vec_global,
&<br>
vec_natural,ierr)<br>
CHKERRQ(ierr)<br>
<br>
<br>
Is there anything missing in the code that
DMPlexGlobalToNatural... does <br>
not work properly?<br>
<br>
Thanks,<br>
<br>
Danyang<br>
<br>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>What most experimenters take for granted before
they begin their experiments is infinitely more
interesting than any results to which their
experiments lead.<br>
-- Norbert Wiener</div>
<div><br>
</div>
<div><a moz-do-not-send="true"
href="http://www.cse.buffalo.edu/%7Eknepley/"
target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>