<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Wed, Nov 28, 2018 at 8:58 PM Danyang Su via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dear All,<br>
<br>
I got the following error when using DMPlexGlobalToNatural function <br>
using 1 processor.<br></blockquote><div><br></div><div>We do not create that mapping on 1 proc because the orderings are the same. Reordering happens</div><div>when we redistribute.</div><div><br></div><div>  Thanks,</div><div><br></div><div>     Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[0]PETSC ERROR: --------------------- Error Message <br>
--------------------------------------------------------------<br>
[0]PETSC ERROR: Object is in wrong state<br>
[0]PETSC ERROR: DM global to natural SF was not created.<br>
You must call DMSetUseNatural() before DMPlexDistribute().<br>
<br>
[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> <br>
for trouble shooting.<br>
[0]PETSC ERROR: Petsc Release Version 3.10.2, Oct, 09, 2018<br>
<br>
The same code does not return error when using more than 2 processors, <br>
however, the vec_natural is always zero after calling <br>
DMPlexGlobalToNaturalEnd.<br>
<br>
<br>
DMSetUseNatural() has already been used before calling DMPlexDistribute. <br>
The code section looks like below<br>
<br>
      if (rank == 0) then<br>
<br>
         call DMPlexCreateFromCellList(Petsc_Comm_World,ndim,0,0,       &<br>
num_nodes_per_cell,                                        &<br>
Petsc_False,dmplex_cells,ndim,                        &  !use Petsc_True <br>
to create intermediate mesh entities (faces, edges),<br>
dmplex_verts,dmda_flow%da,ierr)<br>
         CHKERRQ(ierr)<br>
<br>
      end if<br>
<br>
       !c Set the flag for creating a mapping to the natural order on <br>
distribution<br>
       call DMSetUseNatural(dmda_flow%da,PETSC_TRUE,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       !c distribute mesh over processes<br>
       call DMPlexDistribute(dmda_flow%da,stencil_width,                &<br>
                             PETSC_NULL_SF, distributedMesh,ierr)<br>
<br>
       CHKERRQ(ierr)<br>
<br>
       !c destroy original global mesh after distribution<br>
       if (distributedMesh /= PETSC_NULL_DM) then<br>
         call DMDestroy(dmda_flow%da,ierr)<br>
         CHKERRQ(ierr)<br>
         !c set the global mesh as distributed mesh<br>
         dmda_flow%da = distributedMesh<br>
       end if<br>
<br>
       ...<br>
<br>
         call DMPlexCreateSection(dmda_flow%da,dmda_flow%dim,           &<br>
numFields,pNumComp,pNumDof,           &<br>
numBC,pBcField,                       &<br>
pBcCompIS,pBcPointIS,                 &<br>
PETSC_NULL_IS,                        &<br>
                                  section,ierr)<br>
         CHKERRQ(ierr)<br>
<br>
         call PetscSectionSetFieldName(section,0,'flow',ierr)<br>
         CHKERRQ(ierr)<br>
<br>
<br>
         call DMSetSection(dmda_flow%da,section,ierr)<br>
         CHKERRQ(ierr)<br>
<br>
         call PetscSectionDestroy(section,ierr)<br>
         CHKERRQ(ierr)<br>
<br>
         call DMSetUp(dmda_flow%da,ierr)<br>
         CHKERRQ(ierr)<br>
<br>
      ...<br>
<br>
<br>
       !c global - natural order<br>
<br>
       call DMCreateLocalVector(dmda_flow%da,vec_loc,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       call DMCreateGlobalVector(dmda_flow%da,vec_global,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       call DMCreateGlobalVector(dmda_flow%da,vec_natural,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       !c zero entries<br>
       call VecZeroEntries(vec_loc,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       !Get a pointer to vector data when you need access to the array<br>
       call VecGetArrayF90(vec_loc,vecpointer,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       do inode = 1, num_nodes<br>
         vecpointer(inode) = node_idx_lg2pg(inode) !vector value using <br>
PETSc global order, negative ghost index has been reversed<br>
       end do<br>
<br>
       !Restore the vector when you no longer need access to the array<br>
       call VecRestoreArrayF90(vec_loc,vecpointer,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       !Insert values into global vector<br>
       call DMLocalToGlobalBegin(dmda_flow%da,vec_loc,INSERT_VALUES,    &<br>
                                 vec_global,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       call DMLocalToGlobalEnd(dmda_flow%da,vec_loc,INSERT_VALUES,      &<br>
                                 vec_global,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
<br>
       !c global to natural ordering<br>
       call DMPlexGlobalToNaturalBegin(dmda_flow%da,vec_global,         &<br>
                                       vec_natural,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
       call DMPlexGlobalToNaturalEnd(dmda_flow%da,vec_global,           &<br>
                                     vec_natural,ierr)<br>
       CHKERRQ(ierr)<br>
<br>
<br>
Is there anything missing in the code that DMPlexGlobalToNatural... does <br>
not work properly?<br>
<br>
Thanks,<br>
<br>
Danyang<br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>