<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Mon, Dec 3, 2018 at 8:32 PM Danyang Su <<a href="mailto:danyang.su@gmail.com">danyang.su@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    <p>Hi Matt,</p>
    <p>Attached is the test example with source code, makefile, data and
      screen output I wrote this afternoon. This example reads 2D mesh
      from vtk file and then distribute over all processors. I can get
      correct global order of local nodes after distribution, but the
      natural order of local nodes is always zero after using
      DMPlexGlobalToNaturalBegin/End(). Sounds like it does not take
      effect. <br>
    </p>
    <p>Would you mind helping me to check/test this code at you most
      convenience?</p>
    <p></p></div></blockquote><div>On 2 procs, I get an SEGV</div><div><br></div>





<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span class="gmail-Apple-converted-space">    </span>1 local nodes with ghost <span class="gmail-Apple-converted-space">  </span>47 local cells with ghost <span class="gmail-Apple-converted-space">  </span>75</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span class="gmail-Apple-converted-space">    </span>0 local nodes with ghost <span class="gmail-Apple-converted-space">  </span>52 local cells with ghost <span class="gmail-Apple-converted-space">  </span>78</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: ------------------------------------------------------------------------</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: or try <a href="http://valgrind.org">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: likely location of problem given in stack below</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: ---------------------<span class="gmail-Apple-converted-space">  </span>Stack Frames ------------------------------------</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: <span class="gmail-Apple-converted-space">      </span>INSTEAD the line number of the start of the function</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: <span class="gmail-Apple-converted-space">      </span>is given.</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: Signal received</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: Petsc Development GIT revision: v3.9.3-1021-g8625415<span class="gmail-Apple-converted-space">  </span>GIT Date: 2018-08-02 12:57:14 -0500</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: Unknown Name on a arch-master-debug named MATTHEW-KNEPLEYs-MacBook-Air-2.local by knepley Tue Dec<span class="gmail-Apple-converted-space">  </span>4 14:00:46 2018</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: Configure options --PETSC_ARCH=arch-master-debug --download-chaco --download-cmake=/Users/knepley/Downloads/cmake-3.7.2.tar.gz --download-ctetgen --download-eigen --download-fftw --download-hdf5 --download-med --download-metis --download-mpich --download-netcdf --download-p4est --download-parmetis --download-pnetcdf --download-superlu_dist --download-triangle --with-cc="/Users/knepley/MacSoftware/bin/ccache gcc -Qunused-arguments" --with-cxx="/Users/knepley/MacSoftware/bin/ccache g++ -Qunused-arguments" --with-fc="/Users/knepley/MacSoftware/bin/ccache gfortran" --with-shared-libraries</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC ERROR: #1 User provided function() line 0 in<span class="gmail-Apple-converted-space">  </span>unknown file</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">------------------------------------------------------------------------</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: or try <a href="http://valgrind.org">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: likely location of problem given in stack below</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: ---------------------<span class="gmail-Apple-converted-space">  </span>Stack Frames ------------------------------------</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: <span class="gmail-Apple-converted-space">      </span>INSTEAD the line number of the start of the function</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: <span class="gmail-Apple-converted-space">      </span>is given.</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: Signal received</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: Petsc Development GIT revision: v3.9.3-1021-g8625415<span class="gmail-Apple-converted-space">  </span>GIT Date: 2018-08-02 12:57:14 -0500</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: Unknown Name on a arch-master-debug named MATTHEW-KNEPLEYs-MacBook-Air-2.local by knepley Tue Dec<span class="gmail-Apple-converted-space">  </span>4 14:00:46 2018</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: Configure options --PETSC_ARCH=arch-master-debug --download-chaco --download-cmake=/Users/knepley/Downloads/cmake-3.7.2.tar.gz --download-ctetgen --download-eigen --download-fftw --download-hdf5 --download-med --download-metis --download-mpich --download-netcdf --download-p4est --download-parmetis --download-pnetcdf --download-superlu_dist --download-triangle --with-cc="/Users/knepley/MacSoftware/bin/ccache gcc -Qunused-arguments" --with-cxx="/Users/knepley/MacSoftware/bin/ccache g++ -Qunused-arguments" --with-fc="/Users/knepley/MacSoftware/bin/ccache gfortran" --with-shared-libraries</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC ERROR: #1 User provided function() line 0 in<span class="gmail-Apple-converted-space">  </span>unknown file</p>
<div><span style="font-family:Menlo;font-size:11px">application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0</span></div><div><br></div><div>which valgrind says comes from DMPlexCreateSection()</div><div><br></div>





<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span class="gmail-Apple-converted-space">    </span>0 local nodes with ghost <span class="gmail-Apple-converted-space">  </span>52 local cells with ghost <span class="gmail-Apple-converted-space">  </span>78</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span class="gmail-Apple-converted-space">    </span>1 local nodes with ghost <span class="gmail-Apple-converted-space">  </span>47 local cells with ghost <span class="gmail-Apple-converted-space">  </span>75</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766== Invalid write of size 4</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span class="gmail-Apple-converted-space">    </span>at 0x1151A3B: dmplexcreatesection_ (zplexsectionf90.c:20)</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span class="gmail-Apple-converted-space">    </span>by 0x1000042FA: MAIN__ (in ./natural)</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span class="gmail-Apple-converted-space">    </span>by 0x10000497D: main (in ./natural)</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span class="gmail-Apple-converted-space">  </span>Address 0x600001000 is not stack'd, malloc'd or (recently) free'd</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span class="gmail-Apple-converted-space"> </span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767== Invalid write of size 4</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span class="gmail-Apple-converted-space">    </span>at 0x1151A3B: dmplexcreatesection_ (zplexsectionf90.c:20)</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span class="gmail-Apple-converted-space">    </span>by 0x1000042FA: MAIN__ (in ./natural)</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span class="gmail-Apple-converted-space">    </span>by 0x10000497D: main (in ./natural)</p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span class="gmail-Apple-converted-space">  </span>Address 0x600001000 is not stack'd, malloc'd or (recently) free'd</p>
<div><span style="font-family:Menlo;font-size:11px">==14767==</span><span class="gmail-Apple-converted-space" style="font-family:Menlo;font-size:11px"> </span></div><div><br></div><div>I cannot run the debugger on it because command line arguments are not working with</div><div>my Fortran compiler (Ugh). Do you see this error?</div><div><br></div><div>  Thanks,</div><div><br></div><div>    Matt </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><p>Thanks,</p>
    <p>Danyang<br>
    </p>
    <div class="m_8975814135343234898moz-cite-prefix">On 2018-12-03 1:12 p.m., Danyang Su
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <p><br>
      </p>
      <div class="m_8975814135343234898moz-cite-prefix">On 2018-12-03 12:56 p.m., Matthew
        Knepley wrote:<br>
      </div>
      <blockquote type="cite">
        
        <div dir="ltr">
          <div dir="ltr">
            <div class="gmail_quote">
              <div dir="ltr">On Mon, Dec 3, 2018 at 3:40 PM Danyang Su
                <<a href="mailto:danyang.su@gmail.com" target="_blank">danyang.su@gmail.com</a>>
                wrote:<br>
              </div>
              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                <div bgcolor="#FFFFFF">
                  <p><br>
                  </p>
                  <div class="m_8975814135343234898gmail-m_397419241377719570moz-cite-prefix">On
                    2018-12-03 12:03 p.m., Matthew Knepley wrote:<br>
                  </div>
                  <blockquote type="cite">
                    <div dir="ltr">
                      <div class="gmail_quote">
                        <div dir="ltr">On Mon, Dec 3, 2018 at 2:27 PM
                          Danyang Su <<a href="mailto:danyang.su@gmail.com" target="_blank">danyang.su@gmail.com</a>>
                          wrote:<br>
                        </div>
                        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                          <div bgcolor="#FFFFFF">
                            <p>Hi Matt,</p>
                            <p>Thanks. <br>
                            </p>
                            <p>BTW: DmPlexGetVertexNumbering now can
                              work using the latest develop version. But
                              the index is not in natural ordering when
                              DMSetUseNatural is called. That's why I
                              want to use PetscSFDistributeSection to
                              check if I miss anything in the code.</p>
                          </div>
                        </blockquote>
                        <div>Can you explain that a little more? Maybe
                          you can just push forward what you want using
                          the migrationSF.</div>
                      </div>
                    </div>
                  </blockquote>
                  <p>Hi Matt,</p>
                  <p>Since I cannot figure what is wrong or missing in
                    my code, I followed an old ex26.c example in
                    src/dm/impls/plex/examples/tests to create similar
                    code as shown below to test global to natural
                    ordering. The code may be ugly with unnecessary
                    functions in it. Using DmPlexGetVertexNumbering, I
                    can get the value but it is not in natural order,
                    instead, it is still in default PETSc order without
                    calling DMSetUseNatural(dm,PETSC_TRUE,ierr).</p>
                </div>
              </blockquote>
              <div>I do not understand what you are doing below. You
                just need to call</div>
              <div><br>
              </div>
              <div>
                <div>    ierr =
                  DMSetUseNatural(dm,PETSC_TRUE);CHKERRQ(ierr);</div>
                <div>    ierr =
                  DMPlexDistribute(dm,0,&migrationSF,&pdm);CHKERRQ(ierr);<br>
                </div>
                <div>    if (pdm) {</div>
                <div>      ierr =
                  DMPlexSetMigrationSF(pdm,migrationSF);CHKERRQ(ierr);</div>
              </div>
              <div>    }</div>
              <div> </div>
              <div>and the DMGlobalToNaturalBegin/End() should work.</div>
            </div>
          </div>
        </div>
      </blockquote>
      <p>You mean to use DMPlexGlobalToNaturalBegin/End(), right? That's
        what I tried at first, but without success.</p>
      <p>I will create a test example to make further check if I can
        reproduce the problem. <br>
      </p>
      <p>Thanks,</p>
      <p>Danyang<br>
      </p>
      <blockquote type="cite">
        <div dir="ltr">
          <div dir="ltr">
            <div class="gmail_quote">
              <div><br>
              </div>
              <div>  Thanks,</div>
              <div><br>
              </div>
              <div>     Matt</div>
              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                <div bgcolor="#FFFFFF">
                  <p>          if (rank == 0) then</p>
                              call
                  DMPlexCreateFromCellList(Petsc_Comm_World,ndim,num_cells,
                  num_nodes,num_nodes_per_cell,    &<br>
                                                           
                  Petsc_False,dmplex_cells,ndim, dmplex_verts,dm,ierr)<br>
                              CHKERRQ(ierr)<br>
                            else<br>
                              call
                  DMPlexCreateFromCellList(Petsc_Comm_World,ndim,0,
                  0,num_nodes_per_cell,    &<br>
                                                           
                  Petsc_False,dmplex_cells,ndim,dmplex_verts,dm,ierr)<br>
                              CHKERRQ(ierr)<br>
                            end if<br>
                  <br>
                            if (nprocs > 1) then<br>
                              call DMSetUseNatural(dm,PETSC_TRUE,ierr)<br>
                              CHKERRQ(ierr)<br>
                            end if<br>
                  <br>
                            call
                  DMPlexDistribute(dm,stencil_width,               
                  &<br>
                                                              
                  migrationsf,distributedMesh,ierr)<br>
                            CHKERRQ(ierr)<br>
                  <br>
                            if (distributedMesh /= PETSC_NULL_DM) then<br>
                              call
                  PetscSFCreateInverseSF(migrationsf,migrationsf_inv,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call
DMCreateGlobalToNatural(distributedMesh,migrationsf,migrationsf_inv,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call
                  DMGetSection(distributedMesh,section,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call
                  PetscSectionCreate(Petsc_Comm_World,section_seq,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call
                  PetscSFDistributeSection(migrationsf_inv,section,      
                  &<br>
                                         
                  PETSC_NULL_INTEGER,section_seq,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call
                  DMPlexCreateGlobalToNaturalSF(distributedMesh,         
                  &<br>
                                        
                  section_seq,migrationsf,sf_natural,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call
                  DMSetUseNatural(distributedMesh,PETSC_TRUE,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call PetscSFDestroy(migrationsf,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <br>
                              call PetscSFDestroy(migrationsf_inv,ierr)<br>
                              CHKERRQ(ierr)<br>
                  <p>          end if</p>
                  <p>Thanks,</p>
                  <p>Danyang<br>
                  </p>
                  <blockquote type="cite">
                    <div dir="ltr">
                      <div class="gmail_quote">
                        <div><br>
                        </div>
                        <div>  Thanks,</div>
                        <div><br>
                        </div>
                        <div>     Matt <br>
                        </div>
                        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                          <div bgcolor="#FFFFFF">
                            <p>Regards,</p>
                            <p>Danyang<br>
                            </p>
                            <div class="m_8975814135343234898gmail-m_397419241377719570m_-4663990655342689142moz-cite-prefix">On
                              2018-12-03 5:22 a.m., Matthew Knepley
                              wrote:<br>
                            </div>
                            <blockquote type="cite">
                              <div dir="ltr">I need to write a custom
                                Fortran stub for this one. I will get it
                                done as soon as possible.
                                <div><br>
                                </div>
                                <div>  Thanks,</div>
                                <div><br>
                                </div>
                                <div>    Matt</div>
                              </div>
                              <br>
                              <div class="gmail_quote">
                                <div dir="ltr">On Sat, Dec 1, 2018 at
                                  7:16 PM Danyang Su via petsc-users
                                  <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>>
                                  wrote:<br>
                                </div>
                                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi
                                  All,<br>
                                  <br>
                                  I got a simple compilation error when
                                  use PetscSFDistributeSection in <br>
                                  Fortran. It looks like the required
                                  head files are included and the <br>
                                  parameters are correctly defined.
                                  However, when compile the code, I got
                                  <br>
                                  error undefined reference to
                                  `petscsfdistributesection_'. The code
                                  is <br>
                                  shown below. Did I miss anything here?<br>
                                  <br>
                                  #include
                                  <petsc/finclude/petscsys.h><br>
                                  #include
                                  <petsc/finclude/petscvec.h><br>
                                  #include
                                  <petsc/finclude/petscdm.h><br>
                                  #include
                                  <petsc/finclude/petscdmplex.h><br>
                                         use petscsys<br>
                                         use petscvec<br>
                                         use petscdm<br>
                                         use petscdmplex<br>
                                  <br>
                                         implicit none<br>
                                  <br>
                                         PetscSection ::  section,
                                  section_seq<br>
                                         PetscSF :: migrationsf_inv,
                                  sf_natural<br>
                                         Vec :: vec_global, vec_natural<br>
                                         PetscErrorCode :: ierr<br>
                                  <br>
                                         ...<br>
                                  <br>
                                         call
                                  PetscSFDistributeSection(migrationsf_inv,section, 
                                           &<br>
                                  PETSC_NULL_INTEGER,section_seq,ierr)<br>
                                         CHKERRQ(ierr)<br>
                                  <br>
                                  <br>
                                             call
                                  PetscSFDistributeSection(migrationsf_inv,section, 
                                       &<br>
                                                       
                                   PETSC_NULL_INTEGER,section_seq,ierr)<br>
                                             CHKERRQ(ierr)<br>
                                  <br>
                                  Thanks,<br>
                                  <br>
                                  Danyang<br>
                                  <br>
                                </blockquote>
                              </div>
                              <br clear="all">
                              <div><br>
                              </div>
                              -- <br>
                              <div dir="ltr" class="m_8975814135343234898gmail-m_397419241377719570m_-4663990655342689142gmail_signature">
                                <div dir="ltr">
                                  <div>
                                    <div dir="ltr">
                                      <div>
                                        <div dir="ltr">
                                          <div>What most experimenters
                                            take for granted before they
                                            begin their experiments is
                                            infinitely more interesting
                                            than any results to which
                                            their experiments lead.<br>
                                            -- Norbert Wiener</div>
                                          <div><br>
                                          </div>
                                          <div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </blockquote>
                          </div>
                        </blockquote>
                      </div>
                      <br clear="all">
                      <div><br>
                      </div>
                      -- <br>
                      <div dir="ltr" class="m_8975814135343234898gmail-m_397419241377719570gmail_signature">
                        <div dir="ltr">
                          <div>
                            <div dir="ltr">
                              <div>
                                <div dir="ltr">
                                  <div>What most experimenters take for
                                    granted before they begin their
                                    experiments is infinitely more
                                    interesting than any results to
                                    which their experiments lead.<br>
                                    -- Norbert Wiener</div>
                                  <div><br>
                                  </div>
                                  <div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </blockquote>
                </div>
              </blockquote>
            </div>
            <br clear="all">
            <div><br>
            </div>
            -- <br>
            <div dir="ltr" class="m_8975814135343234898gmail_signature">
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <div>
                      <div dir="ltr">
                        <div>What most experimenters take for granted
                          before they begin their experiments is
                          infinitely more interesting than any results
                          to which their experiments lead.<br>
                          -- Norbert Wiener</div>
                        <div><br>
                        </div>
                        <div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </blockquote>
    </blockquote>
  </div>

</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>