<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">On 2018-12-04 11:06 a.m., Matthew
      Knepley wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAMYG4GkGUBE6P0pnd6LOF28zzuPtDFcRKYZ-xm_Ec_x3g+AdUQ@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div class="gmail_quote">
          <div dir="ltr">On Mon, Dec 3, 2018 at 8:32 PM Danyang Su <<a
              href="mailto:danyang.su@gmail.com" moz-do-not-send="true">danyang.su@gmail.com</a>>
            wrote:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF">
              <p>Hi Matt,</p>
              <p>Attached is the test example with source code,
                makefile, data and screen output I wrote this afternoon.
                This example reads 2D mesh from vtk file and then
                distribute over all processors. I can get correct global
                order of local nodes after distribution, but the natural
                order of local nodes is always zero after using
                DMPlexGlobalToNaturalBegin/End(). Sounds like it does
                not take effect. <br>
              </p>
              <p>Would you mind helping me to check/test this code at
                you most convenience?</p>
            </div>
          </blockquote>
          <div>On 2 procs, I get an SEGV</div>
          <div><br>
          </div>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span
              class="gmail-Apple-converted-space">    </span>1 local
            nodes with ghost <span class="gmail-Apple-converted-space"> 
            </span>47 local cells with ghost <span
              class="gmail-Apple-converted-space">  </span>75</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span
              class="gmail-Apple-converted-space">    </span>0 local
            nodes with ghost <span class="gmail-Apple-converted-space"> 
            </span>52 local cells with ghost <span
              class="gmail-Apple-converted-space">  </span>78</p>
        </div>
      </div>
    </blockquote>
    <p>Interesting. The output on your side is different.  On my
      computer, run the code with 2 processors, I get <br>
    </p>
    <p>rank    0 local nodes with ghost   47 local cells with ghost   76<br>
      rank    1 local nodes with ghost   49 local cells with ghost   74</p>
    <p>and there is no error. I can run this example using 1 to 5
      processors without error. With 6 or more processors, it returns
      some error as shown below. <br>
    </p>
    <p>Error in `./natural': corrupted size vs. prev_size:
      0x0000000001b9a600<br>
    </p>
    <p>This seems reasonable if the mesh is too small with overlap = 1.<br>
    </p>
    <p>The version I use is</p>
    <p>Petsc Development GIT revision: v3.10.2-832-ge28ad50  GIT Date:
      2018-12-02 07:18:52 +0100</p>
    <p>Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran
      --download-mpich --download-scalapack --download-parmetis
      --download-metis --download-ptscotch --download-fblaslapack
      --download-hypre --download-superlu_dist --download-hdf5=yes
      --download-ctetgen --download-zlib --download-netcdf
      --download-pnetcdf --download-exodusii --download-netcdf
      --with-debugging=1</p>
    <p>Thanks,</p>
    <p>Danyang<br>
    </p>
    <blockquote type="cite"
cite="mid:CAMYG4GkGUBE6P0pnd6LOF28zzuPtDFcRKYZ-xm_Ec_x3g+AdUQ@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_quote">
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR:
            ------------------------------------------------------------------------</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: Caught signal number 11 SEGV: Segmentation Violation,
            probably memory access out of range</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: Try option -start_in_debugger or
            -on_error_attach_debugger</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: or see <a
              href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind"
              moz-do-not-send="true">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: or try <a href="http://valgrind.org"
              moz-do-not-send="true">http://valgrind.org</a> on
            GNU/linux and Apple Mac OS X to find memory corruption
            errors</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: likely location of problem given in stack below</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: ---------------------<span
              class="gmail-Apple-converted-space">  </span>Stack Frames
            ------------------------------------</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: Note: The EXACT line numbers in the stack are not
            available,</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: <span class="gmail-Apple-converted-space">      </span>INSTEAD
            the line number of the start of the function</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: <span class="gmail-Apple-converted-space">      </span>is
            given.</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: --------------------- Error Message
            --------------------------------------------------------------</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: Signal received</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: See <a
              href="http://www.mcs.anl.gov/petsc/documentation/faq.html"
              moz-do-not-send="true">http://www.mcs.anl.gov/petsc/documentation/faq.html</a>
            for trouble shooting.</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: Petsc Development GIT revision: v3.9.3-1021-g8625415<span
              class="gmail-Apple-converted-space">  </span>GIT Date:
            2018-08-02 12:57:14 -0500</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: Unknown Name on a arch-master-debug named
            MATTHEW-KNEPLEYs-MacBook-Air-2.local by knepley Tue Dec<span
              class="gmail-Apple-converted-space">  </span>4 14:00:46
            2018</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: Configure options --PETSC_ARCH=arch-master-debug
            --download-chaco
            --download-cmake=/Users/knepley/Downloads/cmake-3.7.2.tar.gz
            --download-ctetgen --download-eigen --download-fftw
            --download-hdf5 --download-med --download-metis
            --download-mpich --download-netcdf --download-p4est
            --download-parmetis --download-pnetcdf
            --download-superlu_dist --download-triangle
            --with-cc="/Users/knepley/MacSoftware/bin/ccache gcc
            -Qunused-arguments"
            --with-cxx="/Users/knepley/MacSoftware/bin/ccache g++
            -Qunused-arguments"
            --with-fc="/Users/knepley/MacSoftware/bin/ccache gfortran"
            --with-shared-libraries</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[1]PETSC
            ERROR: #1 User provided function() line 0 in<span
              class="gmail-Apple-converted-space">  </span>unknown file</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: application called MPI_Abort(MPI_COMM_WORLD, 59) -
            process 1</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">------------------------------------------------------------------------</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: Caught signal number 11 SEGV: Segmentation Violation,
            probably memory access out of range</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: Try option -start_in_debugger or
            -on_error_attach_debugger</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: or see <a
              href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind"
              moz-do-not-send="true">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a></p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: or try <a href="http://valgrind.org"
              moz-do-not-send="true">http://valgrind.org</a> on
            GNU/linux and Apple Mac OS X to find memory corruption
            errors</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: likely location of problem given in stack below</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: ---------------------<span
              class="gmail-Apple-converted-space">  </span>Stack Frames
            ------------------------------------</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: Note: The EXACT line numbers in the stack are not
            available,</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: <span class="gmail-Apple-converted-space">      </span>INSTEAD
            the line number of the start of the function</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: <span class="gmail-Apple-converted-space">      </span>is
            given.</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: --------------------- Error Message
            --------------------------------------------------------------</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: Signal received</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: See <a
              href="http://www.mcs.anl.gov/petsc/documentation/faq.html"
              moz-do-not-send="true">http://www.mcs.anl.gov/petsc/documentation/faq.html</a>
            for trouble shooting.</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: Petsc Development GIT revision: v3.9.3-1021-g8625415<span
              class="gmail-Apple-converted-space">  </span>GIT Date:
            2018-08-02 12:57:14 -0500</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: Unknown Name on a arch-master-debug named
            MATTHEW-KNEPLEYs-MacBook-Air-2.local by knepley Tue Dec<span
              class="gmail-Apple-converted-space">  </span>4 14:00:46
            2018</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: Configure options --PETSC_ARCH=arch-master-debug
            --download-chaco
            --download-cmake=/Users/knepley/Downloads/cmake-3.7.2.tar.gz
            --download-ctetgen --download-eigen --download-fftw
            --download-hdf5 --download-med --download-metis
            --download-mpich --download-netcdf --download-p4est
            --download-parmetis --download-pnetcdf
            --download-superlu_dist --download-triangle
            --with-cc="/Users/knepley/MacSoftware/bin/ccache gcc
            -Qunused-arguments"
            --with-cxx="/Users/knepley/MacSoftware/bin/ccache g++
            -Qunused-arguments"
            --with-fc="/Users/knepley/MacSoftware/bin/ccache gfortran"
            --with-shared-libraries</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">[0]PETSC
            ERROR: #1 User provided function() line 0 in<span
              class="gmail-Apple-converted-space">  </span>unknown file</p>
          <div><span style="font-family:Menlo;font-size:11px">application
              called MPI_Abort(MPI_COMM_WORLD, 59) - process 0</span></div>
          <div><br>
          </div>
          <div>which valgrind says comes from DMPlexCreateSection()</div>
          <div><br>
          </div>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span
              class="gmail-Apple-converted-space">    </span>0 local
            nodes with ghost <span class="gmail-Apple-converted-space"> 
            </span>52 local cells with ghost <span
              class="gmail-Apple-converted-space">  </span>78</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">rank<span
              class="gmail-Apple-converted-space">    </span>1 local
            nodes with ghost <span class="gmail-Apple-converted-space"> 
            </span>47 local cells with ghost <span
              class="gmail-Apple-converted-space">  </span>75</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==
            Invalid write of size 4</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span
              class="gmail-Apple-converted-space">    </span>at
            0x1151A3B: dmplexcreatesection_ (zplexsectionf90.c:20)</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span
              class="gmail-Apple-converted-space">    </span>by
            0x1000042FA: MAIN__ (in ./natural)</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span
              class="gmail-Apple-converted-space">    </span>by
            0x10000497D: main (in ./natural)</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span
              class="gmail-Apple-converted-space">  </span>Address
            0x600001000 is not stack'd, malloc'd or (recently) free'd</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14766==<span
              class="gmail-Apple-converted-space"> </span></p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==
            Invalid write of size 4</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span
              class="gmail-Apple-converted-space">    </span>at
            0x1151A3B: dmplexcreatesection_ (zplexsectionf90.c:20)</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span
              class="gmail-Apple-converted-space">    </span>by
            0x1000042FA: MAIN__ (in ./natural)</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span
              class="gmail-Apple-converted-space">    </span>by
            0x10000497D: main (in ./natural)</p>
          <p class="gmail-p1"
style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo">==14767==<span
              class="gmail-Apple-converted-space">  </span>Address
            0x600001000 is not stack'd, malloc'd or (recently) free'd</p>
          <div><span style="font-family:Menlo;font-size:11px">==14767==</span><span
              class="gmail-Apple-converted-space"
              style="font-family:Menlo;font-size:11px"> </span></div>
          <div><br>
          </div>
          <div>I cannot run the debugger on it because command line
            arguments are not working with</div>
          <div>my Fortran compiler (Ugh). Do you see this error?</div>
          <div><br>
          </div>
          <div>  Thanks,</div>
          <div><br>
          </div>
          <div>    Matt </div>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF">
              <p>Thanks,</p>
              <p>Danyang<br>
              </p>
              <div class="m_8975814135343234898moz-cite-prefix">On
                2018-12-03 1:12 p.m., Danyang Su wrote:<br>
              </div>
              <blockquote type="cite">
                <p><br>
                </p>
                <div class="m_8975814135343234898moz-cite-prefix">On
                  2018-12-03 12:56 p.m., Matthew Knepley wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div class="gmail_quote">
                        <div dir="ltr">On Mon, Dec 3, 2018 at 3:40 PM
                          Danyang Su <<a
                            href="mailto:danyang.su@gmail.com"
                            target="_blank" moz-do-not-send="true">danyang.su@gmail.com</a>>
                          wrote:<br>
                        </div>
                        <blockquote class="gmail_quote"
                          style="margin:0px 0px 0px
                          0.8ex;border-left:1px solid
                          rgb(204,204,204);padding-left:1ex">
                          <div bgcolor="#FFFFFF">
                            <p><br>
                            </p>
                            <div
                              class="m_8975814135343234898gmail-m_397419241377719570moz-cite-prefix">On
                              2018-12-03 12:03 p.m., Matthew Knepley
                              wrote:<br>
                            </div>
                            <blockquote type="cite">
                              <div dir="ltr">
                                <div class="gmail_quote">
                                  <div dir="ltr">On Mon, Dec 3, 2018 at
                                    2:27 PM Danyang Su <<a
                                      href="mailto:danyang.su@gmail.com"
                                      target="_blank"
                                      moz-do-not-send="true">danyang.su@gmail.com</a>>
                                    wrote:<br>
                                  </div>
                                  <blockquote class="gmail_quote"
                                    style="margin:0px 0px 0px
                                    0.8ex;border-left:1px solid
                                    rgb(204,204,204);padding-left:1ex">
                                    <div bgcolor="#FFFFFF">
                                      <p>Hi Matt,</p>
                                      <p>Thanks. <br>
                                      </p>
                                      <p>BTW: DmPlexGetVertexNumbering
                                        now can work using the latest
                                        develop version. But the index
                                        is not in natural ordering when
                                        DMSetUseNatural is called.
                                        That's why I want to use
                                        PetscSFDistributeSection to
                                        check if I miss anything in the
                                        code.</p>
                                    </div>
                                  </blockquote>
                                  <div>Can you explain that a little
                                    more? Maybe you can just push
                                    forward what you want using the
                                    migrationSF.</div>
                                </div>
                              </div>
                            </blockquote>
                            <p>Hi Matt,</p>
                            <p>Since I cannot figure what is wrong or
                              missing in my code, I followed an old
                              ex26.c example in
                              src/dm/impls/plex/examples/tests to create
                              similar code as shown below to test global
                              to natural ordering. The code may be ugly
                              with unnecessary functions in it. Using
                              DmPlexGetVertexNumbering, I can get the
                              value but it is not in natural order,
                              instead, it is still in default PETSc
                              order without calling
                              DMSetUseNatural(dm,PETSC_TRUE,ierr).</p>
                          </div>
                        </blockquote>
                        <div>I do not understand what you are doing
                          below. You just need to call</div>
                        <div><br>
                        </div>
                        <div>
                          <div>    ierr =
                            DMSetUseNatural(dm,PETSC_TRUE);CHKERRQ(ierr);</div>
                          <div>    ierr =
                            DMPlexDistribute(dm,0,&migrationSF,&pdm);CHKERRQ(ierr);<br>
                          </div>
                          <div>    if (pdm) {</div>
                          <div>      ierr =
                            DMPlexSetMigrationSF(pdm,migrationSF);CHKERRQ(ierr);</div>
                        </div>
                        <div>    }</div>
                        <div> </div>
                        <div>and the DMGlobalToNaturalBegin/End() should
                          work.</div>
                      </div>
                    </div>
                  </div>
                </blockquote>
                <p>You mean to use DMPlexGlobalToNaturalBegin/End(),
                  right? That's what I tried at first, but without
                  success.</p>
                <p>I will create a test example to make further check if
                  I can reproduce the problem. <br>
                </p>
                <p>Thanks,</p>
                <p>Danyang<br>
                </p>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div class="gmail_quote">
                        <div><br>
                        </div>
                        <div>  Thanks,</div>
                        <div><br>
                        </div>
                        <div>     Matt</div>
                        <blockquote class="gmail_quote"
                          style="margin:0px 0px 0px
                          0.8ex;border-left:1px solid
                          rgb(204,204,204);padding-left:1ex">
                          <div bgcolor="#FFFFFF">
                            <p>          if (rank == 0) then</p>
                                        call
                            DMPlexCreateFromCellList(Petsc_Comm_World,ndim,num_cells,
                            num_nodes,num_nodes_per_cell,    &<br>
                                                                     
                            Petsc_False,dmplex_cells,ndim,
                            dmplex_verts,dm,ierr)<br>
                                        CHKERRQ(ierr)<br>
                                      else<br>
                                        call
                            DMPlexCreateFromCellList(Petsc_Comm_World,ndim,0,
                            0,num_nodes_per_cell,    &<br>
                                                                     
                            Petsc_False,dmplex_cells,ndim,dmplex_verts,dm,ierr)<br>
                                        CHKERRQ(ierr)<br>
                                      end if<br>
                            <br>
                                      if (nprocs > 1) then<br>
                                        call
                            DMSetUseNatural(dm,PETSC_TRUE,ierr)<br>
                                        CHKERRQ(ierr)<br>
                                      end if<br>
                            <br>
                                      call
                            DMPlexDistribute(dm,stencil_width,               
                            &<br>
                                                                        
                            migrationsf,distributedMesh,ierr)<br>
                                      CHKERRQ(ierr)<br>
                            <br>
                                      if (distributedMesh /=
                            PETSC_NULL_DM) then<br>
                                        call
                            PetscSFCreateInverseSF(migrationsf,migrationsf_inv,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
DMCreateGlobalToNatural(distributedMesh,migrationsf,migrationsf_inv,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
                            DMGetSection(distributedMesh,section,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
                            PetscSectionCreate(Petsc_Comm_World,section_seq,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
                            PetscSFDistributeSection(migrationsf_inv,section,      
                            &<br>
                                                   
                            PETSC_NULL_INTEGER,section_seq,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
                            DMPlexCreateGlobalToNaturalSF(distributedMesh,         
                            &<br>
                                                  
                            section_seq,migrationsf,sf_natural,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
                            DMSetUseNatural(distributedMesh,PETSC_TRUE,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
                            PetscSFDestroy(migrationsf,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <br>
                                        call
                            PetscSFDestroy(migrationsf_inv,ierr)<br>
                                        CHKERRQ(ierr)<br>
                            <p>          end if</p>
                            <p>Thanks,</p>
                            <p>Danyang<br>
                            </p>
                            <blockquote type="cite">
                              <div dir="ltr">
                                <div class="gmail_quote">
                                  <div><br>
                                  </div>
                                  <div>  Thanks,</div>
                                  <div><br>
                                  </div>
                                  <div>     Matt <br>
                                  </div>
                                  <blockquote class="gmail_quote"
                                    style="margin:0px 0px 0px
                                    0.8ex;border-left:1px solid
                                    rgb(204,204,204);padding-left:1ex">
                                    <div bgcolor="#FFFFFF">
                                      <p>Regards,</p>
                                      <p>Danyang<br>
                                      </p>
                                      <div
class="m_8975814135343234898gmail-m_397419241377719570m_-4663990655342689142moz-cite-prefix">On
                                        2018-12-03 5:22 a.m., Matthew
                                        Knepley wrote:<br>
                                      </div>
                                      <blockquote type="cite">
                                        <div dir="ltr">I need to write a
                                          custom Fortran stub for this
                                          one. I will get it done as
                                          soon as possible.
                                          <div><br>
                                          </div>
                                          <div>  Thanks,</div>
                                          <div><br>
                                          </div>
                                          <div>    Matt</div>
                                        </div>
                                        <br>
                                        <div class="gmail_quote">
                                          <div dir="ltr">On Sat, Dec 1,
                                            2018 at 7:16 PM Danyang Su
                                            via petsc-users <<a
                                              href="mailto:petsc-users@mcs.anl.gov"
                                              target="_blank"
                                              moz-do-not-send="true">petsc-users@mcs.anl.gov</a>>
                                            wrote:<br>
                                          </div>
                                          <blockquote
                                            class="gmail_quote"
                                            style="margin:0px 0px 0px
                                            0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Hi All,<br>
                                            <br>
                                            I got a simple compilation
                                            error when use
                                            PetscSFDistributeSection in
                                            <br>
                                            Fortran. It looks like the
                                            required head files are
                                            included and the <br>
                                            parameters are correctly
                                            defined. However, when
                                            compile the code, I got <br>
                                            error undefined reference to
                                            `petscsfdistributesection_'.
                                            The code is <br>
                                            shown below. Did I miss
                                            anything here?<br>
                                            <br>
                                            #include
                                            <petsc/finclude/petscsys.h><br>
                                            #include
                                            <petsc/finclude/petscvec.h><br>
                                            #include
                                            <petsc/finclude/petscdm.h><br>
                                            #include
                                            <petsc/finclude/petscdmplex.h><br>
                                                   use petscsys<br>
                                                   use petscvec<br>
                                                   use petscdm<br>
                                                   use petscdmplex<br>
                                            <br>
                                                   implicit none<br>
                                            <br>
                                                   PetscSection :: 
                                            section, section_seq<br>
                                                   PetscSF ::
                                            migrationsf_inv, sf_natural<br>
                                                   Vec :: vec_global,
                                            vec_natural<br>
                                                   PetscErrorCode ::
                                            ierr<br>
                                            <br>
                                                   ...<br>
                                            <br>
                                                   call
                                            PetscSFDistributeSection(migrationsf_inv,section, 
                                                     &<br>
PETSC_NULL_INTEGER,section_seq,ierr)<br>
                                                   CHKERRQ(ierr)<br>
                                            <br>
                                            <br>
                                                       call
                                            PetscSFDistributeSection(migrationsf_inv,section, 
                                                 &<br>
                                                                 
                                             PETSC_NULL_INTEGER,section_seq,ierr)<br>
                                                       CHKERRQ(ierr)<br>
                                            <br>
                                            Thanks,<br>
                                            <br>
                                            Danyang<br>
                                            <br>
                                          </blockquote>
                                        </div>
                                        <br clear="all">
                                        <div><br>
                                        </div>
                                        -- <br>
                                        <div dir="ltr"
class="m_8975814135343234898gmail-m_397419241377719570m_-4663990655342689142gmail_signature">
                                          <div dir="ltr">
                                            <div>
                                              <div dir="ltr">
                                                <div>
                                                  <div dir="ltr">
                                                    <div>What most
                                                      experimenters take
                                                      for granted before
                                                      they begin their
                                                      experiments is
                                                      infinitely more
                                                      interesting than
                                                      any results to
                                                      which their
                                                      experiments lead.<br>
                                                      -- Norbert Wiener</div>
                                                    <div><br>
                                                    </div>
                                                    <div><a
                                                        href="http://www.cse.buffalo.edu/~knepley/"
                                                        target="_blank"
moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a><br>
                                                    </div>
                                                  </div>
                                                </div>
                                              </div>
                                            </div>
                                          </div>
                                        </div>
                                      </blockquote>
                                    </div>
                                  </blockquote>
                                </div>
                                <br clear="all">
                                <div><br>
                                </div>
                                -- <br>
                                <div dir="ltr"
                                  class="m_8975814135343234898gmail-m_397419241377719570gmail_signature">
                                  <div dir="ltr">
                                    <div>
                                      <div dir="ltr">
                                        <div>
                                          <div dir="ltr">
                                            <div>What most experimenters
                                              take for granted before
                                              they begin their
                                              experiments is infinitely
                                              more interesting than any
                                              results to which their
                                              experiments lead.<br>
                                              -- Norbert Wiener</div>
                                            <div><br>
                                            </div>
                                            <div><a
                                                href="http://www.cse.buffalo.edu/~knepley/"
                                                target="_blank"
                                                moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a><br>
                                            </div>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </blockquote>
                          </div>
                        </blockquote>
                      </div>
                      <br clear="all">
                      <div><br>
                      </div>
                      -- <br>
                      <div dir="ltr"
                        class="m_8975814135343234898gmail_signature">
                        <div dir="ltr">
                          <div>
                            <div dir="ltr">
                              <div>
                                <div dir="ltr">
                                  <div>What most experimenters take for
                                    granted before they begin their
                                    experiments is infinitely more
                                    interesting than any results to
                                    which their experiments lead.<br>
                                    -- Norbert Wiener</div>
                                  <div><br>
                                  </div>
                                  <div><a
                                      href="http://www.cse.buffalo.edu/~knepley/"
                                      target="_blank"
                                      moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a><br>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </blockquote>
              </blockquote>
            </div>
          </blockquote>
        </div>
        <br clear="all">
        <div><br>
        </div>
        -- <br>
        <div dir="ltr" class="gmail_signature"
          data-smartmail="gmail_signature">
          <div dir="ltr">
            <div>
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <div>What most experimenters take for granted before
                      they begin their experiments is infinitely more
                      interesting than any results to which their
                      experiments lead.<br>
                      -- Norbert Wiener</div>
                    <div><br>
                    </div>
                    <div><a href="http://www.cse.buffalo.edu/~knepley/"
                        target="_blank" moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a><br>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
  </body>
</html>