<div dir="ltr"><div dir="ltr">On Wed, Jul 14, 2021 at 4:58 PM Eric Chamberland <<a href="mailto:Eric.Chamberland@giref.ulaval.ca">Eric.Chamberland@giref.ulaval.ca</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div>
    <p>Hi Matthew,</p>
    <p>Ok, I did that but it segfault now.  Here are the order of the
      calls:</p>
    <p>DMPlexCreate</p>
    <p>DMSetDimension</p>
    <p>DMPlexBuildFromCellListParallel(...)</p>
    <p>DMPlexInterpolate</p>
    <p>PetscPartitioner lPart;<br>
      DMPlexGetPartitioner(lDMSansOverlap, &lPart);<br>
      PetscPartitionerSetFromOptions(lPart);<br>
      <br>
      DMSetUseNatural(lDMSansOverlap, PETSC_TRUE)<br>
    </p>
    <p>DMPlexDistribute</p>
    <p>DMPlexGlobalToNaturalBegin</p>
    <p>DMPlexGlobalToNaturalEnd</p>
    <p><br>
    </p>
    <p>But it gives me the following error:</p>
    <p>0]PETSC ERROR: --------------------- Error Message
      --------------------------------------------------------------<br>
      [0]PETSC ERROR: Petsc has generated inconsistent data<br>
      [0]PETSC ERROR: DM global to natural SF not present.<br>
      If DMPlexDistribute() was called and a section was defined, report
      to <a href="mailto:petsc-maint@mcs.anl.gov" target="_blank">petsc-maint@mcs.anl.gov</a>.<br>
      <br>
      [0]PETSC ERROR: See
      <a href="https://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">https://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble
      shooting.<br>
      [0]PETSC ERROR: Petsc Release Version 3.15.0, Mar 30, 2021 <br>
      [0]PETSC ERROR: MEF++.dev on a  named rohan by ericc Wed Jul 14
      16:57:48 2021<br>
      [0]PETSC ERROR: Configure options
      --prefix=/opt/petsc-3.15.0_debug_openmpi-4.1.0_gcc7
      --with-mpi-compilers=1 --with-mpi-dir=/opt/openmpi-4.1.0_gcc7
      --with-cxx-dialect=C++14 --with-make-np=12
      --with-shared-libraries=1 --with-debugging=yes --with-memalign=64
      --with-visibility=0 --with-64-bit-indices=0 --download-ml=yes
      --download-mumps=yes --download-superlu=yes --download-hpddm=yes
      --download-slepc=yes --download-superlu_dist=yes
      --download-parmetis=yes --download-ptscotch=yes
      --download-metis=yes --download-strumpack=yes
      --download-suitesparse=yes --download-hypre=yes
      --with-blaslapack-dir=/opt/intel/oneapi/mkl/2021.1.1/env/../lib/intel64
      --with-mkl_pardiso-dir=/opt/intel/oneapi/mkl/2021.1.1/env/..
      --with-mkl_cpardiso-dir=/opt/intel/oneapi/mkl/2021.1.1/env/..
      --with-scalapack=1
      --with-scalapack-include=/opt/intel/oneapi/mkl/2021.1.1/env/../include
--with-scalapack-lib="-L/opt/intel/oneapi/mkl/2021.1.1/env/../lib/intel64
      -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64"<br>
      [0]PETSC ERROR: #1 DMPlexGlobalToNaturalBegin() at
      /tmp/ompi-opt/petsc-3.15.0-debug/src/dm/impls/plex/plexnatural.c:245<br>
      <br>
    </p>
    <p>What did I missed?</p>
    <p></p></div></blockquote><div>Ah, there was a confusion of intent. GlobalToNatural() is for people that want data transformed back into the original</div><div>order. I thought that was what you wanted. If you just want mesh points in the original order, we give you the</div><div>transformation as part of the output of DMPlexDistribute(). The migrationSF that is output maps the original point to</div><div>the distributed point. You run it backwards to get the original ordering.</div><div><br></div><div>  Thanks,</div><div><br></div><div>     Matt </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p>Thanks a lot!<br>
    </p>
    <p>Eric<br>
    </p>
    <div>On 2021-07-14 3:09 p.m., Matthew
      Knepley wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">
        <div dir="ltr">On Wed, Jul 14, 2021 at 1:18 PM Eric Chamberland
          <<a href="mailto:Eric.Chamberland@giref.ulaval.ca" target="_blank">Eric.Chamberland@giref.ulaval.ca</a>>
          wrote:<br>
        </div>
        <div class="gmail_quote">
          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
            <br>
            I want to use DMPlexDistribute from PETSc for computing
            overlapping and <br>
            play with the different partitioners supported.<br>
            <br>
            However, after calling DMPlexDistribute, I noticed the
            elements are <br>
            renumbered and then the original number is lost.<br>
            <br>
            What would be the best way to keep track of the element
            renumbering?<br>
            <br>
            a) Adding an optional parameter to let the user retrieve a
            vector or <br>
            "IS" giving the old number?<br>
            <br>
            b) Adding a DMLabel (seems a wrong good solution)<br>
            <br>
            c) Other idea?<br>
            <br>
            Of course, I don't want to loose performances with the need
            of this <br>
            "mapping"...<br>
          </blockquote>
          <div><br>
          </div>
          <div>You need to call</div>
          <div><br>
          </div>
          <div>  <a href="https://petsc.org/release/docs/manualpages/DM/DMSetUseNatural.html" target="_blank">https://petsc.org/release/docs/manualpages/DM/DMSetUseNatural.html</a></div>
          <div><br>
          </div>
          <div>before call DMPlexDistribute(). Then you can call</div>
          <div><br>
          </div>
          <div>  <a href="https://petsc.org/release/docs/manualpages/DMPLEX/DMPlexGlobalToNaturalBegin.html" target="_blank">https://petsc.org/release/docs/manualpages/DMPLEX/DMPlexGlobalToNaturalBegin.html</a></div>
          <div><br>
          </div>
          <div>to map back to the original numbering if you want. This
            is the same thing that DMDA is doing.</div>
          <div><br>
          </div>
          <div>  Thanks,</div>
          <div><br>
          </div>
          <div>     Matt</div>
          <div> </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
            Thanks,<br>
            <br>
            Eric<br>
            <br>
            -- <br>
            Eric Chamberland, ing., M. Ing<br>
            Professionnel de recherche<br>
            GIREF/Université Laval<br>
            (418) 656-2131 poste 41 22 42<br>
            <br>
          </blockquote>
        </div>
        <br clear="all">
        <div><br>
        </div>
        -- <br>
        <div dir="ltr">
          <div dir="ltr">
            <div>
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <div>What most experimenters take for granted before
                      they begin their experiments is infinitely more
                      interesting than any results to which their
                      experiments lead.<br>
                      -- Norbert Wiener</div>
                    <div><br>
                    </div>
                    <div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <pre cols="72">-- 
Eric Chamberland, ing., M. Ing
Professionnel de recherche
GIREF/Université Laval
(418) 656-2131 poste 41 22 42</pre>
  </div>

</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>