<div dir="ltr">Dear all,<div><br></div><div>I'm trying to save and load distributed DMPlex in HDF5 format with DMView and DMLoad. The problem is even though there are no errors while loading the DM when I look at the DM loaded from file is not equal to the DM that was saved. Also, when loading the DM if DMSetType is called before the DMLoad, DMLoad produces errors (like line <a href="https://bitbucket.org/petsc/petsc/src/236f5a4d227fe9d0affddb8701edb9509ad39525/src/snes/examples/tutorials/ex12.c?at=master#cl-316">316</a> in src/snes/examples/tutorials/ex12.c ).</div><div><br></div><div>I'm including a small code to replicate the problem here plus an input mesh file. <br clear="all"><div><br></div>Best,
</div><div>Ata</div><div><br></div><div><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><font size="1">/**</font></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><font size="1"> * @file</font></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><font size="1"> *</font></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><font size="1"> * Created by Ataollah Mesgarnejad on 3/6/15.</font></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><span style="font-size:x-small"> </span><span style="font-size:x-small">*</span><br></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><font size="1"> * This is a test to read and write a distributed DM.</font></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><font size="1"> *</font></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)"><font size="1"> */</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"><br></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"><br></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(181,143,174)">static</span><span style="color:rgb(192,197,207)"> </span><span style="color:rgb(181,143,174)">char</span><span style="color:rgb(192,197,207)"> help[] = </span>"An example of the usage of PetscSF to scatter data back and forth between a \</font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1">a serial DM and its parallel counterpart.\n"<span style="color:rgb(192,197,207)">;</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"><br></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(150,181,180)">#include </span><petscdmplex.h></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(150,181,180)">#include </span><petscsnes.h></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(150,181,180)">#include </span><petscsf.h></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(150,181,180)">#include </span><exodusII.h></font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(150,181,180)">#include </span><span style="color:rgb(165,191,142)"><petsc-private/dmimpl.h></span><span style="color:rgb(150,181,180)"> </span>/*I "petscdm.h" I*/</font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(150,181,180)">#include </span><petscsf.h></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(150,181,180)">#include </span><petscviewerhdf5.h></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"><br></font></p><p style="margin:0px;font-family:Menlo;color:rgb(150,181,180)"><font size="1">#undef __FUNCT__</font></p><p style="margin:0px;font-family:Menlo;color:rgb(150,181,180)"><font size="1">#define __FUNCT__ <span style="color:rgb(165,191,142)">"main"</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"><span style="color:rgb(181,143,174)">int</span> main(<span style="color:rgb(181,143,174)">int</span> argc,<span style="color:rgb(181,143,174)">char</span> **argv)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1">{</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscErrorCode</span> ierr;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">DM</span> dm, distDM,distDMold;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">char</span> ifilename[<span style="color:rgb(150,181,180)">PETSC_MAX_PATH_LEN</span>];</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscBool</span> flg;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">int</span> CPU_word_size = <span style="color:rgb(209,135,113)">0</span>,IO_word_size = <span style="color:rgb(209,135,113)">0</span>,exoid;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">float</span> version;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> numproc,rank;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscViewer</span> stdoutViewer,DMPlexHDF5View;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscReal</span> *VArray;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> dim;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> numFields = <span style="color:rgb(209,135,113)">2</span>;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> numComp[<span style="color:rgb(209,135,113)">2</span>] = {<span style="color:rgb(209,135,113)">1</span>,<span style="color:rgb(209,135,113)">1</span>};</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> numDof[<span style="color:rgb(209,135,113)">6</span>] = {<span style="color:rgb(209,135,113)">1</span>, <span style="color:rgb(209,135,113)">0</span>, <span style="color:rgb(209,135,113)">0</span>,</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span><span style="color:rgb(209,135,113)">0</span><span style="color:rgb(192,197,207)">, </span><span style="color:rgb(209,135,113)">0</span><span style="color:rgb(192,197,207)">, </span><span style="color:rgb(209,135,113)">1</span><span style="color:rgb(192,197,207)">}; </span>/*{Vertex, Edge, Cell} */</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> bcFields[<span style="color:rgb(209,135,113)">1</span>] = {<span style="color:rgb(209,135,113)">0</span>}, numBC=<span style="color:rgb(209,135,113)">0</span>;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>//PetscInt *remoteOffsets;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">char</span>** namelist;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> off;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">IS</span> bcPoints[<span style="color:rgb(209,135,113)">1</span>] = {<span style="color:rgb(181,143,174)">NULL</span>};</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">IS</span> *ISlist;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscSection</span> seqSection, distSection;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">Vec</span> distV, V;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscSF</span> pointSF;<span style="color:rgb(79,90,102)">//,pointSFold;</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscInt</span> pStart, pEnd, dof;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">PetscBool</span> load_files = <span style="color:rgb(191,98,107)">PETSC_FALSE</span>, save_files = <span style="color:rgb(191,98,107)">PETSC_FALSE</span>;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(235,202,138)">MPI_Comm</span> comm;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscInitialize</span>(&argc, &argv, (<span style="color:rgb(181,143,174)">char</span>*)<span style="color:rgb(209,135,113)">0</span>, <span style="color:rgb(143,161,178)">help</span>);<span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscViewerASCIIGetStdout</span>(<span style="color:rgb(150,181,180)">PETSC_COMM_SELF</span>,&stdoutViewer);<span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(235,202,138)"><font size="1"><span style="color:rgb(192,197,207)"> comm = </span>PETSC_COMM_WORLD<span style="color:rgb(192,197,207)">;</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>// PetscErrorCode DMCreateSubDM(DM dm, PetscInt numFields, PetscInt fields[], IS *is, DM *subdm)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(192,197,207)"> ierr = </span><span style="color:rgb(150,181,180)">PetscOptionsBegin</span><span style="color:rgb(192,197,207)">(</span><span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span><span style="color:rgb(192,197,207)">,</span>""<span style="color:rgb(192,197,207)">,</span>"PetscSF Test Options"<span style="color:rgb(192,197,207)">,</span>"none"<span style="color:rgb(192,197,207)">);</span><span style="color:rgb(150,181,180)">CHKERRQ</span><span style="color:rgb(192,197,207)">(ierr);</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(192,197,207)"> ierr = </span><span style="color:rgb(150,181,180)">PetscOptionsBool</span><span style="color:rgb(192,197,207)">(</span>"-save_files"<span style="color:rgb(192,197,207)">,</span>"save the distributed vector in HDF5 format"<span style="color:rgb(192,197,207)">,</span>""<span style="color:rgb(192,197,207)">,</span><span style="color:rgb(191,98,107)">PETSC_FALSE</span><span style="color:rgb(192,197,207)">,&save_files,</span><span style="color:rgb(181,143,174)">NULL</span><span style="color:rgb(192,197,207)">);</span><span style="color:rgb(150,181,180)">CHKERRQ</span><span style="color:rgb(192,197,207)">(ierr);</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(192,197,207)"> ierr = </span><span style="color:rgb(150,181,180)">PetscOptionsBool</span><span style="color:rgb(192,197,207)">(</span>"-load_files"<span style="color:rgb(192,197,207)">,</span>"Load the distributed vector in HDF5 format"<span style="color:rgb(192,197,207)">,</span>""<span style="color:rgb(192,197,207)">,</span><span style="color:rgb(191,98,107)">PETSC_FALSE</span><span style="color:rgb(192,197,207)">,&load_files,</span><span style="color:rgb(181,143,174)">NULL</span><span style="color:rgb(192,197,207)">);</span><span style="color:rgb(150,181,180)">CHKERRQ</span><span style="color:rgb(192,197,207)">(ierr);</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(150,181,180)"><font size="1"><span style="color:rgb(192,197,207)"> ierr = </span>PetscOptionsEnd<span style="color:rgb(192,197,207)">();</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscOptionsGetString</span>(<span style="color:rgb(150,181,180)">PETSC_NULL</span>,<span style="color:rgb(165,191,142)">"-i"</span>,ifilename,<span style="color:rgb(181,143,174)">sizeof</span> ifilename,&flg);<span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">MPI_Comm_size</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>,&numproc);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">MPI_Comm_rank</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>,&rank);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">if</span> (!rank)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> exoid = <span style="color:rgb(150,181,180)">ex_open</span>(ifilename,<span style="color:rgb(150,181,180)">EX_READ</span>,&CPU_word_size,&IO_word_size,&version);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(192,197,207)"> </span><span style="color:rgb(181,143,174)">if</span><span style="color:rgb(192,197,207)"> (exoid <= </span><span style="color:rgb(209,135,113)">0</span><span style="color:rgb(192,197,207)">) </span><span style="color:rgb(150,181,180)">SETERRQ1</span><span style="color:rgb(192,197,207)">(</span><span style="color:rgb(150,181,180)">PETSC_COMM_SELF</span><span style="color:rgb(192,197,207)">,</span><span style="color:rgb(150,181,180)">PETSC_ERR_LIB</span><span style="color:rgb(192,197,207)">,</span>"ex_open(\"%s\",...) did not return a valid file ID"<span style="color:rgb(192,197,207)">,ifilename);</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">else</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> exoid = -<span style="color:rgb(209,135,113)">1</span>; <span style="color:rgb(79,90,102)">/* Not used */</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMPlexCreateExodus</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>,exoid,<span style="color:rgb(191,98,107)">PETSC_FALSE</span>,&dm); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscObjectSetName</span>((<span style="color:rgb(235,202,138)">PetscObject</span>) dm,<span style="color:rgb(165,191,142)">"sequential-DM"</span>);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMGetDimension</span>(dm, &dim); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">if</span> (numproc < <span style="color:rgb(209,135,113)">2</span> ) distDM = dm;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">else</span> ierr = <span style="color:rgb(191,98,107)">DMPlexDistribute</span>(dm, <span style="color:rgb(209,135,113)">0</span>, &pointSF, &distDM); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscObjectSetName</span>((<span style="color:rgb(235,202,138)">PetscObject</span>) distDM,<span style="color:rgb(165,191,142)">"Distributed-DM"</span>);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"><br></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMPlexCreateSection</span>(dm, dim, numFields, numComp, numDof,</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> numBC, bcFields, bcPoints,<span style="color:rgb(150,181,180)">PETSC_NULL</span>, &seqSection); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMPlexCreateSection</span>(distDM, dim, numFields, numComp, numDof,</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> numBC, bcFields, bcPoints,<span style="color:rgb(150,181,180)">PETSC_NULL</span>, &distSection); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMSetDefaultSection</span>(dm, seqSection); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMSetDefaultSection</span>(distDM, distSection); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMCreateGlobalVector</span>(dm, &V); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">VecCreate</span>(comm, &distV); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscObjectSetName</span>((<span style="color:rgb(235,202,138)">PetscObject</span>) distV,<span style="color:rgb(165,191,142)">"Distributed-V"</span>);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"><br></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">VecGetArray</span>(V, &VArray); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>// fill vertex data</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMPlexGetDepthStratum</span>(dm, <span style="color:rgb(209,135,113)">0</span>, &pStart, &pEnd); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">for</span> (<span style="color:rgb(181,143,174)">int</span> p = pStart; p < pEnd; p++)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscSectionGetDof</span>(seqSection, p, &dof); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscSectionGetOffset</span>(seqSection, p, &off); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">for</span> (<span style="color:rgb(181,143,174)">int</span> d = <span style="color:rgb(209,135,113)">0</span>; d < dof; d++)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> VArray[off + d] = p;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>// fill cell data</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMPlexGetDepthStratum</span>(dm, <span style="color:rgb(209,135,113)">1</span>, &pStart, &pEnd); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">for</span> (<span style="color:rgb(181,143,174)">int</span> p = pStart; p < pEnd; p++)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscSectionGetDof</span>(seqSection, p, &dof); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscSectionGetOffset</span>(seqSection, p, &off); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">for</span> (<span style="color:rgb(181,143,174)">int</span> d = <span style="color:rgb(209,135,113)">0</span>; d < dof; d++)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> VArray[off + d] = -p-<span style="color:rgb(209,135,113)">1</span>;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">VecRestoreArray</span>(V, &VArray); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMCreateFieldIS</span>(dm, &numFields, &namelist, &ISlist); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMPlexDistributeField</span>(dm, pointSF, seqSection, V, distSection, distV); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(150,181,180)"><font size="1">#if defined(PETSC_HAVE_HDF5)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">if</span> (save_files)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>// distribute fields</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscPrintf</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>, <span style="color:rgb(165,191,142)">"==== Distrubuted DM\n"</span>); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMView</span>(distDM, <span style="color:rgb(150,181,180)">PETSC_VIEWER_STDOUT_WORLD</span>); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(235,202,138)"><font size="1"><span style="color:rgb(192,197,207)"> </span><span style="color:rgb(191,98,107)">PetscSynchronizedFlush</span><span style="color:rgb(192,197,207)">(</span>PETSC_COMM_WORLD<span style="color:rgb(192,197,207)">, </span>PETSC_STDOUT<span style="color:rgb(192,197,207)">); </span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>// Save distributed data</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscViewerHDF5Open</span>(<span style="color:rgb(191,98,107)">PetscObjectComm</span>((<span style="color:rgb(235,202,138)">PetscObject</span>) distDM),<span style="color:rgb(165,191,142)">"distDM.h5"</span>, <span style="color:rgb(191,98,107)">FILE_MODE_WRITE</span>, &DMPlexHDF5View);<span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(192,197,207)"> ierr = </span><span style="color:rgb(191,98,107)">PetscPrintf</span><span style="color:rgb(192,197,207)">(</span><span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span><span style="color:rgb(192,197,207)">,</span>"Writing dist DM ...\n"<span style="color:rgb(192,197,207)">);</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(235,202,138)"><font size="1"><span style="color:rgb(192,197,207)"> </span><span style="color:rgb(191,98,107)">PetscSynchronizedFlush</span><span style="color:rgb(192,197,207)">(</span>PETSC_COMM_WORLD<span style="color:rgb(192,197,207)">, </span>PETSC_STDOUT<span style="color:rgb(192,197,207)">);</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMView</span>(distDM, DMPlexHDF5View); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(191,98,107)">PetscViewerDestroy</span>(&DMPlexHDF5View);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">else</span> <span style="color:rgb(181,143,174)">if</span> (load_files)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscPrintf</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>,<span style="color:rgb(165,191,142)">"Loading dist vectors ...\n"</span>); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscViewerHDF5Open</span>(<span style="color:rgb(150,181,180)">PETSC_COMM_SELF</span>,<span style="color:rgb(165,191,142)">"distDM.h5"</span>, <span style="color:rgb(191,98,107)">FILE_MODE_READ</span>, &DMPlexHDF5View); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMCreate</span>(comm,&distDMold); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMLoad</span>(distDMold,DMPlexHDF5View); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMSetType</span>(distDMold, <span style="color:rgb(150,181,180)">DMPLEX</span>); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>/*ierr = DMPlexCreateSection(distDMold, dim, numFields, numComp, numDof,</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"> numBC, bcFields, bcPoints,PETSC_NULL, &distSectionold); CHKERRQ(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"> ierr = DMSetDefaultSection(distDMold, distSectionold);CHKERRQ(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"> ierr = DMGetPointSF(distDMold,&pointSFold); CHKERRQ(ierr);*/</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(191,98,107)">PetscViewerDestroy</span>(&DMPlexHDF5View);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscPrintf</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>, <span style="color:rgb(165,191,142)">"==== Read DM\n"</span>); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMView</span>(distDMold, <span style="color:rgb(150,181,180)">PETSC_VIEWER_STDOUT_WORLD</span>); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(191,98,107)">PetscSynchronizedFlush</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>, <span style="color:rgb(235,202,138)">PETSC_STDOUT</span>); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(191,98,107)">DMPlexEqual</span>(distDM, distDMold, &flg);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">if</span> (flg)</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(191,98,107)">PetscPrintf</span>(<span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span>,<span style="color:rgb(165,191,142)">"\n DMs equal\n"</span>); </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">else</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> {</font></p><p style="margin:0px;font-family:Menlo;color:rgb(165,191,142)"><font size="1"><span style="color:rgb(192,197,207)"> </span><span style="color:rgb(191,98,107)">PetscPrintf</span><span style="color:rgb(192,197,207)">(</span><span style="color:rgb(235,202,138)">PETSC_COMM_WORLD</span><span style="color:rgb(192,197,207)">,</span>"\n DMs are not equal\n\n"<span style="color:rgb(192,197,207)">);</span></font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> }</font></p><p style="margin:0px;font-family:Menlo;color:rgb(150,181,180)"><font size="1">#endif</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">VecDestroy</span>(&V); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">VecDestroy</span>(&distV); <span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(79,90,102)"><font size="1"><span style="color:rgb(192,197,207)"> </span>//ierr = VecDestroy(&seqV); CHKERRQ(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMDestroy</span>(&dm);<span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">DMDestroy</span>(&distDM);<span style="color:rgb(150,181,180)">CHKERRQ</span>(ierr);</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207);min-height:21px"><font size="1"> </font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> ierr = <span style="color:rgb(191,98,107)">PetscFinalize</span>();</font></p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1"> <span style="color:rgb(181,143,174)">return</span> <span style="color:rgb(209,135,113)">0</span>;</font></p><p style="margin:0px;font-family:Menlo;color:rgb(99,114,125)">
</p><p style="margin:0px;font-family:Menlo;color:rgb(192,197,207)"><font size="1">}</font></p></div><div><br></div><div><br></div></div>