<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    We do exactly this by using the same prefix for each file and bump
    the number with each load step, then paraview does the stacking
    automagically for us.  However we write out VTU files for our FEA
    computations.  <br>
    <br>
    Perhaps you could examine some of the other formats that paraview
    can read and see if they do the trick.<br>
    <br>
    -sanjay<br>
    <pre class="moz-signature" cols="72">
-------------------------------------------------------------------

</pre>
    <div class="moz-cite-prefix">On 1/30/25 11:59 PM, Anna Dalklint via
      petsc-users wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:GVYP280MB0987041BD9C9E904D3132279D2E82@GVYP280MB0987.SWEP280.PROD.OUTLOOK.COM">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <meta name="Generator"
        content="Microsoft Word 15 (filtered medium)">
      <style>@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
        {font-family:Aptos;
        panose-1:2 11 0 4 2 2 2 2 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        font-size:12.0pt;
        font-family:"Aptos",sans-serif;}a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}span.m4625291697624802812gmailsignatureprefix
        {mso-style-name:m_4625291697624802812gmailsignatureprefix;}span.gmailsignatureprefix
        {mso-style-name:gmail_signature_prefix;}.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;
        mso-ligatures:none;}div.WordSection1
        {page:WordSection1;}</style>
      <div class="WordSection1">
        <p class="MsoNormal"><span style="font-size:11.0pt">I want to
            save e.g. the discretized displacement field obtained from a
            quasi-static non-linear finite element simulation using 10
            node tetrahedral elements (i.e. which has edge dofs). As
            mentioned, I use PetscSection to add the additional dofs on
            edges. I have also written my own Newton solver, i.e. I do
            not use SNES. In conclusion, what I want is to be able to
            save the discretized displacement field in each outer
            iteration of the Newton loop (where I increase the
            pseudo-time, i.e. scaling of the load). I would then
            preferably be able to load a stack of these files (call them
            u001, u002, u003… for each “load-step”) and step in “time”
            in ParaView.<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">Thanks,<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">Anna<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <div id="mail-editor-reference-message-container">
          <div>
            <div>
              <div
style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
                <p class="MsoNormal" style="margin-bottom:12.0pt"><b><span
                      style="color:black">From:
                    </span></b><span style="color:black">Matthew Knepley
                    <a class="moz-txt-link-rfc2396E" href="mailto:knepley@gmail.com"><knepley@gmail.com></a><br>
                    <b>Date: </b>Thursday, 30 January 2025 at 16:19<br>
                    <b>To: </b>Anna Dalklint
                    <a class="moz-txt-link-rfc2396E" href="mailto:anna.dalklint@solid.lth.se"><anna.dalklint@solid.lth.se></a><br>
                    <b>Cc: </b>Jed Brown <a class="moz-txt-link-rfc2396E" href="mailto:jed@jedbrown.org"><jed@jedbrown.org></a>,
                    <a class="moz-txt-link-abbreviated" href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>
                    <a class="moz-txt-link-rfc2396E" href="mailto:petsc-users@mcs.anl.gov"><petsc-users@mcs.anl.gov></a><br>
                    <b>Subject: </b>Re: [petsc-users] Visualizing
                    higher order finite element output in ParaView<o:p></o:p></span></p>
              </div>
              <div>
                <div>
                  <p class="MsoNormal">On Thu, Jan 30, 2025 at 9:43<span
                      style="font-family:"Arial",sans-serif"> </span>AM
                    Anna Dalklint <<a
                      href="mailto:anna.dalklint@solid.lth.se"
                      moz-do-not-send="true"
                      class="moz-txt-link-freetext">anna.dalklint@solid.lth.se</a>>
                    wrote:</p>
                </div>
                <div>
                  <blockquote
style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
                    <div>
                      <div>
                        <div>
                          <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
                              style="font-size:11.0pt">I looked deeper
                              into the petsc codebase regarding HDF5.
                              From what I understood (which of course
                              can be wrong), the current version of
                              petsc does not save edge
                              degrees-of-freedom to HDF5? Is this
                              something you plan to allow?</span><o:p></o:p></p>
                        </div>
                      </div>
                    </div>
                  </blockquote>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">We write two different outputs
                      (by default). One has all the data, and one has
                      only cell and vertex data because Paraview does
                      not understand anything else. This can be
                      customized with options. What do you want to save?</p>
                  </div>
                  <div>
                    <p class="MsoNormal"> </p>
                  </div>
                  <blockquote
style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
                    <div>
                      <div>
                        <div>
                          <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
                              style="font-size:11.0pt">Otherwise I’m
                              fine with using CGNS. But could you please
                              explain how I could save timeseries that
                              paraview recognizes using this format?
                              Right now I’m saving files e.g.
                              file0001.cgns, file0002.cgns, … where each
                              .cgns file is written using VecView (i.e.
                              it stores a discretized field). But
                              paraview cannot load this as a timeseries.</span><o:p></o:p></p>
                        </div>
                      </div>
                    </div>
                  </blockquote>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">Jed can explain how this works.</p>
                  </div>
                  <div>
                    <p class="MsoNormal"> </p>
                  </div>
                  <blockquote
style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
                    <div>
                      <div>
                        <div>
                          <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
                              style="font-size:11.0pt">Also, do you have
                              any documentation regarding node (vertex,
                              edge, face, cell) numbering? E.g. how
                              would a 10 node tetrahedral be numbered?
                              From the documentation on your webpage (</span><a
href="https://urldefense.us/v3/__https://petsc.org/release/manual/dmplex/__;!!G_uCfscf7eWS!frd8lJGukGrnZYqiJQyrOszbkCacSP2EftDVAAiYClDx1Ll0dEd7q8th5yDSI1bJgVmvCAGtgrNVnkfcw0i47jUVzWH0vcxIcg$"
                              target="_blank" moz-do-not-send="true"><span
                                style="font-size:11.0pt">https://petsc.org/release/manual/dmplex/</span></a><span
                              style="font-size:11.0pt">) it looks like
                              cell dofs -> vertex dofs-> face
                              dofs-> edge dofs. Is this correct?</span><o:p></o:p></p>
                        </div>
                      </div>
                    </div>
                  </blockquote>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">When you call
                      DMPlexVecGetClosure(), the closure follows the
                      point numbering, in that for each point, we lookup
                      the dofs in the local Section, and push them into
                      the array in order. So then you need the point
                      ordering. For the closure, it goes by dimension,
                      so cell dofs, face dofs, edge dofs, vertex dofs.
                      You can see the definition of faces (and edges)
                      here:</p>
                  </div>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">  <a
href="https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/plexinterpolate.c?ref_type=heads*L196__;Iw!!G_uCfscf7eWS!frd8lJGukGrnZYqiJQyrOszbkCacSP2EftDVAAiYClDx1Ll0dEd7q8th5yDSI1bJgVmvCAGtgrNVnkfcw0i47jUVzWFlsiC2ww$"
                        moz-do-not-send="true">https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/plexinterpolate.c?ref_type=heads#L196</a></p>
                  </div>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">and triangles are ordered here</p>
                  </div>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">  <a
href="https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/plexinterpolate.c?ref_type=heads*L115__;Iw!!G_uCfscf7eWS!frd8lJGukGrnZYqiJQyrOszbkCacSP2EftDVAAiYClDx1Ll0dEd7q8th5yDSI1bJgVmvCAGtgrNVnkfcw0i47jUVzWFvuUov2w$"
                        moz-do-not-send="true">https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/plexinterpolate.c?ref_type=heads#L115</a></p>
                  </div>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">The idea is
                      that DMPlexVecGetClosure() delivers the dofs in a
                      standard order on the element, so that you can
                      write</p>
                  </div>
                  <div>
                    <p class="MsoNormal">your residual function once.
                      Also, for multiple fields, they are stacked
                      contiguously, so the numbering is [field, point,
                      dof on point].</p>
                  </div>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">Let me know if that does not
                      make sense.</p>
                  </div>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">  Thanks,</p>
                  </div>
                  <div>
                    <p class="MsoNormal"><o:p> </o:p></p>
                  </div>
                  <div>
                    <p class="MsoNormal">     Matt</p>
                  </div>
                  <div>
                    <p class="MsoNormal"> </p>
                  </div>
                  <blockquote
style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
                    <div>
                      <div>
                        <div>
                          <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
                              style="font-size:11.0pt">Thanks,</span><o:p></o:p></p>
                          <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
                              style="font-size:11.0pt">Anna</span><o:p></o:p></p>
                          <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
                              style="font-size:11.0pt"> </span><o:p></o:p></p>
                          <div
id="m_4625291697624802812mail-editor-reference-message-container">
                            <div>
                              <div>
                                <div
style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
                                  <p class="MsoNormal"
style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><b><span
                                        style="color:black">From:
                                      </span></b><span
                                      style="color:black">Matthew
                                      Knepley <</span><a
                                      href="mailto:knepley@gmail.com"
                                      target="_blank"
                                      moz-do-not-send="true"
                                      class="moz-txt-link-freetext">knepley@gmail.com</a><span
                                      style="color:black">><br>
                                      <b>Date: </b>Thursday, 30 January
                                      2025 at 00:39<br>
                                      <b>To: </b>Jed Brown <</span><a
                                      href="mailto:jed@jedbrown.org"
                                      target="_blank"
                                      moz-do-not-send="true"
                                      class="moz-txt-link-freetext">jed@jedbrown.org</a><span
                                      style="color:black">><br>
                                      <b>Cc: </b>Anna Dalklint <</span><a
href="mailto:anna.dalklint@solid.lth.se" target="_blank"
                                      moz-do-not-send="true"
                                      class="moz-txt-link-freetext">anna.dalklint@solid.lth.se</a><span
                                      style="color:black">>,
                                    </span><a
href="mailto:petsc-users@mcs.anl.gov" target="_blank"
                                      moz-do-not-send="true"
                                      class="moz-txt-link-freetext">petsc-users@mcs.anl.gov</a><span
                                      style="color:black"> <</span><a
href="mailto:petsc-users@mcs.anl.gov" target="_blank"
                                      moz-do-not-send="true"
                                      class="moz-txt-link-freetext">petsc-users@mcs.anl.gov</a><span
                                      style="color:black">><br>
                                      <b>Subject: </b>Re: [petsc-users]
                                      Visualizing higher order finite
                                      element output in ParaView</span><o:p></o:p></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">That is all
                                    true. If you want lower level pieces
                                    to make it yourself, I have
                                    -dm_plex_high_order_view, which
                                    activates<o:p></o:p></p>
                                  <div>
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">DMPlexCreateHighOrderSurrogate_Internal().
                                      This is a simple function that
                                      refines the mesh lg(p) times to
                                      try and<o:p></o:p></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">resolve the
                                      high order behavior.<o:p></o:p></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">  Thanks,<o:p></o:p></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">     Matt<o:p></o:p></p>
                                  </div>
                                </div>
                                <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
                                <div>
                                  <div>
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Wed, Jan
                                      29, 2025 at 4:55<span
style="font-family:"Arial",sans-serif"> </span>PM Jed Brown
                                      <<a
                                        href="mailto:jed@jedbrown.org"
                                        target="_blank"
                                        moz-do-not-send="true"
                                        class="moz-txt-link-freetext">jed@jedbrown.org</a>>
                                      wrote:<o:p></o:p></p>
                                  </div>
                                  <blockquote
style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
                                    <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I like the
                                      CGNS workflow for this, at least
                                      with quadratic and cubic elements.
                                      You can use options like
                                      -snes_view_solution
                                      cgns:solution.cgns (configure with
                                      --download-cgns). It can also
                                      monitor transient solves with
                                      flexible batch sizes (geometry and
                                      connectivity are stored only once
                                      within a batch of output frames).<br>
                                      <br>
                                      Anna Dalklint via petsc-users <<a
href="mailto:petsc-users@mcs.anl.gov" target="_blank"
                                        moz-do-not-send="true"
                                        class="moz-txt-link-freetext">petsc-users@mcs.anl.gov</a>>
                                      writes:<br>
                                      <br>
                                      > Hello,<br>
                                      ><br>
                                      > We have created a finite
                                      element code in PETSc for
                                      unstructured meshes using DMPlex.
                                      The first order meshes are created
                                      in gmsh and loaded into PETSc. To
                                      introduce higher order elements,
                                      e.g. 10 node tetrahedral elements,
                                      we start from scratch using
                                      PetscSection and loop over the
                                      relevant points it the DM to
                                      introduce additional
                                      degrees-of-freedom (example; for
                                      10 node tets we have 4 vertices
                                      “nodes” and 6 edge “nodes”). The
                                      coordinates of the new “nodes” are
                                      obtained by interpolation using
                                      the finite element basis
                                      functions.<br>
                                      ><br>
                                      > The simulations seem to run
                                      well, but we face issues when
                                      trying to visualize the results in
                                      ParaView. We have tried to use
                                      both CGNS and HDF5+XDMF file
                                      formats for e.g. VecView. CGNS
                                      works, but the edge
                                      degrees-of-freedom appear to not
                                      be interpolated correctly (we
                                      observe oscillations in the
                                      fields, don’t know if this is a
                                      PETSc och ParaView issue). Also,
                                      we would prefer to use another
                                      file format than CGNS since it
                                      does not appear to directly allow
                                      timeseries (at least ParaView
                                      doesn’t recognize it). We haven’t
                                      got the HDF5+XDMF file format to
                                      work at all when running on more
                                      than one core (the mesh is highly
                                      distorted when saving using
                                      VecView and DMView + running the
                                      “petsc_gen_xdmf.py” script on the
                                      .h5 output file).<br>
                                      ><br>
                                      > VTU format works but then
                                      only the vertices’
                                      degrees-of-freedom are visualized.
                                      As far as we have understood it,
                                      this is because VTU/VTK only
                                      supports degrees-of-freedom on
                                      vertices/cell level.<br>
                                      ><br>
                                      > Does anyone have any idea of
                                      how to visualize fields generated
                                      from higher order elements in
                                      ParaView? Or understand what we
                                      might be doing wrong?<br>
                                      ><br>
                                      > Best regards,<br>
                                      > Anna<o:p></o:p></p>
                                  </blockquote>
                                </div>
                                <div>
                                  <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br
                                      clear="all">
                                    <o:p></o:p></p>
                                </div>
                                <div>
                                  <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
                                </div>
                                <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
class="m4625291697624802812gmailsignatureprefix">--
                                  </span><o:p></o:p></p>
                                <div>
                                  <div>
                                    <div>
                                      <div>
                                        <div>
                                          <div>
                                            <div>
                                              <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">What most
                                                experimenters take for
                                                granted before they
                                                begin their experiments
                                                is infinitely more
                                                interesting than any
                                                results to which their
                                                experiments lead.<br>
                                                -- Norbert Wiener<o:p></o:p></p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a
href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!frd8lJGukGrnZYqiJQyrOszbkCacSP2EftDVAAiYClDx1Ll0dEd7q8th5yDSI1bJgVmvCAGtgrNVnkfcw0i47jUVzWH7uFxLgw$"
                                                  target="_blank"
                                                  moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a><o:p></o:p></p>
                                            </div>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </blockquote>
                </div>
                <div>
                  <p class="MsoNormal"><br clear="all">
                  </p>
                </div>
                <div>
                  <p class="MsoNormal"><o:p> </o:p></p>
                </div>
                <p class="MsoNormal"><span class="gmailsignatureprefix">--
                  </span></p>
                <div>
                  <div>
                    <div>
                      <div>
                        <div>
                          <div>
                            <div>
                              <p class="MsoNormal">What most
                                experimenters take for granted before
                                they begin their experiments is
                                infinitely more interesting than any
                                results to which their experiments lead.<br>
                                -- Norbert Wiener</p>
                            </div>
                            <div>
                              <p class="MsoNormal"><o:p> </o:p></p>
                            </div>
                            <div>
                              <p class="MsoNormal"><a
href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!frd8lJGukGrnZYqiJQyrOszbkCacSP2EftDVAAiYClDx1Ll0dEd7q8th5yDSI1bJgVmvCAGtgrNVnkfcw0i47jUVzWH7uFxLgw$"
                                  target="_blank" moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a></p>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>