[petsc-users] PetscViewer with 64bit
Mike Michell
mi.mike1021 at gmail.com
Fri Feb 17 13:13:30 CST 2023
It works as I wanted. Thank you for the change.
Mike
> Okay, this works now. I'm pretty sure I tested this long ago with
> connectivity using Int64 and found that didn't work. That may have been
> ancient history, but I'm hesitant to revamp to match PetscInt. If doing
> that, it would require changing the signature of DMPlexGetVTKConnectivity
> to use PetscInt instead of PetscVTKInt. I'm already underwater and don't
> have the stamina to test it, but this MR will get you going for problems in
> which individual parts don't have more than 2B dofs.
>
>
> https://gitlab.com/petsc/petsc/-/merge_requests/6081/diffs?commit_id=27ba695b8b62ee2bef0e5776c33883276a2a1735
>
> Mike Michell <mi.mike1021 at gmail.com> writes:
>
> > Jed,
> >
> > It does not work for me even with the reproducer case with the small 2D
> > square mesh. Can you run the case that I sent and open the created
> > "sol.vtu" file with paraview?
> >
> > Thanks,
> >
> >
> >> Thanks, Dave.
> >>
> >> Mike, can you test that this branch works with your large problems? I
> >> tested that .vtu works in parallel for small problems, where works =
> loads
> >> correctly in Paraview and VisIt.
> >>
> >> https://gitlab.com/petsc/petsc/-/merge_requests/6081
> >>
> >> Dave May <dave.mayhem23 at gmail.com> writes:
> >>
> >> > On Tue 14. Feb 2023 at 21:27, Jed Brown <jed at jedbrown.org> wrote:
> >> >
> >> >> Dave May <dave.mayhem23 at gmail.com> writes:
> >> >>
> >> >> > On Tue 14. Feb 2023 at 17:17, Jed Brown <jed at jedbrown.org> wrote:
> >> >> >
> >> >> >> Can you share a reproducer? I think I recall the format requiring
> >> >> certain
> >> >> >> things to be Int32.
> >> >> >
> >> >> >
> >> >> > By default, the byte offset used with the appended data format is
> >> >> UInt32. I
> >> >> > believe that’s where the sizeof(int) is coming from. This default
> is
> >> >> > annoying as it limits the total size of your appended data to be <
> 3
> >> GB.
> >> >> > That said, in the opening of the paraview file you can add this
> >> attribute
> >> >> >
> >> >> > header_type="UInt64"
> >> >>
> >> >> You mean in the header of the .vtu?
> >> >
> >> >
> >> > Yeah, within the open VTKFile tag.
> >> > Like this
> >> > < VTKFile type=“xxx”, byte_order="LittleEndian" header_type="UInt64"
> >
> >> >
> >> > Do you happen to have an example or pointers to docs describing this
> >> >> feature?
> >> >
> >> >
> >> > Example yes - will send it to you tomorrow. Docs… not really. Only
> stuff
> >> > like this
> >> >
> >> >
> >>
> https://kitware.github.io/paraview-docs/latest/python/paraview.simple.XMLPStructuredGridWriter.html
> >> >
> >> >
> >> >
> >>
> https://kitware.github.io/paraview-docs/v5.8.0/python/paraview.simple.XMLMultiBlockDataWriter.html
> >> >
> >> > All the writers seem to support it.
> >> >
> >> >
> >> > Can we always do this?
> >> >
> >> >
> >> > Yep!
> >> >
> >> >
> >> > It isn't mentioned in these:
> >> >>
> >> >> https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf (PDF
> was
> >> >> created in 2003)
> >> >>
> >> >>
> >>
> https://kitware.github.io/vtk-examples/site/VTKFileFormats/#xml-file-formats
> >> >>
> >> >
> >> > Yes I know. I’ve tied myself in knots for years because the of the
> >> > assumption that the offset had to be an int.
> >> >
> >> > Credit for the discovery goes to Carsten Uphoff. He showed me this.
> >> >
> >> > Cheers,
> >> > Dave
> >> >
> >> >
> >> >
> >> >> > then the size of the offset is now UInt64 and now large files can
> be
> >> >> > finally written.
> >> >> >
> >> >> >
> >> >> > Cheers,
> >> >> > Dave
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >>
> >> >> >> Mike Michell <mi.mike1021 at gmail.com> writes:
> >> >> >>
> >> >> >> > Thanks for the note.
> >> >> >> > I understood that PETSc calculates the offsets for me through
> >> >> "boffset"
> >> >> >> > variable in plexvtu.c file. Please correct me if it is wrong.
> >> >> >> >
> >> >> >> > If plexvtu.c has a bug, it could be around "write file header"
> >> part in
> >> >> >> > which the boffset is also computed. Is this correct? I am not
> using
> >> >> >> complex
> >> >> >> > number.
> >> >> >> > There are several mixed parts among "Int32, UInt8, PetscInt_FMT,
> >> >> >> > PetscInt64_FMT" in writing the header.
> >> >> >> >
> >> >> >> > Which combination of those flags is correct for 64bit indices?
> I am
> >> >> gonna
> >> >> >> > modify plexvtu.c file with "#if
> defined(PETSC_USE_64BIT_INDICES)"
> >> >> >> > statement, but I do not know what is the correct form of the
> header
> >> >> flag
> >> >> >> > for 64bit indices.
> >> >> >> >
> >> >> >> > It is also confusing to me:
> >> >> >> > boffset += gpiece[r].ncells * sizeof(PetscInt) + sizeof(int);
> >> >> >> > How is sizeof(PetscInt) different from sizeof(int)?
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Mike
> >> >> >> >
> >> >> >> >
> >> >> >> >> On Tue, Feb 14, 2023 at 11:45 AM Mike Michell <
> >> mi.mike1021 at gmail.com
> >> >> >
> >> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >>> I was trying to modify the header flags from "Int32" to
> "Int64",
> >> but
> >> >> >> the
> >> >> >> >>> problem was not resolved. Could I get any additional comments?
> >> >> >> >>>
> >> >> >> >>
> >> >> >> >> The calculated offsets are not correct I think.
> >> >> >> >>
> >> >> >> >> Matt
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>> Thanks,
> >> >> >> >>> Mike
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>> Thanks for the comments.
> >> >> >> >>>> To be precise on the question, the entire part of the header
> of
> >> the
> >> >> >> .vtu
> >> >> >> >>>> file is attached:
> >> >> >> >>>>
> >> >> >> >>>> <?xml version="1.0"?>
> >> >> >> >>>> <VTKFile type="UnstructuredGrid" version="0.1"
> >> >> >> byte_order="LittleEndian">
> >> >> >> >>>> <UnstructuredGrid>
> >> >> >> >>>> <Piece NumberOfPoints="4872" NumberOfCells="4000">
> >> >> >> >>>> <Points>
> >> >> >> >>>> <DataArray type="Float64" Name="Position"
> >> >> >> NumberOfComponents="3"
> >> >> >> >>>> format="appended" offset="0" />
> >> >> >> >>>> </Points>
> >> >> >> >>>> <Cells>
> >> >> >> >>>> <DataArray type="Int32" Name="connectivity"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="116932" />
> >> >> >> >>>> <DataArray type="Int32" Name="offsets"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="372936" />
> >> >> >> >>>> <DataArray type="UInt8" Name="types"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="404940" />
> >> >> >> >>>> </Cells>
> >> >> >> >>>> <CellData>
> >> >> >> >>>> <DataArray type="Int32" Name="Rank"
> >> NumberOfComponents="1"
> >> >> >> >>>> format="appended" offset="408944" />
> >> >> >> >>>> </CellData>
> >> >> >> >>>> <PointData>
> >> >> >> >>>> <DataArray type="Float64"
> >> Name="Vec_0x37c89c0_4Field_0.0"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="424948" />
> >> >> >> >>>> </PointData>
> >> >> >> >>>> </Piece>
> >> >> >> >>>> <Piece NumberOfPoints="4872" NumberOfCells="4000">
> >> >> >> >>>> <Points>
> >> >> >> >>>> <DataArray type="Float64" Name="Position"
> >> >> >> NumberOfComponents="3"
> >> >> >> >>>> format="appended" offset="463928" />
> >> >> >> >>>> </Points>
> >> >> >> >>>> <Cells>
> >> >> >> >>>> <DataArray type="Int32" Name="connectivity"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="580860" />
> >> >> >> >>>> <DataArray type="Int32" Name="offsets"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="836864" />
> >> >> >> >>>> <DataArray type="UInt8" Name="types"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="868868" />
> >> >> >> >>>> </Cells>
> >> >> >> >>>> <CellData>
> >> >> >> >>>> <DataArray type="Int32" Name="Rank"
> >> NumberOfComponents="1"
> >> >> >> >>>> format="appended" offset="872872" />
> >> >> >> >>>> </CellData>
> >> >> >> >>>> <PointData>
> >> >> >> >>>> <DataArray type="Float64"
> >> Name="Vec_0x37c89c0_4Field_0.0"
> >> >> >> >>>> NumberOfComponents="1" format="appended" offset="888876" />
> >> >> >> >>>> </PointData>
> >> >> >> >>>> </Piece>
> >> >> >> >>>> </UnstructuredGrid>
> >> >> >> >>>> <AppendedData encoding="raw">
> >> >> >> >>>>
> >> >> >> >>>>
> >> >> >> >>>> Thanks,
> >> >> >> >>>> Mike
> >> >> >> >>>>
> >> >> >> >>>>
> >> >> >> >>>>> On Sun, Feb 12, 2023 at 6:15 PM Mike Michell <
> >> >> mi.mike1021 at gmail.com>
> >> >> >> >>>>> wrote:
> >> >> >> >>>>>
> >> >> >> >>>>>> Dear PETSc team,
> >> >> >> >>>>>>
> >> >> >> >>>>>> I am a user of PETSc with Fortran. My code uses DMPlex to
> >> handle
> >> >> dm
> >> >> >> >>>>>> object. To print out output variable and mesh
> connectivity, I
> >> use
> >> >> >> VecView()
> >> >> >> >>>>>> by defining PetscSection on that dm and borrow a vector.
> The
> >> type
> >> >> >> of the
> >> >> >> >>>>>> viewer is set to PETSCVIEWERVTK.
> >> >> >> >>>>>>
> >> >> >> >>>>>> With 32bit indices, the above work flow has no issue.
> >> However, if
> >> >> >> >>>>>> PETSc is configured with 64bit indices, my output .vtu file
> >> has
> >> >> an
> >> >> >> error if
> >> >> >> >>>>>> I open the file with visualization tools, such as Paraview
> or
> >> >> >> Tecplot,
> >> >> >> >>>>>> saying that:
> >> >> >> >>>>>> "Cannot read cell connectivity from Cells in piece 0
> because
> >> the
> >> >> >> >>>>>> "offsets" array is not monotonically increasing or starts
> >> with a
> >> >> >> value
> >> >> >> >>>>>> other than 0."
> >> >> >> >>>>>>
> >> >> >> >>>>>> If I open the .vtu file from terminal, I can see such a
> line:
> >> >> >> >>>>>> ...
> >> >> >> >>>>>> <DataArray type="Int32" Name="connectivity"
> >> >> NumberOfComponents="1"
> >> >> >> >>>>>> format="appended" offset="580860" />
> >> >> >> >>>>>> ...
> >> >> >> >>>>>>
> >> >> >> >>>>>> I expected "DataArray type="Int64", since the PETSc has
> 64bit
> >> >> >> indices.
> >> >> >> >>>>>> Could I get recommendations that I need to check to resolve
> >> the
> >> >> >> issue?
> >> >> >> >>>>>>
> >> >> >> >>>>>
> >> >> >> >>>>> This is probably a bug. We will look at it.
> >> >> >> >>>>>
> >> >> >> >>>>> Jed, I saw that Int32 is hardcoded in plexvtu.c, but
> >> >> sizeof(PetscInt)
> >> >> >> >>>>> is used to calculate the offset, which looks inconsistent.
> Can
> >> you
> >> >> >> take a
> >> >> >> >>>>> look?
> >> >> >> >>>>>
> >> >> >> >>>>> Thanks,
> >> >> >> >>>>>
> >> >> >> >>>>> Matt
> >> >> >> >>>>>
> >> >> >> >>>>>
> >> >> >> >>>>>> Thanks,
> >> >> >> >>>>>> Mike
> >> >> >> >>>>>>
> >> >> >> >>>>>
> >> >> >> >>>>>
> >> >> >> >>>>> --
> >> >> >> >>>>> What most experimenters take for granted before they begin
> >> their
> >> >> >> >>>>> experiments is infinitely more interesting than any results
> to
> >> >> which
> >> >> >> their
> >> >> >> >>>>> experiments lead.
> >> >> >> >>>>> -- Norbert Wiener
> >> >> >> >>>>>
> >> >> >> >>>>> https://www.cse.buffalo.edu/~knepley/
> >> >> >> >>>>> <http://www.cse.buffalo.edu/~knepley/>
> >> >> >> >>>>>
> >> >> >> >>>>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> What most experimenters take for granted before they begin
> their
> >> >> >> >> experiments is infinitely more interesting than any results to
> >> which
> >> >> >> their
> >> >> >> >> experiments lead.
> >> >> >> >> -- Norbert Wiener
> >> >> >> >>
> >> >> >> >> https://www.cse.buffalo.edu/~knepley/
> >> >> >> >> <http://www.cse.buffalo.edu/~knepley/>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230217/abcd0308/attachment-0001.html>
More information about the petsc-users
mailing list