<div dir="ltr"><div dir="ltr">Jed,<br><br>It does not work for me even with the reproducer case with the small 2D square mesh. Can you run the case that I sent and open the created "sol.vtu" file with paraview? <br><br>Thanks,<br></div><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>Thanks, Dave.<br>
<br>
Mike, can you test that this branch works with your large problems? I tested that .vtu works in parallel for small problems, where works = loads correctly in Paraview and VisIt.<br>
<br>
<a href="https://gitlab.com/petsc/petsc/-/merge_requests/6081" rel="noreferrer" target="_blank">https://gitlab.com/petsc/petsc/-/merge_requests/6081</a><br>
<br>
Dave May <<a href="mailto:dave.mayhem23@gmail.com" target="_blank">dave.mayhem23@gmail.com</a>> writes:<br>
<br>
> On Tue 14. Feb 2023 at 21:27, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br>
><br>
>> Dave May <<a href="mailto:dave.mayhem23@gmail.com" target="_blank">dave.mayhem23@gmail.com</a>> writes:<br>
>><br>
>> > On Tue 14. Feb 2023 at 17:17, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br>
>> ><br>
>> >> Can you share a reproducer? I think I recall the format requiring<br>
>> certain<br>
>> >> things to be Int32.<br>
>> ><br>
>> ><br>
>> > By default, the byte offset used with the appended data format is<br>
>> UInt32. I<br>
>> > believe that’s where the sizeof(int) is coming from. This default is<br>
>> > annoying as it limits the total size of your appended data to be < 3 GB.<br>
>> > That said, in the opening of the paraview file you can add this attribute<br>
>> ><br>
>> > header_type="UInt64"<br>
>><br>
>> You mean in the header of the .vtu?<br>
><br>
><br>
> Yeah, within the open VTKFile tag.<br>
> Like this<br>
> < VTKFile type=“xxx”, byte_order="LittleEndian" header_type="UInt64" ><br>
><br>
> Do you happen to have an example or pointers to docs describing this<br>
>> feature?<br>
><br>
><br>
> Example yes - will send it to you tomorrow. Docs… not really. Only stuff<br>
> like this<br>
><br>
> <a href="https://kitware.github.io/paraview-docs/latest/python/paraview.simple.XMLPStructuredGridWriter.html" rel="noreferrer" target="_blank">https://kitware.github.io/paraview-docs/latest/python/paraview.simple.XMLPStructuredGridWriter.html</a><br>
><br>
><br>
> <a href="https://kitware.github.io/paraview-docs/v5.8.0/python/paraview.simple.XMLMultiBlockDataWriter.html" rel="noreferrer" target="_blank">https://kitware.github.io/paraview-docs/v5.8.0/python/paraview.simple.XMLMultiBlockDataWriter.html</a><br>
><br>
> All the writers seem to support it.<br>
><br>
><br>
> Can we always do this?<br>
><br>
><br>
> Yep!<br>
><br>
><br>
> It isn't mentioned in these:<br>
>><br>
>> <a href="https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf" rel="noreferrer" target="_blank">https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf</a> (PDF was<br>
>> created in 2003)<br>
>><br>
>> <a href="https://kitware.github.io/vtk-examples/site/VTKFileFormats/#xml-file-formats" rel="noreferrer" target="_blank">https://kitware.github.io/vtk-examples/site/VTKFileFormats/#xml-file-formats</a><br>
>><br>
><br>
> Yes I know. I’ve tied myself in knots for years because the of the<br>
> assumption that the offset had to be an int.<br>
><br>
> Credit for the discovery goes to Carsten Uphoff. He showed me this.<br>
><br>
> Cheers,<br>
> Dave<br>
><br>
><br>
><br>
>> > then the size of the offset is now UInt64 and now large files can be<br>
>> > finally written.<br>
>> ><br>
>> ><br>
>> > Cheers,<br>
>> > Dave<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> >><br>
>> >> Mike Michell <<a href="mailto:mi.mike1021@gmail.com" target="_blank">mi.mike1021@gmail.com</a>> writes:<br>
>> >><br>
>> >> > Thanks for the note.<br>
>> >> > I understood that PETSc calculates the offsets for me through<br>
>> "boffset"<br>
>> >> > variable in plexvtu.c file. Please correct me if it is wrong.<br>
>> >> ><br>
>> >> > If plexvtu.c has a bug, it could be around "write file header" part in<br>
>> >> > which the boffset is also computed. Is this correct? I am not using<br>
>> >> complex<br>
>> >> > number.<br>
>> >> > There are several mixed parts among "Int32, UInt8, PetscInt_FMT,<br>
>> >> > PetscInt64_FMT" in writing the header.<br>
>> >> ><br>
>> >> > Which combination of those flags is correct for 64bit indices? I am<br>
>> gonna<br>
>> >> > modify plexvtu.c file with "#if defined(PETSC_USE_64BIT_INDICES)"<br>
>> >> > statement, but I do not know what is the correct form of the header<br>
>> flag<br>
>> >> > for 64bit indices.<br>
>> >> ><br>
>> >> > It is also confusing to me:<br>
>> >> > boffset += gpiece[r].ncells * sizeof(PetscInt) + sizeof(int);<br>
>> >> > How is sizeof(PetscInt) different from sizeof(int)?<br>
>> >> ><br>
>> >> > Thanks,<br>
>> >> > Mike<br>
>> >> ><br>
>> >> ><br>
>> >> >> On Tue, Feb 14, 2023 at 11:45 AM Mike Michell <<a href="mailto:mi.mike1021@gmail.com" target="_blank">mi.mike1021@gmail.com</a><br>
>> ><br>
>> >> >> wrote:<br>
>> >> >><br>
>> >> >>> I was trying to modify the header flags from "Int32" to "Int64", but<br>
>> >> the<br>
>> >> >>> problem was not resolved. Could I get any additional comments?<br>
>> >> >>><br>
>> >> >><br>
>> >> >> The calculated offsets are not correct I think.<br>
>> >> >><br>
>> >> >> Matt<br>
>> >> >><br>
>> >> >><br>
>> >> >>> Thanks,<br>
>> >> >>> Mike<br>
>> >> >>><br>
>> >> >>><br>
>> >> >>>> Thanks for the comments.<br>
>> >> >>>> To be precise on the question, the entire part of the header of the<br>
>> >> .vtu<br>
>> >> >>>> file is attached:<br>
>> >> >>>><br>
>> >> >>>> <?xml version="1.0"?><br>
>> >> >>>> <VTKFile type="UnstructuredGrid" version="0.1"<br>
>> >> byte_order="LittleEndian"><br>
>> >> >>>> <UnstructuredGrid><br>
>> >> >>>> <Piece NumberOfPoints="4872" NumberOfCells="4000"><br>
>> >> >>>> <Points><br>
>> >> >>>> <DataArray type="Float64" Name="Position"<br>
>> >> NumberOfComponents="3"<br>
>> >> >>>> format="appended" offset="0" /><br>
>> >> >>>> </Points><br>
>> >> >>>> <Cells><br>
>> >> >>>> <DataArray type="Int32" Name="connectivity"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="116932" /><br>
>> >> >>>> <DataArray type="Int32" Name="offsets"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="372936" /><br>
>> >> >>>> <DataArray type="UInt8" Name="types"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="404940" /><br>
>> >> >>>> </Cells><br>
>> >> >>>> <CellData><br>
>> >> >>>> <DataArray type="Int32" Name="Rank" NumberOfComponents="1"<br>
>> >> >>>> format="appended" offset="408944" /><br>
>> >> >>>> </CellData><br>
>> >> >>>> <PointData><br>
>> >> >>>> <DataArray type="Float64" Name="Vec_0x37c89c0_4Field_0.0"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="424948" /><br>
>> >> >>>> </PointData><br>
>> >> >>>> </Piece><br>
>> >> >>>> <Piece NumberOfPoints="4872" NumberOfCells="4000"><br>
>> >> >>>> <Points><br>
>> >> >>>> <DataArray type="Float64" Name="Position"<br>
>> >> NumberOfComponents="3"<br>
>> >> >>>> format="appended" offset="463928" /><br>
>> >> >>>> </Points><br>
>> >> >>>> <Cells><br>
>> >> >>>> <DataArray type="Int32" Name="connectivity"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="580860" /><br>
>> >> >>>> <DataArray type="Int32" Name="offsets"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="836864" /><br>
>> >> >>>> <DataArray type="UInt8" Name="types"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="868868" /><br>
>> >> >>>> </Cells><br>
>> >> >>>> <CellData><br>
>> >> >>>> <DataArray type="Int32" Name="Rank" NumberOfComponents="1"<br>
>> >> >>>> format="appended" offset="872872" /><br>
>> >> >>>> </CellData><br>
>> >> >>>> <PointData><br>
>> >> >>>> <DataArray type="Float64" Name="Vec_0x37c89c0_4Field_0.0"<br>
>> >> >>>> NumberOfComponents="1" format="appended" offset="888876" /><br>
>> >> >>>> </PointData><br>
>> >> >>>> </Piece><br>
>> >> >>>> </UnstructuredGrid><br>
>> >> >>>> <AppendedData encoding="raw"><br>
>> >> >>>><br>
>> >> >>>><br>
>> >> >>>> Thanks,<br>
>> >> >>>> Mike<br>
>> >> >>>><br>
>> >> >>>><br>
>> >> >>>>> On Sun, Feb 12, 2023 at 6:15 PM Mike Michell <<br>
>> <a href="mailto:mi.mike1021@gmail.com" target="_blank">mi.mike1021@gmail.com</a>><br>
>> >> >>>>> wrote:<br>
>> >> >>>>><br>
>> >> >>>>>> Dear PETSc team,<br>
>> >> >>>>>><br>
>> >> >>>>>> I am a user of PETSc with Fortran. My code uses DMPlex to handle<br>
>> dm<br>
>> >> >>>>>> object. To print out output variable and mesh connectivity, I use<br>
>> >> VecView()<br>
>> >> >>>>>> by defining PetscSection on that dm and borrow a vector. The type<br>
>> >> of the<br>
>> >> >>>>>> viewer is set to PETSCVIEWERVTK.<br>
>> >> >>>>>><br>
>> >> >>>>>> With 32bit indices, the above work flow has no issue. However, if<br>
>> >> >>>>>> PETSc is configured with 64bit indices, my output .vtu file has<br>
>> an<br>
>> >> error if<br>
>> >> >>>>>> I open the file with visualization tools, such as Paraview or<br>
>> >> Tecplot,<br>
>> >> >>>>>> saying that:<br>
>> >> >>>>>> "Cannot read cell connectivity from Cells in piece 0 because the<br>
>> >> >>>>>> "offsets" array is not monotonically increasing or starts with a<br>
>> >> value<br>
>> >> >>>>>> other than 0."<br>
>> >> >>>>>><br>
>> >> >>>>>> If I open the .vtu file from terminal, I can see such a line:<br>
>> >> >>>>>> ...<br>
>> >> >>>>>> <DataArray type="Int32" Name="connectivity"<br>
>> NumberOfComponents="1"<br>
>> >> >>>>>> format="appended" offset="580860" /><br>
>> >> >>>>>> ...<br>
>> >> >>>>>><br>
>> >> >>>>>> I expected "DataArray type="Int64", since the PETSc has 64bit<br>
>> >> indices.<br>
>> >> >>>>>> Could I get recommendations that I need to check to resolve the<br>
>> >> issue?<br>
>> >> >>>>>><br>
>> >> >>>>><br>
>> >> >>>>> This is probably a bug. We will look at it.<br>
>> >> >>>>><br>
>> >> >>>>> Jed, I saw that Int32 is hardcoded in plexvtu.c, but<br>
>> sizeof(PetscInt)<br>
>> >> >>>>> is used to calculate the offset, which looks inconsistent. Can you<br>
>> >> take a<br>
>> >> >>>>> look?<br>
>> >> >>>>><br>
>> >> >>>>> Thanks,<br>
>> >> >>>>><br>
>> >> >>>>> Matt<br>
>> >> >>>>><br>
>> >> >>>>><br>
>> >> >>>>>> Thanks,<br>
>> >> >>>>>> Mike<br>
>> >> >>>>>><br>
>> >> >>>>><br>
>> >> >>>>><br>
>> >> >>>>> --<br>
>> >> >>>>> What most experimenters take for granted before they begin their<br>
>> >> >>>>> experiments is infinitely more interesting than any results to<br>
>> which<br>
>> >> their<br>
>> >> >>>>> experiments lead.<br>
>> >> >>>>> -- Norbert Wiener<br>
>> >> >>>>><br>
>> >> >>>>> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
>> >> >>>>> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
>> >> >>>>><br>
>> >> >>>><br>
>> >> >><br>
>> >> >> --<br>
>> >> >> What most experimenters take for granted before they begin their<br>
>> >> >> experiments is infinitely more interesting than any results to which<br>
>> >> their<br>
>> >> >> experiments lead.<br>
>> >> >> -- Norbert Wiener<br>
>> >> >><br>
>> >> >> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
>> >> >> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
>> >> >><br>
>> >><br>
>><br>
</blockquote></div></div>