[petsc-users] Scaling of the Petsc Binary Viewer

[Dave May]

On Wed 7. Jul 2021 at 20:41, Thibault Bridel-Bertomeu <
thibault.bridelbertomeu at gmail.com> wrote:

> Dear all,
>
> I have been having issues with large Vec (based on DMPLex) and massive MPI
> I/O  ... it looks like the data that is written by the Petsc Binary Viewer
> is gibberish for large meshes split on a high number of processes. For
> instance, I am using a mesh that has around 50 million cells, split on 1024
> processors.
> The computation seems to run fine, the timestep computed from the data
> makes sense so I think internally everything is fine. But when I look at
> the solution (one example attached) it's noise - at this point it should
> show a bow shock developing on the left near the step.
> The piece of code I use is below for the output :
>
>                 call DMGetOutputSequenceNumber(dm, save_seqnum,
> save_seqval, ierr); CHKERRA(ierr)
>                 call DMSetOutputSequenceNumber(dm, -1, 0.d0, ierr);
> CHKERRA(ierr)
>                 write(filename,'(A,I8.8,A)') "restart_", stepnum, ".bin"
>                 call PetscViewerCreate(PETSC_COMM_WORLD, binViewer, ierr);
> CHKERRA(ierr)
>                 call PetscViewerSetType(binViewer, PETSCVIEWERBINARY,
> ierr); CHKERRA(ierr)
>                 call PetscViewerFileSetMode(binViewer, FILE_MODE_WRITE,
> ierr); CHKERRA(ierr);
>                 call PetscViewerBinarySetUseMPIIO(binViewer, PETSC_TRUE,
> ierr); CHKERRA(ierr);
>
>

Do you get the correct output if you don’t call the function above (or
equivalently use PETSC_FALSE)


call PetscViewerFileSetName(binViewer, trim(filename), ierr); CHKERRA(ierr)
>                 call VecView(X, binViewer, ierr); CHKERRA(ierr)
>                 call PetscViewerDestroy(binViewer, ierr); CHKERRA(ierr)
>                 call DMSetOutputSequenceNumber(dm, save_seqnum,
> save_seqval, ierr); CHKERRA(ierr)
>
> I do not think there is anything wrong with it but of course I would be
> happy to hear your feedback.
> Nonetheless my question was : how far have you tested the binary mpi i/o
> of a Vec ? Does it make some sense that for a 50 million cell mesh split on
> 1024 processes, it could somehow fail ?
> Or is it my python drawing method that is completely incapable of handling
> this dataset ? (paraview displays the same thing though so I'm not sure ...)
>

Are you using the python provided tools within petsc to load the Vec from
file?


Thanks,
Dave



> Thank you very much for your advice and help !!!
>
> Thibault
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210707/0ccc250a/attachment.html>