[petsc-users] Parallel output in PETSc with pHDF5 and VTK

[Jed Brown]

"Buesing, Henrik" <hbuesing at eonerc.rwth-aachen.de> writes:

> Dear all,
>
> I would like to write HDF5 and VTK files in parallel. I found Vec example 19<http://www.mcs.anl.gov/petsc/petsc-dev/src/vec/vec/examples/tutorials/ex19.c.html>: "Parallel HDF5 Vec Viewing". But I do not understand how I tell PETSc to write a DMDA in parallel when doing VecView. At the moment everything is done on process 0. Can PETSc use parallel HDF5?

Yeah, the implementation is in VecView_MPI_HDF5_DA and uses
H5FD_MPIO_COLLECTIVE if supported.  Did you build your HDF5 with MPI?

> Regarding VTK: Is it possible that every process dumps his part of the Vec in a separate file and let Paraview combine this. I think Firedrake does it in this way.

This creates a filesystem metadata problem and is not supported by
PETSc's VTK viewers.

It would be possible to write binary-appended files by scattering the
metadata back to the processes which then open the output file and seek
to the appropriate place (instead of serializing their data through rank
0).  This would be worthwhile and not particularly hard to implement if
you want to try.  The problem is that VTK readers usually don't
parallelize over such files so you've just moved the problem.  Our usual
advice is to use HDF5 with collective IO if running at large scale.