[petsc-dev] DMplex reader / viewers

[Jed Brown]

Blaise A Bourdin <bourdin at lsu.edu> writes:
> I am not sure if Exodus has a good solution here. As far as I
> understand, exodus is inherently sequential, even when implemented
> with HDF5 instead of netcdf. I would also worry about third party
> support for exodus files using HDF5 as their storage format.  Exodus
> has an parallel extension called nemesis, but I can’t figure out how
> how their concept of ghost point and cells works. The documentation on
> this point is really unclear.

PyLith accesses Exodus files directly via the NetCDF interface rather
than through the Exodus library.  I don't recall if it has true parallel
IO that way, but it's certainly possible.

> Considering the dismal state of parallel FE formats and libraries, it
> seems to me that one needs to chose between two options:
>
>  a. scatter back to a single I/O node and use sequential I/O using the
>  ordering of the original (exodus) mesh. This allows reading and
>  writing on an arbitrary number of processors, but has potential
>  memory footprint and performance issues. How large a mesh can we
>  reasonably expect to be able to handle this way?
>
>  b. Do “poor man” parallel I/O where each CPU does its own I/O, and

This "independent" IO leads to atrocious performance on parallel file
systems.  You can manually aggregate to a smaller number of processes
and have those write, but collective IO is simpler and more sustainable.

>  possibly create interface matching files à la nemesis or SILO. Maybe,
>  we can save enough information on the parallel layout in order to
>  easily write an un-partitionner as a post-processor.
>
> Unless one can come up with a better solution than a or b, I’d like to
> start by writing a very simple ASCII viewer demonstrating the
> communication involved, 

The VTS viewer is mode (a), but could simply be extended to support
writing all the heavy data using MPI-IO collectives.

> then modify it to use exodus, SILO or HDF5 format.

Is SILO expressive enough for high-order, DG, H(div), etc?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140121/caca3716/attachment.sig>