[petsc-users] SNES ex12 visualization

[Kong, Fande]

On Thu, Sep 14, 2017 at 10:35 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
> > On Sep 14, 2017, at 11:10 AM, Kong, Fande <fande.kong at inl.gov> wrote:
> >
> >
> >
> > On Thu, Sep 14, 2017 at 9:47 AM, Matthew Knepley <knepley at gmail.com>
> wrote:
> > On Thu, Sep 14, 2017 at 11:43 AM, Adriano Côrtes <adrimacortes at gmail.com>
> wrote:
> > Dear Matthew,
> >
> > Thank you for your return. It worked, but this prompts another question.
> So why PetscViewer does not write both files (.h5 and .xmf) directly,
> instead of having to post-proc the .h5 file (in serial)?
> >
> > 1) Maintenance: Changing the Python is much easier than changing the C
> you would add to generate it
> >
> > 2) Performance: On big parallel system, writing files is expensive so I
> wanted to minimize what I had to do.
> >
> > 3) Robustness: Moving 1 file around is much easier than remembering 2. I
> just always regenerate the xdmf when needed.
> >
> > And what about big 3D simulations? PETSc always serialize the output of
> the distributed dmplex? Is there a way to output one .h5 per mesh partition?
> >
> > Given the way I/O is structured on big machines, we believe the multiple
> file route is a huge mistake. Also, all our measurements
> > say that sending some data on the network is not noticeable given the
> disk access costs.
> >
> > I have slightly different things here. We tried the serial output, it
> looks really slow for large-scale problems, and the first processor often
> runs out of memory because of gathering all data from other processor cores.
>
>   Where in PETSc is this?  What type of viewer? Is there an example that
> reproduces the problem? Even when we do not use MPI IO in PETSc we attempt
> to not "put the entire object on the first process" so memory should not be
> an issue. For example VecVew() should memory scale both with or without MPI
> IO
>

We manually gather all data to the first processor core, and write it as a
single vtk file.


>
>
> > The parallel IO runs smoothly and much faster than I excepted. We have
> done experiments with ten thousands  of cores for a problem with 1 billion
> of unknowns.
>
>     Is this your own canned IO or something in PETSc?
>

We implement the writer based on the ISView and VecView with HDF5 viewer
 in PETSc to output all data as a single HDF. ISView and VecView do the
magic job for me.



>
> > I did not see any concern so far.
>
>    Ten thousand files is possibly manageable but I question 2 million.
>

Just one single HDF5 file.

Fande,


>
> >
> >
> > Fande,
> >
> >
> >   Thanks,
> >
> >     Matt
> >
> > Best regards,
> > Adriano.
> >
> >
> > 2017-09-14 12:00 GMT-03:00 Matthew Knepley <knepley at gmail.com>:
> > On Thu, Sep 14, 2017 at 10:30 AM, Adriano Côrtes <adrimacortes at gmail.com>
> wrote:
> > Dear all,
> >
> > I am running the SNES ex12  and I'm passing the options -dm_view
> hdf5:sol.h5 -vec_view hdf5:sol.h5::append to generate an output file. The
> .h5 file is generated, but I'm not being able to load it in Paraview
> (5.4.0-64bits). Paraview recognizes the file and offers severel options to
> read it, here is the complete list
> >
> > Chombo Files
> > GTC Files
> > M3DC1 Files
> > Multilevel 3D Plasma Files
> > PFLOTRAN Files
> > Pixie Files
> > Tetrad Files
> > UNIC Files
> > VizSchema Files
> >
> > The problem is none of the options above work :(
> > I'm using the configure option '-download-hdf5' and it installs hdf5
> version 1.8.18
> > Any hint of how to fix it and have the visualization working?
> >
> > Yes, Paraview does not directly read HDF5. It needs you to tell it what
> the data in the HDF5 file means. You do
> > this by creating a *.xdmf file, which is XML. We provide a tool
> >
> >   $PETSC_DIR/bin/petsc_gen_xdmf.py <HDF5 file>
> >
> > which should automatically produce this file for you. Let us know if it
> does not work.
> >
> >   Thanks,
> >
> >     Matt
> >
> >
> > Best regards,
> > Adriano.
> >
> > --
> > Adriano Côrtes
> > =================================================
> > Campus Duque de Caxias and
> > High-performance Computing Center (NACAD/COPPE)
> > Federal University of Rio de Janeiro (UFRJ)
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> > -- Norbert Wiener
> >
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.
> caam.rice.edu_-7Emk51_&d=DwIFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_
> _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m=
> YTLjMkjfS0tYLZ3RxmJFoe8BT56h48axFCzaadZwBXA&s=
> iLsaHQugaY4gj4DKE9gq8XdBt7q3ejdpDRfJ8RFerE0&e=
> >
> >
> >
> > --
> > Adriano Côrtes
> > =================================================
> > Campus Duque de Caxias and
> > High-performance Computing Center (NACAD/COPPE)
> > Federal University of Rio de Janeiro (UFRJ)
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> > -- Norbert Wiener
> >
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.
> caam.rice.edu_-7Emk51_&d=DwIFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_
> _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m=
> YTLjMkjfS0tYLZ3RxmJFoe8BT56h48axFCzaadZwBXA&s=
> iLsaHQugaY4gj4DKE9gq8XdBt7q3ejdpDRfJ8RFerE0&e=
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170914/d9fff70e/attachment.html>