[petsc-users] SNES ex12 visualization

Thu Sep 14 12:26:47 CDT 2017

On Thu, Sep 14, 2017 at 1:07 PM, Kong, Fande <fande.kong at inl.gov> wrote:

>
>
> On Thu, Sep 14, 2017 at 10:35 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>>
>> > On Sep 14, 2017, at 11:10 AM, Kong, Fande <fande.kong at inl.gov> wrote:
>> >
>> >
>> >
>> > On Thu, Sep 14, 2017 at 9:47 AM, Matthew Knepley <knepley at gmail.com>
>> wrote:
>> > On Thu, Sep 14, 2017 at 11:43 AM, Adriano Côrtes <
>> adrimacortes at gmail.com> wrote:
>> > Dear Matthew,
>> >
>> > Thank you for your return. It worked, but this prompts another
>> question. So why PetscViewer does not write both files (.h5 and .xmf)
>> directly, instead of having to post-proc the .h5 file (in serial)?
>> >
>> > 1) Maintenance: Changing the Python is much easier than changing the C
>> you would add to generate it
>> >
>> > 2) Performance: On big parallel system, writing files is expensive so I
>> wanted to minimize what I had to do.
>> >
>> > 3) Robustness: Moving 1 file around is much easier than remembering 2.
>> I just always regenerate the xdmf when needed.
>> >
>> > And what about big 3D simulations? PETSc always serialize the output of
>> the distributed dmplex? Is there a way to output one .h5 per mesh partition?
>> >
>> > Given the way I/O is structured on big machines, we believe the
>> multiple file route is a huge mistake. Also, all our measurements
>> > say that sending some data on the network is not noticeable given the
>> disk access costs.
>> >
>> > I have slightly different things here. We tried the serial output, it
>> looks really slow for large-scale problems, and the first processor often
>> runs out of memory because of gathering all data from other processor cores.
>>
>>   Where in PETSc is this?  What type of viewer? Is there an example that
>> reproduces the problem? Even when we do not use MPI IO in PETSc we attempt
>> to not "put the entire object on the first process" so memory should not be
>> an issue. For example VecVew() should memory scale both with or without MPI
>> IO
>>
>
> We manually gather all data to the first processor core, and write it as a
> single vtk file.
>

Of course I am not doing that. I reduce everything to an ISView or a
VecView call. That way it uses MPI I/O if its turned on.

   Matt

>
>>
>> > The parallel IO runs smoothly and much faster than I excepted. We have
>> done experiments with ten thousands  of cores for a problem with 1 billion
>> of unknowns.
>>
>>     Is this your own canned IO or something in PETSc?
>>
>
> We implement the writer based on the ISView and VecView with HDF5 viewer
>  in PETSc to output all data as a single HDF. ISView and VecView do the
> magic job for me.
>
>
>
>>
>> > I did not see any concern so far.
>>
>>    Ten thousand files is possibly manageable but I question 2 million.
>>
>
> Just one single HDF5 file.
>
> Fande,
>
>
>>
>> >
>> >
>> > Fande,
>> >
>> >
>> >   Thanks,
>> >
>> >     Matt
>> >
>> > Best regards,
>> > Adriano.
>> >
>> >
>> > 2017-09-14 12:00 GMT-03:00 Matthew Knepley <knepley at gmail.com>:
>> > On Thu, Sep 14, 2017 at 10:30 AM, Adriano Côrtes <
>> adrimacortes at gmail.com> wrote:
>> > Dear all,
>> >
>> > I am running the SNES ex12  and I'm passing the options -dm_view
>> hdf5:sol.h5 -vec_view hdf5:sol.h5::append to generate an output file. The
>> .h5 file is generated, but I'm not being able to load it in Paraview
>> (5.4.0-64bits). Paraview recognizes the file and offers severel options to
>> read it, here is the complete list
>> >
>> > Chombo Files
>> > GTC Files
>> > M3DC1 Files
>> > Multilevel 3D Plasma Files
>> > PFLOTRAN Files
>> > Pixie Files
>> > Tetrad Files
>> > UNIC Files
>> > VizSchema Files
>> >
>> > The problem is none of the options above work :(
>> > I'm using the configure option '-download-hdf5' and it installs hdf5
>> version 1.8.18
>> > Any hint of how to fix it and have the visualization working?
>> >
>> > Yes, Paraview does not directly read HDF5. It needs you to tell it what
>> the data in the HDF5 file means. You do
>> > this by creating a *.xdmf file, which is XML. We provide a tool
>> >
>> >   $PETSC_DIR/bin/petsc_gen_xdmf.py <HDF5 file>
>> >
>> > which should automatically produce this file for you. Let us know if it
>> does not work.
>> >
>> >   Thanks,
>> >
>> >     Matt
>> >
>> >
>> > Best regards,
>> > Adriano.
>> >
>> > --
>> > Adriano Côrtes
>> > =================================================
>> > Campus Duque de Caxias and
>> > High-performance Computing Center (NACAD/COPPE)
>> > Federal University of Rio de Janeiro (UFRJ)
>> >
>> >
>> >
>> > --
>> > What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> > -- Norbert Wiener
>> >
>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.caam
>> .rice.edu_-7Emk51_&d=DwIFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_
>> _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&
>> m=YTLjMkjfS0tYLZ3RxmJFoe8BT56h48axFCzaadZwBXA&s=iLsaHQugaY4g
>> j4DKE9gq8XdBt7q3ejdpDRfJ8RFerE0&e=
>> >
>> >
>> >
>> > --
>> > Adriano Côrtes
>> > =================================================
>> > Campus Duque de Caxias and
>> > High-performance Computing Center (NACAD/COPPE)
>> > Federal University of Rio de Janeiro (UFRJ)
>> >
>> >
>> >
>> > --
>> > What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> > -- Norbert Wiener
>> >
>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.caam
>> .rice.edu_-7Emk51_&d=DwIFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_
>> _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&
>> m=YTLjMkjfS0tYLZ3RxmJFoe8BT56h48axFCzaadZwBXA&s=iLsaHQugaY4g
>> j4DKE9gq8XdBt7q3ejdpDRfJ8RFerE0&e=
>> >
>>
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

http://www.caam.rice.edu/~mk51/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170914/dda7c488/attachment-0001.html>