[MOAB-dev] file formats
Nico Schlömer
nico.schloemer at gmail.com
Fri Apr 8 08:43:45 CDT 2016
Iulian,
I attached a typical XDMF file and a link to the accompanying HDF5 file
[1]. You'll see that the XML ("light data") doesn't contain connectivity.
The "Topology" is written into the HDF5 file.
Cheers,
Nico
[1] http://chunk.io/f/58a66720059f4ef0aaae6134afb45387
On Fri, Apr 8, 2016 at 3:35 PM Grindeanu, Iulian R. <iulian at mcs.anl.gov>
wrote:
> Hi Jacob,
> This is very interesting, do you have a small example with xmdf and h5m
> that you can share?
> I don't know about paraview, but visit uses an hdf5 reader that is not
> compiled with parallel option.
> So that means that the xmdf -> hdf5 datasets reads are serial, in my
> understanding (at least when used from visit)
>
> I see some notes in the page you are pointing that xml has "light" data,
> while hdf5 files would have "heavy" data.
> For h5m / moab, connectivity would qualify as heavy or light ?
>
> Thanks,
> Iulian
>
> ________________________________________
> From: moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on
> behalf of Jacob Hochhalter [Jacob.D.Hochhalter at nasa.gov]
> Sent: Friday, April 08, 2016 7:43 AM
> To: moab-dev at mcs.anl.gov
> Subject: Re: [MOAB-dev] file formats
>
> All,
>
> We have had this issue come up too. Our solution, which we have been
> very happy with so far, is to use the .h5m file as part of the XDMF
> concept (see http://www.xdmf.org/index.php/Main_Page). With that
> approach, we simply 'wrap' the .h5m file with a xml file, which
> describes which data we want to visualize (and how). There is no
> duplication of the mesh or results data, since the xml file simply
> 'points' to datasets in the h5m file. The xml file very generation is
> very quick with a simple python script (we have written to general
> python classes for this, if you're interested). An added bonus is that
> both VisIt and Paraview support XDMF inherently.
>
> - Jake
>
> On 04/08/2016 03:31 AM, Nico Schlömer wrote:
> > Thanks for the comment!
> >
> > To be able to do I/O from a visualizable format directly (without
> > mbconvert or any other tool) is in fact more important to me than not
> > wasting space on storing a mesh more than once (although also
> > interesting). Perhaps these two things can be aligned.
> >
> > Cheers,
> > Nico
> >
> > On Thu, Apr 7, 2016 at 11:06 PM Vijay S. Mahadevan <vijay.m at gmail.com
> > <mailto:vijay.m at gmail.com>> wrote:
> >
> > Nico, if the time series data are stored as separate tags, like
> > [SOLUTION_STEP0, SOLUTION_STEP1, .., SOLUTION_STEPN], any of the
> > conversion tools should provide the right data output that can be
> > visualized easily.
> >
> > You should be able to use mbconvert that gets installed with MOAB for
> > the conversion. Of course, if you want exodus output, you will need
> > the NetCDF dependency enabled. VTK is available out of the box
> though.
> >
> > We can also discuss more about the time series data and not writing
> > the mesh more than once, especially for static non-AMR simulations.
> We
> > should be able to find some time in the summer or a little later to
> > implement this functionality within MOAB.
> >
> > Vijay
> >
> > On Wed, Apr 6, 2016 at 2:46 PM, Nico Schlömer
> > <nico.schloemer at gmail.com <mailto:nico.schloemer at gmail.com>> wrote:
> > > Thanks for the comments!
> > >
> > >> the visualization is the tricky bit tere since none of the main
> > tools
> > >> (VisIt, Paraview) understand the
> > >> details of the layout explicitly
> > >> [...]
> > >> if we understand your needs better.
> > >
> > > The question I have to answer the most is how to visualize H5M
> > files. I
> > > usually tell people to grab meshio [1] and convert to a format
> > they use. I
> > > guess I could extend meshio such that it can translate h5m's tag
> > time series
> > > data into another format with native series. Anyhow, more work
> > for me and
> > > for the users (to convert), so I'm not so happy with that yet.
> > >
> > > Cheers,
> > > Nico
> > >
> > > [1] https://pypi.python.org/pypi/meshio
> > >
> > > On Wed, Apr 6, 2016 at 4:49 PM Vijay S. Mahadevan
> > <vijay.m at gmail.com <mailto:vijay.m at gmail.com>> wrote:
> > >>
> > >> Hi NIco,
> > >>
> > >> Our current HDF5-based I/O workflow supports adding arbitrary
> > tags if
> > >> you want to store time-stepping data. This is what Lukasz is
> > doing for
> > >> his needs and should extend arbitrarily to other applications
> using
> > >> MOAB. As Lukasz also mentioned, the visualization is the tricky
> bit
> > >> here since none of the main tools (VisIt, Paraview) understand the
> > >> details of the layout explicitly and so only expose part of the
> > data
> > >> available in h5m files. I've also been think about enhancements to
> > >> support multiple HDF5 files, where the mesh is written out only
> > >> initially and when it changes. But this is a little more
> > involved for
> > >> the book-keeping to remain general.
> > >>
> > >> On a related note, we are working with LLNL to add a native MOAB
> > >> plugin to VisIt, that directly uses MOAB underneath, in
> > parallel, to
> > >> query and visualize solution. Once that is in place, adding
> > capability
> > >> to show time-series data can be added out of the box. Note that
> > MOAB
> > >> already supports only loading partial sets and tags, if you are
> > >> concerned about the data movement with I/O for large simulation
> > runs.
> > >>
> > >> However, we have also been considering alternatives that support
> > >> parallel I/O. Yes, from our discussions, I believe we converged on
> > >> XDMF and Exodus/Nemesis. Currently, this is a bottleneck on the
> > >> resources available to us and so has been in the back burner.
> > There is
> > >> native support for time-series data with the pure NetCDF route
> > >> already, written more focused on climate applications [1, 2].
> > We can
> > >> also talk about extending this to more general use cases, if we
> > >> understand your needs better.
> > >>
> > >> Thoughts ?
> > >>
> > >> Vijay
> > >>
> > >> [1] https://www.ncl.ucar.edu/ParVis.shtml
> > >> [2]
> > https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/template.pdf
> > >>
> > >> On Wed, Apr 6, 2016 at 6:34 AM, Lukasz Kaczmarczyk
> > >> <Lukasz.Kaczmarczyk at glasgow.ac.uk
> > <mailto:Lukasz.Kaczmarczyk at glasgow.ac.uk>> wrote:
> > >> > Hello,
> > >> >
> > >> > I am not convinced that this is the best approach, however I
> > can tell
> > >> > how I
> > >> > do it.
> > >> >
> > >> > I create meshset with number of contained meshsets equal to
> > number of
> > >> > time
> > >> > steps. Each of contained meshsets has tag, which data on all
> > entities on
> > >> > meshset which I store for time step. That can be saved in
> > native HD5
> > >> > file
> > >> > and read back if needed without data redundancies and allow
> > to add time
> > >> > steps. However is a issue of post-processing, that will need
> > some proper
> > >> > reader for post-procssor which I not currently have.
> > >> >
> > >> > There are additional complexities, when you store multiple
> > fields and
> > >> > you
> > >> > store data for variable number of entities for each step. In
> > may case I
> > >> > implemented it like tape recorder, that I have different
> > tapes for which
> > >> > I
> > >> > record and play tapes depending on need. If this will be any
> > use for you
> > >> > pleas look at example,
> > >> >
> > >> >
> >
> http://mofem.eng.gla.ac.uk/mofem/html/record__series__atom_8cpp_source.html
> > >> > In example I have two tapes there and two moab instances, one
> > use for
> > >> > recording another for reading for testing proposes. This
> > works ok, but
> > >> > is
> > >> > missing proper reader for paraview and at the end I produce
> > series of
> > >> > VTKs
> > >> > for paraview. This is very specific to my development by you
> > can use it
> > >> > as
> > >> > idea if you like it.
> > >> >
> > >> > I would be great to have some common interface to record and
> > read time
> > >> > series of tags for moab. That if one will write reader for
> > paraview it
> > >> > could
> > >> > be used by several developers woking on different codes.
> > >> >
> > >> >
> > >> > Regards,
> > >> > Lukasz
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On 6 Apr 2016, at 10:02, Nico Schlömer
> > <nico.schloemer at gmail.com <mailto:nico.schloemer at gmail.com>> wrote:
> > >> >
> > >> > Hi everyone,
> > >> >
> > >> > I'll start tackling a time-series problem rather soon and I'm
> > wondering
> > >> > about file formats here. Ideally, I'd have one time series
> > per file
> > >> > (particularly since the mesh isn't changing over time,
> > perhaps that
> > >> > could be
> > >> > made use of), but I'm not sure if MOAB's custom HDF5 supports
> > that yet.
> > >> >
> > >> > Thinking of which, I faintly remember there had been talk about
> > >> > switching to
> > >> > a better-supported format in the future (xdmf and exodus came
> > to mind).
> > >> > Is
> > >> > this still a thing?
> > >> >
> > >> > Cheers,
> > >> > Nico
> > >> >
> > >> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20160408/eb5e0580/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out.xdmf
Type: application/octet-stream
Size: 1690 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20160408/eb5e0580/attachment.obj>
More information about the moab-dev
mailing list