[MOAB-dev] file formats

Nico Schlömer nico.schloemer at gmail.com
Fri Apr 8 07:49:05 CDT 2016


> The xml file very generation is very quick with a simple python script
> (we have written to general python classes for this, if you're
interested).

Very much so; where do I find the code? Perhaps this could even be included
in MOAB itself.

Cheers,
Nico

On Fri, Apr 8, 2016 at 2:43 PM Jacob Hochhalter <Jacob.D.Hochhalter at nasa.gov>
wrote:

> All,
>
> We have had this issue come up too.  Our solution, which we have been
> very happy with so far, is to use the .h5m file as part of the XDMF
> concept (see http://www.xdmf.org/index.php/Main_Page).  With that
> approach, we simply 'wrap' the .h5m file with a xml file, which
> describes which data we want to visualize (and how).  There is no
> duplication of the mesh or results data, since the xml file simply
> 'points' to datasets in the h5m file.  The xml file very generation is
> very quick with a simple python script (we have written to general
> python classes for this, if you're interested).  An added bonus is that
> both VisIt and Paraview support XDMF inherently.
>
> - Jake
>
> On 04/08/2016 03:31 AM, Nico Schlömer wrote:
> > Thanks for the comment!
> >
> > To be able to do I/O from a visualizable format directly (without
> > mbconvert or any other tool) is in fact more important to me than not
> > wasting space on storing a mesh more than once (although also
> > interesting). Perhaps these two things can be aligned.
> >
> > Cheers,
> > Nico
> >
> > On Thu, Apr 7, 2016 at 11:06 PM Vijay S. Mahadevan <vijay.m at gmail.com
> > <mailto:vijay.m at gmail.com>> wrote:
> >
> >     Nico, if the time series data are stored as separate tags, like
> >     [SOLUTION_STEP0, SOLUTION_STEP1, .., SOLUTION_STEPN], any of the
> >     conversion tools should provide the right data output that can be
> >     visualized easily.
> >
> >     You should be able to use mbconvert that gets installed with MOAB for
> >     the conversion. Of course, if you want exodus output, you will need
> >     the NetCDF dependency enabled. VTK is available out of the box
> though.
> >
> >     We can also discuss more about the time series data and not writing
> >     the mesh more than once, especially for static non-AMR simulations.
> We
> >     should be able to find some time in the summer or a little later to
> >     implement this functionality within MOAB.
> >
> >     Vijay
> >
> >     On Wed, Apr 6, 2016 at 2:46 PM, Nico Schlömer
> >     <nico.schloemer at gmail.com <mailto:nico.schloemer at gmail.com>> wrote:
> >     > Thanks for the comments!
> >     >
> >     >> the visualization is the tricky bit tere since none of the main
> >     tools
> >     >> (VisIt, Paraview) understand the
> >     >> details of the layout explicitly
> >     >> [...]
> >     >>  if we understand your needs better.
> >     >
> >     > The question I have to answer the most is how to visualize H5M
> >     files. I
> >     > usually tell people to grab meshio [1] and convert to a format
> >     they use. I
> >     > guess I could extend meshio such that it can translate h5m's tag
> >     time series
> >     > data into another format with native series. Anyhow, more work
> >     for me and
> >     > for the users (to convert), so I'm not so happy with that yet.
> >     >
> >     > Cheers,
> >     > Nico
> >     >
> >     > [1] https://pypi.python.org/pypi/meshio
> >     >
> >     > On Wed, Apr 6, 2016 at 4:49 PM Vijay S. Mahadevan
> >     <vijay.m at gmail.com <mailto:vijay.m at gmail.com>> wrote:
> >     >>
> >     >> Hi NIco,
> >     >>
> >     >> Our current HDF5-based I/O workflow supports adding arbitrary
> >     tags if
> >     >> you want to store time-stepping data. This is what Lukasz is
> >     doing for
> >     >> his needs and should extend arbitrarily to other applications
> using
> >     >> MOAB. As Lukasz also mentioned, the visualization is the tricky
> bit
> >     >> here since none of the main tools (VisIt, Paraview) understand the
> >     >> details of the layout explicitly and so only expose part of the
> >     data
> >     >> available in h5m files. I've also been think about enhancements to
> >     >> support multiple HDF5 files, where the mesh is written out only
> >     >> initially and when it changes. But this is a little more
> >     involved for
> >     >> the book-keeping to remain general.
> >     >>
> >     >> On a related note, we are working with LLNL to add a native MOAB
> >     >> plugin to VisIt, that directly uses MOAB underneath, in
> >     parallel, to
> >     >> query and visualize solution. Once that is in place, adding
> >     capability
> >     >> to show time-series data can be added out of the box. Note that
> >     MOAB
> >     >> already supports only loading partial sets and tags, if you are
> >     >> concerned about the data movement with I/O for large simulation
> >     runs.
> >     >>
> >     >> However, we have also been considering alternatives that support
> >     >> parallel I/O. Yes, from our discussions, I believe we converged on
> >     >> XDMF and Exodus/Nemesis. Currently, this is a bottleneck on the
> >     >> resources available to us and so has been in the back burner.
> >     There is
> >     >> native support for time-series data with the pure NetCDF route
> >     >> already, written more focused on climate applications [1, 2].
> >     We can
> >     >> also talk about extending this to more general use cases, if we
> >     >> understand your needs better.
> >     >>
> >     >> Thoughts ?
> >     >>
> >     >> Vijay
> >     >>
> >     >> [1] https://www.ncl.ucar.edu/ParVis.shtml
> >     >> [2]
> >     https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/template.pdf
> >     >>
> >     >> On Wed, Apr 6, 2016 at 6:34 AM, Lukasz Kaczmarczyk
> >     >> <Lukasz.Kaczmarczyk at glasgow.ac.uk
> >     <mailto:Lukasz.Kaczmarczyk at glasgow.ac.uk>> wrote:
> >     >> > Hello,
> >     >> >
> >     >> > I am not convinced that this is the best approach, however I
> >     can tell
> >     >> > how I
> >     >> > do it.
> >     >> >
> >     >> > I create meshset with number of contained meshsets equal to
> >     number of
> >     >> > time
> >     >> > steps. Each of contained meshsets has tag, which data on all
> >     entities on
> >     >> > meshset which I store for time step. That can be saved in
> >     native HD5
> >     >> > file
> >     >> > and read  back if needed without data redundancies and allow
> >     to add time
> >     >> > steps. However is a issue of post-processing, that will need
> >     some proper
> >     >> > reader for post-procssor which I not currently have.
> >     >> >
> >     >> > There are additional complexities, when you store multiple
> >     fields and
> >     >> > you
> >     >> > store data for variable number of entities for each step. In
> >     may case I
> >     >> > implemented it like tape recorder, that I have different
> >     tapes for which
> >     >> > I
> >     >> > record and play tapes depending on need. If this will be any
> >     use for you
> >     >> > pleas look at example,
> >     >> >
> >     >> >
> >
> http://mofem.eng.gla.ac.uk/mofem/html/record__series__atom_8cpp_source.html
> >     >> > In example I have two tapes there and two moab instances, one
> >     use for
> >     >> > recording another for reading for testing proposes. This
> >     works ok, but
> >     >> > is
> >     >> > missing proper reader for paraview and at the end I produce
> >     series of
> >     >> > VTKs
> >     >> > for paraview. This is very specific to my development by you
> >     can use it
> >     >> > as
> >     >> > idea if you like it.
> >     >> >
> >     >> > I would be great to have some common interface to record and
> >     read time
> >     >> > series of tags for moab. That if one will write reader for
> >     paraview it
> >     >> > could
> >     >> > be used by several developers woking on different codes.
> >     >> >
> >     >> >
> >     >> > Regards,
> >     >> > Lukasz
> >     >> >
> >     >> >
> >     >> >
> >     >> >
> >     >> >
> >     >> > On 6 Apr 2016, at 10:02, Nico Schlömer
> >     <nico.schloemer at gmail.com <mailto:nico.schloemer at gmail.com>> wrote:
> >     >> >
> >     >> > Hi everyone,
> >     >> >
> >     >> > I'll start tackling a time-series problem rather soon and I'm
> >     wondering
> >     >> > about file formats here. Ideally, I'd have one time series
> >     per file
> >     >> > (particularly since the mesh isn't changing over time,
> >     perhaps that
> >     >> > could be
> >     >> > made use of), but I'm not sure if MOAB's custom HDF5 supports
> >     that yet.
> >     >> >
> >     >> > Thinking of which, I faintly remember there had been talk about
> >     >> > switching to
> >     >> > a better-supported format in the future (xdmf and exodus came
> >     to mind).
> >     >> > Is
> >     >> > this still a thing?
> >     >> >
> >     >> > Cheers,
> >     >> > Nico
> >     >> >
> >     >> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20160408/81a9911f/attachment-0001.html>


More information about the moab-dev mailing list