[MOAB-dev] file formats

Fri Apr 8 02:31:09 CDT 2016

Thanks for the comment!

To be able to do I/O from a visualizable format directly (without mbconvert
or any other tool) is in fact more important to me than not wasting space
on storing a mesh more than once (although also interesting). Perhaps these
two things can be aligned.

Cheers,
Nico

On Thu, Apr 7, 2016 at 11:06 PM Vijay S. Mahadevan <vijay.m at gmail.com>
wrote:

> Nico, if the time series data are stored as separate tags, like
> [SOLUTION_STEP0, SOLUTION_STEP1, .., SOLUTION_STEPN], any of the
> conversion tools should provide the right data output that can be
> visualized easily.
>
> You should be able to use mbconvert that gets installed with MOAB for
> the conversion. Of course, if you want exodus output, you will need
> the NetCDF dependency enabled. VTK is available out of the box though.
>
> We can also discuss more about the time series data and not writing
> the mesh more than once, especially for static non-AMR simulations. We
> should be able to find some time in the summer or a little later to
> implement this functionality within MOAB.
>
> Vijay
>
> On Wed, Apr 6, 2016 at 2:46 PM, Nico Schlömer <nico.schloemer at gmail.com>
> wrote:
> > Thanks for the comments!
> >
> >> the visualization is the tricky bit tere since none of the main tools
> >> (VisIt, Paraview) understand the
> >> details of the layout explicitly
> >> [...]
> >>  if we understand your needs better.
> >
> > The question I have to answer the most is how to visualize H5M files. I
> > usually tell people to grab meshio [1] and convert to a format they use.
> I
> > guess I could extend meshio such that it can translate h5m's tag time
> series
> > data into another format with native series. Anyhow, more work for me and
> > for the users (to convert), so I'm not so happy with that yet.
> >
> > Cheers,
> > Nico
> >
> > [1] https://pypi.python.org/pypi/meshio
> >
> > On Wed, Apr 6, 2016 at 4:49 PM Vijay S. Mahadevan <vijay.m at gmail.com>
> wrote:
> >>
> >> Hi NIco,
> >>
> >> Our current HDF5-based I/O workflow supports adding arbitrary tags if
> >> you want to store time-stepping data. This is what Lukasz is doing for
> >> his needs and should extend arbitrarily to other applications using
> >> MOAB. As Lukasz also mentioned, the visualization is the tricky bit
> >> here since none of the main tools (VisIt, Paraview) understand the
> >> details of the layout explicitly and so only expose part of the data
> >> available in h5m files. I've also been think about enhancements to
> >> support multiple HDF5 files, where the mesh is written out only
> >> initially and when it changes. But this is a little more involved for
> >> the book-keeping to remain general.
> >>
> >> On a related note, we are working with LLNL to add a native MOAB
> >> plugin to VisIt, that directly uses MOAB underneath, in parallel, to
> >> query and visualize solution. Once that is in place, adding capability
> >> to show time-series data can be added out of the box. Note that MOAB
> >> already supports only loading partial sets and tags, if you are
> >> concerned about the data movement with I/O for large simulation runs.
> >>
> >> However, we have also been considering alternatives that support
> >> parallel I/O. Yes, from our discussions, I believe we converged on
> >> XDMF and Exodus/Nemesis. Currently, this is a bottleneck on the
> >> resources available to us and so has been in the back burner. There is
> >> native support for time-series data with the pure NetCDF route
> >> already, written more focused on climate applications [1, 2]. We can
> >> also talk about extending this to more general use cases, if we
> >> understand your needs better.
> >>
> >> Thoughts ?
> >>
> >> Vijay
> >>
> >> [1] https://www.ncl.ucar.edu/ParVis.shtml
> >> [2] https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/template.pdf
> >>
> >> On Wed, Apr 6, 2016 at 6:34 AM, Lukasz Kaczmarczyk
> >> <Lukasz.Kaczmarczyk at glasgow.ac.uk> wrote:
> >> > Hello,
> >> >
> >> > I am not convinced that this is the best approach, however I can tell
> >> > how I
> >> > do it.
> >> >
> >> > I create meshset with number of contained meshsets equal to number of
> >> > time
> >> > steps. Each of contained meshsets has tag, which data on all entities
> on
> >> > meshset which I store for time step. That can be saved in native HD5
> >> > file
> >> > and read  back if needed without data redundancies and allow to add
> time
> >> > steps. However is a issue of post-processing, that will need some
> proper
> >> > reader for post-procssor which I not currently have.
> >> >
> >> > There are additional complexities, when you store multiple fields and
> >> > you
> >> > store data for variable number of entities for each step. In may case
> I
> >> > implemented it like tape recorder, that I have different tapes for
> which
> >> > I
> >> > record and play tapes depending on need. If this will be any use for
> you
> >> > pleas look at example,
> >> >
> >> >
> http://mofem.eng.gla.ac.uk/mofem/html/record__series__atom_8cpp_source.html
> >> > In example I have two tapes there and two moab instances, one use for
> >> > recording another for reading for testing proposes. This works ok, but
> >> > is
> >> > missing proper reader for paraview and at the end I produce series of
> >> > VTKs
> >> > for paraview. This is very specific to my development by you can use
> it
> >> > as
> >> > idea if you like it.
> >> >
> >> > I would be great to have some common interface to record and read time
> >> > series of tags for moab. That if one will write reader for paraview it
> >> > could
> >> > be used by several developers woking on different codes.
> >> >
> >> >
> >> > Regards,
> >> > Lukasz
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On 6 Apr 2016, at 10:02, Nico Schlömer <nico.schloemer at gmail.com>
> wrote:
> >> >
> >> > Hi everyone,
> >> >
> >> > I'll start tackling a time-series problem rather soon and I'm
> wondering
> >> > about file formats here. Ideally, I'd have one time series per file
> >> > (particularly since the mesh isn't changing over time, perhaps that
> >> > could be
> >> > made use of), but I'm not sure if MOAB's custom HDF5 supports that
> yet.
> >> >
> >> > Thinking of which, I faintly remember there had been talk about
> >> > switching to
> >> > a better-supported format in the future (xdmf and exodus came to
> mind).
> >> > Is
> >> > this still a thing?
> >> >
> >> > Cheers,
> >> > Nico
> >> >
> >> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20160408/f7498f0a/attachment-0001.html>