[MOAB-dev] file formats

[Vijay S. Mahadevan]

Nico, if the time series data are stored as separate tags, like
[SOLUTION_STEP0, SOLUTION_STEP1, .., SOLUTION_STEPN], any of the
conversion tools should provide the right data output that can be
visualized easily.

You should be able to use mbconvert that gets installed with MOAB for
the conversion. Of course, if you want exodus output, you will need
the NetCDF dependency enabled. VTK is available out of the box though.

We can also discuss more about the time series data and not writing
the mesh more than once, especially for static non-AMR simulations. We
should be able to find some time in the summer or a little later to
implement this functionality within MOAB.

Vijay

On Wed, Apr 6, 2016 at 2:46 PM, Nico Schlömer <nico.schloemer at gmail.com> wrote:
> Thanks for the comments!
>
>> the visualization is the tricky bit tere since none of the main tools
>> (VisIt, Paraview) understand the
>> details of the layout explicitly
>> [...]
>>  if we understand your needs better.
>
> The question I have to answer the most is how to visualize H5M files. I
> usually tell people to grab meshio [1] and convert to a format they use. I
> guess I could extend meshio such that it can translate h5m's tag time series
> data into another format with native series. Anyhow, more work for me and
> for the users (to convert), so I'm not so happy with that yet.
>
> Cheers,
> Nico
>
> [1] https://pypi.python.org/pypi/meshio
>
> On Wed, Apr 6, 2016 at 4:49 PM Vijay S. Mahadevan <vijay.m at gmail.com> wrote:
>>
>> Hi NIco,
>>
>> Our current HDF5-based I/O workflow supports adding arbitrary tags if
>> you want to store time-stepping data. This is what Lukasz is doing for
>> his needs and should extend arbitrarily to other applications using
>> MOAB. As Lukasz also mentioned, the visualization is the tricky bit
>> here since none of the main tools (VisIt, Paraview) understand the
>> details of the layout explicitly and so only expose part of the data
>> available in h5m files. I've also been think about enhancements to
>> support multiple HDF5 files, where the mesh is written out only
>> initially and when it changes. But this is a little more involved for
>> the book-keeping to remain general.
>>
>> On a related note, we are working with LLNL to add a native MOAB
>> plugin to VisIt, that directly uses MOAB underneath, in parallel, to
>> query and visualize solution. Once that is in place, adding capability
>> to show time-series data can be added out of the box. Note that MOAB
>> already supports only loading partial sets and tags, if you are
>> concerned about the data movement with I/O for large simulation runs.
>>
>> However, we have also been considering alternatives that support
>> parallel I/O. Yes, from our discussions, I believe we converged on
>> XDMF and Exodus/Nemesis. Currently, this is a bottleneck on the
>> resources available to us and so has been in the back burner. There is
>> native support for time-series data with the pure NetCDF route
>> already, written more focused on climate applications [1, 2]. We can
>> also talk about extending this to more general use cases, if we
>> understand your needs better.
>>
>> Thoughts ?
>>
>> Vijay
>>
>> [1] https://www.ncl.ucar.edu/ParVis.shtml
>> [2] https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/template.pdf
>>
>> On Wed, Apr 6, 2016 at 6:34 AM, Lukasz Kaczmarczyk
>> <Lukasz.Kaczmarczyk at glasgow.ac.uk> wrote:
>> > Hello,
>> >
>> > I am not convinced that this is the best approach, however I can tell
>> > how I
>> > do it.
>> >
>> > I create meshset with number of contained meshsets equal to number of
>> > time
>> > steps. Each of contained meshsets has tag, which data on all entities on
>> > meshset which I store for time step. That can be saved in native HD5
>> > file
>> > and read  back if needed without data redundancies and allow to add time
>> > steps. However is a issue of post-processing, that will need some proper
>> > reader for post-procssor which I not currently have.
>> >
>> > There are additional complexities, when you store multiple fields and
>> > you
>> > store data for variable number of entities for each step. In may case I
>> > implemented it like tape recorder, that I have different tapes for which
>> > I
>> > record and play tapes depending on need. If this will be any use for you
>> > pleas look at example,
>> >
>> > http://mofem.eng.gla.ac.uk/mofem/html/record__series__atom_8cpp_source.html
>> > In example I have two tapes there and two moab instances, one use for
>> > recording another for reading for testing proposes. This works ok, but
>> > is
>> > missing proper reader for paraview and at the end I produce series of
>> > VTKs
>> > for paraview. This is very specific to my development by you can use it
>> > as
>> > idea if you like it.
>> >
>> > I would be great to have some common interface to record and read time
>> > series of tags for moab. That if one will write reader for paraview it
>> > could
>> > be used by several developers woking on different codes.
>> >
>> >
>> > Regards,
>> > Lukasz
>> >
>> >
>> >
>> >
>> >
>> > On 6 Apr 2016, at 10:02, Nico Schlömer <nico.schloemer at gmail.com> wrote:
>> >
>> > Hi everyone,
>> >
>> > I'll start tackling a time-series problem rather soon and I'm wondering
>> > about file formats here. Ideally, I'd have one time series per file
>> > (particularly since the mesh isn't changing over time, perhaps that
>> > could be
>> > made use of), but I'm not sure if MOAB's custom HDF5 supports that yet.
>> >
>> > Thinking of which, I faintly remember there had been talk about
>> > switching to
>> > a better-supported format in the future (xdmf and exodus came to mind).
>> > Is
>> > this still a thing?
>> >
>> > Cheers,
>> > Nico
>> >
>> >