[MOAB-dev] file formats
Vijay S. Mahadevan
vijay.m at gmail.com
Fri Apr 8 08:13:35 CDT 2016
Yes this is an interesting extension and we can certainly add something
natively in MOAB.
Jacob, if you can point us to your code and/or want to submit a PR for new
reader/writer, we would happy to add the capability for xdmf.
Vijay
On Apr 8, 2016 7:49 AM, "Nico Schlömer" <nico.schloemer at gmail.com> wrote:
> > The xml file very generation is very quick with a simple python script
> > (we have written to general python classes for this, if you're
> interested).
>
> Very much so; where do I find the code? Perhaps this could even be
> included in MOAB itself.
>
> Cheers,
> Nico
>
> On Fri, Apr 8, 2016 at 2:43 PM Jacob Hochhalter <
> Jacob.D.Hochhalter at nasa.gov> wrote:
>
>> All,
>>
>> We have had this issue come up too. Our solution, which we have been
>> very happy with so far, is to use the .h5m file as part of the XDMF
>> concept (see http://www.xdmf.org/index.php/Main_Page). With that
>> approach, we simply 'wrap' the .h5m file with a xml file, which
>> describes which data we want to visualize (and how). There is no
>> duplication of the mesh or results data, since the xml file simply
>> 'points' to datasets in the h5m file. The xml file very generation is
>> very quick with a simple python script (we have written to general
>> python classes for this, if you're interested). An added bonus is that
>> both VisIt and Paraview support XDMF inherently.
>>
>> - Jake
>>
>> On 04/08/2016 03:31 AM, Nico Schlömer wrote:
>> > Thanks for the comment!
>> >
>> > To be able to do I/O from a visualizable format directly (without
>> > mbconvert or any other tool) is in fact more important to me than not
>> > wasting space on storing a mesh more than once (although also
>> > interesting). Perhaps these two things can be aligned.
>> >
>> > Cheers,
>> > Nico
>> >
>> > On Thu, Apr 7, 2016 at 11:06 PM Vijay S. Mahadevan <vijay.m at gmail.com
>> > <mailto:vijay.m at gmail.com>> wrote:
>> >
>> > Nico, if the time series data are stored as separate tags, like
>> > [SOLUTION_STEP0, SOLUTION_STEP1, .., SOLUTION_STEPN], any of the
>> > conversion tools should provide the right data output that can be
>> > visualized easily.
>> >
>> > You should be able to use mbconvert that gets installed with MOAB
>> for
>> > the conversion. Of course, if you want exodus output, you will need
>> > the NetCDF dependency enabled. VTK is available out of the box
>> though.
>> >
>> > We can also discuss more about the time series data and not writing
>> > the mesh more than once, especially for static non-AMR simulations.
>> We
>> > should be able to find some time in the summer or a little later to
>> > implement this functionality within MOAB.
>> >
>> > Vijay
>> >
>> > On Wed, Apr 6, 2016 at 2:46 PM, Nico Schlömer
>> > <nico.schloemer at gmail.com <mailto:nico.schloemer at gmail.com>> wrote:
>> > > Thanks for the comments!
>> > >
>> > >> the visualization is the tricky bit tere since none of the main
>> > tools
>> > >> (VisIt, Paraview) understand the
>> > >> details of the layout explicitly
>> > >> [...]
>> > >> if we understand your needs better.
>> > >
>> > > The question I have to answer the most is how to visualize H5M
>> > files. I
>> > > usually tell people to grab meshio [1] and convert to a format
>> > they use. I
>> > > guess I could extend meshio such that it can translate h5m's tag
>> > time series
>> > > data into another format with native series. Anyhow, more work
>> > for me and
>> > > for the users (to convert), so I'm not so happy with that yet.
>> > >
>> > > Cheers,
>> > > Nico
>> > >
>> > > [1] https://pypi.python.org/pypi/meshio
>> > >
>> > > On Wed, Apr 6, 2016 at 4:49 PM Vijay S. Mahadevan
>> > <vijay.m at gmail.com <mailto:vijay.m at gmail.com>> wrote:
>> > >>
>> > >> Hi NIco,
>> > >>
>> > >> Our current HDF5-based I/O workflow supports adding arbitrary
>> > tags if
>> > >> you want to store time-stepping data. This is what Lukasz is
>> > doing for
>> > >> his needs and should extend arbitrarily to other applications
>> using
>> > >> MOAB. As Lukasz also mentioned, the visualization is the tricky
>> bit
>> > >> here since none of the main tools (VisIt, Paraview) understand
>> the
>> > >> details of the layout explicitly and so only expose part of the
>> > data
>> > >> available in h5m files. I've also been think about enhancements
>> to
>> > >> support multiple HDF5 files, where the mesh is written out only
>> > >> initially and when it changes. But this is a little more
>> > involved for
>> > >> the book-keeping to remain general.
>> > >>
>> > >> On a related note, we are working with LLNL to add a native MOAB
>> > >> plugin to VisIt, that directly uses MOAB underneath, in
>> > parallel, to
>> > >> query and visualize solution. Once that is in place, adding
>> > capability
>> > >> to show time-series data can be added out of the box. Note that
>> > MOAB
>> > >> already supports only loading partial sets and tags, if you are
>> > >> concerned about the data movement with I/O for large simulation
>> > runs.
>> > >>
>> > >> However, we have also been considering alternatives that support
>> > >> parallel I/O. Yes, from our discussions, I believe we converged
>> on
>> > >> XDMF and Exodus/Nemesis. Currently, this is a bottleneck on the
>> > >> resources available to us and so has been in the back burner.
>> > There is
>> > >> native support for time-series data with the pure NetCDF route
>> > >> already, written more focused on climate applications [1, 2].
>> > We can
>> > >> also talk about extending this to more general use cases, if we
>> > >> understand your needs better.
>> > >>
>> > >> Thoughts ?
>> > >>
>> > >> Vijay
>> > >>
>> > >> [1] https://www.ncl.ucar.edu/ParVis.shtml
>> > >> [2]
>> > https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/template.pdf
>> > >>
>> > >> On Wed, Apr 6, 2016 at 6:34 AM, Lukasz Kaczmarczyk
>> > >> <Lukasz.Kaczmarczyk at glasgow.ac.uk
>> > <mailto:Lukasz.Kaczmarczyk at glasgow.ac.uk>> wrote:
>> > >> > Hello,
>> > >> >
>> > >> > I am not convinced that this is the best approach, however I
>> > can tell
>> > >> > how I
>> > >> > do it.
>> > >> >
>> > >> > I create meshset with number of contained meshsets equal to
>> > number of
>> > >> > time
>> > >> > steps. Each of contained meshsets has tag, which data on all
>> > entities on
>> > >> > meshset which I store for time step. That can be saved in
>> > native HD5
>> > >> > file
>> > >> > and read back if needed without data redundancies and allow
>> > to add time
>> > >> > steps. However is a issue of post-processing, that will need
>> > some proper
>> > >> > reader for post-procssor which I not currently have.
>> > >> >
>> > >> > There are additional complexities, when you store multiple
>> > fields and
>> > >> > you
>> > >> > store data for variable number of entities for each step. In
>> > may case I
>> > >> > implemented it like tape recorder, that I have different
>> > tapes for which
>> > >> > I
>> > >> > record and play tapes depending on need. If this will be any
>> > use for you
>> > >> > pleas look at example,
>> > >> >
>> > >> >
>> >
>> http://mofem.eng.gla.ac.uk/mofem/html/record__series__atom_8cpp_source.html
>> > >> > In example I have two tapes there and two moab instances, one
>> > use for
>> > >> > recording another for reading for testing proposes. This
>> > works ok, but
>> > >> > is
>> > >> > missing proper reader for paraview and at the end I produce
>> > series of
>> > >> > VTKs
>> > >> > for paraview. This is very specific to my development by you
>> > can use it
>> > >> > as
>> > >> > idea if you like it.
>> > >> >
>> > >> > I would be great to have some common interface to record and
>> > read time
>> > >> > series of tags for moab. That if one will write reader for
>> > paraview it
>> > >> > could
>> > >> > be used by several developers woking on different codes.
>> > >> >
>> > >> >
>> > >> > Regards,
>> > >> > Lukasz
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> > On 6 Apr 2016, at 10:02, Nico Schlömer
>> > <nico.schloemer at gmail.com <mailto:nico.schloemer at gmail.com>> wrote:
>> > >> >
>> > >> > Hi everyone,
>> > >> >
>> > >> > I'll start tackling a time-series problem rather soon and I'm
>> > wondering
>> > >> > about file formats here. Ideally, I'd have one time series
>> > per file
>> > >> > (particularly since the mesh isn't changing over time,
>> > perhaps that
>> > >> > could be
>> > >> > made use of), but I'm not sure if MOAB's custom HDF5 supports
>> > that yet.
>> > >> >
>> > >> > Thinking of which, I faintly remember there had been talk about
>> > >> > switching to
>> > >> > a better-supported format in the future (xdmf and exodus came
>> > to mind).
>> > >> > Is
>> > >> > this still a thing?
>> > >> >
>> > >> > Cheers,
>> > >> > Nico
>> > >> >
>> > >> >
>> >
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20160408/f14fa632/attachment.html>
More information about the moab-dev
mailing list