[MOAB-dev] VTK structured mesh outpu

Thu Sep 22 14:41:19 CDT 2016

Hi all,

Just to clarify the motivations....

We prefer the RECTILINEAR_GRID VTK file format because other 
applications (e.g. ParaView) appear to be much more memory efficient 
when reading/storing it.  We are largely unconcerned with the memory 
footprint when this data is stored in MOAB.

The file sizes are also much smaller which turns out to be important 
when transmitting many of them from UW through US ITER (ORNL) to the 
ITER organization in France.

The reason for exploring SCDBox is to explicitly store the X, Y, and Z 
locations of the vertices for easy access rather than to infer them from 
the list of all vertices.  (We currently have a script that does this - 
it reads all the vertices, makes a list of x, y, and z locations, and 
then takes the unique subset, etc...)

Paul

On 09/22/2016 02:26 PM, Grindeanu, Iulian R. wrote:
> Hi Paul,
> So a rectilinear object needs to store just 3 arrays, coordinates along each of the directions; one array along x, let's say xi, i=1,nx, one array along y, yj, j=1,ny, etc.
> Scd box references/stores explicitly all vertices, which means it uses in memory, for coordinates, 3 arrays for each coordinate, of size nx*ny*nz; A lot more than necessary for a rectilinear object in VTK.
>
> I agree that ScdBox can be used, with some new flags, to differentiate it from regular ScdBox
>
> memory saved is still much better than using fully unstructured;
>
> Iulian
> ________________________________________
> From: Paul Wilson [paul.wilson at wisc.edu]
> Sent: Wednesday, September 21, 2016 10:02 AM
> To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
> Cc: Timothy Tautges
> Subject: Re: [MOAB-dev] VTK structured mesh outpu
>
> Hi Iulian,
>
> I'm reasonably sure that the SCDBox class can handle this.  We'll
> investigate further and let you know.
>
> At this point, I just wanted to make sure we weren't duplicating any
> effort :)
>
> Paul
>
>
> On 09/21/2016 09:56 AM, Grindeanu, Iulian R. wrote:
>> Hi Paul,
>> I am not sure what would be the best way to accomplish this.
>> I think it would need a new class/ type, similar to SCDBox. Maybe SCDRectilinear.
>> The entity handle space for vertices AND hexahedra will need to be reserved, the same way hexa entity handle space is reserved for an SCDBox. (Maybe that class can be derived from SCDBox?)
>>
>> Tim developed SCD Interface and SCDBox about 10 years ago, and it is used for climate readers right now.
>> His opinion on this would be very valuable.
>>
>> An SCDRectilinear object could be used for VTK readers/writer only, the rest of the readers/writers would fall back to unstructured representation.
>>
>> Hmmm, maybe I am over-complicating things.
>>
>> Iulian
>>
>> ________________________________________
>> From: Paul Wilson [paul.wilson at wisc.edu]
>> Sent: Wednesday, September 21, 2016 6:14 AM
>> To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
>> Subject: Re: [MOAB-dev] VTK structured mesh outpu
>>
>> Hello Iulian,
>>
>>
>> On 09/20/2016 11:12 AM, Grindeanu, Iulian R. wrote:
>>> Hello Paul,
>>> I am not sure I understand what dagmc needs
>>> (I should probably read the code in moab :)
>> This is not for DAGMC, but related to other ways that we use MOAB to
>> assist with visualization & analysis of mesh data.
>>
>>> Moab SCD interface is aligned with VTK structured grid  (legacy format, we do not do anything with XML)
>>> ( DATASET STRUCTURED_GRID)
>>>
>>> in that, all points' coordinates are explicit, and in a specific order ( I think c style, need to check; need to check if it is the same convention in vtk)
>>> all cells are "implicit", their connectivity is never stored, it is just derived from the dimensions (2d or 3d)
>>>
>>> Moab's SCD interface allows all kinds of access. You can request neighbors, etc, and it works in parallel too, in the sense that each task has a very specific partition , it knows about neighbors, can do ghosting, etc. Pretty cool stuff, but rather verbose. Not sure if you need anything for parallel
>>>
>>> An SCDBox from moab will correspond to a DATASET STRUCTURED_GRID from vtk
>>>
>>> vtk has other "structured" meshes dataset types, that are simpler: STRUCTURED_POINTS and RECTILINEAR_GRID
>>>
>>> Which one of those 3 does dagmc need?
>> We are looking at the "DATASET RECTILINEAR_GRID" option, in which the
>> points are NOT explicitly stored.  Instead, the set of x, y, and z
>> locations are stored in separate lists and the connectivity is never
>> stored.  This change allows some of our larger files to go from 6GB to
>> <900MB!!
>>
>> We're looking into the steps it will take to accomplish this, hopefully
>> as an addition to MOAB.
>>
>> Paul
>>> Iulian
>>>
>>> http://www.vtk.org/Wiki/VTK/Writing_VTK_files_using_python
>>> http://www.vtk.org/wp-content/uploads/2015/04/file-formats.pdf
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: Paul Wilson [paul.wilson at wisc.edu]
>>> Sent: Monday, September 19, 2016 4:13 PM
>>> To: Grindeanu, Iulian R.; moab-dev at mcs.anl.gov
>>> Subject: Re: [MOAB-dev] VTK structured mesh outpu
>>>
>>> Hi Iulian,
>>>
>>> We are much more interested in the writer, but might be convinced to
>>> support a reader as well :)
>>>
>>> We would want to explore the SCD interface to see how it could best
>>> support this.
>>>
>>> Our use case begins with MCNP mesh tally output files that are always
>>> structured mesh.  There is a native reader in MOAB for these files, but
>>> not as rich/robust as we'd like, and it does NOT use the scd interface,
>>> instead making an unstructured representation.  There is also a reader
>>> in the PyNE toolkit that does ultimately use the scd interface, but then
>>> also relies on MOAB's vtk writer which stores an unstructured
>>> representation.
>>>
>>> The key to a structured (Cartesian) mesh VTK writer is having access to
>>> the x,y,z bounds without having to infer them from the set of vertices.
>>>
>>> The rest is probably straightforward.
>>>
>>> Paul
>>>
>>>
>>> On 09/19/2016 03:22 PM, Grindeanu, Iulian R. wrote:
>>>> MOAB does not have a structured writer. Not even for h5m format
>>>> (it has the ability to create structured meshes)
>>>> even in that case, the mesh is written explicit (unstructured)
>>>>
>>>> structured vtk format is relatively simpler, but do you plan to add reader and writer or just writer?
>>>>
>>>> How do you create structured mesh, with scd interface?
>>>>
>>>> thanks,
>>>> Iulian
>>>> ________________________________________
>>>> From: moab-dev-bounces at mcs.anl.gov [moab-dev-bounces at mcs.anl.gov] on behalf of Paul Wilson [paul.wilson at wisc.edu]
>>>> Sent: Monday, September 19, 2016 3:12 PM
>>>> To: moab-dev at mcs.anl.gov
>>>> Subject: [MOAB-dev] VTK structured mesh outpu
>>>>
>>>> Hi all,
>>>>
>>>> Does the MOAB VTK writer have a way to write structured mesh data using
>>>> VTK's structured mesh format?  For data that conforms to that, it saves
>>>> a substantial amount of disk space and also speeds up
>>>> processing/visualization times.
>>>>
>>>> If not, we may be interested in helping add this...
>>>>
>>>> Paul
>>>>
>>>>
>>>> --
>>>> -- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ --
>>>>
>>>> Paul P.H. Wilson
>>>> Grainger Professor of Nuclear Engineering
>>>> 608-263-0807
>>>> paul.wilson at wisc.edu
>>>> 419 Engineering Research Bldg
>>>> 1500 Engineering Dr, Madison, WI 53706
>>>> calendar: http://go.wisc.edu/pphw-cal
>>>>
>>>> Computational Nuclear Engineering Research Group
>>>> cnerg.engr.wisc.edu
>>>>
>>>> Faculty Director, Advanced Computing Initiative
>>>> aci.wisc.edu
>>>>
>>> --
>>> -- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ --
>>>
>>> Paul P.H. Wilson
>>> Grainger Professor of Nuclear Engineering
>>> 608-263-0807
>>> paul.wilson at wisc.edu
>>> 419 Engineering Research Bldg
>>> 1500 Engineering Dr, Madison, WI 53706
>>> calendar: http://go.wisc.edu/pphw-cal
>>>
>>> Computational Nuclear Engineering Research Group
>>> cnerg.engr.wisc.edu
>>>
>>> Faculty Director, Advanced Computing Initiative
>>> aci.wisc.edu
>>>
>> --
>> -- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ --
>>
>> Paul P.H. Wilson
>> Grainger Professor of Nuclear Engineering
>> 608-263-0807
>> paul.wilson at wisc.edu
>> 419 Engineering Research Bldg
>> 1500 Engineering Dr, Madison, WI 53706
>> calendar: http://go.wisc.edu/pphw-cal
>>
>> Computational Nuclear Engineering Research Group
>> cnerg.engr.wisc.edu
>>
>> Faculty Director, Advanced Computing Initiative
>> aci.wisc.edu
>>
> --
> -- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ --
>
> Paul P.H. Wilson
> Grainger Professor of Nuclear Engineering
> 608-263-0807
> paul.wilson at wisc.edu
> 419 Engineering Research Bldg
> 1500 Engineering Dr, Madison, WI 53706
> calendar: http://go.wisc.edu/pphw-cal
>
> Computational Nuclear Engineering Research Group
> cnerg.engr.wisc.edu
>
> Faculty Director, Advanced Computing Initiative
> aci.wisc.edu
>

-- 
-- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ --

Paul P.H. Wilson
Grainger Professor of Nuclear Engineering
608-263-0807
paul.wilson at wisc.edu
419 Engineering Research Bldg
1500 Engineering Dr, Madison, WI 53706
calendar: http://go.wisc.edu/pphw-cal

Computational Nuclear Engineering Research Group
cnerg.engr.wisc.edu

Faculty Director, Advanced Computing Initiative
aci.wisc.edu