sieve-dev loading mesh in parallel

[Shi Jin]

Hi,

Thanks to your help, my CFD code using sieve is close to complete. One last
part to do is to enable each process to load its own grid in parallel.
Currently, I first ask the master process to load the whole mesh and call
MeshDistribute() to distribute the smaller local mesh into each process.
This method works fine if the mesh is small to medium in size. As the mesh
gets very large, the memory require to store the whole mesh on a single
process becomes impossible (this way, the memory of the master process is
much larger than the others). Since each process only needs its own portion
of the mesh anyway, it should just load its own. I am wondering if some help
on this topic can be provided from the sieve-dev team. Thanks a lot.

I have thought about a few possibilities:
1. Like the example codes, I can use tetgen to generate the mesh on the fly
and then refine it. However, this way the domain decomposition is quite
random which does not work with our particle simulation code where a clearly
defined domain boundary (such as some z values) is necessary to track the
particles.
2. I can load a coarse mesh that I am using right now. After calling
MeshDistribute() to obtain the correct domain decomposition, I can then
refine each mesh using tetgen to the required resolution. However, when I
tried this method, the refined mesh is actually repeating the coarse
elements many times (thus although the number of nodes and vertexes are
larger, the mesh is the same as before).
3. Still based on the current code, each process will try to load the same
input global mesh file. Instead of loading everything, each process will
only take the data that is within the pre-defined domain and throw away the
outside data. This way, I can completely separate the mesh generator and the
CFD code (the mesh can be generated with some sophisticated software
separately). But it is important to keep the data structure across process
consistent, i.e., the numbering  and the ghost points must make sense. I
think this is exactly what the MeshDistribute() code does and it is the
perfect solution. However, to do this, one may need a deeper understanding
of the sieve library.

I think the third possibility is the best and if this is not already
implemented, it is probably good to have some kind of function like it in
the library. I am more than happy to contribute my code to this development
and of course I need some guidance. Thank you very much.

Shi

-- 
Sincerely,
Shi Jin, Ph.D.
http://www.ualberta.ca/~sjin1/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.mcs.anl.gov/mailman/private/sieve-dev/attachments/20080308/ea56adbf/attachment.htm>