[MOAB-dev] Interface: Partitioner

[Alexander Nikolas Breuer]

Hey Tim,

A short heads up: I'll have to postpone the work regarding the meshes a bit (December). Let me comment on the remarks anyhow:

> Depends on what you're looking for.  mbpart is our tool to generate partitions, but it works for static partitioning; we don't really do dynamic partitioning at the moment.  mbpart can establish the graph, which it passes to Zoltan to do the actual partitioning.

Static in the sense of AMR? Our mesh doesn't change over time -> If mbpart can create a graph from moabs underlying data management, we should be good to go and get some hints from the implementation of mbpart.

> Zoltan in theory can use metis to generate the partition, though I haven't had much luck with it that way (though maybe Zoltan2, which we haven't updated to, works better) (though now that I say that, it could just as easily be a bug in moab which we haven't found).

Ok, it's probably worth trying out Zoltan2 then, which supports arbitrary large meshes additionally (according to the homepage).

> So, you could take the code from mbpart that generates the graph and passes to parmetis, and that would give you back a partition.  You'd just need to make sure to establish shared mesh between processors correctly before doing that (and possibly exchanging a row of ghost elements), to make sure that both processors sharing vertices would see the same connected elements.  The HelloParMOAB example (http://www.mcs.anl.gov/~fathom/moab-docs/html/HelloParMOAB_8cpp-example.html) shows how to request ghosting, after you've established shared entities.

Yah, I'll do that. Doing it in parallel was on of the main reasons for MOAB and on my schedule anyway. :)

Thanks for your reply & Regards,

Alex

-- 
Dipl.-Math. Alexander Breuer
Technische Universität München
Department of Informatics
Chair of Scientific Computing
Boltzmannstrasse 3, 85748 Garching, Germany
Tel: +49 89 358 31 7811, Fax: +49 89 289 18 607
http://www5.in.tum.de



On Sep 9, 2013, at 5:57 PM, Tim Tautges <tautges at mcs.anl.gov> wrote:

> 
> 
> On 09/08/2013 10:55 AM, Alexander Nikolas Breuer wrote:
>> Hey Timothy Tautges,
>> 
>> I got your e-mail address from a colleague of mine. Maybe you are able to give me some hints on partitioning based on
>> MOAB. My original e-mail is attached to this one.
>> 
> 
> Aha.  I'll cc moab-dev, feel free to email that list with these types of questions too.
>> Best regards,
>> 
>> Alexander Breuer
>> 
> 
> > From: Alexander Nikolas Breuer <breuera at in.tum.de>
> > Subject: Interface: Partitioner
> > Date: September 2, 2013 8:30:09 PM GMT+02:00
> > To: moab at software.sandia.gov
> >
> > Hey MOAB-developers,
> >
> > first of all: Thank's for developing this nice piece of software.
> >
> > I started using MOAB in the pre-processing phase of a finite element-code to cluster time step widths. We plan to go above 10^9 elements soon, so this might be an interesting benchmark. I think I figured out how to use the basics of MOAB, like getting enities or using mesh sets. My next step is the partitioning:
> 
> Great, let us know if you encounter problems.
> 
> >
> > I will have the mesh in MOAB including information for computational and communication costs in the FEM setting. Each of my tetrahedrons is part of a local time stepping cluster, which has a weight for computation - the lower the time step width, the higher the amount of computation in this element, because it has to be updated more often in the LTS scheme. Each face has a weight for communication: Two elements with large time step widths have to communicate much less than neighboured elements with low time step widths.
> >
> > To my question: I need to get that information into a partitioner like ParMetis without writing to disk in between. MOAB seems to have some functionality for that, i.e.: http://www.mcs.anl.gov/~fathom/moab-docs/html/mbpart_8cpp.html#a0ddf1224851353fc92bfbff6f499fa97 Do you have some documentation on the interfaces? Or more examples?
> 
> Depends on what you're looking for.  mbpart is our tool to generate partitions, but it works for static partitioning; we don't really do dynamic partitioning at the moment.  mbpart can establish the graph, which it passes to Zoltan to do the actual partitioning.  Zoltan in theory can use metis to generate the partition, though I haven't had much luck with it that way (though maybe Zoltan2, which we haven't updated to, works better) (though now that I say that, it could just as easily be a bug in moab which we haven't found).
> 
> So, you could take the code from mbpart that generates the graph and passes to parmetis, and that would give you back a partition.  You'd just need to make sure to establish shared mesh between processors correctly before doing that (and possibly exchanging a row of ghost elements), to make sure that both processors sharing vertices would see the same connected elements.  The HelloParMOAB example (http://www.mcs.anl.gov/~fathom/moab-docs/html/HelloParMOAB_8cpp-example.html) shows how to request ghosting, after you've established shared entities.
> 
> - tim
> 
> >
> > ***
> >
> > I hope this is right way to ask questions. If not: I would be happy, if you could point me to the right place.
> >
> > Thanks in advance and regards,
> >
> > Alex
> >
> > --
> > Dipl.-Math. Alexander Breuer
> > Technische Universität München
> > Department of Informatics
> > Chair of Scientific Computing
> > Boltzmannstrasse 3, 85748 Garching, Germany
> > Tel: +49 89 358 31 7811, Fax: +49 89 289 18 607
> > http://www5.in.tum.de
> >
> >
> >
> 
> 
> 
> 
> -- 
> ================================================================
> "You will keep in perfect peace him whose mind is
>  steadfast, because he trusts in you."               Isaiah 26:3
> 
>             Tim Tautges            Argonne National Laboratory
>         (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
> phone (gvoice): (608) 354-1459      1500 Engineering Dr.
>            fax: (608) 263-4499      Madison, WI 53706
>