[petsc-users] Load balancing / redistributing a 1D DM

[Matthew Knepley]

On Mon, Mar 5, 2018 at 4:29 AM, Åsmund Ervik <Asmund.Ervik at sintef.no> wrote:

> Hi all,
>
> We have a code that solves the 1D multiphase Euler equations, using some
> very expensive thermodynamic calls in each cell in each time step. The
> computational time for different cells varies significantly in the spatial
> direction (due to different thermodynamic states), and varies slowly from
> timestep to timestep.
>
> Currently the code runs in serial, but I would like to use a PETSc DM of
> some sort to run it in parallell. There will be no linear on nonlinear
> PETSc solves etc., just a distributed mesh, at least initially. The code is
> Fortran.
>
> Now for my question: Is it possible to do dynamic load balancing using a
> plain 1D DMDA, somehow?


Yes.


> There is some mention of this for PCTELESCOPE, but I guess it only works
> for linear solves?


No, that is just their application.


> Or could I use an index set or some other PETSc structure? Or do I need to
> use a 1D DMPLEX?
>

1D Plex and 1D DA are basically equivalent since 1D has no topology, and
discretizations look mostly the same.

Now that I am looking, the Plex redistribution is exposed, but the DA
redistribution is not.


> If the latter, how do I make a 1D DMPLEX? All the variables are stored in
> cell centers (collocated), so it's a completely trivial "mesh". I tried
> reading the DMPLEX manual, and looking at examples, but I'm having trouble
> penetrating the FEM lingo / abstract nonsense.
>

First question, do you want to create the partition yourself, or have a
program do it for you?

if you make it, then DA is the best bet, and we should just expose the
redistribution code which is
already in PCTELESCOPE, so that you can call it directly.

If you want a partitioner to do it, then Plex is probably the best, In that
case, you need to make a trivial
PetscSection to say how many variables lie on each cell. With this info,
everything else should be pretty
automatic.

  Thanks,

     Matt


> Best regards,
> Åsmund
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20180305/55761109/attachment.html>