[petsc-dev] proposed minor PetscPartitioner changes

[Vaclav Hapla]

> 8. 11. 2017 v 9:06, Lisandro Dalcin <dalcinl at gmail.com>:
> 
> On 8 November 2017 at 05:51, Smith, Barry F. <bsmith at mcs.anl.gov> wrote:
>> 
>>> On Nov 7, 2017, at 1:33 AM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>>> 
>>> The only concern I have about PetscPartitioner is that the API depends
>>> on DM (PetscPartitionerPartition_<TYPE> routines). Maybe
>>> PetscPartitioner should eventually move to became more agnostic, and
>>> that way it can be used to partition matrices and meshes.
>> 
>>   This is certainly a serious flaw if PetscPartitioner is intended as THE API to use for partitioning. If it is not intended as THE API for partitioning then that is also a problem, because why have multiple APIs for what is essentially one set of abstractions.
>> 
> 
> Note however that things looks easy to refactor. I'll try to team up
> with Matt to improve things.

Wait, now we are at the beginning again. I actually wanted to do some refactoring of PetscPartitioner, starting with few cosmetic changes to make it better settable from options. But Barry kept me back of any edits since he think it's anti-systematic to keep two independent classes doing essentially the same. And I agree with that to be honest. It's strange to have two ParMetis, two Scotch and two whatever interfaces. The only thing I don't like on MatPartitioning is its name as it's not just for Mat Partitioning :-)

There are from my point of view multiple issues with PetscPartitioner. Let's look e.g. at PetscPartitionerPartition. It takes as arguments both PetscPartitioner and DM. This DM must be in fact DMPlex which is not checked so it will probably crash somewhere deep in the stack once the first DMPlex specific line is reached. Then there are two output arguments PetscSection partSection and IS *partition. The first must be created beforehand while the second is created inside. And I guess they must keep the same basic information just in two different forms.

Actually the original problem I wanted to solve is that src/dm/impls/plex/examples/tutorials/ex5.c fails with partitioner set to PETSCPARTITIONERPARMETIS for certain numbers of processes, see below. Let me start with pull request altering ex5.c so that partitioner type can be set from options properly and this bug can be reproduced easily.
[ 0] ***ASSERTION failed on line 176 of file /scratch/petsc-dev/arch-linux-gcc-salvus/externalpackages/git.parmetis/libparmetis/comm.c: j == nnbrs
[ 2] ***ASSERTION failed on line 176 of file /scratch/petsc-dev/arch-linux-gcc-salvus/externalpackages/git.parmetis/libparmetis/comm.c: j == nnbrs
ex5: /scratch/petsc-dev/arch-linux-gcc-salvus/externalpackages/git.parmetis/libparmetis/comm.c:176: libparmetis__CommSetup: Assertion `j == nnbrs' failed.
ex5: /scratch/petsc-dev/arch-linux-gcc-salvus/externalpackages/git.parmetis/libparmetis/comm.c:176: libparmetis__CommSetup: Assertion `j == nnbrs' failed.

I'm wondering whether the MatPartitioning interface has the same problem. But anyhow I mean it's maybe time to decide about the two interfaces before chasing all these PetscPartitioner issues.

I propose
- to rename Mat Partitioning to just Partitioning/-er or take over the name PetscPartitioner,
- what is now PetscPartitioner would be MeshPartitioning/-er and it would become sort of adapter from DMPlex meshes to general graphs so that the more general class above can be employed.

Note that I created an issue request as Barry suggested (#192) but there's no feedback yet.

Note my top level intention is to chip in some more distributed mesh loaders - currently there's only one for Salome/MED format which is rather non-standard and not much documented.

Thanks

Vaclav

> 
> 
> -- 
> Lisandro Dalcin
> ============
> Research Scientist
> Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
> Extreme Computing Research Center (ECRC)
> King Abdullah University of Science and Technology (KAUST)
> http://ecrc.kaust.edu.sa/
> 
> 4700 King Abdullah University of Science and Technology
> al-Khawarizmi Bldg (Bldg 1), Office # 0109
> Thuwal 23955-6900, Kingdom of Saudi Arabia
> http://www.kaust.edu.sa
> 
> Office Phone: +966 12 808-0459