[petsc-dev] [petsc-maint #101028] Re: BDDC code

[Jed Brown]

Cc'ing petsc-dev to get other opinions.

On Mon, Jan 9, 2012 at 06:56, Stefano Zampini <stefano.zampini at gmail.com>wrote:

> Because of the way I built the code, different coarse problem types set
> communicators and operators in a different way. Specifically,
>
> - MULTILEVEL_BDDC: mat_type=MATIS, coarse_comm=comm of parent,
> coarse_pc=PCBDDC.
> - PARALLEL_BDDC: mat_type=MATMPIAIJ, coarse_comm=comm of parent,
> coarse_pc=PCREDUNDANT.
>

The only distinction here is the matrix type.


> - SEQUENTIAL_BDDC (only rank 0 creates it): mat_type=MATSEQAIJ,
> coarse_comm=PETSC_COMM_SELF, coarse_pc=PCLU.
>

This should be handled through a mechanism like PCREDUNDANT so that the
caller does not need to know. This "reduce to subcommunicator, broadcast
the result" would be simple to add to PCREDUNDANT.


> - REPLICATED_BDDC (all ranks creates the same sequential ksp):
> mat_type=MATSEQAIJ, coarse_comm=PETSC_COMM_SELF, coarse_pc=PCLU.
>

What does this have over using PCREDUNDANT as in case 2?


>
>
>>
>>
>>>  indeed a possible user code
>>>
>>> PCBDDCGetCoarseKSP(pcbddc,&coarse_ksp)
>>> KSPGetPC(coarse_ksp,&coarse_pc)
>>> PCSetType(coarse_pc,PCBDDC) (or something else)
>>> PCSetup(pcbddc)
>>>
>>> breaks at the second line since pcbddc->coarse_ksp doesn't exist before
>>> calling PCSetup.
>>>
>>
>> You can make PCBDDCGetCoarseKSP() return a KSP that has not had its type
>> set yet.
>>
>>
>
> By creating the KSP object internally to the function call? In this case
> the problem remains since the user should need to know a priori the type of
> communicator associated with the KSP. Can we change the communicator after
> we created the KSP object?
>

No, but I think that the coarse communicator should probably always be the
same as for the whole problem, and then use PCREDUNDANT or similar if we
want to reduce to a subcomm.

But this brings up an issue that also appears in GAMG, how to expose user
access to coarse KSPs even though they are created on adaptively sized
subcomms. If we configure strictly through the options database, then we
are okay, but what if we have to pass any information directly to the KSP?
We can set operators and call PCSetUp(), then pull out levels, but what
should be returned on ranks that are not participating?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120109/96315142/attachment.html>