[petsc-users] Vecs and Mats with non-contiguous parallel layout

[Dave Lee]

Thanks Matt,

I'm currently working through the details of the nested field split. I've
decided to re-organise my application data layout so that the local
portions are contiguous in the global space for consistency with the
default PETSc layout.

Does the IndexSet need to be the same size on each processor in order for
this IndexSet to be supplied to a FieldSplit PC?

Moreover can an IndexSet have 0 entries on a particular processor and still
be part of a global FieldSplit PC?

Cheers, Dave.


On Wed, Jan 30, 2019 at 11:10 PM Matthew Knepley <knepley at gmail.com> wrote:

> On Tue, Jan 29, 2019 at 7:58 PM Dave Lee via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>> Hi,
>>
>> just wondering if its possible to manually specify the parallel
>> decomposition of data within PETSc Vecs and Mats? And if so, will the
>> FieldSplit preconditioning handle this also?
>>
>> I have an application (non-PETSc) data layout as:
>> DATA[16][4][num_procs][3472]
>>
>> and if possible I would like to use FieldSplit preconditioning without
>> re-arranging this layout.
>>
>
> Yes. You need to call
>
>
> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCFieldSplitSetIS.html
>
> in order to define the splits. Note that if you have nested splits, you
> will have to also call it on the
> nested PC, which stinks, but we have no other way of getting the
> information.
>
> Nested splits can also be handled by implementing the DM interface, but
> that is a lot of work. It is,
> however, the way we handle it internally because it makes things
> much easier and more flexible.
>
>   Thanks,
>
>      Matt
>
>
>> Currently I have a nested FieldSplit preconditioning with 16 outer field
>> splits, and a schur complement field split within each. however this throws
>> up an indexing error as FieldSplit assumes that the parallel decomposition
>> is contiguous (I think).
>>
>> [0]PETSC ERROR: #1 ISComplement() line 750 in
>> /home/dlee0035/soft/petsc-3.9.3/src/vec/is/is/utils/iscoloring.c
>>
>> [0]PETSC ERROR: #2 PCSetUp_FieldSplit() line 703 in
>> /home/dlee0035/soft/petsc-3.9.3/src/ksp/pc/impls/fieldsplit/fieldsplit.c
>>
>> [0]PETSC ERROR: #3 PCSetUp() line 923 in
>> /home/dlee0035/soft/petsc-3.9.3/src/ksp/pc/interface/precon.c
>>
>> Cheers, Dave.
>>
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190131/f4b8c1e1/attachment.html>