[petsc-users] Vecs and Mats with non-contiguous parallel layout

[Matthew Knepley]

On Tue, Jan 29, 2019 at 7:58 PM Dave Lee via petsc-users <
petsc-users at mcs.anl.gov> wrote:

> Hi,
>
> just wondering if its possible to manually specify the parallel
> decomposition of data within PETSc Vecs and Mats? And if so, will the
> FieldSplit preconditioning handle this also?
>
> I have an application (non-PETSc) data layout as:
> DATA[16][4][num_procs][3472]
>
> and if possible I would like to use FieldSplit preconditioning without
> re-arranging this layout.
>

Yes. You need to call


https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCFieldSplitSetIS.html

in order to define the splits. Note that if you have nested splits, you
will have to also call it on the
nested PC, which stinks, but we have no other way of getting the
information.

Nested splits can also be handled by implementing the DM interface, but
that is a lot of work. It is,
however, the way we handle it internally because it makes things
much easier and more flexible.

  Thanks,

     Matt


> Currently I have a nested FieldSplit preconditioning with 16 outer field
> splits, and a schur complement field split within each. however this throws
> up an indexing error as FieldSplit assumes that the parallel decomposition
> is contiguous (I think).
>
> [0]PETSC ERROR: #1 ISComplement() line 750 in
> /home/dlee0035/soft/petsc-3.9.3/src/vec/is/is/utils/iscoloring.c
>
> [0]PETSC ERROR: #2 PCSetUp_FieldSplit() line 703 in
> /home/dlee0035/soft/petsc-3.9.3/src/ksp/pc/impls/fieldsplit/fieldsplit.c
>
> [0]PETSC ERROR: #3 PCSetUp() line 923 in
> /home/dlee0035/soft/petsc-3.9.3/src/ksp/pc/interface/precon.c
>
> Cheers, Dave.
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190130/790ed4c8/attachment.html>