[petsc-users] Confusion with MatGetLocalSubMatrix

[Jed Brown]

Lawrence Mitchell <lawrence.mitchell at imperial.ac.uk> writes:

> Dear petsc-users,
>
> I'm trying to setup matrices and data structures for use with MatGetLocalSubMatrix, but I'm rather lost in a world of ISes and block sizes.  I have the layouts and so forth correct where all my fields have block size 1, but am struggling to follow how things fit together for block size > 1.
>
> I have a Taylor-Hood discretisation so a P2 velocity space with block size 2, and a P1 pressure space with block size 1.
>
> On each process, I build the full local to global mapping for both fields.  This has block size 1.

How are you ordering the fields on each process?

> Then I create strided ISes to define the local blocks for each field, and set the block size on each of them (2 for the velocity space, 1 for the pressure).  Aside, when I do ISCreateStride for an IS with a block size > 1, do I provide all the indices, or just the blocked indices?  Should I be using ISCreateBlock for block size > 1 instead?

ISSTRIDE has no concept of block size and can't be used to describe
blocked index sets.  Use ISBLOCK instead.

> Calling MatGetLocalSubMatrix results in an error:
>
> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
> [0]PETSC ERROR: Petsc has generated inconsistent data
> [0]PETSC ERROR: Blocksize of localtoglobalmapping 1 must match that of layout 2

Hmm, I'm concerned that this might not work right after some recent
changes to ISLocalToGlobalMapping.  Can you reproduce this with some
code I can run?

> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
> [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-679-gfd874b2  GIT Date: 2014-10-14 23:50:11 -0500
> [0]PETSC ERROR: foo.py on a arch-darwin-c-dbg named yam-laptop.local by lmitche1 Fri Dec 19 14:50:37 2014
> [0]PETSC ERROR: Configure options --download-chaco=1 --download-ctetgen=1 --download-exodusii=1 --download-hypre=1 --download-metis=1 --download-ml --download-parmetis=1 --download-triangle=1 --with-c2html=0 --with-debugging=1 --with-hdf5-dir=/usr/local --with-hdf5=1 --with-netcdf-dir=/usr/local --with-netcdf=1 --with-shared-libraries=1 PETSC_ARCH=arch-darwin-c-dbg
> [0]PETSC ERROR: #1 PetscLayoutSetBlockSize() line 438 in /Users/lmitche1/Documents/work/mapdes/petsc/src/vec/is/utils/pmap.c
> [0]PETSC ERROR: #2 MatCreateLocalRef() line 259 in /Users/lmitche1/Documents/work/mapdes/petsc/src/mat/impls/localref/mlocalref.c
> [0]PETSC ERROR: #3 MatGetLocalSubMatrix() line 9480 in /Users/lmitche1/Documents/work/mapdes/petsc/src/mat/interface/matrix.c
>
> Should I therefore not set the block size on this IS?  This seems wrong to me, since it's the only way to forward that to the local created sub matrix, such that I can do matsetvaluesblockedlocal on it.  As an aside, I note that the local sub matrix only supports blocked insertion for square blocks, but I think matsetvaluesblocked now works for non-square blocks: should this check be relaxed appropriately?
>
>
> Cheers,
>
> Lawrence
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20141219/5af3c438/attachment.pgp>