<div dir="ltr"><div><div><div>Thank you Barry.<br></div>If I understand well, each process needs to provide the IS of the global number of each local row of the considered field. Right?<br></div>This is what I tried to code. I am gonna check my implementation.<br><br></div>Nicolas<br></div><div class="gmail_extra"><br><div class="gmail_quote">2016-12-02 18:34 GMT+01:00 Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Each process needs to provide the IS that contain only local entries for that process.<br>
<br>
It looks like you might be doing the opposite.<br>
<div><div class="h5"><br>
<br>
> On Dec 2, 2016, at 10:36 AM, Karin&NiKo <<a href="mailto:niko.karin@gmail.com">niko.karin@gmail.com</a>> wrote:<br>
><br>
> Dear all,<br>
><br>
> Thanks to Matt's help, I have been able to set up a fieldsplit preconditioner for a Stokes-like problem. But it was in sequential! Now I am facing new issues when trying to set up the saddle-point structure in parallel.<br>
><br>
> Well, I have a matrix with 38 DOF. In the global numbering, the pressure DOF are numbered : 2,5,8,11,14,17 and the velocity DOF are the others. The matrix is distributed on 2 procs, the rows 0 to 18 on proc0, the rows from 19 to 38 on procs1.<br>
> I have set the following IS in order to pass them to the PCFieldSplit :<br>
> call ISCreateGeneral(PETSC_COMM_<wbr>SELF, nbddl0, vec_ddl0, PETSC_COPY_VALUES, is0, ierr)<br>
> call ISCreateGeneral(PETSC_COMM_<wbr>SELF, nbddl1, vec_ddl1, PETSC_COPY_VALUES, is1, ierr)<br>
><br>
> This is what they contain :<br>
><br>
> is0 on proc0 :<br>
> -------------------<br>
> IS Object: 1 MPI processes<br>
> type: general<br>
> Number of indices in set 19<br>
> 0 19<br>
> 1 20<br>
> 2 21<br>
> 3 22<br>
> 4 23<br>
> 5 24<br>
> 6 25<br>
> 7 26<br>
> 8 27<br>
> 9 28<br>
> 10 29<br>
> 11 30<br>
> 12 31<br>
> 13 32<br>
> 14 33<br>
> 15 34<br>
> 16 35<br>
> 17 36<br>
> 18 37<br>
><br>
> is1 on proc0 :<br>
> -------------------<br>
> IS Object: 1 MPI processes<br>
> type: general<br>
> Number of indices in set 0<br>
><br>
> is0 on proc1 :<br>
> -------------------<br>
> IS Object: 1 MPI processes<br>
> type: general<br>
> Number of indices in set 13<br>
> 0 0<br>
> 1 1<br>
> 2 3<br>
> 3 4<br>
> 4 6<br>
> 5 7<br>
> 6 9<br>
> 7 10<br>
> 8 12<br>
> 9 13<br>
> 10 15<br>
> 11 16<br>
> 12 18<br>
><br>
> is1 on proc1 :<br>
> -------------------<br>
> IS Object: 1 MPI processes<br>
> type: general<br>
> Number of indices in set 6<br>
> 0 2<br>
> 1 5<br>
> 2 8<br>
> 3 11<br>
> 4 14<br>
> 5 17<br>
><br>
> Then I pass them to the FieldSplit :<br>
> call PCFieldSplitSetIS(pc,'0',is0, ierr)<br>
> call PCFieldSplitSetIS(pc,'1',is1, ierr)<br>
><br>
><br>
> But when the PC is set up, PETSc complains about :<br>
><br>
> [1]PETSC ERROR: --------------------- Error Message ------------------------------<wbr>------------------------------<wbr>--<br>
> [1]PETSC ERROR: Nonconforming object sizes<br>
> [1]PETSC ERROR: Local column sizes 32 do not add up to total number of columns 19<br>
> [1]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/<wbr>documentation/faq.html</a> for trouble shooting.<br>
> [1]PETSC ERROR: Petsc Release Version 3.7.2, Jun, 05, 2016<br>
> [1]PETSC ERROR: \C0\E3o on a arch-linux2-c-debug named dsp0780450 by B07947 Fri Dec 2 17:07:54 2016<br>
> [1]PETSC ERROR: Configure options --prefix=/home/B07947/dev/<wbr>codeaster-prerequisites/petsc-<wbr>3.7.2/Install --with-mpi=yes --with-x=yes --download-ml=/home/B07947/<wbr>dev/codeaster-prerequisites/<wbr>petsc-3.7.2/ml-6.2-p3.tar.gz --with-mumps-lib="-L/home/<wbr>B07947/dev/codeaster-<wbr>prerequisites/v13/<wbr>prerequisites/Mumps-502_<wbr>consortium_aster1/MPI/lib -lzmumps -ldmumps -lmumps_common -lpord -L/home/B07947/dev/codeaster-<wbr>prerequisites/v13/<wbr>prerequisites/Scotch_aster-<wbr>604_aster6/MPI/lib -lesmumps -lptscotch -lptscotcherr -lptscotcherrexit -lscotch -lscotcherr -lscotcherrexit -L/home/B07947/dev/codeaster-<wbr>prerequisites/v13/<wbr>prerequisites/Parmetis_aster-<wbr>403_aster/lib -lparmetis -L/home/B07947/dev/codeaster-<wbr>prerequisites/v13/<wbr>prerequisites/Metis_aster-510_<wbr>aster1/lib -lmetis -L/usr/lib -lscalapack-openmpi -L/usr/lib -lblacs-openmpi -lblacsCinit-openmpi -lblacsF77init-openmpi -L/usr/lib/x86_64-linux-gnu -lgomp " --with-mumps-include=/home/<wbr>B07947/dev/codeaster-<wbr>prerequisites/v13/<wbr>prerequisites/Mumps-502_<wbr>consortium_aster1/MPI/include --with-scalapack-lib="-L/usr/<wbr>lib -lscalapack-openmpi" --with-blacs-lib="-L/usr/lib -lblacs-openmpi -lblacsCinit-openmpi -lblacsF77init-openmpi" --with-blas-lib="-L/usr/lib -lopenblas -lcblas" --with-lapack-lib="-L/usr/lib -llapack"<br>
> [1]PETSC ERROR: #1 MatGetSubMatrix_MPIAIJ_<wbr>Private() line 3181 in /home/B07947/dev/codeaster-<wbr>prerequisites/petsc-3.7.2/src/<wbr>mat/impls/aij/mpi/mpiaij.c<br>
> [1]PETSC ERROR: #2 MatGetSubMatrix_MPIAIJ() line 3100 in /home/B07947/dev/codeaster-<wbr>prerequisites/petsc-3.7.2/src/<wbr>mat/impls/aij/mpi/mpiaij.c<br>
> [1]PETSC ERROR: #3 MatGetSubMatrix() line 7825 in /home/B07947/dev/codeaster-<wbr>prerequisites/petsc-3.7.2/src/<wbr>mat/interface/matrix.c<br>
> [1]PETSC ERROR: #4 PCSetUp_FieldSplit() line 560 in /home/B07947/dev/codeaster-<wbr>prerequisites/petsc-3.7.2/src/<wbr>ksp/pc/impls/fieldsplit/<wbr>fieldsplit.c<br>
> [1]PETSC ERROR: #5 PCSetUp() line 968 in /home/B07947/dev/codeaster-<wbr>prerequisites/petsc-3.7.2/src/<wbr>ksp/pc/interface/precon.c<br>
> [1]PETSC ERROR: #6 KSPSetUp() line 390 in /home/B07947/dev/codeaster-<wbr>prerequisites/petsc-3.7.2/src/<wbr>ksp/ksp/interface/itfunc.c<br>
><br>
><br>
> I am doing something wrong but I cannot see how I should specify the layout of my fields.<br>
><br>
> Thanks in advance,<br>
> Nicolas<br>
><br>
><br>
><br>
><br>
><br>
</div></div>> <image.png><br>
><br>
> <Matrix38.ascii><br>
<br>
</blockquote></div><br></div>