How did you create the MatNest? Did you use the same index sets?<br><br><div class="gmail_quote">On Wed, Feb 1, 2012 at 01:42, Klaij, Christiaan <span dir="ltr"><<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">>> Then what would be the best way to create the IS in this case?<br>
>> Can it somehow be deduced from the separate blocks?<br>
>><br>
><br>
>The ISs define the row space of the blocks inside the global matrix.<br>
<br>
Following ex28 I made two stride ISs to define the row space<br>
but I'm still having trouble, this is for 2 procs:<br>
<div class="im"><br>
Matrix object:<br>
type=nest, rows=2, cols=2<br>
MatNest structure:<br>
(0,0) : prefix="a00_", type=mpiaij, rows=24, cols=24<br>
(0,1) : prefix="a01_", type=mpiaij, rows=24, cols=12<br>
(1,0) : prefix="a10_", type=mpiaij, rows=12, cols=24<br>
(1,1) : prefix="a11_", type=mpiaij, rows=12, cols=12<br>
<br>
</div>[0] Index set is permutation<br>
[0] Number of indices in (stride) set 12<br>
[0] 0 0<br>
[0] 1 1<br>
[0] 2 2<br>
[0] 3 3<br>
[0] 4 4<br>
[0] 5 5<br>
[0] 6 6<br>
[0] 7 7<br>
[0] 8 8<br>
[0] 9 9<br>
[0] 10 10<br>
[0] 11 11<br>
[1] Number of indices in (stride) set 12<br>
[1] 0 12<br>
[1] 1 13<br>
[1] 2 14<br>
[1] 3 15<br>
[1] 4 16<br>
[1] 5 17<br>
[1] 6 18<br>
[1] 7 19<br>
[1] 8 20<br>
[1] 9 21<br>
[1] 10 22<br>
[1] 11 23<br></blockquote><div><br></div><div>This really can't be right. Modify ex28 to have it print out the index sets. If distributed evenly over two procs, we might expect the first index set to hold [0..11; 18..29] and the second to hold [12..17; 30..35]. The way you are addressing would force a non-contiguous row partition.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[0] Number of indices in (stride) set 6<br>
[0] 0 24<br>
[0] 1 25<br>
[0] 2 26<br>
[0] 3 27<br>
[0] 4 28<br>
[0] 5 29<br>
[1] Number of indices in (stride) set 6<br>
[1] 0 30<br>
[1] 1 31<br>
[1] 2 32<br>
[1] 3 33<br>
[1] 4 34<br>
[1] 5 35<br>
<div class="im"><br>
[0]PETSC ERROR: --------------------- Error Message ------------------------------------<br>
</div>[0]PETSC ERROR: Arguments are incompatible!<br>
[0]PETSC ERROR: Could not find index set!<br>
<div class="im">[0]PETSC ERROR: ------------------------------------------------------------------------<br>
[0]PETSC ERROR: Petsc Release Version 3.2.0, Patch 5, Sat Oct 29 13:45:54 CDT 2011<br>
[0]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>
[0]PETSC ERROR: See docs/index.html for manual pages.<br>
[0]PETSC ERROR: ------------------------------------------------------------------------<br>
</div>[0]PETSC ERROR: ./matnest-trycpp on a linux_64b named lin0133 by cklaij Wed Feb 1 08:33:40 2012<br>
<div class="im">[0]PETSC ERROR: Libraries linked from /opt/refresco/64bit_intelv11.1_openmpi/petsc-3.2-p5/lib<br>
[0]PETSC ERROR: Configure run at Thu Jan 26 13:44:12 2012<br>
[0]PETSC ERROR: Configure options --prefix=/opt/refresco/64bit_intelv11.1_openmpi/petsc-3.2-p5 --with-mpi-dir=/opt/refresco/64bit_intelv11.1_openmpi/openmpi-1.4.4 --with-x=1 --with-mpe=0 --with-debugging=1 --with-clanguage=c++ --with-hypre-include=/opt/refresco/64bit_intelv11.1_openmpi/hypre-2.7.0b/include --with-hypre-lib=/opt/refresco/64bit_intelv11.1_openmpi/hypre-2.7.0b/lib/libHYPRE.a --with-ml-include=/opt/refresco/64bit_intelv11.1_openmpi/ml-6.2/include --with-ml-lib=/opt/refresco/64bit_intelv11.1_openmpi/ml-6.2/lib/libml.a --with-blas-lapack-dir=/opt/intel/mkl<br>
[0]PETSC ERROR: ------------------------------------------------------------------------<br>
</div>[0]PETSC ERROR: MatNestFindIS() line 292 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/mat/impls/nest/matnest.c<br>
[0]PETSC ERROR: MatNestFindSubMat() line 350 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/mat/impls/nest/matnest.c<br>
[0]PETSC ERROR: MatGetSubMatrix_Nest() line 366 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/mat/impls/nest/matnest.c<br>
[0]PETSC ERROR: MatGetSubMatrix() line 7135 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/mat/interface/matrix.c<br>
[0]PETSC ERROR: PCSetUp_FieldSplit() line 377 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/ksp/pc/impls/fieldsplit/fieldsplit.c<br>
<div><div></div><div class="h5">[0]PETSC ERROR: PCSetUp() line 819 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/ksp/pc/interface/precon.c<br>
[0]PETSC ERROR: KSPSetUp() line 260 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/ksp/ksp/interface/itfunc.c<br>
[0]PETSC ERROR: KSPSolve() line 379 in /home/CKlaij/ReFRESCO/Libraries/build/petsc-3.2-p5/src/ksp/ksp/interface/itfunc.c<br>
<br>
<br>
dr. ir. Christiaan Klaij<br>
CFD Researcher<br>
Research & Development<br>
E mailto:<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a><br>
T <a href="tel:%2B31%20317%2049%2033%2044" value="+31317493344">+31 317 49 33 44</a><br>
<br>
MARIN<br>
2, Haagsteeg, P.O. Box 28, 6700 AA Wageningen, The Netherlands<br>
T <a href="tel:%2B31%20317%2049%2039%2011" value="+31317493911">+31 317 49 39 11</a>, F <a href="tel:%2B31%20317%2049%2032%2045" value="+31317493245">+31 317 49 32 45</a>, I <a href="http://www.marin.nl" target="_blank">www.marin.nl</a><br>
<br>
</div></div></blockquote></div><br>