<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color:rgb(32, 31, 30);font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, system-ui, Roboto, "Helvetica Neue", sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255);display:inline !important">Pierre,</span><br>
</div>
<div style=""><font color="#201f1e"><span style="font-size: 14.6667px;">This is a bug in MatMatMultSymbolic_MPIAIJ_MPIDense() during optimization of block column size of B. Run your code with</span></font></div>
<div style=""><font color="#201f1e"><span style="font-size: 14.6667px;">'-matmatmult_Bbn 1', the infinite loop should not occur. I'll try to figure out a fix tomorrow.<br>
</span></font></div>
<div style=""><font color="#201f1e"><span style="font-size: 14.6667px;">Hong</span></font></div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Zhang, Hong <hzhang@mcs.anl.gov><br>
<b>Sent:</b> Sunday, February 28, 2021 11:05 PM<br>
<b>To:</b> Pierre Jolivet <pierre@joliv.et>; For users of the development version of PETSc <petsc-dev@mcs.anl.gov>; Zhang, Hong <hzhang@mcs.anl.gov><br>
<b>Subject:</b> Re: [petsc-dev] Infinite loop in A*B</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The infinite loop in <span style="background-color:rgb(255,255,255); display:inline!important">MatMatMultNumeric_MPIAIJ_MPIDense()</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
for (i=0; i<BN; i+=n) {<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
}</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="background-color:rgb(255,255,255); display:inline!important">is cause by n=contents->workB->cmap->n=0 (line 590 in mpimatmatmult.c)</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="background-color:rgb(255,255,255); display:inline!important">Hong</span></div>
<div id="x_appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> petsc-dev <petsc-dev-bounces@mcs.anl.gov> on behalf of Zhang, Hong via petsc-dev <petsc-dev@mcs.anl.gov><br>
<b>Sent:</b> Sunday, February 28, 2021 10:33 PM<br>
<b>To:</b> Pierre Jolivet <pierre@joliv.et>; For users of the development version of PETSc <petsc-dev@mcs.anl.gov><br>
<b>Subject:</b> Re: [petsc-dev] Infinite loop in A*B</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I can reproduce the hang with </div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
mpiexec -n 2 ./matmatmult</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
It seems in an infinite loop of calling MatDensePlaceArray() from </div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
#0 MatDensePlaceArray (mat=0xda5c50, array=0xd15e60)
<div> at /home/hongsu/soft/petsc/src/mat/impls/dense/mpi/mpidense.c:2047</div>
<div>#1 0x00007fa0d13bf4f7 in MatDenseGetSubMatrix_SeqDense (A=0xcfb2b0, cbegin=0,</div>
<div> cend=0, v=0xd90370)</div>
<div> at /home/hongsu/soft/petsc/src/mat/impls/dense/seq/dense.c:2997</div>
<div>#2 0x00007fa0d13c574e in MatDenseGetSubMatrix (A=0xcfb2b0, cbegin=0, cend=0,</div>
<div> v=0xd90370) at /home/hongsu/soft/petsc/src/mat/impls/dense/seq/dense.c:3371</div>
<div>#3 0x00007fa0d13db5ce in MatDenseGetSubMatrix_MPIDense (A=0xca5250, cbegin=0,</div>
<div> cend=0, v=0x7ffe87d41de0)</div>
<div> at /home/hongsu/soft/petsc/src/mat/impls/dense/mpi/mpidense.c:1835</div>
<div>#4 0x00007fa0d13c574e in MatDenseGetSubMatrix (A=0xca5250, cbegin=0, cend=0,</div>
<div> v=0x7ffe87d41de0)</div>
<div> at /home/hongsu/soft/petsc/src/mat/impls/dense/seq/dense.c:3371</div>
<div>#5 0x00007fa0d179c2fa in MatMatMultNumeric_MPIAIJ_MPIDense (A=0xc55490,</div>
<div> B=0xca5250, C=0xd282b0)</div>
<div> at /home/hongsu/soft/petsc/src/mat/impls/aij/mpi/mpimatmatmult.c:593</div>
<div>#6 0x00007fa0d1181331 in MatProductNumeric_AB (mat=0xd282b0)</div>
<div> at /home/hongsu/soft/petsc/src/mat/interface/matproduct.c:567</div>
<div>#7 0x00007fa0d1182c14 in MatProductNumeric (mat=0xd282b0)</div>
<div> at /home/hongsu/soft/petsc/src/mat/interface/matproduct.c:679</div>
<div>#8 0x00007fa0d115ef69 in MatProduct_Private (A=0xc55490, B=0xca5250,</div>
<div> scall=MAT_INITIAL_MATRIX, fill=-2, ptype=MATPRODUCT_AB, C=0x7ffe87d42018)</div>
<div> at /home/hongsu/soft/petsc/src/mat/interface/matrix.c:9405</div>
<div>---Type <return> to continue, or q <return> to quit---</div>
<div>#9 0x00007fa0d115f274 in MatMatMult (A=0xc55490, B=0xca5250, scall=MAT_INITIAL_MATRIX, fill=-2,</div>
<div> C=0x7ffe87d42018) at /home/hongsu/soft/petsc/src/mat/interface/matrix.c:9445</div>
#10 0x000000000040130a in main (argc=2, argv=0x7ffe87d42108) at ex1.c:20<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I'll try to figure out what is going on. If anyone has a clue, please help. The above stack comes from 'release' branch.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hong</div>
<div id="x_x_appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> petsc-dev <petsc-dev-bounces@mcs.anl.gov> on behalf of Pierre Jolivet <pierre@joliv.et><br>
<b>Sent:</b> Sunday, February 28, 2021 4:17 PM<br>
<b>To:</b> For users of the development version of PETSc <petsc-dev@mcs.anl.gov><br>
<b>Subject:</b> [petsc-dev] Infinite loop in A*B</font>
<div> </div>
</div>
<div class="x_x_BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="x_x_PlainText">Hello,<br>
The following MWE loops indefinitely for MPI_Comm_size in {2; 3}.<br>
Nothing fancy, just MatAIJ and MatDense.<br>
The problem is either in MatMPIDenseScatter() or MatMatMultSymbolic_MPIAIJ_MPIDense(), I believe, so if someone familiar with those routines can figure out a hot fix, I’m all ears.<br>
I could of course switch to a MatMult(), but the same infinite loop happens in another more complex code with<br>
A = rows=8, cols=35212<br>
B = rows=35212, cols=9<br>
So I’ll need a fix eventually.<br>
<br>
Thanks,<br>
Pierre<br>
<br>
</div>
</span></font></div>
</div>
</div>
</body>
</html>