<div dir="ltr"><div dir="ltr">On Wed, Mar 17, 2021 at 3:27 PM Zhang, Hong via petsc-dev <<a href="mailto:petsc-dev@mcs.anl.gov">petsc-dev@mcs.anl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="color:rgb(32,31,30);font-size:15px;background-color:rgb(255,255,255);display:inline">Pierre,</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="color:rgb(32,31,30);font-size:15px;background-color:rgb(255,255,255);display:inline">Do you mean a
possible bug in C=<span style="background-color:rgb(255,255,255);display:inline">AtB <span style="background-color:rgb(255,255,255);display:inline">MatTransposeMatMult()?</span></span></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="color:rgb(32,31,30);font-size:15px;background-color:rgb(255,255,255);display:inline"><span style="background-color:rgb(255,255,255);display:inline"><span style="background-color:rgb(255,255,255);display:inline">Can
you provide a stand-alone test without hpddm that reproduces this error? </span></span></span></div></div></blockquote><div><br></div><div>Hong, you should be able to just configure with --download-hpddm and then run that ex76 test.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="color:rgb(32,31,30);font-size:15px;background-color:rgb(255,255,255);display:inline"><span style="background-color:rgb(255,255,255);display:inline"><span style="background-color:rgb(255,255,255);display:inline">Hong</span></span></span></div>
<div id="gmail-m_-5972319960547956757appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-5972319960547956757divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> petsc-dev <<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>> on behalf of Pierre Jolivet <<a href="mailto:pierre@joliv.et" target="_blank">pierre@joliv.et</a>><br>
<b>Sent:</b> Wednesday, March 17, 2021 4:31 AM<br>
<b>To:</b> For users of the development version of PETSc <<a href="mailto:petsc-dev@mcs.anl.gov" target="_blank">petsc-dev@mcs.anl.gov</a>><br>
<b>Subject:</b> [petsc-dev] MatTransposeMatMult() bug</font>
<div> </div>
</div>
<div style="overflow-wrap: break-word;">Hello,
<div>While trying out Stefano’s PCApplyMat_MG() code (*), we stumbled upon weird numerical errors when reusing a Mat for both MatProduct_AB and MatProduct_AtB.</div>
<div>This reminded me that there has been a long-standing issue with MatTransposeMatMult(), see <a href="https://www.mcs.anl.gov/petsc/petsc-current/src/ksp/pc/impls/hpddm/hpddm.cxx.html#line608" target="_blank">https://www.mcs.anl.gov/petsc/petsc-current/src/ksp/pc/impls/hpddm/hpddm.cxx.html#line608</a>,
that I never looked into.</div>
<div>I’ve now been trying to figure this out, because this has side effects in multiple places (PCMG and PCHPDDM at least), and thus could impact user-code as well?</div>
<div>With this commit: <a href="https://gitlab.com/petsc/petsc/-/commit/03d8bd538039defc2fcc3e37d523735c4aaceba0" target="_blank">https://gitlab.com/petsc/petsc/-/commit/03d8bd538039defc2fcc3e37d523735c4aaceba0</a></div>
<div>+</div>
<div>$ mpirun -n 4 src/ksp/ksp/tutorials/ex76 -ksp_converged_reason -pc_type hpddm -pc_hpddm_levels_1_eps_nev 20 -ksp_type preonly -mat_type aij -load_dir ${DATAFILESPATH}/matrices/hpddm/GENEO -rhs 2 -pc_hpddm_coarse_correction balanced -C_input_mattransposematmult
-D_output_mattransposematmult</div>
<div>I’m seeing that C is nonzero, but D is full of zeros.</div>
<div>
<div>Mat Object: 4 MPI processes</div>
<div> type: mpidense</div>
<div>5.7098316584361917e-08 1.0159399260517841e-07</div>
<div>1.5812349976211856e-07 2.0688121715350138e-07</div>
<div>2.4887556933361981e-08 4.8111092300772958e-08</div>
<div>1.4606298643602107e-07 1.7213611729839211e-07</div>
</div>
<div>[…]</div>
<div>
<div>Mat Object: 4 MPI processes</div>
<div> type: mpidense</div>
<div>0.0000000000000000e+00 0.0000000000000000e+00</div>
<div>0.0000000000000000e+00 0.0000000000000000e+00</div>
<div>0.0000000000000000e+00 0.0000000000000000e+00</div>
<div>0.0000000000000000e+00 0.0000000000000000e+00</div>
</div>
<div>[…]</div>
<div><br>
</div>
<div>If one switches to a MatType which has no MatProduct_AtB implementation with B of type MPIDense (reminder: in that case, the product is computed column-by-column), e.g., -mat_type sbaij, one gets the expected result.</div>
<div>
<div>Mat Object: 4 MPI processes</div>
<div> type: mpidense</div>
<div>7.2003197398135299e-01 9.5191869895699011e-01</div>
<div>6.1793966541680234e-02 9.3884397585488877e-01</div>
<div>1.0022337823233585e-02 2.4653068080134588e-01</div>
<div>1.4463931936094099e-01 8.6111517670701687e-01</div>
</div>
<div><br>
</div>
<div>Is there a bug somewhere with the MatAIJ implementation, or am I doing something which is not allowed by the MatProduct() machinery?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Pierre</div>
<div><br>
</div>
<div>(*) <a href="https://gitlab.com/petsc/petsc/-/merge_requests/3717" target="_blank">https://gitlab.com/petsc/petsc/-/merge_requests/3717</a> </div>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>