<div dir="ltr"><div dir="ltr">On Tue, May 9, 2023 at 9:04 AM Jose E. Roman <<a href="mailto:jroman@dsic.upv.es">jroman@dsic.upv.es</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I found the bug: the event MAT_MultHermitianTranspose is used but not registered.<br>
I will create a MR.<br></blockquote><div><br></div><div>Great!</div><div><br></div><div> Thanks,</div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Thanks Matt.<br>
<br>
> El 9 may 2023, a las 14:50, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> escribió:<br>
> <br>
> On Tue, May 9, 2023 at 8:41 AM Jose E. Roman <<a href="mailto:jroman@dsic.upv.es" target="_blank">jroman@dsic.upv.es</a>> wrote:<br>
> But MatCreateShell() calls MatInitializePackage() (via MatCreate()) and also the main program creates a regular Mat. The events should have been registered by the time the shell matrix operations are invoked.<br>
> <br>
> The reason I say this is that PetscBarrier is the _first_ event, so if an event is called without initializing, it<br>
> will show up as PetscBarrier. Maybe break in MatMultTranspose, to see who is calling it first?<br>
> <br>
> Thanks,<br>
> <br>
> Matt<br>
> <br>
> > El 9 may 2023, a las 14:13, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> escribió:<br>
> > <br>
> > On Tue, May 9, 2023 at 7:17 AM Jose E. Roman <<a href="mailto:jroman@dsic.upv.es" target="_blank">jroman@dsic.upv.es</a>> wrote:<br>
> > Hi.<br>
> > <br>
> > We are seeing a strange thing in the -log_view output with one of the SLEPc solvers. It is probably an issue with SLEPc, but we don't know how to debug it.<br>
> > <br>
> > It can be reproduced for instance with<br>
> > <br>
> > $ ./ex45 -m 15 -n 20 -p 21 -svd_nsv 4 -svd_ncv 9 -log_view<br>
> > <br>
> > The log_view events are listed at the end of this email. The first one (PetscBarrier) is wrong, because PetscBarrier is never called, if I place a breakpoint in PetscBarrier() it will never be hit. Also, in that event it reports some nonzero Mflop/s, which suggests that it corresponds to another event (not PetscBarrier). Furthermore, the count of the PetscBarrier event always matches the count of MatMultTranspose, so there must be a connection.<br>
> > <br>
> > Does anyone have suggestions how to address this?<br>
> > <br>
> > Hi Jose,<br>
> > <br>
> > Here is my guess. PETSc sets all of the event ids (using Register) when the dynamic libraries get loaded. If they are not loaded,<br>
> > then the library initialization function is called when some function from that library is used. My guess is that we put this init check<br>
> > in MatCreate(), but that is not called when you create your shell matrix and thus the events are not initialized correctly for you until<br>
> > later. Can you check?<br>
> > <br>
> > Thanks,<br>
> > <br>
> > Matt<br>
> > <br>
> > Note: this is with 1 MPI process.<br>
> > Note: the solver creates a shell matrix with MATOP_MULT_TRANSPOSE.<br>
> > <br>
> > Thanks.<br>
> > Jose<br>
> > <br>
> > <br>
> > PetscBarrier 16 1.0 7.5579e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6<br>
> > MatMult 82 1.0 6.1590e-01 1.0 9.25e+05 1.0 0.0e+00 0.0e+00 0.0e+00 81 97 0 0 0 81 97 0 0 0 2<br>
> > MatMultTranspose 16 1.0 7.4625e-04 1.0 4.38e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6<br>
> > MatAssemblyBegin 4 1.0 5.2452e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > MatAssemblyEnd 4 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > MatTranspose 2 1.0 1.6265e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > SVDSetUp 1 1.0 1.8686e-02 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0<br>
> > SVDSolve 1 1.0 5.5965e-01 1.0 7.95e+05 1.0 0.0e+00 0.0e+00 0.0e+00 74 83 0 0 0 74 83 0 0 0 1<br>
> > EPSSetUp 1 1.0 9.5146e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0<br>
> > EPSSolve 1 1.0 5.4082e-01 1.0 7.94e+05 1.0 0.0e+00 0.0e+00 0.0e+00 71 83 0 0 0 71 83 0 0 0 1<br>
> > STSetUp 1 1.0 4.8406e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0<br>
> > STComputeOperatr 1 1.0 1.2653e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > STApply 24 1.0 6.1569e-01 1.0 8.84e+05 1.0 0.0e+00 0.0e+00 0.0e+00 81 92 0 0 0 81 92 0 0 0 1<br>
> > STMatSolve 24 1.0 6.1556e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 0.0e+00 81 92 0 0 0 81 92 0 0 0 1<br>
> > KSPSetUp 1 1.0 2.8465e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > KSPSolve 24 1.0 6.1551e-01 1.0 8.80e+05 1.0 0.0e+00 0.0e+00 0.0e+00 81 92 0 0 0 81 92 0 0 0 1<br>
> > KSPGMRESOrthog 480 1.0 4.3219e-01 1.0 3.98e+05 1.0 0.0e+00 0.0e+00 0.0e+00 57 42 0 0 0 57 42 0 0 0 1<br>
> > PCSetUp 1 1.0 3.8147e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > PCApply 504 1.0 2.1615e-02 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 0.0e+00 3 1 0 0 0 3 1 0 0 0 0<br>
> > BVCopy 27 1.0 3.0560e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > BVMultVec 38 1.0 3.7060e-03 1.0 8.88e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 2<br>
> > BVMultInPlace 3 1.0 1.0681e-04 1.0 4.32e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 40<br>
> > BVDotVec 38 1.0 4.5121e-03 1.0 4.38e+04 1.0 0.0e+00 0.0e+00 0.0e+00 1 5 0 0 0 1 5 0 0 0 10<br>
> > BVOrthogonalizeV 20 1.0 1.3182e-02 1.0 5.33e+04 1.0 0.0e+00 0.0e+00 0.0e+00 2 6 0 0 0 2 6 0 0 0 4<br>
> > BVScale 24 1.0 4.8089e-04 1.0 4.80e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1<br>
> > BVNormVec 4 1.0 5.4097e-04 1.0 3.67e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 7<br>
> > BVNormalize 1 1.0 1.3154e-03 1.0 3.75e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 3<br>
> > BVSetRandom 1 1.0 9.1791e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > BVMatMultVec 19 1.0 4.6828e-01 1.0 6.99e+05 1.0 0.0e+00 0.0e+00 0.0e+00 62 73 0 0 0 62 73 0 0 0 1<br>
> > DSSolve 3 1.0 8.8906e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > DSVectors 7 1.0 2.8920e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > DSOther 12 1.0 3.8576e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > VecDot 4 1.0 8.7500e-05 1.0 1.56e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2<br>
> > VecMDot 488 1.0 6.0463e-04 1.0 1.98e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 21 0 0 0 0 21 0 0 0 327<br>
> > VecNorm 520 1.0 1.0412e-02 1.0 2.02e+04 1.0 0.0e+00 0.0e+00 0.0e+00 1 2 0 0 0 1 2 0 0 0 2<br>
> > VecScale 514 1.0 1.0245e-02 1.0 1.03e+04 1.0 0.0e+00 0.0e+00 0.0e+00 1 1 0 0 0 1 1 0 0 0 1<br>
> > VecCopy 62 1.0 1.8365e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > VecSet 64 1.0 4.2200e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > VecAXPY 32 1.0 4.8995e-04 1.0 1.44e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 3<br>
> > VecMAXPY 504 1.0 6.9070e-04 1.0 2.21e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 23 0 0 0 0 23 0 0 0 320<br>
> > VecPointwiseMult 504 1.0 3.3593e-04 1.0 1.01e+04 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 30<br>
> > VecSetRandom 1 1.0 5.0068e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> > VecNormalize 512 1.0 4.1106e-02 1.0 3.01e+04 1.0 0.0e+00 0.0e+00 0.0e+00 5 3 0 0 0 5 3 0 0 0 1<br>
> > <br>
> > <br>
> > <br>
> > -- <br>
> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
> > -- Norbert Wiener<br>
> > <br>
> > <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
> <br>
> <br>
> <br>
> -- <br>
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
> -- Norbert Wiener<br>
> <br>
> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
<br>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>