<div dir="ltr">Hi, Chris,<br>  First, I had to fix an error in your test by adding " PetscCallA(MatSetFromOptions(AA,ierr))" at line 254.<br><blockquote style="margin:0 0 0 40px;border:none;padding:0px">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[0]PETSC ERROR: Object is in wrong state<br>[0]PETSC ERROR: Mat object's type is not set: Argument # 1<br>...<br>[0]PETSC ERROR: #1 MatSetValues() at /scratch/jczhang/petsc/src/mat/interface/matrix.c:1503<br>[0]PETSC ERROR: #2 ex2f.F90:258</blockquote><div><span style="color:rgb(80,0,80)"><br></span></div>Then I could ran the test without problems<br><blockquote style="margin:0 0 0 40px;border:none;padding:0px">mpirun  -n 2 ./ex2f -pc_type jacobi -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always<br>  0 KSP Residual norm 1.11803<br>  1 KSP Residual norm 0.591608<br>  2 KSP Residual norm 0.316228<br>  3 KSP Residual norm < 1.e-11<br>  0 KSP Residual norm 0.707107<br>  1 KSP Residual norm 0.408248<br>  2 KSP Residual norm < 1.e-11<br>Norm of error < 1.e-12 iterations     3</blockquote><div><span style="color:rgb(80,0,80)"><br></span></div>I used petsc-3.22.4, gcc-11.3, openmpi-5.0.6 and configured with  <br><blockquote style="margin:0 0 0 40px;border:none;padding:0px">./configure --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-openmpi --with-ssl=0 --with-shared-libraries=1 CFLAGS="-std=gnu11 -Wall -funroll-all-loops  -O3 -DNDEBUG" CXXFLAGS="-std=gnu++14 -Wall -funroll-all-loops  -O3 -DNDEBUG " COPTFLAGS="-std=gnu11 -Wall -funroll-all-loops  -O3 -DNDEBUG" CXXOPTFLAGS="-std=gnu++14 -Wall -funroll-all-loops  -O3 -DNDEBUG " FCFLAGS="-Wall -funroll-all-loops -ffree-line-length-0 -Wno-maybe-uninitialized -Wno-target-lifetime -Wno-unused-function -O3 -DNDEBUG" F90FLAGS="-Wall -funroll-all-loops -ffree-line-length-0 -Wno-maybe-uninitialized -Wno-target-lifetime -Wno-unused-function -O3 -DNDEBUG" FOPTFLAGS="-Wall -funroll-all-loops -ffree-line-length-0 -Wno-maybe-uninitialized -Wno-target-lifetime -Wno-unused-function -O3 -DNDEBUG"</blockquote><div><span style="color:rgb(80,0,80)"><br></span></div>Could you fix the error and retry?<div><br><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jul 6, 2025 at 12:57 PM Klaij, Christiaan via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div>Attached is a standalone example of the issue described in the<br>earlier thread "problem with nested logging".  The issue appeared<br>somewhere between petsc 3.19.4 and 3.23.4.<br><br>The example is a variation of ../ksp/tutorials/ex2f.F90, where<br>I've added the nested log viewer with one event as well as the<br>solution of a small system on rank zero.<br><br>When running on mulitple procs the example hangs during<br>PetscLogView with the backtrace below. The configure.log is also<br>attached in the hope that you can replicate the issue.<br><br>Chris<br><br><br>#0  0x000015554c84ea9e in mca_pml_ucx_recv (buf=0x7fffffff9e30, count=1, <br>    datatype=0x15554c9ef900 <ompi_mpi_2dblprec>, src=1, tag=-12, <br>    comm=0x7f1e30, mpi_status=0x0) at pml_ucx.c:700<br>#1  0x000015554c65baff in ompi_coll_base_allreduce_intra_recursivedoubling (<br>    sbuf=0x7fffffff9e20, rbuf=0x7fffffff9e30, count=1, <br>    dtype=0x15554c9ef900 <ompi_mpi_2dblprec>, <br>    op=0x15554ca28980 <ompi_mpi_op_maxloc>, comm=0x7f1e30, module=0xaec630)<br>    at base/coll_base_allreduce.c:247<br>#2  0x000015554c6a7e40 in ompi_coll_tuned_allreduce_intra_do_this (<br>    sbuf=0x7fffffff9e20, rbuf=0x7fffffff9e30, count=1, <br>    dtype=0x15554c9ef900 <ompi_mpi_2dblprec>, <br>    op=0x15554ca28980 <ompi_mpi_op_maxloc>, comm=0x7f1e30, module=0xaec630, <br>    algorithm=3, faninout=0, segsize=0) at coll_tuned_allreduce_decision.c:142<br>#3  0x000015554c6a054f in ompi_coll_tuned_allreduce_intra_dec_fixed (<br>    sbuf=0x7fffffff9e20, rbuf=0x7fffffff9e30, count=1, <br>    dtype=0x15554c9ef900 <ompi_mpi_2dblprec>, <br>    op=0x15554ca28980 <ompi_mpi_op_maxloc>, comm=0x7f1e30, module=0xaec630)<br>    at coll_tuned_decision_fixed.c:216<br>#4  0x000015554c68e160 in mca_coll_hcoll_allreduce (sbuf=0x7fffffff9e20, <br>    rbuf=0x7fffffff9e30, count=1, dtype=0x15554c9ef900 <ompi_mpi_2dblprec>, <br>    op=0x15554ca28980 <ompi_mpi_op_maxloc>, comm=0x7f1e30, module=0xaecb80)<br>    at coll_hcoll_ops.c:217<br>#5  0x000015554c59811a in PMPI_Allreduce (sendbuf=0x7fffffff9e20, <br>    recvbuf=0x7fffffff9e30, count=1, datatype=0x15554c9ef900 <ompi_mpi_2dblprec>, op=0x15554ca28980 <ompi_mpi_op_maxloc>, comm=0x7f1e30) at allreduce.c:123<br>#6  0x0000155553eabede in MPIU_Allreduce_Private () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#7  0x0000155553e50d08 in PetscPrintXMLNestedLinePerfResults () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#8  0x0000155553e5123e in PetscLogNestedTreePrintLine () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#9  0x0000155553e51f3a in PetscLogNestedTreePrint () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#10 0x0000155553e51e96 in PetscLogNestedTreePrint () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#11 0x0000155553e51e96 in PetscLogNestedTreePrint () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#12 0x0000155553e52142 in PetscLogNestedTreePrintTop () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#13 0x0000155553e5257b in PetscLogHandlerView_Nested_XML () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#14 0x0000155553e4e5a0 in PetscLogHandlerView_Nested () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#15 0x0000155553e56232 in PetscLogHandlerView () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#16 0x0000155553e588c3 in PetscLogView () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#17 0x0000155553e40eb5 in petsclogview_ () from /home/cklaij/ReFRESCO/trunk/install/extLibs/lib/libpetsc.so.3.22<br>#18 0x0000000000402c8b in MAIN__ ()<br>#19 0x00000000004023df in main ()<br></div><div dir="ltr" style="font-size:1px;direction:ltr"><table dir="ltr" cellpadding="0" cellspacing="0" border="0" style="width:100%;direction:ltr;border-collapse:collapse;font-size:1px"><tbody><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px;line-height:normal"><tbody><tr style="font-size:0px"><td align="left" style="padding:10px 0px;vertical-align:top"><img src="cid:ii_197ebccaa1d27ee6ef21" width="125" height="40" border="0" alt="" style="width: 125px; min-width: 125px; max-width: 125px; height: 40px; min-height: 40px; max-height: 40px; font-size: 0px;"></td></tr></tbody></table></td></tr><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px"><tbody><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px;color:rgb(0,0,1);font-style:normal;font-weight:400;white-space:nowrap"><tbody><tr style="font-size:14.67px"><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif">dr. ir. </td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif">Christiaan</td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif"> Klaij</td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif"> | </td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif">senior researcher</td><td align="left" style="vertical-align:top;font-size:0px"></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px;color:rgb(0,0,1);font-style:normal;font-weight:400;white-space:nowrap"><tbody><tr style="font-size:14.67px"><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif">Research & Development</td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif"> | </td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif">CFD Development</td></tr></tbody></table></td></tr><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px"><tbody><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px;color:rgb(0,0,1);font-style:normal;font-weight:400;white-space:nowrap"><tbody><tr style="font-size:14.67px"><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif">T <a href="tel:+31%20317%2049%2033%2044" id="m_8650041588025716439m_-4903205649372849687LPlnk689713" style="text-decoration:none;color:rgb(0,0,1)" target="_blank">+31 317 49 33 44</a></td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif"> | </td><td align="left" style="vertical-align:top;font-family:Calibri,Arial,sans-serif"><a href="https://urldefense.us/v3/__https://www.marin.nl/__;!!G_uCfscf7eWS!dAFNrWR8FzE9RrQXQAlok1iR_fA-rZdm9JAi-dlnKTnbdNTOTCViw0Nc-jjU4g72I-mhE1x1MZaf8imk4ivm_tE$" id="m_8650041588025716439m_-4903205649372849687LPlnk689713" style="text-decoration:none;color:rgb(0,0,1)" target="_blank">www.marin.nl</a></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px"><tbody><tr style="font-size:0px"><td align="left" style="padding:5px 0px 0px;vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px"><tbody><tr style="font-size:0px"><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px;line-height:normal"><tbody><tr style="font-size:0px"><td align="left" style="padding:0px 3px 3px 0px;vertical-align:top"><a href="https://urldefense.us/v3/__https://www.facebook.com/marin.wageningen__;!!G_uCfscf7eWS!dAFNrWR8FzE9RrQXQAlok1iR_fA-rZdm9JAi-dlnKTnbdNTOTCViw0Nc-jjU4g72I-mhE1x1MZaf8imkLNCvsiI$" id="m_8650041588025716439m_-4903205649372849687LPlnk689713" style="text-decoration:none" target="_blank"><img src="cid:ii_197ebccaa1dc01308f62" width="15" height="15" border="0" title="Facebook" alt="Facebook" style="width: 15px; min-width: 15px; max-width: 15px; height: 15px; min-height: 15px; max-height: 15px; font-size: 12px;"></a></td></tr></tbody></table></td><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px;line-height:normal"><tbody><tr style="font-size:0px"><td align="left" style="padding:0px 3px 3px 0px;vertical-align:top"><a href="https://urldefense.us/v3/__https://www.linkedin.com/company/marin__;!!G_uCfscf7eWS!dAFNrWR8FzE9RrQXQAlok1iR_fA-rZdm9JAi-dlnKTnbdNTOTCViw0Nc-jjU4g72I-mhE1x1MZaf8imkrb79Ay4$" id="m_8650041588025716439m_-4903205649372849687LPlnk689713" style="text-decoration:none" target="_blank"><img src="cid:ii_197ebccaa1d4e4c65123" width="15" height="15" border="0" title="LinkedIn" alt="LinkedIn" style="width: 15px; min-width: 15px; max-width: 15px; height: 15px; min-height: 15px; max-height: 15px; font-size: 12px;"></a></td></tr></tbody></table></td><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="border-collapse:collapse;font-size:0px;line-height:normal"><tbody><tr style="font-size:0px"><td align="left" style="padding:0px 3px 3px 0px;vertical-align:top"><a href="https://urldefense.us/v3/__https://www.youtube.com/marinmultimedia__;!!G_uCfscf7eWS!dAFNrWR8FzE9RrQXQAlok1iR_fA-rZdm9JAi-dlnKTnbdNTOTCViw0Nc-jjU4g72I-mhE1x1MZaf8imkJiCoeLw$" id="m_8650041588025716439m_-4903205649372849687LPlnk689713" style="text-decoration:none" target="_blank"><img src="cid:ii_197ebccaa1db54640254" width="15" height="15" border="0" title="YouTube" alt="YouTube" style="width: 15px; min-width: 15px; max-width: 15px; height: 15px; min-height: 15px; max-height: 15px; font-size: 12px;"></a></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table><span style="font-family:remialcxesans"> <span style="font-family:template-zjzHWwipEfCqpwAiSIGong"> </span><span style="font-family:zone-1"> </span><span style="font-family:zones-AQ"> </span></span></div></div></blockquote></div>