<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Stefano,<div class="">If we go back to your original question, then you are correct under the additional assumption that main returns 0.</div><div class="">Here is a MWE.</div><div class=""><br class=""></div><div class=""><div class="">$ git diff --unified=1 src/ksp/ksp/tutorials/ex1.c</div><div class="">diff --git a/src/ksp/ksp/tutorials/ex1.c b/src/ksp/ksp/tutorials/ex1.c</div><div class="">index 3b3e776a6d..a5ed63ca18 100644</div><div class="">--- a/src/ksp/ksp/tutorials/ex1.c</div><div class="">+++ b/src/ksp/ksp/tutorials/ex1.c</div><div class="">@@ -33,2 +33,3 @@ int main(int argc,char **args)</div><div class=""> ierr = PetscInitialize(&argc,&args,(char*)0,help);if (ierr) return ierr;</div><div class="">+ ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(0); // not ierr, 0</div><div class=""> ierr = MPI_Comm_size(PETSC_COMM_WORLD,&size);CHKERRQ(ierr);</div><div class=""><div class="">$ make -f gmakefile.test test search='ksp_ksp_tutorials-ex1_1'</div><div class="">Using MAKEFLAGS: search=ksp_ksp_tutorials-ex1_1</div><div class=""> TEST arch-darwin-c-debug-real/tests/counts/ksp_ksp_tutorials-ex1_1.counts</div><div class=""> ok ksp_ksp_tutorials-ex1_1</div><div class=""> ok diff-ksp_ksp_tutorials-ex1_1</div></div><div class="">$ echo $?</div><div class="">0</div><div class="">$ mpirun -n 1 arch-darwin-c-debug-real/tests/ksp/ksp/tutorials/ex1 -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always</div><div class="">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</div><div class="">[0]PETSC ERROR: Invalid argument</div><div class="">[0]PETSC ERROR: Wrong type of object: Parameter # 1</div><div class="">[0]PETSC ERROR: See <a href="https://www.mcs.anl.gov/petsc/documentation/faq.html" class="">https://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</div><div class="">[…]</div><div class=""><br class=""></div><div class="">I’m not sure I’m OK with this (shouldn’t both stderr and stdout be compared, and if there is no stderr to compare against, check that it’s null?), but I guess this is how testing works.</div><div class="">About KSPHPDDM, I’m currently implementing the necessary steps as Matt suggested to go from PetscInt (MatMatMult) -> int (my stuff) -> PetscInt (KSPSolve_HPDDM) to have proper error checking so that in fact main does not return 0. Anyway, as you know since our last iteration the error w.r.t. MatDensePlaceArray is fixed now in your branch.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Pierre</div><div><br class=""><blockquote type="cite" class=""><div class="">On 13 May 2020, at 6:15 PM, Matthew Knepley <<a href="mailto:knepley@gmail.com" class="">knepley@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div dir="ltr" class="">On Wed, May 13, 2020 at 11:33 AM Pierre Jolivet <<a href="mailto:pierre.jolivet@enseeiht.fr" class="">pierre.jolivet@enseeiht.fr</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" class="">I’m only CHKERRQ’ing inside the routine but I’m not propagating this down to the return code of KSPSolve.<div class="">I will try to fix this, or maybe switch to CHKERRABORT instead, because it currently goes like KSPSolve (PETSc) -> my stuff (non PETSc) -> MatMatMult or PCApply (PETSc), and I’m not sure I want to put some more PETSc code inside “my stuff” as it can be a pain to maintain for people that uses “my stuff” without PETSc. Sadly, they do exist.<div class="">Also, I’m using PetscFunction[Begin|End]User, don’t know if switching to PetscFunction[Begin|End] would change something here, though I’m guessing it wouldn't.</div></div></div></blockquote><div class=""><br class=""></div><div class="">No, those work the same.</div><div class=""><br class=""></div><div class="">I think you can propagate just with an integer return code, nothing PETSc dependent.</div><div class=""><br class=""></div><div class=""> Thanks,</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" class=""><div class=""><div class="">I’ll keep you posted, thanks for reporting this,</div><div class="">Pierre<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 13 May 2020, at 3:03 PM, Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank" class="">stefano.zampini@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class="">The error is raised from HPDDM , that does not stop and I think does not return error codes. Pierre?</div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno mer 13 mag 2020 alle ore 16:01 Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>> ha scritto:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class="">I see my stderr with tests. I am running it right now.<div class=""><br class=""></div><div class=""> Matt</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 13, 2020 at 8:55 AM Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank" class="">stefano.zampini@gmail.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class="">I have a test that fails, but it only prints a small amount of information<div class=""><br class=""></div><div class="">[szampini@localhost petsc]$ make -f gmakefile.test test search='ksp%' searchin='ex75_2_icc' PETSC_ARCH=arch-opt<br class="">Using MAKEFLAGS: -- PETSC_ARCH=arch-opt searchin=ex75_2_icc search=ksp%<br class=""> <span class="Apple-converted-space"> </span>TEST arch-opt/tests/counts/ksp_ksp_tutorials-ex75_2_icc.counts<br class=""> ok ksp_ksp_tutorials-ex75_2_icc+reset-false<br class="">not ok diff-ksp_ksp_tutorials-ex75_2_icc+reset-false # Error code: 1<br class=""># 3c3<br class=""># < Linear solve converged due to CONVERGED_RTOL iterations 34<br class=""># ---<br class=""># > Linear solve did not converge due to DIVERGED_ITS iterations 1000<br class=""> ok ksp_ksp_tutorials-ex75_2_icc+reset-true<br class=""> ok diff-ksp_ksp_tutorials-ex75_2_icc+reset-true<br class=""><div class=""><br class=""></div><div class="">If I run the test that fails myself, I get PETSC_ERROR messages</div><div class=""><br class=""></div><div class="">[szampini@localhost tutorials]$ ./ex75 -nmat 3 -pc_type icc -ksp_converged_reason -ksp_type hpddm -ksp_max_it 1000 -ksp_gmres_restart 40 -ksp_rtol 1e-10 -ksp_hpddm_type gcrodr -ksp_hpddm_recycle 20 -reset false -load_dir ${DATAFILESPATH}/matrices/hpddm/GCRODR<br class="">Linear solve converged due to CONVERGED_RTOL iterations 94<br class="">Linear solve converged due to CONVERGED_RTOL iterations 37<br class="">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br class="">[0]PETSC ERROR: Operation done in wrong order<br class="">[0]PETSC ERROR: Need to call MatDenseRestoreArray first<br class="">[0]PETSC ERROR: See<span class="Apple-converted-space"> </span><a href="https://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank" class="">https://www.mcs.anl.gov/petsc/documentation/faq.html</a><span class="Apple-converted-space"> </span>for trouble shooting.<br class="">[0]PETSC ERROR: Petsc Development GIT revision: v3.13.1-192-g23916988c6 GIT Date: 2020-05-09 02:16:44 +0300<br class="">[0]PETSC ERROR: ./ex75 on a arch-debug named localhost.localdomain by szampini Wed May 13 15:51:48 2020<br class="">[0]PETSC ERROR: Configure options --DATAFILESPATH=/home/szampini/Devel/petscdatafiles --download-elemental --download-chaco --download-ctetgen --download-exodusii --download-glvis --download-hara --download-hara-commit=HEAD --download-hdf5 --download-hpddm --download-hpddm-commit=HEAD --download-hypre --download-metis --download-mfem-commit=origin/master --download-mfem=1 --download-ml --download-mumps --download-netcdf --download-p4est --download-parmetis --download-pnetcdf --download-ptscotch --download-revolve --download-scalapack --download-slepc --download-suitesparse --download-superlu --download-superlu_dist --download-triangle --with-blaslapack-include=/usr/include/openblas --with-cxx-dialect=C++11 --with-fortran-bindings=0 --with-opengl --with-openmp --with-thrust-dir=/home/szampini/local --with-valgrind --with-zlib CFLAGS="-Wall -g -O0" CXXFLAGS="-Wall -g -O0" FFLAGS="-g -O0" PETSC_ARCH=arch-debug<br class="">[0]PETSC ERROR: #1 MatDensePlaceArray_SeqDense() line 1364 in /home/szampini/Devel/petsc/src/mat/impls/dense/seq/dense.c<br class="">[0]PETSC ERROR: #2 MatDensePlaceArray() line 1900 in /home/szampini/Devel/petsc/src/mat/impls/dense/mpi/mpidense.c<br class="">[0]PETSC ERROR: #3 GMV() line 128 in /home/szampini/Devel/petsc/arch-debug/include/HPDDM_PETSc.hpp<br class="">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br class="">[0]PETSC ERROR: Operation done in wrong order<br class="">[0]PETSC ERROR: Need to call MatDenseRestoreArray first<br class="">[0]PETSC ERROR: See<span class="Apple-converted-space"> </span><a href="https://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank" class="">https://www.mcs.anl.gov/petsc/documentation/faq.html</a><span class="Apple-converted-space"> </span>for trouble shooting.<br class="">[0]PETSC ERROR: Petsc Development GIT revision: v3.13.1-192-g23916988c6 GIT Date: 2020-05-09 02:16:44 +0300<br class="">[0]PETSC ERROR: ./ex75 on a arch-debug named localhost.localdomain by szampini Wed May 13 15:51:48 2020<br class="">[0]PETSC ERROR: Configure options --DATAFILESPATH=/home/szampini/Devel/petscdatafiles --download-elemental --download-chaco --download-ctetgen --download-exodusii --download-glvis --download-hara --download-hara-commit=HEAD --download-hdf5 --download-hpddm --download-hpddm-commit=HEAD --download-hypre --download-metis --download-mfem-commit=origin/master --download-mfem=1 --download-ml --download-mumps --download-netcdf --download-p4est --download-parmetis --download-pnetcdf --download-ptscotch --download-revolve --download-scalapack --download-slepc --download-suitesparse --download-superlu --download-superlu_dist --download-triangle --with-blaslapack-include=/usr/include/openblas --with-cxx-dialect=C++11 --with-fortran-bindings=0 --with-opengl --with-openmp --with-thrust-dir=/home/szampini/local --with-valgrind --with-zlib CFLAGS="-Wall -g -O0" CXXFLAGS="-Wall -g -O0" FFLAGS="-g -O0" PETSC_ARCH=arch-debug<br class="">[0]PETSC ERROR: #4 MatDensePlaceArray_SeqDense() line 1364 in /home/szampini/Devel/petsc/src/mat/impls/dense/seq/dense.c<br class="">[0]PETSC ERROR: #5 MatDensePlaceArray() line 1900 in /home/szampini/Devel/petsc/src/mat/impls/dense/mpi/mpidense.c<br class="">[0]PETSC ERROR: #6 apply() line 216 in /home/szampini/Devel/petsc/arch-debug/include/HPDDM_PETSc.hpp<br class="">Linear solve did not converge due to DIVERGED_ITS iterations 1000<br class=""></div><div class=""><br class=""></div><div class="">Why is that? are we suppressing stderr when comparing diffs?</div><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div dir="ltr" class="">Stefano</div></div></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div dir="ltr" class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class=""></div></div></div></div></div></div></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div dir="ltr" class="">Stefano</div></div></blockquote></div><br class=""></div></div></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a></div></div></div></div></div></div></div></div></div></blockquote></div><br class=""></div></body></html>