<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 11 Mar 2021, at 8:46 AM, Pierre Jolivet <<a href="mailto:pierre@joliv.et" class="">pierre@joliv.et</a>> wrote:</div><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 11 Mar 2021, at 6:16 AM, Barry Smith <<a href="mailto:bsmith@petsc.dev" class="">bsmith@petsc.dev</a>> wrote:</div><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> Eric,<div class=""><br class=""></div><div class=""> Sorry about not being more immediate. We still have this in our active email so you don't need to submit individual issues. We'll try to get to them as soon as we can.</div></div></div></blockquote><div class=""><br class=""></div><div class="">Indeed, I’m still trying to figure this out.</div><div class="">I realized that some of my configure flags were different than yours, e.g., no --with-memalign.</div><div class="">I’ve also added SuperLU_DIST to my installation.</div><div class="">Still, I can’t reproduce any issue.</div><div class="">I will continue looking into this, it appears I’m seeing some valgrind errors, but I don’t know if this is some side effect of OpenMPI not being valgrind-clean (last time I checked, there was no error with MPICH).</div></div></div></div></blockquote><div><br class=""></div><div>It looks like Valgrind + OpenMPI (+ OpenMP?) is complaining about uninitialized memory in PetscSFGetMultiSF().</div><div>Could you please try out the following branch <a href="https://gitlab.com/petsc/petsc/-/commits/jolivet/fix-valgrind-openmpi" class="">https://gitlab.com/petsc/petsc/-/commits/jolivet/fix-valgrind-openmpi</a> ?</div><div>I’m not sure why there would be such a warning with OpenMPI and not with MPICH, and it is unlikely to fix anything, but for good measure, after compilation, could you please try:</div><div>$ make -f gmakefile test search='snes_tutorials-ex12_quad_hpddm_reuse_baij’</div><div><br class=""></div><div>Thanks,</div><div>Pierre</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><div class="">Thank you for your patience,</div><div class="">Pierre</div><div class=""><br class=""></div><div class=""><div class="">/usr/bin/gmake -f gmakefile test test-fail=1</div><div class="">Using MAKEFLAGS: test-fail=1</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_baij.counts</div><div class=""> ok snes_tutorials-ex12_quad_hpddm_reuse_baij</div><div class=""> ok diff-snes_tutorials-ex12_quad_hpddm_reuse_baij</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist_2.counts</div><div class=""> ok ksp_ksp_tests-ex33_superlu_dist_2</div><div class=""> ok diff-ksp_ksp_tests-ex33_superlu_dist_2</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex49_superlu_dist.counts</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0</div><div class=""> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1</div><div class=""> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex50_tut_2.counts</div><div class=""> ok ksp_ksp_tutorials-ex50_tut_2</div><div class=""> ok diff-ksp_ksp_tutorials-ex50_tut_2</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist.counts</div><div class=""> ok ksp_ksp_tests-ex33_superlu_dist</div><div class=""> ok diff-ksp_ksp_tests-ex33_superlu_dist</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_hypre.counts</div><div class=""> ok snes_tutorials-ex56_hypre</div><div class=""> ok diff-snes_tutorials-ex56_hypre</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex56_2.counts</div><div class=""> ok ksp_ksp_tutorials-ex56_2</div><div class=""> ok diff-ksp_ksp_tutorials-ex56_2</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_elas.counts</div><div class=""> ok snes_tutorials-ex17_3d_q3_trig_elas</div><div class=""> ok diff-snes_tutorials-ex17_3d_q3_trig_elas</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij.counts</div><div class=""> ok snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij</div><div class=""> ok diff-snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_3.counts</div><div class="">not ok ksp_ksp_tutorials-ex5_superlu_dist_3 # Error code: 1</div><div class="">#<span class="Apple-tab-span" style="white-space:pre"> </span>srun: error: Unable to create step for job 1426755: More processors requested than permitted</div><div class=""> ok ksp_ksp_tutorials-ex5_superlu_dist_3 # SKIP Command failed so no diff</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist.counts</div><div class=""> ok ksp_ksp_tutorials-ex5f_superlu_dist # SKIP Fortran required for this test</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_tri_parmetis_hpddm_baij.counts</div><div class=""> ok snes_tutorials-ex12_tri_parmetis_hpddm_baij</div><div class=""> ok diff-snes_tutorials-ex12_tri_parmetis_hpddm_baij</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_tut_3.counts</div><div class=""> ok snes_tutorials-ex19_tut_3</div><div class=""> ok diff-snes_tutorials-ex19_tut_3</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_vlap.counts</div><div class=""> ok snes_tutorials-ex17_3d_q3_trig_vlap</div><div class=""> ok diff-snes_tutorials-ex17_3d_q3_trig_vlap</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_3.counts</div><div class=""> ok ksp_ksp_tutorials-ex5f_superlu_dist_3 # SKIP Fortran required for this test</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist.counts</div><div class=""> ok snes_tutorials-ex19_superlu_dist</div><div class=""> ok diff-snes_tutorials-ex19_superlu_dist</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre.counts</div><div class=""> ok snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre</div><div class=""> ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex49_hypre_nullspace.counts</div><div class=""> ok ksp_ksp_tutorials-ex49_hypre_nullspace</div><div class=""> ok diff-ksp_ksp_tutorials-ex49_hypre_nullspace</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist_2.counts</div><div class=""> ok snes_tutorials-ex19_superlu_dist_2</div><div class=""> ok diff-snes_tutorials-ex19_superlu_dist_2</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_2.counts</div><div class="">not ok ksp_ksp_tutorials-ex5_superlu_dist_2 # Error code: 1</div><div class="">#<span class="Apple-tab-span" style="white-space:pre"> </span>srun: error: Unable to create step for job 1426755: More processors requested than permitted</div><div class=""> ok ksp_ksp_tutorials-ex5_superlu_dist_2 # SKIP Command failed so no diff</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre.counts</div><div class=""> ok snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre</div><div class=""> ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex64_1.counts</div><div class=""> ok ksp_ksp_tutorials-ex64_1</div><div class=""> ok diff-ksp_ksp_tutorials-ex64_1</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist.counts</div><div class="">not ok ksp_ksp_tutorials-ex5_superlu_dist # Error code: 1</div><div class="">#<span class="Apple-tab-span" style="white-space:pre"> </span>srun: error: Unable to create step for job 1426755: More processors requested than permitted</div><div class=""> ok ksp_ksp_tutorials-ex5_superlu_dist # SKIP Command failed so no diff</div><div class=""> TEST arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_2.counts</div><div class=""> ok ksp_ksp_tutorials-ex5f_superlu_dist_2 # SKIP Fortran required for this test</div></div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""> Barry</div><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Mar 10, 2021, at 11:03 PM, Eric Chamberland <<a href="mailto:Eric.Chamberland@giref.ulaval.ca" class="">Eric.Chamberland@giref.ulaval.ca</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div class=""><p class="">Barry,</p><p class="">to get a some follow up on --with-openmp=1 failures, shall I open
gitlab issues for:</p><p class="">a) all hypre failures giving <span style="white-space: pre-wrap;" class="">DIVERGED_INDEFINITE_PC</span></p><p class=""><span style="white-space: pre-wrap;" class="">b) all superlu_dist failures giving different results with </span><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class="">initia and "Exceeded timeout limit of 60 s"</span></span></p><p class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class="">c) hpddm failures "free(): invalid next size (fast)" and "Segmentation Violation"
</span></span></p><p class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class="">d) all tao's </span></span><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class="">"Exceeded timeout limit of 60 s"</span></span></span></span></p><p class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class="">I don't see how I could do all these debugging by myself...</span></span></span></span></p><p class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class="">Thanks,</span></span></span></span></p><p class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class=""><span style="white-space: pre-wrap;" class="">Eric
</span></span></span></span></p>
<br class="">
</div>
</div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></div></div></blockquote></div><br class=""></body></html>