<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<!-- Template generated by Exclaimer Mail Disclaimers on 02:22:32 Wednesday, 28 April 2021 -->
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:"Yu Gothic";
panose-1:2 11 4 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"\@Yu Gothic";
panose-1:2 11 4 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MailQuote, li.MailQuote, div.MailQuote
{mso-style-name:"Mail Quote";
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
border:none;
padding:0in;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
color:black;
mso-fareast-language:EN-GB;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<p></p>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">The modules have automatic freeing in as much as that when a variable that is local to a subroutine is ALLOCATE'd, it is automatically freed when the subroutine
returns. I don't think that is problematic, as MatDestroy is used a lot in the code and normally executes just fine.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">As far as I can see, no specific new communicators are created; MatCreateAIJ or MatCreateSeqAIJ are called with PETSC_COMM_WORLD, resp. PETSC_COMM_SELF as first
argument.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">We also run this with the Intel MPI library, which is based on MPICH. There this problem does not occur.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">The Valgrind run did not produce any new insights (at least not for me), I have pasted the relevant bits at the end of this message. I did a run on debug versions
of PETSc (v3.14.5) and OpenMPI (v 3.1.2) and I find the following stack trace with line numbers for each frame. Maybe that helps in further pinpointing the problem.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">0x0000155540d11719 in ompi_comm_free (comm=0x483f4e0) at /home/mdeij/build-libs-gnu/superbuild/openmpi/src/ompi/communicator/comm.c:1470<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">1470 if ( ! OMPI_COMM_IS_INTRINSIC((*comm)->c_local_comm)) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">Missing separate debuginfos, use: yum debuginfo-install libgcc-8.3.1-5.el8.0.2.x86_64 libgfortran-8.3.1-5.el8.0.2.x86_64 libibumad-47mlnx1-1.47329.x86_64 libibverbs-47mlnx1-1.47329.x86_64
libnl3-3.5.0-1.el8.x86_64 libquadmath-8.3.1-5.el8.0.2.x86_64 librdmacm-47mlnx1-1.47329.x86_64 libstdc++-8.3.1-5.el8.0.2.x86_64 libxml2-2.9.7-7.el8.x86_64 numactl-libs-2.0.12-9.el8.x86_64 opensm-libs-5.5.1.MLNX20191120.0c8dde0-0.1.47329.x86_64 openssl-libs-1.1.1c-15.el8.x86_64
python3-libs-3.6.8-23.el8.x86_64 sssd-client-2.2.3-20.el8.x86_64 ucx-cma-1.7.0-1.47329.x86_64 ucx-ib-1.7.0-1.47329.x86_64 xz-libs-5.2.4-3.el8.x86_64 zlib-1.2.11-16.el8_2.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">(gdb) bt<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#0 0x0000155540d11719 in ompi_comm_free (comm=0x483f4e0) at /home/mdeij/build-libs-gnu/superbuild/openmpi/src/ompi/communicator/comm.c:1470<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#1 0x0000155540d4f1af in PMPI_Comm_free (comm=0x483f4e0) at pcomm_free.c:62<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#2 0x000015555346329a in superlu_gridexit (grid=0x483f4e0) at /home/mdeij/install-gnu/extLibs/Linux-x86_64-Intel/superlu_dist-6.3.0/SRC/superlu_grid.c:174<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#3 0x0000155553ca2ff1 in Petsc_Superlu_dist_keyval_Delete_Fn (comm=0x3921b10, keyval=16, attr_val=0x483f4d0, extra_state=0x0) at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c:97<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#4 0x0000155540d0baa1 in ompi_attr_delete_impl (type=COMM_ATTR, object=0x3921b10, attr_hash=0x377efe0, key=16, predefined=true) at /home/mdeij/build-libs-gnu/superbuild/openmpi/src/ompi/attribute/attribute.c:1062<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#5 0x0000155540d0c039 in ompi_attr_delete_all (type=COMM_ATTR, object=0x3921b10, attr_hash=0x377efe0) at /home/mdeij/build-libs-gnu/superbuild/openmpi/src/ompi/attribute/attribute.c:1166<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#6 0x0000155540d11676 in ompi_comm_free (comm=0x7fffffffc5c0) at /home/mdeij/build-libs-gnu/superbuild/openmpi/src/ompi/communicator/comm.c:1462<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#7 0x0000155540d4f1af in PMPI_Comm_free (comm=0x7fffffffc5c0) at pcomm_free.c:62<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#8 0x000015555393fb68 in PetscCommDestroy (comm=0x3943a60) at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/sys/objects/tagm.c:217<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#9 0x0000155553941e07 in PetscHeaderDestroy_Private (h=0x3943a20) at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/sys/objects/inherit.c:121<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#10 0x000015555408edfe in MatDestroy (A=0x3558c18) at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/mat/interface/matrix.c:1306<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">#11 0x00001555540cb5fa in matdestroy_ (A=0x3558c18, __ierr=0x7fffffffc73c) at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/mat/interface/ftn-auto/matrixf.c:770<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">Valgrind output:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Invalid read of size 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== at 0x19184538: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x6943B61: superlu_gridexit (in /home/mdeij/install-gnu/extLibs/lib/libsuperlu_dist.so.6.3.0)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x56F398E: Petsc_Superlu_dist_keyval_Delete_Fn (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912447B: ompi_attr_delete_impl (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19126FFE: ompi_attr_delete_all (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912ACC6: ompi_comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19184555: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FEE49D: PetscCommDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FF0EE1: PetscHeaderDestroy_Private (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x5317899: MatDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x5336E58: matdestroy_ (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1528710: __fsi_MOD_fem_constructmatricespetscexit (fsi.F90:2297)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Address 0x2ce67398 is 11,112 bytes inside an unallocated block of size 11,232 in arena "client"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905==<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Invalid read of size 8<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== at 0x1912AC9A: ompi_comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19184555: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x6943B61: superlu_gridexit (in /home/mdeij/install-gnu/extLibs/lib/libsuperlu_dist.so.6.3.0)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x56F398E: Petsc_Superlu_dist_keyval_Delete_Fn (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912447B: ompi_attr_delete_impl (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19126FFE: ompi_attr_delete_all (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912ACC6: ompi_comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19184555: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FEE49D: PetscCommDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FF0EE1: PetscHeaderDestroy_Private (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x5317899: MatDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x5336E58: matdestroy_ (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Address 0x2ce673c0 is 11,152 bytes inside an unallocated block of size 11,232 in arena "client"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905==<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Invalid read of size 8<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== at 0x19126E5B: ompi_attr_delete_all (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912ACC6: ompi_comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19184555: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x6943B61: superlu_gridexit (in /home/mdeij/install-gnu/extLibs/lib/libsuperlu_dist.so.6.3.0)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x56F398E: Petsc_Superlu_dist_keyval_Delete_Fn (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912447B: ompi_attr_delete_impl (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19126FFE: ompi_attr_delete_all (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912ACC6: ompi_comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19184555: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FEE49D: PetscCommDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FF0EE1: PetscHeaderDestroy_Private (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x5317899: MatDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Address 0x91 is not stack'd, malloc'd or (recently) free'd<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905==<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905==<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Process terminating with default action of signal 11 (SIGSEGV)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== Access not within mapped region at address 0x91<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== at 0x19126E5B: ompi_attr_delete_all (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912ACC6: ompi_comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19184555: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x6943B61: superlu_gridexit (in /home/mdeij/install-gnu/extLibs/lib/libsuperlu_dist.so.6.3.0)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x56F398E: Petsc_Superlu_dist_keyval_Delete_Fn (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912447B: ompi_attr_delete_impl (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19126FFE: ompi_attr_delete_all (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x1912ACC6: ompi_comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x19184555: PMPI_Comm_free (in /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40.10.2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FEE49D: PetscCommDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x4FF0EE1: PetscHeaderDestroy_Private (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== by 0x5317899: MatDestroy (in /home/mdeij/install-gnu/extLibs/lib/libpetsc.so.3.14.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== If you believe this happened as a result of a stack<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== overflow in your program's main thread (unlikely but<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== possible), you can try to increase the size of the<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== main thread stack using the --main-stacksize= flag.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">==1026905== The main thread stack size used in this run was 16777216.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div></div>
</div>
<br>
<font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;">dr. ir.</font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;">
</font><font style="font-family: Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;">Menno A.</font><font style="font-family: Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;">
</font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;">Deij-van Rijswijk</font><font size="3" face="Segoe UI"></font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;"> |
</font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;">Researcher</font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;"> |
</font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;font-weight:bold;">Research & Development</font><br>
<font style="font-family:Verdana;font-size:9pt;color:#7D7D7D;font-weight:bold;">MARIN</font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;"> | T
</font><font style="font-family:Verdana;font-size:8pt;color:#7D7D7D;">+31 317 49 35 06</font><font color="#7d7d7d" size="2" face="Segoe UI"></font><font style="font-family:Verdana;font-size:8pt;"> |
</font><span style="font-family:Verdana;font-size:8pt;"><a href="mailto:M.Deij@marin.nl" title="" target="" style="font-family:Verdana;font-size:8pt;"><span style="font-family:Verdana; font-size:8pt;">M.Deij@marin.nl</span></a></span><font color="#7d7d7d" size="2" face="Segoe UI"></font><font style="font-family:Verdana;font-size:8pt;"> |
</font><span style="font-family:Verdana;font-size:8pt;"><a href="http://www.marin.nl" title="" target="" style="font-family:Verdana;font-size:8pt;"><span style="font-family:Verdana; font-size:8pt;">www.marin.nl</span></a></span><br>
<br>
<a href="https://www.linkedin.com/company/marin" target=""><img width="16" height="16" style="border: 0px Solid ; " src="cid:imagebf865c.PNG@c1771fab.488b7a50" alt="LinkedIn"></a> <a href="http://www.youtube.com/marinmultimedia" target=""><img width="16" height="16" style="border: 0px Solid ; " src="cid:image1edec1.PNG@016fe9b1.4e9007aa" alt="YouTube"></a> <a href="https://twitter.com/MARIN_nieuws" target=""><img width="16" height="16" style="border: 0px Solid ; " src="cid:imagedbdbd7.PNG@2927a014.4eb382dc" alt="Twitter"></a> <a href="https://www.facebook.com/marin.wageningen" target=""><img width="16" height="16" style="border: 0px Solid ; " src="cid:image4abcc0.PNG@e87c010f.4687f46c" alt="Facebook"></a><br>
<font style="FONT-FAMILY: Verdana; FONT-SIZE: 8pt" color="#7d7d7d" size="2" face="Verdana">MARIN news:
<a href="https://www.marin.nl/news/wasp-webinar-wisp-workshop-april-22" title="" target="" style="">
WASP webinar & WiSP workshop</a></font>
<p></p>
<p id="undefined"></p>
<p></p>
<div class="WordSection1">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Barry Smith <bsmith@petsc.dev>
<br>
<b>Sent:</b> Friday, April 23, 2021 7:09 PM<br>
<b>To:</b> Deij-van Rijswijk, Menno <M.Deij@marin.nl><br>
<b>Cc:</b> petsc-users@mcs.anl.gov<br>
<b>Subject:</b> Re: [petsc-users] MatDestroy problem with multiple matrices and SUPERLU_DIST<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"> Thanks for looking. Do these modules have any "automatic freeing" when variables go out of scope (like C++ classes do)? <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Do you make specific new MPI communicators to use create the matrices? <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Have you tried MPICH or a different version of OpenMPI. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Maybe run the program with valgrind. The stack frames you sent look "funny", that is I would not normally expect them to be in such an order.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Barry<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</EXCLAIMER:ORIGINAL_BODY_AFTER>
<p></p>
</EXCLAIMER:TEXT_ONLY></EXCLAIMER:AD></EXCLAIMER:AD>
</body>
</html>