<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> Thanks for looking. Do these modules have any "automatic freeing" when variables go out of scope (like C++ classes do)? <div class=""><br class=""></div><div class=""> Do you make specific new MPI communicators to use create the matrices? </div><div class=""><br class=""></div><div class=""> Have you tried MPICH or a different version of OpenMPI. </div><div class=""><br class=""></div><div class=""> Maybe run the program with valgrind. The stack frames you sent look "funny", that is I would not normally expect them to be in such an order.</div><div class=""><br class=""></div><div class=""> Barry</div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Apr 23, 2021, at 8:31 AM, Deij-van Rijswijk, Menno <<a href="mailto:M.Deij@marin.nl" class="">M.Deij@marin.nl</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: "Times New Roman", serif; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class="webkit-block-placeholder"></div><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Hi Barry,<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Thank you for looking into this. The code I'm referring to is part of a larger fortran module, and I have tried to isolate the problem in a reproducible test case. Unfortunately, I have not been able to reproduce the problem there. It is probably a subtle bug or misuse on our part, but for now I can't seem to pinpoint the problem.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Menno<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div class=""></div></div><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class="">dr. ir.</font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class=""><span class="Apple-converted-space"> </span></font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class="">Menno A.</font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class=""><span class="Apple-converted-space"> </span></font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class="">Deij-van Rijswijk</font><font size="3" face="Segoe UI" style="caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""></font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class=""><span class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span></font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class="">Researcher</font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class=""><span class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span></font><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125); font-weight: bold;" class="">Research & Development</font><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><font style="font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 9pt; color: rgb(125, 125, 125); font-weight: bold;" class="">MARIN</font><font style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125);" class=""><span class="Apple-converted-space"> </span>| T<span class="Apple-converted-space"> </span></font><font style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt; color: rgb(125, 125, 125);" class="">+31 317 49 35 06</font><font color="#7d7d7d" size="2" face="Segoe UI" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""></font><font style="caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt;" class=""><span class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span></font><span style="caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt;" class=""><a href="mailto:M.Deij@marin.nl" title="" target="" style="color: purple; text-decoration: underline; font-family: Verdana; font-size: 8pt;" class=""><span style="font-family: Verdana; font-size: 8pt;" class="">M.Deij@marin.nl</span></a></span><font color="#7d7d7d" size="2" face="Segoe UI" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""></font><font style="caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt;" class=""> |<span class="Apple-converted-space"> </span></font><span style="caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt;" class=""><a href="http://www.marin.nl/" title="" target="" style="color: purple; text-decoration: underline; font-family: Verdana; font-size: 8pt;" class=""><span style="font-family: Verdana; font-size: 8pt;" class="">www.marin.nl</span></a></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="https://www.linkedin.com/company/marin" target="" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><span id="cid:image578a50.PNG@370aa96f.49ac687a"><image578a50.PNG></span></a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> </span><a href="http://www.youtube.com/marinmultimedia" target="" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><span id="cid:imaged2652e.PNG@7ed8c7d8.4580976c"><imaged2652e.PNG></span></a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> </span><a href="https://twitter.com/MARIN_nieuws" target="" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><span id="cid:image454226.PNG@b44fd166.4d99aa89"><image454226.PNG></span></a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> </span><a href="https://www.facebook.com/marin.wageningen" target="" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><span id="cid:image9421ef.PNG@d2e51b48.418d5dd7"><image9421ef.PNG></span></a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><font color="#7d7d7d" size="2" face="Verdana" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Verdana; font-size: 8pt;" class="">MARIN news:<span class="Apple-converted-space"> </span><a href="https://www.marin.nl/news/report-magazine-131-read-now-and-register-for-future-editions" title="" target="" style="color: purple; text-decoration: underline;" class="">Report magazine 131: read now and register for future editions</a></font><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""></span><div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: "Times New Roman", serif; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class="webkit-block-placeholder"></div><div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: "Times New Roman", serif; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class="webkit-block-placeholder"></div><div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: "Times New Roman", serif; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class="webkit-block-placeholder"></div><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0in 0in;" class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><b class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">From:</span></b><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" style="color: purple; text-decoration: underline;" class="">bsmith@petsc.dev</a>><span class="Apple-converted-space"> </span><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Friday, April 23, 2021 1:18 AM<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Deij-van Rijswijk, Menno <<a href="mailto:M.Deij@marin.nl" style="color: purple; text-decoration: underline;" class="">M.Deij@marin.nl</a>><br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" style="color: purple; text-decoration: underline;" class="">petsc-users@mcs.anl.gov</a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] MatDestroy problem with multiple matrices and SUPERLU_DIST<o:p class=""></o:p></span></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""> Please send a code that reproduces the problem. I cannot reproduce it. It is absolutely expected to work, but there may be an untested corner case you have hit on.<o:p class=""></o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""> The comment <o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">/* This allows reusing the Superlu_DIST communicator and grid when only a single SuperLU_DIST matrix is used at a time */<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">means that __IF__ only a single SuperLU_DIST matrix is used at a time then the Superlu_DIST communicator and grid ARE reused for a next one created after the first has been destroyed. If multiple SuperLU_DIST matrix are needed at the same time then they each get there own Superlu_DIST communicator and grid and there is no reuse. But this does not mean you cannot have as many matrices and solvers outstanding at the same time as you want.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""> Barry<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Yes there could be a subtle bug in our logic but we need a test case to find it.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><br class=""><br class=""><o:p class=""></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">On Apr 21, 2021, at 8:06 AM, Deij-van Rijswijk, Menno <<a href="mailto:M.Deij@marin.nl" style="color: purple; text-decoration: underline;" class="">M.Deij@marin.nl</a>> wrote:<o:p class=""></o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""><o:p class=""> </o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Good afternoon,</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">In our code we're using two matrices that are preconditioned using SUPERLU_dist. Upon freeing the second matrix with MatDestroy, the program segfaults with the following stacktrace (see below). This happens with PETSc versions 3.14.5 and 3.15.0, whereas version 3.11.2 does not have this problem. In the code src\mat\impls\aij\mpi\superlu_dist\superlu_dist.c I see that Petsc_Superlu_dist_keyval_Delete_Fn has been added between 3.11.2 and 3.14.5, and the comment reads that it allows reusing the communicator when only a single matrix is used at the time. I'm wondering if using multiple matrices with SUPERLU_dist is problematic here?</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Note: this is happening on single process and on multiple MPI processes.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Best regards,</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Menno Deij - van Rijswijk</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#0 0x000015554ffc7db4 in ompi_comm_free () from /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#1 0x0000155550021536 in PMPI_Comm_free () from /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#2 0x00001555534262ba in superlu_gridexit (grid=0x3689da0)</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class=""> at /home/mdeij/install-gnu/extLibs/Linux-x86_64-Intel/superlu_dist-6.3.0/SRC/superlu_grid.c:174</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#3 0x0000155553df094b in Petsc_Superlu_dist_keyval_Delete_Fn (comm=0x26a4e50, keyval=16, attr_val=0x3689d90,</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class=""> extra_state=0x0)</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class=""> at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c:97</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#4 0x000015554ffc145c in ompi_attr_delete_impl () from /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#5 0x000015554ffc3fdf in ompi_attr_delete_all () from /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#6 0x000015554ffc7ca7 in ompi_comm_free () from /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#7 0x0000155550021536 in PMPI_Comm_free () from /home/mdeij/install-gnu/extLibs/lib/libmpi.so.40</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#8 0x00001555538f31c5 in PetscCommDestroy (comm=0x29712e0)</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class=""> at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/sys/objects/tagm.c:217</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#9 0x00001555538f55ab in PetscHeaderDestroy_Private (h=0x29712a0)</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class=""> at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/sys/objects/inherit.c:121</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#10 0x0000155553b44626 in MatDestroy (A=0x26e1c98)</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class=""> at /home/mdeij/build-libs-gnu/superbuild/petsc/src/src/mat/interface/matrix.c:1310</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class="">#11 0x0000155553b87a86 in matdestroy_ (x=0x26e1c98, ierr=0x7fffffffc9cc)</span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span lang="EN-US" style="font-size: 11pt; font-family: Consolas;" class=""> </span><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""></o:p></span></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""><br class=""></span><b class=""><span style="font-size: 8pt; font-family: Verdana, sans-serif; color: rgb(125, 125, 125);" class="">dr. ir.<span class="apple-converted-space"> </span>Menno A.<span class="apple-converted-space"> </span>Deij-van Rijswijk<span class="apple-converted-space"> </span>|<span class="apple-converted-space"> </span>Researcher<span class="apple-converted-space"> </span>|<span class="apple-converted-space"> </span>Research & Development</span></b><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""><br class=""></span><b class=""><span style="font-size: 9pt; font-family: Verdana, sans-serif; color: rgb(125, 125, 125);" class="">MARIN</span></b><span class="apple-converted-space"><span style="font-size: 8pt; font-family: Verdana, sans-serif; color: rgb(125, 125, 125);" class=""> </span></span><span style="font-size: 8pt; font-family: Verdana, sans-serif; color: rgb(125, 125, 125);" class="">| T<span class="apple-converted-space"> </span>+31 317 49 35 06</span><span class="apple-converted-space"><span style="font-size: 8pt; font-family: Verdana, sans-serif;" class=""> </span></span><span style="font-size: 8pt; font-family: Verdana, sans-serif;" class="">|<span class="apple-converted-space"> </span><a href="mailto:M.Deij@marin.nl" style="color: purple; text-decoration: underline;" class=""><span style="color: rgb(149, 79, 114);" class="">M.Deij@marin.nl</span></a> |<span class="apple-converted-space"> </span><a href="http://www.marin.nl/" style="color: purple; text-decoration: underline;" class=""><span style="color: rgb(149, 79, 114);" class="">www.marin.nl</span></a></span><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""><br class=""><br class=""></span><a href="https://www.linkedin.com/company/marin" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);" class=""><image350854.PNG></span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""> </span><a href="http://www.youtube.com/marinmultimedia" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);" class=""><image623ac8.PNG></span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""> </span><a href="https://twitter.com/MARIN_nieuws" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);" class=""><image6c0411.PNG></span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""> </span><a href="https://www.facebook.com/marin.wageningen" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: rgb(149, 79, 114);" class=""><imagea99443.PNG></span></a><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif;" class=""><br class=""></span><span style="font-size: 8pt; font-family: Verdana, sans-serif; color: rgb(125, 125, 125);" class="">MARIN news:<span class="apple-converted-space"> </span><a href="https://www.marin.nl/news/iwsa-open-letter-release-for-shipping-decarbonisation" style="color: purple; text-decoration: underline;" class=""><span style="color: rgb(149, 79, 114);" class="">IWSA Open letter release for shipping decarbonisation</span></a></span><o:p class=""></o:p></div></div></blockquote></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 12pt; font-family: "Times New Roman", serif;"><span style="background-color: white; background-position: initial initial; background-repeat: initial initial;" class=""><o:p class=""> </o:p></span></p><p align="center" style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: "Times New Roman", serif; text-align: center;" class=""><b class=""><span style="background-color: white; background-position: initial initial; background-repeat: initial initial;" class="">Help us improve the spam filter. If this message contains SPAM, click<span class="Apple-converted-space"> </span><a href="https://www.mailcontrol.com/sr/rkrirCoveRbGX2PQPOmvUs3oG7lEfVxCKk45h3sl3AVch8EgHsvHBwi-JRbSzaNb4s9osQehojJsVqdfswMBgw==" style="color: purple; text-decoration: underline;" class="">here</a><span class="Apple-converted-space"> </span>to report. Thank you, MARIN Digital Services</span></b></p></div></div></blockquote></div><br class=""></div></body></html>