<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Menlo;
panose-1:2 11 6 9 3 8 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.m2433361434654699384m-1889958021554448809p1, li.m2433361434654699384m-1889958021554448809p1, div.m2433361434654699384m-1889958021554448809p1
{mso-style-name:m_2433361434654699384m-1889958021554448809p1;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.m2433361434654699384m-1889958021554448809s1
{mso-style-name:m_2433361434654699384m-1889958021554448809s1;}
span.m2433361434654699384m-1889958021554448809apple-converted-space
{mso-style-name:m_2433361434654699384m-1889958021554448809apple-converted-space;}
span.m2433361434654699384gmailsignatureprefix
{mso-style-name:m_2433361434654699384gmailsignatureprefix;}
span.gmailsignatureprefix
{mso-style-name:gmail_signature_prefix;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="FR-BE" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">Hello,
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">Sorry for the delay. I attach the file that I obtain when running the code with the debug mode.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">Thanks for your help.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">Best regards,
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">Joauma<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">De :
</span></b><span style="font-size:12.0pt;color:black">Matthew Knepley <knepley@gmail.com><br>
<b>Date : </b>jeudi, 23 novembre 2023 à 15:32<br>
<b>À : </b>Joauma Marichal <joauma.marichal@uclouvain.be><br>
<b>Cc : </b>petsc-maint@mcs.anl.gov <petsc-maint@mcs.anl.gov>, petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Objet : </b>Re: [petsc-maint] DMSwarm on multiple processors<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On Thu, Nov 23, 2023 at 9:01 AM Joauma Marichal <<a href="mailto:joauma.marichal@uclouvain.be">joauma.marichal@uclouvain.be</a>> wrote:<o:p></o:p></span></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Hello,
</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">My problem persists… Is there anything I could try?</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Yes. It appears to be failing from a call inside PetscSFSetUpRanks(). It does allocation, and the failure<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">is in libc, and it only happens on larger examples, so I suspect some allocation problem. Can you rebuild with debugging and run this example? Then we can see if the allocation fails.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"> Thanks,<br>
<br>
Matt<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Thanks a lot.
</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Best regards,
</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt">Joauma</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">De :
</span></b><span style="font-size:12.0pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date : </b>mercredi, 25 octobre 2023 à 14:45<br>
<b>À : </b>Joauma Marichal <<a href="mailto:joauma.marichal@uclouvain.be" target="_blank">joauma.marichal@uclouvain.be</a>><br>
<b>Cc : </b><a href="mailto:petsc-maint@mcs.anl.gov" target="_blank">petsc-maint@mcs.anl.gov</a> <<a href="mailto:petsc-maint@mcs.anl.gov" target="_blank">petsc-maint@mcs.anl.gov</a>>,
<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Objet : </b>Re: [petsc-maint] DMSwarm on multiple processors</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">On Wed, Oct 25, 2023 at 8:32 AM Joauma Marichal via petsc-maint <<a href="mailto:petsc-maint@mcs.anl.gov" target="_blank">petsc-maint@mcs.anl.gov</a>>
wrote:<o:p></o:p></span></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Hello, </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">I am using the DMSwarm library in some Eulerian-Lagrangian approach to have vapor bubbles in water. </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">I have obtained nice results recently and wanted to perform bigger simulations. Unfortunately, when I increase the number
of processors used to run the simulation, I get the following error:</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">free(): invalid size</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] *** Process received signal ***</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] Signal: Aborted (6)</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] Signal code:</span></span><span class="m2433361434654699384m-1889958021554448809apple-converted-space"><span lang="EN-US">
</span></span><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">(-6)</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 0] /lib64/libc.so.6(+0x4eb20)[0x7f56cd4c9b20]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 1] /lib64/libc.so.6(gsignal+0x10f)[0x7f56cd4c9a9f]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 2] /lib64/libc.so.6(abort+0x127)[0x7f56cd49ce05]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 3] /lib64/libc.so.6(+0x91037)[0x7f56cd50c037]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 4] /lib64/libc.so.6(+0x9819c)[0x7f56cd51319c]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 5] /lib64/libc.so.6(+0x99aac)[0x7f56cd514aac]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 6] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(PetscSFSetUpRanks+0x4c4)[0x7f56cea71e64]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 7] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(+0x841642)[0x7f56cea83642]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 8] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(PetscSFSetUp+0x9e)[0x7f56cea7043e]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1">[cns136:590327] [ 9] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(VecScatterCreate+0x164e)[0x7f56cea7bbde]</span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1">[cns136:590327] [10] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetUp_DA_3D+0x3e38)[0x7f56cee84dd8]</span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1">[cns136:590327] [11] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetUp_DA+0xd8)[0x7f56cee9b448]</span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1">[cns136:590327] [12] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetUp+0x20)[0x7f56cededa20]</span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [13] ./cobpor[0x4418dc]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [14] ./cobpor[0x408b63]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [15] /lib64/libc.so.6(__libc_start_main+0xf3)[0x7f56cd4b5cf3]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [16] ./cobpor[0x40bdee]</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] *** End of error message ***</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">Primary job</span></span><span class="m2433361434654699384m-1889958021554448809apple-converted-space"><span lang="EN-US">
</span></span><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">terminated normally, but 1 process returned</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1">a non-zero exit code.
</span><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">Per user-direction, the job has been aborted.</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">mpiexec noticed that process rank 84 with PID 590327 on node cns136 exited on signal 6 (Aborted).</span></span><o:p></o:p></p>
<p class="m2433361434654699384m-1889958021554448809p1"><span class="m2433361434654699384m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">When I reduce the number of processors the error disappears and when I run my code without the vapor bubbles it also works.
</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">The problem seems to take place at this moment:</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMCreate</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(PETSC_COMM_WORLD,swarm);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSetType</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,DMSWARM);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSetDimension</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#B5CEA8">3</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSwarmSetType</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,DMSWARM_PIC);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSwarmSetCellDM</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,*dmcell);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Thanks a lot for your help. </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">Things that would help us track this down:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">1) The smallest example where it fails<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">2) The smallest number of processes where it fails<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">3) A stack trace of the failure<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">4) A simple example that we can run that also fails<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> Matt<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Best regards, </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Joauma</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"><br clear="all">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span class="m2433361434654699384gmailsignatureprefix"><span style="font-size:11.0pt">--
</span></span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><br clear="all">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span class="gmailsignatureprefix"><span style="font-size:11.0pt">--
</span></span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>