<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Menlo;
panose-1:2 11 6 9 3 8 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.m-1889958021554448809p1, li.m-1889958021554448809p1, div.m-1889958021554448809p1
{mso-style-name:m_-1889958021554448809p1;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.p1, li.p1, div.p1
{mso-style-name:p1;
margin:0cm;
font-size:10.5pt;
font-family:Menlo;
color:black;}
span.m-1889958021554448809s1
{mso-style-name:m_-1889958021554448809s1;}
span.m-1889958021554448809apple-converted-space
{mso-style-name:m_-1889958021554448809apple-converted-space;}
span.gmailsignatureprefix
{mso-style-name:gmail_signature_prefix;}
span.s1
{mso-style-name:s1;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="FR-BE" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="p1"><span class="s1"><span lang="EN-US">Hello, <o:p></o:p></span></span></p>
<p class="p1"><span class="s1"><span lang="EN-US"><o:p> </o:p></span></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">Here is a very simple version where I have issues.</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US"><o:p> </o:p></span></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">Which I run as follows:<o:p></o:p></span></span></p>
<p class="p1"><span class="s1"><span lang="EN-US"><o:p> </o:p></span></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">cd Grid_generation</span></span><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">make clean</span></span><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">make all</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">./grid_generation</span></span><span class="apple-converted-space"><span lang="EN-US"> <o:p></o:p></span></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">cd ..</span></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">make clean</span></span><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">make all</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">./cobpor</span></span><span class="apple-converted-space"><span lang="EN-US"> # on 1 proc<o:p></o:p></span></span></p>
<p class="p1"><span class="apple-converted-space"><span lang="EN-US"># OR<o:p></o:p></span></span></p>
<p class="p1"><span lang="EN-US">mpiexec ./cobpor </span><span lang="EN-US" style="color:#C814C9">-ksp_type</span><span lang="EN-US"> cg
</span><span lang="EN-US" style="color:#C814C9">-pc_type</span><span lang="EN-US"> pfmg
</span><span lang="EN-US" style="color:#C814C9">-dm_mat_type</span><span lang="EN-US"> hyprestruct
</span><span lang="EN-US" style="color:#C814C9">-pc_pfmg_skip_relax</span><span lang="EN-US">
</span><span lang="EN-US" style="color:#B42419">1</span><span lang="EN-US"> </span>
<span lang="EN-US" style="color:#C814C9">-pc_pfmg_rap_time</span><span lang="EN-US"> non-Galerkin
<span class="apple-converted-space"># on multiple procs<o:p></o:p></span></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:Menlo;color:#C814C9"><o:p> </o:p></span></p>
<p class="p1"><span lang="EN-US">The error that I get is the following:<o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">munmap_chunk(): invalid pointer</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] *** Process received signal ***</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] Signal: Aborted (6)</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] Signal code:</span></span><span class="apple-converted-space"><span lang="EN-US">
</span></span><span class="s1"><span lang="EN-US">(-6)</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] [ 0] /lib64/libc.so.6(+0x4eb20)[0x7fd7fd194b20]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] [ 1] /lib64/libc.so.6(gsignal+0x10f)[0x7fd7fd194a9f]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] [ 2] /lib64/libc.so.6(abort+0x127)[0x7fd7fd167e05]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] [ 3] /lib64/libc.so.6(+0x91037)[0x7fd7fd1d7037]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1">[cns266:2552391] [ 4] /lib64/libc.so.6(+0x9819c)[0x7fd7fd1de19c]</span></p>
<p class="p1"><span class="s1">[cns266:2552391] [ 5] /lib64/libc.so.6(+0x9844c)[0x7fd7fd1de44c]</span></p>
<p class="p1"><span class="s1">[cns266:2552391] [ 6] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(PetscFreeAlign+0xe)[0x7fd7fe63d50e]</span></p>
<p class="p1"><span class="s1">[cns266:2552391] [ 7] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetMatType+0x3d)[0x7fd7feab87ad]</span></p>
<p class="p1"><span class="s1">[cns266:2552391] [ 8] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetFromOptions+0x109)[0x7fd7feab8b59]</span></p>
<p class="p1"><span class="s1">[cns266:2552391] [ 9] ./cobpor[0x402df9]</span></p>
<p class="p1"><span class="s1">[cns266:2552391] [10] /lib64/libc.so.6(__libc_start_main+0xf3)[0x7fd7fd180cf3]</span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] [11] ./cobpor[0x40304e]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span class="s1"><span lang="EN-US">[cns266:2552391] *** End of error message ***</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="p1"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="p1"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="p1"><span lang="EN-US">Thanks a lot for your help. <o:p></o:p></span></p>
<p class="p1"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="p1"><span lang="EN-US">Best regards, <o:p></o:p></span></p>
<p class="p1"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="p1"><span lang="EN-US">Joauma<o:p></o:p></span></p>
<p class="p1"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span lang="EN-US" style="font-size:12.0pt;color:black">De :
</span></b><span lang="EN-US" style="font-size:12.0pt;color:black">Matthew Knepley <knepley@gmail.com><br>
<b>Date : </b>mercredi, 25 octobre 2023 à 14:45<br>
<b>À : </b>Joauma Marichal <joauma.marichal@uclouvain.be><br>
<b>Cc : </b>petsc-maint@mcs.anl.gov <petsc-maint@mcs.anl.gov>, petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Objet : </b>Re: [petsc-maint] DMSwarm on multiple processors<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">On Wed, Oct 25, 2023 at 8:32 AM Joauma Marichal via petsc-maint <</span><a href="mailto:petsc-maint@mcs.anl.gov"><span lang="EN-US" style="font-size:11.0pt">petsc-maint@mcs.anl.gov</span></a><span lang="EN-US" style="font-size:11.0pt">>
wrote:<o:p></o:p></span></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Hello, </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">I am using the DMSwarm library in some Eulerian-Lagrangian approach to have vapor bubbles in water. </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">I have obtained nice results recently and wanted to perform bigger simulations. Unfortunately, when I increase the number
of processors used to run the simulation, I get the following error:</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">free(): invalid size</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] *** Process received signal ***</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] Signal: Aborted (6)</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] Signal code:</span></span><span class="m-1889958021554448809apple-converted-space"><span lang="EN-US">
</span></span><span class="m-1889958021554448809s1"><span lang="EN-US">(-6)</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 0] /lib64/libc.so.6(+0x4eb20)[0x7f56cd4c9b20]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 1] /lib64/libc.so.6(gsignal+0x10f)[0x7f56cd4c9a9f]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 2] /lib64/libc.so.6(abort+0x127)[0x7f56cd49ce05]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 3] /lib64/libc.so.6(+0x91037)[0x7f56cd50c037]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 4] /lib64/libc.so.6(+0x9819c)[0x7f56cd51319c]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 5] /lib64/libc.so.6(+0x99aac)[0x7f56cd514aac]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 6] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(PetscSFSetUpRanks+0x4c4)[0x7f56cea71e64]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 7] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(+0x841642)[0x7f56cea83642]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 8] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(PetscSFSetUp+0x9e)[0x7f56cea7043e]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [ 9] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(VecScatterCreate+0x164e)[0x7f56cea7bbde]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [10] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetUp_DA_3D+0x3e38)[0x7f56cee84dd8]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [11] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetUp_DA+0xd8)[0x7f56cee9b448]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [12] /gpfs/home/acad/ucl-tfl/marichaj/marha/lib_petsc/lib/libpetsc.so.3.019(DMSetUp+0x20)[0x7f56cededa20]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [13] ./cobpor[0x4418dc]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [14] ./cobpor[0x408b63]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [15] /lib64/libc.so.6(__libc_start_main+0xf3)[0x7f56cd4b5cf3]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] [16] ./cobpor[0x40bdee]</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">[cns136:590327] *** End of error message ***</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">Primary job</span></span><span class="m-1889958021554448809apple-converted-space"><span lang="EN-US">
</span></span><span class="m-1889958021554448809s1"><span lang="EN-US">terminated normally, but 1 process returned</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1">a non-zero exit code.
</span><span class="m-1889958021554448809s1"><span lang="EN-US">Per user-direction, the job has been aborted.</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">mpiexec noticed that process rank 84 with PID 590327 on node cns136 exited on signal 6 (Aborted).</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m-1889958021554448809p1"><span class="m-1889958021554448809s1"><span lang="EN-US">--------------------------------------------------------------------------</span></span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">When I reduce the number of processors the error disappears and when I run my code without the vapor bubbles it also works.
</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">The problem seems to take place at this moment:</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMCreate</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(PETSC_COMM_WORLD,swarm);</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSetType</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,DMSWARM);</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSetDimension</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#B5CEA8">3</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">);</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSwarmSetType</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,DMSWARM_PIC);</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:13.5pt;background:#1E1E1E">
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4"> </span>
<span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#DCDCAA">DMSwarmSetCellDM</span><span lang="EN-US" style="font-size:9.0pt;font-family:Menlo;color:#D4D4D4">(*swarm,*dmcell);</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Thanks a lot for your help. </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Things that would help us track this down:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">1) The smallest example where it fails<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">2) The smallest number of processes where it fails<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">3) A stack trace of the failure<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">4) A simple example that we can run that also fails<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> Matt<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Best regards, </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121"> </span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#212121">Joauma</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><br clear="all">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span class="gmailsignatureprefix"><span lang="EN-US" style="font-size:11.0pt">--
</span></span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank"><span lang="EN-US" style="font-size:11.0pt">https://www.cse.buffalo.edu/~knepley/</span></a><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>