<div dir="ltr"><div><br></div><div>Hi Folks, </div><div><br></div><div>I am using MatZeroRows() to set Dirichlet boundary conditions. This works fine for the serial run and the solver produces correct results (verified through analytical solution). However, when I run the case in parallel, the simulation gets stuck at MatZeroRows(). My understanding is that this function needs to be called after the MatAssemblyBegin{End}() has been called, and should be called by all processors. Here is that bit of the code which calls MatZeroRows() after the matrix has been assembled</div><br clear="all"><div><a href="https://github.com/IBAMR/IBAMR/blob/amneetb/acoustically-driven-flows/src/acoustic_streaming/AcousticStreamingPETScMatUtilities.cpp#L724-L801">https://github.com/IBAMR/IBAMR/blob/amneetb/acoustically-driven-flows/src/acoustic_streaming/AcousticStreamingPETScMatUtilities.cpp#L724-L801</a><br></div><span class="gmail_signature_prefix"><div><span class="gmail_signature_prefix"><br></span></div><div><span class="gmail_signature_prefix">I ran the parallel code (on 3 processors) in the debugger (-start_in_debugger). Below is the call stack from the processor that gets stuck</span></div><div><span class="gmail_signature_prefix"><br></span></div><div><span class="gmail_signature_prefix">
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">amneetb</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">@</span><span class="gmail-s3" style="font-variant-ligatures:no-common-ligatures;color:rgb(47,180,29)">APSB-MBP-16:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">~</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">$ lldb<span class="gmail-Apple-converted-space"> </span>-p 4307<span class="gmail-Apple-converted-space"> </span></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">(lldb) process attach --pid 4307</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">Process 4307 stopped</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">* thread #1, queue = </span><span class="gmail-s3" style="font-variant-ligatures:no-common-ligatures;color:rgb(47,180,29)">'com.apple.main-thread'</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">, stop reason = </span><span class="gmail-s5" style="font-variant-ligatures:no-common-ligatures;color:rgb(180,36,25)">signal SIGSTOP</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #0: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x000000018a2d750c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libsystem_kernel.dylib`__semwait_signal + 8</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">libsystem_kernel.dylib`:</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">-><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x18a2d750c <+8>:<span class="gmail-Apple-converted-space"> </span>b.lo <span class="gmail-Apple-converted-space"> </span>0x18a2d752c <span class="gmail-Apple-converted-space"> </span>; <+40></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)"><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x18a2d7510 <+12>: pacibsp<span class="gmail-Apple-converted-space"> </span></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)"><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x18a2d7514 <+16>: stp<span class="gmail-Apple-converted-space"> </span>x29, x30, [sp, #-0x10]!</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)"><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x18a2d7518 <+20>: mov<span class="gmail-Apple-converted-space"> </span>x29, sp</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">Target 0: (fo_acoustic_streaming_solver_2d) stopped.</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">Executable module set to "/Users/amneetb/Softwares/IBAMR-Git/objs-dbg/tests/IBTK/fo_acoustic_streaming_solver_2d".</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">Architecture set to: arm64-apple-macosx-.</span></p>
<p class="gmail-p2" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgba(97,97,97,0.91);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">(lldb) </span><span class="gmail-s6" style="font-variant-ligatures:no-common-ligatures;color:rgb(242,242,242)">cont</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">Process 4307 resuming</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">Process 4307 stopped</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">* thread #1, queue = </span><span class="gmail-s3" style="font-variant-ligatures:no-common-ligatures;color:rgb(47,180,29)">'com.apple.main-thread'</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">, stop reason = </span><span class="gmail-s5" style="font-variant-ligatures:no-common-ligatures;color:rgb(180,36,25)">signal SIGSTOP</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #0: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000109d281b8</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpmpi.12.dylib`MPIDI_POSIX_mpi_barrier_release_gather + 400</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">libpmpi.12.dylib`MPIDI_POSIX_mpi_barrier_release_gather:</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">-><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x109d281b8 <+400>: ldr<span class="gmail-Apple-converted-space"> </span>w9, [x24]</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)"><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x109d281bc <+404>: cmp<span class="gmail-Apple-converted-space"> </span>w8, w9</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)"><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x109d281c0 <+408>: <a href="http://b.lt">b.lt</a> <span class="gmail-Apple-converted-space"> </span>0x109d281a0 <span class="gmail-Apple-converted-space"> </span>; <+376></span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)"><span class="gmail-Apple-converted-space"> </span></span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> 0x109d281c4 <+412>: bl <span class="gmail-Apple-converted-space"> </span>0x109d28e64 <span class="gmail-Apple-converted-space"> </span>; MPID_Progress_test</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">Target 0: (fo_acoustic_streaming_solver_2d) stopped.</span></p>
<p class="gmail-p2" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgba(97,97,97,0.91);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">(lldb) </span><span class="gmail-s6" style="font-variant-ligatures:no-common-ligatures;color:rgb(242,242,242)">bt</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">* thread #1, queue = </span><span class="gmail-s3" style="font-variant-ligatures:no-common-ligatures;color:rgb(47,180,29)">'com.apple.main-thread'</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">, stop reason = </span><span class="gmail-s5" style="font-variant-ligatures:no-common-ligatures;color:rgb(180,36,25)">signal SIGSTOP</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>* frame #0: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000109d281b8</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpmpi.12.dylib`MPIDI_POSIX_mpi_barrier_release_gather + 400</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #1: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000109d27d14</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpmpi.12.dylib`MPIDI_SHM_mpi_barrier + 224</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #2: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000109d27b60</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpmpi.12.dylib`MPIDI_Barrier_intra_composition_alpha + 44</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #3: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000109d0d490</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpmpi.12.dylib`MPIR_Barrier + 900</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #4: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x000000010224d030</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libmpi.12.dylib`MPI_Barrier + 684</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #5: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x00000001045ea638</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpetsc.3.17.dylib`PetscCommDuplicate(comm_in=-2080374782, comm_out=0x000000010300bcb0, first_tag=0x000000010300bce4) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">tagm.c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">235</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">5</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #6: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x00000001045f2910</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpetsc.3.17.dylib`PetscHeaderCreate_Private(h=0x000000010300bc70, classid=1211227, class_name="PetscSF", descr="Star Forest", mansec="PetscSF", comm=-2080374782, destroy=(libpetsc.3.17.dylib`PetscSFDestroy at sf.c:224), view=(libpetsc.3.17.dylib`PetscSFView at sf.c:841)) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">inherit.c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">62</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">3</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #7: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x00000001049cf820</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpetsc.3.17.dylib`PetscSFCreate(comm=-2080374782, sf=0x000000016f911a50) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">sf.c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">62</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">3</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #8: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000104cd3024</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpetsc.3.17.dylib`MatZeroRowsMapLocal_Private(A=0x00000001170c1270, N=1, rows=0x000000016f912cb4, nr=0x000000016f911df8, olrows=0x000000016f911e00) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">zerorows.c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">36</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">5</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #9: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x000000010504ea50</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpetsc.3.17.dylib`MatZeroRows_MPIAIJ(A=0x00000001170c1270, N=1, rows=0x000000016f912cb4, diag=1, x=0x0000000000000000, b=0x0000000000000000) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">mpiaij.c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">768</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">3</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #10: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000104d95fac</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> libpetsc.3.17.dylib`MatZeroRows(mat=0x00000001170c1270, numRows=1, rows=0x000000016f912cb4, diag=1, x=0x0000000000000000, b=0x0000000000000000) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">matrix.c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">5935</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">3</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #11: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x000000010067d320</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> fo_acoustic_streaming_solver_2d`IBAMR::AcousticStreamingPETScMatUtilities::constructPatchLevelFOAcousticStreamingOp(mat=0x000000016f91c178, omega=1, sound_speed=1, rho_idx=3, mu_idx=2, lambda_idx=4, u_bc_coefs=0x000000016f91c3a8, data_time=NaN, num_dofs_per_proc=size=3, u_dof_index_idx=27, p_dof_index_idx=28, patch_level=Pointer<SAMRAI::hier::PatchLevel<2> > @ 0x000000016f914ed0, mu_interp_type=VC_HARMONIC_INTERP) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">AcousticStreamingPETScMatUtilities.cpp</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">794</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">36</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #12: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000100694bdc</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> fo_acoustic_streaming_solver_2d`IBAMR::FOAcousticStreamingPETScLevelSolver::initializeSolverStateSpecialized(this=0x000000016f91c028, x=0x000000016f91d788, (null)=0x000000016f91d690) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">FOAcousticStreamingPETScLevelSolver.cpp</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">149</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">5</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #13: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x000000010083232c</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> fo_acoustic_streaming_solver_2d`IBTK::PETScLevelSolver::initializeSolverState(this=0x000000016f91c028, x=0x000000016f91d788, b=0x000000016f91d690) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">PETScLevelSolver.cpp</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">340</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">5</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #14: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x00000001004eb230</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> fo_acoustic_streaming_solver_2d`main(argc=11, argv=0x000000016f91f460) at </span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(46,174,187)">fo_acoustic_streaming_solver.cpp</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">400</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures">:</span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">22</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:18px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85)"><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-Apple-converted-space"> </span>frame #15: </span><span class="gmail-s4" style="font-variant-ligatures:no-common-ligatures;color:rgb(159,160,28)">0x0000000189fbbf28</span><span class="gmail-s2" style="font-variant-ligatures:no-common-ligatures"> dyld`start + 2236</span></p></span></div><div><span class="gmail_signature_prefix"><br></span></div><div><span class="gmail_signature_prefix"><br></span></div><div><span class="gmail_signature_prefix">Any suggestions on how to avoid this barrier? Here are all MAT options I am using (in the debug mode), if that is helpful:</span></div><div><span class="gmail_signature_prefix"><br></span></div><div><span class="gmail_signature_prefix"><a href="https://github.com/IBAMR/IBAMR/blob/amneetb/acoustically-driven-flows/src/acoustic_streaming/AcousticStreamingPETScMatUtilities.cpp#L453-L458">https://github.com/IBAMR/IBAMR/blob/amneetb/acoustically-driven-flows/src/acoustic_streaming/AcousticStreamingPETScMatUtilities.cpp#L453-L458</a><br></span></div><div><span class="gmail_signature_prefix"><br></span></div><div><span class="gmail_signature_prefix">Thanks, </span></div>-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>--Amneet <br><br></div><div><br></div><div><br></div></div></div></div>