Yes that is correct. That is the updated code with each node storing its own values. See my previous email to Matt for the old version which segfaults with processors more than 1 and npts =25.<br><br> - Ron<br><br><div class="gmail_quote">
On Mon, Jul 30, 2012 at 4:43 PM, Jed Brown <span dir="ltr"><<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I ran<div><br></div><div>$ mpirun -n 2 ./a.out</div><div><div> ---------------------------------------------------</div>
<div> Running MAS PETSc tests with:</div><div> nr: 23</div><div> np: 27</div>
<div> nt: 25</div><div> N: 46575</div><div> num_steps: 2000</div><div> MPI cores: 2</div><div> ---------------------------------------------------</div><div>
<div> MPI TEST: My rank is: 0</div>
<div> MPI TEST: My rank is: 1</div></div><div> Rank 0 has rows 0 to 23288</div><div> Rank 1 has rows 23288 to 46575</div><div><div> Number of non-zero entries in matrix: 690339</div>
</div><div> Computing y=Ax with RON_CSR_AX...</div><div> ...done!</div><div> ||y||: 821.67460825997637 </div><div> y(5)= 2.7454534359667053 </div><div> Storing MAS matrix into PETSc matrix...</div><div> ...done!</div>
<div> rank 1 about to call MatAssemblyEnd()...</div><div> rank 0 about to call MatAssemblyEnd()...</div><div> Computing y=Ax with PETSc MatMult...</div><div> ...done!</div><div> ||y||: 821.67460825997568 </div>
<div> y(5)= 2.7454534359667053 </div><div> RON_CSR_AX y=Ax time: 201.704498 nsec/mp.</div><div> PETSc y=Ax time: 198.269424 nsec/mp.</div><div> PETSc y=Ax flops: 0.417739183 GFLOPS.</div>
<div> Did not converge! Number of iteratioins: 0</div><div><div><div><br></div><br><div class="gmail_quote">On Mon, Jul 30, 2012 at 3:12 PM, Ronald M. Caplan <span dir="ltr"><<a href="mailto:caplanr@predsci.com" target="_blank">caplanr@predsci.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Attached is the code. The original code which segfaults with more than one core is the code I sent last week.<br><br>
- Ron C<div>
<div><br><br><div class="gmail_quote">On Mon, Jul 30, 2012 at 3:09 PM, Jed Brown <span dir="ltr"><<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div>On Mon, Jul 30, 2012 at 3:04 PM, Ronald M. Caplan <span dir="ltr"><<a href="mailto:caplanr@predsci.com" target="_blank">caplanr@predsci.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>I seem to have solved the problem.<br><br>I was storing my entire matrix on node 0 and then calling MatAssembly (begin and end) on all nodes (which should have worked...). <br><br>Apparently I was using too much space for the buffering or the like, because when I change the code so each node sets its own matrix values, than the MatAssemblyEnd does not seg fault. <br>
</div></blockquote><div><br></div></div><div>Can you send the test case. It shouldn't seg-fault unless the machine runs out of memory (and most desktop systems have overcommit, so the system will kill arbitrary processes, not necessarily the job that did the latest malloc.</div>
<div><br></div><div>In practice, you should call MatAssemblyBegin(...,MAT_FLUSH_ASSEMBLY) periodically.</div><div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<br>Why should this be the case? How many elements of a vector or matrix can a single node "set" before Assembly to distribute over all nodes?</div></blockquote></div></div><br><div><br></div>
</blockquote></div><br>
</div></div></blockquote></div><br></div></div></div>
</blockquote></div><br>