On Thu, Jul 14, 2011 at 1:44 AM, Clemens Domanig <span dir="ltr"><<a href="mailto:clemens.domanig@uibk.ac.at">clemens.domanig@uibk.ac.at</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
I know it is lots of values I send but this will only run on a shared memory system. And to me it is strange that it is just ~20s with 3+ MPI-proc.<br></blockquote><div><br></div><div>You could have hit a level in the memory hierarchy, where it starts swapping to disk. To definitely say would take a lot of work. However,</div>
<div>its very easy to definitely say whether there is deadlock. As both Barry and Jed said, please connect with the debugger and look at the </div><div>stack trace.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Matthew Knepley wrote:<div><div></div><div class="h5"><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Wed, Jul 13, 2011 at 4:56 PM, Clemens Domanig <<a href="mailto:clemens.domanig@uibk.ac.at" target="_blank">clemens.domanig@uibk.ac.at</a> <mailto:<a href="mailto:clemens.domanig@uibk.ac.at" target="_blank">clemens.domanig@uibk.<u></u>ac.at</a>>> wrote:<br>
<br>
I tried with -mat_no_inode - no effect. Thats the output<br>
<br>
[1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>
[0] MatStashScatterBegin_Private()<u></u>__: No of messages: 1<br>
[0] MatStashScatterBegin_Private()<u></u>__: Mesg_to: 1: size: 704692232<br>
<br>
^^^^^^^^^^ Do you really mean to set 700M of off-process values?<br>
<br>
I think Barry is correct that it is just taking forever to send this.<br>
<br>
Matt<br>
<br>
[0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries, uses 13<br>
mallocs.<br>
[0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824; storage<br>
space: 24984360 unneeded,19875384 used<br>
[0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()<br>
is 0<br>
[0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42<br>
<br>
<br>
Am 2011-07-13 22:10, schrieb Matthew Knepley:<br>
<br>
On Wed, Jul 13, 2011 at 4:08 PM, Clemens Domanig<br>
<<a href="mailto:clemens.domanig@uibk.ac.at" target="_blank">clemens.domanig@uibk.ac.at</a> <mailto:<a href="mailto:clemens.domanig@uibk.ac.at" target="_blank">clemens.domanig@uibk.<u></u>ac.at</a>><br>
<mailto:<a href="mailto:clemens.domanig@uibk." target="_blank">clemens.domanig@uibk.</a>_<u></u>_<a href="http://ac.at" target="_blank">ac.at</a><br>
<mailto:<a href="mailto:clemens.domanig@uibk.ac.at" target="_blank">clemens.domanig@uibk.<u></u>ac.at</a>>>> wrote:<br>
<br>
Hi everyone,<br>
<br>
maybe some can offer som debugging-hints for my problem.<br>
<br>
<br>
Its possible that there is a bug in the inode routines. Please try<br>
running with -mat_no_inode<br>
<br>
Thanks,<br>
<br>
Matt<br>
<br>
My FEM-program uses a shell-element that has depending on the<br>
geometry 5 or 6 dof per node.<br>
<br>
The program uses MPI for parallel solving (LU, mumps).<br>
It works fine with all examples that have onyl 5 dof per node and<br>
that have a mixture of 5 and 6 dof per node.<br>
When doing examples that have 6 dof per node this happens:<br>
* when using more than 2 MPI processes everything seems to be<br>
fine.<br>
* when using 1 or 2 MPI processes MatAssemblyBegin() never<br>
finishes<br>
<br>
This is the last output of -info, -mat_view_info, -vec_view_info<br>
(with 2 MPI processes, matrix size 1107648x1107648)<br>
<br>
[1] MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0<br>
mallocs.<br>
[0] MatStashScatterBegin_Private()<u></u>____: No of messages: 1<br>
[0] MatStashScatterBegin_Private()<u></u>____: Mesg_to: 1: size:<br>
704692232<br>
[0] MatAssemblyBegin_MPIAIJ(): Stash has 88086528 entries,<br>
uses 13<br>
mallocs.<br>
[0] MatAssemblyEnd_SeqAIJ(): Matrix size: 553824 X 553824;<br>
storage<br>
space: 24984360 unneeded,19875384 used<br>
[0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during<br>
MatSetValues()<br>
is 0<br>
[0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 42<br>
[0] Mat_CheckInode(): Found 184608 nodes of 553824. Limit<br>
used: 5.<br>
Using Inode routines<br>
<br>
Thx for your help - respectfully C. Domanig<br>
<br>
<br>
<br>
<br>
--<br>
What most experimenters take for granted before they begin their<br>
experiments is infinitely more interesting than any results to which<br>
their experiments lead.<br>
-- Norbert Wiener<br>
<br>
<br>
<br>
<br>
<br>
-- <br>
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</blockquote>
<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>