On Fri, Jan 20, 2012 at 1:52 PM, Wen Jiang <span dir="ltr"><<a href="mailto:jiangwen84@gmail.com">jiangwen84@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Jed,<br><br>Could you cover it a bit more details why it will get deadlock unless the number of elements is *exactly* the same on every process? Thanks.<br></blockquote><div><br></div><div>The flush call is collective. Everyone has to call it the same number of times.</div>
<div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Regards,<br>Wen<br><br>Message: 5<br>
Date: Fri, 20 Jan 2012 11:36:17 -0600<br>
From: Jed Brown <<a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a>><br>
Subject: Re: [petsc-users] generate entries on 'wrong' process<br>
To: PETSc users list <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
Message-ID:<br>
<<a href="mailto:CAM9tzSnQvEDOSTrHbbMZVOyOg2%2ByPA2zeYM-ouk1HGr09tE7hA@mail.gmail.com" target="_blank">CAM9tzSnQvEDOSTrHbbMZVOyOg2+yPA2zeYM-ouk1HGr09tE7hA@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
On Fri, Jan 20, 2012 at 11:31, Wen Jiang <<a href="mailto:jiangwen84@gmail.com" target="_blank">jiangwen84@gmail.com</a>> wrote:<br>
<br>
> The serial job is running without any problems and never stalls. Actually<br>
> the parallel jobs also running successfully on distributed-memory desktop<br>
> or on single node of cluster. It will get stuck if it is running on more<br>
> than one compute node(now it is running on two nodes). Both the serial job<br>
> and parallel job (running on distributed or cluster) I mentioned before<br>
> have the same size(dofs). But If I ran a smaller job on cluster with two<br>
> nodes, it might not get stuck and work fine.<br>
><br>
> As you said before, I add MAT_ASSEMBLY_FLUSH after every element stiffness<br>
> matrix is inserted.<br>
><br>
<br>
This will deadlock unless the number of elements is *exactly* the same on<br>
every process.<br>
<br>
<br>
> I got the output like below, and it gets stuck too.<br>
><br><br><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>