On Fri, Sep 3, 2010 at 5:50 PM, Leo van Kampenhout <span dir="ltr"><<a href="mailto:lvankampenhout@gmail.com">lvankampenhout@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Thanks. Following this conversation closely, i have already built petsc with --with-debugging=0 and found that then thing work better there. Until there is a ultimate fix I'm perfectly happy with that. <br><br>Even on my cluster, I can now run the program for 2,4,8 cores. But there is always a catch... with 10 cores, GMRES breaks down! (see below) Sorry to bother you with this, it seems unrelated at first sight. Should I issue a new thread for this?<br>
</blockquote><div><br></div><div>This is almost certainly somewhere you are not careful about constructing</div><div>a Vec with the proper layout (as the message says). In fact, from the trace</div><div>it looks like the rhs Vec passed in is not compatible with the matrix.</div>
<div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Regards, Leo<br><br>[5]PETSC ERROR: --------------------- Error Message ------------------------------------<br>
[5]PETSC ERROR: Nonconforming object sizes!<br>[5]PETSC ERROR: Mat mat,Vec y: local dim 210161 213282!<br>
[5]PETSC ERROR: ------------------------------------------------------------------------<br>[5]PETSC ERROR: Petsc Development HG revision: 6da7d8c5aaa81099861622f4e6bf1087652253a7 HG Date: Fri Sep 03 15:06:40 2010 +0200<br>
[5]PETSC ERROR: See docs/changes/index.html for recent updates.<br>[5]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>[5]PETSC ERROR: See docs/index.html for manual pages.<br>[5]PETSC ERROR: ------------------------------------------------------------------------<br>
[5]PETSC ERROR: ./main on a linux-opt named node164 by s1459295 Fri Sep 3 17:44:26 2010<br>[5]PETSC ERROR: Libraries linked from /home/s1459295/install/petsc-dev/linux-optimized/lib<br>[5]PETSC ERROR: Configure run at Fri Sep 3 15:28:20 2010<br>
[5]PETSC ERROR: Configure options --with-debugging=0<br>[5]PETSC ERROR: ------------------------------------------------------------------------<br>[5]PETSC ERROR: MatMult() line 1976 in src/mat/interface/matrix.c<br>[5]PETSC ERROR: KSPInitialResidual() line 55 in src/ksp/ksp/interface/itres.c<br>
[5]PETSC ERROR: KSPSolve_GMRES() line 236 in src/ksp/ksp/impls/gmres/gmres.c<br>[5]PETSC ERROR: KSPSolve() line 427 in src/ksp/ksp/interface/itfunc.c<br><br><br><br><br><br><div class="gmail_quote">2010/9/3 Jed Brown <span dir="ltr"><<a href="mailto:jed@59a2.org" target="_blank">jed@59a2.org</a>></span><div>
<div></div><div class="h5"><br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex"><div>On Fri, 3 Sep 2010 09:15:09 -0600 (GMT-06:00), Shri <<a href="mailto:abhyshr@mcs.anl.gov" target="_blank">abhyshr@mcs.anl.gov</a>> wrote:<br>
> Leo,<br>
> Jed and I both tried running a MatLoad example (see src/dm/da/examples/tests/ex35.c) with petsc configured in debug mode and had issues running it with your grid size 202 X 102 X 102. This example runs in a reasonable amount of time with the optimized petsc build.<br>
> I suggest running your code on a smaller grid (100 X100 X 100 seems to work fine in the debug mode) for now, get it working and then using the optimized petsc build (--debugging=0) for the large grid size.<br>
<br>
</div>I wrote an improved sort that doesn't have this bad behavior, but I'm<br>
trying to figure out why the old one is okay in optimized mode. See<br>
here:<br>
<br>
<a href="http://gist.github.com/563948" target="_blank">http://gist.github.com/563948</a><br>
<br>
The MEDIAN macro chooses the median of the first, last, and middle<br>
element in the segment. Performance looks like<br>
<br>
FAST -O1: 2.0 seconds<br>
FAST -O0: 5 seconds<br>
SLOW -O1: 2.3 seconds<br>
SLOW -O0: more than an hour<br>
<br>
SLOW, which is basically the old implementation does a lot of shuffling<br>
with the array and ends up hitting a bad case for quicksort (even with<br>
the median choice). What I don't understand, is that apparently<br>
building SLOW with -O1 transforms the algorithm into something<br>
asymptotically faster. The assembly is a few hundred lines, and it's<br>
not obvious to me what this transform would be.<br>
<font color="#888888"><br>
Jed<br>
</font></blockquote></div></div></div><br>
</blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<br>