On Mon, Dec 20, 2010 at 8:46 AM, Yongjun Chen <span dir="ltr"><<a href="mailto:yjxd.chen@gmail.com">yjxd.chen@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
<p class="MsoNormal"><span lang="DE">Hi everyone,</span></p><p class="MsoNormal"><span lang="DE"><br></span></p>
<p class="MsoNormal"><span lang="DE">I use PETSC
(version 3.1-p5) to solve a linear problem Ax=b. The matrix A and right hand
vector b are read from files. The dimension of A is 1.2Million*1.2Million. I am
pretty sure the matrix A and vector b have been read correctly. </span></p>
<p class="MsoNormal"><span lang="DE">I compiled the
program with optimized version (--with-debugging=0), tested the speed up
performance on two servers, and I have found that the performance is very poor.</span></p>
<p class="MsoNormal"><span lang="DE">For the two
servers, one is 4 cpus * 4 cores per cpu, i.e., with a total 16 cores. And the
other one is 4 cpus * 12 cores per cpu, with a total 48 cores.</span></p>
<p class="MsoNormal"><span lang="DE">On each of them, with
the increasing of computing cores k from 1 to 8 (mpiexec –n k ./Solver_MPI -pc_type jacobi -ksp-type gmres), the speed
up will increase from 1 to 6, but when the computing cores k increase from 9 to
16(for the first server) or 48 (for the second server), the speed up decrease firstly
and then remains a constant value 5.0 (for the first server) or 4.5(for the second
server).</span></p></blockquote><div><br></div><div>We cannot say anything at all without -log_summary data for your runs.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE">Actually, the
program LAMMPS speed up excellently on these two servers.</span></p>
<p class="MsoNormal"><span lang="DE">Any comments are
very appreciated! Thanks!<br></span></p>
<p class="MsoNormal"><span lang="DE"> </span></p><p class="MsoNormal"><span lang="DE">--------------------------------------------------------------------------------------------------------------------------<br>
</span></p>
<p class="MsoNormal"><span lang="DE">PS: the related
codes are as following,</span></p>
<p class="MsoNormal"><span lang="DE"><br></span></p><p class="MsoNormal"><span lang="DE">//firstly read A and b from
files</span></p>
<p class="MsoNormal"><span lang="DE">...</span></p><p class="MsoNormal"><span lang="DE">//then<br></span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = VecAssemblyBegin(b); CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = VecAssemblyEnd(b); CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = MatSetOption(A,MAT_SYMMETRIC,PETSC_TRUE); CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = MatGetRowUpperTriangular(A); CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = KSPCreate(PETSC_COMM_WORLD,&ksp);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = KSPGetPC(ksp,&pc);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr =
KSPSetTolerances(ksp,1.e-7,PETSC_DEFAULT,PETSC_DEFAULT,PETSC_DEFAULT);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = KSPSolve(ksp,b,x);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = KSPView(ksp,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = KSPGetSolution(ksp, &x);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = VecAssemblyBegin(x);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE"><span> </span><span>
</span>ierr = VecAssemblyEnd(x);CHKERRQ(ierr);</span></p>
<p class="MsoNormal"><span lang="DE">...</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<br>
</blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<br>