Slow speed after changing from serial to parallel
Satish Balay
balay at mcs.anl.gov
Fri Apr 18 00:52:14 CDT 2008
On Fri, 18 Apr 2008, Ben Tay wrote:
> Hi,
>
> I've email my school super computing staff and they told me that the queue which I'm using is one meant for testing, hence, it's
> handling of work load is not good. I've sent my job to another queue and it's run on 4 processors. It's my own code because there seems
> to be something wrong with the server displaying the summary when using -log_summary with ex2f.F. I'm trying it again.
Thats wierd. We should first make sure ex2f [or ex2] are running
properly before looking at your code.
>
> Anyway comparing just kspsolve between the two, the speedup is about 2.7. However, I noticed that for the 4 processors one, its
> MatAssemblyBegin is 1.5158e+02, which is more than KSPSolve's 4.7041e+00. So is MatAssemblyBegin's time included in KSPSolve? If not,
> does it mean that there's something wrong about my MatAssemblyBegin?
MatAssemblyBegin is not included in KSPSolve(). Something wierd is
going here. There are 2 possibilities.
- whatever code you have before matrix assembly is unbalanced, so
MatAssemblyBegin() acts as a barrier .
- MPI communication is not optimal within the node.
Its best to first make sure ex2 or ex2f runs fine. As recommended
earlier - you should try latest mpich2 with --with-device=ch3:nemesis:newtcp
and compare ex2/ex2f performance with your current MPI.
Satish
More information about the petsc-users
mailing list