[petsc-users] Scalability of PETSc on vesta.alcf

[Jed Brown]

Roc Wang <pengxwang at hotmail.com> writes:

> Hello, 
>
>    I am testing a petsc program on vesta.alcf.acl.gov. The scalability
>    was fine when the number of ranks is less then 1024. However, when
>    the 2048 ranks were used, 

Are you using c64 mode in all cases or did you run smaller fewer
processes per node out to 1024?  You can't do fair scaling with
different modes because BG/Q has only 16 cores per node.  The four
hardware threads per core only cover latency, but do not significantly
improve memory bandwidth.  We're seeing most of the time in MatSolve,
which does no communication.  (Also MatMult, but your large fill makes
the factors much heavier than the matrix itself.)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140120/edbcdcde/attachment.pgp>