[petsc-users] possible performance issues with PETSc on Cray

[Jed Brown]

Samar Khatiwala <spk at ldeo.columbia.edu> writes:
> CRAY has finally gotten back with a solution,
> which is to use the run-time option -vecscatter_alltoall. Apparently
> this is a known issue and according to the HLRN folks passing this
> command line option to PETSc seems to work nicely.

This option is good when you have nearly-dense rows or columns (in terms
of processors depended on).  For problems with actual dense rows or
columns, it is good to formulate as a sparse matrix plus a low-rank
correction.  The other cases are usually poor dof layout, and reordering
will make the graph sparser.  Sparse problems with good layout usually
run faster with the default VecScatter, though there are exceptions
(mostly non-PDE problems).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140509/11387ee0/attachment.pgp>