[petsc-users] Code speedup after upgrading

[Lawrence Mitchell]

> On 24 Mar 2021, at 01:30, Matthew Knepley <knepley at gmail.com> wrote:
> 
> This is true, but all the PETSc operations are speeding up by a factor 2x. It is hard to believe these were run on the same machine.
> For example, VecScale speeds up!?!  So it is not network, or optimizations. I cannot explain this.
> 

VecMDot speeds up by a factor of 8!

Unrelatedly, one thing I see, which _may_ offer potential for much more speedup, is this:

BuildTwoSided      17548 1.0 4.9331e+00 9.9 0.00e+00 0.0 5.9e+05 4.0e+00 1.8e+04
BuildTwoSidedF     17547 1.0 5.0489e+00 7.3 0.00e+00 0.0 1.2e+06 3.6e+03 1.8e+04

...

MatAssemblyBegin   17547 1.0 8.8252e+00 1.1 0.00e+00 0.0 1.2e+06 3.6e+03 1.8e+04
MatAssemblyEnd     17547 1.0 2.6903e+00 2.8 2.79e+07 2.7 2.1e+02 2.0e+02 1.0e+01

I think these BuildTwoSided calls are coming from the MatAssemblyBegin/End pairs.

If you preallocate and fill your matrices with zeros in all the possible places that you might end up putting a non-zero, then calling

MatSetOption(mat, MAT_SUBSET_OFF_PROC_ENTRIES, PETSC_TRUE)

on the matrix you create will reduce this time in BuildTwoSided to almost zero.

Lawrence