[petsc-dev] Question re parallel sparse Matrix-vector / KSP solvers

[Barry Smith]

On May 13, 2011, at 10:49 AM, David Henty wrote:

> I see in the example for how to solve A.x_i = b_i (ie multiple right-hand-sides for the same matrix), it simply loops over multiple separate KSP calls.

   Correct.
> 
> Wouldn't there be some benefit in having a matrix-vector routine that computed y_i = A.x_i for multiple "i" values?

   Absolutely!

> The parallel overhead comes from communicating the off-process x values. As this is probably latency dominated (especially as we go to many processes), the comms cost of doing the communications would rise quite slowly for additional vectors. Likewise, the cache utilisation of the sparse A matrix entries would be helped by doing sevefal multiplications at once.

   Completely correct.

> 
> Has this already been implemented, or maybe in the pipleline?

    No, no. 

> Or am I somehow missing the point ...

   No. You are correct that solving "multiple right hand side systems" can offer a good improvement over solving them separately.    To "do it right" with PETSc would require a major overhaul in the PETSc "plumbing", we've never had the resources nor the driving motivating applications to make a push in this direction. 

   Barry

There is a technical sticking point that I think is non-trivial. For block Krylov methods it can happen that the elements of the generated Krylov space at an iteration become not linearly independent. Care has to be taken to detect this cases properly (numerical issues of when things are linearly dependent are potentially tricky to compute) and handle them efficiently. Presumably these issues could be overcome with careful work.


   
> 
> Thanks!
> 
> David
> 
> -- 
> Dr David Henty                        EPCC, The University of Edinburgh
> HPC Training and Support              Edinburgh EH9 3JZ, UK
> d.henty at epcc.ed.ac.uk                 Tel: +44 (0)131 650 5960
> http://www.epcc.ed.ac.uk/~dsh/        Fax: +44 (0)131 650 6555
> 
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>