[petsc-users] Mixed sparse and dense CG

[Ross Brodie]

Hello PETSc Users

I have previously successfully tackled the problem of solving 

(J'DJ + aL'DL + cK'EK) x = b 

where real J,  L, K were sparse matrices and D and E diagonal matrices and a and c real scalars using PETSc by using a shell matrix approach, i.e., never actually explicitly forming J'DJ etc but but supplying the function to KSP (PCG) to compute the products using PETSc functions with a vector on the fly. 

This did the trick for sparse J.  I am pretty sure this can be done with a dense J, in PETSc as well just a lot slower.  J could be the order of 10e5 x 10e6.

I have considered using libElemental for the dense part (to make use of the grid cyclic layout)  and PETSc for the sparse part of the PCG matvec product but I am not sure it is wise to mix packages.

I also note that the recent release of PETSc includes an interface to Elemental via the MATELEMENTAL matrix type.  Would this be a good choice?

On a different note could you clarify the terminology for the sequential vector VECSEQ. Does it mean:  (i) It is an independent uniprocessor object such that the data storage is not distributed and if you change an element on one process the change is NOT reflected on other processes, or (ii) does it mean the data storage is not distributed but it is not necessarily uniprocessor independent (ie such that each processor carries around redundant copies of the same data and changes on one process ARE reflected on other processes)  depending on whether or not it is constructed using PETSC_COMM_SELF or PETSC_COMM_WORLD.  Does it ever make sense to construct a VECSEQ with anything other than PETSC_COMM_SELF?  I suspect it is (i) but want to make sure?

Ross