[petsc-dev] trying to make MKL work with GAMG

[Richard Tran Mills]

Hi Mark,

I'd never looked at the code for MatCreateMPIAIJWithSeqAIJ(), but it looks
like if you are using MPIAIJ matrices but you've set "-mat_seqaij_type
seqaijmkl", then the "diagonal" portion of the MPIAIJ matrix ends up as a
SEQAIJMKL instance, but the off-diagonal portion ends up as a plain SEQAIJ
when it is created with MatCreateSeqAIJWithArrays(). (I have not stepped
through this with a debugger to verify that this is actually what happens.)
So, if we want the off-diagonal matrix to also be a SEQAIJMKL, then we
either need to add some logic after the MatCreateSeqAIJWithArrays() call to
check to see if a different subtype of SEQAIJ is being used and then
convert it to the correct type, or we need a variant of
MatCreateSeqAIJWithArrays() that will honor the subtype of SEQAIJ that we
are using. Barry, since you added the concept of subtyping for SEQAIJ, what
do you prefer?

I note that in some (most?) cases, I think we are OK or even better off
with the off-diagonal matrix in the MPIAIJ not being a SEQAIJMKL one,
because the PETSc routines can used the "compressed row" stuff to speed up
handling the many zero rows that often end up in the off-diagonal, whereas
the MKL routines cannot.

--Richard

On Tue, Jul 3, 2018 at 4:31 PM, Mark Adams <mfadams at lbl.gov> wrote:

> I have having to fix AIJ methods that don't get the type from the args to
> set created matrix type, so as to keep the MKL typing.
>
> I am not sure how to fix this because I don't know to say 'make MPI from
> SEQ' in this method
>
> PetscErrorCode MatCreateMPIAIJWithSeqAIJ(MPI_Comm comm,Mat A,Mat B,const
> PetscInt garray[],Mat *mat)
> {
>  ....
>   ierr = MatCreate(comm,mat);CHKERRQ(ierr);
>   ierr = MatGetSize(A,&m,&n);CHKERRQ(ierr);
> ....
>   ierr = MatSetSizes(*mat,m,n,PETSC_DECIDE,N);CHKERRQ(ierr);
>   ierr = MatSetType(*mat,MATMPIAIJ);CHKERRQ(ierr);
>
> How should I do this?  Other places in the code I get the type from input
> (MPI) args and set created matrix types accordingly. I could just put a
> switch with an error for any new types that might come along in the future
> ...
>
> Mark
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180704/24557f9d/attachment-0001.html>