[petsc-users] MatSetValues error with ViennaCL types

[Manuel Valera]

Hello PETSc devs,

I am running into an error when trying to use the MATMPIAIJVIENNACL Matrix
type in MPI calls, the same code runs for MATSEQAIJVIENNACL type in one
processor. The error happens when calling MatSetValues for this specific
configuration. It does not occur when using MPI DMMatrix types only.

Any help will be appreciated,

Thanks,



My program call:

mpirun -n 2 ./gcmLEP.GPU tc=TestCases/LockRelease/LE_6x6x6/
jid=tiny_cuda_test_n2 -ksp_type cg -dm_vec_type viennacl -dm_mat_type
aijviennacl -pc_type saviennacl -log_view


The error (repeats after each MatSetValues call):

[1]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[1]PETSC ERROR: Argument out of range
[1]PETSC ERROR: Inserting a new nonzero at global row/column (75, 50) into
matrix
[1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
trouble shooting.
[1]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53  GIT
Date: 2018-05-31 17:31:13 +0300
[1]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera Wed Aug
15 13:10:44 2018
[1]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug
--with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
--FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
--CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64 --download-viennacl
[1]PETSC ERROR: #1 MatSetValues_MPIAIJ() line 608 in
/home/valera/petsc/src/mat/impls/aij/mpi/mpiaij.c
[1]PETSC ERROR: #2 MatSetValues() line 1339 in
/home/valera/petsc/src/mat/interface/matrix.c


My Code structure:

call DMCreateMatrix(daDummy,A,ierr)
> call MatSetFromOptions(A,ierr)
> call MPI_Comm_size(PETSC_COMM_WORLD, numprocs, ierr)
> if (numprocs > 1) then  ! set matrix type parallel
>     ! Get local size
>     call DMDACreateNaturalVector(daDummy,Tmpnat,ierr)
>     call VecGetLocalSize(Tmpnat,locsize,ierr)
>     call VecDestroy(Tmpnat,ierr)
>     ! Set matrix
> #ifdef GPU
>     call MatSetType(A,MATAIJVIENNACL,ierr)
>     call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr)
>     call DMSetVecType(daDummy,VECMPIVIENNACL,ierr)
>     print*,'SETTING GPU TYPES'
> #else
>     call DMSetMatType(daDummy,MATMPIAIJ,ierr)
>     call DMSetMatType(daDummy,VECMPI,ierr)
>     call MatSetType(A,MATMPIAIJ,ierr)!
> #endif
>     call
> MatMPIAIJSetPreallocation(A,19,PETSC_NULL_INTEGER,19,PETSC_NULL_INTEGER,ierr)
> else                    ! set matrix type sequential
> #ifdef GPU
>     call DMSetMatType(daDummy,MATSEQAIJVIENNACL,ierr)
>     call DMSetVecType(daDummy,VECSEQVIENNACL,ierr)
>     call MatSetType(A,MATSEQAIJVIENNACL,ierr)
>     print*,'SETTING GPU TYPES'
> #else
>     call DMSetMatType(daDummy,MATSEQAIJ,ierr)
>     call DMSetMatType(daDummy,VECSEQ,ierr)
>     call MatSetType(A,MATSEQAIJ,ierr)
> #endif
> call MatSetUp(A,ierr)
> call getCenterInfo(daGrid,xstart,ystart,zstart,xend,yend,zend)
>


> do k=zstart,zend-1
>     do j=ystart,yend-1
>         do i=xstart,xend-1
> [..]
>            call
> MatSetValues(A,1,row,sumpos,pos(0:iter-1),vals(0:iter-1),INSERT_VALUES,ierr)
> [..]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20180815/69a805ad/attachment.html>