[petsc-users] SLEPC: partitioning for CISS

[Povolotskyi, Mykhailo]

Hello,
I'm currently using CISS via SLEPc.

I would like to use parallelization by partitioning over quadrature points, using the option -eps_ciss_partitions.

I have done the following:

1. created a matrix on each MPI rank

 MatCreateDense(MPI_COMM_SELF, matrix_size, matrix_size, PETSC_DECIDE,PETSC_DECIDE,NULL,&matrix_petsc);

2. Created EPS object

EPSCreate(MPI_COMM_WORLD,&eps);
EPSSetOperators( eps,matrix_petsc,NULL);
EPSSetType(eps,EPSCISS);
EPSSetProblemType(eps, EPS_NHEP);

EPSSetFromOptions(eps);


EPSGetRG(eps,&rg);
RGSetType(rg,RGRING);

RGRingSetParameters(rg,center,radius,vscale,start_ang,end_ang,width);

EPSSolve(eps);

3. Then I run the code:

mpiexec -n 2 ./a.out -eps_ciss_partitions 2

The code gives an error:

[1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[1]PETSC ERROR: Arguments are incompatible
[1]PETSC ERROR: MatMatMultSymbolic requires A, seqdense, to be compatible with B, mpidense
[1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[1]PETSC ERROR: Petsc Release Version 3.8.4, Mar, 24, 2018
[1]PETSC ERROR: ../lib/a.out on a linux-complex named brown-a337.rcac.purdue.edu by mpovolot Tue Sep 10 23:43:44 2019
[1]PETSC ERROR: Configure options --with-scalar-type=complex --with-x=0 --with-hdf5 --download-hdf5=1 --with-single-library=1 --with-pic=1 --with-shared-libraries=0 --with-log=0 --with-clanguage=C++ --CXXFLAGS="-fopenmp -fPIC" --CFLAGS="-fopenmp -fPIC" --with-fortran=0 --FFLAGS="-fopenmp -fPIC" --with-debugging=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx COPTFLAGS= CXXOPTFLAGS= FOPTFLAGS= --download-metis=1 --download-parmetis=1 --with-valgrind-dir=/apps/brown/valgrind/3.13.0_gcc-4.8.5 --download-mumps=1 --with-fortran-kernels=0 --download-superlu_dist=1 --with-blaslapack-lib="-L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core " --with-blacs-lib=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so --with-blacs-include=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/include --with-scalapack-lib="-Wl,-rpath,/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core  -lpthread  -L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64" --with-scalapack-include=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/include
[1]PETSC ERROR: #463 MatMatMultSymbolic() line 9692 in /depot/kildisha/apps/brown/nemo5/libs/petsc/build-cplx/src/mat/interface/matrix.c
[1]PETSC ERROR: #464 BVMatMult_Svec() line 229 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/impls/svec/svec.c
[1]PETSC ERROR: #465 BVMatMult() line 589 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/interface/bvops.c
[1]PETSC ERROR: #466 BVMatProject_MatMult() line 903 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/interface/bvglobal.c
[1]PETSC ERROR: #467 BVMatProject() line 1151 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/sys/classes/bv/interface/bvglobal.c
[1]PETSC ERROR: #468 EPSSolve_CISS() line 1066 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/eps/impls/ciss/ciss.c
[1]PETSC ERROR: #469 EPSSolve() line 147 in /depot/kildisha/apps/brown/nemo5/libs/slepc/build-cplx/src/eps/interface/epssolve.c

Could you, please, tell me what am I doing wrong?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190911/95b38951/attachment-0001.html>