[petsc-dev] [GPU] SEGV with: mpiexec -np 2 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi -sub_pc_type jacobi -mat_type aijcusp -vec_type cusp

[Karl Rupp]

Hi,


 > Another crash when using ex2 (src/ksp/ksp/examples/tutorials) in
> parallel on GPU, when playing with
> precondtionners:
>
> mpiexec -np 1 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi
> -sub_pc_type jacobi
> Norm of error 5.71895e-06 iterations 14
> mpiexec -np 1 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi
> -sub_pc_type jacobi -mat_type aijcusp -vec_type cusp
> Norm of error 5.71895e-06 iterations 14
> mpiexec -np 2 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi
> -sub_pc_type jacobi
> Norm of error 5.71895e-06 iterations 14
> mpiexec -np 2 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type bjacobi
> -sub_pc_type jacobi -mat_type aijcusp -vec_type cusp
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
> probably memory access out of range
> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [0]PETSC ERROR: or see
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC
> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to
> find memory corruption errors
> [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link,
> and run
> [0]PETSC ERROR: to get more information on the crash.
> [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
>
> Whereas the last one should work exactly as:
> mpiexec -np 2 ./ex2 -ksp_type cg -m 10 -n 10 -pc_type jacobi -mat_type
> aijcusp -vec_type cusp
> Norm of error 5.71895e-06 iterations 14
>
> Or I miss something ?

this is a known problem with block-jacobi for which Dominic created a 
pull request here:
https://bitbucket.org/petsc/petsc/pull-request/178/make-cusp-and-cusparse-solver-packages

I was talking to Dominic yesterday and we plan to get the series of 
GPU-related pull requests merged into next and master within the next 
couple of days. If it's urgent, please try that feature branch in the 
meanwhile.

Best regards,
Karli