[petsc-users] MatSetValues error with ViennaCL types
Manuel Valera
mvalera-w at sdsu.edu
Wed Aug 29 16:21:30 CDT 2018
Ok, executing with:
mpirun -n 2 ./gcmLEP.GPU tc=TestCases/LockRelease/LE_6x6x6/
jid=tiny_cuda_test_n1 -pc_type bjacobi -pc_sub_type saviennacl -ksp_view
I get:
SETTING GPU TYPES
Matrix type: mpiaijviennacl
Of sizes: 125 x 125
Matrix type: mpiaijviennacl
Of sizes: 125 x 125
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: No support for this operation for this object type
[0]PETSC ERROR: Currently only handles ViennaCL matrices
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.9.3-1264-g6fb0cd4 GIT
Date: 2018-08-29 12:42:36 -0600
[0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera Wed Aug
29 14:20:42 2018
[0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug --with-cc=mpicc
--with-cxx=mpic++ --with-fc=mpifort --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
--FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
--CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
--download-viennacl --download-cusp
[0]PETSC ERROR: #1 PCSetUp_SAVIENNACL() line 47 in
/home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu
[0]PETSC ERROR: #2 PCSetUp() line 932 in
/home/valera/petsc/src/ksp/pc/interface/precon.c
[1]PETSC ERROR:
------------------------------------------------------------------------
[1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
probably memory access out of range
[1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[1]PETSC ERROR: or see
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X
to find memory corruption errors
[1]PETSC ERROR: likely location of problem given in stack below
[1]PETSC ERROR: --------------------- Stack Frames
------------------------------------
[1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[1]PETSC ERROR: INSTEAD the line number of the start of the function
[1]PETSC ERROR: is given.
[1]PETSC ERROR: [1] PetscTraceBackErrorHandler line 182
/home/valera/petsc/src/sys/error/errtrace.c
[1]PETSC ERROR: [1] PetscError line 352
/home/valera/petsc/src/sys/error/err.c
[1]PETSC ERROR: [1] PCSetUp_SAVIENNACL line 45
/home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu
[1]PETSC ERROR: [1] PCSetUp line 894
/home/valera/petsc/src/ksp/pc/interface/precon.c
[1]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[1]PETSC ERROR: Signal received
[1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
trouble shooting.
[1]PETSC ERROR: Petsc Development GIT revision: v3.9.3-1264-g6fb0cd4 GIT
Date: 2018-08-29 12:42:36 -0600
[1]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera Wed Aug
29 14:20:42 2018
[1]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug --with-cc=mpicc
--with-cxx=mpic++ --with-fc=mpifort --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
--FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
--CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
--download-viennacl --download-cusp
[1]PETSC ERROR: #1 User provided function() line 0 in unknown file
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 59.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[0]PETSC ERROR:
------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the
batch system) has told this process to end
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X
to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: --------------------- Stack Frames
------------------------------------
[0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[0]PETSC ERROR: INSTEAD the line number of the start of the function
[0]PETSC ERROR: is given.
[0]PETSC ERROR: [0] PetscCommDuplicate line 130
/home/valera/petsc/src/sys/objects/tagm.c
[0]PETSC ERROR: [0] PetscHeaderCreate_Private line 41
/home/valera/petsc/src/sys/objects/inherit.c
[0]PETSC ERROR: [0] ISCreate line 35
/home/valera/petsc/src/vec/is/is/interface/isreg.c
[0]PETSC ERROR: [0] ISCreateGeneral line 658
/home/valera/petsc/src/vec/is/is/impls/general/general.c
[0]PETSC ERROR: [0] PCSetUp_SAVIENNACL line 45
/home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu
[0]PETSC ERROR: [0] PCSetUp line 894
/home/valera/petsc/src/ksp/pc/interface/precon.c
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.9.3-1264-g6fb0cd4 GIT
Date: 2018-08-29 12:42:36 -0600
[0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera Wed Aug
29 14:20:42 2018
[0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug --with-cc=mpicc
--with-cxx=mpic++ --with-fc=mpifort --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
--FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
--CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
--download-viennacl --download-cusp
[0]PETSC ERROR: #3 User provided function() line 0 in unknown file
[node50:77836] 1 more process has sent help message help-mpi-api.txt /
mpi-abort
[node50:77836] Set MCA parameter "orte_base_help_aggregate" to 0 to see all
help / error messages
On Wed, Aug 29, 2018 at 2:02 PM, Smith, Barry F. <bsmith at mcs.anl.gov> wrote:
> Please send complete error message
>
>
> > On Aug 29, 2018, at 3:40 PM, Manuel Valera <mvalera-w at sdsu.edu> wrote:
> >
> > Yeah, no sorry, i get the same error with -pc_type bjacobi -sub_pc_type
> SAVIENNACL : "Currently only handles ViennaCL matrices"
> >
> >
> > Thanks and let me know of any progress on this issue,
> >
> >
> >
> > On Wed, Aug 29, 2018 at 1:37 PM, Manuel Valera <mvalera-w at sdsu.edu>
> wrote:
> > Awesome, thanks!
> >
> > On Wed, Aug 29, 2018 at 1:29 PM, Smith, Barry F. <bsmith at mcs.anl.gov>
> wrote:
> >
> >
> > > On Aug 29, 2018, at 3:26 PM, Manuel Valera <mvalera-w at sdsu.edu> wrote:
> > >
> > >
> > >
> > > You may need to use just plain PCBJACOBI or PCASM for parallelism and
> then SAVIENNACL sequentially on each block.
> > >
> > >
> > > Can you elaborate a little in how to do this? do you mean i can only
> use viennacl in serial at the moment or this is a suggestion of a
> workaround on how to make it work?
> >
> > -pc_type bjacobi -sub_pc_type SAVIENNACL
> >
> > This will run in parallel and utilize the GPU for each block of the
> preconditioner.
> >
> > Barry
> >
> > >
> > >
> > > Thanks,
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > > On Aug 29, 2018, at 1:50 PM, Manuel Valera <mvalera-w at sdsu.edu>
> wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > Thanks for your responses, i understand communicating on this way to
> this level of technicality can be hard, i still think we can work ways to
> solve this problem though,
> > > >
> > > > I can say the following at this point:
> > > >
> > > > • Program works without issues in any other non-gpu
> preconditioner, just by calling -pc_type ### .
> > > > • Program works correctly with ViennaCL preconditioners with
> mpirun -n 1, that is on one processor.
> > > > • Program breaks with every ViennaCL preconditioner when
> attempting any more than 1 processors.
> > > > • I haven't tried other GPU preconditioners because ViennaCL
> gave the best performance in 1 processor before.
> > > > • I have made sure the matrix type is mpiaijviennacl as it is
> printed right before the error log.
> > > > • I am attaching what i think are the backtrace logs of the
> run with two cores, as i have limited experience with this kind of
> debugging, and i cannot make sense of what they say, please let me know if
> you need me to do something else.
> > > >
> > > > Current options i can think of are giving you permission to clone my
> model repo, or for you to guide me trough the process remotely,
> > > >
> > > > Thanks,
> > > >
> > > > .-.-.-.-
> > > >
> > > >
> > > > The error i get is still:
> > > >
> > > > Matrix type: mpiaijviennacl
>
> > > > Of sizes: 125 x 125
> > > > Matrix type: mpiaijviennacl
>
> > > > Of sizes: 125 x 125
> > > > [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> > > > [0]PETSC ERROR: No support for this operation for this object type
> > > > [0]PETSC ERROR: Currently only handles ViennaCL matrices
> > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/
> documentation/faq.html for trouble shooting.
> > > > [0]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53
> GIT Date: 2018-05-31 17:31:13 +0300
> > > > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera
> Wed Aug 29 11:43:25 2018
> > > > [0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug
> --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
> --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
> --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
> --download-viennacl
> > > > [0]PETSC ERROR: #1 PCSetUp_SAVIENNACL() line 47 in
> /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu
> > > > [0]PETSC ERROR: #2 PCSetUp() line 932 in
> /home/valera/petsc/src/ksp/pc/interface/precon.c
> > > > [0]PETSC ERROR: #3 KSPSetUp() line 381 in /home/valera/petsc/src/ksp/
> ksp/interface/itfunc.c
> > > > [1]PETSC ERROR: ------------------------------
> ------------------------------------------
> > > > [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation
> Violation, probably memory access out of range
> > > > [1]PETSC ERROR: Try option -start_in_debugger or
> -on_error_attach_debugger
> > > > [1]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/
> documentation/faq.html#valgrind
> > > > [1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple
> Mac OS X to find memory corruption errors
> > > > [1]PETSC ERROR: likely location of problem given in stack below
> > > > [1]PETSC ERROR: --------------------- Stack Frames
> ------------------------------------
> > > > [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> available,
> > > > [1]PETSC ERROR: INSTEAD the line number of the start of the
> function
> > > > [1]PETSC ERROR: is given.
> > > > [1]PETSC ERROR: [1] PetscTraceBackErrorHandler line 182
> /home/valera/petsc/src/sys/error/errtrace.c
> > > > [1]PETSC ERROR: [1] PetscError line 352 /home/valera/petsc/src/sys/
> error/err.c
> > > > [1]PETSC ERROR: [1] PCSetUp_SAVIENNACL line 45
> /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu
> > > > [1]PETSC ERROR: [1] PCSetUp line 894 /home/valera/petsc/src/ksp/pc/
> interface/precon.c
> > > > [1]PETSC ERROR: [1] KSPSetUp line 294 /home/valera/petsc/src/ksp/
> ksp/interface/itfunc.c
> > > > [1]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> > > > [1]PETSC ERROR: Signal received
> > > > [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/
> documentation/faq.html for trouble shooting.
> > > > [1]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53
> GIT Date: 2018-05-31 17:31:13 +0300
> > > > [1]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera
> Wed Aug 29 11:43:25 2018
> > > > [1]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug
> --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
> --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
> --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
> --download-viennacl
> > > > [1]PETSC ERROR: #1 User provided function() line 0 in unknown file
> > > > ------------------------------------------------------------
> --------------
> > > > MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
> > > > with errorcode 59.
> > > >
> > > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
> > > > You may or may not see output from other processes, depending on
> > > > exactly when Open MPI kills them.
> > > > ------------------------------------------------------------
> --------------
> > > > [0]PETSC ERROR: ------------------------------
> ------------------------------------------
> > > > [0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or
> the batch system) has told this process to end
> > > > [0]PETSC ERROR: Try option -start_in_debugger or
> -on_error_attach_debugger
> > > > [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/
> documentation/faq.html#valgrind
> > > > [0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple
> Mac OS X to find memory corruption errors
> > > > [0]PETSC ERROR: likely location of problem given in stack below
> > > > [0]PETSC ERROR: --------------------- Stack Frames
> ------------------------------------
> > > > [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> available,
> > > > [0]PETSC ERROR: INSTEAD the line number of the start of the
> function
> > > > [0]PETSC ERROR: is given.
> > > > [0]PETSC ERROR: [0] MatSetErrorIfFailure line 116
> /home/valera/petsc/src/mat/utils/gcreate.c
> > > > [0]PETSC ERROR: [0] PCSetUp line 894 /home/valera/petsc/src/ksp/pc/
> interface/precon.c
> > > > [0]PETSC ERROR: [0] PCSetUp_SAVIENNACL line 45
> /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu
> > > > [0]PETSC ERROR: [0] PCSetUp line 894 /home/valera/petsc/src/ksp/pc/
> interface/precon.c
> > > > [0]PETSC ERROR: [0] KSPSetUp line 294 /home/valera/petsc/src/ksp/
> ksp/interface/itfunc.c
> > > > [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> > > > [0]PETSC ERROR: Signal received
> > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/
> documentation/faq.html for trouble shooting.
> > > > [0]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53
> GIT Date: 2018-05-31 17:31:13 +0300
> > > > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera
> Wed Aug 29 11:43:25 2018
> > > > [0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug
> --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
> --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
> --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
> --download-viennacl
> > > > [0]PETSC ERROR: #4 User provided function() line 0 in unknown file
> > > > [node50:32783] 1 more process has sent help message help-mpi-api.txt
> / mpi-abort
> > > > [node50:32783] Set MCA parameter "orte_base_help_aggregate" to 0 to
> see all help / error messages
> > > >
> > > >
> > > > On Tue, Aug 28, 2018 at 9:34 PM, Karl Rupp <rupp at iue.tuwien.ac.at>
> wrote:
> > > > Hi Manuel,
> > > >
> > > > as Barry said, it is hard for us to provide any help without having
> a more complete picture of what is going on.
> > > >
> > > > The error you report seems to come from the AMG preconditioner in
> ViennaCL. This can have many origins. Do other preconditioners run without
> error? Can you run in a debugger and provide a complete backtrace?
> > > >
> > > > Thanks and best regards,
> > > > Karli
> > > >
> > > >
> > > > On 08/29/2018 01:33 AM, Manuel Valera wrote:
> > > > Talked too fast,
> > > >
> > > > After fixing that problem, i tried more than one mpi processor and
> got the following:
> > > >
> > > > Matrix type: mpiaijviennacl
> > > > Of sizes: 125 x 125
> > > > Matrix type: mpiaijviennacl
> > > > Of sizes: 125 x 125
> > > > [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> > > > [0]PETSC ERROR: No support for this operation for this object type
> > > > [0]PETSC ERROR: Currently only handles ViennaCL matrices
> > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/
> documentation/faq.html for trouble shooting.
> > > > [0]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53
> GIT Date: 2018-05-31 17:31:13 +0300
> > > > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera
> Tue Aug 28 16:30:02 2018
> > > > [0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug
> --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
> --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
> --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
> --download-viennacl
> > > > [0]PETSC ERROR: #1 PCSetUp_SAVIENNACL() line 47 in
> /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu <
> http://saviennacl.cu>
> > > > [0]PETSC ERROR: #2 PCSetUp() line 932 in
> /home/valera/petsc/src/ksp/pc/interface/precon.c
> > > > [1]PETSC ERROR: ------------------------------
> ------------------------------------------
> > > > [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation
> Violation, probably memory access out of range
> > > > [1]PETSC ERROR: Try option -start_in_debugger or
> -on_error_attach_debugger
> > > > [1]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/
> documentation/faq.html#valgrind
> > > > [1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple
> Mac OS X to find memory corruption errors
> > > > [1]PETSC ERROR: likely location of problem given in stack below
> > > > [1]PETSC ERROR: --------------------- Stack Frames
> ------------------------------------
> > > > [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> available,
> > > > [1]PETSC ERROR: INSTEAD the line number of the start of the
> function
> > > > [1]PETSC ERROR: is given.
> > > > [1]PETSC ERROR: [1] PetscTraceBackErrorHandler line 182
> /home/valera/petsc/src/sys/error/errtrace.c
> > > > [1]PETSC ERROR: [1] PetscError line 352 /home/valera/petsc/src/sys/
> error/err.c
> > > > [1]PETSC ERROR: [1] PCSetUp_SAVIENNACL line 45
> /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu<
> http://saviennacl.cu>
> > > >
> > > > [1]PETSC ERROR: [1] PCSetUp line 894 /home/valera/petsc/src/ksp/pc/
> interface/precon.c
> > > > [1]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> > > > [1]PETSC ERROR: Signal received
> > > > [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/
> documentation/faq.html for trouble shooting.
> > > > [1]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53
> GIT Date: 2018-05-31 17:31:13 +0300
> > > > [1]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera
> Tue Aug 28 16:30:02 2018
> > > > [1]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug
> --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
> --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
> --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
> --download-viennacl
> > > > [1]PETSC ERROR: #1 User provided function() line 0 in unknown file
> > > > ------------------------------------------------------------
> --------------
> > > > MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
> > > > with errorcode 59.
> > > >
> > > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
> > > > You may or may not see output from other processes, depending on
> > > > exactly when Open MPI kills them.
> > > > ------------------------------------------------------------
> --------------
> > > > [0]PETSC ERROR: ------------------------------
> ------------------------------------------
> > > > [0]PETSC ERROR: Caught signal number 15 Terminate: Some process (or
> the batch system) has told this process to end
> > > > [0]PETSC ERROR: Try option -start_in_debugger or
> -on_error_attach_debugger
> > > > [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/
> documentation/faq.html#valgrind
> > > > [0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple
> Mac OS X to find memory corruption errors
> > > > [0]PETSC ERROR: likely location of problem given in stack below
> > > > [0]PETSC ERROR: --------------------- Stack Frames
> ------------------------------------
> > > > [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> available,
> > > > [0]PETSC ERROR: INSTEAD the line number of the start of the
> function
> > > > [0]PETSC ERROR: is given.
> > > > [0]PETSC ERROR: [0] PetscCommDuplicate line 130
> /home/valera/petsc/src/sys/objects/tagm.c
> > > > [0]PETSC ERROR: [0] PetscHeaderCreate_Private line 34
> /home/valera/petsc/src/sys/objects/inherit.c
> > > > [0]PETSC ERROR: [0] ISCreate line 35 /home/valera/petsc/src/vec/is/
> is/interface/isreg.c
> > > > [0]PETSC ERROR: [0] ISCreateGeneral line 668
> /home/valera/petsc/src/vec/is/is/impls/general/general.c
> > > > [0]PETSC ERROR: [0] PCSetUp_SAVIENNACL line 45
> /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu<
> http://saviennacl.cu>
> > > > [0]PETSC ERROR: [0] PCSetUp line 894 /home/valera/petsc/src/ksp/pc/
> interface/precon.c
> > > > [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> > > > [0]PETSC ERROR: Signal received
> > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/
> documentation/faq.html for trouble shooting.
> > > > [0]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53
> GIT Date: 2018-05-31 17:31:13 +0300
> > > > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera
> Tue Aug 28 16:30:02 2018
> > > > [0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug
> --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2
> --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1
> --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64
> --download-viennacl
> > > > [0]PETSC ERROR: #3 User provided function() line 0 in unknown file
> > > > [node50:30582] 1 more process has sent help message help-mpi-api.txt
> / mpi-abort
> > > > [node50:30582] Set MCA parameter "orte_base_help_aggregate" to 0 to
> see all help / error messages
> > > >
> > > >
> > > >
> > > > It is currently running in 1 mpi processor + GPU but i would like to
> call at least 16 mpi processors + GPU to do the rest of the data management
> who is not part of the main laplacian on the mpi and the laplacian solution
> on the GPU, is this currently possible?
> > > >
> > > > Thanks for your help,
> > > >
> > > >
> > > >
> > > > On Tue, Aug 28, 2018 at 4:21 PM, Manuel Valera <mvalera-w at sdsu.edu
> <mailto:mvalera-w at sdsu.edu>> wrote:
> > > >
> > > > Ok, i found the culprit and we can close this thread,
> > > >
> > > > The problem was a missing variable for setting the maximum
> columns,
> > > > which i deleted at some point without realizing. The error
> message
> > > > was too ambiguous to catch this so i had to compare with a
> previous
> > > > working version of the arguments of MatSetValues, it was evident
> then.
> > > >
> > > > Good news is that i can now set the values with the viennacl
> types too,
> > > >
> > > > Thanks for your kind help,
> > > >
> > > > Manuel
> > > >
> > > > On Tue, Aug 28, 2018 at 11:25 AM, Smith, Barry F.
> > > > <bsmith at mcs.anl.gov <mailto:bsmith at mcs.anl.gov>> wrote:
> > > >
> > > >
> > > > 1) PetscMalloc() is never valid or needed in Fortran
> > > >
> > > > 2) there is no reason to use
> DMSetMatrixPreallocateOnly()
> > > > just use DMCreateMatrix() assuming that using a DM (DMDA,
> > > > DMPLEX, etc) is suitable for your problem.
> > > >
> > > > At this end we are totally guessing at what you are
> doing
> > > > and so have little help we can provide. A simple, nonworking
> > > > code that tries to do what you would need would help us a
> great
> > > > deal in understanding that you are trying to do.
> > > >
> > > > Barry
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > On Aug 28, 2018, at 1:18 PM, Manuel Valera
> > > > <mvalera-w at sdsu.edu <mailto:mvalera-w at sdsu.edu>> wrote:
> > > > >
> > > > > Matthew, PetscMalloc gives the same error,
> > > > >
> > > > > Barry, it would be very hard for me to get the code to a
> > > > minimum working example, i guess all i need to understand is
> how
> > > > to setup a DM matrix with DMSetMatrixPreallocateOnly()
> instead
> > > > of MatMPIAIJSetPreallocation() as we were doing before, is
> there
> > > > a simple example who does this in Fortran?
> > > > >
> > > > > Is the PetscMalloc call needed? is 'call
> > > > PetscMalloc(1,row,ierr)' a valid, compilable call to
> > > > PetscMalloc? what other reason may there be for this error to
> > > > happen ?
> > > > >
> > > > > Just remembering, that trying to setup the matrix with the
> > > > MatAIJSetPreallocation() brings up an error to acknowledge
> the
> > > > viennacl datatypes and that's why i'm trying to make this
> change
> > > > on your recommendation,
> > > > >
> > > > > Thanks for your help,
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Aug 27, 2018 at 7:35 PM, Smith, Barry F.
> > > > <bsmith at mcs.anl.gov <mailto:bsmith at mcs.anl.gov>> wrote:
> > > > >
> > > > > Send your code in a way we can compile and run it; it
> must
> > > > be some simple issue that is hard to communicate in email.
> > > > >
> > > > > Barry
> > > > >
> > > > >
> > > > > > On Aug 27, 2018, at 5:51 PM, Manuel Valera
> > > > <mvalera-w at sdsu.edu <mailto:mvalera-w at sdsu.edu>> wrote:
> > > > > >
> > > > > > Hello everyone,
> > > > > >
> > > > > > I just had time to work on this again, and checked the
> code
> > > > for errors on the matrix entries, this is the exact code i
> was
> > > > using for creating the matrix without
> > > > DMSetMatrixPreallocateOnly, using MatMPIAIJSetPreallocation
> and
> > > > it worked that way, but trying this way i get the same
> 'Column
> > > > too large' error using any number at the column position of
> > > > MatSetValues,
> > > > > >
> > > > > > I have set up my code to print the column argument (n)
> of
> > > > MatSetValues and in this case is 7 (lower than 124), it still
> > > > gives error, even entering a specific number in the
> MatSetValues
> > > > column argument position gives the same error.
> > > > > >
> > > > > > So next i went back to ex.50 here:
> > > > http://www.mcs.anl.gov/petsc/petsc-current/src/ts/examples/
> tutorials/ex50.c.html
> > > > <http://www.mcs.anl.gov/petsc/petsc-current/src/ts/examples/
> tutorials/ex50.c.html>
> > > > and it has a very similar structure except the PetscMalloc1()
> > > > call, so i tried adding that and got:
> > > > > >
> > > > > > /home/valera/ParGCCOM/Src/DMDALaplacian.f90:114:
> undefined
> > > > reference to `petscmalloc1_'
> > > > > >
> > > > > > Any ideas on this behaviour?
> > > > > >
> > > > > > Thanks so much,
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Aug 16, 2018 at 11:20 AM, Smith, Barry F.
> > > > <bsmith at mcs.anl.gov <mailto:bsmith at mcs.anl.gov>> wrote:
> > > > > >
> > > > > > Column too large: col 10980 max 124
> > > > > >
> > > > > > You need to check the code that is generating the
> matrix
> > > > entries. The matrix has 124 columns but you are attempting to
> > > > put a value at column 10980
> > > > > >
> > > > > > Barry
> > > > > >
> > > > > >
> > > > > > > On Aug 15, 2018, at 9:44 PM, Manuel Valera
> > > > <mvalera-w at sdsu.edu <mailto:mvalera-w at sdsu.edu>> wrote:
> > > > > > >
> > > > > > > Thanks Matthew and Barry,
> > > > > > >
> > > > > > > Now my code looks like:
> > > > > > >
> > > > > > > call DMSetMatrixPreallocateOnly(
> daDummy,PETSC_TRUE,ierr)
> > > > > > > call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr)
> > > > > > > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr)
> > > > > > > call DMCreateMatrix(daDummy,A,ierr)
> > > > > > > call MatSetFromOptions(A,ierr)
> > > > > > > call MatSetUp(A,ierr)
> > > > > > > [...]
> > > > > > > call
> > > > MatSetValues(A,1,row,sumpos,pos(0:iter-1),vals(0:iter-1),
> INSERT_VALUES,ierr)
> > > > > > > [...]
> > > > > > > call MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY, ierr)
> > > > > > > call MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY, ierr)
> > > > > > >
> > > > > > > And i get a different error, now is:
> > > > > > >
> > > > > > > [0]PETSC ERROR: --------------------- Error Message
> > > > ------------------------------------------------------------
> --
> > > > > > > [0]PETSC ERROR: Argument out of range
> > > > > > > [0]PETSC ERROR: Column too large: col 10980 max 124
> > > > > > > [0]PETSC ERROR: See
> > > > http://www.mcs.anl.gov/petsc/documentation/faq.html
> > > > <http://www.mcs.anl.gov/petsc/documentation/faq.html> for
> > > > trouble shooting.
> > > > > > > [0]PETSC ERROR: Petsc Development GIT revision:
> > > > v3.9.2-549-g779ab53 GIT Date: 2018-05-31 17:31:13 +0300
> > > > > > > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named
> node50
> > > > by valera Wed Aug 15 19:40:00 2018
> > > > > > > [0]PETSC ERROR: Configure options
> PETSC_ARCH=cuda-debug
> > > > --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2
> > > > --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-shared-libraries=1
> > > > --with-debugging=1 --with-cuda=1 --CUDAFLAGS=-arch=sm_60
> > > > --with-blaslapack-dir=/usr/lib64 --download-viennacl
> > > > > > > [0]PETSC ERROR: #1 MatSetValues_SeqAIJ() line 442 in
> > > > /home/valera/petsc/src/mat/impls/aij/seq/aij.c
> > > > > > > [0]PETSC ERROR: #2 MatSetValues() line 1339 in
> > > > /home/valera/petsc/src/mat/interface/matrix.c
> > > > > > >
> > > > > > >
> > > > > > > Thanks again,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Aug 15, 2018 at 7:02 PM, Smith, Barry F.
> > > > <bsmith at mcs.anl.gov <mailto:bsmith at mcs.anl.gov>> wrote:
> > > > > > >
> > > > > > > Should be
> > > > > > >
> > > > > > > call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr)
> > > > > > > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr)
> > > > > > > call DMCreateMatrix(daDummy,A,ierr)
> > > > > > >
> > > > > > > and remove the rest. You need to set the type of Mat
> > > > you want the DM to return BEFORE you create the matrix.
> > > > > > >
> > > > > > > Barry
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > On Aug 15, 2018, at 4:45 PM, Manuel Valera
> > > > <mvalera-w at sdsu.edu <mailto:mvalera-w at sdsu.edu>> wrote:
> > > > > > > >
> > > > > > > > Ok thanks for clarifying that, i wasn't sure if
> there
> > > > were different types,
> > > > > > > >
> > > > > > > > Here is a stripped down version of my code, it seems
> > > > like the preallocation is working now since the matrix
> > > > population part is working without problem, but here it is
> for
> > > > illustration purposes:
> > > > > > > >
> > > > > > > > call DMSetMatrixPreallocateOnly(
> daDummy,PETSC_TRUE,ierr)
> > > > > > > > call DMCreateMatrix(daDummy,A,ierr)
> > > > > > > > call MatSetFromOptions(A,ierr)
> > > > > > > > call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr)
> > > > > > > > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr)
> > > > > > > > call
> > > > MatMPIAIJSetPreallocation(A,19,PETSC_NULL_INTEGER,19,
> PETSC_NULL_INTEGER,ierr)
> > > > > > > > call MatSetUp(A,ierr)
> > > > > > > > [...]
> > > > > > > > call
> > > > MatSetValues(A,1,row,sumpos,pos(0:iter-1),vals(0:iter-1),
> INSERT_VALUES,ierr)
> > > > > > > > [...]
> > > > > > > > call MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY, ierr)
> > > > > > > > call MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY, ierr)
> > > > > > > >
> > > > > > > > Adding the first line there did the trick,
> > > > > > > >
> > > > > > > > Now the problem seems to be the program is not
> > > > recognizing the matrix as ViennaCL type when i try with more
> > > > than one processor, i get now:
> > > > > > > >
> > > > > > > > [0]PETSC ERROR: --------------------- Error Message
> > > > ------------------------------------------------------------
> --
> > > > > > > > [0]PETSC ERROR: No support for this operation for
> this
> > > > object type
> > > > > > > > [0]PETSC ERROR: Currently only handles ViennaCL
> matrices
> > > > > > > > [0]PETSC ERROR: See
> > > > http://www.mcs.anl.gov/petsc/documentation/faq.html
> > > > <http://www.mcs.anl.gov/petsc/documentation/faq.html> for
> > > > trouble shooting.
> > > > > > > > [0]PETSC ERROR: Petsc Development GIT revision:
> > > > v3.9.2-549-g779ab53 GIT Date: 2018-05-31 17:31:13 +0300
> > > > > > > > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named
> > > > node50 by valera Wed Aug 15 14:44:22 2018
> > > > > > > > [0]PETSC ERROR: Configure options
> PETSC_ARCH=cuda-debug
> > > > --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2
> > > > --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-shared-libraries=1
> > > > --with-debugging=1 --with-cuda=1 --CUDAFLAGS=-arch=sm_60
> > > > --with-blaslapack-dir=/usr/lib64 --download-viennacl
> > > > > > > > [0]PETSC ERROR: #1 PCSetUp_SAVIENNACL() line 47 in
> > > > /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennac
> l.cu
> > > > <http://saviennacl.cu>
> > > > > > > > [0]PETSC ERROR: #2 PCSetUp() line 932 in
> > > > /home/valera/petsc/src/ksp/pc/interface/precon.c
> > > > > > > > [0]PETSC ERROR: #3 KSPSetUp() line 381 in
> > > > /home/valera/petsc/src/ksp/ksp/interface/itfunc.c
> > > > > > > >
> > > > > > > > When running with:
> > > > > > > >
> > > > > > > > mpirun -n 1 ./gcmLEP.GPU
> > > > tc=TestCases/LockRelease/LE_6x6x6/ jid=tiny_cuda_test_n2
> > > > -ksp_type cg -dm_vec_type viennacl -dm_mat_type aijviennacl
> > > > -pc_type saviennacl -log_view
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 15, 2018 at 2:32 PM, Matthew Knepley
> > > > <knepley at gmail.com <mailto:knepley at gmail.com>> wrote:
> > > > > > > > On Wed, Aug 15, 2018 at 5:20 PM Manuel Valera
> > > > <mvalera-w at sdsu.edu <mailto:mvalera-w at sdsu.edu>> wrote:
> > > > > > > > It seems to be resumed on: I do not know how to
> > > > preallocate a DM Matrix correctly.
> > > > > > > >
> > > > > > > > There is only one matrix type, Mat. There are no
> > > > separate DM matrices. A DM can create a matrix for you
> > > > > > > > using DMCreateMatrix(), but that is a Mat and it is
> > > > preallocated correctly. I am not sure what you are doing.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Matt
> > > > > > > >
> > > > > > > > The interesting part is that it only breaks when i
> need
> > > > to populate a GPU matrix from MPI, so kudos on that, but it
> > > > seems i need to do better on my code to get this setup
> working,
> > > > > > > >
> > > > > > > > Any help would be appreciated,
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 15, 2018 at 2:15 PM, Matthew Knepley
> > > > <knepley at gmail.com <mailto:knepley at gmail.com>> wrote:
> > > > > > > > On Wed, Aug 15, 2018 at 4:53 PM Manuel Valera
> > > > <mvalera-w at sdsu.edu <mailto:mvalera-w at sdsu.edu>> wrote:
> > > > > > > > Thanks Matthew,
> > > > > > > >
> > > > > > > > I try to do that when calling:
> > > > > > > >
> > > > > > > > call
> > > > MatMPIAIJSetPreallocation(A,19,PETSC_NULL_INTEGER,19,
> PETSC_NULL_INTEGER,ierr)
> > > > > > > >
> > > > > > > > But i am not aware on how to do this for the DM if
> it
> > > > needs something more specific/different,
> > > > > > > >
> > > > > > > > The error says that your preallocation is wrong for
> the
> > > > values you are putting in. The DM does not control either,
> > > > > > > > so I do not understand your email.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Matt
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > On Wed, Aug 15, 2018 at 1:51 PM, Matthew Knepley
> > > > <knepley at gmail.com <mailto:knepley at gmail.com>> wrote:
> > > > > > > > On Wed, Aug 15, 2018 at 4:39 PM Manuel Valera
> > > > <mvalera-w at sdsu.edu <mailto:mvalera-w at sdsu.edu>> wrote:
> > > > > > > > Hello PETSc devs,
> > > > > > > >
> > > > > > > > I am running into an error when trying to use the
> > > > MATMPIAIJVIENNACL Matrix type in MPI calls, the same code
> runs
> > > > for MATSEQAIJVIENNACL type in one processor. The error
> happens
> > > > when calling MatSetValues for this specific configuration. It
> > > > does not occur when using MPI DMMatrix types only.
> > > > > > > >
> > > > > > > > The DM properly preallocates the matrix. I am
> assuming
> > > > you do not here.
> > > > > > > >
> > > > > > > > Matt
> > > > > > > >
> > > > > > > > Any help will be appreciated,
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > My program call:
> > > > > > > >
> > > > > > > > mpirun -n 2 ./gcmLEP.GPU
> > > > tc=TestCases/LockRelease/LE_6x6x6/ jid=tiny_cuda_test_n2
> > > > -ksp_type cg -dm_vec_type viennacl -dm_mat_type aijviennacl
> > > > -pc_type saviennacl -log_view
> > > > > > > >
> > > > > > > >
> > > > > > > > The error (repeats after each MatSetValues call):
> > > > > > > >
> > > > > > > > [1]PETSC ERROR: --------------------- Error Message
> > > > ------------------------------------------------------------
> --
> > > > > > > > [1]PETSC ERROR: Argument out of range
> > > > > > > > [1]PETSC ERROR: Inserting a new nonzero at global
> > > > row/column (75, 50) into matrix
> > > > > > > > [1]PETSC ERROR: See
> > > > http://www.mcs.anl.gov/petsc/documentation/faq.html
> > > > <http://www.mcs.anl.gov/petsc/documentation/faq.html> for
> > > > trouble shooting.
> > > > > > > > [1]PETSC ERROR: Petsc Development GIT revision:
> > > > v3.9.2-549-g779ab53 GIT Date: 2018-05-31 17:31:13 +0300
> > > > > > > > [1]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named
> > > > node50 by valera Wed Aug 15 13:10:44 2018
> > > > > > > > [1]PETSC ERROR: Configure options
> PETSC_ARCH=cuda-debug
> > > > --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2
> > > > --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-shared-libraries=1
> > > > --with-debugging=1 --with-cuda=1 --CUDAFLAGS=-arch=sm_60
> > > > --with-blaslapack-dir=/usr/lib64 --download-viennacl
> > > > > > > > [1]PETSC ERROR: #1 MatSetValues_MPIAIJ() line 608 in
> > > > /home/valera/petsc/src/mat/impls/aij/mpi/mpiaij.c
> > > > > > > > [1]PETSC ERROR: #2 MatSetValues() line 1339 in
> > > > /home/valera/petsc/src/mat/interface/matrix.c
> > > > > > > >
> > > > > > > >
> > > > > > > > My Code structure:
> > > > > > > >
> > > > > > > > call DMCreateMatrix(daDummy,A,ierr)
> > > > > > > > call MatSetFromOptions(A,ierr)
> > > > > > > > call MPI_Comm_size(PETSC_COMM_WORLD, numprocs,
> ierr)
> > > > > > > > if (numprocs > 1) then ! set matrix type parallel
> > > > > > > > ! Get local size
> > > > > > > > call DMDACreateNaturalVector(
> daDummy,Tmpnat,ierr)
> > > > > > > > call VecGetLocalSize(Tmpnat,locsize,ierr)
> > > > > > > > call VecDestroy(Tmpnat,ierr)
> > > > > > > > ! Set matrix
> > > > > > > > #ifdef GPU
> > > > > > > > call MatSetType(A,MATAIJVIENNACL,ierr)
> > > > > > > > call DMSetMatType(daDummy,
> MATMPIAIJVIENNACL,ierr)
> > > > > > > > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr)
> > > > > > > > print*,'SETTING GPU TYPES'
> > > > > > > > #else
> > > > > > > > call DMSetMatType(daDummy,MATMPIAIJ,ierr)
> > > > > > > > call DMSetMatType(daDummy,VECMPI,ierr)
> > > > > > > > call MatSetType(A,MATMPIAIJ,ierr)!
> > > > > > > > #endif
> > > > > > > > call
> > > > MatMPIAIJSetPreallocation(A,19,PETSC_NULL_INTEGER,19,
> PETSC_NULL_INTEGER,ierr)
> > > > > > > > else ! set matrix type sequential
> > > > > > > > #ifdef GPU
> > > > > > > > call DMSetMatType(daDummy,
> MATSEQAIJVIENNACL,ierr)
> > > > > > > > call DMSetVecType(daDummy,VECSEQVIENNACL,ierr)
> > > > > > > > call MatSetType(A,MATSEQAIJVIENNACL,ierr)
> > > > > > > > print*,'SETTING GPU TYPES'
> > > > > > > > #else
> > > > > > > > call DMSetMatType(daDummy,MATSEQAIJ,ierr)
> > > > > > > > call DMSetMatType(daDummy,VECSEQ,ierr)
> > > > > > > > call MatSetType(A,MATSEQAIJ,ierr)
> > > > > > > > #endif
> > > > > > > > call MatSetUp(A,ierr)
> > > > > > > > call
> > > > getCenterInfo(daGrid,xstart,ystart,zstart,xend,yend,zend)
> > > > > > > >
> > > > > > > > do k=zstart,zend-1
> > > > > > > > do j=ystart,yend-1
> > > > > > > > do i=xstart,xend-1
> > > > > > > > [..]
> > > > > > > > call
> > > > MatSetValues(A,1,row,sumpos,pos(0:iter-1),vals(0:iter-1),
> INSERT_VALUES,ierr)
> > > > > > > > [..]
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > What most experimenters take for granted before they
> > > > begin their experiments is infinitely more interesting than
> any
> > > > results to which their experiments lead.
> > > > > > > > -- Norbert Wiener
> > > > > > > >
> > > > > > > > https://www.cse.buffalo.edu/~knepley/
> > > > <https://www.cse.buffalo.edu/%7Eknepley/>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > What most experimenters take for granted before they
> > > > begin their experiments is infinitely more interesting than
> any
> > > > results to which their experiments lead.
> > > > > > > > -- Norbert Wiener
> > > > > > > >
> > > > > > > > https://www.cse.buffalo.edu/~knepley/
> > > > <https://www.cse.buffalo.edu/%7Eknepley/>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > What most experimenters take for granted before they
> > > > begin their experiments is infinitely more interesting than
> any
> > > > results to which their experiments lead.
> > > > > > > > -- Norbert Wiener
> > > > > > > >
> > > > > > > > https://www.cse.buffalo.edu/~knepley/
> > > > <https://www.cse.buffalo.edu/%7Eknepley/>
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > <backt-2.log><backt.log>
> >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20180829/277ef655/attachment-0001.html>
More information about the petsc-users
mailing list