[petsc-dev] [External] Re: Issue with sacusp preconditioner.

[Das, Rajeev]

UNRESTRICTED / ILLIMITÉE

Hi:

 

The CUDA details are:

 

nvcc: NVIDIA (R) Cuda compiler driver

Copyright (c) 2005-2015 NVIDIA Corporation

Built on Mon_Feb_16_22:59:02_CST_2015

Cuda compilation tools, release 7.0, V7.0.27

 

and the OS details are

 

Linux version 2.6.32-573.1.1.el6.x86_64 (mockbuild at c6b9.bsys.dev.centos.org) 

(gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC) )

 

 

 

Regards,

 

Rajeev Das. 

 

From: Steven Dalton [mailto:sdalton1 at gmail.com] 
Sent: March-23-16 9:59 AM
To: Das, Rajeev
Cc: For users of the development version of PETSc; rajeevdas25 at gmail.com
Subject: [External] Re: [petsc-dev] Issue with sacusp preconditioner.

 

Hello,

 

  What version of CUDA is installed? OS?

  Can you send your example file?

 

Thanks,

  Steve

 

On Wed, Mar 23, 2016 at 9:39 AM, Das, Rajeev <rajeev.das at cnl.ca> wrote:

UNRESTRICTED / ILLIMITÉE

Hi:

 

The scenario that I considered involves a structured grid of size 32 x 32 x 32 (using Pflotran here).  When I used the bcgs solver and the preconditioner jacobi, there were no issues, but when the preconditioner sacusp was used I received the following error:

 

terminate called after throwing an instance of 'thrust::system::system_error'

  what():  function_attributes(): after cudaFuncGetAttributes: an illegal memory access was encountered.

 

I am attaching the screen shots of my debugger (idb) that will provide some detail regarding the source of this error. My Petsc configuration is 

 

--download-mpich=yes --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-shared-libraries=0 --with-debugging=1 --with-valgrind=1 --download-parmetis=yes --download-metis=yes --download-hypre=yes --download-fblaslapack=yes --with-c2html=0 --with-cuda=1 --with-cusp=1 --with-thrust=1 --with-64-bit-indices=1

 

I have to use 64 bit indices, if I avoid it, I receive out of memory issues. Machines used are Intel Xeon E5-2620 and Tesla K20c card having the compute capability of 3.5. When I reduced the grid size to 16 x 16 x 16 I no more receive the above error with sacusp.

 

Can you point me what could be the problem here.

 

 

Regards,

 

Rajeev Das.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20160323/f167e504/attachment.html>