[petsc-users] cudaSetDevice

[Harshad Sahasrabudhe]

I'll try doing that. We will have to take control of cudaSetDevice in our code, instead of PETSc.

Thanks,
Harshad

----- Original Message -----
From: "Paul Mullowney" <paulmullowney at gmail.com>
To: "Jed Brown" <jedbrown at mcs.anl.gov>
Cc: "Harshad Sahasrabudhe" <hsahasra at purdue.edu>, petsc-users at mcs.anl.gov
Sent: Tuesday, October 8, 2013 8:46:35 PM
Subject: Re: [petsc-users] cudaSetDevice



Perhaps one can do cudaGetDevice() and cudaDeviceReset() in between the two calls to PetscInitialize in the application code? 
Provided no device data was allocated in between the two calls, this might eliminate the error. 

If you are doing this with multiple host threads (especially where the # of CPU threads > # of GPUs), special care is required. 

-Paul 



On Tue, Oct 8, 2013 at 6:04 PM, Jed Brown < jedbrown at mcs.anl.gov > wrote: 



Harshad Sahasrabudhe < hsahasra at purdue.edu > writes: 
> You are right, it doesn't if you use just one PETSc. In our case, we 
> use 2 PETSc libraries (compiled separately, one each for complex and 
> double). 

Please remind us of this when you write. We don't know everyone in your 
group at Purdue and it would simplify our figuring out what could cause 
the weird problems you encounter. 


> And in that case, when the second PETSc library is initialized, 
> cudaSetDevice throws an error. I can't think of any way in which both 
> the PETSc libraries could be initialized with CUDA on the same 
> process. 

This is a general issue of interoperability between multiple libraries 
that want to use CUDA. How can we find out if someone else has already 
called cudaSetDevice?