[mpich-discuss] MPI jobs don't have access to CUDA devices

[Florin Ciubotaru]

Hi Jayesh,

Thanks for the quick response.
Indeed, running mpiexec with -localroot does allow gpu access, but as I
understand, it only limits the execution to one node.
I tried to change the "slave" smpd service settings using the Windows
service manager (services.msc), but apparently all connections to those
nodes fail after allowing the service to be interactive.

After a bit of digging I found a partial solution that probably has some
performance impact. For this I disabled smpd as a windows service on the
cuda enabled machines and started the process in debug mode using "smpd -d
0".
The linux versions seem to have more solid approaches but for my case debug
mode should do for now.

Thanks,
Florin Ciubotaru

On Tue, Feb 22, 2011 at 7:02 PM, Jayesh Krishna <jayesh at mcs.anl.gov> wrote:

> Hi,
>  You are right, the launched processes do not have an interactive console
> session. This prevents the MPI processes from using GPUs. This is a feature
> that can be added in future (Although I don't have a timeline for it).
>  You can however use GPUs on a local node using the "-localroot" option of
> mpiexec.
>
> Regards,
> Jayesh
>
> ----- Original Message -----
> From: "Florin Ciubotaru" <florinciu at gmail.com>
> To: mpich-discuss at mcs.anl.gov
> Sent: Tuesday, February 22, 2011 10:53:45 AM
> Subject: [mpich-discuss] MPI jobs don't have access to CUDA devices
>
>
> Hi,
>
>
> I'm running mpi jobs on a small cluster that has several machines with CUDA
> enabled devices. I'm trying to add gpu acceleration to my computations, but
> I'm having issues executing CUDA code while running the processes as an mpi
> job. The gpu logic works fine when the application is executed as an
> independent process.
> I guess there's something in the way smpd launches the job and prevents the
> process from properly accessing the CUDA driver(and device). Most probable
> cause is that smpd is running as a Windows service and not as an interactive
> console session. Is there a way to tweak smpd or mpiexec in order to have
> CUDA access for the jobs?
>
>
> Thanks,
> Florin Ciubotaru
>
>
>
>
> _______________________________________________
> mpich-discuss mailing list
> mpich-discuss at mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20110223/75a5d604/attachment.htm>