Hi Jayesh,<div><br></div><div>Thanks for the quick response.</div><div>Indeed, running mpiexec with -localroot does allow gpu access, but as I understand, it only limits the execution to one node.</div><div>I tried to change the "slave" smpd service settings using the Windows service manager (services.msc), but apparently all connections to those nodes fail after allowing the service to be interactive.</div>
<div><br></div><div>After a bit of digging I found a partial solution that probably has some performance impact. For this I disabled smpd as a windows service on the cuda enabled machines and started the process in debug mode using "smpd -d 0".</div>
<div>The linux versions seem to have more solid approaches but for my case debug mode should do for now.</div><div><br></div><div>Thanks,</div><div>Florin Ciubotaru </div><div><br><div class="gmail_quote">On Tue, Feb 22, 2011 at 7:02 PM, Jayesh Krishna <span dir="ltr"><<a href="mailto:jayesh@mcs.anl.gov">jayesh@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hi,<br>
You are right, the launched processes do not have an interactive console session. This prevents the MPI processes from using GPUs. This is a feature that can be added in future (Although I don't have a timeline for it).<br>
You can however use GPUs on a local node using the "-localroot" option of mpiexec.<br>
<br>
Regards,<br>
Jayesh<br>
<div><div></div><div class="h5"><br>
----- Original Message -----<br>
From: "Florin Ciubotaru" <<a href="mailto:florinciu@gmail.com">florinciu@gmail.com</a>><br>
To: <a href="mailto:mpich-discuss@mcs.anl.gov">mpich-discuss@mcs.anl.gov</a><br>
Sent: Tuesday, February 22, 2011 10:53:45 AM<br>
Subject: [mpich-discuss] MPI jobs don't have access to CUDA devices<br>
<br>
<br>
Hi,<br>
<br>
<br>
I'm running mpi jobs on a small cluster that has several machines with CUDA enabled devices. I'm trying to add gpu acceleration to my computations, but I'm having issues executing CUDA code while running the processes as an mpi job. The gpu logic works fine when the application is executed as an independent process.<br>
I guess there's something in the way smpd launches the job and prevents the process from properly accessing the CUDA driver(and device). Most probable cause is that smpd is running as a Windows service and not as an interactive console session. Is there a way to tweak smpd or mpiexec in order to have CUDA access for the jobs?<br>
<br>
<br>
Thanks,<br>
Florin Ciubotaru<br>
<br>
<br>
<br>
<br>
</div></div>_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
</blockquote></div><br></div>