<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:times new roman, new york, times, serif;font-size:12pt"><DIV>I am a litle confused with what is discussed in this thread. </DIV>
<DIV> </DIV>
<DIV>Jayesh, are you saying if we call MPI_Init_thread with THREAD_MULTIPLE, the default </DIV>
<DIV>channel is socket ? Even if MPICH2 is built with THREAD_MULTIPLE ?</DIV>
<DIV> </DIV>
<DIV>I thought (or was told) that the default channel is 'nemesis', and MPICH2 runtime will pick</DIV>
<DIV>the proper thread safe channel depending on the situation. So, if I build MPICH2 (2.1.1+)</DIV>
<DIV>without specifying the channel and threading, and I call MPI_Init_Thread with THREAD_MULTIPLE,</DIV>
<DIV>the channel will be thread-safe nemesis, is this remotely correct ?<BR></DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">tan</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><BR> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><FONT size=2 face=Tahoma>
<HR SIZE=1>
<B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Jayesh Krishna <jayesh@mcs.anl.gov><BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Lubomir Radev <lubomirr@yahoo.com><BR><B><SPAN style="FONT-WEIGHT: bold">Cc:</SPAN></B> mpich-discuss@mcs.anl.gov<BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Friday, September 25, 2009 7:18:46 AM<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> Re: [mpich-discuss] MPI_THREAD_MULTIPLE<BR></FONT><BR>
<STYLE type=text/css>DIV {
MARGIN:0px;}
</STYLE>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009>Hi,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009> If you are starting your program without mpiexec the default channel is picked. You can control the channel used by the library using the "MPICH2_CHANNEL" environment variable.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009> Please set MPICH2_CHANNEL environment variable as follows (and then run your MPI program),</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009>1) To use the Nemesis channel, MPICH2_CHANNEL=nemesis</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009>2) To use the multithreaded channel, MPICH2_CHANNEL=mt</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009> The default channel (MPICH2_CHANNEL=sock) is the single threaded sock channel.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009> Let us know if it works for you.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009>Regards,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=320011514-25092009>Jayesh</SPAN></FONT></DIV><BR>
<DIV dir=ltr lang=en-us class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>From:</B> Lubomir Radev [mailto:lubomirr@yahoo.com] <BR><B>Sent:</B> Tuesday, September 22, 2009 11:05 AM<BR><B>To:</B> Jayesh Krishna<BR><B>Cc:</B> mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> Re: [mpich-discuss] MPI_THREAD_MULTIPLE<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV style="FONT-FAMILY: 'times new roman', 'new york', times, serif; FONT-SIZE: 12pt">
<DIV>I will do that, but in the meantime, could you run your own test code below directly, not using mpiexec, and see the result. For me, it's returning MPI_THREAD_SINGLE - is that how it's supposed to be, i.e. the thread-safe support can only be enabled if the application is run via mpiexec? </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><BR>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><FONT size=2 face=Tahoma>
<HR SIZE=1>
<B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Jayesh Krishna <jayesh@mcs.anl.gov><BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Lubomir Radev <lubomirr@yahoo.com><BR><B><SPAN style="FONT-WEIGHT: bold">Cc:</SPAN></B> mpich-discuss@mcs.anl.gov<BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Tuesday, September 22, 2009 8:21:44 AM<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE: [mpich-discuss] MPI_THREAD_MULTIPLE<BR></FONT><BR>
<STYLE type=text/css>DIV {
MARGIN:0px;}
</STYLE>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=501422015-22092009>Hi,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=501422015-22092009> Can you send us a test program ?</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=501422015-22092009></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=501422015-22092009>Regards,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=501422015-22092009>Jayesh</SPAN></FONT></DIV><BR>
<DIV dir=ltr lang=en-us class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>From:</B> Lubomir Radev [mailto:lubomirr@yahoo.com] <BR><B>Sent:</B> Tuesday, September 22, 2009 2:27 AM<BR><B>To:</B> Jayesh Krishna; mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> Re: [mpich-discuss] MPI_THREAD_MULTIPLE<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV style="FONT-FAMILY: 'times new roman', 'new york', times, serif; FONT-SIZE: 12pt">
<DIV>This works for me too, returning MPI_THREAD_MULTIPLE, but only when started via mpiexec.</DIV>
<DIV><BR></DIV>
<DIV>However, I need to launch my worker processes programmatically by using MPI_Comm_spawn_multiple(). That's necessitated by the nature of my app - it's actually a library that clients will be linking against. So the client process has to spawn workers and communicate with them, which works just fine from one thread and fails with two. And if I use MPI_Init_thread() from the client/master process (which isn't started by mpiexec), it always returns MPI_THREAD_SINGLE.</DIV>
<DIV><BR></DIV>
<DIV>Should I try to pack all of my MPI communications in one thread or can you suggest a solution other than mpiexec for multiple threads? Perhaps I could somehow perform the same thread support initialization that mpiexec apparently does internally?</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><BR></DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">Thanks very much.</DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><BR>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><FONT size=2 face=Tahoma>
<HR SIZE=1>
<B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Jayesh Krishna <jayesh@mcs.anl.gov><BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> mpich-discuss@mcs.anl.gov<BR><B><SPAN style="FONT-WEIGHT: bold">Cc:</SPAN></B> Lubomir Radev <lubomirr@yahoo.com><BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Monday, September 21, 2009 9:04:17 AM<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE: [mpich-discuss] MPI_THREAD_MULTIPLE<BR></FONT><BR>
<STYLE type=text/css>DIV {
MARGIN:0px;}
</STYLE>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009>Hi,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009> Are you launching your MPI job using the "-channel" option of mpiexec (mpiexec -n 2 -channel nemesis thread_multiple_test.exe)?</SPAN></FONT><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009> A simple MPI program (see below) that queries for the thread level support returns MPI_THREAD_MULTIPLE for me.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009>=========== Thread multiple test ===================</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009><FONT color=#0000ff size=2><FONT color=#0000ff size=2></FONT></FONT>
<P><FONT color=#0000ff size=2><FONT color=#0000ff size=2>#include</FONT></FONT><FONT color=#000000 size=2> </FONT><FONT color=#a31515 size=2><FONT color=#a31515 size=2><iostream></FONT></FONT></P><FONT color=#a31515 size=2><FONT color=#a31515 size=2></FONT></FONT><FONT color=#0000ff size=2><FONT color=#0000ff size=2></FONT></FONT>
<P><FONT color=#0000ff size=2><FONT color=#0000ff size=2>#include</FONT></FONT><FONT color=#000000 size=2> </FONT><FONT color=#a31515 size=2><FONT color=#a31515 size=2>"mpi.h"</FONT></FONT></P><FONT color=#a31515 size=2><FONT color=#a31515 size=2></FONT></FONT><FONT color=#0000ff size=2><FONT color=#0000ff size=2></FONT></FONT>
<P><FONT color=#0000ff size=2><FONT color=#0000ff size=2>int</FONT></FONT><FONT color=#000000 size=2> main(</FONT><FONT color=#0000ff size=2><FONT color=#0000ff size=2>int</FONT></FONT><FONT color=#000000 size=2> argc, </FONT><FONT color=#0000ff size=2><FONT color=#0000ff size=2>char</FONT></FONT><FONT size=2><FONT color=#000000> *argv[])</FONT></FONT></P><FONT size=2>
<P>{</P></FONT>
<P><FONT color=#0000ff size=2><FONT color=#0000ff size=2>int</FONT></FONT><FONT size=2> provided;</FONT></P><FONT size=2>
<P>provided = MPI::Init_thread(MPI_THREAD_MULTIPLE);</P></FONT>
<P><FONT color=#0000ff size=2><FONT color=#0000ff size=2>if</FONT></FONT><FONT size=2>(provided == MPI_THREAD_MULTIPLE){</FONT></P><FONT size=2></FONT>
<P><FONT size=2>std::cout << </FONT><FONT color=#a31515 size=2><FONT color=#a31515 size=2>"Thread multiple supported"</FONT></FONT><FONT size=2> << std::endl;</FONT></P><FONT size=2>
<P>}</P></FONT>
<P><FONT color=#0000ff size=2><FONT color=#0000ff size=2>else</FONT></FONT><FONT size=2>{</FONT></P><FONT size=2></FONT>
<P><FONT size=2>std::cout << </FONT><FONT color=#a31515 size=2><FONT color=#a31515 size=2>"Thread multiple NOT supported"</FONT></FONT><FONT size=2> << std::endl;</FONT></P><FONT size=2>
<P>}</P>
<P>MPI::Finalize();</P>
<P>}</P>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009>=========== Thread multiple test ===================</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009>Regards,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN class=408565815-21092009>Jayesh</SPAN></FONT></DIV></FONT></SPAN></FONT></DIV><BR>
<DIV dir=ltr lang=en-us class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>From:</B> mpich-discuss-bounces@mcs.anl.gov [mailto:mpich-discuss-bounces@mcs.anl.gov] <B>On Behalf Of </B>Lubomir Radev<BR><B>Sent:</B> Sunday, September 20, 2009 7:23 PM<BR><B>To:</B> mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> Re: [mpich-discuss] MPI_THREAD_MULTIPLE<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV style="FONT-FAMILY: 'times new roman', 'new york', times, serif; FONT-SIZE: 12pt">
<DIV>Yes. I was using 1.0.8p1 originally, but after your suggestion got 1.1.1p1 and the problem persists.</DIV>
<DIV><BR></DIV>
<DIV>I could reduce my code to a test case exhibiting the issue, but I need to understand why MPI_Init_thread() won't provide me with MPI_THREAD_MULTIPLE support and always returns MPI_THREAD_SINGLE regardless of the channel configuration. I need to get that working first so my multi-threaded code can assume it operates in a correctly initialized environment and can legitimately call into MPICH2 simultaneously from more than one thread.</DIV>
<DIV><BR></DIV>
<DIV>I'm using Boost.MPI wrapper (and Boost.Thread), but that shouldn't matter.</DIV>
<DIV><BR></DIV>
<DIV>Thanks</DIV>
<DIV><BR></DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt"><FONT size=2 face=Tahoma>
<HR SIZE=1>
<B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Jayesh Krishna <jayesh@mcs.anl.gov><BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> mpich-discuss@mcs.anl.gov<BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Friday, September 18, 2009 8:42:38 AM<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> Re: [mpich-discuss] MPI_THREAD_MULTIPLE<BR></FONT><BR>
<P><FONT size=2>Hi,<BR><SPAN><SPAN><SPAN> Are you trying the latest stable release, 1.1.1p1, of MPICH2 (http://www.mcs.anl.gov/research/projects/mpich2/downloads/index.php?s=downloads) ?</SPAN></SPAN></SPAN><BR> Can you send us a test program ?<BR><BR>Regards,<BR>Jayesh<BR><BR>-----Original Message-----<BR>From: mpich-discuss-bounces@mcs.anl.gov [<A href="mailto:mpich-discuss-bounces@mcs.anl.gov" rel=nofollow target=_blank ymailto="mailto:mpich-discuss-bounces@mcs.anl.gov">mailto:mpich-discuss-bounces@mcs.anl.gov</A>] On Behalf Of Lubomir Radev<BR>Sent: Thursday, September 17, 2009 10:08 PM<BR>To: mpich-discuss@mcs.anl.gov<BR>Subject: [mpich-discuss] MPI_THREAD_MULTIPLE<BR><BR>Hey guys,<BR><BR>I need to call into the thread-safe version of MPICH2 so I tried to initialize with MPI_Init_thread() and MPI_THREAD_MULTIPLE, but no matter how I configured the channels on my nodes - tried both mt and nemesis - that function returns MPI_THREAD_SINGLE as
provided thread support.<BR><BR>I'm using 1.0.8p1 on Windows if that matters. Installed from the provided MSI binary. Perhaps I need to build from sources, configuring it with thread support?<BR><BR>Thanks<BR></FONT></P></DIV></DIV>
<DIV></DIV></DIV></DIV></DIV>
<DIV></DIV></DIV></DIV></DIV>
<DIV></DIV></DIV></DIV></div><br>
</body></html>