<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.6000.16414" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=431090800-27022007><FONT face=Arial
color=#0000ff size=2>You could directly use the ch3:sock channel instead of
ch3:shm. It would behave better in this case because it waits on a poll() as you
point out, but the rest of the communication would be slower with ch3:sock than
ch3:shm.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=431090800-27022007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=431090800-27022007><FONT face=Arial
color=#0000ff size=2>Rajeev</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Sudarshan Raghunathan
[mailto:rdarshan@gmail.com] <BR><B>Sent:</B> Monday, February 26, 2007 5:58
PM<BR><B>To:</B> Rajeev Thakur<BR><B>Cc:</B>
mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> Re: [MPICH] MPI_Barrier on
ch3:shm and sched_yield<BR></FONT><BR></DIV>
<DIV></DIV>Rajeev,<BR><BR>Thank you for the reply.<BR><BR>I was running four
MPI processes and then started another computationally intensive task (running
another MPI program on 2 processes, for example). The other task gets around
50% of the CPU, but seems to be constantly competing with the first MPI
program which isn't really doing any heavy lifting except waiting on the
barrier. <BR><BR>I looked into a few options, all of which seem to have
problems:<BR>(a) put a sleep in the MPIDI_CH3I_Progress code so that the
processes waiting on the barrier sleep for a while instead of spinning in a
sched_yield loop and compete with other processes which might actually need to
use the CPU. The problem with this approach is that it might slow down
portions of the application where a fast barrier _is_ required <BR>(b) use
semaphores. It looks like it might touch a lot of places in the implementation
and again might cause performance problems where a fast barrier is
needed<BR>(c) use sockets. The sock device in MPICH seems to be calling poll
with a time-out and that seems to be behaving much better than the shm device.
I'm not sure how much of the existing MPICH code can be reused. <BR><BR>Any
other saner/more feasible options or advice on which of the above three
options is simplest to implement would be most appreciated.<BR><BR>Thanks
again,<BR>Sudarshan<BR><BR>
<DIV><SPAN class=gmail_quote>On 26/02/07, <B class=gmail_sendername>Rajeev
Thakur</B> <<A onclick="return top.js.OpenExtLink(window,event,this)"
href="mailto:thakur@mcs.anl.gov" target=_blank>thakur@mcs.anl.gov </A>>
wrote:</SPAN>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2>Sudarshan,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2>
How many MPI processes are you running on the 4-way SMP and how many
processes of the other application? Does the other application get any CPU
at all? sched_yield just gives up the current time slice for the process and
moves it to the end of the run queue. It will get run again when its turn
comes. In other words, the process doesn't sleep until some event happens.
It shares the CPU with other processes. </FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2>Rajeev</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2></FONT></SPAN><BR></DIV></DIV></BLOCKQUOTE></DIV><BR></BLOCKQUOTE></BODY></HTML>