<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7651.59">
<TITLE>Problem with -machinefile</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P><FONT SIZE=2 FACE="Arial">I am having a problem running mpiexec with the -machine file option. (Red Hat Enterprise 4 - 64 bit)</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">When I use the -machinefile option, my application hangs (deadlocks) while attempting communication. The master is sending, the workers are receiving, but nothing happens. Any thoughts?</FONT></P>
<P><FONT SIZE=2 FACE="Arial">I start my MPD ring as follows:</FONT>
</P>
<P><FONT SIZE=2 FACE="Courier New">> mpdboot -n 3 -f mpd.hosts</FONT>
<BR><FONT SIZE=2 FACE="Courier New">> cat mpd.hosts</FONT>
<BR><FONT SIZE=2 FACE="Courier New">pad-lnx52:2</FONT>
<BR><FONT SIZE=2 FACE="Courier New">noclue:2</FONT>
<BR><FONT SIZE=2 FACE="Courier New">question:4</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">I can then run my application with the -host option or by letting the MPD ring choose the systems using either of the following command lines:</FONT></P>
<P><FONT SIZE=2 FACE="Courier New">> mpiexec -l -n 1 -host pad-lnx52 lithorun dev/LithoWare/Samples/FEM1D.xml Output.xml : -n 2 -host noclue lithorun : -n 2 -host question lithorun</FONT></P>
<P><FONT SIZE=2 FACE="Courier New">> mpiexec -l -n 5 lithorun dev/LithoWare/Samples/FEM1D.xml Output.xml</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">But when I try to use the -machine file option, my application hangs. The master is sending; all of the workers are receiving, but no communication appears to actually be happening.</FONT></P>
<P><FONT SIZE=2 FACE="Courier New">> mpiexec -machinefile mpd.hosts -l -n 5 lithorun dev/LithoWare/Samples/FEM1D.xml Output.xml</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Here is trace of the process when it hangs. You can see that the workers have been started and are waiting for a work packet in a MPI::COMM_WORLD.Probe call. The master has divided up the work and is attempting to send the first packet using a MPI::COMM_WORLD.Send call. Then, nothing else happens. This only occurs when I am trying to use the -machinefile option. </FONT></P>
<P><FONT SIZE=2 FACE="Courier New">> mpiexec -machinefile mpd.hosts -l -n 5 lithorun dev/LithoWare/Samples/FEM1D.xml Output.xml</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Worker on noclue</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Worker on noclue</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">1: Worker on pad-lnx52.kla-tencor.com</FONT>
<BR><FONT SIZE=2 FACE="Courier New">1: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Master on pad-lnx52.kla-tencor.com</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Loading dev/LithoWare/Samples/FEM1D.xml</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Found Factorial(FEM1D)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Worker on question.kla-tencor.com</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Loading Sample.plt</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Distributing Factorial(FEM1D) with 45 experiments over 4 processes with 3 work packets</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Sending work(1625)</FONT>
</P>
<BR>
<P><FONT SIZE=2 FACE="Arial">For a point of reference here is a trace of the process when it works:</FONT>
</P>
<P><FONT SIZE=2 FACE="Courier New">> mpiexec -l -n 5 lithorun dev/LithoWare/Samples/FEM1D.xml Output.xml</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Master on pad-lnx52.kla-tencor.com</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Loading dev/LithoWare/Samples/FEM1D.xml</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Found Factorial(FEM1D)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Loading Sample.plt</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Distributing Factorial(FEM1D) with 45 experiments over 4 processes with 3 work packets</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Sending work(1625)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">1: Worker on noclue</FONT>
<BR><FONT SIZE=2 FACE="Courier New">1: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Worker on noclue</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Worker on question.kla-tencor.com</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Sent work(1625)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Sending work(1619)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Worker on question.kla-tencor.com</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Received work(1625)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Found Factorial(FEM1D)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Loading Sample.plt</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Sent work(1619)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Sending work(1622)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Running Factorial(FEM1D) with 15 experiments</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Received work(1619)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Found Factorial(FEM1D)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Loading Sample.plt</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Running Factorial(FEM1D) with 15 experiments</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Sent work(1622)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Waiting for results...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Received work(1622)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Found Factorial(FEM1D)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Loading Sample.plt</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Running Factorial(FEM1D) with 15 experiments</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Received results(1672)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Waiting for results...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Factorial(FEM1D) complete (0.04175)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Sending results(1672)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Factorial(FEM1D) complete (0.0420239)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Received results(1652)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Waiting for results...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Sending results(1652)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Factorial(FEM1D) complete (0.0852771)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Received results(1400)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">0: Factorial(FEM1D) complete (0.136751)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Sending results(1400)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Waiting for work...</FONT>
<BR><FONT SIZE=2 FACE="Courier New">1: Received work(0)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">4: Received work(0)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">3: Received work(0)</FONT>
<BR><FONT SIZE=2 FACE="Courier New">2: Received work(0)</FONT>
</P>
</BODY>
</HTML>