[mpich-discuss] command line ordering of hosts matters?
jayesh at mcs.anl.gov
jayesh at mcs.anl.gov
Fri Jul 23 14:39:11 CDT 2010
Hi,
From the error message, gethostbyname() fails for ipaddress 10.0.0.6. Looks like the DNS entries for the host is incomplete. Can you (or a sysadmin) verify that the DNS entries for the host is actually valid/complete ?
Can you also try the newer nemesis channel (which will be the default channel, replacing the sock channel, in 1.3 series) and see if it works (mpiexec -channel nemesis -hosts 2 10.0.0.6 1 10.0.0.1 1 helloworld.exe)?
Regards,
Jayesh
----- Original Message -----
From: "David Lowinger" <David_Lowinger at ea.epson.com>
To: mpich-discuss at mcs.anl.gov
Sent: Friday, July 23, 2010 2:24:02 PM GMT -06:00 US/Canada Central
Subject: Re: [mpich-discuss] command line ordering of hosts matters?
Hi Jayesh,
I do indeed see 1 instance of my MPI program ("myapp.exe") on each machine's Process Manager whenever I run any test, whether the MPI_Bcast fails or not. So, it looks like both machines are successfully launching processes.
Yes, I am running the 64-bit version of MPICH2 on both the machines. I just tried recompiling the code separately on both the machines, and I see the exact same result as before ( ie. the MPI_Bcast never completes if I put 10.0.0.101 before 10.0.0.6 ).
I tried both the simple hello world program and the compute pi example that you sent in the links. The simple "hello world" program actually crashes... this is very odd, because I've successfully run much more complex apps than this with no problems. Please see the attached file ("helloworld_output.jpg") for the error I get when I run the https://svn.mcs.anl.gov/repos/mpi/mpich2/trunk/examples/hellow.c example. I see this crash as long as I use 2 hosts, listed in either order on the command line. If I only use 1 host (ie. "mpiexec -np 5 helloworld.exe"), it runs fine.
The PI example seems to have the same "host ordering" issue I initially described to you. If I place 10.0.0.6 first in the command line, there's no problem (apart from me being unable to input the value for 'n' using scanf... instead, I just commented out the scanf section and hardcoded 'n' to be 5). If 10.0.0.6 is not listed first, the PI calculation app never gets past the initial call to 'MPI_Bcast()'.
David
Jayesh Krishna <jayesh at mcs.anl.gov>
07/13/2010 08:56 PM
Expire Date: 07/22/2012
To mpich-discuss at mcs.anl.gov
cc David_Lowinger at ea.epson.com
Subject Re: [mpich-discuss] command line ordering of hosts matters?
Hi,
Sorry for the delayed response.
Since you are able to run "hostname" on the two machines the process manager should be able to launch processes on the two machines (In the process manager tab you should see 1 instance of your MPI program, MYMPIPGM.exe, on each machine).
Are you running a 64-bit version of MPICH2 on both the machines ? Have you recompiled your code on both the machines ? Did you try a simple hello world program to see if it works (https://svn.mcs.anl.gov/repos/mpi/mpich2/trunk/examples/hellow.c)? Try compiling the compute pi example to see if it works (https://svn.mcs.anl.gov/repos/mpi/mpich2/trunk/examples/icpi.c - Make sure that you compile the code separately on both the machines)?
Let us know the results.
Regards,
Jayesh
----- Original Message -----
From: "David Lowinger" <David_Lowinger at ea.epson.com>
To: mpich-discuss at mcs.anl.gov
Sent: Monday, July 5, 2010 11:11:16 AM GMT -06:00 US/Canada Central
Subject: Re: [mpich-discuss] command line ordering of hosts matters?
Hi Jayesh,
Yes, I can run a simple non-MPI program (ie. the program "hostname") successfully with " mpiexec -hosts 2 10.0.0.101 1 10.0.0.6 1 hostname ". I see both hostnames get printed out, and then the program exits. The ordering of hosts does not matter in this case.
I see the exact same behavior as before when I provide the complete path to the executable; ie. the MPI_Bcase never completes if I put 10.0.0.101 before 10.0.0.6.
I see the exact same behavior as before when I use the machinefile option.
With regards to checking the control panel, I assume you mean the "Processes" tab of the Windows Task Manager? If so, which processes should I be looking for as the program runs? I changed the MPI "hello world" app to MPI_Bcast a very large buffer over and over, so that I can look for some kind of process in the Windows Task Manager. What is the name of the process I should be looking for? Thanks,
David
Jayesh Krishna <jayesh at mcs.anl.gov>
07/01/2010 11:54 AM
Expire Date: 07/04/2012
To mpich-discuss at mcs.anl.gov
cc David_Lowinger at ea.epson.com
Subject Re: [mpich-discuss] command line ordering of hosts matters?
Hi,
Along with checking whether the processes are launched, please provide us with the information below,
# Are you able to run a simple non-MPI program (mpiexec -hosts 2 10.0.0.101 1 10.0.0.6 1 hostname)?
# Try providing the complete path to the executable (mpiexec -hosts 2 10.0.0.101 1 10.0.0.6 1 c:\temp\helloworld.exe). Without the complete path you might actually be executing two different MPI programs on the hosts.
# Does the machinefile option work (mpiexec -n 2 -machinefile mf.txt c:\temp\helloworld.exe ; Where the file mf.txt contains the host ipaddresses)?
Regards,
Jayesh
----- Original Message -----
From: "David Lowinger" <David_Lowinger at ea.epson.com>
To: mpich-discuss at mcs.anl.gov
Sent: Wednesday, June 30, 2010 4:26:16 PM GMT -06:00 US/Canada Central
Subject: Re: [mpich-discuss] command line ordering of hosts matters?
Hi Jayesh,
On both machines, when I type, "smpd -version", both display "1.2.1p1".
When I call "MPI_Get_version()" on both machines, they both show version 2, subversion 2.
Both machines are running 64-bit Windows Vista.
I'll check the control panel to see if the MPI processes are being launched.
David
Jayesh Krishna <jayesh at mcs.anl.gov>
06/30/2010 04:53 PM
Expire Date: 06/29/2012
To mpich-discuss at mcs.anl.gov
cc David_Lowinger at ea.epson.com
Subject Re: [mpich-discuss] command line ordering of hosts matters?
Hi,
Which version of MPICH2 are you using ?
Do the two machines have the same underlying architecture (MPICH2 currently does not support heterogeneous systems - So you cannot run your job across 32-bit and 64-bit machines/MPICH2_libs)?
Also try checking the control panel to see if the MPI processes are being launched on the machines.
Regards,
Jayesh
----- Original Message -----
From: "David Lowinger" <David_Lowinger at ea.epson.com>
To: mpich-discuss at mcs.anl.gov
Sent: Wednesday, June 30, 2010 3:42:22 PM GMT -06:00 US/Canada Central
Subject: Re: [mpich-discuss] command line ordering of hosts matters?
Firewall is turned off on both machines.
There is no error message... the MPI_Bcase simply never completes. I've left it running for 10 minutes, and the second printf ("completed MPI_Bcast") never appears when I use the second host ordering below ("10.0.0.101 1 10.0.0.6 1").
When I run "smpd -status 10.0.0.6" from 10.0.0.101, I see the message "smpd running on 10.0.0.6". When I run "smpd -status 10.0.0.101" from 10.0.0.6, I see the message "smpd running on 10.0.0.101".
jayesh at mcs.anl.gov
06/30/2010 10:50 AM
Expire Date: 06/29/2012
To mpich-discuss at mcs.anl.gov
cc David_Lowinger at ea.epson.com
Subject Re: [mpich-discuss] command line ordering of hosts matters?
Hi,
Do you have a firewall running on any of these machines (If so, can you try running your job after turning off the firewall)?
What is the error message that you get when you run your job ?
Can you try running "smpd -status REMOTE_MACHINE" from each of the machines and let us know the results ("smpd -status 10.0.0.6" from 10.0.0.101 & "smpd -status 10.0.0.101" from 10.0.0.6)?
Regards,
Jayesh
----- Original Message -----
From: "David Lowinger" <David_Lowinger at ea.epson.com>
To: mpich-discuss at mcs.anl.gov
Sent: Tuesday, June 29, 2010 5:53:07 PM GMT -06:00 US/Canada Central
Subject: [mpich-discuss] command line ordering of hosts matters?
Hi,
When running a very basic "hello world" app, I've found that the app's behavior depends on the order I use for hosts in the command line. For example, if I use:
mpiexec -hosts 2 10.0.0.6 1 10.0.0.101 1 helloworld.exe
The program executes flawlessly. But, if I use:
mpiexec -hosts 2 10.0.0.101 1 10.0.0.6 1 helloworld.exe
then the program never gets past the call to "MPI_Bcast()". Here is my code:
-------------------
#include "mpi.h"
#define MPI_FLUSH() fflush(stdout)
int main( int argc, char* argv[] )
{
int g_Thread_ID, g_Num_Threads;
int test = 0;
/**************************************************\
* MPI Initialization *
\**************************************************/
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &g_Thread_ID);
MPI_Comm_size(MPI_COMM_WORLD, &g_Num_Threads);
printf("thread %d: main: About to execute MPI_Bcast\n", g_Thread_ID);
MPI_FLUSH();
// Broadcast integer
int err = MPI_Bcast(&test, 1, MPI_INT, 0, MPI_COMM_WORLD);
printf("thread %d: completed MPI_Bcast\n", g_Thread_ID);
MPI_FLUSH();
MPI_Finalize();
}
------------------
I am running Windows Vista on both machines. Has anyone seen this before? Thanks,
David
_______________________________________________
mpich-discuss mailing list
mpich-discuss at mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
_______________________________________________
mpich-discuss mailing list
mpich-discuss at mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
_______________________________________________
mpich-discuss mailing list
mpich-discuss at mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
_______________________________________________
mpich-discuss mailing list
mpich-discuss at mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
_______________________________________________
mpich-discuss mailing list
mpich-discuss at mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
More information about the mpich-discuss
mailing list