[mpich-discuss] The working of mapall option on windows

Fri Nov 5 18:08:42 CDT 2010

Hi

Just to bring a closure to this issue: We recently found in our program (100K lines of Fortran77) a variable that was uninitialized for rank > 0 processes. We think that it explains the erratic behavior previously seen with our parallel runs. After fixing this bug, the -localonly and -mapall seem to work as expected.

Thanks for your help
Kottalam

-----Original Message-----
From: Jayesh Krishna [mailto:jayesh at mcs.anl.gov] 
Sent: Friday, March 26, 2010 8:16 AM
To: Jeyapandian Kottalam
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] The working of mapall option on windows

Hi,
>> On this machine I had trouble with -mapall, -localonly works and -host localhost does not work even with -mapall, even when the machine is always on the network

 So is this problem (-mapall does not work) specific to a machine ? Does the option work on other machines ? Is the program, myMPIpgm.exe (in your prev email), located on a network drive (I did not understand what you meant when you mentioned that the program failed when there are network drives.)?

 SMPD does have a debug logging option (-d/-traceon options). I would like to understand your problem better before proceeding with logging.

(PS: There is no special network configuration required for MPICH2 if you are running your job on the local machine.)
Regards,
Jayesh
----- Original Message -----
From: "Jeyapandian Kottalam" <JKottalam at reactiondesign.com>
To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
Cc: mpich-discuss at mcs.anl.gov
Sent: Thursday, March 25, 2010 7:15:13 PM GMT -06:00 US/Canada Central
Subject: RE: [mpich-discuss] The working of mapall option on windows

Hi Jayesh

We will consider upgrading MPICH2. Meanwhile the use of the -machinefile mf.txt in place of -host hostname did not change the behavior. I should have mentioned that I started exploring these options only after the straight 'mpiexec -n 2 myMPIpgm.exe' failed when there are mapped drives.

In general, are there any special network configuration requirements for MPICH2 on Windows, even if I am only running on a local machine?

Does SMPD write a log file? Is there any way of getting verbose output from its progress?

Thanks
Kottalam

-----Original Message-----
From: Jayesh Krishna [mailto:jayesh at mcs.anl.gov] 
Sent: Thursday, March 25, 2010 7:21 AM
To: Jeyapandian Kottalam
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] The working of mapall option on windows

Hi,
 The bug associated with the "-localonly" option ("shutdown failed") should be fixed in the latest release (http://www.mcs.anl.gov/research/projects/mpich2/downloads/index.php?s=downloads). As I mentioned before, SMPD launches all the processes locally if you don't specify "-host" or the "-machinefile" option. So if you just want to launch procs locally you don't need the "-localonly" option.
 Does the option work with a machinefile (mpiexec -n 2 -mapall -machinefile mf.txt myMPIpgm.exe)?

Regards,
Jayesh
----- Original Message -----
From: "Jeyapandian Kottalam" <JKottalam at reactiondesign.com>
To: jayesh at mcs.anl.gov
Cc: mpich-discuss at mcs.anl.gov
Sent: Wednesday, March 24, 2010 11:52:33 PM GMT -06:00 US/Canada Central
Subject: RE: [mpich-discuss] The working of mapall option on windows

Hi Jayesh

It does not work with -host localhost -mapall even when localhost is replaced with its hostname or its IP address. There is no error message printed. Within a second of typing the mpiexec command, I get the command prompt back. 

However, with -localonly (and without -mapall) the program runs to completion, but at the end there is the following error message:

shutdown failed, sock 860, error 10093
closesocket failed, sock860, error 10093

Does this tell us anything useful?

Thanks
Kottalam

-----Original Message-----
From: jayesh at mcs.anl.gov [mailto:jayesh at mcs.anl.gov] 
Sent: Monday, March 22, 2010 7:28 AM
To: Jeyapandian Kottalam
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] The working of mapall option on windows

Hi,
 The "-localonly" option is used to specify that all MPI processes be launched on the localhost. This option also performs some bypassing of the normal launching mechanism for the MPI job. The "-mapall" option should internally be turned off in this case since the drive names are implicitly mapped to the drive names on the local machine (The library is trying to map the same network drive again using the same drive letter). I would recommend that you don't use the "-mapall" option with the "-localonly" option. The MPI job is launched on the local machine if you don't specify any hosts (-host, -machinefile) anyway, so you should not need to use the "-localonly" option (Unless you have GUI components in your MPI app).
 Try using a complete hostname (instead of localhost) or the ipaddress and see if the "-mapall" option works with "-host" option.
 Let us know the results.

Regards,
Jayesh
----- Original Message -----
From: "Jeyapandian Kottalam" <JKottalam at reactiondesign.com>
To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
Cc: mpich-discuss at mcs.anl.gov
Sent: Sunday, March 21, 2010 3:54:50 PM GMT -06:00 US/Canada Central
Subject: RE: [mpich-discuss] The working of mapall option on windows

Thank you. This is helpful. My take-home message is that I should use the -mapall option on the mpiexec command whenever I expect the program to do i/o on mapped drives.

I have another question on mpiexec options. What exactly is the difference between using -localonly and -host localhost? On this machine I had trouble with -mapall, -localonly works and -host localhost does not work even with -mapall, even when the machine is always on the network. How do these options impact the ability to undock laptops while an MPI program is running?

Thanks
Kottalam

-----Original Message-----
From: Jayesh Krishna [mailto:jayesh at mcs.anl.gov] 
Sent: Friday, March 19, 2010 12:07 PM
To: Jeyapandian Kottalam
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] The working of mapall option on windows

Hi,
 The mapped network drives should be visible to the process launcher, mpiexec.exe, on the local machine since the process launcher runs under the security context of the local user (i.e., You run the job launcher as the user logged on to the machine). However the mapped folders are not available to the process manager on the remote machine unless you explicitly map them (On the remote machine SMPD impersonates as the user running the job, however network drives are not mapped automatically).
 You should also note that although the MPI program on a network mapped drive is visible to the process launcher without any explicit mapping, the MPI process itself won't be able to access the mapped drive (read a file from network mapped drive within your MPI program) unless you explicitly map them.
 Hope it helps.

Regards,
Jayesh
----- Original Message -----
From: "Jeyapandian Kottalam" <JKottalam at reactiondesign.com>
To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
Cc: mpich-discuss at mcs.anl.gov
Sent: Thursday, March 18, 2010 5:00:10 PM GMT -06:00 US/Canada Central
Subject: RE: [mpich-discuss] The working of mapall option on windows

I just compared the Properties and Advanced Security tabs of this drive on both computers. These are desktop computers plugged in to the network all the time. So they have not been made available offline. The only thing that is turned on, on both machines is "Allow indexing services" for fast searching.

- Kottalam

-----Original Message-----
From: Jayesh Krishna [mailto:jayesh at mcs.anl.gov] 
Sent: Thursday, March 18, 2010 7:10 AM
To: Jeyapandian Kottalam
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] The working of mapall option on windows

Hi,
 Do you have any special settings for the network drives (eg: "Make available offline" )?

Regards,
Jayesh
----- Original Message -----
From: "Jeyapandian Kottalam" <JKottalam at reactiondesign.com>
To: jayesh at mcs.anl.gov
Cc: mpich-discuss at mcs.anl.gov
Sent: Wednesday, March 17, 2010 4:35:10 PM GMT -06:00 US/Canada Central
Subject: RE: [mpich-discuss] The working of mapall option on windows

Hi Jayesh

I am just running on a single machine with multiple cores. On both machines I start the program using the commands

mpiexec -n 2 program
mpiexec -mapall -n 2 program

The latter behaves the same on both machines, but the former works on only one of them. On the other machine it stops without finding the input file (which is on a mapped drive).

Thanks
Kottalam

-----Original Message-----
From: jayesh at mcs.anl.gov [mailto:jayesh at mcs.anl.gov] 
Sent: Wednesday, March 17, 2010 10:51 AM
To: Jeyapandian Kottalam
Cc: mpich-discuss at mcs.anl.gov
Subject: Re: [mpich-discuss] The working of mapall option on windows

Hi,
 Can you provide the complete mpiexec command that you use to launch your MPI program ? Are you running the MPI jobs locally on each machine (MPI processes belonging to the job are launched only on the localhost)?

Regards,
Jayesh
----- Original Message -----
From: "Jeyapandian Kottalam" <JKottalam at reactiondesign.com>
To: mpich-discuss at mcs.anl.gov
Sent: Wednesday, March 17, 2010 10:49:31 AM GMT -06:00 US/Canada Central
Subject: [mpich-discuss] The working of mapall option on windows

Hi 

I have built my program with MPICH2 1.0.8p1 on Windows XP. My input and output files are on mapped drives. I run the program on two different Windows XP machines and get different behaviors. On one machine, I can run the program with or without the -mapall option on mpiexec. On the other machine, the program works only with the -mapall option. On both machines the drives are always mapped at the OS level. Both machines are configured identically as far as I can tell; in fact one was cloned from the other. For supporting my users, I would like to understand why it works on one machine without the -mapall option but not on the other. Where would I look for the differences in configuration? 

Thanks 

Kottalam 
_______________________________________________
mpich-discuss mailing list
mpich-discuss at mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss