[mpich-discuss] Problem mixing 32 bit and 64 bit machines

[Jayesh Krishna]

Hi,
 As long as you use the same data model for your MPI application (which depends on the MPICH2 library you use to compile/link your application) you should be good to go. The process managers (the process managers don't participate in the communication btw MPI processes) communicate with each other using the PMI protocol which should work across machines with different data models.

Regards,
Jayesh
----- Original Message -----
From: "Peter Smith" <smithpd at lanl.gov>
To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
Sent: Monday, March 22, 2010 7:14:40 PM GMT -06:00 US/Canada Central
Subject: Re: [mpich-discuss] Problem mixing 32 bit and 64 bit machines

Jayesh,

That's interesting.  While waiting for your reply, I realized that I did have a test configuration I could try, so I innocently tried the mixed versions of mpich2 that you said would not work:

You said:
>The following configuration won't work,
># IA32 Machine A with 32-bit version of MPICH2 + x64 Machine B with 64-bit version of MPICH2.

It did work for me. I wonder if I did the impossible or there was something unique about my application.  I'll explain how I did it and the nature of the data passed by my MPI app.

I used a machine file that started two processes on each of two machines, 2 on IA32 and 2 on x64.  The machine file had the following format:

mach32_IP:2
mach64_IP:2

A colleague has since done it with more hosts.

The program being launched in both cases was an IA32 EXE.  The IA32 machine, of course, will not run the x64 EXE.  

I checked the MPICH2 DLLS to be sure that the native versions of MPICH2 were installed on each machine.  Also, the IA32 and the X64 versions of MPICH2 were both in the folder Program Files (native).

I launched my 32 bit EXE using mpiexec, first invoking the IA32 version of mpieced on the IA32 machine and then invoking the x64 version of mpiexec on the X64 machine. Both cases 
worked.

The mpiexec executables are both dated 4/9/2009.  
ia32 version date = 4/9/2009, 11:08 AM
x64 version date = 4/9/2009, 11:31 AM

I am pretty sure that the install files were:
mpich2-1.0.8p1-win-ia32.msi
mpich2-1.0.8p1-win-x86-64.msi

My MPI application transfers mostly single 32 bit integers between hosts, but in one case (included in these test runs) the code also one transfers a struct containing two 32 bit floats. The struct is converted to an MPI data type, MPI_Type_struct.  If you want, I can send you the code for creating the MPI_Type_struct.  The command that sends the data is:

if(MPI_SUCCESS != MPI_Send(&cen_data, 1, mpiCenDataStruct, slave, 0, world))
{// print error message}

Does that simplified (or rigorous) data passing procedure make it possible?

Peter


At 01:06 PM 3/22/2010, you wrote:
>Hi,
> You cannot run MPI jobs using MPICH2 across MPICH2 installations with different data models.
>
>The following configuration won't work,
># IA32 Machine A with 32-bit version of MPICH2 + x64 Machine B with 64-bit version of MPICH2.
>
>The following tested configurations should work,
># IA32 Machine A with 32-bit version of MPICH2 + IA32 machine B with 32-bit version of MPICH2
># x64 Machine A with 64-bit version of MPICH2 + x64 machine B with 64-bit version of MPICH2
>
>The following untested configuration should work,
># IA32 Machine A with 32-bit version of MPICH2 + x64 Machine B with 32-bit version of MPICH2.
>
>Regards,
>Jayesh
>----- Original Message -----
>From: "Peter Smith" <smithpd at lanl.gov>
>To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
>Sent: Monday, March 22, 2010 1:21:36 PM GMT -06:00 US/Canada Central
>Subject: Re: [mpich-discuss] Problem mixing 32 bit and 64 bit machines
>
>Thanks.
>
>OK, one more question.  How do you think it will it work if I have native MPICH2 on each machine?  Can 32 bit MPICH2 running on a 32 bit machine communicate with 64 bit MPICH2 running on a 64 bit machine?  If so, does it matter which machine the mpiexec launch is run from?
>
>Peter
>
>At 12:14 PM 3/22/2010, you wrote:
>>Hi,
>> Yes, installing the 32-bit version of MPICH2 on all the machines should work for you (We don't routinely test these hetero envs but it should work).
>> Let us know if you have any issues.
>>
>>(PS: Make sure that you have only one version of MPICH2 installed on a machine.)
>>Regards,
>>Jayesh
>>----- Original Message -----
>>From: "Peter Smith" <smithpd at lanl.gov>
>>To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
>>Sent: Monday, March 22, 2010 12:12:25 PM GMT -06:00 US/Canada Central
>>Subject: Re: [mpich-discuss] Problem mixing 32 bit and 64 bit machines
>>
>>Jayesh,
>>
>>You have confirmed my suspicion about installing 2 versions of mpich2 on one machine.
>>
>>What you describe for copying the libraries is exactly what I have done to build the application on both 32 bit and 64 bit platforms using my 64 bit machine.
>>
>>My main question, though, was how to run the application under mpiexec on a mixture of 32 bit and 64 bit platforms. I quote from the previous message"
>>
>>"The real problem -- What I would like to be able to do is to launch a 32 bit MPI executable on a mix of 32 bit and 64 bit machines.  I suppose we could install the 32 bit version of mpiexec on all these machines including the 64 bit machines.  Would it work then in mixed hardware environments?"
>>
>>Can you explain that?
>>
>>Thanks,
>>
>>Peter Smith
>>
>>
>>
>>At 07:38 AM 3/21/2010, you wrote:
>>>Hi,
>>> MPICH2 has separate dlls (although the dll names are the same) for IA32 and x64 applications. So in your case the 32-bit application does not work (missing dll error message) because you don't have the IA32 version of the dll installed in your system.
>>> We currently do not support installing IA32 and x64 versions of MPICH2 on the same machine (It is in our todo list though).
>>> So in your case you will have to manually copy the libs (MPICH2\lib\*), the header files (MPICH2\include\*) and the dlls (c:\windows\system32\mpich*.dll , c:\windows\system32\fmpich*.dll, c:\windows\system32\mpe*.dll) from an IA32 installation to your x64 machine. Copy the dlls to the local directory containing your executable. I know it is not an optimal solution, however that is the best solution for you as of now.
>>> Let us know if it works for you.
>>>
>>>Regards,
>>>Jayesh
>>>----- Original Message -----
>>>From: "Peter Smith" <smithpd at lanl.gov>
>>>To: mpich-discuss at mcs.anl.gov
>>>Sent: Friday, March 19, 2010 5:09:49 PM GMT -06:00 US/Canada Central
>>>Subject: [mpich-discuss] Problem mixing 32 bit and 64 bit machines
>>>
>>>Background -- I have installed the 64 bit version of MPICH2 on my x64 computer, and I have both 32 and 64 bit versions of the libraries that I can use to build 32 bit and 64 bit versions of my code.  These code versions run fine on their respective target platforms.  However, when running the 32 bit MPI application on my 64 bit machine (either by itself or launched by mpiexec), it complains about a missing DLL, mpich2mpi.dll. This DLL name exists in C:\WINDOWS\system32.  Evidently that is a 64 bit version of the DLL, and it is incompatible with the launching of a 32 bit EXE.
>>>
>>>The real problem -- What I would like to be able to do is to launch a 32 bit MPI executable on a mix of 32 bit and 64 bit machines.  I suppose we could install the 32 bit version of mpiexec on all these machines including the 64 bit machines.  Would it work then in mixed hardware environments?
>>>
>>>Also, is it possible to have 64 bit and 32 bit MPICH2 coexist on one machine?  Since mpich2mpi.dll has the same name, it seems unlikely now, but I am just asking if it is somehow possible.  If not, it might be a useful feature for the future.  Perhaps you could use unique names to avoid conflict.
>>>
>>>It is not clear to me what are the ground rules for mixing hardware platforms to run MPICH2 software builds: what works and what doesn't work, and why.  Kindly explain or point me to some documentation.  If you know a way to do this hardware mixing, please let me know.
>>>
>>>_______________________________________________
>>>mpich-discuss mailing list
>>>mpich-discuss at mcs.anl.gov
>>>https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss