[mpich2-dev] build issues with MPE on Windows(32) and MinGW

Mon Nov 2 16:21:13 CST 2009

Hi,
 Where do you have libmpe.a (It is not installed with MPICH2 on windows and you seem to be linking your code with "-lmpe")?

(PS: The problem with multiple installations of MPICH2 was a result of us not upgrading the product versions correctly for MPICH2. We expected users to uninstall their previous version of MPICH2 before installing a new version. This will not happen for future releases.)

Regards,
Jayesh
----- Original Message -----
From: "Lisandro Dalcin" <dalcinl at gmail.com>
To: mpich2-dev at mcs.anl.gov
Cc: jayesh at mcs.anl.gov
Sent: Monday, November 2, 2009 4:10:38 PM GMT -06:00 US/Canada Central
Subject: Re: [mpich2-dev] build issues with MPE on Windows(32) and MinGW

On Mon, Nov 2, 2009 at 7:41 PM,  <jayesh at mcs.anl.gov> wrote:
> Hi,
>  Each MPICH2 release on windows is independent of each other (You shouldn't have to install a version of MPICH2 before installing another). Let us know the problem you faced and we can sort it out.

Well, at this point I'm not sure what happened. As I got confused with
your previous mail, I decided to unistall all the MPI's I had in my
Windows system. I went to "Add/Remove Programs" (not sure if it is the
exact translation, using a Spanish version of Windows here) and had
three MPICH2 entries; I've uninstalled the latest one with success,
but I was not able to uninstall the other two, a box "missing program
required for uninstall" or something like that appeared. Then I had no
other choice to manually search for and remove any file (mostly DLL's
in c:\windows\system32 ) let out (and use some utilities to remove the
MPICH2 entries for the installed program lists...)

Perhaps all this nightmare was just my fault... I really do not have
any Windows experience, I'm just making sure that my Python bindings
can be built and used on Windows against MPICH2, DeinoMPI and MS HPC
Cluster Pack 08, with either MSVC and MinGW...

>
>  Which library did you use to compile/link your code (Please provide us with the path/name of the libraries)? Is your code written in C/python/C++/fortran ? Please provide us more details regarding your program.
>

You have attached the whole output I get when building mpi4py on Win32
with MPICH2 and MinGW.

I pasted below the most relevant info which appear at the end of the
log, i.e, the part where MPE is being actually used (this output is
from some monkeypatched Python's distutils in order to make it more
featured):

building 'mpi4py.MPE' extension
C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -DHAVE_MPE=1
-IC:\Python26\include -IC:\Python26\PC "-IC:\Archivos de
programa\MPICH2\include" -c src\MPE.c -o
build\temp.win32-2.6\Release\src\mpe.o
writing build\temp.win32-2.6\Release\src\MPE.def
C:\MinGW\bin\gcc.exe -mno-cygwin -shared -s
build\temp.win32-2.6\Release\src\mpe.o
build\temp.win32-2.6\Release\src\MPE.def -LC:\Python26\libs
-LC:\Python26\PCbuild "-LC:\Archivos de programa\MPICH2\lib" -lmpe
-lpython26 -lmsvcr90 -lmpi -o build\lib.win32-2.6\mpi4py\MPE.pyd

After that, I'm able to "from mpi4py import MPE", and the
MPE-instrumented script here:
http://code.google.com/p/mpi4py/source/browse/trunk/demo/mpe-logging/cpilog.py
works just fine, generating a "cpilog.clog2" file at the end of the
run. No MPI calls are logged (because of the lack of RTLD_GLOBAL
semantics on Win), just the user-defined states and events.

So this makes me think that using MinGW and linking against MPE does
actually work. However, I point it again, these two includes (stdint.h
and intypes.h) are required to workaround build failures. Again, I
paste to my mpi4py pach where you can see these two headers included
right above including mpe.h

http://code.google.com/p/mpi4py/source/diff?spec=svn522&r=522&format=side&path=/trunk/src/MPE/mpe-log.c.

>
> (PS: You should be able to do stuff similar to dlopen() with LoadLibrary() on windows...)
>

Do you know of any way of getting RTLD_GLOBAL semantics? I know about
LoadLibrary(), but AFAIK there is not equivalent do dlopen() with
mode=RTLD_GLOBAL... Perhaps I'm plain wrong, as I have no experience
on Windows... Any point on this will be appreciated.

> -Jayesh
>
> ----- Original Message -----
> From: "Lisandro Dalcin" <dalcinl at gmail.com>
> To: mpich2-dev at mcs.anl.gov
> Cc: "Jayesh Krishna" <jayesh at mcs.anl.gov>
> Sent: Monday, November 2, 2009 3:20:50 PM GMT -06:00 US/Canada Central
> Subject: Re: [mpich2-dev] build issues with MPE on Windows(32) and MinGW
>
> On Mon, Nov 2, 2009 at 5:35 PM,  <jayesh at mcs.anl.gov> wrote:
>> Hi,
>
> Sorry for taking so long to get back... I have little Windows
> experience, and your mail REALLY confused me. I had to start from
> scratch and uninstall DeinoMPI, MPICH2 and MS HPC Cluster Pack from my
> Windows system to make sure I was not being bitten by the infamous
> "DLL Hell" :-) ... In the process, I've discovered that for upgrading
> MPICH2, fist I have to uninstall the previous version :-( (BTW, is
> this right?)... But I think I was able to get things cleaned-up and
> then re-installed latest MPICH2-1.2 using the MSI installer.
>
>>
>>  We currently don't provide a mingw compatible library for MPE on windows.
>>
>
> What that means? This is the part that confused me...
>
> I've just built a Python extension module linked against MPE...
> because of the way my code works and Windows limitations (no
> equivalent to dlopen() with RTLD_GLOBAL, AFAIK), I cannot log MPI
> calls, but the logging of user-defined event and states is definitely
> working (to be honest, just a sequential try, no mpiexec involved).
>
> So I think you are wrong... MPE (at least some parts) do work with
> MinGW on Windows(32) ... Why did you said that this is not supported??
>
>>
>> Why are you using the MPE header files provided with MPICH2 on windows with mingw?
>>
>
> Because I want my code (i.e, mpi4py) to work with MinGW. No more, no
> less... And part of MPE (I mean, at least the logging of user-defined
> states and events) DO WORK with MinGW after using the fix commented in
> my previous mail (I mean, #include stdint.h and inttypes.h)
>
>>
>> (PS: The CLOG types are defined using pre-defined windows types. There is no need to include <stdint.h> or <inttypes.h> on windows.)
>>
>
> But if you want to use MinGW, it seems you have to include these two
> headers to make GCC "understand" these pre-defined windows types.
>
>
> PS: Am I being clear enough?
>
>
>> Regards,
>> Jayesh
>> ----- Original Message -----
>> From: "Lisandro Dalcin" <dalcinl at gmail.com>
>> To: mpich2-dev at mcs.anl.gov
>> Sent: Monday, November 2, 2009 1:02:46 PM GMT -06:00 US/Canada Central
>> Subject: [mpich2-dev] build issues with MPE on Windows(32) and MinGW
>>
>> Anthony, this is unrelated to our previous discussions about -fPIC, so
>> I'm reporting the issue here. I hope it is the right place. If not,
>> let me know (perhaps mpich-discuss ??).
>>
>> This diff at Google Code will be clear enough, I think:
>>
>> http://code.google.com/p/mpi4py/source/diff?spec=svn522&r=522&format=side&path=/trunk/src/MPE/mpe-log.c
>>
>> Without these two headers included, my Python ext module fails bad to
>> build with MinGW. Interestingly, "clog_inttypes.h" has these two
>> includes commented out on Windows (using the binary installer here).
>> So perhaps protecting the inclusion of these two headers as shown in
>> the link above will do the trick? To put it clear, I'm talking about
>> using this:
>>
>>   #if defined(_WIN32) && defined(__GNUC__)
>>     #include <stdint.h>
>>     #include <inttypes.h>
>>   #endif
>>
>>
>> --
>> Lisandro Dalcín
>> ---------------
>> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
>> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
>> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
>> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>>
>
>
>
> --
> Lisandro Dalcín
> ---------------
> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
>

-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594