[mpich2-dev] build issues with MPE on Windows(32) and MinGW

Mon Nov 2 16:49:03 CST 2009

On Mon, Nov 2, 2009 at 8:21 PM,  <jayesh at mcs.anl.gov> wrote:
> Hi,
>  Where do you have libmpe.a (It is not installed with MPICH2 on windows and you seem to be linking your code with "-lmpe")?
>

Well, I have installed a "mpe.lib" (import lib?) in
%ProgramFiles%\MPICH2\lib, and a "mpe.dll" in c:\windows\system32 ...
So my code is likely reading stuff from mpe.lib and linking with
mpe.dll, right? I think libmpe.a is not required at all... BTW, all
this is on Windows XP.

Remember, I've manually removed any trace of other MPI/MPE installs,
so I got these libs after re-installing the latest MPICH2 release
(moreover, the dates of the installed files are all the same). So I
think the MPE libs are from my last MPICH2 install...

>
> (PS: The problem with multiple installations of MPICH2 was a result of us not upgrading the product versions correctly for MPICH2. We expected users to uninstall their previous version of MPICH2 before installing a new version. This will not happen for future releases.)
>

OK... Good to know you are going to fix this, hunting for files left
around after a broken uninstall is not a pleasure :-) ...

> Regards,
> Jayesh
> ----- Original Message -----
> From: "Lisandro Dalcin" <dalcinl at gmail.com>
> To: mpich2-dev at mcs.anl.gov
> Cc: jayesh at mcs.anl.gov
> Sent: Monday, November 2, 2009 4:10:38 PM GMT -06:00 US/Canada Central
> Subject: Re: [mpich2-dev] build issues with MPE on Windows(32) and MinGW
>
> On Mon, Nov 2, 2009 at 7:41 PM,  <jayesh at mcs.anl.gov> wrote:
>> Hi,
>>  Each MPICH2 release on windows is independent of each other (You shouldn't have to install a version of MPICH2 before installing another). Let us know the problem you faced and we can sort it out.
>
> Well, at this point I'm not sure what happened. As I got confused with
> your previous mail, I decided to unistall all the MPI's I had in my
> Windows system. I went to "Add/Remove Programs" (not sure if it is the
> exact translation, using a Spanish version of Windows here) and had
> three MPICH2 entries; I've uninstalled the latest one with success,
> but I was not able to uninstall the other two, a box "missing program
> required for uninstall" or something like that appeared. Then I had no
> other choice to manually search for and remove any file (mostly DLL's
> in c:\windows\system32 ) let out (and use some utilities to remove the
> MPICH2 entries for the installed program lists...)
>
> Perhaps all this nightmare was just my fault... I really do not have
> any Windows experience, I'm just making sure that my Python bindings
> can be built and used on Windows against MPICH2, DeinoMPI and MS HPC
> Cluster Pack 08, with either MSVC and MinGW...
>
>
>>
>>  Which library did you use to compile/link your code (Please provide us with the path/name of the libraries)? Is your code written in C/python/C++/fortran ? Please provide us more details regarding your program.
>>
>
> You have attached the whole output I get when building mpi4py on Win32
> with MPICH2 and MinGW.
>
> I pasted below the most relevant info which appear at the end of the
> log, i.e, the part where MPE is being actually used (this output is
> from some monkeypatched Python's distutils in order to make it more
> featured):
>
> building 'mpi4py.MPE' extension
> C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -DHAVE_MPE=1
> -IC:\Python26\include -IC:\Python26\PC "-IC:\Archivos de
> programa\MPICH2\include" -c src\MPE.c -o
> build\temp.win32-2.6\Release\src\mpe.o
> writing build\temp.win32-2.6\Release\src\MPE.def
> C:\MinGW\bin\gcc.exe -mno-cygwin -shared -s
> build\temp.win32-2.6\Release\src\mpe.o
> build\temp.win32-2.6\Release\src\MPE.def -LC:\Python26\libs
> -LC:\Python26\PCbuild "-LC:\Archivos de programa\MPICH2\lib" -lmpe
> -lpython26 -lmsvcr90 -lmpi -o build\lib.win32-2.6\mpi4py\MPE.pyd
>
> After that, I'm able to "from mpi4py import MPE", and the
> MPE-instrumented script here:
> http://code.google.com/p/mpi4py/source/browse/trunk/demo/mpe-logging/cpilog.py
> works just fine, generating a "cpilog.clog2" file at the end of the
> run. No MPI calls are logged (because of the lack of RTLD_GLOBAL
> semantics on Win), just the user-defined states and events.
>
> So this makes me think that using MinGW and linking against MPE does
> actually work. However, I point it again, these two includes (stdint.h
> and intypes.h) are required to workaround build failures. Again, I
> paste to my mpi4py pach where you can see these two headers included
> right above including mpe.h
>
> http://code.google.com/p/mpi4py/source/diff?spec=svn522&r=522&format=side&path=/trunk/src/MPE/mpe-log.c.
>
>
>>
>> (PS: You should be able to do stuff similar to dlopen() with LoadLibrary() on windows...)
>>
>
> Do you know of any way of getting RTLD_GLOBAL semantics? I know about
> LoadLibrary(), but AFAIK there is not equivalent do dlopen() with
> mode=RTLD_GLOBAL... Perhaps I'm plain wrong, as I have no experience
> on Windows... Any point on this will be appreciated.
>
>
>> -Jayesh
>>
>> ----- Original Message -----
>> From: "Lisandro Dalcin" <dalcinl at gmail.com>
>> To: mpich2-dev at mcs.anl.gov
>> Cc: "Jayesh Krishna" <jayesh at mcs.anl.gov>
>> Sent: Monday, November 2, 2009 3:20:50 PM GMT -06:00 US/Canada Central
>> Subject: Re: [mpich2-dev] build issues with MPE on Windows(32) and MinGW
>>
>> On Mon, Nov 2, 2009 at 5:35 PM,  <jayesh at mcs.anl.gov> wrote:
>>> Hi,
>>
>> Sorry for taking so long to get back... I have little Windows
>> experience, and your mail REALLY confused me. I had to start from
>> scratch and uninstall DeinoMPI, MPICH2 and MS HPC Cluster Pack from my
>> Windows system to make sure I was not being bitten by the infamous
>> "DLL Hell" :-) ... In the process, I've discovered that for upgrading
>> MPICH2, fist I have to uninstall the previous version :-( (BTW, is
>> this right?)... But I think I was able to get things cleaned-up and
>> then re-installed latest MPICH2-1.2 using the MSI installer.
>>
>>>
>>>  We currently don't provide a mingw compatible library for MPE on windows.
>>>
>>
>> What that means? This is the part that confused me...
>>
>> I've just built a Python extension module linked against MPE...
>> because of the way my code works and Windows limitations (no
>> equivalent to dlopen() with RTLD_GLOBAL, AFAIK), I cannot log MPI
>> calls, but the logging of user-defined event and states is definitely
>> working (to be honest, just a sequential try, no mpiexec involved).
>>
>> So I think you are wrong... MPE (at least some parts) do work with
>> MinGW on Windows(32) ... Why did you said that this is not supported??
>>
>>>
>>> Why are you using the MPE header files provided with MPICH2 on windows with mingw?
>>>
>>
>> Because I want my code (i.e, mpi4py) to work with MinGW. No more, no
>> less... And part of MPE (I mean, at least the logging of user-defined
>> states and events) DO WORK with MinGW after using the fix commented in
>> my previous mail (I mean, #include stdint.h and inttypes.h)
>>
>>>
>>> (PS: The CLOG types are defined using pre-defined windows types. There is no need to include <stdint.h> or <inttypes.h> on windows.)
>>>
>>
>> But if you want to use MinGW, it seems you have to include these two
>> headers to make GCC "understand" these pre-defined windows types.
>>
>>
>> PS: Am I being clear enough?
>>
>>
>>> Regards,
>>> Jayesh
>>> ----- Original Message -----
>>> From: "Lisandro Dalcin" <dalcinl at gmail.com>
>>> To: mpich2-dev at mcs.anl.gov
>>> Sent: Monday, November 2, 2009 1:02:46 PM GMT -06:00 US/Canada Central
>>> Subject: [mpich2-dev] build issues with MPE on Windows(32) and MinGW
>>>
>>> Anthony, this is unrelated to our previous discussions about -fPIC, so
>>> I'm reporting the issue here. I hope it is the right place. If not,
>>> let me know (perhaps mpich-discuss ??).
>>>
>>> This diff at Google Code will be clear enough, I think:
>>>
>>> http://code.google.com/p/mpi4py/source/diff?spec=svn522&r=522&format=side&path=/trunk/src/MPE/mpe-log.c
>>>
>>> Without these two headers included, my Python ext module fails bad to
>>> build with MinGW. Interestingly, "clog_inttypes.h" has these two
>>> includes commented out on Windows (using the binary installer here).
>>> So perhaps protecting the inclusion of these two headers as shown in
>>> the link above will do the trick? To put it clear, I'm talking about
>>> using this:
>>>
>>>   #if defined(_WIN32) && defined(__GNUC__)
>>>     #include <stdint.h>
>>>     #include <inttypes.h>
>>>   #endif
>>>
>>>
>>> --
>>> Lisandro Dalcín
>>> ---------------
>>> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
>>> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
>>> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
>>> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
>>> Tel/Fax: +54-(0)342-451.1594
>>>
>>
>>
>>
>> --
>> Lisandro Dalcín
>> ---------------
>> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
>> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
>> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
>> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>>
>
>
>
> --
> Lisandro Dalcín
> ---------------
> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
>

-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594