[mpich-discuss] undefined symbol: MPIR_Thread

Matthias Demuzere Matthias.demuzere at ees.kuleuven.be
Thu Feb 3 09:48:41 CST 2011 

yes, ccsm.exe is created again, as a new folder is created each time I 
run the building process (meaning that the old one is removed).
So once the compilation starts, all is stored in folder /test1/....

In attach the log file for creating the ccsm executable, as mct is not 
the module that creates the ccsm.exe file.

On 02/03/2011 04:34 PM, Anthony Chan wrote:
> Your MCT build log does not contain the link command that creates ccsm.exe,
> so I am not sure if ccsm.exe has actually been recreated with the mpich2-1.3.1,
> i.e. you could be still use the old ccsm.exe that was linked with ubuntu mpich.
>
> So you need to remove all .o, .a (or .so) and ccsm.exe (or all .exe) files before
> compiling again to make sure everything is compiled and linked correctly with
> mpich2-1.3.1...
>
> A.Chan
>
> ----- Original Message -----
>> Ok!
>>
>> I realised that in first instance I installed mpich via the ubuntu
>> repository, and afterwards i built it myself.
>> That was the reason why I had another MPI installation present.
>>
>> So I un-installed the first version, and built the other one again, to
>> be sure.
>> I have set the references as you said:
>>
>> FC := /usr/local/mpich2-1.3.1/bin/mpif90
>> CC := /usr/local/mpich2-1.3.1/bin/mpicc
>> F90 := /usr/local/mpich2-1.3.1/bin/mpif90
>> MPICC := /usr/local/mpich2-1.3.1/bin/mpicc
>> MPIF90 := /usr/local/mpich2-1.3.1/bin/mpif90
>>
>> and compiled the model again. This went ok, an executable was created
>> (see attach for example of one of the modules)
>> Running the executable unfortunately gave me the same error:
>> ./ccsm.exe: symbol lookup error: ./ccsm.exe: undefined symbol:
>> MPIR_Thread
>>
>> So back to square one?
>>
>> Any new ideas?
>> Matthias
>>
>>
>> On 02/02/2011 07:08 PM, Anthony Chan wrote:
>>> You indicated mpich2-1.3.1 was installed in /usr/local/mpich2-1.3.1
>>> so /usr/local/bin/mpif90 refers to mpif90 from other MPI
>>> implementation.
>>> So you MUST specify fullpath to mpif90 in all your build scripts
>>> and/or
>>> makefiles. I also notice your bldlog uses compiler variables besides
>>> FC and CC,
>>> i.e. MPICC, MPIF90, F90..., and some of these fortran
>>>
>>> CC = $(MPICH_PATH)/bin/mpicc
>>> MPICC = $(MPICH_PATH)/bin/mpicc
>>> FC = $(MPICH_PATH)/bin/mpif90
>>> F90 = $(MPICH_PATH)/bin/mpif90
>>> MPIF90 = $(MPICH_PATH)/bin/mpif90
>>>
>>> and leave LIB_MPI and INC_MPI blank (that would help you locate
>>> which script
>>> and/or makefile that set FC to "gfortran" or "mpif90").
>>>
>>> The other option is to simply prefix your PATH with
>>> /usr/local/mpich2-1.3.1 to
>>> avoid /usr/local/bin's MPI compiler wrappers being used in CCSM
>>> build system,
>>> but then you have to check your environment every time running CCSM
>>> build system,
>>> and your build output won't show correct MPI implementation is being
>>> used or not.
>>> So I think you should follow the 1st approach that was suggested.
>>>
>>> A.Chan
>>>
>>> ----- Original Message -----
>>>> The files that I send were indeed with the LIB and INC definitions
>>>> included.
>>>> But I did the test as well without these indications (and setting
>>>> FC
>>>> and
>>>> CC to MPICH2/etc/etc/...), and the results were the same?
>>>> see example built log in attach.
>>>>
>>>> The "which mpif90" statement gives me:
>>>> /usr/local/bin/mpif90
>>>>
>>>> Matthias
>>>>
>>>> On 02/02/2011 04:48 PM, Anthony Chan wrote:
>>>>> All your bldlog files indicate that you set FC to mpif90 and
>>>>> NOT $(MPICH_PATH)/bin/mpif90 as I suggested earlier. You should
>>>>> set both LIB_MPI and INC_MPI to empty strings. Do you have other
>>>>> MPI implementations (older MPICH2 or other MPI implementations)
>>>>> installed on your system besides MPICH2-1.3.1 ? Do a "which
>>>>> mpif90"
>>>>> to find out.
>>>>>
>>>>> A.Chan
>>>>>
>>>>> ----- Original Message -----
>>>>>> Dear Anthony!
>>>>>>
>>>>>> I have now done what you suggested below:
>>>>>> Set FC and CC to mpif90 and mpicc and removing the inc_ and lib_
>>>>>> references.
>>>>>> Again, I could built the executable, but running the latter
>>>>>> results
>>>>>> in
>>>>>> ./ccsm.exe: undefined symbol: MPIR_Thread
>>>>>>
>>>>>> Therefore, I set everything back to before and added this to the
>>>>>> LIB_MPI
>>>>>> LIB_MPI := $(MPICH_PATH)/lib -lmpichf90 -lmpichf90 -lmpich -lopa
>>>>>> -lmpl -lrt -lpthread
>>>>>>
>>>>>> As the last arguments were shown by mpif90 -show. But again, same
>>>>>> error.
>>>>>>
>>>>>> For every module of the model, a log file is saved. I attached
>>>>>> the
>>>>>> 3
>>>>>> most important ones:
>>>>>> - pio: parallel IO
>>>>>> - lnd: the land module that I would like to test
>>>>>> - ccsm: the log for building the executable
>>>>>>
>>>>>> As far as I can see, there could be something in pio, but I'm not
>>>>>> sure,
>>>>>> and it doesn't seem to be enough to stop the compilation process.
>>>>>>
>>>>>> Wouldn't there be any other way to deal with this thread problem?
>>>>>> Kind regards,
>>>>>>
>>>>>> Matthias
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 02/02/2011 01:12 AM, Anthony Chan wrote:
>>>>>>> I took a quick look at the scripts that you sent but they ain't
>>>>>>> very
>>>>>>> helpful
>>>>>>> as these files source some other files which ain't included in
>>>>>>> your
>>>>>>> attachment.
>>>>>>> In any case, my knowledge on CCSM is limited. Since the build
>>>>>>> problem is CCSM
>>>>>>> specific, you may want to contact CCSM developers to see what it
>>>>>>> should be done.
>>>>>>>
>>>>>>> Given your final executable (or one of the executables) has
>>>>>>> unresolved symbol
>>>>>>> MPIR_Thread, so it seems it didn't link in MPICH2 libraries
>>>>>>> correctly.
>>>>>>> I would first set FC=to $(MPICH_PATH)/bin/mpif90 and
>>>>>>> CC=$(MPICH_PATH)/bin/mpicc
>>>>>>> and leave LIB_MPI and INC_MPI empty since mpif90 and mpicc
>>>>>>> should
>>>>>>> take care of
>>>>>>> LIB_MPI and INC_MPI. You may want to save the compile and
>>>>>>> linking
>>>>>>> output of
>>>>>>> whole build process to see what goes wrong.
>>>>>>>
>>>>>>> A.Chan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> mmm, I am not sure if I can answer this question.
>>>>>>>> It is a global climate model, with a lot of packages, and in
>>>>>>>> order
>>>>>>>> to
>>>>>>>> compile it I follow the installation guide of the model.
>>>>>>>> So I am not completely sure what I should provide you with?
>>>>>>>>
>>>>>>>> There is a seperate Macros files in which I have to adapt libs
>>>>>>>> and
>>>>>>>> flags
>>>>>>>> for my system. The contents are as follows:
>>>>>>>> # Note that CPPDEFS is set in Macros.cppdefs
>>>>>>>> CPPDEFS += -DLINUX -DSEQ_$(FRAMEWORK) -DFORTRANUNDERSCORE
>>>>>>>> -DNO_R16
>>>>>>>> -DNO_SHR_VMATH
>>>>>>>>
>>>>>>>> ifeq ($(compile_threaded), true)
>>>>>>>> CPPDEFS += -DTHREADED_OMP
>>>>>>>> endif
>>>>>>>>
>>>>>>>> FC := mpif90
>>>>>>>> CC := mpicc
>>>>>>>>
>>>>>>>> HDF5_PATH := /usr/local/hdf5
>>>>>>>> HDF5_LIB := $(HDF5_PATH)/lib -lhdf5 -lhdf5_hl
>>>>>>>> NETCDF_PATH := /usr/local/netcdf-4.1.1
>>>>>>>> INC_NETCDF := $(NETCDF_PATH)/include
>>>>>>>> LIB_NETCDF := $(NETCDF_PATH)/lib -lnetcdf -L/usr/local/hdf5/lib
>>>>>>>> -lhdf5_hl -lhdf5 -L/usr/local/zlib/lib -lz -lm
>>>>>>>> MOD_NETCDF := $(NETCDF_PATH)/include
>>>>>>>> MPICH_PATH := /usr/local/mpich2-1.3.1
>>>>>>>> INC_MPI := $(MPICH_PATH)/include
>>>>>>>> LIB_MPI := $(MPICH_PATH)/lib
>>>>>>>> MPI_LIB_NAME := mpich
>>>>>>>> PNETCDF_PATH :=
>>>>>>>> INC_PNETCDF :=
>>>>>>>> LIB_PNETCDF :=
>>>>>>>> LAPACK_LIBDIR :=
>>>>>>>> #=====================================================
>>>>>>>>
>>>>>>>> CFLAGS := $(CPPDEFS)
>>>>>>>> FIXEDFLAGS := -132
>>>>>>>> FREEFLAGS := -FR
>>>>>>>> ##FFLAGS := $(CPPDEFS) -g -fp-model precise -convert big_endian
>>>>>>>> -assume byterecl -ftz -traceback
>>>>>>>> FFLAGS := $(CPPDEFS)-g -O2 -I/usr/local/include
>>>>>>>> FFLAGS_NOOPT := $(FFLAGS) -O0
>>>>>>>> FFLAGS_OPT := -O2
>>>>>>>> LDFLAGS :=
>>>>>>>> AR := ar
>>>>>>>> MOD_SUFFIX := mod
>>>>>>>> CONFIG_SHELL :=
>>>>>>>>
>>>>>>>> So maybe something is wrong here???
>>>>>>>> In attach I also sent you the configure file, and the build
>>>>>>>> file
>>>>>>>> to
>>>>>>>> compile the model.
>>>>>>>> That works all fine. Only when running the obtained executable,
>>>>>>>> I
>>>>>>>> get
>>>>>>>> the error MPIR_Thread...
>>>>>>>> to run the .exe, I use the following:
>>>>>>>> mpirun ./ccsm.exe>&! ccsm.log.$LID
>>>>>>>>
>>>>>>>> as is mentioned in the test2.Latitude_E6400.run file in attach.
>>>>>>>>
>>>>>>>> I do not know if you can do something with this, if not, please
>>>>>>>> let
>>>>>>>> me
>>>>>>>> know....
>>>>>>>>
>>>>>>>> Matthias
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02/01/2011 09:17 PM, Anthony Chan wrote:
>>>>>>>>> ----- Original Message -----
>>>>>>>>>
>>>>>>>>>> I can also perform the test as in the installation guide:
>>>>>>>>>> mpiexec -n 5 ./examples/cpi
>>>>>>>>>>
>>>>>>>>>> Process 1 of 5 is on MaffieLE6400
>>>>>>>>>> Process 3 of 5 is on MaffieLE6400
>>>>>>>>>> Process 0 of 5 is on MaffieLE6400
>>>>>>>>>> Process 2 of 5 is on MaffieLE6400
>>>>>>>>>> Process 4 of 5 is on MaffieLE6400
>>>>>>>>>> pi is approximately 3.1415926544231230, Error is
>>>>>>>>>> 0.0000000008333298
>>>>>>>>>> wall clock time = 0.000435
>>>>>>>>>>
>>>>>>>>>> as far as I can see all works fine?
>>>>>>>>> Yes.
>>>>>>>>>
>>>>>>>>>> The reason I configured with the --with-thread-package=none
>>>>>>>>>> the
>>>>>>>>>> second
>>>>>>>>>> time was to deal with the MPIR_Thread error...
>>>>>>>>>> So I wonder how I can deal with this error?
>>>>>>>>> Let's just focus on using the mpich2 install that was
>>>>>>>>> configured
>>>>>>>>> without
>>>>>>>>> --with-thread-package. How did you compile your application ?
>>>>>>>>>
>>>>>>>>>> Thanks for your help!
>>>>>>>>>>
>>>>>>>>>> Matthias
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 02/01/2011 08:24 PM, Anthony Chan wrote:
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>> Dear Anthony,
>>>>>>>>>>>>
>>>>>>>>>>>> My apologies for posting to the wrong email list. I will
>>>>>>>>>>>> register
>>>>>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> other one so that in the future I can put my questions
>>>>>>>>>>>> there.
>>>>>>>>>>>> For now i'll post this directly to you, hope that is fine??
>>>>>>>>>>> Yes. I cc'ed my reply to mpich-discuss as someone else may
>>>>>>>>>>> have
>>>>>>>>>>> things to add.
>>>>>>>>>>>
>>>>>>>>>>>> I am not sure if I understand what you mean with:
>>>>>>>>>>>>
>>>>>>>>>>>> Did you use mpif90/mpif77 to link your code or just use
>>>>>>>>>>>> above
>>>>>>>>>>>> link
>>>>>>>>>>>> command which seems incomplete ?
>>>>>>>>>>>>
>>>>>>>>>>>> I used the following line to check how mpich2 was built, as
>>>>>>>>>>>> I
>>>>>>>>>>>> first
>>>>>>>>>>>> got
>>>>>>>>>>>> an error with f90 vs f95.
>>>>>>>>>>>> So with
>>>>>>>>>>>> /usr/local/mpich2-1.3.1/bin/mpif90 -show
>>>>>>>>>>>>
>>>>>>>>>>>> I get finally got:
>>>>>>>>>>>> gfortran -O2 -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>> -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>> -L/usr/local/mpich2-1.3.1/lib
>>>>>>>>>>>> -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread
>>>>>>>>>>>>
>>>>>>>>>>>> Do you mean that there is something missing here? And
>>>>>>>>>>>> should
>>>>>>>>>>>> I
>>>>>>>>>>>> add
>>>>>>>>>>>> it
>>>>>>>>>>>> than in my definition of INC_MPI and LIB_MPI?
>>>>>>>>>>> Let's focus if your mpich2 installation is correct first.
>>>>>>>>>>>> One other question. I was thinking that it could be useful
>>>>>>>>>>>> to
>>>>>>>>>>>> built
>>>>>>>>>>>> mpich2 without threading, so I used:
>>>>>>>>>>>> ./configure -prefix=/usr/local/mpich2-1.3.1
>>>>>>>>>>>> --with-thread-package=none
>>>>>>>>>>> Try "./configure --prefix=/usr/local/mpich2-1.3.1", without
>>>>>>>>>>> --with-thread-package.
>>>>>>>>>>>
>>>>>>>>>>> A.Chan
>>>>>>>>>>>
>>>>>>>>>>>> But then I get a whole list of errors, e.g.:
>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getn.o):
>>>>>>>>>>>> In
>>>>>>>>>>>> function `MPIU_Thread_CS_exit_lockname_recursive_impl_':
>>>>>>>>>>>> info_getn.c:(.text+0x124): undefined reference to
>>>>>>>>>>>> `MPIU_Thread_mutex_unlock'
>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getnth.o):
>>>>>>>>>>>> In
>>>>>>>>>>>> function `MPIU_Thread_CS_enter_lockname_recursive_impl_':
>>>>>>>>>>>> info_getnth.c:(.text+0x32): undefined reference to
>>>>>>>>>>>> `MPIU_Thread_mutex_lock'
>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getnth.o):
>>>>>>>>>>>> In
>>>>>>>>>>>> function `MPIU_Thread_CS_exit_lockname_recursive_impl_':
>>>>>>>>>>>> info_getnth.c:(.text+0xf4): undefined reference to
>>>>>>>>>>>> `MPIU_Thread_mutex_unlock'
>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getvallen.o):
>>>>>>>>>>>> In function
>>>>>>>>>>>> `MPIU_Thread_CS_enter_lockname_recursive_impl_':
>>>>>>>>>>>> info_getvallen.c:(.text+0x32): undefined reference to
>>>>>>>>>>>> `MPIU_Thread_mutex_lock'
>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getvallen.o):
>>>>>>>>>>>> In function `MPIU_Thread_CS_exit_lockname_recursive_impl_':
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Any ideas here?
>>>>>>>>>>>> Matthias
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -------- Original Message --------
>>>>>>>>>>>> Subject: Re: [mpich2-dev] undefined symbol: MPIR_Thread
>>>>>>>>>>>> Date: Tue, 1 Feb 2011 19:41:54 +0100
>>>>>>>>>>>> From: Anthony Chan<chan at mcs.anl.gov>
>>>>>>>>>>>> Reply-To: Anthony Chan<chan at mcs.anl.gov>,
>>>>>>>>>>>> "mpich2-dev at mcs.anl.gov"
>>>>>>>>>>>> <mpich2-dev at mcs.anl.gov>
>>>>>>>>>>>> To: mpich2-dev at mcs.anl.gov<mpich2-dev at mcs.anl.gov>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> mpich2-dev for mpich2 development not general MPICH usage
>>>>>>>>>>>> questions,
>>>>>>>>>>>> please send your question to mpich-discuss at mcs.anl.gov.
>>>>>>>>>>>>
>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>        All,
>>>>>>>>>>>>>
>>>>>>>>>>>>>        when running a model with mpirun ./exe
>>>>>>>>>>>>>        I get the following error:
>>>>>>>>>>>>>        ./exe: symbol lookup error: ./exe: undefined symbol:
>>>>>>>>>>>>>        MPIR_Thread
>>>>>>>>>>>>>
>>>>>>>>>>>>>        I have no idea how to deal with this. I use
>>>>>>>>>>>>>        mpich2-1.3.1,
>>>>>>>>>>>>>        compiled
>>>>>>>>>>>>>        with
>>>>>>>>>>>>>        gfortran, with the following libraries:
>>>>>>>>>>>>>        gfortran -O2 -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>>>        -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>>>        -L/usr/local/mpich2-1.3.1/lib
>>>>>>>>>>>>>        -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt
>>>>>>>>>>>>>        -lpthread
>>>>>>>>>>>> Did you use mpif90/mpif77 to link your code or just use
>>>>>>>>>>>> above
>>>>>>>>>>>> link
>>>>>>>>>>>> command which seems incomplete ?
>>>>>>>>>>>>
>>>>>>>>>>>> A.Chan
>>>>>>>>>>>>
>>>>>>>>>>>>>        Could anybody help me with this?
>>>>>>>>>>>>>
>>>>>>>>>>>>>        Regards,
>>>>>>>>>>>>>        Matthias
>>>>>>>>>>>>>
>>>>>>>>>>>>>        --
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>        -------------------------------------------------------------
>>>>>>>>>>>>>        FWO Post-doctoral Research Fellow
>>>>>>>>>>>>>        Department of Earth&  Environmental Sciences
>>>>>>>>>>>>>        Physical and Regional Geography Research Group
>>>>>>>>>>>>>        Regional climate studies
>>>>>>>>>>>>>
>>>>>>>>>>>>>        Celestijnenlaan 200E
>>>>>>>>>>>>>        3001 Heverlee (Leuven)
>>>>>>>>>>>>>        BELGIUM
>>>>>>>>>>>>>
>>>>>>>>>>>>>        Tel: + 32 16 326424
>>>>>>>>>>>>>        Fax: + 32 16 322980
>>>>>>>>>>>>>
>>>>>>>>>>>>>        http://geo.kuleuven.be/aow/
>>>>>>>>>>>>>        www.kuleuven.be/geography
>>>>>>>>>>>>>        -------------------------------------------------------------
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>> FWO Post-doctoral Research Fellow
>>>>>>>>>> Department of Earth&  Environmental Sciences
>>>>>>>>>> Physical and Regional Geography Research Group
>>>>>>>>>> Regional climate studies
>>>>>>>>>>
>>>>>>>>>> Celestijnenlaan 200E
>>>>>>>>>> 3001 Heverlee (Leuven)
>>>>>>>>>> BELGIUM
>>>>>>>>>>
>>>>>>>>>> Tel: + 32 16 326424
>>>>>>>>>> Fax: + 32 16 322980
>>>>>>>>>>
>>>>>>>>>> http://geo.kuleuven.be/aow/
>>>>>>>>>> www.kuleuven.be/geography
>>>>>>>>>> -------------------------------------------------------------
>>>>>>>> --
>>>>>>>>
>>>>>>>>
>>>>>>>> -------------------------------------------------------------
>>>>>>>> FWO Post-doctoral Research Fellow
>>>>>>>> Department of Earth&  Environmental Sciences
>>>>>>>> Physical and Regional Geography Research Group
>>>>>>>> Regional climate studies
>>>>>>>>
>>>>>>>> Celestijnenlaan 200E
>>>>>>>> 3001 Heverlee (Leuven)
>>>>>>>> BELGIUM
>>>>>>>>
>>>>>>>> Tel: + 32 16 326424
>>>>>>>> Fax: + 32 16 322980
>>>>>>>>
>>>>>>>> http://geo.kuleuven.be/aow/
>>>>>>>> www.kuleuven.be/geography
>>>>>>>> -------------------------------------------------------------
>>>>>> --
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------------
>>>>>> FWO Post-doctoral Research Fellow
>>>>>> Department of Earth&  Environmental Sciences
>>>>>> Physical and Regional Geography Research Group
>>>>>> Regional climate studies
>>>>>>
>>>>>> Celestijnenlaan 200E
>>>>>> 3001 Heverlee (Leuven)
>>>>>> BELGIUM
>>>>>>
>>>>>> Tel: + 32 16 326424
>>>>>> Fax: + 32 16 322980
>>>>>>
>>>>>> http://geo.kuleuven.be/aow/
>>>>>> www.kuleuven.be/geography
>>>>>> -------------------------------------------------------------
>>>> --
>>>>
>>>>
>>>> -------------------------------------------------------------
>>>> FWO Post-doctoral Research Fellow
>>>> Department of Earth&  Environmental Sciences
>>>> Physical and Regional Geography Research Group
>>>> Regional climate studies
>>>>
>>>> Celestijnenlaan 200E
>>>> 3001 Heverlee (Leuven)
>>>> BELGIUM
>>>>
>>>> Tel: + 32 16 326424
>>>> Fax: + 32 16 322980
>>>>
>>>> http://geo.kuleuven.be/aow/
>>>> www.kuleuven.be/geography
>>>> -------------------------------------------------------------
>> --
>>
>>
>> -------------------------------------------------------------
>> FWO Post-doctoral Research Fellow
>> Department of Earth&  Environmental Sciences
>> Physical and Regional Geography Research Group
>> Regional climate studies
>>
>> Celestijnenlaan 200E
>> 3001 Heverlee (Leuven)
>> BELGIUM
>>
>> Tel: + 32 16 326424
>> Fax: + 32 16 322980
>>
>> http://geo.kuleuven.be/aow/
>> www.kuleuven.be/geography
>> -------------------------------------------------------------

-- 


-------------------------------------------------------------
FWO Post-doctoral Research Fellow
Department of Earth&  Environmental Sciences
Physical and Regional Geography Research Group
Regional climate studies

Celestijnenlaan 200E
3001 Heverlee (Leuven)
BELGIUM

Tel: + 32 16 326424
Fax: + 32 16 322980

http://geo.kuleuven.be/aow/
www.kuleuven.be/geography
-------------------------------------------------------------

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ccsm.bldlog.110203-164317
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20110203/5d0668e4/attachment-0001.diff>

More information about the mpich-discuss mailing list