[mpich-discuss] undefined symbol: MPIR_Thread
Matthias Demuzere
Matthias.demuzere at ees.kuleuven.be
Thu Feb 3 12:44:36 CST 2011
I attached the .run script, but end the end this is the command that is
used:
mpiexec ./ccsm.exe
or if I just start it in my shell session directly:
/usr/local/mpich2-1.3.1/bin/mpiexec ./ccsm.exe
On 02/03/2011 07:10 PM, Anthony Chan wrote:
> What is the exact command you used to launch ccsm.exe ?
>
> ----- Original Message -----
>> MPICH2-1.3.1's default process manager is hydra not mpd,
>> so you don't need to start mpd... Older mpich2 requires
>> mpd but not 1.3.1 MPIR_Thread is defined in libmpich.a (AFAICT)
>> I don't know why your ccsm.exe still says it can't find MPIR_Thread.
>>
>> Is everything in CCSM statically linked ?
>>
>>
>> ----- Original Message -----
>>> I was wondering whether this has also something to do with the
>>> problem.
>>> On a website of a person installing ccsm on his laptop (also using
>>> mpich2) he states that before running the executable, he has to
>>> start
>>> mpi first
>>>> mpd&
>>> Now, if I do that, then I get:
>>> /usr/workdir/cesm1_0_2/scripts$ config: problems opening file
>>> /etc/mpd.conf for reading: Permission denied
>>>
>>> or as root
>>> listen: Failed to listen on localhost (line 69): Address already in
>>> use:
>>>
>>> Does this mean it is already on, or that there is an error?
>>>
>>>
>>> On 02/02/2011 07:08 PM, Anthony Chan wrote:
>>>> You indicated mpich2-1.3.1 was installed in
>>>> /usr/local/mpich2-1.3.1
>>>> so /usr/local/bin/mpif90 refers to mpif90 from other MPI
>>>> implementation.
>>>> So you MUST specify fullpath to mpif90 in all your build scripts
>>>> and/or
>>>> makefiles. I also notice your bldlog uses compiler variables
>>>> besides
>>>> FC and CC,
>>>> i.e. MPICC, MPIF90, F90..., and some of these fortran
>>>>
>>>> CC = $(MPICH_PATH)/bin/mpicc
>>>> MPICC = $(MPICH_PATH)/bin/mpicc
>>>> FC = $(MPICH_PATH)/bin/mpif90
>>>> F90 = $(MPICH_PATH)/bin/mpif90
>>>> MPIF90 = $(MPICH_PATH)/bin/mpif90
>>>>
>>>> and leave LIB_MPI and INC_MPI blank (that would help you locate
>>>> which script
>>>> and/or makefile that set FC to "gfortran" or "mpif90").
>>>>
>>>> The other option is to simply prefix your PATH with
>>>> /usr/local/mpich2-1.3.1 to
>>>> avoid /usr/local/bin's MPI compiler wrappers being used in CCSM
>>>> build system,
>>>> but then you have to check your environment every time running
>>>> CCSM
>>>> build system,
>>>> and your build output won't show correct MPI implementation is
>>>> being
>>>> used or not.
>>>> So I think you should follow the 1st approach that was suggested.
>>>>
>>>> A.Chan
>>>>
>>>> ----- Original Message -----
>>>>> The files that I send were indeed with the LIB and INC
>>>>> definitions
>>>>> included.
>>>>> But I did the test as well without these indications (and setting
>>>>> FC
>>>>> and
>>>>> CC to MPICH2/etc/etc/...), and the results were the same?
>>>>> see example built log in attach.
>>>>>
>>>>> The "which mpif90" statement gives me:
>>>>> /usr/local/bin/mpif90
>>>>>
>>>>> Matthias
>>>>>
>>>>> On 02/02/2011 04:48 PM, Anthony Chan wrote:
>>>>>> All your bldlog files indicate that you set FC to mpif90 and
>>>>>> NOT $(MPICH_PATH)/bin/mpif90 as I suggested earlier. You should
>>>>>> set both LIB_MPI and INC_MPI to empty strings. Do you have other
>>>>>> MPI implementations (older MPICH2 or other MPI implementations)
>>>>>> installed on your system besides MPICH2-1.3.1 ? Do a "which
>>>>>> mpif90"
>>>>>> to find out.
>>>>>>
>>>>>> A.Chan
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> Dear Anthony!
>>>>>>>
>>>>>>> I have now done what you suggested below:
>>>>>>> Set FC and CC to mpif90 and mpicc and removing the inc_ and
>>>>>>> lib_
>>>>>>> references.
>>>>>>> Again, I could built the executable, but running the latter
>>>>>>> results
>>>>>>> in
>>>>>>> ./ccsm.exe: undefined symbol: MPIR_Thread
>>>>>>>
>>>>>>> Therefore, I set everything back to before and added this to
>>>>>>> the
>>>>>>> LIB_MPI
>>>>>>> LIB_MPI := $(MPICH_PATH)/lib -lmpichf90 -lmpichf90 -lmpich
>>>>>>> -lopa
>>>>>>> -lmpl -lrt -lpthread
>>>>>>>
>>>>>>> As the last arguments were shown by mpif90 -show. But again,
>>>>>>> same
>>>>>>> error.
>>>>>>>
>>>>>>> For every module of the model, a log file is saved. I attached
>>>>>>> the
>>>>>>> 3
>>>>>>> most important ones:
>>>>>>> - pio: parallel IO
>>>>>>> - lnd: the land module that I would like to test
>>>>>>> - ccsm: the log for building the executable
>>>>>>>
>>>>>>> As far as I can see, there could be something in pio, but I'm
>>>>>>> not
>>>>>>> sure,
>>>>>>> and it doesn't seem to be enough to stop the compilation
>>>>>>> process.
>>>>>>>
>>>>>>> Wouldn't there be any other way to deal with this thread
>>>>>>> problem?
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> Matthias
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 02/02/2011 01:12 AM, Anthony Chan wrote:
>>>>>>>> I took a quick look at the scripts that you sent but they
>>>>>>>> ain't
>>>>>>>> very
>>>>>>>> helpful
>>>>>>>> as these files source some other files which ain't included in
>>>>>>>> your
>>>>>>>> attachment.
>>>>>>>> In any case, my knowledge on CCSM is limited. Since the build
>>>>>>>> problem is CCSM
>>>>>>>> specific, you may want to contact CCSM developers to see what
>>>>>>>> it
>>>>>>>> should be done.
>>>>>>>>
>>>>>>>> Given your final executable (or one of the executables) has
>>>>>>>> unresolved symbol
>>>>>>>> MPIR_Thread, so it seems it didn't link in MPICH2 libraries
>>>>>>>> correctly.
>>>>>>>> I would first set FC=to $(MPICH_PATH)/bin/mpif90 and
>>>>>>>> CC=$(MPICH_PATH)/bin/mpicc
>>>>>>>> and leave LIB_MPI and INC_MPI empty since mpif90 and mpicc
>>>>>>>> should
>>>>>>>> take care of
>>>>>>>> LIB_MPI and INC_MPI. You may want to save the compile and
>>>>>>>> linking
>>>>>>>> output of
>>>>>>>> whole build process to see what goes wrong.
>>>>>>>>
>>>>>>>> A.Chan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> mmm, I am not sure if I can answer this question.
>>>>>>>>> It is a global climate model, with a lot of packages, and in
>>>>>>>>> order
>>>>>>>>> to
>>>>>>>>> compile it I follow the installation guide of the model.
>>>>>>>>> So I am not completely sure what I should provide you with?
>>>>>>>>>
>>>>>>>>> There is a seperate Macros files in which I have to adapt
>>>>>>>>> libs
>>>>>>>>> and
>>>>>>>>> flags
>>>>>>>>> for my system. The contents are as follows:
>>>>>>>>> # Note that CPPDEFS is set in Macros.cppdefs
>>>>>>>>> CPPDEFS += -DLINUX -DSEQ_$(FRAMEWORK) -DFORTRANUNDERSCORE
>>>>>>>>> -DNO_R16
>>>>>>>>> -DNO_SHR_VMATH
>>>>>>>>>
>>>>>>>>> ifeq ($(compile_threaded), true)
>>>>>>>>> CPPDEFS += -DTHREADED_OMP
>>>>>>>>> endif
>>>>>>>>>
>>>>>>>>> FC := mpif90
>>>>>>>>> CC := mpicc
>>>>>>>>>
>>>>>>>>> HDF5_PATH := /usr/local/hdf5
>>>>>>>>> HDF5_LIB := $(HDF5_PATH)/lib -lhdf5 -lhdf5_hl
>>>>>>>>> NETCDF_PATH := /usr/local/netcdf-4.1.1
>>>>>>>>> INC_NETCDF := $(NETCDF_PATH)/include
>>>>>>>>> LIB_NETCDF := $(NETCDF_PATH)/lib -lnetcdf
>>>>>>>>> -L/usr/local/hdf5/lib
>>>>>>>>> -lhdf5_hl -lhdf5 -L/usr/local/zlib/lib -lz -lm
>>>>>>>>> MOD_NETCDF := $(NETCDF_PATH)/include
>>>>>>>>> MPICH_PATH := /usr/local/mpich2-1.3.1
>>>>>>>>> INC_MPI := $(MPICH_PATH)/include
>>>>>>>>> LIB_MPI := $(MPICH_PATH)/lib
>>>>>>>>> MPI_LIB_NAME := mpich
>>>>>>>>> PNETCDF_PATH :=
>>>>>>>>> INC_PNETCDF :=
>>>>>>>>> LIB_PNETCDF :=
>>>>>>>>> LAPACK_LIBDIR :=
>>>>>>>>> #=====================================================
>>>>>>>>>
>>>>>>>>> CFLAGS := $(CPPDEFS)
>>>>>>>>> FIXEDFLAGS := -132
>>>>>>>>> FREEFLAGS := -FR
>>>>>>>>> ##FFLAGS := $(CPPDEFS) -g -fp-model precise -convert
>>>>>>>>> big_endian
>>>>>>>>> -assume byterecl -ftz -traceback
>>>>>>>>> FFLAGS := $(CPPDEFS)-g -O2 -I/usr/local/include
>>>>>>>>> FFLAGS_NOOPT := $(FFLAGS) -O0
>>>>>>>>> FFLAGS_OPT := -O2
>>>>>>>>> LDFLAGS :=
>>>>>>>>> AR := ar
>>>>>>>>> MOD_SUFFIX := mod
>>>>>>>>> CONFIG_SHELL :=
>>>>>>>>>
>>>>>>>>> So maybe something is wrong here???
>>>>>>>>> In attach I also sent you the configure file, and the build
>>>>>>>>> file
>>>>>>>>> to
>>>>>>>>> compile the model.
>>>>>>>>> That works all fine. Only when running the obtained
>>>>>>>>> executable,
>>>>>>>>> I
>>>>>>>>> get
>>>>>>>>> the error MPIR_Thread...
>>>>>>>>> to run the .exe, I use the following:
>>>>>>>>> mpirun ./ccsm.exe>&! ccsm.log.$LID
>>>>>>>>>
>>>>>>>>> as is mentioned in the test2.Latitude_E6400.run file in
>>>>>>>>> attach.
>>>>>>>>>
>>>>>>>>> I do not know if you can do something with this, if not,
>>>>>>>>> please
>>>>>>>>> let
>>>>>>>>> me
>>>>>>>>> know....
>>>>>>>>>
>>>>>>>>> Matthias
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02/01/2011 09:17 PM, Anthony Chan wrote:
>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>
>>>>>>>>>>> I can also perform the test as in the installation guide:
>>>>>>>>>>> mpiexec -n 5 ./examples/cpi
>>>>>>>>>>>
>>>>>>>>>>> Process 1 of 5 is on MaffieLE6400
>>>>>>>>>>> Process 3 of 5 is on MaffieLE6400
>>>>>>>>>>> Process 0 of 5 is on MaffieLE6400
>>>>>>>>>>> Process 2 of 5 is on MaffieLE6400
>>>>>>>>>>> Process 4 of 5 is on MaffieLE6400
>>>>>>>>>>> pi is approximately 3.1415926544231230, Error is
>>>>>>>>>>> 0.0000000008333298
>>>>>>>>>>> wall clock time = 0.000435
>>>>>>>>>>>
>>>>>>>>>>> as far as I can see all works fine?
>>>>>>>>>> Yes.
>>>>>>>>>>
>>>>>>>>>>> The reason I configured with the --with-thread-package=none
>>>>>>>>>>> the
>>>>>>>>>>> second
>>>>>>>>>>> time was to deal with the MPIR_Thread error...
>>>>>>>>>>> So I wonder how I can deal with this error?
>>>>>>>>>> Let's just focus on using the mpich2 install that was
>>>>>>>>>> configured
>>>>>>>>>> without
>>>>>>>>>> --with-thread-package. How did you compile your application
>>>>>>>>>> ?
>>>>>>>>>>
>>>>>>>>>>> Thanks for your help!
>>>>>>>>>>>
>>>>>>>>>>> Matthias
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 02/01/2011 08:24 PM, Anthony Chan wrote:
>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>> Dear Anthony,
>>>>>>>>>>>>>
>>>>>>>>>>>>> My apologies for posting to the wrong email list. I will
>>>>>>>>>>>>> register
>>>>>>>>>>>>> to
>>>>>>>>>>>>> the
>>>>>>>>>>>>> other one so that in the future I can put my questions
>>>>>>>>>>>>> there.
>>>>>>>>>>>>> For now i'll post this directly to you, hope that is
>>>>>>>>>>>>> fine??
>>>>>>>>>>>> Yes. I cc'ed my reply to mpich-discuss as someone else may
>>>>>>>>>>>> have
>>>>>>>>>>>> things to add.
>>>>>>>>>>>>
>>>>>>>>>>>>> I am not sure if I understand what you mean with:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Did you use mpif90/mpif77 to link your code or just use
>>>>>>>>>>>>> above
>>>>>>>>>>>>> link
>>>>>>>>>>>>> command which seems incomplete ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I used the following line to check how mpich2 was built,
>>>>>>>>>>>>> as
>>>>>>>>>>>>> I
>>>>>>>>>>>>> first
>>>>>>>>>>>>> got
>>>>>>>>>>>>> an error with f90 vs f95.
>>>>>>>>>>>>> So with
>>>>>>>>>>>>> /usr/local/mpich2-1.3.1/bin/mpif90 -show
>>>>>>>>>>>>>
>>>>>>>>>>>>> I get finally got:
>>>>>>>>>>>>> gfortran -O2 -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>>> -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>>> -L/usr/local/mpich2-1.3.1/lib
>>>>>>>>>>>>> -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread
>>>>>>>>>>>>>
>>>>>>>>>>>>> Do you mean that there is something missing here? And
>>>>>>>>>>>>> should
>>>>>>>>>>>>> I
>>>>>>>>>>>>> add
>>>>>>>>>>>>> it
>>>>>>>>>>>>> than in my definition of INC_MPI and LIB_MPI?
>>>>>>>>>>>> Let's focus if your mpich2 installation is correct first.
>>>>>>>>>>>>> One other question. I was thinking that it could be
>>>>>>>>>>>>> useful
>>>>>>>>>>>>> to
>>>>>>>>>>>>> built
>>>>>>>>>>>>> mpich2 without threading, so I used:
>>>>>>>>>>>>> ./configure -prefix=/usr/local/mpich2-1.3.1
>>>>>>>>>>>>> --with-thread-package=none
>>>>>>>>>>>> Try "./configure --prefix=/usr/local/mpich2-1.3.1",
>>>>>>>>>>>> without
>>>>>>>>>>>> --with-thread-package.
>>>>>>>>>>>>
>>>>>>>>>>>> A.Chan
>>>>>>>>>>>>
>>>>>>>>>>>>> But then I get a whole list of errors, e.g.:
>>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getn.o):
>>>>>>>>>>>>> In
>>>>>>>>>>>>> function `MPIU_Thread_CS_exit_lockname_recursive_impl_':
>>>>>>>>>>>>> info_getn.c:(.text+0x124): undefined reference to
>>>>>>>>>>>>> `MPIU_Thread_mutex_unlock'
>>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getnth.o):
>>>>>>>>>>>>> In
>>>>>>>>>>>>> function `MPIU_Thread_CS_enter_lockname_recursive_impl_':
>>>>>>>>>>>>> info_getnth.c:(.text+0x32): undefined reference to
>>>>>>>>>>>>> `MPIU_Thread_mutex_lock'
>>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getnth.o):
>>>>>>>>>>>>> In
>>>>>>>>>>>>> function `MPIU_Thread_CS_exit_lockname_recursive_impl_':
>>>>>>>>>>>>> info_getnth.c:(.text+0xf4): undefined reference to
>>>>>>>>>>>>> `MPIU_Thread_mutex_unlock'
>>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getvallen.o):
>>>>>>>>>>>>> In function
>>>>>>>>>>>>> `MPIU_Thread_CS_enter_lockname_recursive_impl_':
>>>>>>>>>>>>> info_getvallen.c:(.text+0x32): undefined reference to
>>>>>>>>>>>>> `MPIU_Thread_mutex_lock'
>>>>>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getvallen.o):
>>>>>>>>>>>>> In function
>>>>>>>>>>>>> `MPIU_Thread_CS_exit_lockname_recursive_impl_':
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any ideas here?
>>>>>>>>>>>>> Matthias
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> -------- Original Message --------
>>>>>>>>>>>>> Subject: Re: [mpich2-dev] undefined symbol: MPIR_Thread
>>>>>>>>>>>>> Date: Tue, 1 Feb 2011 19:41:54 +0100
>>>>>>>>>>>>> From: Anthony Chan<chan at mcs.anl.gov>
>>>>>>>>>>>>> Reply-To: Anthony Chan<chan at mcs.anl.gov>,
>>>>>>>>>>>>> "mpich2-dev at mcs.anl.gov"
>>>>>>>>>>>>> <mpich2-dev at mcs.anl.gov>
>>>>>>>>>>>>> To: mpich2-dev at mcs.anl.gov<mpich2-dev at mcs.anl.gov>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> mpich2-dev for mpich2 development not general MPICH usage
>>>>>>>>>>>>> questions,
>>>>>>>>>>>>> please send your question to mpich-discuss at mcs.anl.gov.
>>>>>>>>>>>>>
>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>> All,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> when running a model with mpirun ./exe
>>>>>>>>>>>>>> I get the following error:
>>>>>>>>>>>>>> ./exe: symbol lookup error: ./exe: undefined
>>>>>>>>>>>>>> symbol:
>>>>>>>>>>>>>> MPIR_Thread
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have no idea how to deal with this. I use
>>>>>>>>>>>>>> mpich2-1.3.1,
>>>>>>>>>>>>>> compiled
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> gfortran, with the following libraries:
>>>>>>>>>>>>>> gfortran -O2 -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>>>> -I/usr/local/mpich2-1.3.1/include
>>>>>>>>>>>>>> -L/usr/local/mpich2-1.3.1/lib
>>>>>>>>>>>>>> -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt
>>>>>>>>>>>>>> -lpthread
>>>>>>>>>>>>> Did you use mpif90/mpif77 to link your code or just use
>>>>>>>>>>>>> above
>>>>>>>>>>>>> link
>>>>>>>>>>>>> command which seems incomplete ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> A.Chan
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Could anybody help me with this?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Matthias
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>>>>>> FWO Post-doctoral Research Fellow
>>>>>>>>>>>>>> Department of Earth& Environmental Sciences
>>>>>>>>>>>>>> Physical and Regional Geography Research Group
>>>>>>>>>>>>>> Regional climate studies
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Celestijnenlaan 200E
>>>>>>>>>>>>>> 3001 Heverlee (Leuven)
>>>>>>>>>>>>>> BELGIUM
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Tel: + 32 16 326424
>>>>>>>>>>>>>> Fax: + 32 16 322980
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://geo.kuleuven.be/aow/
>>>>>>>>>>>>>> www.kuleuven.be/geography
>>>>>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>>> FWO Post-doctoral Research Fellow
>>>>>>>>>>> Department of Earth& Environmental Sciences
>>>>>>>>>>> Physical and Regional Geography Research Group
>>>>>>>>>>> Regional climate studies
>>>>>>>>>>>
>>>>>>>>>>> Celestijnenlaan 200E
>>>>>>>>>>> 3001 Heverlee (Leuven)
>>>>>>>>>>> BELGIUM
>>>>>>>>>>>
>>>>>>>>>>> Tel: + 32 16 326424
>>>>>>>>>>> Fax: + 32 16 322980
>>>>>>>>>>>
>>>>>>>>>>> http://geo.kuleuven.be/aow/
>>>>>>>>>>> www.kuleuven.be/geography
>>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -------------------------------------------------------------
>>>>>>>>> FWO Post-doctoral Research Fellow
>>>>>>>>> Department of Earth& Environmental Sciences
>>>>>>>>> Physical and Regional Geography Research Group
>>>>>>>>> Regional climate studies
>>>>>>>>>
>>>>>>>>> Celestijnenlaan 200E
>>>>>>>>> 3001 Heverlee (Leuven)
>>>>>>>>> BELGIUM
>>>>>>>>>
>>>>>>>>> Tel: + 32 16 326424
>>>>>>>>> Fax: + 32 16 322980
>>>>>>>>>
>>>>>>>>> http://geo.kuleuven.be/aow/
>>>>>>>>> www.kuleuven.be/geography
>>>>>>>>> -------------------------------------------------------------
>>>>>>> --
>>>>>>>
>>>>>>>
>>>>>>> -------------------------------------------------------------
>>>>>>> FWO Post-doctoral Research Fellow
>>>>>>> Department of Earth& Environmental Sciences
>>>>>>> Physical and Regional Geography Research Group
>>>>>>> Regional climate studies
>>>>>>>
>>>>>>> Celestijnenlaan 200E
>>>>>>> 3001 Heverlee (Leuven)
>>>>>>> BELGIUM
>>>>>>>
>>>>>>> Tel: + 32 16 326424
>>>>>>> Fax: + 32 16 322980
>>>>>>>
>>>>>>> http://geo.kuleuven.be/aow/
>>>>>>> www.kuleuven.be/geography
>>>>>>> -------------------------------------------------------------
>>>>> --
>>>>>
>>>>>
>>>>> -------------------------------------------------------------
>>>>> FWO Post-doctoral Research Fellow
>>>>> Department of Earth& Environmental Sciences
>>>>> Physical and Regional Geography Research Group
>>>>> Regional climate studies
>>>>>
>>>>> Celestijnenlaan 200E
>>>>> 3001 Heverlee (Leuven)
>>>>> BELGIUM
>>>>>
>>>>> Tel: + 32 16 326424
>>>>> Fax: + 32 16 322980
>>>>>
>>>>> http://geo.kuleuven.be/aow/
>>>>> www.kuleuven.be/geography
>>>>> -------------------------------------------------------------
>>> --
>>>
>>>
>>> -------------------------------------------------------------
>>> FWO Post-doctoral Research Fellow
>>> Department of Earth& Environmental Sciences
>>> Physical and Regional Geography Research Group
>>> Regional climate studies
>>>
>>> Celestijnenlaan 200E
>>> 3001 Heverlee (Leuven)
>>> BELGIUM
>>>
>>> Tel: + 32 16 326424
>>> Fax: + 32 16 322980
>>>
>>> http://geo.kuleuven.be/aow/
>>> www.kuleuven.be/geography
>>> -------------------------------------------------------------
--
-------------------------------------------------------------
FWO Post-doctoral Research Fellow
Department of Earth& Environmental Sciences
Physical and Regional Geography Research Group
Regional climate studies
Celestijnenlaan 200E
3001 Heverlee (Leuven)
BELGIUM
Tel: + 32 16 326424
Fax: + 32 16 322980
http://geo.kuleuven.be/aow/
www.kuleuven.be/geography
-------------------------------------------------------------
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test1.Latitude_E6400.run
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20110203/6b328f7d/attachment-0001.diff>
More information about the mpich-discuss
mailing list