[mpich-discuss] undefined symbol: MPIR_Thread

Anthony Chan chan at mcs.anl.gov
Thu Feb 3 12:10:40 CST 2011 

What is the exact command you used to launch ccsm.exe ?

----- Original Message -----
> MPICH2-1.3.1's default process manager is hydra not mpd,
> so you don't need to start mpd... Older mpich2 requires
> mpd but not 1.3.1 MPIR_Thread is defined in libmpich.a (AFAICT)
> I don't know why your ccsm.exe still says it can't find MPIR_Thread.
> 
> Is everything in CCSM statically linked ?
> 
> 
> ----- Original Message -----
> > I was wondering whether this has also something to do with the
> > problem.
> > On a website of a person installing ccsm on his laptop (also using
> > mpich2) he states that before running the executable, he has to
> > start
> > mpi first
> > > mpd &
> >
> > Now, if I do that, then I get:
> > /usr/workdir/cesm1_0_2/scripts$ config: problems opening file
> > /etc/mpd.conf for reading: Permission denied
> >
> > or as root
> > listen: Failed to listen on localhost (line 69): Address already in
> > use:
> >
> > Does this mean it is already on, or that there is an error?
> >
> >
> > On 02/02/2011 07:08 PM, Anthony Chan wrote:
> > > You indicated mpich2-1.3.1 was installed in
> > > /usr/local/mpich2-1.3.1
> > > so /usr/local/bin/mpif90 refers to mpif90 from other MPI
> > > implementation.
> > > So you MUST specify fullpath to mpif90 in all your build scripts
> > > and/or
> > > makefiles. I also notice your bldlog uses compiler variables
> > > besides
> > > FC and CC,
> > > i.e. MPICC, MPIF90, F90..., and some of these fortran
> > >
> > > CC = $(MPICH_PATH)/bin/mpicc
> > > MPICC = $(MPICH_PATH)/bin/mpicc
> > > FC = $(MPICH_PATH)/bin/mpif90
> > > F90 = $(MPICH_PATH)/bin/mpif90
> > > MPIF90 = $(MPICH_PATH)/bin/mpif90
> > >
> > > and leave LIB_MPI and INC_MPI blank (that would help you locate
> > > which script
> > > and/or makefile that set FC to "gfortran" or "mpif90").
> > >
> > > The other option is to simply prefix your PATH with
> > > /usr/local/mpich2-1.3.1 to
> > > avoid /usr/local/bin's MPI compiler wrappers being used in CCSM
> > > build system,
> > > but then you have to check your environment every time running
> > > CCSM
> > > build system,
> > > and your build output won't show correct MPI implementation is
> > > being
> > > used or not.
> > > So I think you should follow the 1st approach that was suggested.
> > >
> > > A.Chan
> > >
> > > ----- Original Message -----
> > >> The files that I send were indeed with the LIB and INC
> > >> definitions
> > >> included.
> > >> But I did the test as well without these indications (and setting
> > >> FC
> > >> and
> > >> CC to MPICH2/etc/etc/...), and the results were the same?
> > >> see example built log in attach.
> > >>
> > >> The "which mpif90" statement gives me:
> > >> /usr/local/bin/mpif90
> > >>
> > >> Matthias
> > >>
> > >> On 02/02/2011 04:48 PM, Anthony Chan wrote:
> > >>> All your bldlog files indicate that you set FC to mpif90 and
> > >>> NOT $(MPICH_PATH)/bin/mpif90 as I suggested earlier. You should
> > >>> set both LIB_MPI and INC_MPI to empty strings. Do you have other
> > >>> MPI implementations (older MPICH2 or other MPI implementations)
> > >>> installed on your system besides MPICH2-1.3.1 ? Do a "which
> > >>> mpif90"
> > >>> to find out.
> > >>>
> > >>> A.Chan
> > >>>
> > >>> ----- Original Message -----
> > >>>> Dear Anthony!
> > >>>>
> > >>>> I have now done what you suggested below:
> > >>>> Set FC and CC to mpif90 and mpicc and removing the inc_ and
> > >>>> lib_
> > >>>> references.
> > >>>> Again, I could built the executable, but running the latter
> > >>>> results
> > >>>> in
> > >>>> ./ccsm.exe: undefined symbol: MPIR_Thread
> > >>>>
> > >>>> Therefore, I set everything back to before and added this to
> > >>>> the
> > >>>> LIB_MPI
> > >>>> LIB_MPI := $(MPICH_PATH)/lib -lmpichf90 -lmpichf90 -lmpich
> > >>>> -lopa
> > >>>> -lmpl -lrt -lpthread
> > >>>>
> > >>>> As the last arguments were shown by mpif90 -show. But again,
> > >>>> same
> > >>>> error.
> > >>>>
> > >>>> For every module of the model, a log file is saved. I attached
> > >>>> the
> > >>>> 3
> > >>>> most important ones:
> > >>>> - pio: parallel IO
> > >>>> - lnd: the land module that I would like to test
> > >>>> - ccsm: the log for building the executable
> > >>>>
> > >>>> As far as I can see, there could be something in pio, but I'm
> > >>>> not
> > >>>> sure,
> > >>>> and it doesn't seem to be enough to stop the compilation
> > >>>> process.
> > >>>>
> > >>>> Wouldn't there be any other way to deal with this thread
> > >>>> problem?
> > >>>> Kind regards,
> > >>>>
> > >>>> Matthias
> > >>>>
> > >>>>
> > >>>>
> > >>>> On 02/02/2011 01:12 AM, Anthony Chan wrote:
> > >>>>> I took a quick look at the scripts that you sent but they
> > >>>>> ain't
> > >>>>> very
> > >>>>> helpful
> > >>>>> as these files source some other files which ain't included in
> > >>>>> your
> > >>>>> attachment.
> > >>>>> In any case, my knowledge on CCSM is limited. Since the build
> > >>>>> problem is CCSM
> > >>>>> specific, you may want to contact CCSM developers to see what
> > >>>>> it
> > >>>>> should be done.
> > >>>>>
> > >>>>> Given your final executable (or one of the executables) has
> > >>>>> unresolved symbol
> > >>>>> MPIR_Thread, so it seems it didn't link in MPICH2 libraries
> > >>>>> correctly.
> > >>>>> I would first set FC=to $(MPICH_PATH)/bin/mpif90 and
> > >>>>> CC=$(MPICH_PATH)/bin/mpicc
> > >>>>> and leave LIB_MPI and INC_MPI empty since mpif90 and mpicc
> > >>>>> should
> > >>>>> take care of
> > >>>>> LIB_MPI and INC_MPI. You may want to save the compile and
> > >>>>> linking
> > >>>>> output of
> > >>>>> whole build process to see what goes wrong.
> > >>>>>
> > >>>>> A.Chan
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> ----- Original Message -----
> > >>>>>> mmm, I am not sure if I can answer this question.
> > >>>>>> It is a global climate model, with a lot of packages, and in
> > >>>>>> order
> > >>>>>> to
> > >>>>>> compile it I follow the installation guide of the model.
> > >>>>>> So I am not completely sure what I should provide you with?
> > >>>>>>
> > >>>>>> There is a seperate Macros files in which I have to adapt
> > >>>>>> libs
> > >>>>>> and
> > >>>>>> flags
> > >>>>>> for my system. The contents are as follows:
> > >>>>>> # Note that CPPDEFS is set in Macros.cppdefs
> > >>>>>> CPPDEFS += -DLINUX -DSEQ_$(FRAMEWORK) -DFORTRANUNDERSCORE
> > >>>>>> -DNO_R16
> > >>>>>> -DNO_SHR_VMATH
> > >>>>>>
> > >>>>>> ifeq ($(compile_threaded), true)
> > >>>>>> CPPDEFS += -DTHREADED_OMP
> > >>>>>> endif
> > >>>>>>
> > >>>>>> FC := mpif90
> > >>>>>> CC := mpicc
> > >>>>>>
> > >>>>>> HDF5_PATH := /usr/local/hdf5
> > >>>>>> HDF5_LIB := $(HDF5_PATH)/lib -lhdf5 -lhdf5_hl
> > >>>>>> NETCDF_PATH := /usr/local/netcdf-4.1.1
> > >>>>>> INC_NETCDF := $(NETCDF_PATH)/include
> > >>>>>> LIB_NETCDF := $(NETCDF_PATH)/lib -lnetcdf
> > >>>>>> -L/usr/local/hdf5/lib
> > >>>>>> -lhdf5_hl -lhdf5 -L/usr/local/zlib/lib -lz -lm
> > >>>>>> MOD_NETCDF := $(NETCDF_PATH)/include
> > >>>>>> MPICH_PATH := /usr/local/mpich2-1.3.1
> > >>>>>> INC_MPI := $(MPICH_PATH)/include
> > >>>>>> LIB_MPI := $(MPICH_PATH)/lib
> > >>>>>> MPI_LIB_NAME := mpich
> > >>>>>> PNETCDF_PATH :=
> > >>>>>> INC_PNETCDF :=
> > >>>>>> LIB_PNETCDF :=
> > >>>>>> LAPACK_LIBDIR :=
> > >>>>>> #=====================================================
> > >>>>>>
> > >>>>>> CFLAGS := $(CPPDEFS)
> > >>>>>> FIXEDFLAGS := -132
> > >>>>>> FREEFLAGS := -FR
> > >>>>>> ##FFLAGS := $(CPPDEFS) -g -fp-model precise -convert
> > >>>>>> big_endian
> > >>>>>> -assume byterecl -ftz -traceback
> > >>>>>> FFLAGS := $(CPPDEFS)-g -O2 -I/usr/local/include
> > >>>>>> FFLAGS_NOOPT := $(FFLAGS) -O0
> > >>>>>> FFLAGS_OPT := -O2
> > >>>>>> LDFLAGS :=
> > >>>>>> AR := ar
> > >>>>>> MOD_SUFFIX := mod
> > >>>>>> CONFIG_SHELL :=
> > >>>>>>
> > >>>>>> So maybe something is wrong here???
> > >>>>>> In attach I also sent you the configure file, and the build
> > >>>>>> file
> > >>>>>> to
> > >>>>>> compile the model.
> > >>>>>> That works all fine. Only when running the obtained
> > >>>>>> executable,
> > >>>>>> I
> > >>>>>> get
> > >>>>>> the error MPIR_Thread...
> > >>>>>> to run the .exe, I use the following:
> > >>>>>> mpirun ./ccsm.exe>&! ccsm.log.$LID
> > >>>>>>
> > >>>>>> as is mentioned in the test2.Latitude_E6400.run file in
> > >>>>>> attach.
> > >>>>>>
> > >>>>>> I do not know if you can do something with this, if not,
> > >>>>>> please
> > >>>>>> let
> > >>>>>> me
> > >>>>>> know....
> > >>>>>>
> > >>>>>> Matthias
> > >>>>>>
> > >>>>>>
> > >>>>>> On 02/01/2011 09:17 PM, Anthony Chan wrote:
> > >>>>>>> ----- Original Message -----
> > >>>>>>>
> > >>>>>>>> I can also perform the test as in the installation guide:
> > >>>>>>>> mpiexec -n 5 ./examples/cpi
> > >>>>>>>>
> > >>>>>>>> Process 1 of 5 is on MaffieLE6400
> > >>>>>>>> Process 3 of 5 is on MaffieLE6400
> > >>>>>>>> Process 0 of 5 is on MaffieLE6400
> > >>>>>>>> Process 2 of 5 is on MaffieLE6400
> > >>>>>>>> Process 4 of 5 is on MaffieLE6400
> > >>>>>>>> pi is approximately 3.1415926544231230, Error is
> > >>>>>>>> 0.0000000008333298
> > >>>>>>>> wall clock time = 0.000435
> > >>>>>>>>
> > >>>>>>>> as far as I can see all works fine?
> > >>>>>>> Yes.
> > >>>>>>>
> > >>>>>>>> The reason I configured with the --with-thread-package=none
> > >>>>>>>> the
> > >>>>>>>> second
> > >>>>>>>> time was to deal with the MPIR_Thread error...
> > >>>>>>>> So I wonder how I can deal with this error?
> > >>>>>>> Let's just focus on using the mpich2 install that was
> > >>>>>>> configured
> > >>>>>>> without
> > >>>>>>> --with-thread-package. How did you compile your application
> > >>>>>>> ?
> > >>>>>>>
> > >>>>>>>> Thanks for your help!
> > >>>>>>>>
> > >>>>>>>> Matthias
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On 02/01/2011 08:24 PM, Anthony Chan wrote:
> > >>>>>>>>> ----- Original Message -----
> > >>>>>>>>>> Dear Anthony,
> > >>>>>>>>>>
> > >>>>>>>>>> My apologies for posting to the wrong email list. I will
> > >>>>>>>>>> register
> > >>>>>>>>>> to
> > >>>>>>>>>> the
> > >>>>>>>>>> other one so that in the future I can put my questions
> > >>>>>>>>>> there.
> > >>>>>>>>>> For now i'll post this directly to you, hope that is
> > >>>>>>>>>> fine??
> > >>>>>>>>> Yes. I cc'ed my reply to mpich-discuss as someone else may
> > >>>>>>>>> have
> > >>>>>>>>> things to add.
> > >>>>>>>>>
> > >>>>>>>>>> I am not sure if I understand what you mean with:
> > >>>>>>>>>>
> > >>>>>>>>>> Did you use mpif90/mpif77 to link your code or just use
> > >>>>>>>>>> above
> > >>>>>>>>>> link
> > >>>>>>>>>> command which seems incomplete ?
> > >>>>>>>>>>
> > >>>>>>>>>> I used the following line to check how mpich2 was built,
> > >>>>>>>>>> as
> > >>>>>>>>>> I
> > >>>>>>>>>> first
> > >>>>>>>>>> got
> > >>>>>>>>>> an error with f90 vs f95.
> > >>>>>>>>>> So with
> > >>>>>>>>>> /usr/local/mpich2-1.3.1/bin/mpif90 -show
> > >>>>>>>>>>
> > >>>>>>>>>> I get finally got:
> > >>>>>>>>>> gfortran -O2 -I/usr/local/mpich2-1.3.1/include
> > >>>>>>>>>> -I/usr/local/mpich2-1.3.1/include
> > >>>>>>>>>> -L/usr/local/mpich2-1.3.1/lib
> > >>>>>>>>>> -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lpthread
> > >>>>>>>>>>
> > >>>>>>>>>> Do you mean that there is something missing here? And
> > >>>>>>>>>> should
> > >>>>>>>>>> I
> > >>>>>>>>>> add
> > >>>>>>>>>> it
> > >>>>>>>>>> than in my definition of INC_MPI and LIB_MPI?
> > >>>>>>>>> Let's focus if your mpich2 installation is correct first.
> > >>>>>>>>>> One other question. I was thinking that it could be
> > >>>>>>>>>> useful
> > >>>>>>>>>> to
> > >>>>>>>>>> built
> > >>>>>>>>>> mpich2 without threading, so I used:
> > >>>>>>>>>> ./configure -prefix=/usr/local/mpich2-1.3.1
> > >>>>>>>>>> --with-thread-package=none
> > >>>>>>>>> Try "./configure --prefix=/usr/local/mpich2-1.3.1",
> > >>>>>>>>> without
> > >>>>>>>>> --with-thread-package.
> > >>>>>>>>>
> > >>>>>>>>> A.Chan
> > >>>>>>>>>
> > >>>>>>>>>> But then I get a whole list of errors, e.g.:
> > >>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getn.o):
> > >>>>>>>>>> In
> > >>>>>>>>>> function `MPIU_Thread_CS_exit_lockname_recursive_impl_':
> > >>>>>>>>>> info_getn.c:(.text+0x124): undefined reference to
> > >>>>>>>>>> `MPIU_Thread_mutex_unlock'
> > >>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getnth.o):
> > >>>>>>>>>> In
> > >>>>>>>>>> function `MPIU_Thread_CS_enter_lockname_recursive_impl_':
> > >>>>>>>>>> info_getnth.c:(.text+0x32): undefined reference to
> > >>>>>>>>>> `MPIU_Thread_mutex_lock'
> > >>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getnth.o):
> > >>>>>>>>>> In
> > >>>>>>>>>> function `MPIU_Thread_CS_exit_lockname_recursive_impl_':
> > >>>>>>>>>> info_getnth.c:(.text+0xf4): undefined reference to
> > >>>>>>>>>> `MPIU_Thread_mutex_unlock'
> > >>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getvallen.o):
> > >>>>>>>>>> In function
> > >>>>>>>>>> `MPIU_Thread_CS_enter_lockname_recursive_impl_':
> > >>>>>>>>>> info_getvallen.c:(.text+0x32): undefined reference to
> > >>>>>>>>>> `MPIU_Thread_mutex_lock'
> > >>>>>>>>>> /home/matthias/Downloads/mpich2-1.3.1/lib/libmpich.a(info_getvallen.o):
> > >>>>>>>>>> In function
> > >>>>>>>>>> `MPIU_Thread_CS_exit_lockname_recursive_impl_':
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Any ideas here?
> > >>>>>>>>>> Matthias
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> -------- Original Message --------
> > >>>>>>>>>> Subject: Re: [mpich2-dev] undefined symbol: MPIR_Thread
> > >>>>>>>>>> Date: Tue, 1 Feb 2011 19:41:54 +0100
> > >>>>>>>>>> From: Anthony Chan<chan at mcs.anl.gov>
> > >>>>>>>>>> Reply-To: Anthony Chan<chan at mcs.anl.gov>,
> > >>>>>>>>>> "mpich2-dev at mcs.anl.gov"
> > >>>>>>>>>> <mpich2-dev at mcs.anl.gov>
> > >>>>>>>>>> To: mpich2-dev at mcs.anl.gov<mpich2-dev at mcs.anl.gov>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> mpich2-dev for mpich2 development not general MPICH usage
> > >>>>>>>>>> questions,
> > >>>>>>>>>> please send your question to mpich-discuss at mcs.anl.gov.
> > >>>>>>>>>>
> > >>>>>>>>>> ----- Original Message -----
> > >>>>>>>>>>>       All,
> > >>>>>>>>>>>
> > >>>>>>>>>>>       when running a model with mpirun ./exe
> > >>>>>>>>>>>       I get the following error:
> > >>>>>>>>>>>       ./exe: symbol lookup error: ./exe: undefined
> > >>>>>>>>>>>       symbol:
> > >>>>>>>>>>>       MPIR_Thread
> > >>>>>>>>>>>
> > >>>>>>>>>>>       I have no idea how to deal with this. I use
> > >>>>>>>>>>>       mpich2-1.3.1,
> > >>>>>>>>>>>       compiled
> > >>>>>>>>>>>       with
> > >>>>>>>>>>>       gfortran, with the following libraries:
> > >>>>>>>>>>>       gfortran -O2 -I/usr/local/mpich2-1.3.1/include
> > >>>>>>>>>>>       -I/usr/local/mpich2-1.3.1/include
> > >>>>>>>>>>>       -L/usr/local/mpich2-1.3.1/lib
> > >>>>>>>>>>>       -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt
> > >>>>>>>>>>>       -lpthread
> > >>>>>>>>>> Did you use mpif90/mpif77 to link your code or just use
> > >>>>>>>>>> above
> > >>>>>>>>>> link
> > >>>>>>>>>> command which seems incomplete ?
> > >>>>>>>>>>
> > >>>>>>>>>> A.Chan
> > >>>>>>>>>>
> > >>>>>>>>>>>       Could anybody help me with this?
> > >>>>>>>>>>>
> > >>>>>>>>>>>       Regards,
> > >>>>>>>>>>>       Matthias
> > >>>>>>>>>>>
> > >>>>>>>>>>>       --
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>       -------------------------------------------------------------
> > >>>>>>>>>>>       FWO Post-doctoral Research Fellow
> > >>>>>>>>>>>       Department of Earth& Environmental Sciences
> > >>>>>>>>>>>       Physical and Regional Geography Research Group
> > >>>>>>>>>>>       Regional climate studies
> > >>>>>>>>>>>
> > >>>>>>>>>>>       Celestijnenlaan 200E
> > >>>>>>>>>>>       3001 Heverlee (Leuven)
> > >>>>>>>>>>>       BELGIUM
> > >>>>>>>>>>>
> > >>>>>>>>>>>       Tel: + 32 16 326424
> > >>>>>>>>>>>       Fax: + 32 16 322980
> > >>>>>>>>>>>
> > >>>>>>>>>>>       http://geo.kuleuven.be/aow/
> > >>>>>>>>>>>       www.kuleuven.be/geography
> > >>>>>>>>>>>       -------------------------------------------------------------
> > >>>>>>>> --
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> -------------------------------------------------------------
> > >>>>>>>> FWO Post-doctoral Research Fellow
> > >>>>>>>> Department of Earth& Environmental Sciences
> > >>>>>>>> Physical and Regional Geography Research Group
> > >>>>>>>> Regional climate studies
> > >>>>>>>>
> > >>>>>>>> Celestijnenlaan 200E
> > >>>>>>>> 3001 Heverlee (Leuven)
> > >>>>>>>> BELGIUM
> > >>>>>>>>
> > >>>>>>>> Tel: + 32 16 326424
> > >>>>>>>> Fax: + 32 16 322980
> > >>>>>>>>
> > >>>>>>>> http://geo.kuleuven.be/aow/
> > >>>>>>>> www.kuleuven.be/geography
> > >>>>>>>> -------------------------------------------------------------
> > >>>>>> --
> > >>>>>>
> > >>>>>>
> > >>>>>> -------------------------------------------------------------
> > >>>>>> FWO Post-doctoral Research Fellow
> > >>>>>> Department of Earth& Environmental Sciences
> > >>>>>> Physical and Regional Geography Research Group
> > >>>>>> Regional climate studies
> > >>>>>>
> > >>>>>> Celestijnenlaan 200E
> > >>>>>> 3001 Heverlee (Leuven)
> > >>>>>> BELGIUM
> > >>>>>>
> > >>>>>> Tel: + 32 16 326424
> > >>>>>> Fax: + 32 16 322980
> > >>>>>>
> > >>>>>> http://geo.kuleuven.be/aow/
> > >>>>>> www.kuleuven.be/geography
> > >>>>>> -------------------------------------------------------------
> > >>>> --
> > >>>>
> > >>>>
> > >>>> -------------------------------------------------------------
> > >>>> FWO Post-doctoral Research Fellow
> > >>>> Department of Earth& Environmental Sciences
> > >>>> Physical and Regional Geography Research Group
> > >>>> Regional climate studies
> > >>>>
> > >>>> Celestijnenlaan 200E
> > >>>> 3001 Heverlee (Leuven)
> > >>>> BELGIUM
> > >>>>
> > >>>> Tel: + 32 16 326424
> > >>>> Fax: + 32 16 322980
> > >>>>
> > >>>> http://geo.kuleuven.be/aow/
> > >>>> www.kuleuven.be/geography
> > >>>> -------------------------------------------------------------
> > >> --
> > >>
> > >>
> > >> -------------------------------------------------------------
> > >> FWO Post-doctoral Research Fellow
> > >> Department of Earth& Environmental Sciences
> > >> Physical and Regional Geography Research Group
> > >> Regional climate studies
> > >>
> > >> Celestijnenlaan 200E
> > >> 3001 Heverlee (Leuven)
> > >> BELGIUM
> > >>
> > >> Tel: + 32 16 326424
> > >> Fax: + 32 16 322980
> > >>
> > >> http://geo.kuleuven.be/aow/
> > >> www.kuleuven.be/geography
> > >> -------------------------------------------------------------
> >
> > --
> >
> >
> > -------------------------------------------------------------
> > FWO Post-doctoral Research Fellow
> > Department of Earth& Environmental Sciences
> > Physical and Regional Geography Research Group
> > Regional climate studies
> >
> > Celestijnenlaan 200E
> > 3001 Heverlee (Leuven)
> > BELGIUM
> >
> > Tel: + 32 16 326424
> > Fax: + 32 16 322980
> >
> > http://geo.kuleuven.be/aow/
> > www.kuleuven.be/geography
> > -------------------------------------------------------------

More information about the mpich-discuss mailing list