[mpich-discuss] building 32 bit shmem on 64 bit linux system

[Daniel Fetchinson]

Okay, thanks a lot, I'll do that. I was just hoping I don't have to do
this since compiling arpack is always a pain in the a$$.

Thanks for all the help everybody,
Daniel


> The unresolved symbols from libparpack looks like fortran symbols.
>
> You need to recompile arpack with mpicc/mpif77/mpif90 from ch_shmem
> and relink arpack with your code.
>
> A.Chan
>
>> No.
>>
>> I was actually never recompiling it, always was using the same 32-bit
>> archive and it was always working on a number of platforms (intel
>> core
>> 2 duo, intel xeon (8 cores), etc) which made me believe I don't have
>> to recompile it.
>>
>> Only on this new intel core 2 quad it doesn't seem to work.
>>
>> Do you think it's possible that just because the platform on which it
>> was compiled is slightly different from the current one (I'm not even
>> sure intel core 2 duo is a different platform from intel core 2 quad)
>> the linker would complain about undefined references? I thought it
>> would find the references in any case, at worst the program would
>> segfault.
>>
>> Daniel
>>
>>
>> > Did you also recompile the parallel arpack library with the newly
>> installed
>> > mpicc/mpif77?
>> >
>> > Rajeev
>>
>> >>
>> >> I started the whole thing from scratch and deleted all build
>> >> directories, unpacked mpich and now I'm pretty certain I've got a
>> >> ch_shmem configured, built, installed.
>> >>
>> >> But there is another problem: I use the parallel arpack library in
>> my
>> >> program and while linking the whole thing together mpicc complains
>> >> about undefined references:
>> >>
>> >> libs/libparpack.a(pznaupd.o): In function `pznaupd_':
>> >> pznaupd.f:(.text+0x3db): undefined reference to `mpi_comm_rank__'
>> >> libs/libparpack.a(pznaupd.o):(.data+0x0): undefined reference
>> >> to `mpi_dup_fn__'
>> >> libs/libparpack.a(pznaupd.o):(.data+0x4): undefined reference to
>> >> `mpi_null_delete_fn__'
>> >> libs/libparpack.a(pznaupd.o):(.data+0x8): undefined reference to
>> >> `mpi_null_copy_fn__'
>> >> libs/libparpack.a(pivout.o): In function `pivout_':
>> >> pivout.f:(.text+0x16): undefined reference to `mpi_comm_rank__'
>> >> libs/libparpack.a(pivout.o):(.data+0x0): undefined reference
>> >> to `mpi_dup_fn__'
>> >> libs/libparpack.a(pivout.o):(.data+0x4): undefined reference to
>> >> `mpi_null_delete_fn__'
>> >> libs/libparpack.a(pivout.o):(.data+0x8): undefined reference to
>> >> `mpi_null_copy_fn__'
>> >>
>> >> etc. etc..............
>> >>
>> >> Although /home/fetchinson/mpich32shmem/lib/libmpich.a is given to
>> >> mpicc for linking along with all other object files where
>> >> /home/fetchinson/mpich32shmem is the installation directory of
>> mpich
>> >> (the one given to ./configure via --prefix). So it seems I have
>> some
>> >> linking problems still.
>> >>
>> >> By the way, the only file that is needed to be given to mpicc for
>> >> static linking is libmpich.a, right? What is strange is that on
>> >> another machine the same Makefile works with the same installation
>> >> directory for mpich.
>> >>
>> >> Cheers,
>> >> Daniel
>> >>
>> >>
>> >>
>> >>
>> >> > In the outputs of the configure and make steps, does it
>> >> look like it is
>> >> > building ch_p4 or ch_shmem?
>> >> >
>> >> > Rajeev
>> >> >>
>> >> >> Out of desperation I looked at the output of
>> >> >>
>> >> >> strings my_test_program
>> >> >>
>> >> >> where my_test_program was compiled using mpicc (using
>> >> absolute paths)
>> >> >> from the 32-bit shmem mpi library. Strangely, ch_p4 appears in
>> the
>> >> >> output several times while ch_shmem doesn't. On another
>> >> machine where
>> >> >> I'm successfully compiling 32-bit shmem executables the output
>> of
>> >> >> 'strings my_test_program' only contains ch_shmem and not ch_p4.
>> >> >>
>> >> >> So there must be something wrong with my mpicc, although I
>> can't
>> >> >> imagine what, I never compiled to ch_p4.
>> >> >>
>> >> >> Thanks for the advice guys, there seems to be progress :)
>> >> >> Daniel
>> >> >>
>> >> >>
>> >> >> > Did you do a "make install" after "make" ?  If not, the
>> >> >> install directory
>> >> >> > may still have an older installation of mpich-1 which
>> >> could be ch_p4
>> >> >> > and not ch_shmem that you need.
>> >> >> >
>> >> >> > A.Chan
>> >> >> >
>> >> >> > ----- "Daniel Fetchinson" <fetchinson at googlemail.com> wrote:
>> >> >> >
>> >> >> >> > It must be the mpirun then. Make sure you use the
>> >> mpirun from the
>> >> >> >> same
>> >> >> >> > build. Give the full path if necessary.
>> >> >> >> >
>> >> >> >> > Rajeev
>> >> >> >>
>> >> >> >> Thanks, but there is only one installation of mpi on the
>> >> >> machine and
>> >> >> >> that installation is the one I keep compiling. So even
>> >> if it wanted
>> >> >> >> to
>> >> >> >> execute the wrong mpicc/mpirun/etc it couldn't because
>> >> >> there is only
>> >> >> >> 1
>> >> >> >> copy at any given time.
>> >> >> >>
>> >> >> >> Nevertheless I recompiled now mpi as well as my program
>> >> and still I
>> >> >> >> get the same error:
>> >> >> >>
>> >> >> >> 0 - MPI_INIT : MPIRUN chose the wrong device ch_shmem;
>> >> >> program needs
>> >> >> >> device ch_p4
>> >> >> >> /home/fetchinson/mpich32shmem/bin/mpirun.ch_shmem: line
>> >> 91: 21976
>> >> >> >> Segmentation fault      /home/fetchinson/test "testinput"
>> >> >> >>
>> >> >> >> Any ideas?
>> >> >> >> Daniel
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> >> > Did you recompile the program? Make sure you use the
>> mpicc
>> >> >> >> >> from this
>> >> >> >> >> > build and not some other mpicc from another directory.
>> Give
>> >> >> >> >> the full
>> >> >> >> >> > path if necessary.
>> >> >> >> >> >
>> >> >> >> >> > Rajeev
>> >> >> >> >>
>> >> >> >> >> Yes, I did recompile the program using mpicc from the
>> newly
>> >> >> >> >> built mpi library. Actually, there is only one mpicc on
>> the
>> >> >> >> >> whole machine.
>> >> >> >> >>
>> >> >> >> >> Cheers,
>> >> >> >> >> Daniel
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> >> Actually, I spoke a bit too soon. Configure, make,
>> make
>> >> >> >> >> install all
>> >> >> >> >> >> work but the code still doesn't run, saying:
>> >> >> >> >> >>
>> >> >> >> >> >> 0 - MPI_INIT : MPIRUN chose the wrong device ch_shmem;
>> >> >> >> >> program needs
>> >> >> >> >> >> device ch_p4
>> >> >> >> >> >> /home/fetchinson/mpi32shmem/bin/mpirun.ch_shmem:
>> >> >> line 91: 17761
>> >> >> >> >> >> Segmentation fault      /home/fetchinson/test
>> "testinput"
>> >> >> >> >> >>
>> >> >> >> >> >> The exact same code runs on another machine where
>> >> >> someone else
>> >> >> >> >> >> installed a 32-bit shmem mpich1 library, I'm trying to
>> do
>> >> >> >> >> the same on
>> >> >> >> >> >> a different machine now. It's an intel quad core
>> >> so I suppose
>> >> >> >> the
>> >> >> >> >> >> shmem device should work. The configure options I used
>> for
>> >> >> >> >> compiling
>> >> >> >> >> >> are
>> >> >> >> >> >>
>> >> >> >> >> >> export CC='gcc -m32'
>> >> >> >> >> >> export F77='g77 -m32'
>> >> >> >> >> >> ./configure  --with-device=ch_shmem
>> >> >> >> >> >>
>> >> >> >> >> >> Do I need to specify the --with-arch option? Going
>> through
>> >> >> >> >> the list
>> >> >> >> >> >> of available --with-arch options I'm not sure
>> >> what I need for
>> >> >> >> an
>> >> >> >> >> >> Intel(R)
>> >> >> >> >> >> Core(TM)2 Quad CPU Q6600 @ 2.40GHz.
>> >> >> >> >> >>
>> >> >> >> >> >> What am I doing wrong?
>> >> >> >> >> >>
>> >> >> >> >> >> Cheers,
>> >> >> >> >> >> Daniel
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> > Thanks Anthony, that worked!
>> >> >> >> >> >> >
>> >> >> >> >> >> > Concerning MPICH2: I'm not in a position to change
>> the
>> >> >> >> >> >> communication
>> >> >> >> >> >> > routines in the code  I use so this would only
>> >> be an option
>> >> >> >> >> >> if MPICH2
>> >> >> >> >> >> > is fully backward compatible with MPICH1. Is
>> >> this the case?
>> >> >> >> >> >> Can I just
>> >> >> >> >> >> > start using MPICH2 without changing anything in
>> >> the code if
>> >> >> >> >> >> it works
>> >> >> >> >> >> > with MPICH1?
>> >> >> >> >> >> >
>> >> >> >> >> >> > Cheers,
>> >> >> >> >> >> > Daniel
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> >> Since you are using MPICH-1, you can try setting
>> >> >> CC, F77 and
>> >> >> >> F90
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> CC="gcc -m32"
>> >> >> >> >> >> >> F77="g77 -m32"
>> >> >> >> >> >> >> ....
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> BTW, can you use MPICH2 instead ?  We have
>> >> shifted all our
>> >> >> >> >> >> >> development effort to MPICH2 which is more robust
>> than
>> >> >> >> >> >> MPICH-1.  If
>> >> >> >> >> >> >> you are using MPICH2, setting CFLAGS and FFLAGS
>> should
>> >> >> >> >> >> work, and you
>> >> >> >> >> >> >> can use nemesis which shared memory for intranode
>> >> >> >> communication.
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> A.Chan
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> ----- "Daniel Fetchinson"
>> >> >> <fetchinson at googlemail.com> wrote:
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>> Hi folks,
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> I'd like to build the 32-bit shmem mpi library
>> because
>> >> >> >> >> >> the code I'll
>> >> >> >> >> >> >>> use with mpi only works in 32-bit because of some
>> >> >> >> >> 32-bit specific
>> >> >> >> >> >> >>> assembly code. The machine is a 64-bit machine with
>> a
>> >> >> >> >> >> 64-bit linux
>> >> >> >> >> >> >>> distribution (suse).
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> I tried configuring mpich-1.2.7p1 in a number of
>> ways:
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> ./configure --with-device=ch_shmem ./configure
>> >> >> >> >> --with-arch=LINUX
>> >> >> >> >> >> >>> --with-device=ch_shmem ./configure
>> >> >> >> >> >> >>> --with-arch=LINUX32 --with-device=ch_shmem
>> ./configure
>> >> >> >> >> >> >>> --with-arch=i386 --with-device=ch_shmem
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> with and without setting the following environment
>> >> >> >> >> variables (in
>> >> >> >> >> >> >>> bash):
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> export CFLAGS=-m32
>> >> >> >> >> >> >>> export FFLAGS=-m32
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> Configuring went all right but none of the above
>> >> >> >> >> >> combinations worked
>> >> >> >> >> >> >>> with 'make' unfortunately :( The error in 'make'
>> is
>> >> >> >> >> the following:
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> /home/fetchinson/mpich-1.2.7p1/bin/mpicc  -o
>> overtake
>> >> >> >> >> overtake.o
>> >> >> >> >> >> >>> test.o
>> >> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> /usr/lib64/gcc/x86_64-suse-linux/4.2.1/../../../../x86_64-suse
>> >> >> >> >> >> -linux/bin/ld:
>> >> >> >> >> >> >>> skipping incompatible
>> >> >> >> >> >> /home/fetchinson/mpich-1.2.7p1/lib/libmpich.a
>> >> >> >> >> >> >>> when searching for -lmpich
>> >> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> /usr/lib64/gcc/x86_64-suse-linux/4.2.1/../../../../x86_64-suse
>> >> >> >> >> >> -linux/bin/ld:
>> >> >> >> >> >> >>> cannot find -lmpich
>> >> >> >> >> >> >>> collect2: ld returned 1 exit status
>> >> >> >> >> >> >>> make[4]: *** [overtake] Error 1
>> >> >> >> >> >> >>> make[3]: [linktest1] Error 2 (ignored) Could not
>> link
>> >> >> >> >> a C program
>> >> >> >> >> >> >>> with MPI libraries
>> >> >> >> >> >> >>> make[3]: *** [linktest1] Error 1
>> >> >> >> >> >> >>> make[2]: *** [linktest] Error 2
>> >> >> >> >> >> >>> make[1]: *** [mpi-lib-test] Error 2
>> >> >> >> >> >> >>> make: *** [mpi] Error 2
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> Which makes me believe that I was not able to
>> >> >> convince the
>> >> >> >> >> >> >>> compiler/linker/etc that I really want everything
>> in
>> >> >> >> 32-bit.
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> What would be the correct way of compiling
>> >> the whole mpi
>> >> >> >> >> >> library to
>> >> >> >> >> >> >>> 32-bit?
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> Cheers,
>> >> >> >> >> >> >>> Daniel
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> > --
>> >> >> >> >> >> > Psss, psss, put it down! -
>> >> >> http://www.cafepress.com/putitdown
>> >> >> >> >> >> >
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> --
>> >> >> >> >> >> Psss, psss, put it down! -
>> >> http://www.cafepress.com/putitdown
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> Psss, psss, put it down! -
>> http://www.cafepress.com/putitdown
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Psss, psss, put it down! -
>> http://www.cafepress.com/putitdown
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Psss, psss, put it down! - http://www.cafepress.com/putitdown
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Psss, psss, put it down! - http://www.cafepress.com/putitdown
>> >>
>> >>
>> >
>> >
>>
>>
>> --
>> Psss, psss, put it down! - http://www.cafepress.com/putitdown
>
>


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown