[petsc-users] configure mpirun problem

[Satish Balay]

On Mon, 23 May 2011, Matthew Knepley wrote:

> On Mon, May 23, 2011 at 2:16 PM, charles reid <charlesreid1 at gmail.com>wrote:
> 
> > Hi -
> >
> > I'm trying to install Petsc and linking to OpenMPI, but running into a
> > conflict with the compiler test.  If I specify my MPI installation directory
> > and my mpiexec command, like so:
> >
> > ./configure \
> >  --prefix=/uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/petsc/3.0.0_incendio \
> >  --with-x=false \
> >  --with-matlab=false \
> >  --download-f-blas-lapack=ifneeded \
> >  \
> >  --with-mpi-dir=/uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3\
> >  --with-mpiexec=/uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpirun \
> >  \
> >  --with-blas-lapack-dir=/usr/lib \Checking for program /uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc...found
> >           Defined make macro "CC" to "/uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc"
> >           Pushing language C
> > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o   conftest.c
> > Executing: /uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o
> > conftest.o   conftest.c
> > sh:
> > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o   conftest.c
> > Executing: /uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o
> > conftest.o   conftest.c
> > sh:
> >                                           Pushing language C
> >                                           Popping language C
> >                                           Pushing language Cxx
> >                                           Popping language Cxx
> >                                           Pushing language FC
> >                                           Popping language FC
> >                   Pushing language C                  Popping language C
> > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc
> > -o conftest     conftest.o
> > Executing: /uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc  -o
> > conftest     conftest.o
> > sh:
> > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc-c -o conftest.o   conftest.c
> > Executing: /uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc -c -o
> > conftest.o   conftest.c
> > sh:                      Pushing language C
> >                       Popping language C
> > sh: /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc
> > -o conftest     conftest.o
> > Executing: /uufs/
> > chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc  -o
> > conftest     conftest.o
> > sh:Executing: ./conftest
> > sh: ./conftest
> > Executing: ./conftestsh:
> > ERROR while running executable: Could not execute './conftest':./conftest:
> > error while loading shared libraries: libmpi.so.0: cannot open shared object
> > file: No such file or directory
> >  --with-shared=0 \
> >
> >
> >
> > The underlying problem can be demonstrated with a simple C hello world
> > program:
> >
> > $ cat test.c
> >
> > #include <stdio.h>
> >
> > main()
> > {
> >   printf ("Hello World!\n");
> > }
> >
> > $ /uufs/chpc.utah.edu/common/home/u0552682/pkg/openmpi/1.4.3/bin/mpicc  -o
> > test.o test.c
> >
> > $ ./test.o
> > ./test.o: error while loading shared libraries: libmpi.so.0: cannot open
> > shared object file: No such file or directory
> >
> >
> >
> > Running the executable with mpirun alone shows the same error (similar to
> > this thread at OpenMPI list:
> > http://www.open-mpi.org/community/lists/users/2006/07/1691.php)
> >
> > $ mpirun -np 1 ./test.o
> > ./test.o: error while loading shared libraries: libmpi.so.0: cannot open
> > shared object file: No such file or directory
> >
> >
> >
> > and ultimately the resolution is to specify the full path of mpirun when
> > running the executable (as per the response to the above thread,
> > http://www.open-mpi.org/community/lists/users/2006/07/1692.php):
> >
> > $ `which mpirun` -np 1 ./test.o
> > Hello World!
> >
> >
> >
> > So my question is, why does Petsc's compiler test insist on running the
> > mpicc-built test executable in an incorrect way (i.e. ./conftestsh instead
> > of /path/to/mpiexec ./conftest)?  And how can I compile with OpenMPI
> > support?  I'm unable to bypass this by specifying CC and CXX, as I just run
> > into problems later on with mpi_init not being found.
> 
> 
> There are many, many ways to run an executable. We count on 'sh
> <executable>' as a default, and if that does not work
>  then you can run configure using --with-batch.

With openMPI - the primary issue is:

They [openmpi folks] expect openmpi users to set LD_LIBRARY_PATH
before using their compilers. So you should set that before running
PETSc configure.

Also - you are using older 3.0.0 version. 3.1 might have some
workarrounds for openmpi [for sure petsc-dev does..]

Satish

> 
>     Matt
> 
> 
> >
> > Charles
> >
> 
> 
> 
>