[petsc-users] make test freeze
Patrick Begou
Patrick.Begou at legi.grenoble-inp.fr
Fri Jan 6 07:52:45 CST 2017
It is not the first time I have this problem and my aim was now to try to solve
it instead of ignoring tests. The environment seams coherent (see below).
I'll try to run in debug mode to investigate where the code hangs.
Patrick
[begou at kareline tutorials]$ make ex19
*mpicc* -o ex19.o -c -Wall -Wwrite-strings -Wno-strict-aliasing
-Wno-unknown-pragmas -fvisibility=hidden -g3
-I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/include
-I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/include `pwd`/ex19.c
mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
-fvisibility=hidden -g3 -o ex19 ex19.o
-L/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/lib -lpetsc
-llapack -lblas -lX11 -lhwloc -lssl -lcrypto -L/opt/openmpi173-GCC48-node/lib
-L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
-L/opt/GCC48c/lib -lmpi_usempi -lmpi_mpifh -lgfortran -lm -lgfortran -lm
-lquadmath -lm -lmpi_cxx -lstdc++ -L/opt/openmpi173-GCC48-node/lib
-L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
-L/opt/GCC48c/lib -ldl -lmpi -lgcc_s -lpthread -ldl
/bin/rm -f ex19.o
[begou at kareline tutorials]$ *which mpiexec**
**/opt/openmpi173-GCC48-node/bin/mpiexec*
[begou at kareline tutorials]$ *mpicc --showme**
**gcc -I/opt/openmpi173-GCC48-node/include -pthread
-L/opt/openmpi173-GCC48-node/lib -lmpi*
[begou at kareline tutorials]$ *ldd ./ex19*
linux-vdso.so.1 => (0x00007ffe771ea000)
liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x00007f5ac8596000)
libblas.so.3 => /usr/lib64/libblas.so.3 (0x00007f5ac833e000)
libX11.so.6 => /usr/lib64/libX11.so.6 (0x0000003c66600000)
libhwloc.so.5 => /usr/lib64/libhwloc.so.5 (0x0000003dde600000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x0000003c71600000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x0000003c69a00000)
libmpi_usempi.so.1 => */opt/openmpi173-GCC48-node/lib/libmpi_usempi.so.1*
(0x00007f5ac80a4000)
libmpi_mpifh.so.2 => */opt/openmpi173-GCC48-node/lib/libmpi_mpifh.so.2*
(0x00007f5ac7e5a000)
libgfortran.so.3 => /opt/GCC48c/lib64/libgfortran.so.3 (0x00007f5ac7b43000)
libm.so.6 => /lib64/libm.so.6 (0x0000003c63200000)
libquadmath.so.0 => /opt/GCC48c/lib64/libquadmath.so.0 (0x00007f5ac7907000)
libmpi_cxx.so.1 => */opt/openmpi173-GCC48-node/lib/libmpi_cxx.so.1*
(0x00007f5ac76ed000)
libstdc++.so.6 => /opt/GCC48c/lib64/libstdc++.so.6 (0x00007f5ac73e4000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003c63a00000)
libmpi.so.1 => */opt/openmpi173-GCC48-node/lib/libmpi.so.1*
(0x00007f5ac7115000)
libgcc_s.so.1 => /opt/GCC48c/lib64/libgcc_s.so.1 (0x00007f5ac6eff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c63600000)
libc.so.6 => /lib64/libc.so.6 (0x0000003c62e00000)
libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x00007f5ac6cdf000)
libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x00007f5ac6abe000)
libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x0000003c66200000)
libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x0000003ddee00000)
libpci.so.3 => /lib64/libpci.so.3 (0x0000003ddea00000)
libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x0000003c6e200000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x0000003c70a00000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x0000003c70e00000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003c69e00000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x0000003c6f200000)
libz.so.1 => /lib64/libz.so.1 (0x0000003c64200000)
libopen-rte.so.6 => */opt/openmpi173-GCC48-node/lib/libopen-rte.so.6*
(0x00007f5ac684a000)
libopen-pal.so.6 => */opt/openmpi173-GCC48-node/lib/libopen-pal.so.6*
(0x00007f5ac6575000)
librt.so.1 => /lib64/librt.so.1 (0x0000003c63e00000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003c71a00000)
libutil.so.1 => /lib64/libutil.so.1 (0x0000003c6ba00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003c62a00000)
libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f5ac5f18000)
libXau.so.6 => /usr/lib64/libXau.so.6 (0x0000003c66a00000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003c64e00000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x0000003c6fe00000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003c6f600000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003c64600000)
Matthew Knepley a écrit :
> On Fri, Jan 6, 2017 at 2:39 AM, Patrick Begou
> <Patrick.Begou at legi.grenoble-inp.fr
> <mailto:Patrick.Begou at legi.grenoble-inp.fr>> wrote:
>
> Hi Matthew,
>
> Launching manualy ex19 shows only one process consuming cpu time, after
> 952mn I've killed the job this morning.
>
> [begou at kareline tutorials]$ make ex19
> mpicc -o ex19.o -c -Wall -Wwrite-strings -Wno-strict-aliasing
> -Wno-unknown-pragmas -fvisibility=hidden -g3
> -I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/include
> -I/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/include
> `pwd`/ex19.c
> mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
> -fvisibility=hidden -g3 -o ex19 ex19.o
> -L/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git/GCC48/lib -lpetsc
> -llapack -lblas -lX11 -lhwloc -lssl -lcrypto
> -L/opt/openmpi173-GCC48-node/lib
> -L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
> -L/opt/GCC48c/lib -lmpi_usempi -lmpi_mpifh -lgfortran -lm -lgfortran -lm
> -lquadmath -lm -lmpi_cxx -lstdc++ -L/opt/openmpi173-GCC48-node/lib
> -L/opt/GCC48c/lib/gcc/x86_64-unknown-linux-gnu/4.8.1 -L/opt/GCC48c/lib64
> -L/opt/GCC48c/lib -ldl -lmpi -lgcc_s -lpthread -ldl
> /bin/rm -f ex19.o
> [begou at kareline tutorials]$ mpiexec -n 2 ./ex19 -snes_monitor
>
> top command shows:
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 32184 begou 20 0 249m 7152 5132 R 99.8 0.0 952:15.97 ex19
> 32183 begou 20 0 71676 3508 2264 S 0.0 0.0 0:00.04 mpiexec
> 32185 begou 20 0 185m 7132 5124 S 0.0 0.0 0:00.04 ex19
>
> looks like the first process waiting for something that never occur in MPI
> communication....
>
>
> 1000s of people run this every day, so I am skeptical of that explanation.
>
> However, this could happen if the 'mpiexec' in your path does not match the
> MPI libraries that PETSc is linked to.
>
> Matt
>
> Patrick
>
> Matthew Knepley a écrit :
>> On Thu, Jan 5, 2017 at 6:31 AM, Patrick Begou
>> <Patrick.Begou at legi.grenoble-inp.fr
>> <mailto:Patrick.Begou at legi.grenoble-inp.fr>> wrote:
>>
>> I am unable to run any test on petsc. It looks like if the ex19 run
>> freeze on the server as it do not use any cpu time and pstree shows
>>
>> sshd---bash-+-gedit
>> `-make---sh-+-gmake---sh---gmake---sh---mpiexec---ex19
>> `-tee
>> I've tested petsc-3.7.5.tar.gz and the latest sources on the Git
>> repository.
>>
>>
>> All make is doing is running ex19, which you can do by hand. What do you
>> get for
>>
>> cd $PETSC_DIR
>> cd src/snes/examples/tutorials
>> make ex19
>> mpiexec -n 2 ./ex19 -snes_monitor
>>
>> Thanks,
>>
>> Matt
>>
>> Setup from the Git repo:
>> ./configure
>> --prefix=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git-binaries \
>> --PETSC_ARCH=GCC48 \
>> --PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git \
>> --with-shared-libraries=0 \
>> --with-fortran-interfaces=1 \
>> --with-fortran-kernels=1 \
>> --with-cc=mpicc \
>> --with-fc=mpif90 \
>> --with-cxx=mpicxx
>>
>> make PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git
>> PETSC_ARCH=GCC48 all
>>
>> make PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git
>> PETSC_ARCH=GCC48 install
>>
>> make
>> PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git-binaries
>> PETSC_ARCH="" test
>>
>>
>> In the log file I've just:
>>
>> Running test examples to verify correct installation
>> Using
>> PETSC_DIR=/kareline/data/begou/YALES2_1.0.0/PREREQUIS/petsc-git-binaries
>> and PETSC_ARCH=
>>
>> I'm using:
>> gcc version 4.8.1
>> Open MPI: 1.7.3 (build with gcc 4.8.1)
>> (This environment is in production for a while for many local
>> software and works fine)
>>
>> Any suggestion is welcome
>>
>> Patrick
>>
>> --
>> ===================================================================
>> | Equipe M.O.S.T. | |
>> | Patrick BEGOU | mailto:Patrick.Begou at grenoble-inp.fr
>> <mailto:Patrick.Begou at grenoble-inp.fr> |
>> | LEGI | |
>> | BP 53 X | Tel 04 76 82 51 35 |
>> | 38041 GRENOBLE CEDEX | Fax 04 76 82 52 71 |
>> ===================================================================
>>
>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which
>> their experiments lead.
>> -- Norbert Wiener
>
>
> --
> ===================================================================
> | Equipe M.O.S.T. | |
> | Patrick BEGOU |mailto:Patrick.Begou at grenoble-inp.fr <mailto:Patrick.Begou at grenoble-inp.fr> |
> | LEGI | |
> | BP 53 X | Tel 04 76 82 51 35 |
> | 38041 GRENOBLE CEDEX | Fax 04 76 82 52 71 |
> ===================================================================
>
> --
> What most experimenters take for granted before they begin their experiments
> is infinitely more interesting than any results to which their experiments
> lead. -- Norbert Wiener
--
===================================================================
| Equipe M.O.S.T. | |
| Patrick BEGOU | mailto:Patrick.Begou at grenoble-inp.fr |
| LEGI | |
| BP 53 X | Tel 04 76 82 51 35 |
| 38041 GRENOBLE CEDEX | Fax 04 76 82 52 71 |
===================================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170106/21cb7d32/attachment-0001.html>
More information about the petsc-users
mailing list