[petsc-users] PETSC MPI error on PetscGatherMessageLengths2

[Hong]

Hui :
It is impossible to know what causes the problem from these error messages.
I notice that you use Petsc-3.5.2.
I would suggest
1. update to the latest petsc-3.6;
2. if problem remains, send us your matrices C and B in petsc binary
format, we can investigate. With matrix files, one can test
MatTransposeMatMult()
using
petsc/src/mat/examples/tests/ex94.c

Hong


 I'm running some code written by myself, using PETSC with MPI. It runs
> fine with less than or equal to 12 cores. However, if I ran it with 16
> cores, it gives me an error. By looking at the error message, it seems that
> the error happens at a line where I have something like this:
>
>   ierr = MatTransposeMatMult(C,B,MAT_INITIAL_MATRIX,1,&A);CHKERRQ(ierr);
>
>  And I have attached the error message here. Does anyone sort of having
> an idea of what might be causing the problem here? Or is there any hint of
> what might be a direction for looking into this bug?
>
>   [hus003 at tscc-login2 test]$ cat /home/hus003/output/errFluid3d | grep
> '\[4\]'
>
> [4]PETSC ERROR:
> ------------------------------------------------------------------------
>
> [4]PETSC ERROR: Caught signal number 15 Terminate: Some process (or the
> batch system) has told this process to end
>
> [4]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>
> [4]PETSC ERROR: or see
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[4]PETSC
> ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find
> memory corruption errors
>
> [4]PETSC ERROR: likely location of problem given in stack below
>
> [4]PETSC ERROR: ---------------------  Stack Frames
> ------------------------------------
>
> [4]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> available,
>
> [4]PETSC ERROR:       INSTEAD the line number of the start of the function
>
> [4]PETSC ERROR:       is given.
>
> [4]PETSC ERROR: [4] PetscGatherMessageLengths2 line 180
> /state/partition1/jjh/All-Rolls/math-roll/BUILD/sdsc-petsc_intel_openmpi_ib-3.5.2/petsc-3.5.2/src/sys/utils/mpimesg.c
>
> [4]PETSC ERROR: [4] MatTransposeMatMultSymbolic_MPIAIJ_MPIAIJ line 1573
> /state/partition1/jjh/All-Rolls/math-roll/BUILD/sdsc-petsc_intel_openmpi_ib-3.5.2/petsc-3.5.2/src/mat/impls/aij/mpi/mpimatmatmult.c
>
> [4]PETSC ERROR: [4] MatTransposeMatMult_MPIAIJ_MPIAIJ line 812
> /state/partition1/jjh/All-Rolls/math-roll/BUILD/sdsc-petsc_intel_openmpi_ib-3.5.2/petsc-3.5.2/src/mat/impls/aij/mpi/mpimatmatmult.c
>
> [4]PETSC ERROR: [4] MatTransposeMatMult line 8945
> /state/partition1/jjh/All-Rolls/math-roll/BUILD/sdsc-petsc_intel_openmpi_ib-3.5.2/petsc-3.5.2/src/mat/interface/matrix.c
>
> [4]PETSC ERROR: [4] Schur_Prec line 2885
> /home/hus003/DISM/LSM_DISM3D/test/../Stokes3D.h
>
> [4]PETSC ERROR: [4] StokesSetup line 140
> /home/hus003/DISM/LSM_DISM3D/test/../Stokes3D.h
>
> [4]PETSC ERROR: [4] test line 102
> /home/hus003/DISM/LSM_DISM3D/test/test.cpp
>
> [4]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
>
> [4]PETSC ERROR: Signal received
>
> [4]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
>
> [4]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014
>
> [4]PETSC ERROR: ./test on a intel-openmpi_ib named tscc-gpu-6-5.sdsc.edu
> by hus003 Mon Jul 13 23:57:09 2015
>
> [4]PETSC ERROR: Configure options CFLAGS= FFLAGS= CXXFLAGS=
> --prefix=/opt/petsc/intel/openmpi_ib --with-mpi=1
> --download-pastix=../pastix_5.2.2.12.tar.bz2
> --download-ptscotch=../scotch_6.0.0_esmumps.tar.gz
> --with-blas-lib="-Wl,--start-group
> /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_intel_lp64.a
>
> /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_sequential.a
> /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_core.a
>     -Wl,--end-group -lpthread -lm" --with-lapack-lib="-Wl,--start-group
> /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_intel_lp64.a
>
> /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_sequential.a
> /opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/libmkl_core.a
>     -Wl,--end-group -lpthread -lm"
> --with-superlu_dist-include=/opt/superlu/intel/openmpi_ib/include
> --with-superlu_dist-lib="-L/opt/superlu/intel/openmpi_ib/lib -lsuperlu"
> --with-parmetis-dir=/opt/parmetis/intel/openmpi_ib
> --with-metis-dir=/opt/parmetis/intel/openmpi_ib
> --with-mpi-dir=/opt/openmpi/intel/ib
> --with-scalapack-dir=/opt/scalapack/intel/openmpi_ib
> --download-mumps=../MUMPS_4.10.0-p3.tar.gz
> --download-blacs=../blacs-dev.tar.gz
> --download-fblaslapack=../fblaslapack-3.4.2.tar.gz --with-pic=1
> --with-shared-libraries=1
>
> [4]PETSC ERROR: #1 User provided function() line 0 in  unknown file
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150714/8e683fde/attachment-0001.html>