[petsc-users] MUMPS Error

Tue Jan 8 03:37:53 CST 2019

>
> How large is the matrix, its dimension and number of non-zeros?
>
Can you run it sequentially or with less number of processors?
>
The matrix is 139603x139603 and with 15975113 of non-zero elements.
I tried using the iterative solver and started on one node then increased
to two there was no performance increase in using 2 nodes with 32 cores on
each node vs using 1 node with 32 cores (both took ~12min). Increasing the
number of nodes seems to crash it though.

 Can you run your case on different machine?
>
Running it on the work machine, it seems to work using the iterative solver
but will give me similar error I first posted if I use MUMPS (INFOG(1)=-9,
INFO(2)=0)
.

On Mon, Jan 7, 2019 at 4:47 PM Zhang, Hong <hzhang at mcs.anl.gov> wrote:

> Sal:
>
>> Recent versions of MUMPS have bugs when using SCALAPACK. You may also try
>>> to disable parallelism on the root node by using
>>
>> tried that, does not seem to help... but it is odd. this is not a big
>> system, it is small and made for testing.
>>
>
> How large is the matrix, its dimension and number of non-zeros?
> Can you run it sequentially or with less number of processors?
>
>>
>> perhaps there is something wrong with the MPI....If anyone is using the
>> PBSPro for queueing system, could it be the incompatibility between the
>> MPICH and the PBS? Should one use OpenMPI?
>>
>  Can you run your case on different machine?
> Hong
>
>>
>> On Thu, Jan 3, 2019 at 3:55 PM Stefano Zampini <stefano.zampini at gmail.com>
>> wrote:
>>
>>> Recent versions of MUMPS have bugs when using SCALAPACK. You may also
>>> try to disable parallelism on the root node by using
>>>
>>>  -mat_mumps_icntl_13 1
>>>
>>> On Jan 3, 2019, at 4:33 PM, Zhang, Hong via petsc-users <
>>> petsc-users at mcs.anl.gov> wrote:
>>>
>>> You may try different matrix orderings, or try superlu_dist.
>>> Hong
>>>
>>>
>>> On Thu, Jan 3, 2019 at 7:39 AM Matthew Knepley via petsc-users <
>>> petsc-users at mcs.anl.gov> wrote:
>>>
>>>> On Thu, Jan 3, 2019 at 5:58 AM Sal Am via petsc-users <
>>>> petsc-users at mcs.anl.gov> wrote:
>>>>
>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [0]PETSC ERROR: Error in external library
>>>>> [0]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=-22193
>>>>>
>>>>> [0]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [0]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [0]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [0]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>> [0]PETSC ERROR: [4]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [4]PETSC ERROR: Error in external library
>>>>> [4]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=-24197
>>>>>
>>>>> [4]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [4]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [4]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [4]PETSC ERROR: [16]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [16]PETSC ERROR: Error in external library
>>>>> [16]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>
>>>>> [16]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [16]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [16]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [16]PETSC ERROR: [20]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [20]PETSC ERROR: Error in external library
>>>>> [20]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>
>>>>> [20]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [20]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [20]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [20]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>> [20]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>> [24]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [24]PETSC ERROR: Error in external library
>>>>> [24]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>
>>>>> [24]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [24]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [24]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [24]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>> [24]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>> [24]PETSC ERROR: #2 MatLUFactorNumeric() line 3065 in
>>>>> /lustre/home/vef002/petsc/src/mat/interface/matrix.c
>>>>> [24]PETSC ERROR: #3 PCSetUp_LU() line 131 in
>>>>> /lustre/home/vef002/petsc/src/ksp/pc/impls/factor/lu/lu.c
>>>>> [24]PETSC ERROR: #4 PCSetUp() line 932 in
>>>>> /lustre/home/vef002/petsc/src/ksp/pc/interface/precon.c
>>>>> [24]PETSC ERROR: #5 KSPSetUp() line 391 in
>>>>> /lustre/home/vef002/petsc/src/ksp/ksp/interface/itfunc.c
>>>>> [24]PETSC ERROR: [32]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [32]PETSC ERROR: Error in external library
>>>>> [32]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>
>>>>> [32]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [32]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [32]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [32]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>> [32]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>> [32]PETSC ERROR: #2 MatLUFactorNumeric() line 3065 in
>>>>> /lustre/home/vef002/petsc/src/mat/interface/matrix.c
>>>>> [32]PETSC ERROR: #3 PCSetUp_LU() line 131 in
>>>>> /lustre/home/vef002/petsc/src/ksp/pc/impls/factor/lu/lu.c
>>>>> [32]PETSC ERROR: #4 PCSetUp() line 932 in
>>>>> /lustre/home/vef002/petsc/src/ksp/pc/interface/precon.c
>>>>> [32]PETSC ERROR: #5 KSPSetUp() line 391 in
>>>>> /lustre/home/vef002/petsc/src/ksp/ksp/interface/itfunc.c
>>>>> [32]PETSC ERROR: [8]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [8]PETSC ERROR: Error in external library
>>>>> [8]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=-35665
>>>>>
>>>>> [8]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [8]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [8]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [8]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>> [8]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>> [8]PETSC ERROR: #2 MatLUFactorNumeric() line 3065 in
>>>>> /lustre/home/vef002/petsc/src/mat/interface/matrix.c
>>>>> [8]PETSC ERROR: #3 PCSetUp_LU() line 131 in
>>>>> /lustre/home/vef002/petsc/src/ksp/pc/impls/factor/lu/lu.c
>>>>> [8]PETSC ERROR: #4 PCSetUp() line 932 in
>>>>> /lustre/home/vef002/petsc/src/ksp/pc/interface/precon.c
>>>>> [8]PETSC ERROR: [12]PETSC ERROR: --------------------- Error Message
>>>>> --------------------------------------------------------------
>>>>> [12]PETSC ERROR: Error in external library
>>>>> [12]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>> phase: INFOG(1)=-13, INFO(2)=-16661
>>>>>
>>>>> [12]PETSC ERROR: See
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>> shooting.
>>>>> [12]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>> [12]PETSC ERROR: ./solveCSys on a linux-opt named r03n13 by vef002 Thu
>>>>> Jan  3 04:32:43 2019
>>>>> [12]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>> [12]PETSC ERROR: #6 main() line 68 in
>>>>> /lustre/home/vef002/tests/solveCmplxLinearSys.cpp
>>>>> [24]PETSC ERROR: PETSc Option Table entries:
>>>>> [24]PETSC ERROR: -ksp_error_if_not_converged
>>>>> [24]PETSC ERROR: -ksp_max_it 1
>>>>> [24]PETSC ERROR: -ksp_monitor_true_residual
>>>>> [24]PETSC ERROR: -ksp_type richardson
>>>>> [24]PETSC ERROR: -log_view
>>>>> [24]PETSC ERROR: -mat_mumps_icntl_14 100
>>>>> [24]PETSC ERROR: -pc_factor_mat_solver_type mumps
>>>>> [24]PETSC ERROR: -pc_type lu
>>>>> [24]PETSC ERROR: ----------------End of Error Message -------send
>>>>> entire error message to petsc-maint at mcs.anl.gov----------
>>>>>
>>>>> Sorry to bother again, but increasing ICNTL(14) does not seem to have
>>>>> helped... I am not sure what I am doing wrong exactly. As you can see I
>>>>> have increased it by 100%, prior, I tried to increase it with 40-70% but
>>>>> none worked.
>>>>>
>>>>
>>>> I think this is for the MUMPS list, but it appears you have a lot of
>>>> fill-in for this matrix and it might be too much for a direct solver.
>>>>
>>>>   Thanks,
>>>>
>>>>     Matt
>>>>
>>>>
>>>>> On Wed, Dec 12, 2018 at 3:38 PM Zhang, Junchao <jczhang at mcs.anl.gov>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 12, 2018 at 7:14 AM Sal Am via petsc-users <
>>>>>> petsc-users at mcs.anl.gov> wrote:
>>>>>>
>>>>>>> Hi I am getting an error using MUMPS.
>>>>>>> How I run it:
>>>>>>> bash-4.2$ mpiexec -n 16 ./solveCSys -ksp_type richardson -pc_type lu
>>>>>>> -pc_factor_mat_solver_type mumps -ksp_max_it 1 -ksp_monitor_true_residual
>>>>>>> -log_view -ksp_error_if_not_converged
>>>>>>>
>>>>>>> The error output:
>>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [0]PETSC ERROR: Error in external library
>>>>>>> [0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [1]PETSC ERROR: Error in external library
>>>>>>> [1]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [1]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [6]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [6]PETSC ERROR: Error in external library
>>>>>>> [6]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [6]PETSC ERROR: [8]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [8]PETSC ERROR: Error in external library
>>>>>>> [8]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [8]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [8]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [8]PETSC ERROR: [9]PETSC ERROR: [15]PETSC ERROR:
>>>>>>> --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [15]PETSC ERROR: Error in external library
>>>>>>> [15]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=-36536
>>>>>>>
>>>>>>> [15]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [15]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [15]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [15]PETSC ERROR: Configure options PETSC_ARCH=linux-opt
>>>>>>> --with-cc=gcc --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [15]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=-81813
>>>>>>>
>>>>>>> [0]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [0]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [1]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [1]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [1]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [1]PETSC ERROR: [2]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [2]PETSC ERROR: Error in external library
>>>>>>> [2]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [2]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [2]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [2]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [2]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [2]PETSC ERROR: [3]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [3]PETSC ERROR: Error in external library
>>>>>>> [3]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=-28194
>>>>>>>
>>>>>>> [3]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [3]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [3]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [3]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [3]PETSC ERROR: [4]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [4]PETSC ERROR: Error in external library
>>>>>>> [4]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [4]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [4]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [4]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [4]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [5]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [5]PETSC ERROR: Error in external library
>>>>>>> [5]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=-27685
>>>>>>>
>>>>>>> [5]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [5]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [5]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [5]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [5]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [6]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [6]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [6]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [6]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [6]PETSC ERROR: [7]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [7]PETSC ERROR: Error in external library
>>>>>>> [7]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [7]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [7]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [7]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [7]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [7]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [7]PETSC ERROR: #2 MatLUFactorNumeric() line 3065 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/interface/matrix.c
>>>>>>> ./solveCSys on a linux-opt named r02g03 by vef002 Wed Dec 12
>>>>>>> 10:44:13 2018
>>>>>>> [8]PETSC ERROR: Configure options PETSC_ARCH=linux-opt --with-cc=gcc
>>>>>>> --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [8]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [8]PETSC ERROR: #2 MatLUFactorNumeric() line 3065 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/interface/matrix.c
>>>>>>> [8]PETSC ERROR: #3 PCSetUp_LU() line 131 in
>>>>>>> /lustre/home/vef002/petsc/src/ksp/pc/impls/factor/lu/lu.c
>>>>>>> [10]PETSC ERROR: --------------------- Error Message
>>>>>>> --------------------------------------------------------------
>>>>>>> [10]PETSC ERROR: Error in external library
>>>>>>> [10]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [10]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [10]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [10]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [10]PETSC ERROR: Configure options PETSC_ARCH=linux-opt
>>>>>>> --with-cc=gcc --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [10]PETSC ERROR: [11]PETSC ERROR: --------------------- Error
>>>>>>> Message --------------------------------------------------------------
>>>>>>> [11]PETSC ERROR: Error in external library
>>>>>>> [11]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=-30519
>>>>>>>
>>>>>>> [11]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [11]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [11]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [11]PETSC ERROR: Configure options PETSC_ARCH=linux-opt
>>>>>>> --with-cc=gcc --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [11]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [11]PETSC ERROR: [12]PETSC ERROR: --------------------- Error
>>>>>>> Message --------------------------------------------------------------
>>>>>>> [12]PETSC ERROR: Error in external library
>>>>>>> [12]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=-28435
>>>>>>>
>>>>>>> [12]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [12]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [12]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [12]PETSC ERROR: Configure options PETSC_ARCH=linux-opt
>>>>>>> --with-cc=gcc --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [12]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [12]PETSC ERROR: [13]PETSC ERROR: --------------------- Error
>>>>>>> Message --------------------------------------------------------------
>>>>>>> [13]PETSC ERROR: Error in external library
>>>>>>> [13]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=-30110
>>>>>>>
>>>>>>> [13]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [13]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [13]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [13]PETSC ERROR: Configure options PETSC_ARCH=linux-opt
>>>>>>> --with-cc=gcc --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [13]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [13]PETSC ERROR: [14]PETSC ERROR: --------------------- Error
>>>>>>> Message --------------------------------------------------------------
>>>>>>> [14]PETSC ERROR: Error in external library
>>>>>>> [14]PETSC ERROR: Error reported by MUMPS in numerical factorization
>>>>>>> phase: INFOG(1)=-13, INFO(2)=0
>>>>>>>
>>>>>>> [14]PETSC ERROR: See
>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble
>>>>>>> shooting.
>>>>>>> [14]PETSC ERROR: Petsc Release Version 3.10.2, unknown
>>>>>>> [14]PETSC ERROR: ./solveCSys on a linux-opt named r02g03 by vef002
>>>>>>> Wed Dec 12 10:44:13 2018
>>>>>>> [14]PETSC ERROR: Configure options PETSC_ARCH=linux-opt
>>>>>>> --with-cc=gcc --with-fc=gfortran --with-cxx=g++ --with-clanguage=cxx
>>>>>>> --download-superlu_dist --download-mumps --with-scalar-type=complex
>>>>>>> --with-debugging=no --download-scalapack --download-superlu
>>>>>>> --download-mpich --download-fblaslapack=1 --download-cmake
>>>>>>> [14]PETSC ERROR: #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [14]PETSC ERROR: #2 MatLUFactorNumeric() line 3065 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/interface/matrix.c
>>>>>>> #1 MatFactorNumeric_MUMPS() line 1230 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/impls/aij/mpi/mumps/mumps.c
>>>>>>> [15]PETSC ERROR: #2 MatLUFactorNumeric() line 3065 in
>>>>>>> /lustre/home/vef002/petsc/src/mat/interface/matrix.c
>>>>>>> [15]PETSC ERROR: #3 PCSetUp_LU() line 131 in
>>>>>>> /lustre/home/vef002/petsc/src/ksp/pc/impls/factor/lu/lu.c
>>>>>>> [15]PETSC ERROR: #4 PCSetUp() line 932 in
>>>>>>> /lustre/home/vef002/petsc/src/ksp/pc/interface/precon.c
>>>>>>> [15]PETSC ERROR: #5 KSPSetUp() line 391 in
>>>>>>> /lustre/home/vef002/petsc/src/ksp/ksp/interface/itfunc.c
>>>>>>> [15]PETSC ERROR: #6 main() line 68 in
>>>>>>> /lustre/home/vef002/tests/solveCmplxLinearSys.cpp
>>>>>>> [15]PETSC ERROR: PETSc Option Table entries:
>>>>>>> [15]PETSC ERROR: -ksp_error_if_not_converged
>>>>>>> [15]PETSC ERROR: -ksp_max_it 1
>>>>>>> [15]PETSC ERROR: -ksp_monitor_true_residual
>>>>>>> [15]PETSC ERROR: -ksp_type richardson
>>>>>>> [15]PETSC ERROR: -log_view
>>>>>>> [15]PETSC ERROR: -pc_factor_mat_solver_type mumps
>>>>>>> [15]PETSC ERROR: -pc_type lu
>>>>>>> [15]PETSC ERROR: ----------------End of Error Message -------send
>>>>>>> entire error message to petsc-maint at mcs.anl.gov----------
>>>>>>>
>>>>>>> From the MUMPS manual it seems INFOG(1)=-13, INFO(2)=0 :
>>>>>>> Problem of workspace allocation of size INFO(2) during the
>>>>>>> factorization or solve steps. The size that the package tried to allocate
>>>>>>> with a Fortran ALLOCATE statement is available in INFO(2). If INFO(2) is
>>>>>>> negative, then the size that the package requested is obtained by
>>>>>>> multiplying the absolute value of INFO(2) by 1 million. In general, the
>>>>>>> unit for INFO(2) is the number of scalar entries of  the type of the input
>>>>>>> matrix (real, complex, single or double precision).
>>>>>>>
>>>>>>> Not quite sure what to make of it as the first error indicates size
>>>>>>> INFO(2)= 0?. Would I need to make use of -mat_mumps_icntl_14?
>>>>>>>
>>>>>> But you can see there are processes with negative INFO(2). According
>>>>>> to MUMPS manual, you can try to increase ICNTL(14).
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>> https://www.cse.buffalo.edu/~knepley/
>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190108/a9ec2130/attachment-0001.html>