<div dir="ltr">More precisely, on all 16 processes, same error message:<br><br>[13]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[13]PETSC ERROR: Nonconforming object sizes<br>[13]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[13]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[13]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[13]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[13]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[13]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[14]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[14]PETSC ERROR: Nonconforming object sizes<br>[14]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[14]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[14]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[14]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[14]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[14]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[14]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[14]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[14]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[14]PETSC ERROR: [0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[1]PETSC ERROR: Nonconforming object sizes<br>[1]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[1]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[1]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[1]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[1]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[1]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[1]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[1]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[1]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[1]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[6]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[6]PETSC ERROR: Nonconforming object sizes<br>[6]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[6]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[6]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[6]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[6]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[6]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[6]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[6]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[6]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[6]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[7]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[7]PETSC ERROR: Nonconforming object sizes<br>[7]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[7]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[7]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[7]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[7]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[7]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[7]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[7]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[7]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[7]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[8]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[8]PETSC ERROR: Nonconforming object sizes<br>[8]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[8]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[8]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[8]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[8]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[8]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[8]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[8]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[8]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[8]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[9]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[9]PETSC ERROR: Nonconforming object sizes<br>[9]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[9]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[9]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[9]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[9]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[9]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[9]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[9]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[9]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[9]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[10]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[10]PETSC ERROR: Nonconforming object sizes<br>[10]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[10]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[10]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[10]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[10]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[10]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[10]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[10]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[10]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[10]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[11]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[11]PETSC ERROR: Nonconforming object sizes<br>[11]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[11]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[11]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[11]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[11]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[11]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[11]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[11]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[11]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[11]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[12]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[12]PETSC ERROR: Nonconforming object sizes<br>[12]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[12]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[12]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[12]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[12]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[12]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[12]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[12]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[12]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[12]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[13]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[13]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[13]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[13]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[15]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[15]PETSC ERROR: Nonconforming object sizes<br>[15]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[15]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[15]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[15]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[15]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[15]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[15]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[15]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[15]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[15]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>#5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[2]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[2]PETSC ERROR: Nonconforming object sizes<br>[2]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[2]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[2]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[2]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[2]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[2]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[2]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[2]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[2]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[2]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[3]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[3]PETSC ERROR: Nonconforming object sizes<br>[3]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[3]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[3]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[3]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[3]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[3]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[3]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[3]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[3]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[3]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[4]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[4]PETSC ERROR: Nonconforming object sizes<br>[4]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[4]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[4]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[4]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[4]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[4]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[4]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[4]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[4]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[4]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>[5]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[5]PETSC ERROR: Nonconforming object sizes<br>[5]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br>[5]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[5]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[5]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[5]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[5]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[5]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[5]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[5]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[5]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br>--------------------- Error Message --------------------------------------------------------------<br>[0]PETSC ERROR: Nonconforming object sizes<br>[0]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[0]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[0]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[0]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[0]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[0]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[0]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[0]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[0]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br></div><div class="gmail_extra"><br><div class="gmail_quote">2015-11-13 10:39 GMT+09:00 Timothée Nicolas <span dir="ltr"><<a href="mailto:timothee.nicolas@gmail.com" target="_blank">timothee.nicolas@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Sorry, here is the full error message<br><br>[0]PETSC ERROR: Nonconforming object sizes<br>[0]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 71808<br>[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[0]PETSC ERROR: Petsc Release Version 3.6.0, Jun, 09, 2015 <br>[0]PETSC ERROR: ./mips_implicit on a arch-linux2-c-opt named helios90 by tnicolas Fri Nov 13 10:39:14 2015<br>[0]PETSC ERROR: Configure options --prefix=/csc/softs/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real --with-debugging=0 --with-x=0 --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-fortran --known-mpi-shared-libraries=1 --with-scalar-type=real --with-precision=double --CFLAGS="-g -O3 -mavx -mkl" --CXXFLAGS="-g -O3 -mavx -mkl" --FFLAGS="-g -O3 -mavx -mkl"<br>[0]PETSC ERROR: #1 PCApply() line 472 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/pc/interface/precon.c<br>[0]PETSC ERROR: #2 KSP_PCApply() line 242 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/include/petsc/private/kspimpl.h<br>[0]PETSC ERROR: #3 KSPInitialResidual() line 63 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itres.c<br>[0]PETSC ERROR: #4 KSPSolve_GMRES() line 235 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/impls/gmres/gmres.c<br>[0]PETSC ERROR: #5 KSPSolve() line 604 in /csc/releases/buildlog/anl/petsc-3.6.0/intel-15.0.0.090/bullxmpi-1.2.8.2/real/petsc-3.6.0/src/ksp/ksp/interface/itfunc.c<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">2015-11-13 10:38 GMT+09:00 Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Thu, Nov 12, 2015 at 6:47 PM, Timothée Nicolas <span dir="ltr"><<a href="mailto:timothee.nicolas@gmail.com" target="_blank">timothee.nicolas@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div>Hi all,<br><br></div>In the manual and the documentation, the syntax for the routine to be given as argument of PCMGSetResidual:<br><br><pre><a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/PC/PCMGSetResidual.html#PCMGSetResidual" target="_blank">PCMGSetResidual</a>(PC pc,<a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Sys/PetscInt.html#PetscInt" target="_blank">PetscInt</a> l,<a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Sys/PetscErrorCode.html#PetscErrorCode" target="_blank">PetscErrorCode</a> (*residual)(<a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Mat/Mat.html#Mat" target="_blank">Mat</a>,<a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Vec/Vec.html#Vec" target="_blank">Vec</a>,<a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Vec/Vec.html#Vec" target="_blank">Vec</a>,<a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Vec/Vec.html#Vec" target="_blank">Vec</a>),<a href="http://www.mcs.anl.gov/petsc/petsc-3.6/docs/manualpages/Mat/Mat.html#Mat" target="_blank">Mat</a> mat)</pre><br></div>is not specified. I mean that the order of the vectors is not specified. I suppose it is something like<br></div>residual(Mat,b,x,r) with r = b - Mat*x, but it could as well be any combination like residual(Mat,r,x,b). There is no example in the documentation of the usage so I am confused. Does it absolutely need to be set ? I find the manual a bit confusing on this point. Is it only if matrix-free matrices are used ?<br><br></div>In the present situation, I use matrix-free operators in a multigrid preconditioner (but the interpolation and restriction are not matrix free) and have not set this residual function yet. I get the following error:<br></div></div></div></div></blockquote><div><br></div></span><div>Always always always give the entire error message. We want the stack.</div><div><br></div><div>The problem here looks like the preconditioner is reporting -1 rows for process 13.</div><div><br></div><div>  Matt</div><span><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>[13]PETSC ERROR: Preconditioner number of local rows -1 does not equal resulting vector number of rows 67584<br><br></div>Could this be related ? By the way, I don't understand what is meant by the "preconditioner number of local rows". I have separately tested the operators at each level and they are fine.<br><br></div>Best<br><br></div>Timothee<br></div>
</blockquote></span></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>