<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 8, 2015 at 12:57 AM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
It is getting a zero pivot when factoring the little dense diagonal blocks of the smoother matrix (to do SOR).<br>
<br>
What happens if you run with the additional option -mat_no_inode?<br></blockquote><div><br>With -mat_no_inode I see that the fieldsplit_0_solve has preconditioned norm converging but not the true resid norm. And therefore fieldsplit_1_solve not converge.<br><br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm 0.000000000000e+00 ||r(i)||/||b|| -nan<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 5.366616065799e-01 true resid norm 1.763251719324e+01 ||r(i)||/||b|| 1.000000000000e+00<br> 1 KSP preconditioned resid norm 2.051153486997e-15 true resid norm 1.763377056173e+01 ||r(i)||/||b|| 1.000071082789e+00<br> Residual norms for fieldsplit_1_ solve.<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm 0.000000000000e+00 ||r(i)||/||b|| -nan<br> 0 KSP preconditioned resid norm 7.379043098663e+00 true resid norm 7.379043147181e+00 ||r(i)||/||b|| 1.000000000000e+00<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 1.013832784140e-01 true resid norm 1.194932744284e+00 ||r(i)||/||b|| 1.000000000000e+00<br> 1 KSP preconditioned resid norm 3.651850464015e-16 true resid norm 1.195413534155e+00 ||r(i)||/||b|| 1.000402357265e+00<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 9.468489343208e-01 true resid norm 1.115983634787e+01 ||r(i)||/||b|| 1.000000000000e+00<br> 1 KSP preconditioned resid norm 4.573576948730e-15 true resid norm 1.116432658910e+01 ||r(i)||/||b|| 1.000402357265e+00<br> 1 KSP preconditioned resid norm 7.376179991979e+00 true resid norm 7.376180019146e+00 ||r(i)||/||b|| 9.996119919645e-01<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 4.141204698273e+00 true resid norm 1.010004734003e+00 ||r(i)||/||b|| 1.000000000000e+00<br> 1 KSP preconditioned resid norm 3.094717134282e-14 true resid norm 1.612661239170e+00 ||r(i)||/||b|| 1.596686812327e+00<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 9.488116584954e-01 true resid norm 2.657897132073e+01 ||r(i)||/||b|| 1.000000000000e+00<br> 1 KSP preconditioned resid norm 2.486948812256e-15 true resid norm 2.658134372650e+01 ||r(i)||/||b|| 1.000089258750e+00<br> 2 KSP preconditioned resid norm 7.376160898815e+00 true resid norm 7.376160928269e+00 ||r(i)||/||b|| 9.996094047893e-01<br> .......................<br> .......................<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 3.198579793302e+00 true resid norm 1.508640813109e+00 ||r(i)||/||b|| 1.000000000000e+00<br> 1 KSP preconditioned resid norm 1.839267393927e-14 true resid norm 1.825155061421e+00 ||r(i)||/||b|| 1.209800931781e+00<br> Residual norms for fieldsplit_0_ solve.<br> 0 KSP preconditioned resid norm 1.497257742332e+01 true resid norm 4.292799680684e+14 ||r(i)||/||b|| 1.000000000000e+00<br> 1 KSP preconditioned resid norm 1.138874539002e-13 true resid norm 4.292799680684e+14 ||r(i)||/||b|| 1.000000000000e+00<br> 43 KSP preconditioned resid norm 6.372901415746e+00 true resid norm 7.990778566794e+00 ||r(i)||/||b|| 1.082901726878e+00<br> .... And continues<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Run with the additional option -start_in_debugger noxterm<br>
<br>
In the debugger type<br>
<br>
cont<br>
<br>
when it crashes type<br>
<br>
bt<br>
<br>
then use<br>
<br>
up<br>
<br>
as many times as needed to get to the routine MatSOR() then type<br>
<br>
call MatView(mat,0)<br>
<br>
you should see that diagonal subblock from the 3rd row to 7th row is singular. Normally we would expect this from the operator<br>
mu lap(u) unless the mu has zero entries?<br></blockquote><div><br></div><div>Sorry I didn't understand what exactly the diagonal sublock mean here? <br>mat is of a big size, so it was not easy to see with MatView. Looking at its output, what I see is that every now and then there are 3 or 4 contiguous rows (i, i+1, i+2) having mostly 0 values. No mu does not have zero entries, I set it to 1 everywhere except at Dirichlet boundaries where mu is not used.<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
What happens if you run -fieldsplit_0_pc_type sor ? Does it also give a zero factorization?<br></blockquote><div><br>No it does not give zero factorization, the system converges in 2 iterations. <br>The -fieldsplit_0_ solves mostly in around 27 iterations, and for -fieldsplit_1_:<br>Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 17<br><br><br></div><div></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class=""><div class="h5"><br>
<br>
> On Dec 7, 2015, at 5:35 PM, Bishesh Khanal <<a href="mailto:bisheshkh@gmail.com">bisheshkh@gmail.com</a>> wrote:<br>
><br>
> Hello,<br>
> I'm revisiting my code for solving a system of eqs similar to Stokes to get better convergence for big jumps in viscosity (mu) and wanted to experiment with petsc's gamg:<br>
> mu lap(u) + grad(p) = f1<br>
> div(u) = f2<br>
> Dirichlet boundary condition of zero velocity.<br>
> The implementation is a finite difference with a staggered grid discretization.<br>
><br>
> Using Schur FieldSplit PC type, and for constant viscosity, using hypre PC for A00 block had been serving well. Now when trying gamg, I get zero pivot error. From some of the past discussions in the archives of the mailing list, it seems I've to play around with ..._sub_pc_factor_shift_type but I could not figure out the exact set of options.<br>
><br>
> Setting mu = 1 everywhere; Using the following options:<br>
> -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_schur_precondition self -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type gamg<br>
> I get an error.<br>
> When I use -fieldsplit_0_pc_type lu , it converges in a single iteration so I guess it should mean that A00 is not singular.<br>
> Below are the results of using:<br>
> 1. -fieldsplit_0_pc_type hypre (converges)<br>
> 2. -ksp_view -fieldsplit_0_pc_type lu (converges)<br>
> 3. -fieldsplit_0_pc_type gamg (all error output)<br>
><br>
><br>
> 1. ----------------------------------------<br>
> With -fieldsplit_0_pc_type hypre<br>
> ----------------------------------------<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 17<br>
> Linear solve converged due to CONVERGED_RTOL iterations 2<br>
><br>
> 2. --------------------------------------------<br>
> With -fieldsplit_0_pc_type lu -ksp_view<br>
> -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_schur_precondition self -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type lu -fieldsplit_1_ksp_rtol 1.0e-10 -fieldsplit_1_ksp_converged_reason -ksp_converged_reason -ksp_view<br>
><br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_ATOL iterations 0<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
> ...<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 22<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
> ...<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 22<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
> Linear solve converged due to CONVERGED_RTOL iterations 1<br>
><br>
> KSP Object: 1 MPI processes<br>
> type: gmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=10000, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using PRECONDITIONED norm type for convergence test<br>
> PC Object: 1 MPI processes<br>
> type: fieldsplit<br>
> FieldSplit with Schur preconditioner, blocksize = 4, factorization FULL<br>
> Preconditioner for the Schur complement formed from S itself<br>
> Split info:<br>
> Split number 0 Fields 0, 1, 2<br>
> Split number 1 Fields 3<br>
> KSP solver for A00 block<br>
> KSP Object: (fieldsplit_0_) 1 MPI processes<br>
> type: gmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=10000, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using PRECONDITIONED norm type for convergence test<br>
> PC Object: (fieldsplit_0_) 1 MPI processes<br>
> type: lu<br>
> LU: out-of-place factorization<br>
> tolerance for zero pivot 2.22045e-14<br>
> matrix ordering: nd<br>
> factor fill ratio given 5, needed 20.772<br>
> Factored matrix follows:<br>
> Mat Object: 1 MPI processes<br>
> type: seqaij<br>
> rows=54243, cols=54243, bs=3<br>
> package used to perform factorization: petsc<br>
> total: nonzeros=8.41718e+07, allocated nonzeros=8.41718e+07<br>
> total number of mallocs used during MatSetValues calls =0<br>
> using I-node routines: found 17801 nodes, limit used is 5<br>
> linear system matrix = precond matrix:<br>
> Mat Object: (fieldsplit_0_) 1 MPI processes<br>
> type: seqaij<br>
> rows=54243, cols=54243, bs=3<br>
> total: nonzeros=4.05217e+06, allocated nonzeros=4.05217e+06<br>
> total number of mallocs used during MatSetValues calls =0<br>
> using I-node routines: found 18081 nodes, limit used is 5<br>
> KSP solver for S = A11 - A10 inv(A00) A01<br>
> KSP Object: (fieldsplit_1_) 1 MPI processes<br>
> type: gmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=10000, initial guess is zero<br>
> tolerances: relative=1e-10, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using PRECONDITIONED norm type for convergence test<br>
> PC Object: (fieldsplit_1_) 1 MPI processes<br>
> type: none<br>
> linear system matrix = precond matrix:<br>
> Mat Object: (fieldsplit_1_) 1 MPI processes<br>
> type: schurcomplement<br>
> rows=18081, cols=18081<br>
> has attached null space<br>
> Schur complement A11 - A10 inv(A00) A01<br>
> A11<br>
> Mat Object: (fieldsplit_1_) 1 MPI processes<br>
> type: seqaij<br>
> rows=18081, cols=18081<br>
> total: nonzeros=450241, allocated nonzeros=450241<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node routines<br>
> A10<br>
> Mat Object: 1 MPI processes<br>
> type: seqaij<br>
> rows=18081, cols=54243<br>
> total: nonzeros=1.35072e+06, allocated nonzeros=1.35072e+06<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node routines<br>
> KSP of A00<br>
> KSP Object: (fieldsplit_0_) 1 MPI processes<br>
> type: gmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=10000, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using PRECONDITIONED norm type for convergence test<br>
> PC Object: (fieldsplit_0_) 1 MPI processes<br>
> type: lu<br>
> LU: out-of-place factorization<br>
> tolerance for zero pivot 2.22045e-14<br>
> matrix ordering: nd<br>
> factor fill ratio given 5, needed 20.772<br>
> Factored matrix follows:<br>
> Mat Object: 1 MPI processes<br>
> type: seqaij<br>
> rows=54243, cols=54243, bs=3<br>
> package used to perform factorization: petsc<br>
> total: nonzeros=8.41718e+07, allocated nonzeros=8.41718e+07<br>
> total number of mallocs used during MatSetValues calls =0<br>
> using I-node routines: found 17801 nodes, limit used is 5<br>
> linear system matrix = precond matrix:<br>
> Mat Object: (fieldsplit_0_) 1 MPI processes<br>
> type: seqaij<br>
> rows=54243, cols=54243, bs=3<br>
> total: nonzeros=4.05217e+06, allocated nonzeros=4.05217e+06<br>
> total number of mallocs used during MatSetValues calls =0<br>
> using I-node routines: found 18081 nodes, limit used is 5<br>
> A01<br>
> Mat Object: 1 MPI processes<br>
> type: seqaij<br>
> rows=54243, cols=18081, rbs=3, cbs = 1<br>
> total: nonzeros=1.35072e+06, allocated nonzeros=1.35072e+06<br>
> total number of mallocs used during MatSetValues calls =0<br>
> using I-node routines: found 18081 nodes, limit used is 5<br>
> linear system matrix = precond matrix:<br>
> Mat Object: 1 MPI processes<br>
> type: seqaij<br>
> rows=72324, cols=72324, bs=4<br>
> total: nonzeros=7.20386e+06, allocated nonzeros=7.20386e+06<br>
> total number of mallocs used during MatSetValues calls =0<br>
> has attached null space<br>
> using I-node routines: found 18081 nodes, limit used is 5<br>
><br>
><br>
> 3. ----------------------------------------<br>
> when using gamg: -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_schur_precondition self -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type gamg -fieldsplit_0_ksp_converged_reason -fieldsplit_1_ksp_converged_reason -ksp_converged_reason -ksp_view<br>
> ----------------------------------------<br>
><br>
> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>
> [0]PETSC ERROR: Zero pivot in LU factorization: <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#zeropivot" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#zeropivot</a><br>
> [0]PETSC ERROR: Zero pivot, row 3<br>
> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
> [0]PETSC ERROR: Petsc Release Version 3.6.3, Dec, 03, 2015<br>
> [0]PETSC ERROR: /user/bkhanal/home/works/AdLemModel/build/src/AdLemMain on a arch-linux2-cxx-debug named delorme by bkhanal Mon Dec 7 23:13:00 2015<br>
> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-scalapack --download-mpich --download-superlu_dist --download-mumps --download-hypre --download-metis --download-parmetis<br>
> [0]PETSC ERROR: #1 PetscKernel_A_gets_inverse_A_5() line 48 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/impls/baij/seq/dgefa5.c<br>
> [0]PETSC ERROR: #2 MatSOR_SeqAIJ_Inode() line 2808 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/impls/aij/seq/inode.c<br>
> [0]PETSC ERROR: #3 MatSOR() line 3697 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/interface/matrix.c<br>
> [0]PETSC ERROR: #4 PCApply_SOR() line 37 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/sor/sor.c<br>
> [0]PETSC ERROR: #5 PCApply() line 483 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: #6 KSP_PCApply() line 242 in /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: #7 KSPInitialResidual() line 63 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: #8 KSPSolve_GMRES() line 235 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: #9 KSPSolve() line 604 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: #10 KSPSolve_Chebyshev() line 381 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/cheby/cheby.c<br>
> [0]PETSC ERROR: #11 KSPSolve() line 604 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: #12 PCMGMCycle_Private() line 19 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: #13 PCMGMCycle_Private() line 48 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: #14 PCApply_MG() line 340 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: #15 PCApply() line 483 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: #16 KSP_PCApply() line 242 in /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: #17 KSPInitialResidual() line 63 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: #18 KSPSolve_GMRES() line 235 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: #19 KSPSolve() line 604 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: #20 MatMult_SchurComplement() line 105 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/utils/schurm.c<br>
> [0]PETSC ERROR: #21 MatNullSpaceTest() line 431 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/interface/matnull.c<br>
><br>
> WARNING: not a valid pressure null space<br>
> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>
> [0]PETSC ERROR: Object is in wrong state<br>
> [0]PETSC ERROR: Vec is locked read only, argument # 1<br>
> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
> [0]PETSC ERROR: Petsc Release Version 3.6.3, Dec, 03, 2015<br>
> [0]PETSC ERROR: /user/bkhanal/home/works/AdLemModel/build/src/AdLemMain on a arch-linux2-cxx-debug named delorme by bkhanal Mon Dec 7 23:13:00 2015<br>
> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-scalapack --download-mpich --download-superlu_dist --download-mumps --download-hypre --download-metis --download-parmetis<br>
> [0]PETSC ERROR: #22 VecSet() line 573 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/vec/vec/interface/rvector.c<br>
> [0]PETSC ERROR: #23 MatMultTranspose_SeqAIJ() line 1263 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/impls/aij/seq/aij.c<br>
> [0]PETSC ERROR: #24 MatMultTranspose() line 2282 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/interface/matrix.c<br>
> [0]PETSC ERROR: #25 MatRestrict() line 7829 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/interface/matrix.c<br>
> [0]PETSC ERROR: #26 PCMGMCycle_Private() line 44 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: #27 PCApply_MG() line 340 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: #28 PCApply() line 483 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: #29 KSP_PCApply() line 242 in /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: #30 KSPInitialResidual() line 63 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: #31 KSPSolve_GMRES() line 235 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: #32 KSPSolve() line 604 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: #33 PCApply_FieldSplit_Schur() line 898 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/fieldsplit/fieldsplit.c<br>
> [0]PETSC ERROR: #34 PCApply() line 483 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: #35 KSP_PCApply() line 242 in /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: #36 KSPInitialResidual() line 63 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: #37 KSPSolve_GMRES() line 235 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: #38 KSPSolve() line 604 in /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: #39 solveModel() line 313 in /home/bkhanal/works/AdLemModel/src/PetscAdLemTaras3D.cxx<br>
> [0]PETSC ERROR: ------------------------------------------------------------------------<br>
> [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range<br>
> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
> [0]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a><br>
> [0]PETSC ERROR: or try <a href="http://valgrind.org" rel="noreferrer" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors<br>
> [0]PETSC ERROR: likely location of problem given in stack below<br>
> [0]PETSC ERROR: --------------------- Stack Frames ------------------------------------<br>
> [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,<br>
> [0]PETSC ERROR: INSTEAD the line number of the start of the function<br>
> [0]PETSC ERROR: is given.<br>
> [0]PETSC ERROR: [0] getSolVelocityAt line 137 /home/bkhanal/works/AdLemModel/src/PetscAdLem3D.cxx<br>
> [0]PETSC ERROR: [0] VecSet line 568 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/vec/vec/interface/rvector.c<br>
> [0]PETSC ERROR: [0] MatMultTranspose_SeqAIJ line 1262 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/impls/aij/seq/aij.c<br>
> [0]PETSC ERROR: [0] MatMultTranspose line 2263 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/interface/matrix.c<br>
> [0]PETSC ERROR: [0] MatRestrict line 7817 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/interface/matrix.c<br>
> [0]PETSC ERROR: [0] PCMGMCycle_Private line 17 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: [0] PCApply_MG line 325 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: [0] PCApply line 463 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: [0] KSP_PCApply line 240 /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: [0] KSPInitialResidual line 44 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: [0] KSPSolve_GMRES line 225 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: [0] KSPSolve line 510 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: [0] PCApply_FieldSplit_Schur line 848 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/fieldsplit/fieldsplit.c<br>
> [0]PETSC ERROR: [0] PCApply line 463 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: [0] KSP_PCApply line 240 /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: [0] KSPInitialResidual line 44 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: [0] KSPSolve_GMRES line 225 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: [0] KSPSolve line 510 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: [0] PetscKernel_A_gets_inverse_A_5 line 25 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/impls/baij/seq/dgefa5.c<br>
> [0]PETSC ERROR: [0] MatSOR_SeqAIJ_Inode line 2764 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/impls/aij/seq/inode.c<br>
> [0]PETSC ERROR: [0] MatSOR line 3678 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/mat/interface/matrix.c<br>
> [0]PETSC ERROR: [0] PCApply_SOR line 35 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/sor/sor.c<br>
> [0]PETSC ERROR: [0] PCApply line 463 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: [0] KSP_PCApply line 240 /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: [0] KSPInitialResidual line 44 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: [0] KSPSolve_GMRES line 225 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: [0] KSPSolve line 510 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: [0] KSPSolve_Chebyshev line 358 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/cheby/cheby.c<br>
> [0]PETSC ERROR: [0] KSPSolve line 510 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itfunc.c<br>
> [0]PETSC ERROR: [0] PCMGMCycle_Private line 17 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: [0] PCMGMCycle_Private line 17 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: [0] PCApply_MG line 325 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/impls/mg/mg.c<br>
> [0]PETSC ERROR: [0] PCApply line 463 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/pc/interface/precon.c<br>
> [0]PETSC ERROR: [0] KSP_PCApply line 240 /home/bkhanal/Documents/softwares/petsc-3.6.3/include/petsc/private/kspimpl.h<br>
> [0]PETSC ERROR: [0] KSPInitialResidual line 44 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/interface/itres.c<br>
> [0]PETSC ERROR: [0] KSPSolve_GMRES line 225 /home/bkhanal/Documents/softwares/petsc-3.6.3/src/ksp/ksp/impls/gmres/gmres.c<br>
> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>
> [0]PETSC ERROR: Signal received<br>
> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
> [0]PETSC ERROR: Petsc Release Version 3.6.3, Dec, 03, 2015<br>
> [0]PETSC ERROR: /user/bkhanal/home/works/AdLemModel/build/src/AdLemMain on a arch-linux2-cxx-debug named delorme by bkhanal Mon Dec 7 23:13:00 2015<br>
> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-scalapack --download-mpich --download-superlu_dist --download-mumps --download-hypre --download-metis --download-parmetis<br>
> [0]PETSC ERROR: #40 User provided function() line 0 in unknown file<br>
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0<br>
> [unset]: aborting job:<br>
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0<br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div></div>