<div dir="ltr">GAMG does set PREONLY but it should be overridable. Perhaps that has failed in some way.</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jul 24, 2014 at 4:23 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Ok, GAMG does call KSPComputeExtremeSingularValues() so that is likely why it is being called.<br>
<br>
You could run in the debugger -start_in_debugger and see where it detects that error and then go up through the stack frames to see why the error doesn’t cause the program to end (and complete more error message).<br>
<br>
Regarding -mg_coarse_ksp_type richardson not seeming to be changed, it could be that something changes it back to preonly later. You could run in the debugger and put a break point in KSPSetType() and look at each call until you find the ones for the coarse grid and see if one is turning it back to preonly and where and why<br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
Barry<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Jul 24, 2014, at 8:30 AM, John Mousel <<a href="mailto:john.mousel@gmail.com">john.mousel@gmail.com</a>> wrote:<br>
<br>
> Barry,<br>
><br>
> I'm using petsc-dev that I cloned about 2 days ago after you pushed a change in VecSet.<br>
><br>
> There was no extra part of the error message, which confused me as well.<br>
><br>
> -options_left returns:<br>
> -pres_redistribute_gamg_type value: agg<br>
><br>
> John<br>
><br>
><br>
> On Wed, Jul 23, 2014 at 4:11 PM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
><br>
> On Jul 23, 2014, at 9:54 AM, John Mousel <<a href="mailto:john.mousel@gmail.com">john.mousel@gmail.com</a>> wrote:<br>
><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > I'm getting an FPE in LAPACKgesvd of KSPComputeExtremeSingularValues_GMRES.<br>
><br>
> ierr = PetscFPTrapPush(PETSC_FP_TRAP_OFF);CHKERRQ(ierr);<br>
> #if !defined(PETSC_USE_COMPLEX)<br>
> PetscStackCallBLAS("LAPACKgesvd",LAPACKgesvd_("N","N",&bn,&bn,R,&bN,realpart,&sdummy,&idummy,&sdummy,&idummy,work,&lwork,&lierr));<br>
> #else<br>
> PetscStackCallBLAS("LAPACKgesvd",LAPACKgesvd_("N","N",&bn,&bn,R,&bN,realpart,&sdummy,&idummy,&sdummy,&idummy,work,&lwork,realpart+N,&lierr));<br>
> #endif<br>
> if (lierr) SETERRQ1(PETSC_COMM_SELF,PETSC_ERR_LIB,"Error in SVD Lapack routine %d",(int)lierr);<br>
> ierr = PetscFPTrapPop();CHKERRQ(ierr);<br>
><br>
> What version of PETSc are you using? In our current release we turn off the trapping for that routine but maybe yours is a system that cannot turn it off<br>
><br>
> > I'm not sure where this routine is required during the algorithm as I'm using BiCG + richardson/SOR on all multigrid levels.<br>
><br>
> Good question. Yes it would normally not use the KSPComputeExtremeSingularValues_GMRES() Is there more error message that shows where the functions are called from?<br>
><br>
> > Also, even though I've set mg_coarse_ksp_type richardson, KSPView is showing that preonly is used on the coarsest level. Am I misunderstanding something about the options I'm using?<br>
><br>
> They seem to be correct. If you run with -options_left does it indicate that option is never used?<br>
><br>
> Suggest upgrading to PETSc 3.5<br>
><br>
> Barry<br>
><br>
> ><br>
> > -pres_ksp_type preonly -pres_pc_type redistribute -pres_redistribute_ksp_type bcgsl -pres_redistribute_pc_type gamg -pres_redistribute_pc_gamg_threshold 0.01 -pres_redistribute_mg_levels_ksp_type richardson -pres_redistribute_mg_levels_pc_type sor -pres_redistribute_mg_coarse_ksp_type richardson -pres_redistribute_mg_coarse_pc_type sor -pres_redistribute_mg_coarse_pc_sor_its 4 -pres_redistribute_pc_gamg_agg_nsmooths 1 -pres_redistribute_pc_gamg_sym_graph true -pres_redistribute_gamg_type agg -pres_ksp_view<br>
> ><br>
> ><br>
> ><br>
> > [8]PETSC ERROR: ------------------------------------------------------------------------<br>
> > [8]PETSC ERROR: Caught signal number 8 FPE: Floating Point Exception,probably divide by zero<br>
> > [8]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
> > [8]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[8]PETSC" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[8]PETSC</a> ERROR: or try <a href="http://valgrind.org" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors<br>
> > [40]PETSC ERROR: ------------------------------------------------------------------------<br>
> > [40]PETSC ERROR: Caught signal number 8 FPE: Floating Point Exception,probably divide by zero<br>
> > [40]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
> > [40]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[40]PETSC" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[40]PETSC</a> ERROR: or try <a href="http://valgrind.org" target="_blank">http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors<br>
> > [40]PETSC ERROR: likely location of problem given in stack below<br>
> > [40]PETSC ERROR: --------------------- Stack Frames ------------------------------------<br>
> > [40]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,<br>
> > [40]PETSC ERROR: INSTEAD the line number of the start of the function<br>
> > [40]PETSC ERROR: is given.<br>
> > [40]PETSC ERROR: [40] LAPACKgesvd line 42 /Users/jmousel/SOFT/petsc/src/ksp/ksp/impls/gmres/gmreig.c<br>
> > [40]PETSC ERROR: [40] KSPComputeExtremeSingularValues_GMRES line 24 /Users/jmousel/SOFT/petsc/src/ksp/ksp/impls/gmres/gmreig.c<br>
> > [40]PETSC ERROR: [40] KSPComputeExtremeSingularValues line 51 /Users/jmousel/SOFT/petsc/src/ksp/ksp/interface/itfunc.c<br>
> ><br>
> ><br>
> ><br>
> > KSP Object:proj_ksp(pres_) 96 MPI processes<br>
> > type: preonly<br>
> > maximum iterations=1000, initial guess is zero<br>
> > tolerances: relative=1e-50, absolute=0.01, divergence=10000<br>
> > left preconditioning<br>
> > using NONE norm type for convergence test<br>
> > PC Object:(pres_) 96 MPI processes<br>
> > type: redistribute<br>
> > Number rows eliminated 7115519 Percentage rows eliminated 44.1051<br>
> > Redistribute preconditioner:<br>
> > KSP Object: (pres_redistribute_) 96 MPI processes<br>
> > type: bcgsl<br>
> > BCGSL: Ell = 2<br>
> > BCGSL: Delta = 0<br>
> > maximum iterations=1000, initial guess is zero<br>
> > tolerances: relative=1e-50, absolute=0.01, divergence=10000<br>
> > left preconditioning<br>
> > has attached null space<br>
> > using PRECONDITIONED norm type for convergence test<br>
> > PC Object: (pres_redistribute_) 96 MPI processes<br>
> > type: gamg<br>
> > MG: type is MULTIPLICATIVE, levels=4 cycles=v<br>
> > Cycles per PCApply=1<br>
> > Using Galerkin computed coarse grid matrices<br>
> > Coarse grid solver -- level -------------------------------<br>
> > KSP Object: (pres_redistribute_mg_coarse_) 96 MPI processes<br>
> > type: preonly<br>
> > maximum iterations=1, initial guess is zero<br>
> > tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> > left preconditioning<br>
> > using NONE norm type for convergence test<br>
> > PC Object: (pres_redistribute_mg_coarse_) 96 MPI processes<br>
> > type: sor<br>
> > SOR: type = local_symmetric, iterations = 4, local iterations = 1, omega = 1<br>
> > linear system matrix = precond matrix:<br>
> > Mat Object: 96 MPI processes<br>
> > type: mpiaij<br>
> > rows=465, cols=465<br>
> > total: nonzeros=84523, allocated nonzeros=84523<br>
> > total number of mallocs used during MatSetValues calls =0<br>
> > not using I-node (on process 0) routines<br>
> > Down solver (pre-smoother) on level 1 -------------------------------<br>
> > KSP Object: (pres_redistribute_mg_levels_1_) 96 MPI processes<br>
> > type: richardson<br>
> > Richardson: damping factor=1<br>
> > maximum iterations=2<br>
> > tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> > left preconditioning<br>
> > using nonzero initial guess<br>
> > using NONE norm type for convergence test<br>
> > PC Object: (pres_redistribute_mg_levels_1_) 96 MPI processes<br>
> > type: sor<br>
> > SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1<br>
> > linear system matrix = precond matrix:<br>
> > Mat Object: 96 MPI processes<br>
> > type: mpiaij<br>
> > rows=13199, cols=13199<br>
> > total: nonzeros=2.09436e+06, allocated nonzeros=2.09436e+06<br>
> > total number of mallocs used during MatSetValues calls =0<br>
> > not using I-node (on process 0) routines<br>
> > Up solver (post-smoother) same as down solver (pre-smoother)<br>
> > Down solver (pre-smoother) on level 2 -------------------------------<br>
> > KSP Object: (pres_redistribute_mg_levels_2_) 96 MPI processes<br>
> > type: richardson<br>
> > Richardson: damping factor=1<br>
> > maximum iterations=2<br>
> > tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> > left preconditioning<br>
> > using nonzero initial guess<br>
> > using NONE norm type for convergence test<br>
> > PC Object: (pres_redistribute_mg_levels_2_) 96 MPI processes<br>
> > type: sor<br>
> > SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1<br>
> > linear system matrix = precond matrix:<br>
> > Mat Object: 96 MPI processes<br>
> > type: mpiaij<br>
> > rows=568202, cols=568202<br>
> > total: nonzeros=2.33509e+07, allocated nonzeros=2.33509e+07<br>
> > total number of mallocs used during MatSetValues calls =0<br>
> > not using I-node (on process 0) routines<br>
> > Up solver (post-smoother) same as down solver (pre-smoother)<br>
> > Down solver (pre-smoother) on level 3 -------------------------------<br>
> > KSP Object: (pres_redistribute_mg_levels_3_) 96 MPI processes<br>
> > type: richardson<br>
> > Richardson: damping factor=1<br>
> > maximum iterations=2<br>
> > tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> > left preconditioning<br>
> > using nonzero initial guess<br>
> > using NONE norm type for convergence test<br>
> > PC Object: (pres_redistribute_mg_levels_3_) 96 MPI processes<br>
> > type: sor<br>
> > SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1<br>
> > linear system matrix = precond matrix:<br>
> > Mat Object: 96 MPI processes<br>
> > type: mpiaij<br>
> > rows=9017583, cols=9017583<br>
> > total: nonzeros=1.01192e+08, allocated nonzeros=1.01192e+08<br>
> > total number of mallocs used during MatSetValues calls =0<br>
> > not using I-node (on process 0) routines<br>
> > Up solver (post-smoother) same as down solver (pre-smoother)<br>
> > linear system matrix = precond matrix:<br>
> > Mat Object: 96 MPI processes<br>
> > type: mpiaij<br>
> > rows=9017583, cols=9017583<br>
> > total: nonzeros=1.01192e+08, allocated nonzeros=1.01192e+08<br>
> > total number of mallocs used during MatSetValues calls =0<br>
> > not using I-node (on process 0) routines<br>
> > linear system matrix = precond matrix:<br>
> > Mat Object: proj_A 96 MPI processes<br>
> > type: mpiaij<br>
> > rows=16133102, cols=16133102<br>
> > total: nonzeros=1.09807e+08, allocated nonzeros=1.57582e+08<br>
> > total number of mallocs used during MatSetValues calls =0<br>
> > not using I-node (on process 0) routines<br>
> > Residual norms for vel_ solve.<br>
> ><br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div>