<div dir="ltr">That's what happens when you try to VecAXPY/MAXPY/whatever with a NaN or inf.  We should really have a better error message for this.<div><br></div><div>- Peter</div></div><div class="gmail_extra"><br><br>
<div class="gmail_quote">On Fri, Mar 28, 2014 at 7:50 AM, Mark Adams <span dir="ltr"><<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div>I get this error in the middle of a GMRES iteration.  Any ideas?</div><div><br></div><div>  0 SNES Function norm 6.067786990073e+17                                                                                                                                </div>

<div>    Residual norms for fsa_ solve.                                                                                                                                       </div><div>    0 KSP Residual norm 6.067786990073e+17                                                                                                                               </div>

<div>    1 KSP Residual norm 6.037192129591e+17                                                                                                                               </div><div>    2 KSP Residual norm 5.852390318109e+17                                                                                                                               </div>

<div>    3 KSP Residual norm 5.562063100790e+17                                                                                                                               </div><div>    4 KSP Residual norm 5.284787094948e+17                                                                                                                               </div>

<div>    5 KSP Residual norm 5.183753281867e+17                                                                                                                               </div><div>    6 KSP Residual norm 4.813052731918e+17                                                                                                                               </div>

<div>    7 KSP Residual norm 4.571658388437e+17                                                                                                                               </div><div>    8 KSP Residual norm 4.326765203018e+17                                                                                                                               </div>

<div>    9 KSP Residual norm 4.044786063878e+17                                                                                                                               </div><div>   10 KSP Residual norm 3.852338418686e+17                                                                                                                               </div>

<div>   11 KSP Residual norm 3.533696733608e+17                                                                                                                               </div><div>   12 KSP Residual norm 3.242503119640e+17                                                                                                                               </div>

<div>   13 KSP Residual norm 2.946010215248e+17                                                                                                                               </div><div>   14 KSP Residual norm 2.642699328492e+17                                                                                                                               </div>

<div>   15 KSP Residual norm 2.365537803535e+17                                                                                                                               </div><div>   16 KSP Residual norm 1.841438363878e+17                                                                                                                               </div>

<div>   17 KSP Residual norm 1.333683638359e+17                                                                                                                               </div><div>   18 KSP Residual norm 7.325375319129e+16                                                                                                                               </div>

<div>   19 KSP Residual norm 2.964254455676e+16                                                                                                                               </div><div>   20 KSP Residual norm 1.216650547542e+16                    [3]PETSC ERROR: --------------------- Error Message ------------------------------------                                                                                 <br>

</div><div>[3]PETSC ERROR: Invalid argument!                                                                                                                                        </div><div>[3]PETSC ERROR: Scalar value must be same on all processes, argument # 3!                                                                                                </div>

<div>[3]PETSC ERROR: ------------------------------------------------------------------------                                                                                 </div><div>[3]PETSC ERROR: Petsc Development GIT revision: a9e422f38a085385bc6ffdad704c9d6a22342ac9  GIT Date: 2013-12-31 01:26:24 -0700                                            </div>

<div>[3]PETSC ERROR: See docs/changes/index.html for recent updates.                                                                                                          </div><div>[3]PETSC ERROR: See docs/faq.html for hints about trouble shooting.                                                                                                      </div>

<div>[3]PETSC ERROR: See docs/index.html for manual pages.                                                                                                                    </div><div>[3]PETSC ERROR: ------------------------------------------------------------------------                                                                                 </div>

<div>[3]PETSC ERROR: ../../XGC1_3/xgc2 on a arch-xc30-dbg named nid04306 by madams Fri Mar 28 05:34:54 2014                                                                   </div><div>[3]PETSC ERROR: Libraries linked from /global/homes/m/madams/petsc_private/arch-xc30-dbg/lib                                                                             </div>

<div>[3]PETSC ERROR: Configure run at Tue Dec 31 14:53:42 2013                                                                                                                </div><div>[3]PETSC ERROR: Configure options --COPTFLAGS="-g -no-ipo" --CXXOPTFLAGS="-g -no-ipo" --FOPTFLAGS="-g -no-ipo" --download-hypre --download-metis --download-parmetis --with-cc=cc --with-clib-autodetect=0 --with-cxx=CC --with-cxxlib-autodetect=0 --with-debugging=1 --with-fc=ftn --with-fortranlib-autodetect=0 --with-hdf5-dir=/opt/cray/hdf5-parallel/1.8.9/intel/120/ --with-shared-libraries=0 --with-x=0 LIBS=-lstdc++ PETSC_ARCH=arch-xc30-dbg                                                                   </div>

<div>[3]PETSC ERROR: ------------------------------------------------------------------------                                                                                 </div><div>[3]PETSC ERROR: VecMAXPY() line 1257 in /global/u2/m/madams/petsc_private/src/vec/vec/interface/rvector.c                                                                </div>

<div>[3]PETSC ERROR: KSPGMRESBuildSoln() line 353 in /global/u2/m/madams/petsc_private/src/ksp/ksp/impls/gmres/gmres.c                                                        </div><div>[3]PETSC ERROR: KSPGMRESCycle() line 211 in /global/u2/m/madams/petsc_private/src/ksp/ksp/impls/gmres/gmres.c                                                            </div>

<div>[3]PETSC ERROR: KSPSolve_GMRES() line 235 in /global/u2/m/madams/petsc_private/src/ksp/ksp/impls/gmres/gmres.c                                                           </div><div>[3]PETSC ERROR: KSPSolve() line 432 in /global/u2/m/madams/petsc_private/src/ksp/ksp/interface/itfunc.c                                                                  </div>

<div>[3]PETSC ERROR: SNESSolve_KSPONLY() line 44 in /global/u2/m/madams/petsc_private/src/snes/impls/ksponly/ksponly.c                                                        </div><div>[3]PETSC ERROR: SNESSolve() line 3812 in /global/u2/m/madams/petsc_private/src/snes/interface/snes.c                                                                     </div>

<div>Rank 3 [Fri Mar 28 05:35:04 2014] [c6-2c1s4n2] application called MPI_Abort(MPI_COMM_WORLD, 62) - process 3             </div></div>
</blockquote></div><br></div>