If the operating system is not crashing regularly, then it is probably not a hardware failure. Do what I wrote in the last email.<div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Nov 24, 2012 at 7:06 PM, Roc Wang <span dir="ltr"><<a href="mailto:pengxwang@hotmail.com" target="_blank">pengxwang@hotmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr">
Thanks, Jed.<div><br></div><div> It seems the error comes more often when I asked a larger array or Mat than a smaller one. Is it because the outage of memories on the node? Thanks. <br><br><div><div></div><hr>Date: Sat, 24 Nov 2012 18:26:10 +0100<br>
From: <a href="mailto:jedbrown@mcs.anl.gov" target="_blank">jedbrown@mcs.anl.gov</a><br>To: <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>Subject: Re: [petsc-users] PETSC ERROR: Floating point exception!<div>
<div class="h5"><br><br>Lack of reproducibility points its huge foam finger at memory corruption. You can try<div><br></div><div>1. running with -fp_trap to see where the bad values are first computed<div>2. running in valgrind to look for memory corruption<br>
</div></div><div>3. linking to a different BLAS/Lapack to rule out linking problems with MKL</div><div><br><br><div>On Sat, Nov 24, 2012 at 6:17 PM, Roc Wang <span dir="ltr"><<a href="mailto:pengxwang@hotmail.com" target="_blank">pengxwang@hotmail.com</a>></span> wrote:<br>
<blockquote style="border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr">
Hello,<div><br></div><div> I met a wierd problem. I am trying to solve linear system by calling KSPSolve(). But sometimes, the error information of <span style="font-size:10pt">Floating point exception! I tried to track down the error, but u</span><span style="font-size:10pt;white-space:pre-wrap">nfortunately, it's pretty random. S</span><span style="font-size:10pt;white-space:pre-wrap">ometimes the executable runs fine. </span><span style="font-size:10pt;white-space:pre-wrap">Other times it will crash with floating point exceptions as below. I cannot find the exact line causing the error. Could you please give me some suggestions or hints? Thanks a lot!</span></div>
<div><span style="font-size:10pt;white-space:pre-wrap"><br></span></div><div><span style="white-space:pre-wrap">[0]PETSC ERROR: --------------------- Error Message ------------------------------------
[0]PETSC ERROR: Floating point exception!
[0]PETSC ERROR: Infinite or not-a-number generated in norm!
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Petsc Release Version 3.2.0, Patch 7, Thu Mar 15 09:30:51 CDT 2012
[0]PETSC ERROR: See docs/changes/index.html for recent updates.
[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[0]PETSC ERROR: See docs/index.html for manual pages.
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: /gpfs/home/pzw2/work/MLPIC03_3dCart_20121118/a.out on a arch-linu named <a href="http://lionxo64.hpc.aset.psu.edu" target="_blank">lionxo64.hpc.aset.psu.edu</a> by pzw2 Sat Nov 24 11:01:49 2012
[0]PETSC ERROR: Libraries linked from /usr/global/petsc/3.2-p7/lib
[0]PETSC ERROR: Configure run at Thu Apr 5 10:32:29 2012
[0]PETSC ERROR: Configure options --prefix=/usr/global/petsc/3.2-p7/ --with-blas-lapack-dir=/usr/global/intel/mkl/<a href="http://10.2.6.038/lib/em64t/" target="_blank">10.2.6.038/lib/em64t/</a>
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: VecNorm() line 167 in src/vec/vec/interface/rvector.c
[0]PETSC ERROR: VecNormalize() line 261 in src/vec/vec/interface/rvector.c
[0]PETSC ERROR: GMREScycle() line 128 in src/ksp/ksp/impls/gmres/gmres.c
[0]PETSC ERROR: KSPSolve_GMRES() line 231 in src/ksp/ksp/impls/gmres/gmres.c
[0]PETSC ERROR: KSPSolve() line 423 in src/ksp/ksp/interface/itfunc.c</span><span style="font-size:10pt;white-space:pre-wrap">
</span></div><div><span style="font-size:10pt;white-space:pre-wrap"><br></span></div> </div></div>
</blockquote></div><br></div></div></div></div></div> </div></div>
</blockquote></div><br></div>