<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<br>
<div class="moz-cite-prefix">On 15-04-25 06:26 PM, Matthew Knepley
wrote:<br>
</div>
<blockquote
cite="mid:CAMYG4GkZ645gzME9m9-0R+Oz7PZJvVxo4qhiAfUgeUcSrrMyPQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Sat, Apr 25, 2015 at 8:23 PM,
Danyang Su <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:danyang.su@gmail.com" target="_blank">danyang.su@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
On 15-04-25 06:03 PM, Barry Smith wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
If this is what you got in your last run<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
at ../../gas_advection/velocity_g.F90:1344<br>
1344 cinfrt =
cinfrt_dg(i1) * diff(ic,idim)
!diff is a very small value, e.g.,
1.0d-316<br>
</blockquote>
then it is still catching floating point underflow,
which we do not want. This means either the change I
suggested you make in the fp.c code didn't work or it
actually uses a different floating point trap than that
one. BTW: absurd numbers like 1.0d-316 are often a
symptom of uninitialized data; could that be a problem
that diff is not filled correctly for all the ic, idim
you are using?<br>
<br>
This going round and round is very frustrating and a
waste of time. You need to be more proactive yourself
and explore the code and poke around to figure out how
to solve the problem.<br>
<br>
Please email
$PETSC_DIR/$PETSC_ARCH/include/petscvariables.h so I can
see what FP trap is being used on your machine.<br>
<br>
Barry<br>
</blockquote>
Do you mean $PETSC_DIR/$PETSC_ARCH/conf/petscvariables?
Otherwise I cannot find this file.<br>
</blockquote>
<div><br>
</div>
<div>Its include/petscconf.h</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Do I need to reconfigure PETSc after changing the code you
mentioned?<br>
</blockquote>
<div><br>
</div>
<div>No, but you need to rebuild.</div>
</div>
</div>
</div>
</blockquote>
Yes, I have done 'gnumake'.<br>
<blockquote
cite="mid:CAMYG4GkZ645gzME9m9-0R+Oz7PZJvVxo4qhiAfUgeUcSrrMyPQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Danyang<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
On Apr 25, 2015, at 2:24 PM, Danyang Su <<a
moz-do-not-send="true"
href="mailto:danyang.su@gmail.com" target="_blank">danyang.su@gmail.com</a>>
wrote:<br>
<br>
<br>
<br>
On 15-04-25 11:55 AM, Barry Smith wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
On Apr 25, 2015, at 1:51 PM, Danyang Su <<a
moz-do-not-send="true"
href="mailto:danyang.su@gmail.com"
target="_blank">danyang.su@gmail.com</a>><br>
wrote:<br>
<br>
<br>
<br>
On 15-04-25 11:32 AM, Barry Smith wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
I told you this yesterday.<br>
<br>
It is probably stopping here on a harmless
underflow. You need to edit the PETSc code to
not worry about underflow.<br>
<br>
Edit the file
/home/dsu/Soft/PETSc/petsc-3.5.2/src/sys/error/fp.c
and locate<br>
<br>
#elif defined PETSC_HAVE_XMMINTRIN_H<br>
_MM_SET_EXCEPTION_MASK(_MM_MASK_INEXACT);<br>
#else<br>
<br>
change it to<br>
<br>
#elif defined PETSC_HAVE_XMMINTRIN_H<br>
_MM_SET_EXCEPTION_MASK(_MM_MASK_INEXACT |
_MM_MASK_UNDERFLOW);<br>
#else<br>
<br>
Then run make gnumake in the PETSc directory
to compile the new version. Now link and run the
program again with -fp_trap and see where it
gets stuck this time.<br>
<br>
Did you do this?<br>
<br>
Barry<br>
<br>
</blockquote>
Yes, I did change the code in fp.c and run 'make
gnumake' in the PETSc directory. I just did a
double check and ran make gnumake again and got
the following information this time.<br>
<br>
<br>
<a class="moz-txt-link-abbreviated" href="mailto:dsu@nwmop:~/Soft/PETSc/petsc-3.5.2$">dsu@nwmop:~/Soft/PETSc/petsc-3.5.2$</a><br>
make gnumake<br>
Building PETSc using GNU Make with 10 build
threads<br>
==========================================<br>
make[1]: Entering directory
`/home/dsu/Soft/PETSc/petsc-3.5.2'<br>
make[1]: Nothing to be done for `all'.<br>
make[1]: Leaving directory
`/home/dsu/Soft/PETSc/petsc-3.5.2'<br>
=========================================<br>
<br>
<br>
Then I recompiled the codes, ran with -fp_trap and
still got the following error<br>
<br>
Backtrace for this error:<br>
Note: The EXACT line numbers in the stack are not
available,<br>
[2]PETSC ERROR: INSTEAD the line number of
the start of the function<br>
[2]PETSC ERROR: is given.<br>
[2]PETSC ERROR: [2] PetscDefaultFPTrap line 379
/home/dsu/Soft/PETSc/petsc-3.5.2/src/sys/error/fp.c<br>
INSTEAD the line number of the start of the
function<br>
[3]PETSC ERROR: is given.<br>
[3]PETSC ERROR: [3] PetscDefaultFPTrap line 379
/home/dsu/Soft/PETSc/petsc-3.5.2/src/sys/error/fp.c<br>
[2]PETSC ERROR: User provided function() line 0 in
Unknown file trapped floating point error<br>
[3]PETSC ERROR: User provided function() line 0 in
Unknown file trapped floating point error<br>
<br>
<br>
</blockquote>
This is different then what you sent a few
minutes ago where it crashed in hypre.<br>
<br>
Anyways you need to use the -start_in_debugger
business I sent in the previous email to see the
exact place the problem occurs.<br>
<br>
</blockquote>
Here is the information shown on gdb screen<br>
<br>
Program received signal SIGFPE, Arithmetic exception.<br>
0x00000000006c2bef in velocity_g (l_sufx=1,
suffix=..., nmax=12, njamxc=34,<br>
cinfradx=..., radial_coordx=.FALSE., _suffix=3)<br>
at ../../gas_advection/velocity_g.F90:1344<br>
1344 cinfrt =
cinfrt_dg(i1) * diff(ic,idim)
!diff is a very small value, e.g.,
1.0d-316<br>
(gdb)<br>
<br>
After type cont on gdb screen, I got error information
as below<br>
<br>
[1]PETSC ERROR: *** unknown floating point error
occurred ***<br>
[1]PETSC ERROR: The specific exception can be
determined by running in a debugger. When the<br>
[1]PETSC ERROR: debugger traps the signal, the
exception can be found with fetestexcept(0x3d)<br>
[1]PETSC ERROR: where the result is a bitwise OR of
the following flags:<br>
[1]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4
FE_OVERFLOW=0x8 FE_UNDERFLOW=0x10 FE_INEXACT=0x20<br>
[1]PETSC ERROR: Try option -start_in_debugger<br>
[1]PETSC ERROR: likely location of problem given in
stack below<br>
[1]PETSC ERROR: --------------------- Stack Frames
------------------------------------<br>
[1]PETSC ERROR: Note: The EXACT line numbers in the
stack are not available,<br>
[1]PETSC ERROR: INSTEAD the line number of the
start of the function<br>
[1]PETSC ERROR: is given.<br>
[1]PETSC ERROR: [1] PetscDefaultFPTrap line 379
/home/dsu/Soft/PETSc/petsc-3.5.2/src/sys/error/fp.c<br>
[1]PETSC ERROR: User provided function() line 0 in
Unknown file trapped floating point error<br>
[0]PETSC ERROR: *** unknown floating point error
occurred ***<br>
[0]PETSC ERROR: The specific exception can be
determined by running in a debugger. When the<br>
[0]PETSC ERROR: debugger traps the signal, the
exception can be found with fetestexcept(0x3d)<br>
[0]PETSC ERROR: where the result is a bitwise OR of
the following flags:<br>
[0]PETSC ERROR: FE_INVALID=0x1 FE_DIVBYZERO=0x4
FE_OVERFLOW=0x8 FE_UNDERFLOW=0x10 FE_INEXACT=0x20<br>
[0]PETSC ERROR: Try option -start_in_debugger<br>
[0]PETSC ERROR: likely location of problem given in
stack below<br>
<br>
Thanks,<br>
<br>
Danyang<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks,<br>
<br>
Danyang<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
On Apr 25, 2015, at 1:05 AM, Danyang Su <<a
moz-do-not-send="true"
href="mailto:danyang.su@gmail.com"
target="_blank">danyang.su@gmail.com</a>><br>
wrote:<br>
<br>
Hi Barry and Satish,<br>
<br>
How can I get rid of unknown floating point
error when a very small value is multiplied.<br>
<br>
e.g.,<br>
cinfrt_dg(i1) and diff(ic,idim) are
1.0250235986806329E-008
8.6178408169776945E-317 respectively,<br>
<br>
cinfrt = cinfrt_dg(i1) * diff(ic,idim)<br>
<br>
I get the following error when run with
"-fp_trap -start_in_debugger".<br>
<br>
Backtrace for this error:<br>
*** unknown floating point error occurred ***<br>
[2]PETSC ERROR: The specific exception can be
determined by running in a debugger. When the<br>
[2]PETSC ERROR: debugger traps the signal, the
exception can be found with fetestexcept(0x3d)<br>
[2]PETSC ERROR: cinfrt_dg(i1),diff(ic,idim)
1.0250235986806329E-008
8.6178408169776945E-317<br>
where the result is a bitwise OR of the
following flags:<br>
[2]PETSC ERROR: FE_INVALID=0x1
FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8
FE_UNDERFLOW=0x10 FE_INEXACT=0x20<br>
[2]PETSC ERROR: Try option -start_in_debugger<br>
[2]PETSC ERROR: likely location of problem
given in stack below<br>
<br>
Thanks,<br>
<br>
Danyang<br>
<br>
On 15-04-24 01:54 PM, Danyang Su wrote:<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
On 15-04-24 01:23 PM, Satish Balay wrote:<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
c 4 1.0976214263087059E-067<br>
<br>
I don't think this number can be stored in
a real*4.<br>
<br>
Satish<br>
<br>
</blockquote>
Thanks, Satish. It is caused by this number.<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
On Fri, 24 Apr 2015, Danyang Su wrote:<br>
<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
On 15-04-24 11:12 AM, Barry Smith wrote:<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
On Apr 24, 2015, at 1:05 PM, Danyang
Su <<a moz-do-not-send="true"
href="mailto:danyang.su@gmail.com"
target="_blank">danyang.su@gmail.com</a>><br>
wrote:<br>
<br>
Hi All,<br>
<br>
One of my case crashes because of
floating point exception when using
4<br>
processors, as shown below. But if I
run this case with 1 processor, it<br>
works fine. I have tested the codes
with around 100 cases up to 768<br>
processors, all other cases work
fine. I just wonder if this kind of
error<br>
is caused because of NaN in jacobi
matrix, RHS or preconditioner?<br>
<br>
</blockquote>
Yes, almost for sure it is one of
these places.<br>
<br>
First run the bad case with
-fp_trap if all goes well you'll see
the<br>
function where the FPE is generated.
Then run also with -start_in_debugger<br>
and<br>
type cont in all four debugger
windows. When the FPE happens the
debugger<br>
should stop showing exactly where the
FPE happens.<br>
<br>
Barry<br>
<br>
</blockquote>
Hi Barry,<br>
<br>
If run with -fp_trap -start_in_debugger,
I got the following error<br>
<br>
[0]PETSC ERROR: *** unknown floating
point error occurred ***<br>
[0]PETSC ERROR: The specific exception
can be determined by running in a<br>
debugger. When the<br>
[0]PETSC ERROR: debugger traps the
signal, the exception can be found with<br>
fetestexcept(0x3d)<br>
[0]PETSC ERROR: where the result is a
bitwise OR of the following flags:<br>
[0]PETSC ERROR: FE_INVALID=0x1
FE_DIVBYZERO=0x4 FE_OVERFLOW=0x8<br>
FE_UNDERFLOW=0x10 FE_INEXACT=0x20<br>
[0]PETSC ERROR: Try option
-start_in_debugger<br>
[0]PETSC ERROR: likely location of
problem given in stack below<br>
[0]PETSC ERROR: ---------------------
Stack Frames<br>
------------------------------------<br>
[0]PETSC ERROR: Note: The EXACT line
numbers in the stack are not available,<br>
[0]PETSC ERROR: INSTEAD the line
number of the start of the function<br>
[0]PETSC ERROR: is given.<br>
[0]PETSC ERROR: [0] PetscDefaultFPTrap
line 379<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/sys/error/fp.c<br>
[0]PETSC ERROR: User provided function()
line 0 in Unknown file trapped<br>
floating point error<br>
<br>
Program received signal SIGABRT: Process
abort signal.<br>
<br>
Backtrace for this error:<br>
#0 0x7F4FEAB1C7D7<br>
#1 0x7F4FEAB1CDDE<br>
#2 0x7F4FE9E1AD3F<br>
#3 0x7F4FE9E1ACC9<br>
#4 0x7F4FE9E1E0D7<br>
#5 0x7F4FEB0B6DCB<br>
#6 0x7F4FEB0B1825<br>
#7 0x7F4FEB0B817F<br>
#8 0x7F4FE9E1AD3F<br>
#9 0x6972C8 in tprfrtlc_ at
tprfrtlc.F90:2393 (discriminator 3)<br>
#10 0x4C6C87 in gcreact_ at
gcreact.F90:678<br>
#11 0x707E19 in initicrt_ at
initicrt.F90:589<br>
#12 0x4F42D0 in initprob_ at
initprob.F90:430<br>
#13 0x5AAF72 in driver_pc at
driver_pc.F90:438<br>
<br>
I checked the code at
tprfrtlc.F90:2393,<br>
<br>
realbuffer_gb(1:nvars) =
(/time,(c(ic),ic=1,nc-1), &<br>
(cx(ix),ix=1,nxout)/)<br>
<br>
All the values (time, c, cx) are
reasonable, as shown below. The only<br>
possibility is that realbuffer_gb is in
declared as real*4 if using sing<br>
precision output while time, c, cx are
declared in real*8. I have a lot of<br>
similar data conversion from real*8 to
real*4 output, other code does not<br>
return error.<br>
<br>
time 0.0000000000000000<br>
c 1
9.9999999999999995E-008<br>
c 2
3.1555251077549618E-003<br>
c 3
7.1657814842179362E-008<br>
c 4
1.0976214263087059E-067<br>
c 5
5.2879822292305797E-004<br>
c 6
9.9999999999999964E-005<br>
c 7
6.4055731968811337E-005<br>
c 8
3.4607572892578404E-020<br>
cx 1
3.4376650636008101E-005<br>
cx 2
7.3989678854017763E-012<br>
cx 3
9.5317170613607207E-012<br>
cx 4
2.2344525794718353E-015<br>
cx 5
3.0624685689695889E-008<br>
cx 6
1.0046157902783967E-007<br>
cx 7
1.5320169154914984E-004<br>
cx 8
8.6930292776346176E-014<br>
cx 9
3.5944267559348721E-005<br>
cx 10
3.0072645866951157E-018<br>
cx 11
2.3592486321095017E-013<br>
<br>
Thanks,<br>
<br>
Danyang<br>
<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
I can check all the entries of
jacobi matrix to see if the value is
valid,<br>
but this seems not a good idea as it
takes a long time to reach this<br>
point. If I restart the simulation
from a specified time (e.g., 7.685
in<br>
this case), then the error does not
occur.<br>
<br>
Would you please give me any
suggestion on debugging this case?<br>
<br>
Thanks and Regards,<br>
<br>
Danyang<br>
<br>
<br>
timestep: 2730 time: 7.665E+00
years delt: 1.000E-02 years iter:
1<br>
timestep: max.sia: 0.000E+00
tol.sia: 0.000E+00<br>
timestep: 2731 time: 7.675E+00
years delt: 1.000E-02 years iter:
1<br>
timestep: max.sia: 0.000E+00
tol.sia: 0.000E+00<br>
timestep: 2732 time: 7.685E+00
years delt: 1.000E-02 years iter:
1<br>
timestep: max.sia: 0.000E+00
tol.sia: 0.000E+00<br>
timestep: 2733 time: 7.695E+00
years delt: 1.000E-02 years iter:
1<br>
timestep: max.sia: 0.000E+00
tol.sia: 0.000E+00<br>
timestep: 2734 time: 7.705E+00
years delt: 1.000E-02 years iter:
1<br>
timestep: max.sia: 0.000E+00
tol.sia: 0.000E+00<br>
Reduce time step for reactive
transport<br>
timestep: 2734 time: 7.700E+00
years delt: 5.000E-03 years iter:
1<br>
timestep: max.sia: 0.000E+00
tol.sia: 0.000E+00<br>
Reduce time step for reactive
transport<br>
timestep: 2734 time: 7.697E+00
years delt: 2.500E-03 years iter:
1<br>
timestep: max.sia: 0.000E+00
tol.sia: 0.000E+00<br>
[1]PETSC ERROR:
--------------------- Error Message<br>
--------------------------------------------------------------<br>
[1]PETSC ERROR: Floating point
exception<br>
[2]PETSC ERROR:
--------------------- Error Message<br>
--------------------------------------------------------------<br>
[2]PETSC ERROR: Floating point
exception<br>
[2]PETSC ERROR: Vec entry at local
location 0 is not-a-number or
infinite<br>
at end of function: Parameter number
3<br>
[2]PETSC ERROR: See<br>
<a moz-do-not-send="true"
href="http://www.mcs.anl.gov/petsc/documentation/faq.html"
target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a><br>
<br>
for trouble shooting.<br>
[2]PETSC ERROR: Petsc Release
Version 3.5.2, Sep, 08, 2014<br>
[2]PETSC ERROR: [1]PETSC ERROR: Vec
entry at local location 0 is<br>
not-a-number or infinite at end of
function: Parameter number 3<br>
[1]PETSC ERROR: See<br>
<a moz-do-not-send="true"
href="http://www.mcs.anl.gov/petsc/documentation/faq.html"
target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a><br>
<br>
for trouble shooting.<br>
[1]PETSC ERROR: Petsc Release
Version 3.5.2, Sep, 08, 2014<br>
[1]PETSC ERROR:
../min3p_thcm_petsc_dbg on a
linux-gnu-dbg named nwmop by<br>
dsu Thu Apr 23 15:38:52 2015<br>
[1]PETSC ERROR: Configure options
PETSC_ARCH=linux-gnu-dbg
--with-cc=gcc<br>
--with-cxx=g++ --with-fc=gfortran
--download-fblaslapack
--download-mpich<br>
--download-mumps --download-hypre
--download-superlu_dist
--download-metis<br>
--download-parmetis
--download-scalapack<br>
[1]PETSC ERROR: #1 VecValidValues()
line 34 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c<br>
../min3p_thcm_petsc_dbg on a
linux-gnu-dbg named nwmop by dsu Thu
Apr 23<br>
15:38:52 2015<br>
[2]PETSC ERROR: Configure options
PETSC_ARCH=linux-gnu-dbg
--with-cc=gcc<br>
--with-cxx=g++ --with-fc=gfortran
--download-fblaslapack
--download-mpich<br>
--download-mumps --download-hypre
--download-superlu_dist
--download-metis<br>
--download-parmetis
--download-scalapack<br>
[2]PETSC ERROR: #1 VecValidValues()
line 34 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c<br>
[2]PETSC ERROR: [1]PETSC ERROR: #2
PCApply() line 442 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c<br>
[1]PETSC ERROR: #2 PCApply() line
442 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c<br>
[2]PETSC ERROR: #3 KSP_PCApply()
line 230 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h<br>
#3 KSP_PCApply() line 230 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h<br>
[1]PETSC ERROR: #4
KSPInitialResidual() line 63 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c<br>
[2]PETSC ERROR: #4
KSPInitialResidual() line 63 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c<br>
[1]PETSC ERROR: #5 KSPSolve_GMRES()
line 234 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/ksp/impls/gmres/gmres.c<br>
[2]PETSC ERROR: #5 KSPSolve_GMRES()
line 234 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/ksp/impls/gmres/gmres.c<br>
[2]PETSC ERROR: #6 KSPSolve() line
459 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c<br>
[1]PETSC ERROR: #6 KSPSolve() line
459 in<br>
/home/dsu/Soft/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c<br>
^C[mpiexec@nwmop] Sending Ctrl-C to
processes as requested<br>
[mpiexec@nwmop] Press Ctrl-C again
to force abort<br>
<br>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature">What most experimenters take for
granted before they begin their experiments is infinitely
more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>