<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 01/14/14 16:41, Karl Rupp wrote:<br>
</div>
<blockquote cite="mid:52D55AB8.9060301@mcs.anl.gov" type="cite">Hi
Pierre,
<br>
<br>
> I try running in parallel the ex19 test case on CPU and GPU:
<br>
<blockquote type="cite">
<br>
OPT="_opt"
<br>
dir=/ccc/cont002/home/den/triou/Version_test_airain-hybrid/Trio_U/lib/src/LIBPETSC/petsc/linux$OPT/src/snes/examples/tutorials
<br>
<br>
option="-pc_type none -ksp_type fgmres -snes_monitor_short
-snes_rtol
<br>
1.e-5 -log_summary -ksp_view -cuda_show_devices"
<br>
mpirun -np 2 $dir/ex19 $option 1>cpu$OPT.log 2>&1
<br>
mpirun -np 2 $dir/ex19 -dm_vec_type cusp -dm_mat_type aijcusp
$option
<br>
1>gpu$OPT.log 2>&1
<br>
<br>
With OPT="", PETSc optimized library is used, parallel
calculation runs
<br>
well on CPU and GPU.
<br>
With OPT="_opt", PETSc non optimized library is used, parallel
<br>
calculation crashes on GPU (it is OK on CPU).
<br>
<br>
I join the log files. The only difference seems that PETSc-dev
is built
<br>
with -O3 intead of -g...
<br>
I could try to rebuild PETSc with -O2 but do you have any idea
of the
<br>
problem ?
<br>
</blockquote>
<br>
I could reproduce the problem and also get some uninitialized
variable warnings in Valgrind. The debug version detects these
errors, hence you only see the errors in the debug build. For the
optimized build, chances are good that the computed values are
either wrong or may become wrong in other environments. I'll see
what I can do when I'm again at GPU machine tomorrow (parallel GPU
debugging via SSH is not great...)
<br>
</blockquote>
Sorry, I mean:<br>
<br>
Parallel calculation on CPU or GPU run well with PETSc non optimized
library<br>
Parallel calculation on GPU crashes with PETSc optimized library (on
CPU it is OK)<br>
<br>
I could add that the "mpirun -np 1 ex19" runs well for all builds on
CPU and GPU.<br>
<br>
Pierre<br>
<blockquote cite="mid:52D55AB8.9060301@mcs.anl.gov" type="cite">
<br>
Best regards,
<br>
Karli
<br>
<br>
<br>
</blockquote>
<br>
<br>
<div class="moz-signature">-- <br>
<b>Trio_U support team</b>
<br>
Marthe ROUX (Saclay)
<br>
Pierre LEDAC (Grenoble)
</div>
</body>
</html>