<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I have done some more testing of the problem, continuing with
src/ksp/ksp/examples/tutorials/ex2.c.<br>
<br>
The behavior I am seeing is that with smaller problems sizes
superlu_dist is behaving properly<br>
but with larger problem sizes things seem to go wrong and what goes
wrong is apparently consistent; the error appears both with my intel
build as well as with my gcc build.<br>
<br>
I have two run lines:<br>
<br>
runex2superlu:<br>
-@${MPIEXEC} -n 2 ./ex2 -ksp_monitor_short -m 100 -n 100
-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package
superlu_dist<br>
<br>
runex2spooles:<br>
-@${MPIEXEC} -n 2 ./ex2 -ksp_monitor_short -m 100 -n 100
-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package spooles<br>
<br>
From my intel build, I get<br>
<br>
sg-macbook-prolocal:tutorials sg$ make runex2superlu<br>
Norm of error 7.66145e-13 iterations 1<br>
sg-macbook-prolocal:tutorials sg$ make runex2spooles<br>
Norm of error 2.21422e-12 iterations 1<br>
<br>
From my GCC build, I get<br>
sg-macbook-prolocal:tutorials sg$ make runex2superlu<br>
Norm of error 7.66145e-13 iterations 1<br>
sg-macbook-prolocal:tutorials sg$ make runex2spooles<br>
Norm of error 2.21422e-12 iterations 1<br>
<br>
If I change the -m 100 -n 100 to -m 500 -n 500, I get for my intel
build<br>
<br>
sg-macbook-prolocal:tutorials sg$ make runex2superlu<br>
Norm of error 419.953 iterations 1<br>
sg-macbook-prolocal:tutorials sg$ make runex2spooles<br>
Norm of error 2.69468e-10 iterations 1<br>
<br>
From my GCC build with -m 500 -n 500, I get<br>
<br>
sg-macbook-prolocal:tutorials sg$ make runex2superlu<br>
Norm of error 419.953 iterations 1<br>
sg-macbook-prolocal:tutorials sg$ make runex2spooles<br>
Norm of error 2.69468e-10 iterations 1<br>
<br>
<br>
Any suggestions will be greatly appreciated.<br>
<br>
-sanjay<br>
<br>
<br>
<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 12/23/12 6:42 PM, Matthew Knepley
wrote:<br>
</div>
<blockquote
cite="mid:CAMYG4GmW7FfEqvV6YEPygXFjpxjD2sDA9fkroAJ+DXXbjiW9nw@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra">On Sun, Dec 23, 2012 at 9:37 PM, Sanjay
Govindjee <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:s_g@berkeley.edu" target="_blank">s_g@berkeley.edu</a>></span>
wrote:<br>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> I decided to go
with ksp/ksp/exampeles/tutorials/ex2.c; I was unsure how
to convert the run lines for snes/examples/ex5.c to work
with a direct solver as I am not versed in SNES options.<br>
<br>
Notwithstanding something strange is happening only on
select examples. With ksp/ksp/exampeles/tutorials/ex2.c
and the run line:<br>
<br>
-@${MPIEXEC} -n 2 ./ex2 -ksp_monitor_short -m 20 -n 20
-ksp_type preonly -pc_type lu
-pc_factor_mat_solver_package superlu_dist<br>
<br>
I get good results (of the order):<br>
<br>
Norm of error 1.85464e-14 iterations 1<br>
<br>
using both superlu_dist and spooles.<br>
<br>
My BLAS/LAPACK: -llapack -lblas (so native to my
machine).<br>
<br>
If you can guide me on a run line for the snes ex5.c I
can try that too. I'll also try to construct a GCC
build later to see if that is an issue.<br>
</div>
</blockquote>
<div><br>
</div>
<div style="">Same line on ex5, but ex2 is good enough.
However, it will not tell us anything new. Try another
build.</div>
<div style=""><br>
</div>
<div style=""> Matt</div>
<div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> -sanjay<br>
<br>
<br>
<div>On 12/23/12 5:58 PM, Matthew Knepley wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">On Sun, Dec 23, 2012 at 8:08 PM, Sanjay
Govindjee <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:s_g@berkeley.edu" target="_blank">s_g@berkeley.edu</a>></span>
wrote:<br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Not
sure what you mean by where is your matrix?
I am simply running ex6 in the
ksp/examples/tests directory.<br>
<br>
The reason I ran this test is because I was
seeing the same behavior with my finite
element code (on perfectly benign problems).<br>
<br>
Is there a built-in test that you use to
check that superlu_dist is working properly
with petsc?<br>
i.e. something you know that works with with
petsc 3.3-p5?<br>
</div>
</blockquote>
<div><br>
</div>
<div>1) Run it on a SNES ex5 (or KSP ex2), which
is a nice Laplacian</div>
<div><br>
</div>
<div>2) Compare with MUMPS</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> -sanjay
<div>
<div><br>
<br>
<br>
<div>On 12/23/12 4:56 PM, Jed Brown
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Where is your matrix?
It might be ending up with a very
bad pivot. If the problem can be
reproduced, it should be reported to
the SuperLU_DIST developers to fix.
(Note that we do not see this with
other matrices.) You can also try
MUMPS.</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Sun, Dec
23, 2012 at 6:48 PM, Sanjay
Govindjee <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:s_g@berkeley.edu"
target="_blank">s_g@berkeley.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000"> I wanted to
use SuperLU Dist to perform a
direct solve but seem to be
encountering<br>
a problem. I was wonder if
this is a know issue and if
there is a solution for it.<br>
<br>
The problem is easily observed
using ex6.c in
src/ksp/ksp/examples/tests.<br>
<br>
Out of the box: make runex6
produces a residual error of
O(1e-11), all is well.<br>
<br>
I then changed the run to run
on two processors and add the
flag<br>
-pc_factor_mat_solver_package
spooles this produces a
residual error of O(1e-11),
all is still well.<br>
<br>
I then switch over to
-pc_factor_mat_solver_package
superlu_dist and the<br>
residual error comes back as
22.6637! Something seems very
wrong.<br>
<br>
My build is perfectly vanilla:<br>
<br>
export
PETSC_DIR=/Users/sg/petsc-3.3-p5/<br>
export PETSC_ARCH=intel<br>
<br>
./configure --with-cc=icc
--with-fc=ifort \
-download-{spooles,parmetis,superlu_dist,prometheus,mpich,ml,hypre,metis}<br>
<br>
make
PETSC_DIR=/Users/sg/petsc-3.3-p5/
PETSC_ARCH=intel all<br>
make
PETSC_DIR=/Users/sg/petsc-3.3-p5/
PETSC_ARCH=intel test<span><font
color="#888888"><br>
<br>
-sanjay<br>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
<pre cols="72">--
-----------------------------------------------
Sanjay Govindjee, PhD, PE
Professor of Civil Engineering
Vice Chair for Academic Affairs
779 Davis Hall
Structural Engineering, Mechanics and Materials
Department of Civil Engineering
University of California
Berkeley, CA 94720-1710
Voice: <a moz-do-not-send="true" href="tel:%2B1%20510%20642%206060" value="+15106426060" target="_blank">+1 510 642 6060</a>
FAX: <a moz-do-not-send="true" href="tel:%2B1%20510%20643%205264" value="+15106435264" target="_blank">+1 510 643 5264</a>
<a moz-do-not-send="true" href="mailto:s_g@berkeley.edu" target="_blank">s_g@berkeley.edu</a>
<a moz-do-not-send="true" href="http://www.ce.berkeley.edu/%7Esanjay" target="_blank">http://www.ce.berkeley.edu/~sanjay</a>
-----------------------------------------------
New Books:
Engineering Mechanics of Deformable
Solids: A Presentation with Exercises
<a moz-do-not-send="true" href="http://www.oup.com/us/catalog/general/subject/Physics/MaterialsScience/?view=usa&ci=9780199651641" target="_blank">http://www.oup.com/us/catalog/general/subject/Physics/MaterialsScience/?view=usa&ci=9780199651641</a>
<a moz-do-not-send="true" href="http://ukcatalogue.oup.com/product/9780199651641.do" target="_blank">http://ukcatalogue.oup.com/product/9780199651641.do</a>
<a moz-do-not-send="true" href="http://amzn.com/0199651647" target="_blank">http://amzn.com/0199651647</a>
Engineering Mechanics 3 (Dynamics)
<a moz-do-not-send="true" href="http://www.springer.com/materials/mechanics/book/978-3-642-14018-1" target="_blank">http://www.springer.com/materials/mechanics/book/978-3-642-14018-1</a>
<a moz-do-not-send="true" href="http://amzn.com/3642140181" target="_blank">http://amzn.com/3642140181</a>
-----------------------------------------------
</pre>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<span class="HOEnZb"><font color="#888888">
<div><br>
</div>
-- <br>
What most experimenters take for granted
before they begin their experiments is
infinitely more interesting than any results
to which their experiments lead.<br>
-- Norbert Wiener </font></span></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
What most experimenters take for granted before they begin
their experiments is infinitely more interesting than any
results to which their experiments lead.<br>
-- Norbert Wiener
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">
</pre>
</body>
</html>