<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<base href="x-msg://200/">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"MS Mincho";
panose-1:2 2 6 9 4 2 5 8 3 4;}
@font-face
{font-family:"MS Mincho";
panose-1:2 2 6 9 4 2 5 8 3 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"MS Mincho";
panose-1:2 2 6 9 4 2 5 8 3 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:99.25pt 85.05pt 85.05pt 85.05pt;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple style='word-wrap: break-word;
-webkit-nbsp-mode: space;-webkit-line-break: after-white-space'>
<div class=WordSection1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Barry,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Your performance data is identical with mine. Could you repost?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Thanks,<o:p></o:p></span></p>
<div>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:#1F497D'>================================<br>
Keita Teranishi<br>
Scientific Library Group<br>
Cray, Inc.<br>
keita@cray.com<br>
================================</span><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'><o:p></o:p></span></p>
</div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> petsc-dev-bounces@mcs.anl.gov
[mailto:petsc-dev-bounces@mcs.anl.gov] <b>On Behalf Of </b>Barry Smith<br>
<b>Sent:</b> Tuesday, August 31, 2010 1:38 PM<br>
<b>To:</b> For users of the development version of PETSc<br>
<b>Subject:</b> Re: [petsc-dev] [GPU] Performance of ex19<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<p class=MsoNormal> Interesting. Some numbers are worse than our
older system (MAXPY), some are a bit better, nothing is huge amounts better.
Here is the older one<o:p></o:p></p>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<div>
<p class=MsoNormal>VecDot
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0 0
0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecMDot 2024 1.0
1.1760e+00 1.0 2.54e+09 1.0 0.0e+00 0.0e+00 0.0e+00 18 29 0 0
0 32 29 0 0 0 2163<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecNorm 2096 1.0
3.1199e-01 1.0 1.68e+08 1.0 0.0e+00 0.0e+00 0.0e+00 5 2 0
0 0 9 2 0 0 0 537<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecScale 2092 1.0
1.7600e-01 1.0 8.37e+07 1.0 0.0e+00 0.0e+00 0.0e+00 3 1 0
0 0 5 1 0 0 0 475<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecCopy 2072 1.0
9.1996e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0
0 0 3 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecSet
70 1.0 3.9999e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0 0
0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecAXPY 108
1.0 1.5999e-02 1.0 8.64e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0
0 0 0 0 0 0 0 540<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecWAXPY 68
1.0 7.9999e-03 1.0 2.72e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0
0 0 0 0 0 0 0 340<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecMAXPY 2092 1.0
7.0399e-01 1.0 2.71e+09 1.0 0.0e+00 0.0e+00 0.0e+00 11 31 0 0
0 19 31 0 0 0 3844<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecScatterBegin 5 1.0 0.0000e+00
0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecReduceArith 2 1.0 0.0000e+00
0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecReduceComm 1 1.0
0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0
0 0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecCUDACopyTo 10 1.0 0.0000e+00
0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>VecCUDACopyFrom 5 1.0 0.0000e+00
0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>SNESSolve 1
1.0 3.6199e+00 1.0 8.87e+09 1.0 0.0e+00 0.0e+00 0.0e+00 56100 0 0
0 100100 0 0 0 2451<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>SNESLineSearch 2 1.0 7.9999e-03
1.0 5.49e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 687<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>SNESFunctionEval 3 1.0 3.9999e-03 1.0
2.52e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0
0 0 0 0 0 630<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>SNESJacobianEval 2 1.0 3.0399e-01 1.0
3.85e+07 1.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0 0
8 0 0 0 0 127<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>KSPGMRESOrthog 2024 1.0 1.8280e+00 1.0
5.09e+09 1.0 0.0e+00 0.0e+00 0.0e+00 28 57 0 0 0 50 57
0 0 0 2783<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>KSPSetup 2
1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0
0 0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>KSPSolve 2
1.0 3.3079e+00 1.0 8.83e+09 1.0 0.0e+00 0.0e+00 0.0e+00 51 99 0 0
0 91 99 0 0 0 2668<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>PCSetUp
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0
0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>PCApply 2024 1.0
8.7996e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0
0 0 2 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>MatMult 2092 1.0
8.3197e-01 1.0 3.32e+09 1.0 0.0e+00 0.0e+00 0.0e+00 13 37 0 0
0 23 37 0 0 0 3991<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>MatAssemblyBegin 2 1.0 0.0000e+00 0.0
0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0
0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>MatAssemblyEnd 2 1.0 7.9989e-03
1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>MatZeroEntries 2 1.0 4.0002e-03
1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 0<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>MatFDColorApply 2 1.0 3.0399e-01
1.0 3.85e+07 1.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0
0 8 0 0 0 0 127<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>MatFDColorFunc 42 1.0 1.2000e-02
1.0 3.53e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0
0 0 0 0 0 0 2940<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<div>
<p class=MsoNormal>On Aug 31, 2010, at 2:21 PM, Keita Teranishi wrote:<o:p></o:p></p>
</div>
<p class=MsoNormal><br>
<br>
<o:p></o:p></p>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Barry,</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Here it is. The flops rate is better, but the solver is
not multilevel anymore<span class=apple-converted-space> </span></span><span
style='font-size:11.0pt;font-family:Wingdings;color:#1F497D'>L</span><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>.</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Thanks,</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>--- Event Stage 0: Main Stage</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>PetscBarrier
1 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>--- Event Stage 1: SetUp</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatAssemblyBegin 1 1.0
0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatAssemblyEnd 1
1.0 8.0001e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 9 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatFDColorCreate 1 1.0
3.5999e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0
0 0 0 41 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>--- Event Stage 2: Solve</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecDot
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0
0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecMDot
2024 1.0 1.1760e+00 1.0 2.54e+09 1.0 0.0e+00 0.0e+00 0.0e+00 18 29
0 0 0 32 29 0 0 0 2163</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecNorm
2096 1.0 3.1199e-01 1.0 1.68e+08 1.0 0.0e+00 0.0e+00 0.0e+00 5
2 0 0 0 9 2 0 0
0 537</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecScale
2092 1.0 1.7600e-01 1.0 8.37e+07 1.0 0.0e+00 0.0e+00 0.0e+00 3
1 0 0 0 5 1 0 0
0 475</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecCopy
2072 1.0 9.1996e-02 1.0 0.00e+00
0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0
3 0 0 0 0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecSet
70 1.0 3.9999e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecAXPY
108 1.0 1.5999e-02 1.0 8.64e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 540</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecWAXPY
68 1.0 7.9999e-03 1.0 2.72e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 340</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecMAXPY
2092 1.0 7.0399e-01 1.0 2.71e+09 1.0 0.0e+00 0.0e+00 0.0e+00 11 31
0 0 0 19 31 0 0 0 3844</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecScatterBegin 5 1.0
0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecReduceArith 2
1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecReduceComm
1 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecCUDACopyTo 10
1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>VecCUDACopyFrom 5 1.0
0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>SNESSolve
1 1.0 3.6199e+00 1.0 8.87e+09 1.0 0.0e+00 0.0e+00 0.0e+00 56100 0
0 0 100100 0 0 0 2451</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>SNESLineSearch 2
1.0 7.9999e-03 1.0 5.49e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0 687</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>SNESFunctionEval 3 1.0
3.9999e-03 1.0 2.52e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0 630</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>SNESJacobianEval 2 1.0
3.0399e-01 1.0 3.85e+07 1.0 0.0e+00 0.0e+00 0.0e+00 5 0
0 0 0 8 0 0 0 0 127</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>KSPGMRESOrthog 2024 1.0 1.8280e+00
1.0 5.09e+09 1.0 0.0e+00 0.0e+00 0.0e+00 28 57 0 0 0 50
57 0 0 0 2783</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>KSPSetup
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>KSPSolve
2 1.0 3.3079e+00 1.0 8.83e+09 1.0 0.0e+00 0.0e+00 0.0e+00 51 99 0
0 0 91 99 0 0 0 2668</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>PCSetUp
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>PCApply
2024 1.0 8.7996e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1
0 0 0 0 2 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatMult
2092 1.0 8.3197e-01 1.0 3.32e+09 1.0 0.0e+00 0.0e+00 0.0e+00 13 37
0 0 0 23 37 0 0 0 3991</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatAssemblyBegin 2 1.0
0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatAssemblyEnd 2
1.0 7.9989e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatZeroEntries 2
1.0 4.0002e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatFDColorApply 2 1.0
3.0399e-01 1.0 3.85e+07 1.0 0.0e+00 0.0e+00 0.0e+00 5 0
0 0 0 8 0 0 0 0 127</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>MatFDColorFunc 42 1.0
1.2000e-02 1.0 3.53e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0 2940</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>------------------------------------------------------------------------------------------------------------------------</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><o:p></o:p></p>
</div>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in;
border-width:initial;border-color:initial'>
<div>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
class=apple-converted-space><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> </span></span><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'><a
href="mailto:petsc-dev-bounces@mcs.anl.gov">petsc-dev-bounces@mcs.anl.gov</a><span
class=apple-converted-space> </span>[mailto:petsc-dev-bounces@mcs.anl.gov]<span
class=apple-converted-space> </span><b>On Behalf Of<span
class=apple-converted-space> </span></b>Barry Smith<br>
<b>Sent:</b><span class=apple-converted-space> </span>Tuesday, August 31,
2010 10:53 AM<br>
<b>To:</b><span class=apple-converted-space> </span>For users of the
development version of PETSc<br>
<b>Subject:</b><span class=apple-converted-space> </span>Re: [petsc-dev]
[GPU] Performance of ex19</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
</div>
<div>
<p class=MsoNormal> Please run with the options ./ex19
-da_vec_type seqcuda -da_mat_type seqaijcuda -pc_type none -dmmg_nlevels 1
-da_grid_x 100 -da_grid_y 100 -log_summary -mat_no_inode -preload off
-cuda_synchronize<o:p></o:p></p>
</div>
<div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class=MsoNormal>On Aug 31, 2010, at 11:45 AM, Keita Teranishi wrote:<o:p></o:p></p>
</div>
</div>
<div>
<p class=MsoNormal><br>
<br>
<br>
<o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Hi
PETSc Developer team,</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>I
have just measured the performance of ex19 program running on Fermi GPU.
I hope it will help you to develop GPU-enabled PETSc further.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Thanks,</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Keita</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>./ex19
-pc_type jacobi -dmmg_nlevels 5 -da_vec_type cuda -da_mat_type aijcuda
-log_summary -cuda_synchronize</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>---
Event Stage 0: Main Stage</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>PetscBarrier
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0
0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>---
Event Stage 1: SetUp</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecSet
8 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0
0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecCUDACopyFrom
8 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatMultTranspose
4 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 58 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatAssemblyBegin
9 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatAssemblyEnd
9 1.0 3.9999e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 14 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatFDColorCreate
5 1.0 1.2000e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 43 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>---
Event Stage 2: Solve</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecDot
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0
0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecMDot
980 1.0 5.5599e-01 1.0 2.95e+08 1.0 0.0e+00 0.0e+00 0.0e+00 10 14 0
0 0 39 28 0 0 0 530</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecNorm
1025 1.0 1.2399e-01 1.0 1.95e+07 1.0 0.0e+00 0.0e+00 0.0e+00 2
1 0 0 0 9 2 0 0
0 158</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecScale
1013 1.0 9.9998e-02 1.0 9.73e+06 1.0 0.0e+00 0.0e+00 0.0e+00 2
0 0 0 0 7 1 0 0
0 97</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecCopy
208 1.0 3.9999e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecSet
45 1.0 7.9989e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 1 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecAXPY
233 1.0 3.9999e-03 1.0 1.68e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 419</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecWAXPY
33 1.0 3.9990e-03 1.0 3.17e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 79</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecMAXPY
1013 1.0 2.9199e-01 1.0 3.14e+08 1.0 0.0e+00 0.0e+00 0.0e+00 5 15
0 0 0 21 30 0 0 0 1074</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecPointwiseMult
988 1.0 9.5995e-02 1.0 9.42e+06 1.0 0.0e+00 0.0e+00 0.0e+00 2
0 0 0 0 7 1 0 0
0 98</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecScatterBegin
13 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecReduceArith
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecReduceComm
1 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecCUDACopyTo
24 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>VecCUDACopyFrom
21 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatMult
1013 1.0 1.3600e-01 1.0 3.83e+08 1.0 0.0e+00 0.0e+00 0.0e+00 2 18
0 0 0 10 37 0 0 0 2815</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatMultTranspose
8 1.0 3.9999e-03 1.0 1.15e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 29</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatAssemblyBegin
10 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatAssemblyEnd
10 1.0 8.0001e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 1 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatZeroEntries
10 1.0 4.0002e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0
0 0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatFDColorApply
10 1.0 8.7998e-02 1.0 1.26e+07 1.0 0.0e+00 0.0e+00 0.0e+00 2
1 0 0 0 6 1 0 0
0 143</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>MatFDColorFunc
210 1.0 1.2000e-02 1.0 1.15e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0
1 0 0 0 1 1 0 0
0 958</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>SNESSolve
1 1.0 1.4160e+00 1.0 1.04e+09 1.0 0.0e+00 0.0e+00 0.0e+00 25 50 0
0 0 100100 0 0 0 737</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>SNESLineSearch
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>SNESFunctionEval
3 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>SNESJacobianEval
2 1.0 9.1998e-02 1.0 1.27e+07 1.0 0.0e+00 0.0e+00 0.0e+00 2 1
0 0 0 6 1 0 0 0 138</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>KSPGMRESOrthog
980 1.0 8.3199e-01 1.0 5.89e+08 1.0 0.0e+00 0.0e+00 0.0e+00 15 28 0
0 0 59 56 0 0 0 708</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>KSPSetup
2 1.0 0.0000e+00 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0
0 0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>KSPSolve
2 1.0 1.3240e+00 1.0 1.03e+09 1.0 0.0e+00 0.0e+00 0.0e+00 23 49 0
0 0 93 99 0 0 0 778</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>PCSetUp
2 1.0 3.9999e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0
0 0 0 0 0 0 0 0
0</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>PCApply
980 1.0 9.5995e-02 1.0 9.41e+06 1.0 0.0e+00 0.0e+00 0.0e+00 2
0 0 0 0 7 1 0 0
0 98</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</div>
</body>
</html>