<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple style='word-wrap: break-word;-webkit-nbsp-mode: space;-webkit-line-break: after-white-space'><div class=WordSection1><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark, Matt,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>We recently downloaded the petsc development version, to test the gamg package.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>This works in serial : we tried for small cases. The parallel case (even with 2 partitions) just hangs. As of now, we have not set any parameters. So I guess that the default parameters are being used.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Do we need to explicitly set the block size to MatSetBlockSize(mat,1) for a PARALLEL run? Our solver solves U,V,W and P separately. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Any input on this would be great.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> petsc-dev-bounces@mcs.anl.gov [mailto:petsc-dev-bounces@mcs.anl.gov] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Thursday, December 15, 2011 1:18 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Dec 15, 2011, at 1:56 PM, Matthew Knepley wrote:<o:p></o:p></p></div><p class=MsoNormal><br><br><o:p></o:p></p><p class=MsoNormal>On Thu, Dec 15, 2011 at 10:23 AM, Ravi Kannan <<a href="mailto:rxk@cfdrc.com">rxk@cfdrc.com</a>> wrote:<o:p></o:p></p><div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in;z-index:auto'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Dear All,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>This is Ravi Kannan from CFD Research Corporation. Recently, we are experimenting with the BoomerAMG preconditioner for some “stiff” CFD problems. In that regard, all the other standard solver-preconditioner combinations failed for the current CFD problem. The boomer is the only one which is able to provide with “converged” solutions.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>We noticed that the scalability of this boomer preconditioner is really poor. For instance, even with a cell size of 2 million, we cannot scale to even 16 partitions (in contrast, the other solver-preconditioner combinations like the BI-CGS/BJacobi gave good enough scalability).</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Are we missing something? Do we need to use a more latest version of boomer?</span><o:p></o:p></p></div></div></blockquote><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Have you tried -pc_type gamg in petsc-dev?<o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>For gamg you also want to use MPIAIJ matrices and set the block size MatSetBlockSize(mat,3), for a 3D velocity field, for instance. You can also try '-pc_gamg_type pa' or '-pc_gamg_type sa'. "pa", for plain aggregation might be better for CFD problems.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Mark<o:p></o:p></p></div><p class=MsoNormal><br><br><o:p></o:p></p><div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> Matt<o:p></o:p></p></div><div><p class=MsoNormal> <span class=apple-style-span><span style='font-size:14.5pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span></span><o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi. </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><o:p> </o:p></p></div></div></blockquote></div><p class=MsoNormal><br clear=all><o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div></body></html>