<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple style='word-wrap: break-word;-webkit-nbsp-mode: space;-webkit-line-break: after-white-space'><div class=WordSection1><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,Hong<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks for the observation w.r.t the proc 0 having 2 equations. This is a bug from our end. We will fix it and get back to you if needed. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> petsc-dev-bounces@mcs.anl.gov [mailto:petsc-dev-bounces@mcs.anl.gov] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Thursday, January 12, 2012 10:03 PM<br><b>To:</b> Hong Zhang<br><b>Cc:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Ravi, can you run with -ksp_view_binary? This will produce two files.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Hong, ex10 will read in these files and solve them. I will probably not be able to get to this until Monday.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Also, this matrix has just two equations on proc 0 and and about 11000 on proc 1 so its is strangely balanced, in case that helps ...<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Mark<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><div><p class=MsoNormal>On Jan 12, 2012, at 10:35 PM, Hong Zhang wrote:<o:p></o:p></p></div><p class=MsoNormal><br><br><o:p></o:p></p><p class=MsoNormal>Ravi,<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I need more info for debugging. Can you provide a simple stand alone code and matrices in petsc<o:p></o:p></p></div><div><p class=MsoNormal>binary format that reproduce the error?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>MatTransposeMatMult() for mpiaij is a newly developed subroutine - less than one month old <o:p></o:p></p></div><div><p class=MsoNormal>and not well tested yet :-(<o:p></o:p></p></div><div><p class=MsoNormal>I used petsc-dev/src/mat/examples/tests/ex94.c for testing.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thanks,<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>Hong<o:p></o:p></p><div><p class=MsoNormal>On Thu, Jan 12, 2012 at 9:17 PM, Mark F. Adams <<a href="mailto:mark.adams@columbia.edu">mark.adams@columbia.edu</a>> wrote:<o:p></o:p></p><div><p class=MsoNormal>It looks like the problem is in MatTransposeMatMult and Hong (cc'ed) is working on it.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I'm hoping that your output will be enough for Hong to figure this out but I could not reproduce this problem with any of my tests.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>If Hong can not figure this out then we will need to get the matrix from you to reproduce this.<o:p></o:p></p></div><div><p class=MsoNormal><span style='color:#888888'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span class=hoenzb><span style='color:#888888'>Mark</span></span><o:p></o:p></p><div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><div><p class=MsoNormal>On Jan 12, 2012, at 6:25 PM, Ravi Kannan wrote:<o:p></o:p></p></div><p class=MsoNormal><br><br><o:p></o:p></p><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Any luck with the gamg bug fix?</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial'><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Wednesday, January 11, 2012 1:54 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>This seems to be dying earlier than it was last week, so it looks like a new bug in MatTransposeMatMult.<o:p></o:p></p></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div><div><div><p class=MsoNormal> <o:p></o:p></p></div><div><div><div><p class=MsoNormal>On Jan 11, 2012, at 1:59 PM, Matthew Knepley wrote:<o:p></o:p></p></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p></div><div><p class=MsoNormal>On Wed, Jan 11, 2012 at 12:23 PM, Ravi Kannan <<a href="mailto:rxk@cfdrc.com" target="_blank">rxk@cfdrc.com</a>> wrote:<o:p></o:p></p></div><div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt;border-width:initial;border-color:initial'><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I downloaded the dev version again. This time, the program crashes even earlier. Attached is the serial and parallel info outputs.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Could you kindly take a look.</span><o:p></o:p></p></div></div></div></blockquote><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal>It looks like this is a problem with MatMatMult(). Can you try to reproduce this using KSP ex10? You put<o:p></o:p></p></div></div><div><div><p class=MsoNormal>your matrix in binary format and use -pc_type gamg. Then you can send us the matrix and we can track<o:p></o:p></p></div></div><div><div><p class=MsoNormal>it down. Or are you running an example there?<o:p></o:p></p></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal> Thanks,<o:p></o:p></p></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal> Matt<o:p></o:p></p></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt;border-width:initial;border-color:initial'><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial'><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Monday, January 09, 2012 3:08 PM</span><o:p></o:p></p></div><div><div><div><p class=MsoNormal><br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability<o:p></o:p></p></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>Yes its all checked it, just pull from dev.<o:p></o:p></p></div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div><div><div><div><p class=MsoNormal>On Jan 9, 2012, at 2:54 PM, Ravi Kannan wrote:<o:p></o:p></p></div></div><p class=MsoNormal style='margin-bottom:12.0pt'> <o:p></o:p></p><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks for your efforts.</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Do I need to do the install from scratch once again? Or some particular files (check out gamg.c for instance)?</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><div style='border:none;border-top:solid windowtext 3.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Friday, January 06, 2012 10:30 AM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal>I think I found the problem. You will need to use petsc-dev to get the fix.<o:p></o:p></p></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><div><div><p class=MsoNormal>On Jan 6, 2012, at 8:55 AM, Mark F. Adams wrote:<o:p></o:p></p></div></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p></div><div><div><div><p class=MsoNormal>Ravi, I forgot but you can just use -ksp_view_binary to output the matrix data (two files). You could run it with two procs and a Jacobi solver to get it past the solve, where it writes the matrix (I believe).<o:p></o:p></p></div></div><div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><div><div><p class=MsoNormal>On Jan 5, 2012, at 6:19 PM, Ravi Kannan wrote:<o:p></o:p></p></div></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Just send in another email with the attachment.</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div><div style='border:none;border-top:solid windowtext 3.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Jed Brown<br><b>Sent:</b> Thursday, January 05, 2012 5:15 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><div><p class=MsoNormal>On Thu, Jan 5, 2012 at 17:12, Ravi Kannan <<a href="mailto:rxk@cfdrc.com" target="_blank">rxk@cfdrc.com</a>> wrote:<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>I have attached the verbose+info outputs for both the serial and the parallel (2 partitions). NOTE: the serial output at some location says PC=Jacobi! Is it implicitly converting the PC to a Jacobi?<o:p></o:p></p></div></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><div><p class=MsoNormal>Looks like you forgot the attachment.<o:p></o:p></p></div></div></div></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div></blockquote></div><div><p class=MsoNormal><br><br clear=all><o:p></o:p></p></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><p class=MsoNormal>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<o:p></o:p></p></div></div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p></div></div></div></body></html>