<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple style='word-wrap: break-word;-webkit-nbsp-mode: space;-webkit-line-break: after-white-space'><div class=WordSection1><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark, Hong,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>As you might remember, the reason for this whole exercise was to obtain a solution for a very stiff problem.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>We did have Hypre Boomer amg. This did not scale, but gives correct solution. So we wanted an alternative; hence we approached you for gamg.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>However for certain cases, gamg crashes. Even for the working cases, it takes about 15-20 times more sweeps than the boomer-hypre. Hence it is cost-prohibitive.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hopefully this gamg solver can be improved in the near future, for users like us. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Warm Regards,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Mark F. Adams [mailto:mark.adams@columbia.edu] <br><b>Sent:</b> Wednesday, January 18, 2012 9:56 AM<br><b>To:</b> Hong Zhang<br><b>Cc:</b> rxk@cfdrc.com<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Hong and Ravi,<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I fixed a bug with the 6x6 problem. There seemed to be a bug in MatTranposeMat with funny decomposition, that was not really verified. So we can wait for Ravi to continue with his tests a fix them as they arise.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Mark<o:p></o:p></p></div><div><p class=MsoNormal>ps, Ravi, I may not have cc'ed so I will send again.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Jan 17, 2012, at 7:37 PM, Hong Zhang wrote:<o:p></o:p></p></div><p class=MsoNormal><br><br><o:p></o:p></p><p class=MsoNormal>Ravi,<o:p></o:p></p><div><p class=MsoNormal>I wrote a simple test ex163.c (attached) on MatTransposeMatMult().<o:p></o:p></p></div><div><p class=MsoNormal>Loading your 6x6 matrix gives no error from MatTransposeMatMult()<o:p></o:p></p></div><div><p class=MsoNormal>using 1,2,...7 processes.<o:p></o:p></p></div><div><p class=MsoNormal>For example,<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><p class=MsoNormal>petsc-dev/src/mat/examples/tests>mpiexec -n 4 ./ex163 -f /Users/hong/Downloads/repetscdevboomeramgscalability/binaryoutput<o:p></o:p></p></div><div><p class=MsoNormal>A:<o:p></o:p></p></div><div><p class=MsoNormal>Matrix Object: 1 MPI processes<o:p></o:p></p></div><div><p class=MsoNormal> type: mpiaij<o:p></o:p></p></div><div><p class=MsoNormal>row 0: (0, 1.66668e+06) (1, -1.35) (3, -0.6) <o:p></o:p></p></div><div><p class=MsoNormal>row 1: (0, -1.35) (1, 1.66667e+06) (2, -1.35) (4, -0.6) <o:p></o:p></p></div><div><p class=MsoNormal>row 2: (1, -1.35) (2, 1.66667e+06) (5, -0.6) <o:p></o:p></p></div><div><p class=MsoNormal>row 3: (0, -0.6) (3, 1.66668e+06) (4, -1.35) <o:p></o:p></p></div><div><p class=MsoNormal>row 4: (1, -0.6) (3, -1.35) (4, 1.66667e+06) (5, -1.35) <o:p></o:p></p></div><div><p class=MsoNormal>row 5: (2, -0.6) (4, -1.35) (5, 1.66667e+06) <o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>C = A^T * A:<o:p></o:p></p></div><div><p class=MsoNormal>Matrix Object: 1 MPI processes<o:p></o:p></p></div><div><p class=MsoNormal> type: mpiaij<o:p></o:p></p></div><div><p class=MsoNormal>row 0: (0, 2.77781e+12) (1, -4.50002e+06) (2, 1.8225) (3, -2.00001e+06) (4, 1.62) <o:p></o:p></p></div><div><p class=MsoNormal>row 1: (0, -4.50002e+06) (1, 2.77779e+12) (2, -4.50001e+06) (3, 1.62) (4, -2.00001e+06) (5, 1.62) <o:p></o:p></p></div><div><p class=MsoNormal>row 2: (0, 1.8225) (1, -4.50001e+06) (2, 2.7778e+12) (4, 1.62) (5, -2.00001e+06) <o:p></o:p></p></div><div><p class=MsoNormal>row 3: (0, -2.00001e+06) (1, 1.62) (3, 2.77781e+12) (4, -4.50002e+06) (5, 1.8225) <o:p></o:p></p></div><div><p class=MsoNormal>row 4: (0, 1.62) (1, -2.00001e+06) (2, 1.62) (3, -4.50002e+06) (4, 2.77779e+12) (5, -4.50001e+06) <o:p></o:p></p></div><div><p class=MsoNormal>row 5: (1, 1.62) (2, -2.00001e+06) (3, 1.8225) (4, -4.50001e+06) (5, 2.7778e+12)<o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Do I miss something?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Hong<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Sat, Jan 14, 2012 at 3:37 PM, Mark F. Adams <<a href="mailto:mark.adams@columbia.edu">mark.adams@columbia.edu</a>> wrote:<o:p></o:p></p><div><p class=MsoNormal>Ravi, this system is highly diagonally dominate. I've fixed the code so you can pull and try again.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I've decided to basically just do a one level method with DD systems. I don't know if that is the best semantics, I think Barry will hate it, because it gives you a one level solver when you asked for MG. It now picks up the coarse grid solver as the solver, which is wrong, so I need to fix this if we decide to stick with the current semantics.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>And again thanks for helping to pound on this code.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Mark<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p><div><div><div><div><p class=MsoNormal>On Jan 13, 2012, at 6:33 PM, Ravi Kannan wrote:<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark, Hong,</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Lets make it simpler. I fixed my partitiotion bug (in metis). Now there is a equidivision of cells.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>To simplify even further, lets run a much smaller case : with 6 cells (equations) in SERIAL. This one crashes. The out and the ksp_view_binary files are attached.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>RAvi.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial'><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Friday, January 13, 2012 3:00 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>Well, we do have a bug here. It should work with zero elements on a proc, but the code is being actively developed so you are really helping us to find these cracks.<o:p></o:p></p></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal>If its not too hard it would be nice if you could give use these matrices, before you fix it, so we can fix this bug. You can just send it to Hong and I (cc'ed).<o:p></o:p></p></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><div><div><p class=MsoNormal>On Jan 13, 2012, at 12:16 PM, Ravi Kannan wrote:<o:p></o:p></p></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,Hong</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks for the observation w.r.t the proc 0 having 2 equations. This is a bug from our end. We will fix it and get back to you if needed.</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div><div><div style='border:none;border-top:solid windowtext 3.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Thursday, January 12, 2012 10:03 PM<br><b>To:</b> Hong Zhang<br><b>Cc:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><p class=MsoNormal>Ravi, can you run with -ksp_view_binary? This will produce two files.<o:p></o:p></p></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>Hong, ex10 will read in these files and solve them. I will probably not be able to get to this until Monday.<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>Also, this matrix has just two equations on proc 0 and and about 11000 on proc 1 so its is strangely balanced, in case that helps ...<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><div><div><p class=MsoNormal>On Jan 12, 2012, at 10:35 PM, Hong Zhang wrote:<o:p></o:p></p></div></div></div><div><div><p class=MsoNormal style='margin-bottom:12.0pt'><br><br><o:p></o:p></p></div></div><div><div><p class=MsoNormal>Ravi,<o:p></o:p></p></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>I need more info for debugging. Can you provide a simple stand alone code and matrices in petsc<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>binary format that reproduce the error?<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>MatTransposeMatMult() for mpiaij is a newly developed subroutine - less than one month old <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>and not well tested yet :-(<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>I used petsc-dev/src/mat/examples/tests/ex94.c for testing.<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>Thanks,<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>Hong<o:p></o:p></p><div><div><div><p class=MsoNormal>On Thu, Jan 12, 2012 at 9:17 PM, Mark F. Adams <<a href="mailto:mark.adams@columbia.edu" target="_blank">mark.adams@columbia.edu</a>> wrote:<o:p></o:p></p></div></div><div><div><div><p class=MsoNormal>It looks like the problem is in MatTransposeMatMult and Hong (cc'ed) is working on it.<o:p></o:p></p></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>I'm hoping that your output will be enough for Hong to figure this out but I could not reproduce this problem with any of my tests.<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>If Hong can not figure this out then we will need to get the matrix from you to reproduce this.<o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='color:#888888'> </span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='color:#888888'>Mark</span><o:p></o:p></p></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><div><div><p class=MsoNormal>On Jan 12, 2012, at 6:25 PM, Ravi Kannan wrote:<o:p></o:p></p></div></div></div><div><div><p class=MsoNormal style='margin-bottom:12.0pt'><br><br><o:p></o:p></p></div></div><div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Any luck with the gamg bug fix?</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div><div><div style='border:none;border-top:solid windowtext 3.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Wednesday, January 11, 2012 1:54 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>This seems to be dying earlier than it was last week, so it looks like a new bug in MatTransposeMatMult.<o:p></o:p></p></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><div><div><p class=MsoNormal>On Jan 11, 2012, at 1:59 PM, Matthew Knepley wrote:<o:p></o:p></p></div></div></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'> <o:p></o:p></p></div><div><div><div><p class=MsoNormal>On Wed, Jan 11, 2012 at 12:23 PM, Ravi Kannan <<a href="mailto:rxk@cfdrc.com" target="_blank">rxk@cfdrc.com</a>> wrote:<o:p></o:p></p></div></div></div><div><blockquote style='border:none;border-left:solid windowtext 3.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I downloaded the dev version again. This time, the program crashes even earlier. Attached is the serial and parallel info outputs.</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Could you kindly take a look.</span><o:p></o:p></p></div></div></div></div></div></blockquote><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal>It looks like this is a problem with MatMatMult(). Can you try to reproduce this using KSP ex10? You put<o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal>your matrix in binary format and use -pc_type gamg. Then you can send us the matrix and we can track<o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal>it down. Or are you running an example there?<o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal> Thanks,<o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal> Matt<o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><blockquote style='border:none;border-left:solid windowtext 3.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.</span><o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div><div><div style='border:none;border-top:solid windowtext 3.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Monday, January 09, 2012 3:08 PM</span><o:p></o:p></p></div></div></div><div><div><div><div><div><p class=MsoNormal><br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability<o:p></o:p></p></div></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div><div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><p class=MsoNormal>Yes its all checked it, just pull from dev.<o:p></o:p></p></div></div></div><div><div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div><div><div><div><div><div><p class=MsoNormal>On Jan 9, 2012, at 2:54 PM, Ravi Kannan wrote:<o:p></o:p></p></div></div></div></div><p class=MsoNormal style='margin-bottom:12.0pt'> <o:p></o:p></p><div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Mark,</span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks for your efforts.</span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Do I need to do the install from scratch once again? Or some particular files (check out gamg.c for instance)?</span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks,</span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Ravi.</span><o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div></div><div><div style='border:none;border-top:solid windowtext 3.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Friday, January 06, 2012 10:30 AM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><p class=MsoNormal>I think I found the problem. You will need to use petsc-dev to get the fix.<o:p></o:p></p></div></div></div></div><div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div><div><div><div><div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div></div></div></div><div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><div><div><p class=MsoNormal>On Jan 6, 2012, at 8:55 AM, Mark F. Adams wrote:<o:p></o:p></p></div></div></div></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'> <o:p></o:p></p></div><div><div><div><div><div><p class=MsoNormal>Ravi, I forgot but you can just use -ksp_view_binary to output the matrix data (two files). You could run it with two procs and a Jacobi solver to get it past the solve, where it writes the matrix (I believe).<o:p></o:p></p></div></div></div></div><div><div><div><div><div><p class=MsoNormal>Mark<o:p></o:p></p></div></div></div></div></div><div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><div><div><div><p class=MsoNormal>On Jan 5, 2012, at 6:19 PM, Ravi Kannan wrote:<o:p></o:p></p></div></div></div></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'> <o:p></o:p></p></div><div><div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Just send in another email with the attachment.</span><o:p></o:p></p></div></div></div></div></div><div><div><div><div><div><p class=MsoNormal><span style='font-size:14.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div></div></div></div></div><div style='border:none;border-top:solid windowtext 3.0pt;padding:3.0pt 0in 0in 0in;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial;border-width:initial;border-color:initial'><div><div><div><div><div><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Jed Brown<br><b>Sent:</b> Thursday, January 05, 2012 5:15 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></p></div></div></div></div></div></div><div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div><div><div><div><div><div><div><p class=MsoNormal>On Thu, Jan 5, 2012 at 17:12, Ravi Kannan <<a href="mailto:rxk@cfdrc.com" target="_blank">rxk@cfdrc.com</a>> wrote:<o:p></o:p></p></div></div></div></div></div><div><div><div><div><div><p class=MsoNormal>I have attached the verbose+info outputs for both the serial and the parallel (2 partitions). NOTE: the serial output at some location says PC=Jacobi! Is it implicitly converting the PC to a Jacobi?<o:p></o:p></p></div></div></div></div></div></div><div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div><div><div><div><div><div><div><p class=MsoNormal>Looks like you forgot the attachment.<o:p></o:p></p></div></div></div></div></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div></div></div></blockquote></div><div><div><div><p class=MsoNormal><br><br clear=all><o:p></o:p></p></div></div></div><div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><div><p class=MsoNormal>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<o:p></o:p></p></div></div></div></div><div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div><div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div><div><p class=MsoNormal> <o:p></o:p></p></div></div></div></div></div></div></div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif"'><out><binaryoutput><<a href="http://binaryoutput.info/" target="_blank">binaryoutput.info</a>><o:p></o:p></span></p></div></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal><ex163.c><o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div></div></body></html>