<html><head><base href="x-msg://29/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Well, we do have a bug here. It should work with zero elements on a proc, but the code is being actively developed so you are really helping us to find these cracks.<div><br></div><div>If its not too hard it would be nice if you could give use these matrices, before you fix it, so we can fix this bug. You can just send it to Hong and I (cc'ed).<br><div><br></div><div>Mark<br><div><br></div><div><div><div>On Jan 13, 2012, at 12:16 PM, Ravi Kannan wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div lang="EN-US" link="blue" vlink="purple" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Hi Mark,Hong<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thanks for the observation w.r.t the proc 0 having 2 equations. This is a bug from our end. We will fix it and get back to you if needed.<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thanks,<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Ravi.<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div><div style="border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; border-top-style: solid; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding-top: 3pt; padding-right: 0in; padding-bottom: 0in; padding-left: 0in; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "><span class="Apple-converted-space"> </span><a href="mailto:petsc-dev-bounces@mcs.anl.gov" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a><span class="Apple-converted-space"> </span>[mailto:petsc-dev-bounces@mcs.anl.gov]<span class="Apple-converted-space"> </span><b>On Behalf Of<span class="Apple-converted-space"> </span></b>Mark F. Adams<br><b>Sent:</b><span class="Apple-converted-space"> </span>Thursday, January 12, 2012 10:03 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Hong Zhang<br><b>Cc:</b><span class="Apple-converted-space"> </span>For users of the development version of PETSc<br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-dev] boomerAmg scalability<o:p></o:p></span></div></div></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Ravi, can you run with -ksp_view_binary? This will produce two files.<o:p></o:p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Hong, ex10 will read in these files and solve them. I will probably not be able to get to this until Monday.<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Also, this matrix has just two equations on proc 0 and and about 11000 on proc 1 so its is strangely balanced, in case that helps ...<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Mark<o:p></o:p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Jan 12, 2012, at 10:35 PM, Hong Zhang wrote:<o:p></o:p></div></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Ravi,<o:p></o:p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">I need more info for debugging. Can you provide a simple stand alone code and matrices in petsc<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">binary format that reproduce the error?<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">MatTransposeMatMult() for mpiaij is a newly developed subroutine - less than one month old <o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">and not well tested yet :-(<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">I used petsc-dev/src/mat/examples/tests/ex94.c for testing.<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Thanks,<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 12pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Hong<o:p></o:p></p><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Thu, Jan 12, 2012 at 9:17 PM, Mark F. Adams <<a href="mailto:mark.adams@columbia.edu" style="color: blue; text-decoration: underline; ">mark.adams@columbia.edu</a>> wrote:<o:p></o:p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">It looks like the problem is in MatTransposeMatMult and Hong (cc'ed) is working on it.<o:p></o:p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">I'm hoping that your output will be enough for Hong to figure this out but I could not reproduce this problem with any of my tests.<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">If Hong can not figure this out then we will need to get the matrix from you to reproduce this.<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="color: rgb(136, 136, 136); "><o:p> </o:p></span></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span class="hoenzb"><span style="color: rgb(136, 136, 136); ">Mark</span></span><o:p></o:p></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Jan 12, 2012, at 6:25 PM, Ravi Kannan wrote:<o:p></o:p></div></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Hi Mark,</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Any luck with the gamg bug fix?</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thanks,</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Ravi.</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div><div><div style="border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; border-top-style: solid; padding-top: 3pt; padding-right: 0in; padding-bottom: 0in; padding-left: 0in; border-width: initial; border-color: initial; "><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Wednesday, January 11, 2012 1:54 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></div></div></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">This seems to be dying earlier than it was last week, so it looks like a new bug in MatTransposeMatMult.<o:p></o:p></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Mark<o:p></o:p></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Jan 11, 2012, at 1:59 PM, Matthew Knepley wrote:<o:p></o:p></div></div></div><div><p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 12pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Wed, Jan 11, 2012 at 12:23 PM, Ravi Kannan <<a href="mailto:rxk@cfdrc.com" target="_blank" style="color: blue; text-decoration: underline; ">rxk@cfdrc.com</a>> wrote:<o:p></o:p></div></div><div><blockquote style="border-top-style: none; border-right-style: none; border-bottom-style: none; border-width: initial; border-color: initial; border-left-style: solid; padding-top: 0in; padding-right: 0in; padding-bottom: 0in; padding-left: 6pt; margin-left: 4.8pt; margin-top: 5pt; margin-right: 0in; margin-bottom: 5pt; border-width: initial; border-color: initial; "><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Hi Mark,</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">I downloaded the dev version again. This time, the program crashes even earlier. Attached is the serial and parallel info outputs.</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Could you kindly take a look.</span><o:p></o:p></div></div></div></div></blockquote><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">It looks like this is a problem with MatMatMult(). Can you try to reproduce this using KSP ex10? You put<o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">your matrix in binary format and use -pc_type gamg. Then you can send us the matrix and we can track<o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">it down. Or are you running an example there?<o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> Thanks,<o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> Matt<o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><blockquote style="border-top-style: none; border-right-style: none; border-bottom-style: none; border-width: initial; border-color: initial; border-left-style: solid; padding-top: 0in; padding-right: 0in; padding-bottom: 0in; padding-left: 6pt; margin-left: 4.8pt; margin-top: 5pt; margin-right: 0in; margin-bottom: 5pt; border-width: initial; border-color: initial; "><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thanks,</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Ravi.</span><o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div><div><div style="border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; border-top-style: solid; padding-top: 3pt; padding-right: 0in; padding-bottom: 0in; padding-left: 0in; border-width: initial; border-color: initial; "><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Monday, January 09, 2012 3:08 PM</span><o:p></o:p></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability<o:p></o:p></div></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Yes its all checked it, just pull from dev.<o:p></o:p></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Mark<o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Jan 9, 2012, at 2:54 PM, Ravi Kannan wrote:<o:p></o:p></div></div></div><p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 12pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></p><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Hi Mark,</span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thanks for your efforts.</span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Do I need to do the install from scratch once again? Or some particular files (check out gamg.c for instance)?</span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thanks,</span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Ravi.</span><o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div></div><div><div style="border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; border-top-style: solid; padding-top: 3pt; padding-right: 0in; padding-bottom: 0in; padding-left: 0in; border-width: initial; border-color: initial; border-width: initial; border-color: initial; "><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Mark F. Adams<br><b>Sent:</b> Friday, January 06, 2012 10:30 AM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></div></div></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">I think I found the problem. You will need to use petsc-dev to get the fix.<o:p></o:p></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Mark<o:p></o:p></div></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Jan 6, 2012, at 8:55 AM, Mark F. Adams wrote:<o:p></o:p></div></div></div></div><div><p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 12pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></p></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Ravi, I forgot but you can just use -ksp_view_binary to output the matrix data (two files). You could run it with two procs and a Jacobi solver to get it past the solve, where it writes the matrix (I believe).<o:p></o:p></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Mark<o:p></o:p></div></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Jan 5, 2012, at 6:19 PM, Ravi Kannan wrote:<o:p></o:p></div></div></div></div><div><p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 12pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></p></div><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Just send in another email with the attachment.</span><o:p></o:p></div></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 14pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span><o:p></o:p></div></div></div></div><div style="border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; border-top-style: solid; padding-top: 3pt; padding-right: 0in; padding-bottom: 0in; padding-left: 0in; border-width: initial; border-color: initial; border-width: initial; border-color: initial; border-width: initial; border-color: initial; "><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank" style="color: blue; text-decoration: underline; ">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Jed Brown<br><b>Sent:</b> Thursday, January 05, 2012 5:15 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] boomerAmg scalability</span><o:p></o:p></div></div></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">On Thu, Jan 5, 2012 at 17:12, Ravi Kannan <<a href="mailto:rxk@cfdrc.com" target="_blank" style="color: blue; text-decoration: underline; ">rxk@cfdrc.com</a>> wrote:<o:p></o:p></div></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">I have attached the verbose+info outputs for both the serial and the parallel (2 partitions). NOTE: the serial output at some location says PC=Jacobi! Is it implicitly converting the PC to a Jacobi?<o:p></o:p></div></div></div></div></div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Looks like you forgot the attachment.<o:p></o:p></div></div></div></div></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div></div></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div></div></div></div></blockquote></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br clear="all"><o:p></o:p></div></div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<o:p></o:p></div></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div></div></div></div></div></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div></div></div></div></div></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div></div></div></div></span></blockquote></div><br></div></div></div></body></html>