<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">2. 7. 2018 v 16:38, Matthew Knepley <<a href="mailto:knepley@gmail.com" class="">knepley@gmail.com</a>>:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="gmail_quote"><div dir="ltr" class="">On Mon, Jul 2, 2018 at 9:33 AM Vaclav Hapla <<a href="mailto:vaclav.hapla@erdw.ethz.ch" class="">vaclav.hapla@erdw.ethz.ch</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">2. 7. 2018 v 15:50, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>>:</div><br class="m_6618602247234769429Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div dir="ltr" class="">On Mon, Jul 2, 2018 at 8:28 AM Vaclav Hapla <<a href="mailto:vaclav.hapla@erdw.ethz.ch" target="_blank" class="">vaclav.hapla@erdw.ethz.ch</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class="">2. 7. 2018 v 15:05, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>>:</div><br class="m_6618602247234769429m_-2599313670520163405Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><div class="gmail_quote"><div dir="ltr" class="">On Mon, Jul 2, 2018 at 7:54 AM Vaclav Hapla <<a href="mailto:vaclav.hapla@erdw.ethz.ch" target="_blank" class="">vaclav.hapla@erdw.ethz.ch</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">2. 7. 2018 v 14:48, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>>:</div><br class="m_6618602247234769429m_-2599313670520163405m_4254946542362599331Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div dir="ltr" class="">On Mon, Jul 2, 2018 at 3:48 AM Vaclav Hapla <<a href="mailto:vaclav.hapla@erdw.ethz.ch" target="_blank" class="">vaclav.hapla@erdw.ethz.ch</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class="">Barry wrote:</div><div class=""><blockquote type="cite" class=""><span style="font-family: Menlo-Regular; float: none; display: inline !important;" class=""> This could get ugly real fast, for example, for vector operations, there may be dozens of named vectors and each one gets its own logging? You'd have to make sure that only the objects you care about get named, is that possible?</span><br style="font-family: Menlo-Regular;" class=""><br style="font-family: Menlo-Regular;" class=""><span style="font-family: Menlo-Regular; float: none; display: inline !important;" class=""> I don't know if there is a good solution within the PETSc logging infrastructure to get what you want but maybe what you propose is the best possible.</span><br style="font-family: Menlo-Regular;" class=""></blockquote><br class=""></div><div class="">As I suggest, this behavior would be only triggered by a specific option.</div><div class=""><br class=""></div><div class="">I think there are actually 4 strings which could be used as an event name suffix in log view:</div><div class="">1) name</div><div class="">2) prefix</div><div class="">3) type</div><div class="">4) custom string (set by something like PetscObjectSetLogViewSuffix)</div><div class="">I think the best would be to let user choose by offering -log_view_by_{name,prefix,type,suffix}.</div><div class=""><br class=""></div><div class="">For example, with -log_view_by_prefix, you could readily distinguish PCTelescope outer and inner apply, because you would see a separate "PCApply (telescope_)" event.</div><div class="">With -log_view_by_type, you would see PCApply (telescope).</div><div class=""><br class=""></div><div class="">I think this would be useful because the current class-wide events like MatMult or PCApply aggregate very different operations from which some are for free and some form hotspots.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Stefano wrote:</div><div class=""><blockquote type="cite" class=""><div style="font-family: Menlo-Regular;" class="">The issue with this sort of “dynamic” logging is that now PETSc requires PetscLogEvent created during the registration of the class, so that all the ranks in PETSC_COMM_WORLD have the same events registered.</div><div style="font-family: Menlo-Regular;" class="">What you propose is not generally supported for this specific reason.</div><div style="font-family: Menlo-Regular;" class=""><br class=""></div><div style="font-family: Menlo-Regular;" class="">Your “log_name” may work if users register their own classes (with their own LogEvents created properly), and currently we don’t have support (maybe I’m wrong) to add an “InitializePackage” method for the users’ registered classes.</div></blockquote></div><div class=""><br class=""></div><div class="">I don't agree. What I suggest is basically an ability to allow automatically created object-wise events, so it _can't_ be managed during the class registration. In presence of respective option, the event would be created during PetscLogEventBegin by taking the class-wide event's name, concatenating the suffix and registering a new event. The event id would be stored in the PetscObject structure.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Matt wrote:</div><div class=""><blockquote type="cite" class=""><div style="font-family: Menlo-Regular;" class="">As people have pointed out, this would not work well for Events. However, this is exactly what stages are for.</div><div style="font-family: Menlo-Regular;" class="">Use separate stages for the different types of MatMult. I did this, for example, when looking at performance</div><div style="font-family: Menlo-Regular;" class="">on different MG levels.</div></blockquote><br class=""></div><div class="">Yes, performance on different MG levels is a nice use case. I don't understand how you inject stages into MatMults. To me it's exactly the same problem as with events - you have to define MatMult_custom where you take the original mult and wrap into PetscStageLogPush/Pop and then use MatSetOperation to redefine MatMult. Or do you mean something more elegant?</div></div></blockquote><div class=""><br class=""></div><div class="">You could do that, but usually I think of stages as being structural. I think for your example I would push/pop the stage</div><div class="">inside your Mat operation wrapper (I don't see why you need another one), and this behavior could be controlled with</div><div class="">another option so you could turn it off.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">I meant hierarchies of typically Mats or PCs, where you don't define any custom operations but compose together existing types (which should be promoted I believe). So no "my" wrapper. As I wrote below:</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class=""><blockquote type="cite" style="font-family: Menlo-Regular;" class=""> Think e.g. of having additive MATCOMPOSITE wrapping multiplicative MATCOMPOSITE wrapping MATTRANSPOSE wrapping MATAIJ. You want to measure this MATAIJ instance's MatMult separately but you surely don't want to rewrite implementation of MatMult_Transpose or force yourself to use MATSHELL just to hang the events on MatMult*.</blockquote></div></blockquote></div></div></blockquote></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div><div class="">Its not enough to make separate stages for additive MC, multiplicative MC, and MT? If you want stages for every single</div><div class="">combination created dynamically, you can push another stage when each of these combinations is created using GetTag()</div><div class="">or something like that. You could switch between these behaviors with an option.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">I'm not sure I understand. Do you mean registering and pushing/popping these stages in the user's code? You can surely call PetscStageLogRegister somewhere after PetscInitialize, but where do you place your PetscStageLogPush/Pop calls?</div></div></div></blockquote><div class=""><br class=""></div><div class="">No. You would create a stage when the MATCOMPOSITE is created (or once when any MATCOMPOSITE is created), and push/pop</div><div class="">on application.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">There's surely no problem with creating that stage. But still I don't see how can you push/pop on specific MatMult if it's aggregated together with another MatMults in a higher level MatMult without redefining the latter? And there can be arbitrary number of such levels. If you push/pop your stage in the code calling the top-level MatMult, you can't distinguish different MatMults occurring inside.</div></div></div></blockquote><div class=""><br class=""></div><div class="">I do not understand what you mean. Each operation would happen in a separate stage. Its time for a specific, simple example</div><div class="">where you think this would break. This could be coded in 10 minutes.</div></div></div></div></blockquote><div><br class=""></div><div>OK, now I got you. But I didn't mean creating a bunch of tests for any possible hierarchy and running them just to get the separated time. I hoped that any hierarchy specifiable just from options could be flexibly reflected in log view without any coding, additionally with realistic times and counts.</div><div><br class=""></div><div>I see a lot of possible benefits. Within PETSc, it's not that obvious with MatMult but think of PCSetUp, PCApply, KSPSetUp, KSPSolve. Think of your talk's slide-wide example of composed KSPs and PCs specified just from options. Don't you really think it would be nice to be able to get separated counts, times, flops, MPI messages and so on per user-specified stages flexibly?</div><div><br class=""></div><div>In my opinion, flexibility of the log view lags behind the flexibility of always advertised composable solvers. With my proposal,<span style="font-family: Menlo-Regular;" class=""> you would get it just by specifying</span> -<span style="font-family: Menlo-Regular;" class="">log_view_by_prefix.</span></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><div class="">And I don't understand what's in this context the advantage of using stages (which are typically registered by user and from his perspective they are generally spanning multiple different operations/function calls) against events which are precisely meant for single operations/function calls (at least I hope so based on thousands of use cases in PETSc itself).</div></div></div></blockquote><div class=""><br class=""></div><div class="">Events are semantically associated with given operations. Stages are semantically associated with the context of an operation.</div><div class="">So defining a bunch of events for MatMult does not make sense to me, whereas defining a bunch of stages for different MatMults</div><div class="">definitely does.</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><div class="">You want me to create a stage which is shorter than then the wrapping event which is I think much uglier than what I propose :-)</div></div></div></blockquote><div class=""><br class=""></div><div class="">I don't. I think it follows exactly the toplevel design.</div></div></div></div></blockquote><div><br class=""></div><div>OK, I really don't like it but don't want to argue about aesthetics. But a hard argument can be e.g. that only one stage can be active at a time (hence Push/Pop name) so what I wanted to have is simply not doable your way. From this reason there's probably no hard-wired stage anywhere in PETSc.</div><div><br class=""></div><div>Vaclav</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="gmail_quote"><div class=""><br class=""></div><div class=""> Thanks,</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><div class="">Vaclav</div><div class=""><br class=""></div><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><div class="">Thanks</div><div class=""><br class=""></div><div class="">Vaclav</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><div class="gmail_quote"><div class=""><br class=""></div><div class="">The reason I think this is preferable is that we do not mess with any logging infrastructure, we just use stages inside of other objects.</div><div class=""><br class=""></div><div class=""> Thanks,</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><div class="">Thanks</div><div class=""><br class=""></div><div class="">Vaclav</div><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class="">Thanks</div><div class=""><br class=""></div><div class="">Vaclav</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""><blockquote type="cite" class=""><div class="">29. 6. 2018 v 22:42, Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank" class="">bsmith@mcs.anl.gov</a>>:</div><br class="m_6618602247234769429m_-2599313670520163405m_4254946542362599331m_71980157714340757Apple-interchange-newline"><div class=""><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class="">On Jun 29, 2018, at 9:33 AM, Vaclav Hapla <<a href="mailto:vaclav.hapla@erdw.ethz.ch" target="_blank" class="">vaclav.hapla@erdw.ethz.ch</a>> wrote:<br class=""><br class=""><br class=""><br class=""><blockquote type="cite" class="">22. 6. 2018 v 17:47, Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank" class="">bsmith@mcs.anl.gov</a>>:<br class=""><br class=""><br class=""><br class=""><blockquote type="cite" class="">On Jun 22, 2018, at 5:43 AM, Pierre Jolivet <<a href="mailto:pierre.jolivet@enseeiht.fr" target="_blank" class="">pierre.jolivet@enseeiht.fr</a>> wrote:<br class=""><br class="">Hello,<br class="">I’m solving a system using a MATSHELL and PCGAMG.<br class="">The MPIAIJ Mat I’m giving to GAMG has a specific structure (inherited from the MATSHELL) I’d like to exploit during the solution phase when the smoother on the finest level is doing MatMults.<br class=""><br class="">Is there some way to:<br class="">1) decouple in -log_view the time spent in the MATSHELL MatMult and in the smoothers MatMult<br class=""></blockquote><br class="">You can register a new event and then inside your MATSHELL MatMult() call PetscLogEventBegin/End on your new event.<br class=""><br class=""> Note that the MatMult() like will still contain the time for your MatShell mult so you will need to subtract it off to get the time for your non-shell matmults.<br class=""></blockquote><br class="">In PERMON, we sometimes have quite complicated hierarchy of wrapped matrices and want to measure MatMult{,Transpose,Add,TransposeAdd} separately for particular ones. Think e.g. of having additive MATCOMPOSITE wrapping multiplicative MATCOMPOSITE wrapping MATTRANSPOSE wrapping MATAIJ. You want to measure this MATAIJ instance's MatMult separately but you surely don't want to rewrite implementation of MatMult_Transpose or force yourself to use MATSHELL just to hang the events on MatMult*.<br class=""><br class="">We had a special wrapper type just adding some prefix to the events for the given object but this is not nice. What about adding a functionality to PetscLogEventBegin/End that would distinguish based on the first PetscObject's name or option prefix? Of course optionally not to break guys relying on current behavior - e.g. under something like -log_view_by_name. To me it's quite an elegant solution working for any PetscObject and any event.<br class=""></blockquote><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><span style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; float: none; display: inline !important;" class=""> This could get ugly real fast, for example, for vector operations, there may be dozens of named vectors and each one gets its own logging? You'd have to make sure that only the objects you care about get named, is that possible?</span><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><span style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; float: none; display: inline !important;" class=""> I don't know if there is a good solution within the PETSc logging infrastructure to get what you want but maybe what you propose is the best possible.</span><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><span style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; float: none; display: inline !important;" class=""> Barry</span><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><br style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Menlo-Regular; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><br class="">I can do that if I get some upvotes.<br class=""><br class="">Vaclav<br class=""><br class=""><blockquote type="cite" class=""><br class=""><blockquote type="cite" class="">2) hardwire a specific MatMult implementation for the smoother on the finest level<br class=""></blockquote><br class="">In the latest release you do MatSetOperation() to override the normal matrix vector product with anything else you want.<span class="m_6618602247234769429m_-2599313670520163405m_4254946542362599331m_71980157714340757Apple-converted-space"> </span><br class=""><br class=""><blockquote type="cite" class=""><br class="">Thanks in advance,<br class="">Pierre<br class=""><br class="">PS : here is what I have right now,<br class="">MatMult 118 1.0 1.0740e+02 1.6 1.04e+13 1.6 1.7e+06 6.1e+05 0.0e+00 47100 90 98 0 47100 90 98 0 81953703<br class="">[…]<br class="">PCSetUp 2 1.0 8.6513e+00 1.0 1.01e+09 1.7 2.6e+05 4.0e+05 1.8e+02 5 0 14 10 66 5 0 14 10 68 94598<br class="">PCApply 14 1.0 8.0373e+01 1.1 9.06e+12 1.6 1.3e+06 6.0e+05 2.1e+01 45 87 72 78 8 45 87 72 78 8 95365211 // I’m guessing a lot of time here is being wasted in doing inefficient MatMults on the finest level but this is only speculation<br class=""><br class="">Same code with -pc_type none -ksp_max_it 13,<br class="">MatMult 14 1.0 1.2936e+01 1.7 1.35e+12 1.6 2.0e+05 6.1e+05 0.0e+00 15100 78 93 0 15100 78 93 0 88202079<br class=""><br class="">The grid itself is rather simple (two levels, extremely aggressive coarsening),<br class="">type is MULTIPLICATIVE, levels=2 cycles=v<br class="">KSP Object: (mg_coarse_) 1024 MPI processes<br class="">linear system matrix = precond matrix:<br class=""> Mat Object: 1024 MPI processes<br class=""> type: mpiaij<br class=""> rows=775, cols=775<br class=""> total: nonzeros=1793, allocated nonzeros=1793<br class=""><br class="">linear system matrix followed by preconditioner matrix:<br class="">Mat Object: 1024 MPI processes<br class="">type: shell<br class="">rows=1369307136, cols=1369307136<br class="">Mat Object: 1024 MPI processes<br class="">type: mpiaij<br class="">rows=1369307136, cols=1369307136<br class="">total: nonzeros=19896719360, allocated nonzeros=19896719360</blockquote></blockquote></blockquote></div></blockquote></div><br class=""></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="m_6618602247234769429m_-2599313670520163405Apple-converted-space"> </span><br class=""><div dir="ltr" class="m_6618602247234769429m_-2599313670520163405m_4254946542362599331gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.caam.rice.edu/~mk51/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class=""></div></div></div></div></div></div></div></blockquote></div><br class=""></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="m_6618602247234769429m_-2599313670520163405Apple-converted-space"> </span><br class=""><div dir="ltr" class="m_6618602247234769429m_-2599313670520163405gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.caam.rice.edu/~mk51/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a></div></div></div></div></div></div></div></blockquote></div><br class=""></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div dir="ltr" class="m_6618602247234769429gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.caam.rice.edu/~mk51/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class=""></div></div></div></div></div></div></div></blockquote></div><br class=""></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.caam.rice.edu/~mk51/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a></div></div></div></div></div></div></div></blockquote></div><br class=""></body></html>