<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> Sure, but have you ever seen such a large jump in time in going from one to two MPI ranks, and are there any algorithms to do the aggregation that would not require this very expensive parallel symmetrization?<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Sep 17, 2022, at 9:07 AM, Mark Adams <<a href="mailto:mfadams@lbl.gov" class="">mfadams@lbl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Symetrix graph make a transpose and then adds them.<div class="">I imagine adding two different matrices is expensive.</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 16, 2022 at 8:30 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" class="">bsmith@petsc.dev</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br class="">
Mark,<br class="">
<br class="">
I have runs of GAMG on one and two ranks with -pc_gamg_symmetrize_graph because the matrix is far from symmetric and some of GAMG is taking a huge amount more time with 2 ranks than one. (While other stuff like VecNorm shows improvement with two ranks). I've attached the two files<br class="">
<br class="">
Have you seen this before, is there anything that can be done about? If going to two ranks causes almost a doubling in GAMG setup time that makes using parallelism not useful,<br class="">
<br class="">
Barry<br class="">
<br class="">
</blockquote></div>
</div></blockquote></div><br class=""></body></html>