[petsc-dev] large cost of symmetrizing for GAMG?

[Mark Adams]

I don't see a problem here other than the network looks bad relative to the
problem size.

All the graph methods (PCGAMGCreateG and MIS) are 2x slower.
  - THe symmetrization must be in PCGAMGCreateG.
  - MIS is pretty old code (the algorithm and original code are 25 years
old)
RAPs are about the same.
KSPGMRESOrthog and MatMult are nowhere near perfect.

The graph setup work gets amortized by (most) applications and benchmarkers
that know how to benchmark, so it is not highly engineered like the RAP and
MatMult.
Note, this application is building the graph work for every linear solve.
I am guessing they want '-pc_gamg_reuse_interpolation true' or are doing a
single step/stage TS with a linear problem and AMR every time step, which
would be pretty pathological.
I'm doing an MR right now, maybe I should change the default for
-pc_gamg_reuse_interpolation?

Mark



On Sat, Sep 17, 2022 at 10:12 AM Barry Smith <bsmith at petsc.dev> wrote:

>
>   Sure, but have you ever seen such a large jump in time in going from one
> to two MPI ranks, and are there any algorithms to do the aggregation that
> would not require this very expensive parallel symmetrization?
>
> On Sep 17, 2022, at 9:07 AM, Mark Adams <mfadams at lbl.gov> wrote:
>
> Symetrix graph make a transpose and then adds them.
> I imagine adding two different matrices is expensive.
>
> On Fri, Sep 16, 2022 at 8:30 PM Barry Smith <bsmith at petsc.dev> wrote:
>
>>
>> Mark,
>>
>>    I have runs of GAMG on one and two ranks with
>> -pc_gamg_symmetrize_graph because the matrix is far from symmetric and some
>> of GAMG is taking a huge amount more time with 2 ranks than one. (While
>> other stuff like VecNorm shows improvement with two ranks). I've attached
>> the two files
>>
>>   Have you seen this before, is there anything that can be done about? If
>> going to two ranks causes almost a doubling in GAMG setup time that makes
>> using parallelism not useful,
>>
>>   Barry
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20220917/67aa9f46/attachment.html>