[petsc-users] Guidance on GAMG preconditioning

[Matthew Knepley]

On Thu, Jun 4, 2015 at 4:47 PM, Justin Chang <jychang48 at gmail.com> wrote:

> Thank you Matt and Mark for the clarification. Matt, if you recall our
> discussion about calculating the arithmetic intensity from the earlier
> threads, it seems GAMG now has a myriad of all these additional vector and
> matrix operations that were not present in the CG/Jacobi case. Running with
> the command line options you and Mark suggested, I now have these
> additional operations to deal with:
>
> VecMDot
> VecAXPBYCZ
> VecMAXPY
> VecSetRandom
> VecNormalize
> MatMultAdd
> MatMultTranspose
> MatSolve
> MatConvert
> MatScale
> MatResidual
> MatCoarsen
> MatAXPY
> MatMatMult
> MatMatMultSym
> MatMatMultNum
> MatPtAP
> MatPtAPSymbolic
> PatPtAPNumeric
> MatTrnMatMult
> MatTrnMatMultSym
> MatTrnMatMultNum
> MatGetSymTrans
> KSPGMRESOrthog
> PCGAMGGraph_AGG
> PCGAMGCoarse_AGG
> PCGAMGProl_AGG
> PCGAMGPOpt_AGG
> GAMG: createProl and all of its associated events.
> GAMG: partLevel
> PCSetUpOnBlocks
>
> Attached is the output from -log_summary showing the exact counts for the
> case I am running.
>
> I have the following questions:
>

I will do my best


> 1) For the Vec operations VecMDot and VecMAXPY, it seems the estimation of
> total bytes transferred (TBT) relies on knowing how many vectors there are.
> Is there a way to figure this out? Or at least with gamg what would it be,
> three vectors?
>

AMG is very complex. I think that the simple performance model a) might be
too simple, and b) is really hard
to model this way all at once. It really needs to be broken into parts.
Thus I am not sure how much you learn
from this model for AMG.


> 2) It seems there are a lot of matrix manipulations and multiplications.
> Is it safe to say that the size and number of non zeroes is the same? Or
> will it change?
>

Nope, since they happen on different levels.


> 3) If I follow the TBT tabulation as in that paper you pointed me to,
> would MatMultTranspose follow the same formula if the Jacobian is symmetric?
>

Yes.


> 4) How do I calculate anything that requires the multiplication of at
> least two matrices?
>

This is a hard problem.


> 5) More importantly, are any of the above calculations necessary? Because
> log_summary seems to indicate that MatMult() has the greatest amount of
> workload and number of calls. My only hesitation is how much traffic
> MatMatMults may take (assuming I go off of the same assumptions as in that
> paper).
>

The overwhleming cost of AMG is the Galerkin triple-product RAP.

  Thanks,

     Matt


> 6) And/or, are there any other functions that I missed that might be
> important to calculate as well?
>
> Thanks,
> Justin
>
> On Thu, Jun 4, 2015 at 11:33 AM, Mark Adams <mfadams at lbl.gov> wrote:
>
>>
>>
>> On Thu, Jun 4, 2015 at 12:29 PM, Matthew Knepley <knepley at gmail.com>
>> wrote:
>>
>>> On Thu, Jun 4, 2015 at 10:31 AM, Justin Chang <jychang48 at gmail.com>
>>> wrote:
>>>
>>>> Yeah I saw his recommendation and am trying it out. But I am not sure
>>>> what most of those parameters mean. For instance:
>>>>
>>>> 1) What does -pc_gamg_agg_nsmooths refer to?
>>>>
>>>
>>> This is always 1 (its the definition of smoothed aggregation). Mark
>>> allows 0 to support unsmoothed aggregation, which may be
>>> better for easy problems on extremely large machines.
>>>
>>>
>>>> 2) Does increase in the threshold of -pc_gamg_threshold translate to
>>>> making the coarsening faster?
>>>>
>>>
>>> Yes, I believe so (easy to check).
>>>
>>
>> Other way around.
>>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150605/f5ef4b16/attachment-0001.html>