[petsc-users] comparing HYPRE on CPU vs GPU

[Mark Adams]

Also, the coarsening algorithms may be different on the CPU and GPU runs
(hypre does coarsening on the GPU). Even if the algorithms are
nominally the same there could be differences in, for example, the vertex
ordering in greedy coarsening algorithms that results in slightly different
"C" point selection on the device, which will change the preconditioner
slightly.

On Thu, Nov 24, 2022 at 2:40 PM Barry Smith <bsmith at petsc.dev> wrote:

>
>   Probably some of these questions are best asked to the hypre folks. But
> I can answer some
>
> > On Nov 24, 2022, at 11:19 AM, nicola varini <nicola.varini at gmail.com>
> wrote:
> >
> > Dear all, I am comparing the HYPRE boomeramg preconditioner on CPU and
> GPU.
> > It looks like the defaults are different, therefore I tried to specify
> the match the options.
> > I have a few observations/questions:
> > 1) Why are the residuals at step 0 different?
>
>   By default, PETSc uses left preconditioning; thus, the norms reported
> are for the preconditioner residual and will be different for different
> preconditioner options (and possibly slightly different for numerical
> differences even if run with the same preconditioner options.  You can use
> -ksp_pc_side right to have the true residual norm printed, this should be
> the same except for possibly, hopefully, small numerics effects.
>
> > 2) If the options are the same the number of iterations should be the
> same?
>
>   Numerical changes could affect these. Hopefully, not much if all the
> algorithms are truly the same.
>
> > 3) Why on the GPU side the PCApply doesn't seem to be done on GPU?
> > I noticed that if I change certain options, like
> pc_hypre_boomeramg_relax_type_all symmetric SOR/Jacobi, the KSPSolve get
> slower and performs on CPU.
>
>    Hypre probably does not have, by default, a GPU version of SOR, so it
> must be run on a CPU (Petsc also does not currently have a GPU SOR).
>
> > However, how can I diagnose if the computations are perfomed on GPU or
> not?
>
>   Run with -log_view -log_view_gpu_time and look at the final columns that
> give some information on time and communication to the GPU. But since hypre
> does not log flops, the flop rates for computations done by hypre are
> meaningless.
>
>
> >
> > Thanks very much,
> >
> > Nicola
> > <log_cpu><log_gpu>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20221125/fca0e2a4/attachment-0001.html>