[petsc-users] hypre / hip usage

[Jed Brown]

"Paul T. Bauman" <ptbauman at gmail.com> writes:

> On Fri, Jan 21, 2022 at 8:19 AM Jed Brown <jed at jedbrown.org> wrote:
>
>> Mark Adams <mfadams at lbl.gov> writes:
>>
>> > Two questions about hypre on HIP:
>> >
>> > * I am doing this now. Is this correct?
>> >
>> >     '--download-hypre',
>> >     '--download-hypre-configure-arguments=--enable-unified-memory',
>>
>
> Apologies for interjecting, but I want to point out here that a pretty good
> chunk of BoomerAMG is ported to the GPU and you may not need this
> unified-memory option. I point this out because you will get substantially
> better performance without this option, i.e. using "native" GPU memory. I
> do not know the intricacies of the PETSc/HYPRE/GPU interaction so maybe
> PETSc won't handle the CPU->GPU memcopies for you (I'm assuming vecs, mats
> are assembled on the CPU) in which case you might need the option. And if
> you do run into code paths in BoomerAMG that are not ported to the GPU and
> you want to use them, I'd be very interested to know what the options are
> that are missing a GPU port.

We have matrices and vectors assembled on the device and logic to pass the device data to Hypre. Stefano knows the details.

Will the option --enable-unified-memory hurt performance if we provide all data on the device?