[petsc-users] Matvecs and KSPSolves with multiple vectors

[Sreeram R Venkat]

Oh, in that case I will try out BoomerAMG. Getting AMGX to build correctly
was also tricky so hopefully the HYPRE build will be easier.

Thanks,
Sreeram

On Thu, Dec 7, 2023, 3:03 PM Pierre Jolivet <pierre at joliv.et> wrote:

>
>
> On 7 Dec 2023, at 9:37 PM, Sreeram R Venkat <srvenkat at utexas.edu> wrote:
>
> Thank you Barry and Pierre; I will proceed with the first option.
>
> I want to use the AMGX preconditioner for the KSP. I will try it out and
> see how it performs.
>
>
> Just FYI, AMGX does not handle systems with multiple RHS, and thus has no
> PCMatApply() implementation.
> BoomerAMG does, and there is a PCMatApply_HYPRE_BoomerAMG() implementation.
> But let us know if you need assistance figuring things out.
>
> Thanks,
> Pierre
>
> Thanks,
> Sreeram
>
> On Thu, Dec 7, 2023 at 2:02 PM Pierre Jolivet <pierre at joliv.et> wrote:
>
>> To expand on Barry’s answer, we have observed repeatedly that MatMatMult
>> with MatAIJ performs better than MatMult with MatMAIJ, you can reproduce
>> this on your own with
>> https://petsc.org/release/src/mat/tests/ex237.c.html.
>> Also, I’m guessing you are using some sort of preconditioner within your
>> KSP.
>> Not all are “KSPMatSolve-ready”, i.e., they may treat blocks of
>> right-hand sides column by column, which is very inefficient.
>> You could run your code with -info dump and send us dump.0 to see what
>> needs to be done on our end to make things more efficient, should you not
>> be satisfied with the current performance of the code.
>>
>> Thanks,
>> Pierre
>>
>> On 7 Dec 2023, at 8:34 PM, Barry Smith <bsmith at petsc.dev> wrote:
>>
>>
>>
>> On Dec 7, 2023, at 1:17 PM, Sreeram R Venkat <srvenkat at utexas.edu> wrote:
>>
>> I have 2 sequential matrices M and R (both MATSEQAIJCUSPARSE of size n x
>> n) and a vector v of size n*m. v = [v_1 , v_2 ,... , v_m] where v_i has
>> size n. The data for v can be stored either in column-major or row-major
>> order.  Now, I want to do 2 types of operations:
>>
>> 1. Matvecs of the form M*v_i = w_i, for i = 1..m.
>> 2. KSPSolves of the form R*x_i = v_i, for i = 1..m.
>>
>> From what I have read on the documentation, I can think of 2 approaches.
>>
>> 1. Get the pointer to the data in v (column-major) and use it to create a
>> dense matrix V. Then do a MatMatMult with M*V = W, and take the data
>> pointer of W to create the vector w. For KSPSolves, use KSPMatSolve with R
>> and V.
>>
>> 2. Create a MATMAIJ using M/R and use that for matvecs directly with the
>> vector v. I don't know if KSPSolve with the MATMAIJ will know that it is a
>> multiple RHS system and act accordingly.
>>
>> Which would be the more efficient option?
>>
>>
>> Use 1.
>>
>>
>> As a side-note, I am also wondering if there is a way to use row-major
>> storage of the vector v.
>>
>>
>> No
>>
>> The reason is that this could allow for more coalesced memory access when
>> doing matvecs.
>>
>>
>>   PETSc matrix-vector products use BLAS GMEV matrix-vector products for
>> the computation so in theory they should already be well-optimized
>>
>>
>> Thanks,
>> Sreeram
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20231207/4f10e11d/attachment.html>