[petsc-dev] MatMPIAIJGetLocalMat problem with GPUs

[Stefano Zampini]

How do you use amgx->localA after it is created?

On Thu, Jun 23, 2022, 20:25 Mark Adams <mfadams at lbl.gov> wrote:

> We have a bug in the AMGx test snes_tests-ex13_amgx in parallel.
> Matt Martineau found that MatMPIAIJGetLocalMat worked in the first pass in
> the code below, where the local matrix is created (INITIAL), but in the
> next pass, when "REUSE" is used, he sees an invalid pointer.
> Matt found that it does have offloadmask == CPU.
> Maybe it is missing logic to put the output in same state as the input?
>
> Any ideas on this or should I just dig into it?
>
> Thanks,
>
> bool partial_setup_allowed = (pc->setupcalled && pc->flag != DIFFERENT_NONZERO_PATTERN);
> 199   if (amgx->nranks > 1) {
> 200     if (partial_setup_allowed) {
> 202       PetscCall(MatMPIAIJGetLocalMat(Pmat, MAT_REUSE_MATRIX, &amgx->localA)); // This path seems doesn't work by the time we reach AmgX API
> 203     } else {
> 205       PetscCall(MatMPIAIJGetLocalMat(Pmat, MAT_INITIAL_MATRIX, &amgx->localA)); // This path works
> 206     }
> 207   } else {
> 208     amgx->localA = Pmat;
> 209   }
> 210
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20220623/0b854dad/attachment.html>