[petsc-users] Near nullspace lost in fieldsplit

[Jeremy Theler (External)]

Hi Barry

>   7279 does change the code for MatZeroRowsColumns_MPIAIJ(). But perhaps that does not resolve the problem you are seeing? If that is the case we will need a reproducible example so we can determine exactly what else is happening in your code to cause the difficulties.
>
>Here is the diff for  MatZeroRowsColumns_MPIAIJ()
>
>@@ -1026,7 +1023,7 @@ static PetscErrorCode MatZeroRowsColumns_MPIAIJ(Mat A, PetscInt N, const PetscIn
>
>   PetscCall(PetscFree(lrows));
>
>   /* only change matrix nonzero state if pattern was allowed to be changed */
> -  if (!((Mat_SeqAIJ *)(l->A->data))->keepnonzeropattern) {
> +  if (!((Mat_SeqAIJ *)(l->A->data))->nonew) {
>      PetscObjectState state = l->A->nonzerostate + l->B->nonzerostate;
>      PetscCall(MPIU_Allreduce(&state, &A->nonzerostate, 1, MPIU_INT64, MPI_SUM, PetscObjectComm((PetscObject)A)));
>    }

Fair enough, I might have overlooked this single line. But this change does not fix our issue. The if condition is still true.
I'll need some time to come up with a reproducible example because it involves setting up the PC of the KSP of an SNES of a TS before doing TSSolve() with the DM-allocated jacobian.

However, regarding bug #1, I do have a MWE. It is not CI-friendly. I tried to modify ex47.c to illustrate the issue but could not make it work.
Anyway, the attached tarball:

  1.
reads a global matrix A from a file.
  2.
reads two ISs from another file
  3.
sets up a field split PC
  4.
attaches a near nullspace (read from another file) to one of the sub-KSPs
  5.
sets "dirichlet BCs" with MatZeroRowsColumns() or MatZeroRows() depending if -columns was given in the command line

The issue comes when calling MatZeroRowsColumns() or MatZeroRows().
In the first case, the near nullspace is not lost but it is in the second:

$ make lost
$ ./lost -columns -ksp_view | grep "near null"
            has attached near null space
        has attached near null space
$ mpiexec -n 2 ./lost -columns -ksp_view | grep "near null"
            has attached near null space
        has attached near null space
$ ./lost -ksp_view | grep "near null"
$ mpiexec -n 2 ./lost -ksp_view | grep "near null"
$

When using MatZeroRows(), the code passes through fieldsplit.c:692 which looses the near nullspace.


Note that the original issue we see in our code is that there is a difference between the serial and parallel implementation of MatZeroRowsColumns().
We loose the near nullspace only in parallel but not in serial because of a combination of bugs #1 and #2.


--
jeremy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240213/10d4eeae/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lost.tar.gz
Type: application/gzip
Size: 467498 bytes
Desc: lost.tar.gz
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240213/10d4eeae/attachment-0001.bin>