[petsc-dev] FieldSplit with different nonzero pattern

[Pierre Jolivet]

Hong,
Thank you for the patch.
Our application is not crashing anymore.
Hopefully this will be merged quickly in master.

Thanks,
Pierre

> On 7 May 2018, at 10:07 PM, Hong <hzhang at mcs.anl.gov> wrote:
> 
> Pierre:
> I added this support 
> https://bitbucket.org/petsc/petsc/commits/ef7efd37182b12f7b7ab7c34390aae8e592a83c2 <https://bitbucket.org/petsc/petsc/commits/ef7efd37182b12f7b7ab7c34390aae8e592a83c2>
> 
> It is in branch hzhang/fix-pc-fieldsplit-reuse.
> You may give it a try. Once it passed all our regression tests, it will be merged to petsc-master branch.
> 
> Your modified ex42.c is very helpful, thanks!
> Hong
> 
> On Fri, May 4, 2018 at 10:15 AM, Hong <hzhang at mcs.anl.gov <mailto:hzhang at mcs.anl.gov>> wrote:
> Pierre:
> Thanks for creating this use case. I'll check it and get back to you sometime next week.
> Hong
> 
> Hong,
> Thanks for your reply.
> It’s not so minimalist, but at least it is based on one of your example.
> diff --git a/src/ksp/ksp/examples/tutorials/ex42.c b/src/ksp/ksp/examples/tutorials/ex42.c
> index 8be4389a2d..520c82569a 100644
> --- a/src/ksp/ksp/examples/tutorials/ex42.c
> +++ b/src/ksp/ksp/examples/tutorials/ex42.c
> @@ -1953,6 +1953,15 @@ static PetscErrorCode solve_stokes_3d_coupled(PetscInt mx,PetscInt my,PetscInt m
>      }
>    }
>    ierr = KSPSolve(ksp_S,f,X);CHKERRQ(ierr);
> +  PetscViewerPushFormat(PETSC_VIEWER_STDOUT_WORLD, PETSC_VIEWER_ASCII_INFO);
> +  MatView(A, PETSC_VIEWER_STDOUT_WORLD);
> +  Mat C;
> +  MatMatMult(A, A, MAT_INITIAL_MATRIX, PETSC_DEFAULT, &C);
> +  MatView(C, PETSC_VIEWER_STDOUT_WORLD);
> +  PetscViewerPopFormat(PETSC_VIEWER_STDOUT_WORLD);
> +  ierr = KSPSetOperators(ksp_S,C,C);CHKERRQ(ierr);
> +  ierr = KSPSolve(ksp_S,f,X);CHKERRQ(ierr);
> +  MatDestroy(&C);
> 
>    ierr = PetscOptionsGetBool(NULL,NULL,"-write_pvts",&write_output,NULL);CHKERRQ(ierr);
>    if (write_output) {ierr = DAView3DPVTS(da_Stokes,X,"up");CHKERRQ(ierr);}
> 
> $ mpirun -np 1 ./ex42 -mx 2 -my 2 -mz 2 -stokes_pc_type fieldsplit
> This should give you this exact error.
> Do you know of a way to bypass the error by flushing/destroying the PC while still keeping the options for the following KSPSolve?
> 
> Thanks in advance,
> Pierre
> 
>> On 3 May 2018, at 5:07 PM, Hong <hzhang at mcs.anl.gov <mailto:hzhang at mcs.anl.gov>> wrote:
>> 
>> Pierre :
>> The code crashes inside
>> if (scall == MAT_REUSE_MATRIX) {
>> ...
>> } 
>> which assumes both original matrix and its submatrices data structures remain unchanged.
>> We did not anticipate the usage of looping
>> 
>> 3) change the sparsity pattern of the Mat for one of the field
>> 4) do a KSPSolve
>>  
>> Can you send us a stand-alone code for this type of application.
>> We will investigate how to support it.
>> 
>> Hong
>> 
>> I cannot change this calling sequence, because I’d like to nest 3) and 4) inside a loop.
>> However, I end up with this error during 4):
>> [0]PETSC ERROR: Nonconforming object sizes
>> [0]PETSC ERROR: Cannot reuse matrix. wrong no of nonzeros
>> [0]PETSC ERROR: Petsc Development GIT revision: v3.9-13-g05d412bc58  GIT Date: 2018-04-09 08:39:52 -0500
>> 
>> [0]PETSC ERROR: #1 MatCreateSubMatrix_SeqAIJ() line 2330 in /Users/jolivet/Documents/repositories/petsc/src/mat/impls/aij/seq/aij.c
>> [0]PETSC ERROR: #2 MatCreateSubMatrix_MPIAIJ_SameRowColDist() line 3184 in /Users/jolivet/Documents/repositories/petsc/src/mat/impls/aij/mpi/mpiaij.c
>> [0]PETSC ERROR: #3 MatCreateSubMatrix_MPIAIJ() line 3310 in /Users/jolivet/Documents/repositories/petsc/src/mat/impls/aij/mpi/mpiaij.c
>> [0]PETSC ERROR: #4 MatCreateSubMatrix() line 7813 in /Users/jolivet/Documents/repositories/petsc/src/mat/interface/matrix.c
>> [0]PETSC ERROR: #5 PCSetUp_FieldSplit() line 564 in /Users/jolivet/Documents/repositories/petsc/src/ksp/pc/impls/fieldsplit/fieldsplit.c
>> [0]PETSC ERROR: #6 PCSetUp() line 923 in /Users/jolivet/Documents/repositories/petsc/src/ksp/pc/interface/precon.c
>> [0]PETSC ERROR: #7 PCApply() line 453 in /Users/jolivet/Documents/repositories/petsc/src/ksp/pc/interface/precon.c
>> [0]PETSC ERROR: #8 KSP_PCApply() line 276 in /Users/jolivet/repo/petsc/include/petsc/private/kspimpl.h
>> [0]PETSC ERROR: #9 KSPFGMRESCycle() line 166 in /Users/jolivet/Documents/repositories/petsc/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
>> [0]PETSC ERROR: #10 KSPSolve_FGMRES() line 291 in /Users/jolivet/Documents/repositories/petsc/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
>> [0]PETSC ERROR: #11 KSPSolve() line 669 in /Users/jolivet/Documents/repositories/petsc/src/ksp/ksp/interface/itfunc.c
>> 
>> Is this the expected behaviour? Would you need a MWE?
>> What I find funny is that if I turn -info on, then just before the error, I get:
>> [0] PCSetUp(): Setting up PC with different nonzero pattern
>> So why is it trying to reuse the matrix? I am missing a call to some function?
>> 
>> Thanks in advance,
>> Pierre
>> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180508/4fb82be1/attachment.html>