[petsc-users] Problem setting Fieldsplit fields
Matthew Knepley
knepley at gmail.com
Mon Feb 6 10:57:13 CST 2023
On Mon, Feb 6, 2023 at 11:45 AM Nicolas Barnafi <nbarnafi at cmm.uchile.cl>
wrote:
> Thank you Matt,
>
> Again, at the bottom of this message you will find the -info output. I
> don't see any output related to the fields,
>
If the splits were done automatically, you would see an info message from
here:
https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L1595
Thus it must be setup here
https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L380
There are info statements, but you do not see them, I do not see a way
around using a small example
to understand how you are setting up the system, since it is working as
expected in the PETSc examples.
Thanks,
Matt
> Best
>
>
> ------ -info
>
> [0] <sys> PetscDetermineInitialFPTrap(): Floating point trapping is on by
> default 13
> [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType
> host available, initializing
> [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDevice host
> initialized, default device id 0, view FALSE, init type lazy
> [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType
> cuda not available
> [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType
> hip not available
> [0] <sys> PetscDeviceInitializeTypeFromOptions_Private(): PetscDeviceType
> sycl not available
> [0] <sys> PetscInitialize_Common(): PETSc successfully started: number of
> processors = 1
> [0] <sys> PetscGetHostName(): Rejecting domainname, likely is NIS
> nico-santech.(none)
> [0] <sys> PetscInitialize_Common(): Running on machine: nico-santech
> [0] <sys> SlepcInitialize(): SLEPc successfully started
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960
> 94770087780768 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770087780768
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960
> is being unlinked from inner PETSc comm 94770087780768
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770087780768
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960
> 94770087780768 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770087780768
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960
> is being unlinked from inner PETSc comm 94770087780768
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770087780768
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960
> 94770087780768 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770087780768
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960
> is being unlinked from inner PETSc comm 94770087780768
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770087780768
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960
> 94770087780768 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770087780768
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960
> is being unlinked from inner PETSc comm 94770087780768
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770087780768
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960
> 94770087780768 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770087780768
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066936960
> is being unlinked from inner PETSc comm 94770087780768
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770087780768
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770087780768
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066936960
> 94770087780768 max tags = 2147483647
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066936960 94770087780768
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 1219 X 1219; storage
> space: 0 unneeded,26443 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 150
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 1219) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 1160 nodes out of 1219 rows. Not
> using Inode routines
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066936960 94770087780768
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066936960 94770087780768
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066936960 94770087780768
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066936960 94770087780768
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066936960 94770087780768
> [0] <sys> PetscGetHostName(): Rejecting domainname, likely is NIS
> nico-santech.(none)
> [0] <pc> PCSetUp(): Setting up PC for first time
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 615 X 615; storage space:
> 0 unneeded,9213 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 117
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 615) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 561 nodes out of 615 rows. Not
> using Inode routines
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048
> 94770110251424 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770110251424
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048
> is being unlinked from inner PETSc comm 94770110251424
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770110251424
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770110251424
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 64 X 64; storage space: 0
> unneeded,0 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 0
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 64)/(num_localrows 64) > 0.6. Use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 13 nodes of 64. Limit used: 5.
> Using Inode routines
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048
> 94770100861088 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770100861088
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048
> is being unlinked from inner PETSc comm 94770100861088
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770100861088
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770100861088
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 240 X 240; storage space:
> 0 unneeded,2140 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 11
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 240) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 235 nodes out of 240 rows. Not
> using Inode routines
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048
> 94770100861088 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770100861088
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048
> is being unlinked from inner PETSc comm 94770100861088
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770100861088
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770100861088
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 300 X 300; storage space:
> 0 unneeded,2292 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 33
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 300) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 300 nodes out of 300 rows. Not
> using Inode routines
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048
> 94770100861088 max tags = 2147483647
> [0] <sys> Petsc_OuterComm_Attr_Delete_Fn(): Removing reference to PETSc
> communicator embedded in a user MPI_Comm 94770100861088
> [0] <sys> Petsc_InnerComm_Attr_Delete_Fn(): User MPI_Comm 94770066934048
> is being unlinked from inner PETSc comm 94770100861088
> [0] <sys> PetscCommDestroy(): Deleting PETSc MPI_Comm 94770100861088
> [0] <sys> Petsc_Counter_Attr_Delete_Fn(): Deleting counter data in an
> MPI_Comm 94770100861088
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 615 X 1219; storage space:
> 0 unneeded,11202 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 150
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 615) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 561 nodes out of 615 rows. Not
> using Inode routines
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 64 X 1219; storage space:
> 0 unneeded,288 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 6
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 64) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 64 nodes out of 64 rows. Not using
> Inode routines
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 240 X 1219; storage space:
> 0 unneeded,8800 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 78
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 240) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 235 nodes out of 240 rows. Not
> using Inode routines
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 300 X 1219; storage space:
> 0 unneeded,6153 used
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues()
> is 0
> [0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 89
> [0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows
> 0)/(num_localrows 300) < 0.6. Do not use CompressedRow routines.
> [0] <mat> MatSeqAIJCheckInode(): Found 300 nodes out of 300 rows. Not
> using Inode routines
> [0] <sys> PetscCommDuplicate(): Duplicating a communicator 94770066934048
> 94770100861088 max tags = 2147483647
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 94770100861088
> 0 KSP Residual norm 2.541418258630e+01
> [0] <ksp> KSPConvergedDefault(): user has provided nonzero initial guess,
> computing 2-norm of RHS
> [0] <pc> PCSetUp(): Leaving PC with identical preconditioner since
> operator is unchanged
> [0] <pc> PCSetUp(): Leaving PC with identical preconditioner since
> operator is unchanged
> [0] <pc> PCSetUp(): Setting up PC for first time
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 94770100861088
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 94770100861088
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 94770100861088
> [0] <pc> PCSetUp(): Leaving PC with identical preconditioner since
> operator is unchanged
> [0] <pc> PCSetUp(): Setting up PC for first time
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 94770100861088
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 94770100861088
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 94770100861088
> [0] <sys> PetscCommDuplicate(): Using internal PETSc communicator
> 94770066934048 947701008610882
>
>
> On 03/02/23 21:11, Matthew Knepley wrote:
>
> On Fri, Feb 3, 2023 at 3:03 PM Nicolas Barnafi <nbarnafi at cmm.uchile.cl>
> wrote:
>
>> > There are a number of common errors:
>> >
>> > 1) Your PC has a prefix
>> >
>> > 2) You have not called KSPSetFromOptions() here
>> >
>> > Can you send the -ksp_view output?
>>
>> The PC at least has no prefix. I had to set ksp_rtol to 1 to get through
>> the solution process, you will find both the petsc_rc and the ksp_view
>> at the bottom of this message.
>>
>> Options are indeed being set from the options file, but there must be
>> something missing at a certain level. Thanks for looking into this.
>>
>
> Okay, the next step is to pass
>
> -info
>
> and send the output. This will tell us how the default splits were done.
> If that
> is not conclusive, we will have to use the debugger.
>
> Thanks,
>
> Matt
>
>
>> Best
>>
>> ---- petsc_rc file
>>
>> -ksp_monitor
>> -ksp_type gmres
>> -ksp_view
>> -mat_type aij
>> -ksp_norm_type unpreconditioned
>> -ksp_atol 1e-14
>> -ksp_rtol 1
>> -pc_type fieldsplit
>> -pc_fieldsplit_type multiplicative
>>
>> ---- ksp_view
>>
>> KSP Object: 1 MPI process
>> type: gmres
>> restart=500, using Classical (unmodified) Gram-Schmidt
>> Orthogonalization with no iterative refinement
>> happy breakdown tolerance 1e-30
>> maximum iterations=10000, nonzero initial guess
>> tolerances: relative=1., absolute=1e-14, divergence=10000.
>> right preconditioning
>> using UNPRECONDITIONED norm type for convergence test
>> PC Object: 1 MPI process
>> type: fieldsplit
>> FieldSplit with MULTIPLICATIVE composition: total splits = 4
>> Solver info for each split is in the following KSP objects:
>> Split number 0 Defined by IS
>> KSP Object: (fieldsplit_0_) 1 MPI process
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using DEFAULT norm type for convergence test
>> PC Object: (fieldsplit_0_) 1 MPI process
>> type: ilu
>> PC has not been set up so information may be incomplete
>> out-of-place factorization
>> 0 levels of fill
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: natural
>> matrix solver type: petsc
>> matrix not yet factored; no additional information available
>> linear system matrix = precond matrix:
>> Mat Object: (fieldsplit_0_) 1 MPI process
>> type: seqaij
>> rows=615, cols=615
>> total: nonzeros=9213, allocated nonzeros=9213
>> total number of mallocs used during MatSetValues calls=0
>> not using I-node routines
>> Split number 1 Defined by IS
>> KSP Object: (fieldsplit_1_) 1 MPI process
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using DEFAULT norm type for convergence test
>> PC Object: (fieldsplit_1_) 1 MPI process
>> type: ilu
>> PC has not been set up so information may be incomplete
>> out-of-place factorization
>> 0 levels of fill
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: natural
>> matrix solver type: petsc
>> matrix not yet factored; no additional information available
>> linear system matrix = precond matrix:
>> Mat Object: (fieldsplit_1_) 1 MPI process
>> type: seqaij
>> rows=64, cols=64
>> total: nonzeros=0, allocated nonzeros=0
>> total number of mallocs used during MatSetValues calls=0
>> using I-node routines: found 13 nodes, limit used is 5
>> Split number 2 Defined by IS
>> KSP Object: (fieldsplit_2_) 1 MPI process
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using DEFAULT norm type for convergence test
>> PC Object: (fieldsplit_2_) 1 MPI process
>> type: ilu
>> PC has not been set up so information may be incomplete
>> out-of-place factorization
>> 0 levels of fill
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: natural
>> matrix solver type: petsc
>> matrix not yet factored; no additional information available
>> linear system matrix = precond matrix:
>> Mat Object: (fieldsplit_2_) 1 MPI process
>> type: seqaij
>> rows=240, cols=240
>> total: nonzeros=2140, allocated nonzeros=2140
>> total number of mallocs used during MatSetValues calls=0
>> not using I-node routines
>> Split number 3 Defined by IS
>> KSP Object: (fieldsplit_3_) 1 MPI process
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using DEFAULT norm type for convergence test
>> PC Object: (fieldsplit_3_) 1 MPI process
>> type: ilu
>> PC has not been set up so information may be incomplete
>> out-of-place factorization
>> 0 levels of fill
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: natural
>> matrix solver type: petsc
>> matrix not yet factored; no additional information available
>> linear system matrix = precond matrix:
>> Mat Object: (fieldsplit_3_) 1 MPI process
>> type: seqaij
>> rows=300, cols=300
>> total: nonzeros=2292, allocated nonzeros=2292
>> total number of mallocs used during MatSetValues calls=0
>> not using I-node routines
>> linear system matrix = precond matrix:
>> Mat Object: 1 MPI process
>> type: seqaij
>> rows=1219, cols=1219
>> total: nonzeros=26443, allocated nonzeros=26443
>> total number of mallocs used during MatSetValues calls=0
>> not using I-node routines
>> solving time: 0.00449609
>> iterations: 0
>> estimated error: 25.4142
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230206/315bf385/attachment-0001.html>
More information about the petsc-users
mailing list