[petsc-users] PCCOMPOSITE with PCBJACOBI
Robert Annewandter
robert.annewandter at opengosim.com
Tue Jun 27 11:24:41 CDT 2017
Dear PETSc folks,
I want a Block Jacobi PC to be the second PC in a two-stage
preconditioning scheme implemented via multiplicative PCCOMPOSITE, with
the outermost KSP an FGMRES.
However, PCBJacobiGetSubKSP
(https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCBJacobiGetSubKSP.html#PCBJacobiGetSubKSP)
requires to call KSPSetUp (or PCSetUp) first on its parent KSP, which I
struggle in succeeding. I wonder which KSP (or if so PC) that is.
This is how I attempt to do it (using PCKSP to provide a parent KSP for
PCBJacobiGetSubKSP):
call KSPGetPC(solver%ksp, solver%pc, ierr); CHKERRQ(ierr)
call PCSetType(solver%pc, PCCOMPOSITE, ierr); CHKERRQ(ierr)
call PCCompositeSetType(solver%pc, PC_COMPOSITE_MULTIPLICATIVE, ierr);
CHKERRQ(ierr)
! 1st Stage
call PCCompositeAddPC(solver%pc, PCGALERKIN, ierr); CHKERRQ(ierr)
call PCCompositeGetPC(solver%pc, 0, T1, ierr); CHKERRQ(ierr)
! KSPPREONLY-PCNONE for testing
call PCGalerkinGetKSP(T1, Ap_ksp, ierr); CHKERRQ(ierr)
call KSPSetType(Ap_ksp, KSPPREONLY, ierr); CHKERRQ(ierr)
call KSPGetPC(Ap_ksp, Ap_pc, ierr); CHKERRQ(ierr)
call PCSetType(Ap_pc, PCNONE, ierr); CHKERRQ(ierr)
! 2nd Stage
call PCCompositeAddPC(solver%pc, PCKSP, ierr); CHKERRQ(ierr)
call PCCompositeGetPC(solver%pc, 1, T2, ierr); CHKERRQ(ierr)
call PCKSPGetKSP(T2, BJac_ksp, ierr); CHKERRQ(ierr)
call KSPSetType(BJac_ksp, KSPPREONLY, ierr); CHKERRQ(ierr)
call KSPGetPC(BJac_ksp, BJac_pc, ierr); CHKERRQ(ierr)
call PCSetType(BJac_pc, PCBJACOBI, ierr); CHKERRQ(ierr)
call KSPSetUp(solver%ksp, ierr); CHKERRQ(ierr)
! call KSPSetUp(BJac_ksp, ierr); CHKERRQ(ierr)
! call PCSetUp(T2, ierr); CHKERRQ(ierr)
! call PCSetUp(BJac_pc, ierr); CHKERRQ(ierr)
call PCBJacobiGetSubKSP(BJac_pc, nsub_ksp, first_sub_ksp,
PETSC_NULL_KSP, ierr); CHKERRQ(ierr)
allocate(sub_ksps(nsub_ksp))
call PCBJacobiGetSubKSP(BJac_pc, nsub_ksp, first_sub_ksp,
sub_ksps,ierr); CHKERRQ(ierr)
do i = 1, nsub_ksp
call KSPGetPC(sub_ksps(i), BJac_pc_sub, ierr); CHKERRQ(ierr)
call PCFactorSetShiftType(BJac_pc_sub, MAT_SHIFT_INBLOCKS, ierr);
CHKERRQ(ierr)
call PCFactorSetZeroPivot(BJac_pc_sub, solver%linear_zero_pivot_tol,
ierr); CHKERRQ(ierr)
end do
deallocate(sub_ksps)
nullify(sub_ksps)
Is using PCKSP a good idea at all?
With KSPSetUp(solver%ksp) -> FGMRES
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: You requested a vector from a KSP that cannot provide one
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.7.5-3167-g03c0fad
GIT Date: 2017-03-30 14:27:53 -0500
[0]PETSC ERROR: pflotran on a debug_g-6.2 named mother by pujjad Tue Jun
27 16:55:14 2017
[0]PETSC ERROR: Configure options --download-mpich=yes
--download-hdf5=yes --download-fblaslapack=yes --download-metis=yes
--download-parmetis=yes --download-eigen=yes --download-hypre=yes
--download-superlu_dist=yes --download-superlu=yes --with-cc=gcc-6
--with-cxx=g++-6 --with-fc=gfortran-6 PETSC_ARCH=debug_g-6.2
PETSC_DIR=/home/pujjad/Repositories/petsc
[0]PETSC ERROR: #1 KSPCreateVecs() line 939 in
/home/pujjad/Repositories/petsc/src/ksp/ksp/interface/iterativ.c
[0]PETSC ERROR: #2 KSPSetUp_GMRES() line 85 in
/home/pujjad/Repositories/petsc/src/ksp/ksp/impls/gmres/gmres.c
[0]PETSC ERROR: #3 KSPSetUp_FGMRES() line 41 in
/home/pujjad/Repositories/petsc/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
[0]PETSC ERROR: #4 KSPSetUp() line 338 in
/home/pujjad/Repositories/petsc/src/ksp/ksp/interface/itfunc.c
application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0
[mpiexec at mother] handle_pmi_cmd (./pm/pmiserv/pmiserv_cb.c:52):
Unrecognized PMI command: abort | cleaning up processes
[mpiexec at mother] control_cb (./pm/pmiserv/pmiserv_cb.c:289): unable to
process PMI command
[mpiexec at mother] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status
[mpiexec at mother] HYD_pmci_wait_for_completion
(./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event
[mpiexec at mother] main (./ui/mpich/mpiexec.c:405): process manager error
waiting for completion
With KSPSetUp(BJac_ksp) -> KSPPREONLY
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Arguments are incompatible
[0]PETSC ERROR: Both n and N cannot be PETSC_DECIDE
likely a call to VecSetSizes() or MatSetSizes() is wrong.
See http://www.mcs.anl.gov/petsc/documentation/faq.html#split
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.7.5-3167-g03c0fad
GIT Date: 2017-03-30 14:27:53 -0500
[0]PETSC ERROR: pflotran on a debug_g-6.2 named mother by pujjad Tue Jun
27 16:52:57 2017
[0]PETSC ERROR: Configure options --download-mpich=yes
--download-hdf5=yes --download-fblaslapack=yes --download-metis=yes
--download-parmetis=yes --download-eigen=yes --download-hypre=yes
--download-superlu_dist=yes --download-superlu=yes --with-cc=gcc-6
--with-cxx=g++-6 --with-fc=gfortran-6 PETSC_ARCH=debug_g-6.2
PETSC_DIR=/home/pujjad/Repositories/petsc
[0]PETSC ERROR: #1 PetscSplitOwnership() line 77 in
/home/pujjad/Repositories/petsc/src/sys/utils/psplit.c
[0]PETSC ERROR: #2 PetscLayoutSetUp() line 137 in
/home/pujjad/Repositories/petsc/src/vec/is/utils/pmap.c
[0]PETSC ERROR: #3 VecCreate_Seq_Private() line 847 in
/home/pujjad/Repositories/petsc/src/vec/vec/impls/seq/bvec2.c
[0]PETSC ERROR: #4 VecCreateSeqWithArray() line 899 in
/home/pujjad/Repositories/petsc/src/vec/vec/impls/seq/bvec2.c
[0]PETSC ERROR: #5 PCSetUp_BJacobi_Singleblock() line 786 in
/home/pujjad/Repositories/petsc/src/ksp/pc/impls/bjacobi/bjacobi.c
[0]PETSC ERROR: #6 PCSetUp_BJacobi() line 136 in
/home/pujjad/Repositories/petsc/src/ksp/pc/impls/bjacobi/bjacobi.c
[0]PETSC ERROR: #7 PCSetUp() line 924 in
/home/pujjad/Repositories/petsc/src/ksp/pc/interface/precon.c
[0]PETSC ERROR: #8 KSPSetUp() line 379 in
/home/pujjad/Repositories/petsc/src/ksp/ksp/interface/itfunc.c
[mpiexec at mother] handle_pmi_cmd (./pm/pmiserv/pmiserv_cb.c:52):
Unrecognized PMI command: abort | cleaning up processes
[mpiexec at mother] control_cb (./pm/pmiserv/pmiserv_cb.c:289): unable to
process PMI command
[mpiexec at mother] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status
[mpiexec at mother] HYD_pmci_wait_for_completion
(./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event
[mpiexec at mother] main (./ui/mpich/mpiexec.c:405): process manager error
waiting for completion
[mpiexec at mother] handle_pmi_cmd (./pm/pmiserv/pmiserv_cb.c:52):
Unrecognized PMI command: abort | cleaning up processes
[mpiexec at mother] control_cb (./pm/pmiserv/pmiserv_cb.c:289): unable to
process PMI command
[mpiexec at mother] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status
[mpiexec at mother] HYD_pmci_wait_for_completion
(./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event
[mpiexec at mother] main (./ui/mpich/mpiexec.c:405): process manager error
waiting for completion
With PCSetUp(T2) -> PCKSP
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Matrix must be set first
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.7.5-3167-g03c0fad
GIT Date: 2017-03-30 14:27:53 -0500
[0]PETSC ERROR: pflotran on a debug_g-6.2 named mother by pujjad Tue Jun
27 16:51:23 2017
[0]PETSC ERROR: Configure options --download-mpich=yes
--download-hdf5=yes --download-fblaslapack=yes --download-metis=yes
--download-parmetis=yes --download-eigen=yes --download-hypre=yes
--download-superlu_dist=yes --download-superlu=yes --with-cc=gcc-6
--with-cxx=g++-6 --with-fc=gfortran-6 PETSC_ARCH=debug_g-6.2
PETSC_DIR=/home/pujjad/Repositories/petsc
[0]PETSC ERROR: #1 PCSetUp() line 888 in
/home/pujjad/Repositories/petsc/src/ksp/pc/interface/precon.c
application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0
[mpiexec at mother] handle_pmi_cmd (./pm/pmiserv/pmiserv_cb.c:52):
Unrecognized PMI command: abort | cleaning up processes
[mpiexec at mother] control_cb (./pm/pmiserv/pmiserv_cb.c:289): unable to
process PMI command
[mpiexec at mother] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status
[mpiexec at mother] HYD_pmci_wait_for_completion
(./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event
[mpiexec at mother] main (./ui/mpich/mpiexec.c:405): process manager error
waiting for completion
With PCSetUp(BJac_pc) -> PCBJACOBI
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Matrix must be set first
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.7.5-3167-g03c0fad
GIT Date: 2017-03-30 14:27:53 -0500
[0]PETSC ERROR: pflotran on a debug_g-6.2 named mother by pujjad Tue Jun
27 16:42:10 2017
[0]PETSC ERROR: Configure options --download-mpich=yes
--download-hdf5=yes --download-fblaslapack=yes --download-metis=yes
--download-parmetis=yes --download-eigen=yes --download-hypre=yes
--download-superlu_dist=yes --download-superlu=yes --with-cc=gcc-6
--with-cxx=g++-6 --with-fc=gfortran-6 PETSC_ARCH=debug_g-6.2
PETSC_DIR=/home/pujjad/Repositories/petsc
[0]PETSC ERROR: #1 PCSetUp() line 888 in
/home/pujjad/Repositories/petsc/src/ksp/pc/interface/precon.c
[mpiexec at mother] handle_pmi_cmd (./pm/pmiserv/pmiserv_cb.c:52):
Unrecognized PMI command: abort | cleaning up processes
[mpiexec at mother] control_cb (./pm/pmiserv/pmiserv_cb.c:289): unable to
process PMI command
[mpiexec at mother] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status
[mpiexec at mother] HYD_pmci_wait_for_completion
(./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event
[mpiexec at mother] main (./ui/mpich/mpiexec.c:405): process manager error
waiting for completion
Grateful for any help!
Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170627/c140885a/attachment.html>
More information about the petsc-users
mailing list