<div dir="ltr"><div dir="ltr"><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div>Jed, Junchao,</div><div><br></div><div>Thanks for getting me back! </div><div> </div><div>Junchao, I cannot provide you with an input file as I am plugging just part of that code into mine. I am building blocks of the full matrix on my own, not reading from the file unfortunately. I can share a branch of my code with a 3x3 cavity toy case, but that would require compiling it and it might be a bit inconvenient for you. </div><div><br></div><div>Jed, I am using fieldsplit indeed, it's just the matrix that is assembled in a 2x2 block form. Here is the full backtrace:</div><div><br></div></div></div></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: No support for this operation for this object type</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: No method getinfo for Mat of type nest</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: WARNING! There are unused option(s) set! Could be the program crashed before usage or a spelling mistake, etc!</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Option left: name:-UPeqn_fieldsplit_p_pc_type value: hypre source: command line</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Option left: name:-UPeqn_fieldsplit_u_pc_type value: bjacobi source: command line</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: See <a href="https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!a6ejiT9ML4BpeAAcqVuIMX-J7sA75OiyYcHeUd6-INXK8sR-gcQPA3ndqsgpzphK4tIBnE13dbzycfmUkwST7aXF6oxXipY$">https://petsc.org/release/faq/</a> for trouble shooting.</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Petsc Release Version 3.20.4, unknown </i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: flubio_coupled on a linux_x86_64 named localhost.localdomain by edo Thu Oct 3 14:17:20 2024</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Configure options PETSC_ARCH=linux_x86_64 FOPTFLAGS=-O3 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 -with-debugging=no -download-fblaslapack=1 -download-superlu_dist -download-mumps -download-hypre -download-metis -download-parmetis -download-scalapack -download-ml -download-slepc -download-hpddm -download-cmake -with-mpi-dir=/home/edo/software_repo/openmpi-5.0.1/build/</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #1 MatGetInfo() at /home/edo/software_repo/petsc/src/mat/interface/matrix.c:3005</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #2 PCSetUp_GAMG() at /home/edo/software_repo/petsc/src/ksp/pc/impls/gamg/gamg.c:619</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #3 PCSetUp() at /home/edo/software_repo/petsc/src/ksp/pc/interface/precon.c:1080</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: It appears a new error in the code was triggered after a previous error, possibly because:</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: - The first error was not properly handled via (for example) the use of</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: PetscCall(TheFunctionThatErrors()); or</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: - The second error was triggered while handling the first error.</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Above is the traceback for the previous unhandled error, below the traceback for the next error</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: ALL ERRORS in the PETSc libraries are fatal, you should add the appropriate error checking to the code</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: No support for this operation for this object type</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: No method getinfo for Mat of type nest</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: WARNING! There are unused option(s) set! Could be the program crashed before usage or a spelling mistake, etc!</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Option left: name:-UPeqn_fieldsplit_p_pc_type value: hypre source: command line</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Option left: name:-UPeqn_fieldsplit_u_pc_type value: bjacobi source: command line</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: See <a href="https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!a6ejiT9ML4BpeAAcqVuIMX-J7sA75OiyYcHeUd6-INXK8sR-gcQPA3ndqsgpzphK4tIBnE13dbzycfmUkwST7aXF6oxXipY$">https://petsc.org/release/faq/</a> for trouble shooting.</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Petsc Release Version 3.20.4, unknown </i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: flubio_coupled on a linux_x86_64 named localhost.localdomain by edo Thu Oct 3 14:17:20 2024</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Configure options PETSC_ARCH=linux_x86_64 FOPTFLAGS=-O3 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 -with-debugging=no -download-fblaslapack=1 -download-superlu_dist -download-mumps -download-hypre -download-metis -download-parmetis -download-scalapack -download-ml -download-slepc -download-hpddm -download-cmake -with-mpi-dir=/home/edo/software_repo/openmpi-5.0.1/build/</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #1 MatGetInfo() at /home/edo/software_repo/petsc/src/mat/interface/matrix.c:3005</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #2 PCSetUp_GAMG() at /home/edo/software_repo/petsc/src/ksp/pc/impls/gamg/gamg.c:619</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #3 PCSetUp() at /home/edo/software_repo/petsc/src/ksp/pc/interface/precon.c:1080</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #4 KSPSetUp() at /home/edo/software_repo/petsc/src/ksp/ksp/interface/itfunc.c:415</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: It appears a new error in the code was triggered after a previous error, possibly because:</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: - The first error was not properly handled via (for example) the use of</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: PetscCall(TheFunctionThatErrors()); or</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: - The second error was triggered while handling the first error.</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Above is the traceback for the previous unhandled error, below the traceback for the next error</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: ALL ERRORS in the PETSc libraries are fatal, you should add the appropriate error checking to the code</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: No support for this operation for this object type</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: No method getinfo for Mat of type nest</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: WARNING! There are unused option(s) set! Could be the program crashed before usage or a spelling mistake, etc!</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Option left: name:-UPeqn_fieldsplit_p_pc_type value: hypre source: command line</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Option left: name:-UPeqn_fieldsplit_u_pc_type value: bjacobi source: command line</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: See <a href="https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!a6ejiT9ML4BpeAAcqVuIMX-J7sA75OiyYcHeUd6-INXK8sR-gcQPA3ndqsgpzphK4tIBnE13dbzycfmUkwST7aXF6oxXipY$">https://petsc.org/release/faq/</a> for trouble shooting.</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Petsc Release Version 3.20.4, unknown </i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: flubio_coupled on a linux_x86_64 named localhost.localdomain by edo Thu Oct 3 14:17:20 2024</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: Configure options PETSC_ARCH=linux_x86_64 FOPTFLAGS=-O3 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 -with-debugging=no -download-fblaslapack=1 -download-superlu_dist -download-mumps -download-hypre -download-metis -download-parmetis -download-scalapack -download-ml -download-slepc -download-hpddm -download-cmake -with-mpi-dir=/home/edo/software_repo/openmpi-5.0.1/build/</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #1 MatGetInfo() at /home/edo/software_repo/petsc/src/mat/interface/matrix.c:3005</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #2 PCSetUp_GAMG() at /home/edo/software_repo/petsc/src/ksp/pc/impls/gamg/gamg.c:619</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #3 PCSetUp() at /home/edo/software_repo/petsc/src/ksp/pc/interface/precon.c:1080</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #4 KSPSetUp() at /home/edo/software_repo/petsc/src/ksp/ksp/interface/itfunc.c:415</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #5 KSPSolve_Private() at /home/edo/software_repo/petsc/src/ksp/ksp/interface/itfunc.c:832</i></div></div></div></div></div></div><div><div><div class="gmail_signature"><div><div><div><i>[0]PETSC ERROR: #6 KSPSolve() at /home/edo/software_repo/petsc/src/ksp/ksp/interface/itfunc.c:1079</i></div></div></div></div></div></div></blockquote><div dir="ltr"><br></div><div>This the code I am doing right now:</div><div><br></div><div><i> M = this%setMomentumBlockMatrix()<br> C = this%setPressureBlockMatrix()<br> G = this%setPressureGradBlockMatrix()<br> D = this%setDivBlockMatrix()<br> bhat = this%setPressureMomentumRhs()<br><br> !------------------!<br> ! Create Ahat !<br> !-----------------!<br><br> ! Bottom left block of Ahat<br> Mat_array(4) = M <br><br> ! Extraction of the diagonal of M<br> call MatCreateVecs(M, PETSC_NULL_VEC, v, ierr) !v has the parallel distribution of M<br> call MatGetDiagonal(M, v, ierr)<br><br> call MatCreateDiagonal(v, diag_2M, ierr)<br> call MatScale(diag_2M, 2.0, ierr) ! store 2*diagonal part of M<br> call MatConvert(diag_2M, MATAIJ, MAT_INPLACE_MATRIX, diag_2M, ierr)<br> call VecReciprocal(v, ierr)<br><br> ! Creation of D_M_inv_G = D_M_inv*G<br> call MatDuplicate(G,MAT_COPY_VALUES, D_M_inv_G, ierr) ! D_M_inv_G contains G<br> call MatCreateVecs(D_M_inv_G, PETSC_NULL_VEC, v_redistributed, ierr) ! v_redistributed has the parallel distribution of D_M_inv_G<br><br> call VecGetOwnershipRange(v, col_min, col_max, ierr) <br> call ISCreateStride(PETSC_COMM_WORLD, col_max-col_min, col_min, 1, is_from, ierr)<br> call VecGetOwnershipRange(v_redistributed, col_min, col_max, ierr)<br> call ISCreateStride(PETSC_COMM_WORLD, col_max-col_min, col_min, 1, is_to, ierr)<br><br> call VecScatterCreate(v,is_from,v_redistributed,is_to, scat, ierr)<br> call VecScatterBegin(scat, v, v_redistributed,INSERT_VALUES,SCATTER_FORWARD, ierr)<br> call VecScatterEnd( scat, v, v_redistributed,INSERT_VALUES,SCATTER_FORWARD, ierr)<br> call MatDiagonalScale( D_M_inv_G, v_redistributed, PETSC_NULL_VEC, ierr) ! D_M_inv_G contains D_M_inv*G<br><br> call ISDestroy(is_to, ierr)<br> call ISDestroy(is_from, ierr)<br><br> ! Creation of Chat (PETSC_DEFAULT_REAL -> PETSC_DETERMINE)<br> call MatMatMult(D, D_M_inv_G, MAT_INITIAL_MATRIX, PETSC_DEFAULT_REAL, Chat, ierr) ! Chat contains D*D_M_inv*G<br> call MatAXPY( Chat, 1.0, C, SUBSET_NONZERO_PATTERN, ierr) ! Chat contains C + D*D_M_inv*G<br> Mat_array(1) = Chat ! Top left block of Ahat<br> <br> ! Creation of Ghat<br> call MatMatMult(M, D_M_inv_G, MAT_INITIAL_MATRIX, PETSC_DEFAULT_REAL, Ghat, ierr) ! Ghat contains M*D_M_inv*G<br> call MatAYPX( Ghat, -1.0, G, UNKNOWN_NONZERO_PATTERN, ierr) ! Ghat contains G - M*D_M_inv*G<br> Mat_array(3) = Ghat ! Bottom left block of Ahat<br> <br> ! Creation of -D<br> call MatScale(D,-1.0, ierr)<br> Mat_array(2) = D ! Top right block of Ahat<br> <br> ! Creation of Ahat = transformed+ reordered A_input<br> call MatCreateNest(PETSC_COMM_WORLD, 2, PETSC_NULL_IS, 2, PETSC_NULL_IS, Mat_array, Ahat, ierr)<br> <br> ! Creation of Pmat<br> Mat_array(4) = diag_2M<br> Mat_array(2) = PETSC_NULL_MAT ! Cancel top right block<br> call MatCreateNest(PETSC_COMM_WORLD, 2, PETSC_NULL_IS, 2, PETSC_NULL_IS, Mat_array, Pmat, ierr);<br> <br> ! Finalisation of the preconditioner<br> call ISCreateStride(PETSC_COMM_WORLD, numberOfElements, 0, 1, is_P_hat, ierr); <br> call ISCreateStride(PETSC_COMM_WORLD, (3-bdim)*numberOfElements, numberOfElements, 1, is_U_hat, ierr);<br><br> !----------------!<br> ! KSP PART !<br> !----------------!<br><br> call KSPSetOperators(this%ksp, Ahat, Pmat, ierr)<br> call KSPGetPC(this%ksp, pc, ierr)<br><br> call PCSetFromOptions(pc, ierr)<br> call PCSetUp(pc, ierr)<br> call KSPSetFromOptions(this%ksp, ierr)<br> call KSPSetUp(this%ksp, ierr)<br><br> call VecDuplicate(bhat, xhat, ierr)<br><br> ! I think it makes sense to move this part where fieldsplit preconditioner is defined<br> call PCFieldSplitSetIS(pc, "phat", is_P_hat, ierr)<br> call PCFieldSplitSetIS(pc, "uhat", is_U_hat, ierr)<br><br> call KSPSolve(this%ksp, bhat, xhat, ierr)</i><br></div><div><br></div><div>KSP id FGMRES + fieldsplit with default settings.</div><div><br></div><div>Many thanks!</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno gio 3 ott 2024 alle ore 19:01 Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com">junchao.zhang@gmail.com</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Could you also provide input files to run your code?<div><br clear="all"><div><div dir="ltr" class="gmail_signature"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 3, 2024 at 5:14 AM Edoardo alinovi <<a href="mailto:edoardo.alinovi@gmail.com" target="_blank">edoardo.alinovi@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div>Hi Barry,</div><div><br></div><div>I am trying to port this preconditioner I found presented at the last PETSc conference into may code:</div><div><a href="https://urldefense.us/v3/__https://github.com/ndjinga/SOLVERLAB/blob/master/CoreFlows/examples/C/SaddlePointPreconditioner/SaddlePointLinearSolver.c__;!!G_uCfscf7eWS!a-JnrBsta75n_WDtxUNd7Xb8_m7tg1pWSxwWNTmUBSfpBwISq4IxJkbM6QgADd-x23Q_2s-d6d6bOWCKGRMxMzHyc9DDm7A$" target="_blank">https://github.com/ndjinga/SOLVERLAB/blob/master/CoreFlows/examples/C/SaddlePointPreconditioner/SaddlePointLinearSolver.c</a></div><div><br></div><div>So far so good, I am just translating C to Fortran, however when I try to solve my linear system with: </div><div><br></div></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_signature"><div><div><div>call KSPSetOperators(ksp, Ahat, Pmat, ierr)</div></div></div></div><div class="gmail_signature"><div><div><div>call KSPSolve(ksp, bhat, xhat, ierr)</div></div></div><div><br></div></div></blockquote><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div>with Ahat and Pmat being indeed two nest MatNest, created from an array of Mat as:</div><div><br></div></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_signature"><div><div><div>call MatCreateNest(PETSC_COMM_WORLD, 2, PETSC_NULL_IS, 2, PETSC_NULL_IS, Mat_array, Ahat, ierr)</div></div></div></div><div class="gmail_signature"><div><div><div>call MatCreateNest(PETSC_COMM_WORLD, 2, PETSC_NULL_IS, 2, PETSC_NULL_IS, Mat_array, Pmat, ierr)</div></div></div></div></blockquote><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div><br></div><div>However I am getting this error on getinfo:</div><div><br></div><div>from C to Fortran</div><div><br></div><div>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[0]PETSC ERROR: No support for this operation for this object type<br>[0]PETSC ERROR: No method getinfo for Mat of type nest<br>[0]PETSC ERROR: See <a href="https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!a-JnrBsta75n_WDtxUNd7Xb8_m7tg1pWSxwWNTmUBSfpBwISq4IxJkbM6QgADd-x23Q_2s-d6d6bOWCKGRMxMzHyRUMnyvE$" target="_blank">https://petsc.org/release/faq/</a> for trouble shooting.<br>[0]PETSC ERROR: Petsc Release Version 3.20.4, unknown <br></div><div><br></div><div>How can I get rid of this issue?</div><div><br></div><div>Many thanks!</div><div><br></div></div></div></div></div>
</blockquote></div>
</blockquote></div></div>