<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">1) PCHPDDM handles AIJ, BAIJ, SBAIJ, IS, NORMAL, NORMALHERMITIAN, SCHURCOMPLEMENT, HTOOL<div>2) This PC is based on domain decomposition, with no support yet for “over decomposition”. If you run with a single process, it’s like PCASM or PCBJACOBI, you’ll get the same behavior as if you were just using the sub PC (in this case, an exact factorization)<br><div>3) The error you are seeing is likely due to a failure while coarsening, I will ask you for some info</div><div>4) Unrelated, but you should probably not use --with-cxx-dialect=C++11 and instead stick to --with-cxx-dialect=11 (unless you have a good reason to)</div><div><br></div><div>Thanks,</div><div>Pierre<br><div><br><blockquote type="cite"><div>On 18 Apr 2023, at 1:26 AM, Alexander Lindsay <alexlindsay239@gmail.com> wrote:</div><br class="Apple-interchange-newline"><div><div dir="ltr">I don't really get much more of a stack trace out:<div><br></div><div>[0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[0]PETSC ERROR: Invalid argument<br>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br> <br>[0]PETSC ERROR: WARNING! There are option(s) set that were not used! Could be the program crashed before they were used or a spelling mistake, etc!<br>[1]PETSC ERROR: Invalid argument<br>[1]PETSC ERROR: <br>[0]PETSC ERROR: Option left: name:-i value: full_upwinding_2D.i source: command line<br>[0]PETSC ERROR: [1]PETSC ERROR: WARNING! There are option(s) set that were not used! Could be the program crashed before they were used or a spelling mistake, etc!<br> Option left: name:-ksp_converged_reason value: ::failed source: code<br>[0]PETSC ERROR: Option left: name:-pc_hpddm_coarse_mat_type value: baij source: command line<br>[0]PETSC ERROR: Option left: name:-pc_hpddm_coarse_pc_type value: lu source: command line<br>[1]PETSC ERROR: Option left: name:-i value: full_upwinding_2D.i source: command line<br>[1]PETSC ERROR: Option left: name:-ksp_converged_reason value: ::failed source: code<br>[0]PETSC ERROR: Option left: name:-snes_converged_reason value: ::failed source: code<br>[0]PETSC ERROR: See <a href="https://petsc.org/release/faq/">https://petsc.org/release/faq/</a> for trouble shooting.<br>[0]PETSC ERROR: Petsc Development GIT revision: v3.17.4-3368-g5a48edb989d GIT Date: 2023-04-16 17:35:24 +0000<br>[1]PETSC ERROR: Option left: name:-pc_hpddm_coarse_mat_type value: baij source: command line<br>[1]PETSC ERROR: Option left: name:-pc_hpddm_coarse_pc_type value: lu source: command line<br>[0]PETSC ERROR: ../../../moose_test-opt on a arch-moose named <a href="http://rod.hpc.inl.gov/">rod.hpc.inl.gov</a> by lindad Mon Apr 17 16:11:09 2023<br>[0]PETSC ERROR: Configure options --download-hypre=1 --with-shared-libraries=1 --download-hdf5=1 --with-hdf5-fortran-bindings=0 --with-debugging=no --download-fblaslapack=1 --download-metis=1 --download-ptscotch=1 --download-parmetis=1 --download-superlu_dist=1 --download-mumps=1 --download-strumpack=1 --download-scalapack=1 --download-slepc=1 --with-mpi=1 --with-openmp=1 --with-cxx-dialect=C++11 --with-fortran-bindings=0 --with-sowing=0 --with-64-bit-indices --with-make-np=256 --download-hpddm<br>[1]PETSC ERROR: Option left: name:-snes_converged_reason value: ::failed source: code<br>[1]PETSC ERROR: See <a href="https://petsc.org/release/faq/">https://petsc.org/release/faq/</a> for trouble shooting.<br>[1]PETSC ERROR: [0]PETSC ERROR: #1 buildTwo() at /raid/lindad/moose/petsc/arch-moose/include/HPDDM_schwarz.hpp:1012<br>Petsc Development GIT revision: v3.17.4-3368-g5a48edb989d GIT Date: 2023-04-16 17:35:24 +0000<br>[1]PETSC ERROR: ../../../moose_test-opt on a arch-moose named <a href="http://rod.hpc.inl.gov/">rod.hpc.inl.gov</a> by lindad Mon Apr 17 16:11:09 2023<br>[1]PETSC ERROR: Configure options --download-hypre=1 --with-shared-libraries=1 --download-hdf5=1 --with-hdf5-fortran-bindings=0 --with-debugging=no --download-fblaslapack=1 --download-metis=1 --download-ptscotch=1 --download-parmetis=1 --download-superlu_dist=1 --download-mumps=1 --download-strumpack=1 --download-scalapack=1 --download-slepc=1 --with-mpi=1 --with-openmp=1 --with-cxx-dialect=C++11 --with-fortran-bindings=0 --with-sowing=0 --with-64-bit-indices --with-make-np=256 --download-hpddm<br>[1]PETSC ERROR: #1 buildTwo() at /raid/lindad/moose/petsc/arch-moose/include/HPDDM_schwarz.hpp:1012<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 17, 2023 at 4:55 PM Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I don't think so. Can you show the whole stack?<div><br></div><div> THanks,</div><div><br></div><div> Matt</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 17, 2023 at 6:24 PM Alexander Lindsay <<a href="mailto:alexlindsay239@gmail.com" target="_blank">alexlindsay239@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">If it helps: if I use those exact same options in serial, then no errors and the linear solve is beautiful :-) </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 17, 2023 at 4:22 PM Alexander Lindsay <<a href="mailto:alexlindsay239@gmail.com" target="_blank">alexlindsay239@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I'm likely revealing a lot of ignorance, but in order to use HPDDM as a preconditioner does my system matrix (I am using the same matrix for A and P) need to be block type, e.g. baij or sbaij ? In MOOSE our default is aij and I am currently getting<div><br><div>[1]PETSC ERROR: #1 buildTwo() at /raid/lindad/moose/petsc/arch-moose/include/HPDDM_schwarz.hpp:1012<br></div><div><br></div><div>with options:</div><div><br></div><div>-pc_type hpddm -pc_hpddm_block_splitting -pc_hpddm_coarse_mat_type baij -pc_hpddm_coarse_pc_type lu -pc_hpddm_define_subdomains -pc_hpddm_levels_1_eps_gen_non_hermitian -pc_hpddm_levels_1_eps_nev 50 -pc_hpddm_levels_1_eps_threshold 0.1 -pc_hpddm_levels_1_st_matstructure SAME -pc_hpddm_levels_1_st_share_sub_ksp -pc_hpddm_levels_1_sub_pc_factor_mat_solver_type mumps -pc_hpddm_levels_1_sub_pc_type lu</div><div><br></div><div>Alex<br></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div>
</blockquote></div>
</div></blockquote></div><br></div></div></body></html>