<div dir="ltr">Hi all,<div><br></div><div>I'm solving a Navier Stokes system. I am having a problem whereby when I try to use asm as a preconditioner and make the number of blocks more than one per mpi proc I get an error from within parmetis (when I attempt the same with bjacobi I don't get an error). Also when I run with exactly one block per processor there are no issues. Any tips? Error message below.</div>
<div><br></div><div>Cheers,</div><div>James</div><div><br></div><div><div>PARMETIS ERROR: The sum of tpwgts for constraint #0 is not 1.0</div><div>PARMETIS ERROR: The sum of tpwgts for constraint #0 is not 1.0</div><div>
--------------------------------------------------------------------------</div>
<div>MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD </div><div>with errorcode 76.</div><div><br></div><div>NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.</div><div>You may or may not see output from other processes, depending on</div>
<div>exactly when Open MPI kills them.</div><div>--------------------------------------------------------------------------</div><div>[0]PETSC ERROR: --------------------- Error Message ------------------------------------</div>
<div>[1]PETSC ERROR: --------------------- Error Message ------------------------------------</div><div>[1]PETSC ERROR: Error in external library!</div><div>[1]PETSC ERROR: ParMETIS general error in ParMETIS_V3_PartKway!</div>
<div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 </div><div>[1]PETSC ERROR: See docs/changes/index.html for recent updates.</div>
<div>[1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div><div>[1]PETSC ERROR: See docs/index.html for manual pages.</div><div>[1]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[1]PETSC ERROR: /home/james/libmesh-0.9.3/examples/fem_system/fem_system_ex1/.libs/lt-example-opt on a linux-opt named james-SATELLITE-C855-18D by james Fri Aug 1 10:45:36 2014</div><div>[1]PETSC ERROR: Libraries linked from /home/james/petsc-3.4.2/linux-opt/lib</div>
<div>[1]PETSC ERROR: Configure run at Wed Jul 9 14:22:40 2014</div><div>[1]PETSC ERROR: Configure options ./configure --with-mpi=1 --with-x=0 --download-scalapack=1 --download-metis=1 --download-parmetis=1 --download-mumps=1 --download-hypre=1 --download-f-blas-lapack=1 --download-ml=1 --download-superlu_dist=1 --download-superlu=1 --download-umfpack=1 --download-ptscotch=1 --download-gamg=1 --download-pastix=1 --with-debugging=0</div>
<div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: MatPartitioningApply_Parmetis() line 104 in /home/james/petsc-3.4.2/src/mat/partition/impls/pmetis/pmetis.c</div>
<div>[0]PETSC ERROR: Error in external library!</div><div>[0]PETSC ERROR: ParMETIS general error in ParMETIS_V3_PartKway!</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 </div><div>[0]PETSC ERROR: See docs/changes/index.html for recent updates.</div><div>[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div>
<div>[0]PETSC ERROR: See docs/index.html for manual pages.</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[0]PETSC ERROR: /home/james/libmesh-0.9.3/examples/fem_system/fem_system_ex1/.libs/lt-example-opt on a linux-opt named james-SATELLITE-C855-18D by james Fri Aug 1 10:45:36 2014</div>
<div>[0]PETSC ERROR: Libraries linked from /home/james/petsc-3.4.2/linux-opt/lib</div><div>[0]PETSC ERROR: Configure run at Wed Jul 9 14:22:40 2014</div><div>[0]PETSC ERROR: Configure options ./configure --with-mpi=1 --with-x=0 --download-scalapack=1 --download-metis=1 --download-parmetis=1 --download-mumps=1 --download-hypre=1 --download-f-blas-lapack=1 --download-ml=1 --download-superlu_dist=1 --download-superlu=1 --download-umfpack=1 --download-ptscotch=1 --download-gamg=1 --download-pastix=1 --with-debugging=0</div>
<div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[0]PETSC ERROR: MatPartitioningApply_Parmetis() line 104 in /home/james/petsc-3.4.2/src/mat/partition/impls/pmetis/pmetis.c</div>
<div>[0]PETSC ERROR: MatPartitioningApply() line 225 in /home/james/petsc-3.4.2/src/mat/partition/partition.c</div><div>[0]PETSC ERROR: PCASMCreateSubdomains() line 1210 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div>
<div>[0]PETSC ERROR: PCSetUp_ASM() line 232 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div><div>[0]PETSC ERROR: PCSetUp() line 890 in /home/james/petsc-3.4.2/src/ksp/pc/interface/precon.c</div><div>[0]PETSC ERROR: KSPSetUp() line 278 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div>
<div>[0]PETSC ERROR: KSPSolve() line 399 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div><div>[0]PETSC ERROR: solve() line 763 in "unknowndirectory/"src/solvers/petsc_linear_solver.C</div><div>[1]PETSC ERROR: MatPartitioningApply() line 225 in /home/james/petsc-3.4.2/src/mat/partition/partition.c</div>
<div>[1]PETSC ERROR: PCASMCreateSubdomains() line 1210 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div><div>[1]PETSC ERROR: PCSetUp_ASM() line 232 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div><div>
[1]PETSC ERROR: PCSetUp() line 890 in /home/james/petsc-3.4.2/src/ksp/pc/interface/precon.c</div><div>[1]PETSC ERROR: KSPSetUp() line 278 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div><div>[1]PETSC ERROR: KSPSolve() line 399 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div>
<div>[1]PETSC ERROR: solve() line 763 in "unknowndirectory/"src/solvers/petsc_linear_solver.C</div><div>--------------------------------------------------------------------------</div><div>mpirun has exited due to process rank 1 with PID 5356 on</div>
<div>node james-SATELLITE-C855-18D exiting without calling "finalize". This may</div><div>have caused other processes in the application to be</div><div>terminated by signals sent by mpirun (as reported hPARMETIS ERROR: The sum of tpwgts for constraint #0 is not 1.0</div>
<div>PARMETIS ERROR: The sum of tpwgts for constraint #0 is not 1.0</div><div>--------------------------------------------------------------------------</div><div>MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD </div>
<div>with errorcode 76.</div><div><br></div><div>NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.</div><div>You may or may not see output from other processes, depending on</div><div>exactly when Open MPI kills them.</div>
<div>--------------------------------------------------------------------------</div><div>[0]PETSC ERROR: --------------------- Error Message ------------------------------------</div><div>[1]PETSC ERROR: --------------------- Error Message ------------------------------------</div>
<div>[1]PETSC ERROR: Error in external library!</div><div>[1]PETSC ERROR: ParMETIS general error in ParMETIS_V3_PartKway!</div><div>[1]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[1]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 </div><div>[1]PETSC ERROR: See docs/changes/index.html for recent updates.</div><div>[1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div>
<div>[1]PETSC ERROR: See docs/index.html for manual pages.</div><div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: /home/james/libmesh-0.9.3/examples/fem_system/fem_system_ex1/.libs/lt-example-opt on a linux-opt named james-SATELLITE-C855-18D by james Fri Aug 1 10:45:36 2014</div>
<div>[1]PETSC ERROR: Libraries linked from /home/james/petsc-3.4.2/linux-opt/lib</div><div>[1]PETSC ERROR: Configure run at Wed Jul 9 14:22:40 2014</div><div>[1]PETSC ERROR: Configure options ./configure --with-mpi=1 --with-x=0 --download-scalapack=1 --download-metis=1 --download-parmetis=1 --download-mumps=1 --download-hypre=1 --download-f-blas-lapack=1 --download-ml=1 --download-superlu_dist=1 --download-superlu=1 --download-umfpack=1 --download-ptscotch=1 --download-gamg=1 --download-pastix=1 --with-debugging=0</div>
<div>[1]PETSC ERROR: ------------------------------------------------------------------------</div><div>[1]PETSC ERROR: MatPartitioningApply_Parmetis() line 104 in /home/james/petsc-3.4.2/src/mat/partition/impls/pmetis/pmetis.c</div>
<div>[0]PETSC ERROR: Error in external library!</div><div>[0]PETSC ERROR: ParMETIS general error in ParMETIS_V3_PartKway!</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: Petsc Release Version 3.4.2, Jul, 02, 2013 </div><div>[0]PETSC ERROR: See docs/changes/index.html for recent updates.</div><div>[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.</div>
<div>[0]PETSC ERROR: See docs/index.html for manual pages.</div><div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[0]PETSC ERROR: /home/james/libmesh-0.9.3/examples/fem_system/fem_system_ex1/.libs/lt-example-opt on a linux-opt named james-SATELLITE-C855-18D by james Fri Aug 1 10:45:36 2014</div>
<div>[0]PETSC ERROR: Libraries linked from /home/james/petsc-3.4.2/linux-opt/lib</div><div>[0]PETSC ERROR: Configure run at Wed Jul 9 14:22:40 2014</div><div>[0]PETSC ERROR: Configure options ./configure --with-mpi=1 --with-x=0 --download-scalapack=1 --download-metis=1 --download-parmetis=1 --download-mumps=1 --download-hypre=1 --download-f-blas-lapack=1 --download-ml=1 --download-superlu_dist=1 --download-superlu=1 --download-umfpack=1 --download-ptscotch=1 --download-gamg=1 --download-pastix=1 --with-debugging=0</div>
<div>[0]PETSC ERROR: ------------------------------------------------------------------------</div><div>[0]PETSC ERROR: MatPartitioningApply_Parmetis() line 104 in /home/james/petsc-3.4.2/src/mat/partition/impls/pmetis/pmetis.c</div>
<div>[0]PETSC ERROR: MatPartitioningApply() line 225 in /home/james/petsc-3.4.2/src/mat/partition/partition.c</div><div>[0]PETSC ERROR: PCASMCreateSubdomains() line 1210 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div>
<div>[0]PETSC ERROR: PCSetUp_ASM() line 232 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div><div>[0]PETSC ERROR: PCSetUp() line 890 in /home/james/petsc-3.4.2/src/ksp/pc/interface/precon.c</div><div>[0]PETSC ERROR: KSPSetUp() line 278 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div>
<div>[0]PETSC ERROR: KSPSolve() line 399 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div><div>[0]PETSC ERROR: solve() line 763 in "unknowndirectory/"src/solvers/petsc_linear_solver.C</div><div>[1]PETSC ERROR: MatPartitioningApply() line 225 in /home/james/petsc-3.4.2/src/mat/partition/partition.c</div>
<div>[1]PETSC ERROR: PCASMCreateSubdomains() line 1210 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div><div>[1]PETSC ERROR: PCSetUp_ASM() line 232 in /home/james/petsc-3.4.2/src/ksp/pc/impls/asm/asm.c</div><div>
[1]PETSC ERROR: PCSetUp() line 890 in /home/james/petsc-3.4.2/src/ksp/pc/interface/precon.c</div><div>[1]PETSC ERROR: KSPSetUp() line 278 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div><div>[1]PETSC ERROR: KSPSolve() line 399 in /home/james/petsc-3.4.2/src/ksp/ksp/interface/itfunc.c</div>
<div>[1]PETSC ERROR: solve() line 763 in "unknowndirectory/"src/solvers/petsc_linear_solver.C</div><div>--------------------------------------------------------------------------</div><div>mpirun has exited due to process rank 1 with PID 5356 on</div>
<div>node james-SATELLITE-C855-18D exiting without calling "finalize". This may</div><div>have caused other processes in the application to be</div><div>terminated by signals sent by mpirun (as reported here).</div>
<div>--------------------------------------------------------------------------</div><div>[james-SATELLITE-C855-18D:05354] 1 more process has sent help message help-mpi-api.txt / mpi-abort</div><div>[james-SATELLITE-C855-18D:05354] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages</div>
<div>ere).</div><div>--------------------------------------------------------------------------</div><div>[james-SATELLITE-C855-18D:05354] 1 more process has sent help message help-mpi-api.txt / mpi-abort</div><div>[james-SATELLITE-C855-18D:05354] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages</div>
</div><div><br></div></div>