[petsc-users] Unknown PETSc-HYPRE Error

[Alistair Bentley]

Hello all,

I'm a first time PETSc mailing list user, so I apologize in advance if this
is not the right mailing list for this post!

I've been using PETSc with HYPRE (BoomerAMG) as part of a Schur complement
block preconditioner for a time-dependent 3D Navier-Stokes equation.

Specifically, for the Schur complement approximation, I've been using
PETSc's selfp method (given the simulation's short time steps, this has
been producing reasonable results) and a single application of the default
HYPRE BoomerAMG.

When I run on my local machine (up to 4 processors), the simulation runs to
completion smoothly.  The GMRES iterations for the global system (as well
as the block solves) appear to scale well with h.

Next I pushed to job to our HPC resource and ran a small job (stabilized
P1P1 elements on 329013 tetrahedra and 56970 nodes) with 36 cores.  This
time, however, the simulation did not run to completion.  Rather, I began
to encounter the following error:

[ 6] KSPSolve() line 666 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/ksp/interface/itfunc.c
[ 6] KSPSolve_FGMRES() line 291 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
[ 6] KSPFGMRESCycle() line 166 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/ksp/impls/gmres/fgmres/fgmres.c
[ 6] KSP_PCApply() line 275 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/include/petsc/private/kspimpl.h
[ 6] PCApply() line 458 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/pc/interface/precon.c
[ 6] PCApply_FieldSplit_Schur() line 900 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/pc/impls/fieldsplit/fieldsplit.c
[ 6] KSPSolve() line 666 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/ksp/interface/itfunc.c
[ 6] KSPSolve_PREONLY() line 22 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/ksp/impls/preonly/preonly.c
[ 6] KSP_PCApply() line 275 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/include/petsc/private/kspimpl.h
[ 6] PCApply() line 458 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/pc/interface/precon.c
[ 6] PCApply_HYPRE() line 351 in
/p/home/abentle/.hashdist/tmp/petsc-mtyptrixhukj/src/ksp/pc/impls/hypre/hypre.c
[ 6] Error in external library
[ 6] Error in HYPRE solver, error code 1

Unfortunately, HYPRE lists an error code1 as a *generic error* which does
not give me much to go on.  Further, while this error appears consistently,
it seems to occur randomly.  For instance, the error message above occurred
on the 6th MPI process after the simulation had reached t = 0.1.  A
previous run with the exact same configuration occurred on the 28th MPI
process after the simulation reached t = 1.1 seconds.

I've been experimenting with different configurations to try and find a
stable solution, but I'm at a bit of a loss as to how to proceed.  Any
suggestions for moving forward would be greatly appreciated.

Thanks!

- Alistair
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20180718/e122f034/attachment.html>