<div dir="ltr">Satish fixed this error. I believe the fix is now in master.<div><br></div><div>  Thanks,</div><div><br></div><div>     Matt</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Feb 19, 2017 at 3:05 AM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Hi all,<br><br></div>So I am attempting to employ the DG1 finite element method on the poisson equation using GAMG. When I attempt to solve a problem with roughly 4 million DOFs across 20 cores, i get this error:<br><br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error: error code 63<br>[ 6] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 6] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 6] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 6] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 6] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 6] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 6] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 6] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 6] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 6] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 6] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 6] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 6] Argument out of range<br>[ 6] A really huge hash is being requested.. cannot process: 4096000<br>------------------------------<wbr>------------------------------<wbr>--------------<br>MPI_ABORT was invoked on rank 6 in communicator MPI_COMM_WORLD <br>with errorcode 1.<br><br>NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.<br>You may or may not see output from other processes, depending on<br>exactly when Open MPI kills them.<br>------------------------------<wbr>------------------------------<wbr>--------------<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>Traceback (most recent call last):<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "pFiredrake.py", line 109, in <module><br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>    _solve_varproblem(*args, **kwargs)<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>      File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>solver.solve()<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    self.snes.solve(None, v)<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>petsc4py.PETSc.Errorpetsc4py.<wbr>PETSc.Errorpetsc4py.PETSc.<wbr>Error:     solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>: error code 63<br>[ 3] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 3] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 3] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 3] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 3] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 3] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 3] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 3] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 3] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 3] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 3] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 3] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 3] Argument out of range<br>[ 3] A really huge hash is being requested.. cannot process: 4096000<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>: error code 63<br>[ 1] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 1] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 1] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 1] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 1] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 1] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 1] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 1] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 1] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 1] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 1] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 1] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 1] Argument out of range<br>[ 1] A really huge hash is being requested.. cannot process: 4096000<br>: error code 63<br>[17] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[17] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[17] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[17] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[17] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[17] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[17] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[17] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[17] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[17] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[17] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[17] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[17] Argument out of range<br>[17] A really huge hash is being requested.. cannot process: 4096000<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>error code 63<br>[ 5] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 5] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 5] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 5] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 5] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 5] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 5] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 5] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 5] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 5] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 5] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 5] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 5] Argument out of range<br>[ 5] A really huge hash is being requested.. cannot process: 4096000<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    _solve_varproblem(*args, **kwargs)<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>petsc4py.PETSc.Error: error code 63<br>[ 9] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 9] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 9] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 9] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 9] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 9] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 9] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 9] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 9] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 9] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 9] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 9] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 9] Argument out of range<br>[ 9] A really huge hash is being requested.. cannot process: 4096000<br>    self.snes.solve(None, v)<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>    _solve_varproblem(*args, **kwargs)<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error: error code 63<br>[ 4] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 4] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 4] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 4] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 4] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 4] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 4] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 4] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 4] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 4] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 4] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 4] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 4] Argument out of range<br>[ 4] A really huge hash is being requested.. cannot process: 4096000<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    solver.solve()<br>    _solve_varproblem(*args, **kwargs)<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>    _solve_varproblem(*args, **kwargs)<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    solver.solve()<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    self.snes.solve(None, v)<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>petsc4py.PETSc.Error: error code 63<br>[13] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[13] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[13] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[13] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[13] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[13] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[13] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[13] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[13] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[13] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[13] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[13] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[13] Argument out of range<br>[13] A really huge hash is being requested.. cannot process: 4096000<br>    solver.solve()<br>Traceback (most recent call last):<br>  File "pFiredrake.py", line 109, in <module><br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>    solver.solve()<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>: error code 63<br>[11] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[11] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[11] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[11] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[11] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[11] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[11] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[11] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[11] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[11] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[11] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[11] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[11] Argument out of range<br>[11] A really huge hash is being requested.. cannot process: 4096000<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    solve(a==L,solution,options_<wbr>prefix='fe_',solver_<wbr>parameters=solver_params)<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    solver.solve()<br>    self.snes.solve(None, v)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    _solve_varproblem(*args, **kwargs)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    _solve_varproblem(*args, **kwargs)<br>    self.snes.solve(None, v)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 122, in solve<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    self.snes.solve(None, v)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error: error code 63<br>[14] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[14] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[14] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[14] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[14] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[14] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[14] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[14] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[14] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[14] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[14] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[14] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[14] Argument out of range<br>[14] A really huge hash is being requested.. cannot process: 4096000<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>    self.snes.solve(None, v)<br>petsc4py.PETSc.Error: error code 63<br>[19] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[19] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[19] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[19] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[19] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[19] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[19] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[19] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[19] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[19] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[19] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[19] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[19] Argument out of range<br>[19] A really huge hash is being requested.. cannot process: 4096000<br>    _solve_varproblem(*args, **kwargs)<br>    self.snes.solve(None, v)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    _solve_varproblem(*args, **kwargs)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error: error code 63<br>[15] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[15] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[15] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[15] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[15] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[15] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[15] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[15] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[15] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[15] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[15] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[15] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[15] Argument out of range<br>[15] A really huge hash is being requested.. cannot process: 4096000<br>    self.snes.solve(None, v)<br>petsc4py.PETSc.Error: error code 63<br>[ 7] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 7] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 7] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 7] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 7] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 7] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 7] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 7] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 7] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 7] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 7] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 7] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 7] Argument out of range<br>[ 7] A really huge hash is being requested.. cannot process: 4096000<br>    solver.solve()<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>: error code 63<br>[12] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[12] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[12] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[12] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[12] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[12] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[12] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[12] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[12] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[12] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[12] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[12] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[12] Argument out of range<br>[12] A really huge hash is being requested.. cannot process: 4096000<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>    solver.solve()<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/solving.py", line 152, in _solve_varproblem<br>petsc4py.PETSc.Error:   File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>error code 63<br>[ 2] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 2] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 2] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 2] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 2] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 2] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 2] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 2] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 2] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 2] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 2] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 2] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 2] Argument out of range<br>[ 2] A really huge hash is being requested.. cannot process: 4096000<br>petsc4py.PETSc.Error  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>: error code 63<br>[ 8] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 8] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 8] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 8] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 8] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 8] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 8] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 8] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 8] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 8] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 8] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 8] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 8] Argument out of range<br>[ 8] A really huge hash is being requested.. cannot process: 4096000<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    solver.solve()<br>    solver.solve()<br>    self.snes.solve(None, v)<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>  File "/home/jchang23/Software/<wbr>firedrake/src/firedrake/<wbr>firedrake/variational_solver.<wbr>py", line 220, in solve<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Error    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>: error code 63<br>[18] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[18] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[18] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[18] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[18] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[18] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[18] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[18] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[18] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[18] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[18] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[18] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[18] Argument out of range<br>[18] A really huge hash is being requested.. cannot process: 4096000<br>    self.snes.solve(None, v)<br>  File "PETSc/SNES.pyx", line 537, in petsc4py.PETSc.SNES.solve (src/petsc4py.PETSc.c:172359)<br>petsc4py.PETSc.Errorpetsc4py.<wbr>PETSc.Error: error code 63<br>[ 0] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[ 0] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[ 0] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 0] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[ 0] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[ 0] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[ 0] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[ 0] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[ 0] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 0] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[ 0] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 0] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[ 0] Argument out of range<br>[ 0] A really huge hash is being requested.. cannot process: 4096000<br>: error code 63<br>[16] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[16] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[16] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[16] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[16] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[16] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[16] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[16] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[16] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[16] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[16] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[16] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[16] Argument out of range<br>[16] A really huge hash is being requested.. cannot process: 4096000<br>petsc4py.PETSc.Error: error code 63<br>[10] SNESSolve() line 4128 in /tmp/pip-FNpsya-build/src/<wbr>snes/interface/snes.c<br>[10] SNESSolve_KSPONLY() line 40 in /tmp/pip-FNpsya-build/src/<wbr>snes/impls/ksponly/ksponly.c<br>[10] KSPSolve() line 620 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[10] KSPSetUp() line 393 in /tmp/pip-FNpsya-build/src/ksp/<wbr>ksp/interface/itfunc.c<br>[10] PCSetUp() line 968 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/interface/precon.c<br>[10] PCSetUp_GAMG() line 524 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/gamg.c<br>[10] PCGAMGCoarsen_AGG() line 955 in /tmp/pip-FNpsya-build/src/ksp/<wbr>pc/impls/gamg/agg.c<br>[10] MatTransposeMatMult() line 9962 in /tmp/pip-FNpsya-build/src/mat/<wbr>interface/matrix.c<br>[10] MatTransposeMatMult_MPIAIJ_<wbr>MPIAIJ() line 902 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[10] MatTransposeMatMultSymbolic_<wbr>MPIAIJ_MPIAIJ() line 1676 in /tmp/pip-FNpsya-build/src/mat/<wbr>impls/aij/mpi/mpimatmatmult.c<br>[10] PetscTableCreate() line 52 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[10] PetscTableCreateHashSize() line 28 in /tmp/pip-FNpsya-build/src/sys/<wbr>utils/ctable.c<br>[10] Argument out of range<br>[10] A really huge hash is being requested.. cannot process: 4096000<br>[compute-0-0.local:31182] 19 more processes have sent help message help-mpi-api.txt / mpi-abort<br>[compute-0-0.local:31182] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages<br><br></div><div>However, if I use either ML or HYPRE i do not. When I downscale the problem to roughly 1.7 million dofs, I get no such error with GAMG, but both ML and HYPRE are still faster.<br><br></div><div>Something tells me I need to tune the GAMG parameters. Here's some -GAMG related info that may help:<br><br>[0] PCSetUp_GAMG(): level 0) N=4096000, n data rows=1, n data cols=1, nnz/row (ave)=55, np=20<br>[0] PCGAMGFilterGraph():      78.5842% nnz after filtering, with threshold 0., 55.8 nnz ave. (N=4096000)<br>[0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square<br><br></div><div>And if I run with -pc_gamg_threshold 0.08:<br><br>[0] PCSetUp_GAMG(): level 0) N=4096000, n data rows=1, n data cols=1, nnz/row (ave)=55, np=20<br>[0] PCGAMGFilterGraph():      28.5842% nnz after filtering, with threshold 0.08, 55.8 nnz ave. (N=4096000)<br>[0] PCGAMGCoarsen_AGG(): Square Graph on level 1 of 1 to square<br><br></div><div>Any idea what to do here?<br><br></div><div>Thanks,<br></div><div>Justin<br></div><div><br></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div>