[petsc-dev] GAMG key not found in table

Sun Jun 24 16:06:12 CDT 2018

On Sun, Jun 24, 2018 at 4:18 PM Pierre Jolivet <pierre.jolivet at enseeiht.fr>
wrote:

>
> On 24 Jun 2018, at 6:15 PM, Mark Adams <mfadams at lbl.gov> wrote:
>
> First, -pc_gamg_threshold 0.8  is crazy. 0.08 is very high. probably best
> to start with 0.
>
>
> I know it’s high, but that is what works for my application (otherwise the
> setup takes forever for just one or two iterations less).
> The problem is related to my previous email from Friday ([petsc-dev] GAMG
> and custom MatMults in smoothers, thanks Barry for the thorough answer), I
> basically need only two or three levels (extremely aggressive coarsening).
>

OK, you are coarsening somehow. The normal GAMG solver with 0.8 will
coarsen so slow that it is crazy.

>
> I don’t know if this can help, but depending on the value I use for the
> threshold, I also get this error:
> ksp/pc/impls/gamg/agg.c:          if (flid < 0)
> SETERRQ(PETSC_COMM_SELF,PETSC_ERR_PLIB,"Cannot find gid1 in table”);
>
>
This could also be from the messed up hash function.

> Of course, if I switch to an even crazier threshold, e.g., 0.999999, and
> end-up with a one-level method, I don’t see such an error.
>

Yea, I put logic in there to switch silently because it happens if you use
GAMG for a mass matrix (happens all the time). If you run with -info and
grep on GAMG you will see a message that it is doing this.

>
> I know I am to blame here but my application does not use PetscInt as
> default integers (plain int instead). I use a lot
> of MatMPIAIJSetPreallocationCSR, so it is not as easy as simply recompiling
> with --with-64-bit-indices for me.
> I’d gladly try anything that you suggest which would still use 32 bit
> indices as "primary PetscInt type".
>

If you feel like remaking PETSc (no reconfigure) with line 89 of
pcgamgimpl.h changed as Barry suggested. I think it will fix your problems.

Also, you are using a MatShell. GAMG requires AIJ because it digs into its
data structures. Not properly abstracted.

>
> Thanks,
> Pierre
>
> This threshold will delete all of your edges in the coarsening graph,
> probably, and GAMG will switch to a one level solve (silently although
> -info will tell you).
>
> And yes, this looks like a 64 bit indices problem, we have:
>
> #define GAMG_HASH(key) ((((PetscInt)7)*key)%a_tab->size)
>
> We should probably cast everything to (long long) or (unsigned PetscInt) I
> guess ... I'm not sure how this should be done exactly.  Any ideas from the
> peanut gallery?
>
>
>
> On Sun, Jun 24, 2018 at 2:34 AM Pierre Jolivet <pierre.jolivet at enseeiht.fr>
> wrote:
>
>> Hello,
>> Any ideas on how to debug this?
>> [406]PETSC ERROR: key 328214936 not found in table
>> [406]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
>> for trouble shooting.
>> [406]PETSC ERROR: Petsc Development GIT revision: v3.9.2-603-gceafe64
>> GIT Date: 2018-06-10 12:46:16 -0500
>> [406]PETSC ERROR:
>> /ccc/work/cont003/rndm/rndm/FreeFem-sources/src/mpi/FreeFem++-mpi on a
>> arch-linux2-c-debug-bullxmpi named curie2698 by jolivetp Sun Jun 24
>> 07:06:07 2018
>> [406]PETSC ERROR: Configure options --with-scalar-type=real
>> --with-hypre-dir=arch-linux2-c-opt-bullxmpi
>> --with-metis-dir=arch-linux2-c-opt-bullxmpi
>> --with-mumps-dir=arch-linux2-c-opt-bullxmpi --with-mkl_cpardiso=1
>> --with-mkl_pardiso=1 --with-parmetis-dir=arch-linux2-c-opt-bullxmpi
>> --with-ptscotch-dir=arch-linux2-c-opt-bullxmpi
>> --with-sowing-dir=arch-linux2-c-opt-bullxmpi
>> --with-blacs-include=/ccc/products/mkl-18.0.1.163/default/
>> 18.0.1.163/mkl/include
>> --with-blacs-lib=/ccc/products/mkl-18.0.1.163/default/
>> 18.0.1.163/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so
>> --with-blaslapack-dir=/ccc/products/mkl-18.0.1.163/default/
>> 18.0.1.163/mkl/lib/intel64 --with-x=0 --with-debugging=1
>> --with-errorchecking=1
>> --with-mkl_pardiso-dir=/ccc/products/mkl-18.0.1.163/default/
>> 18.0.1.163/mkl
>> --with-mkl_cpardiso-dir=/ccc/products/mkl-18.0.1.163/default/
>> 18.0.1.163/mkl
>> --with-scalapack-include=/ccc/products/mkl-18.0.1.163/default/
>> 18.0.1.163/mkl/include
>> --with-scalapack-lib="[/ccc/products/mkl-18.0.1.163/default/
>> 18.0.1.163/mkl/lib/intel64/libmkl_scalapack_lp64.so,/ccc/products/mkl-18.0.1.163/default/18.0.1.163/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so]"
>> --with-fortran-bindings=0 PETSC_ARCH=arch-linux2-c-debug-bullxmpi
>> [406]PETSC ERROR: #1 PCGAMGHashTableFind() line 106 in
>> /ccc/work/cont003/rndm/rndm/petsc/include/petsc/private/pcgamgimpl.h
>> [406]PETSC ERROR: #2 smoothAggs() line 526 in
>> /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/impls/gamg/agg.c
>> [406]PETSC ERROR: #3 PCGAMGCoarsen_AGG() line 970 in
>> /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/impls/gamg/agg.c
>> [406]PETSC ERROR: #4 PCSetUp_GAMG() line 518 in
>> /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/impls/gamg/gamg.c
>> [406]PETSC ERROR: #5 PCSetUp() line 932 in
>> /ccc/work/cont003/rndm/rndm/petsc/src/ksp/pc/interface/precon.c
>> [406]PETSC ERROR: #6 KSPSetUp() line 381 in
>> /ccc/work/cont003/rndm/rndm/petsc/src/ksp/ksp/interface/itfunc.c
>>
>> For smaller problems, this works fine. Should I compile using 64 bit
>> indices?
>> I’m using -pc_type gamg -pc_gamg_threshold 0.8 -pc_gamg_agg_nsmooths 1
>> -pc_gamg_sym_graph true -pc_gamg_square_graph 10
>>
>> Thanks,
>> Pierre
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180624/723f9e36/attachment-0001.html>