[petsc-users] SEGV using asm + icc + empty processors

[Tobin Isaac]

I've set up PCMG using PCML with repartitioning, which gives some
processors empty partitions on all by the finest levels.  As smoothers
I want to use block incomplete factorizations with one block per
processor.  My command line looks like this:

-info -pc_ml_PrintLevel 10 -pc_ml_maxCoarseSize 128 -pc_ml_repartition
-pc_ml_repartitionType Zoltan -pc_ml_Reusable -pc_ml_KeepAggInfo
-pc_mg_cycle_type v -pc_mg_smoothup 1 -pc_mg_smoothdown 1
-mg_coarse_pc_type redundant -mg_coarse_redundant_pc_type cholesky
-mg_levels_pc_type asm -on_error_attach_debugger
-mg_levels_1_sub_pc_type icc

I get a SEGV from the empty processors:

[1]PETSC ERROR: ------------------------------------------------------------------------
[1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[1]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[1]PETSC ERROR: likely location of problem given in stack below
[1]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[1] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -2080374780
[1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,
[1]PETSC ERROR:       INSTEAD the line number of the start of the function
[1]PETSC ERROR:       is given.
[1]PETSC ERROR: [1] MatCholeskyFactorNumeric_SeqAIJ line 2094 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/mat/impls/aij/seq/aijfact.c
[1]PETSC ERROR: [1] MatCholeskyFactorNumeric line 3019 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/mat/interface/matrix.c
[1]PETSC ERROR: [1] PCSetup_ICC line 13 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/impls/factor/icc/icc.c
[1]PETSC ERROR: [1] PCSetUp line 810 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/interface/precon.c
[1]PETSC ERROR: [1] KSPSetUp line 182 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/ksp/interface/itfunc.c
[1]PETSC ERROR: [1] PCSetUpOnBlocks_ASM line 416 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/impls/asm/asm.c
[1]PETSC ERROR: [1] PCSetUpOnBlocks line 861 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/interface/precon.c
[1]PETSC ERROR: [1] KSPSetUpOnBlocks line 151 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/ksp/interface/itfunc.c
[1]PETSC ERROR: [1] KSPSolve line 351 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/ksp/interface/itfunc.c
[1]PETSC ERROR: [1] PCMGMCycle_Private line 17 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/impls/mg/mg.c
[1]PETSC ERROR: [1] PCMGMCycle_Private line 17 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/impls/mg/mg.c
[1]PETSC ERROR: [1] PCApply_MG line 311 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/impls/mg/mg.c
[1]PETSC ERROR: [1] PCApply line 373 /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/ksp/pc/interface/precon.c
[1]PETSC ERROR: User provided function() line 0 in unknown directory unknown file

The debugger pulled up line 2129:

#9  0x00007f72b65cb992 in MatCholeskyFactorNumeric_SeqAIJ (B=0x2f367b0, A=0x2798410, info=0x2796678) at /org/centers/ccgo/local/ubuntu/lucid/apps/petsc/build/dev/src/mat/impls/aij/seq/aijfact.c:2129
2129        il[0] = 0;
(gdb) list
2124     
2125      do {
2126        sctx.newshift = PETSC_FALSE;
2127
2128        for (i=0; i<mbs; i++) c2r[i] = mbs; 
2129        il[0] = 0;
2130     
2131        for (k = 0; k<mbs; k++){
2132          /* zero rtmp */
2133          nz = bi[k+1] - bi[k];
(gdb) p il
$1 = (PetscInt *) 0x0

This doesn't happen if I swap in ilu for icc.

Thanks,
Toby
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120904/f65c1568/attachment.pgp>