[petsc-dev] asm / gasm

Sun Jul 3 09:04:15 CDT 2016

Barry, I have never seen an empty matrix for this problem.  I am in my
branch (mark/fix-gamg-asm-aggs), which basically just has the fix to add a
dummy block for BCs to make ASM work properly, added to your branch.

This (appended) is what I get with 1 proc (it is fine, it is solving) .
With 8 procs I get this stack trace.  I will keep digging.

#8 main (argc=36, args=0x7fffffff4628) at
/global/homes/m/madams/petsc/src/ksp/ksp/examples/tutorials/ex56.c:280 (at
0x0000000000409f2e)
#7 KSPSetUp (ksp=0x46e2f20) at
/global/u2/m/madams/petsc/src/ksp/ksp/interface/itfunc.c:393 (at
0x000000000075d99f)
#6 PCSetUp (pc=0x46e2f20) at
/global/u2/m/madams/petsc/src/ksp/pc/interface/precon.c:968 (at
0x00000000007531da)
#5 PCSetUp_GAMG (pc=0x46e2f20) at
/global/u2/m/madams/petsc/src/ksp/pc/impls/gamg/gamg.c:672 (at
0x0000000000c28a81)
#4 PCSetUp_MG (pc=0x4760330) at
/global/u2/m/madams/petsc/src/ksp/pc/impls/mg/mg.c:781 (at
0x0000000000c87f7e)
#3 KSPSetUp (ksp=0x46e2f20) at
/global/u2/m/madams/petsc/src/ksp/ksp/interface/itfunc.c:393 (at
0x000000000075d99f)
#2 PCSetUp (pc=0x46e2f20) at
/global/u2/m/madams/petsc/src/ksp/pc/interface/precon.c:968 (at
0x00000000007531da)
#1 PCSetUp_ASM (pc=0x46e2f20) at
/global/u2/m/madams/petsc/src/ksp/pc/impls/asm/asm.c:383 (at
0x0000000000c5c499)
#0 MatGetSubMatrices (mat=0x46e2f20, n=63418000, irow=0x21, icol=0x4a8,
scall=MAT_INITIAL_MATRIX, submat=0x2) at
/global/u2/m/madams/petsc/src/mat/interface/matrix.c:6780 (at
0x00000000004e8bcd)

06:32 134 nid00015  ~/petsc/src/ksp/ksp/examples/tutorials$ srun -n 1
./ex56 -ne 3 -alpha 1.e-3 -ksp_type cg -pc_type gamg -pc_gamg_type agg
-pc_gamg_agg_nsmooths 1 -pc_gamg_coarse_eq_limit 10
-pc_gamg_reuse_interpolation true -two_solves true -ksp_converged_reason
-use_mat_nearnullspace -mg_levels_esteig_ksp_type cg -pc_gamg_square_graph
1 -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_esteig
0,0.05,0,1.05 -gamg_est_ksp_type cg -gamg_est_ksp_max_it 20
-pc_gamg_asm_use_agg -mg_levels_sub_pc_type lu -ksp_monitor

  0 KSP Residual norm 2.553528059796e+02

  1 KSP Residual norm 1.285760737388e+01

  2 KSP Residual norm 5.896306650166e-01

  3 KSP Residual norm 3.206061310861e-02

  4 KSP Residual norm 3.287356003543e-03

  5 KSP Residual norm 1.695272457611e-04

Linear solve converged due to CONVERGED_RTOL iterations 5

  0 KSP Residual norm 2.553528059796e-03

  1 KSP Residual norm 1.285760737388e-04

  2 KSP Residual norm 5.896306650166e-06

  3 KSP Residual norm 3.206061310861e-07

  4 KSP Residual norm 3.287356003543e-08

  5 KSP Residual norm 1.695272457611e-09

Linear solve converged due to CONVERGED_RTOL iterations 5

  0 KSP Residual norm 2.553528059796e-08

  1 KSP Residual norm 1.285760737388e-09

  2 KSP Residual norm 5.896306650166e-11

  3 KSP Residual norm 3.206061310861e-12

  4 KSP Residual norm 3.287356003543e-13

  5 KSP Residual norm 1.695272457611e-14

Linear solve converged due to CONVERGED_RTOL iterations 5

[0]main |b-Ax|/|b|=9.327411e-06, |b|=7.453560e+00, emax=9.907752e-01

On Wed, Jun 29, 2016 at 1:05 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
>    Mark
>
>    When I run with exactly the options in this file on either 1 or 8
> process and MatView() the matrix you pass to KSP the matrix has ALL zeros.
> So of course the GAMG solver is not happy. Is the matrix suppose to have
> all zeros? When I run the command line options in the make file for runex56
> the matrix has nonzero values in it and GAMG runs fine.
>
>    Barry
>
> > On Jun 27, 2016, at 6:56 AM, Mark Adams <mfadams at lbl.gov> wrote:
> >
> > Barry,
> >
> > I am having problems with your branch on ksp/ex56. First I was getting
> an error with the increase of the overlap that you added (I don't want that
> to be the default).  So I set this back to 0 in gamg.c.
> >
> > Next, I get the same error with the vec out of range because ex56 has
> BCs in the matrix.  I created a branch mark/fix-gamg-asm-aggs from your
> branch and added the dummy ASM block. But I am getting all kinds of
> errors.  I get errors in get MatSubMatrix and a funny error when it tries
> to free a prefix string.
> >
> > I've attached the valgrind output of one process, of 8.  There is a lot
> of noise with string operations but it looks like the first non-string
> thing is in MatGetSubMatrices.  ex56 is elasticity. Maybe there is a block
> size problem?
> >
> >
> > <out_val_1>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20160703/ee16fba8/attachment.html>