<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Sorry, I should have made myself more clear. I changed the three
7 passed to DMDACreate3d to 33 to make the example a bit more
realistic, as I also use "U-cycles", i.e. my coarsest level is
still big enough to make use of some GPU parallelism. I should
have just put that into the given command line argument string
with -da_grid_x 33 -da_grid_y 33 -da_grid_z 33<br>
</p>
<div class="moz-cite-prefix">On 2/28/23 18:43, Barry Smith wrote:<br>
</div>
<blockquote type="cite"
cite="mid:BF9DF7E7-3411-4408-8345-A7E0BF9E58E1@petsc.dev">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div><br>
</div>
I am sorry, I cannot reproduce what you describe. I am using
src/ksp/ksp/tutorials/ex45.c in the main branch (should be same as
release for this purpose).
<div><br>
</div>
<div> No change to the code I get </div>
<div><br>
</div>
<div>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">$ ./ex45
-ksp_converged_reason -ksp_type richardson -ksp_rtol 1e-09
-pc_type mg -pc_mg_levels 3 -mg_levels_ksp_type richardson
-mg_levels_ksp_max_it 6 -mg_levels_ksp_converged_maxits
-mg_levels_pc_type jacobi -mg_coarse_ksp_type richardson
-mg_coarse_ksp_max_it 6 -mg_coarse_ksp_converged_maxits
-mg_coarse_pc_type jacobi -ksp_monitor_true_residual
-ksp_view</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 0 KSP
preconditioned resid norm 1.851257578045e+01 true resid norm
1.476491378857e+01 ||r(i)||/||b|| 1.000000000000e+00</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 1 KSP
preconditioned resid norm 3.720545622095e-01 true resid norm
5.171053311198e-02 ||r(i)||/||b|| 3.502257707188e-03</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 2 KSP
preconditioned resid norm 1.339047557616e-02 true resid norm
1.866765310863e-03 ||r(i)||/||b|| 1.264325235890e-04</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 3 KSP
preconditioned resid norm 4.833887599029e-04 true resid norm
6.867629264754e-05 ||r(i)||/||b|| 4.651316873974e-06</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 4 KSP
preconditioned resid norm 1.748167886388e-05 true resid norm
3.398334857479e-06 ||r(i)||/||b|| 2.301628648933e-07</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 5 KSP
preconditioned resid norm 6.570567424652e-07 true resid norm
4.304483984231e-07 ||r(i)||/||b|| 2.915346507180e-08</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 6 KSP
preconditioned resid norm 4.013427896557e-08 true resid norm
7.502068698790e-08 ||r(i)||/||b|| 5.081010838410e-09</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 7 KSP
preconditioned resid norm 5.934811016347e-09 true resid norm
1.333884145638e-08 ||r(i)||/||b|| 9.034147877457e-10</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">Linear
solve converged due to CONVERGED_RTOL iterations 7</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">KSP
Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> type:
richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=10000, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-09, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> left
preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> using
PRECONDITIONED norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">PC
Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> type:
mg</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> type
is MULTIPLICATIVE, levels=3 cycles=v</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Cycles per PCApply=1</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Not using Galerkin computed coarse grid matrices</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Coarse
grid solver -- level 0 -------------------------------</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> KSP
Object: (mg_coarse_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=6, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
left preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
using PRECONDITIONED norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> PC
Object: (mg_coarse_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: jacobi</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type DIAGONAL</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
linear system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Mat Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=8, cols=8</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=32, allocated nonzeros=32</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Down
solver (pre-smoother) on level 1
-------------------------------</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> KSP
Object: (mg_levels_1_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=6, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
left preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
using NONE norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> PC
Object: (mg_levels_1_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: jacobi</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type DIAGONAL</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
linear system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Mat Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=64, cols=64</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=352, allocated nonzeros=352</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Up
solver (post-smoother) same as down solver (pre-smoother)</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Down
solver (pre-smoother) on level 2
-------------------------------</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> KSP
Object: (mg_levels_2_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=6, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
left preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
using NONE norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> PC
Object: (mg_levels_2_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: jacobi</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type DIAGONAL</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
linear system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Mat Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=343, cols=343</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=2107, allocated nonzeros=2107</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Up
solver (post-smoother) same as down solver (pre-smoother)</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> linear
system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Mat
Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=343, cols=343</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=2107, allocated nonzeros=2107</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">Residual
norm 1.33388e-08</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">~/Src/petsc/src/ksp/ksp/tutorials</span><span
style="font-variant-ligatures: no-common-ligatures; color:
#c814c9"><b> (main=)</b></span><span
style="font-variant-ligatures: no-common-ligatures">
arch-main</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">$ </span></p>
</div>
<div><span style="font-variant-ligatures: no-common-ligatures"><br>
</span></div>
<div><span style="font-variant-ligatures: no-common-ligatures"><br>
</span></div>
<div> Now change code with </div>
<div><br>
</div>
<div>
<div> if (i == 0 || j == 0 || k == 0 || i == mx - 1 || j
== my - 1 || k == mz - 1) {</div>
<div> barray[k][j][i] = 0; //2.0 * (HxHydHz + HxHzdHy +
HyHzdHx);</div>
<div> } else {</div>
<div> barray[k][j][i] = 1; //Hx * Hy * Hz;</div>
<div> }</div>
<div><br>
</div>
<div> I do not understand where I am suppose to change the
dimension to 33 so I ignore that statement. Same command line
with change above gives</div>
<div><br>
</div>
<div>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">$
./ex45 -ksp_converged_reason -ksp_type richardson
-ksp_rtol 1e-09 -pc_type mg -pc_mg_levels 3
-mg_levels_ksp_type richardson -mg_levels_ksp_max_it 6
-mg_levels_ksp_converged_maxits -mg_levels_pc_type jacobi
-mg_coarse_ksp_type richardson -mg_coarse_ksp_max_it 6
-mg_coarse_ksp_converged_maxits -mg_coarse_pc_type jacobi
-ksp_monitor_true_residual -ksp_view</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 0
KSP preconditioned resid norm 7.292257119299e+01 true
resid norm 1.118033988750e+01 ||r(i)||/||b||
1.000000000000e+00</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 1
KSP preconditioned resid norm 2.534913491362e+00 true
resid norm 3.528425353826e-01 ||r(i)||/||b||
3.155919577875e-02</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 2
KSP preconditioned resid norm 9.145057509152e-02 true
resid norm 1.279725352471e-02 ||r(i)||/||b||
1.144621152262e-03</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 3
KSP preconditioned resid norm 3.302446009474e-03 true
resid norm 5.122622088691e-04 ||r(i)||/||b||
4.581812485342e-05</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 4
KSP preconditioned resid norm 1.204504429329e-04 true
resid norm 4.370692051248e-05 ||r(i)||/||b||
3.909265814124e-06</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 5
KSP preconditioned resid norm 5.339971695523e-06 true
resid norm 7.229991776815e-06 ||r(i)||/||b||
6.466701235889e-07</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 6
KSP preconditioned resid norm 5.856425044706e-07 true
resid norm 1.282860114273e-06 ||r(i)||/||b||
1.147424968455e-07</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 7
KSP preconditioned resid norm 1.007137752126e-07 true
resid norm 2.283009757390e-07 ||r(i)||/||b||
2.041986004328e-08</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> 8
KSP preconditioned resid norm 1.790021892548e-08 true
resid norm 4.063263596129e-08 ||r(i)||/||b||
3.634293444578e-09</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">Linear
solve converged due to CONVERGED_RTOL iterations 8</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">KSP
Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=10000, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-09, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> left
preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
using PRECONDITIONED norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">PC
Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: mg</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type is MULTIPLICATIVE, levels=3 cycles=v</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Cycles per PCApply=1</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Not using Galerkin computed coarse grid matrices</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Coarse grid solver -- level 0
-------------------------------</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
KSP Object: (mg_coarse_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=6, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
left preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
using PRECONDITIONED norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> PC
Object: (mg_coarse_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: jacobi</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type DIAGONAL</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
linear system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Mat Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=8, cols=8</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=32, allocated nonzeros=32</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Down
solver (pre-smoother) on level 1
-------------------------------</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
KSP Object: (mg_levels_1_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=6, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
left preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
using NONE norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> PC
Object: (mg_levels_1_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: jacobi</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type DIAGONAL</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
linear system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Mat Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=64, cols=64</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=352, allocated nonzeros=352</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Up
solver (post-smoother) same as down solver (pre-smoother)</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Down
solver (pre-smoother) on level 2
-------------------------------</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
KSP Object: (mg_levels_2_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: richardson</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
damping factor=1.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
maximum iterations=6, nonzero initial guess</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
left preconditioning</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
using NONE norm type for convergence test</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> PC
Object: (mg_levels_2_) 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: jacobi</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type DIAGONAL</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
linear system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
Mat Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=343, cols=343</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=2107, allocated nonzeros=2107</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Up
solver (post-smoother) same as down solver (pre-smoother)</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
linear system matrix = precond matrix:</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"> Mat
Object: 1 MPI process</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
type: seqaij</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
rows=343, cols=343</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total: nonzeros=2107, allocated nonzeros=2107</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
total number of mallocs used during MatSetValues calls=0</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">
not using I-node routines</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">Residual
norm 4.06326e-08</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">~/Src/petsc/src/ksp/ksp/tutorials</span><span
style="font-variant-ligatures: no-common-ligatures; color:
#c814c9"><b> (main *=)</b></span><span
style="font-variant-ligatures: no-common-ligatures">
arch-main</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">$ </span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"><br>
</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">In
neither case is it taking 25 iterations. What am I doing
wrong?</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"><br>
</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">Normally
one expects only trivial changes in the convergence of
multigrid methods when one changes values in the right
hand side as with the run above.</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"><br>
</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures">Barry</span></p>
<p style="margin: 0px; font-stretch: normal; font-size: 14px;
line-height: normal; font-family: Menlo;"><span
style="font-variant-ligatures: no-common-ligatures"><br>
</span></p>
</div>
<div><span style="font-variant-ligatures: no-common-ligatures"><br>
</span></div>
<div><br>
<blockquote type="cite">
<div>On Feb 27, 2023, at 7:16 PM, Paul Grosse-Bley
<a class="moz-txt-link-rfc2396E" href="mailto:paul.grosse-bley@ziti.uni-heidelberg.de"><paul.grosse-bley@ziti.uni-heidelberg.de></a> wrote:</div>
<br class="Apple-interchange-newline">
<div>The scaling might be the problem, especially since I
don't know what you mean by scaling it according to FE.<br>
<br>
For reproducing the issue with a smaller problem:<br>
Change the ComputeRHS function in ex45.c<br>
<br>
if (i == 0 || j == 0 || k == 0 || i == mx - 1 || j == my -
1 || k == mz - 1) {<br>
barray[k][j][i] = 0.0;<br>
} else {<br>
barray[k][j][i] = 1.0;<br>
}<br>
<br>
Change the dimensions to e.g. 33 (I scaled it down, so it
goes quick without a GPU) instead of 7 and then run with<br>
<br>
-ksp_converged_reason -ksp_type richardson -ksp_rtol 1e-09
-pc_type mg -pc_mg_levels 3 -mg_levels_ksp_type richardson
-mg_levels_ksp_max_it 6 -mg_levels_ksp_converged_maxits
-mg_levels_pc_type jacobi -mg_coarse_ksp_type richardson
-mg_coarse_ksp_max_it 6 -mg_coarse_ksp_converged_maxits
-mg_coarse_pc_type jacobi<br>
<br>
You will find that it takes 145 iterations instead of 25
for the original ex45 RHS. My hpgmg-cuda implementation
(using 32^3) takes 41 iterations.<br>
<br>
To what do I have to change the diagonal entries of the
matrix for the boundary according to FE? Right now the
diagonal is completely constant.<br>
<br>
Paul<br>
<br>
On Tuesday, February 28, 2023 00:23 CET, Barry Smith
<a class="moz-txt-link-rfc2396E" href="mailto:bsmith@petsc.dev"><bsmith@petsc.dev></a> wrote:<br>
<blockquote type="cite"
cite="x-msg://41/EAFA975B-BDD5-4414-8569-D419EB6401ED@petsc.dev"><br>
I have not seen explicitly including, or excluding, the
Dirichlet boundary values in the system having a
significant affect on the convergence so long as you
SCALE the diagonal rows (of those Dirichlet points) by a
value similar to the other entries along the diagonal.
If they are scaled completely differently, that can
screw up the convergence. For src/ksp/ksp/ex45.c I see
that the appropriate scaling is used (note the scaling
should come from a finite element view of the
discretization even if the discretization is finite
differences as is done in ex45.c)<br>
<br>
Are you willing to share the two codes so we can take a
look with experienced eyes to try to figure out the
difference?<br>
<br>
Barry<br>
<br>
<br>
<br>
<br>
> On Feb 27, 2023, at 5:48 PM, Paul Grosse-Bley
<a class="moz-txt-link-rfc2396E" href="mailto:paul.grosse-bley@ziti.uni-heidelberg.de"><paul.grosse-bley@ziti.uni-heidelberg.de></a> wrote:<br>
><br>
> Hi Barry,<br>
><br>
> the reason why I wanted to change to ghost
boundaries is that I was worrying about the effect of
PCMGs coarsening on these boundary values.<br>
><br>
> As mentioned before, I am trying to reproduce
results from the hpgmg-cuda benchmark (a modified
version of it, e.g. using 2nd order instead of 4th
etc.).<br>
> I am trying to solve the Poisson equation -\nabla^2
u = 1 with u = 0 on the boundary with rtol=1e-9. While
my MG solver implemented in hpgmg solves this in 40
V-cycles (I weakened it a lot by only doing smooths at
the coarse level instead of CG). When I run the "same"
MG solver built in PETSc on this problem, it starts out
reducing the residual norm as fast or even faster for
the first 20-30 iterations. But for the last order of
magnitude in the residual norm it needs more than 300
V-cycles, i.e. it gets very slow. At this point I am
pretty much out of ideas about what is the cause,
especially since e.g. adding back cg at the coarsest
level doesn't seem to change the number of iterations at
all. Therefore I am suspecting the discretization to be
the problem. HPGMG uses an even number of points per
dimension (e.g. 256), while PCMG wants an odd number
(e.g. 257). So I also tried adding another layer of
boundary values for the discretization to effectively
use only 254 points per dimension. This caused the
solver to get even slightly worse.<br>
><br>
> So can the explicit boundary values screw with the
coarsening, especially when they are not finite? Because
with the problem as stated in ex45 with finite (i.e.
non-zero) boundary values, the MG solver takes only 18
V-cycles.<br>
><br>
> Best,<br>
> Paul<br>
><br>
><br>
><br>
> On Monday, February 27, 2023 18:17 CET, Barry Smith
<a class="moz-txt-link-rfc2396E" href="mailto:bsmith@petsc.dev"><bsmith@petsc.dev></a> wrote:<br>
><br>
>><br>
>> Paul,<br>
>><br>
>> DM_BOUNDARY_GHOSTED would result in the extra
ghost locations in the local vectors (obtained with
DMCreateLocalVector() but they will not appear in the
global vectors obtained with DMCreateGlobalVector();
perhaps this is the issue? Since they do not appear in
the global vector they will not appear in the linear
system so there will be no diagonal entries for you to
set since those rows/columns do not exist in the linear
system. In other words, using DM_BOUNDARY_GHOSTED is a
way to avoid needing to put the Dirichlet values
explicitly into the system being solved;
DM_BOUNDARY_GHOSTED is generally more helpful for
nonlinear systems than linear systems.<br>
>><br>
>> Barry<br>
>><br>
>> > On Feb 27, 2023, at 12:08 PM, Paul
Grosse-Bley
<a class="moz-txt-link-rfc2396E" href="mailto:paul.grosse-bley@ziti.uni-heidelberg.de"><paul.grosse-bley@ziti.uni-heidelberg.de></a> wrote:<br>
>> ><br>
>> > Hi,<br>
>> ><br>
>> > I would like to modify
src/ksp/ksp/tutorials/ex45.c to implement Dirichlet
boundary conditions using DM_BOUNDARY_GHOSTED instead of
using DM_BOUNDARY_NONE and explicitly implementing the
boundary by adding diagnonal-only rows.<br>
>> ><br>
>> > My assumption was that with
DM_BOUNDARY_GHOSTED all vectors from that DM have the
extra memory for the ghost entries and that I can
basically use DMDAGetGhostCorners instead of
DMDAGetCorners to access the array gotten via
DMDAVecGetArray. But when I access (gxs, gys, gzs) =
(-1,-1,-1) I get a segmentation fault. When looking at
the implementation of DMDAVecGetArray it looked to me as
if accessing (-1, -1, -1) should work as DMDAVecGetArray
passes the ghost corners to VecGetArray3d which then
adds the right offsets.<br>
>> ><br>
>> > I could not find any example using
DM_BOUNDARY_GHOSTED and then actually accessing the
ghost/boundary elements. Can I assume that they are set
to zero for the solution vector, i.e. the u=0 on
\del\Omega and I do not need to access them at all?<br>
>> ><br>
>> > Best,<br>
>> > Paul Große-Bley<br>
>><br>
</blockquote>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</body>
</html>