[petsc-users] solving stokes-like equation in 3D staggered grid in irregular domain with petscsection
Bishesh Khanal
bisheshkh at gmail.com
Wed Dec 4 10:34:25 CST 2013
On Tue, Dec 3, 2013 at 5:13 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> Bishesh Khanal <bisheshkh at gmail.com> writes:
>
> > With options ksp_converged_reason and ksp_monitor_true_residual set only
> > for the fieldsplit_1 , I get the result as below. -ksp_view reports that
> > the preconditioner for the schur complement is built from A11.
> > preconditioned resid norm below looks to be the same and in the order of
> > 10e+14 for every iteration. Does this imply the precondtioner is
> singular?
> > Does it mean that the problem is possibly in assembling the A11 block?
>
> I would suspect that, yes. The preconditioner is amplifying the
> residual, but then it is not converging either, which indicates that the
> original system may also be singular, with an inconsistent right hand
> side.
>
I did some experiments and it seems that the matrix is assembled all right,
because the solver converges for some RHS and not for some others. But I
have not understood why RHS could be inconsistent. I present below 4 cases
out of which 3 cases converge. The only difference between 2nd and 4th is
the RHS. Do the following results give clues on what I should look at or
consider about ?
Experiments:
The domain is a 11^3 sized cuboid partitioned into A and B where B is a
sphere in the center with diameter 4 units.
To make it easier to refer to, here are the equations I'm solving:
div( mu (grad(u) )) - grad(p) = f(x)
div(u) + c(x) p = g(x)
WITH mu = 1 everywhere.
The options I give: -pc_type fieldsplit -pc_fieldsplit_type schur
-pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2
-pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type lu
-fieldsplit_0_pc_factor_mat_solver_package superlu_dist
-fieldsplit_1_ksp_converged_reason -ksp_converged_reason
-fieldsplit_1_ksp_monitor_true_residual -ksp_monitor_true_residual
Now, different cases starting with the ones where the solver converges.
1. A driven cavity test: c(x) = 0 everywhere; f(x) = g(x) = 0 everywhere;
boundary condition ux = 5;
It converges and results in the expected velocity field. Here is the output:
Residual norms for fieldsplit_1_ solve.
0 KSP preconditioned resid norm 5.054223508734e+14 true resid norm
1.122436826478e+01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.749535352327e+14 true resid norm
6.107840938579e+00 ||r(i)||/||b|| 5.441589935838e-01
2 KSP preconditioned resid norm 1.505961910867e+14 true resid norm
3.348977798988e+00 ||r(i)||/||b|| 2.983667071487e-01
3 KSP preconditioned resid norm 5.174359504409e+13 true resid norm
1.162887407911e+00 ||r(i)||/||b|| 1.036038180928e-01
4 KSP preconditioned resid norm 2.080045117083e+13 true resid norm
4.956790811875e-01 ||r(i)||/||b|| 4.416097810535e-02
5 KSP preconditioned resid norm 7.755059044492e+12 true resid norm
2.492038982077e-01 ||r(i)||/||b|| 2.220204222892e-02
6 KSP preconditioned resid norm 2.466612045429e+12 true resid norm
1.883096820091e-01 ||r(i)||/||b|| 1.677686240925e-02
7 KSP preconditioned resid norm 8.579343124832e+11 true resid norm
1.811484167702e-01 ||r(i)||/||b|| 1.613885187094e-02
8 KSP preconditioned resid norm 2.905820616232e+11 true resid norm
1.802566230397e-01 ||r(i)||/||b|| 1.605940029652e-02
9 KSP preconditioned resid norm 1.050448005003e+11 true resid norm
1.801573367439e-01 ||r(i)||/||b|| 1.605055469439e-02
10 KSP preconditioned resid norm 3.403790595963e+10 true resid norm
1.801439682968e-01 ||r(i)||/||b|| 1.604936367439e-02
11 KSP preconditioned resid norm 1.299335081666e+10 true resid norm
1.801425947866e-01 ||r(i)||/||b|| 1.604924130579e-02
12 KSP preconditioned resid norm 4.493652071254e+09 true resid norm
1.801423944430e-01 ||r(i)||/||b|| 1.604922345681e-02
Linear solve converged due to CONVERGED_RTOL iterations 12
0 KSP preconditioned resid norm 6.460680353553e+01 true resid norm
1.049466531148e+02 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for fieldsplit_1_ solve.
0 KSP preconditioned resid norm 7.823051492951e+12 true resid norm
1.737068454898e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 4.255798896540e+12 true resid norm
9.449771864126e-02 ||r(i)||/||b|| 5.440068776497e-01
2 KSP preconditioned resid norm 2.330965266450e+12 true resid norm
5.175786465381e-02 ||r(i)||/||b|| 2.979609957678e-01
3 KSP preconditioned resid norm 8.009006255864e+11 true resid norm
1.778356846780e-02 ||r(i)||/||b|| 1.023769006780e-01
4 KSP preconditioned resid norm 3.219545074418e+11 true resid norm
7.148826788310e-03 ||r(i)||/||b|| 4.115454844715e-02
5 KSP preconditioned resid norm 1.200344957869e+11 true resid norm
2.665301219518e-03 ||r(i)||/||b|| 1.534367406191e-02
6 KSP preconditioned resid norm 3.817859308610e+10 true resid norm
8.477351604027e-04 ||r(i)||/||b|| 4.880263400170e-03
7 KSP preconditioned resid norm 1.327911515975e+10 true resid norm
2.948555889112e-04 ||r(i)||/||b|| 1.697432177068e-03
8 KSP preconditioned resid norm 4.497669563823e+09 true resid norm
9.986832819219e-05 ||r(i)||/||b|| 5.749245397360e-04
9 KSP preconditioned resid norm 1.625897964247e+09 true resid norm
3.610218721966e-05 ||r(i)||/||b|| 2.078339924824e-04
10 KSP preconditioned resid norm 5.268465651593e+08 true resid norm
1.169834375386e-05 ||r(i)||/||b|| 6.734532378890e-05
11 KSP preconditioned resid norm 2.011138479503e+08 true resid norm
4.465624568191e-06 ||r(i)||/||b|| 2.570782144826e-05
12 KSP preconditioned resid norm 6.955390134082e+07 true resid norm
1.544407052087e-06 ||r(i)||/||b|| 8.890881920815e-06
Linear solve converged due to CONVERGED_RTOL iterations 12
1 KSP preconditioned resid norm 5.821935836469e-04 true resid norm
1.801423947884e-01 ||r(i)||/||b|| 1.716513956775e-03
Linear solve converged due to CONVERGED_RTOL iterations 1
2. Same as 1 but with c(x) = 1 in the sphere B, 0 everywhere else;
It converges and results in expected velocity field. Interesting thing in
the output is that preconditioned norm is exactly the same as true resid
norm in this case:
using user defined split
Residual norms for fieldsplit_1_ solve.
0 KSP preconditioned resid norm 1.069474142891e+01 true resid norm
1.069474142891e+01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 1.426854491662e+00 true resid norm
1.426854491662e+00 ||r(i)||/||b|| 1.334164552875e-01
2 KSP preconditioned resid norm 9.997206363303e-02 true resid norm
9.997206363303e-02 ||r(i)||/||b|| 9.347777531377e-03
3 KSP preconditioned resid norm 1.010655985247e-02 true resid norm
1.010655985247e-02 ||r(i)||/||b|| 9.450027305152e-04
4 KSP preconditioned resid norm 8.627873205985e-04 true resid norm
8.627873205984e-04 ||r(i)||/||b|| 8.067397667664e-05
5 KSP preconditioned resid norm 6.055739652458e-05 true resid norm
6.055739652441e-05 ||r(i)||/||b|| 5.662352561486e-06
Linear solve converged due to CONVERGED_RTOL iterations 5
0 KSP preconditioned resid norm 4.960608132875e+01 true resid norm
9.486832980505e+01 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for fieldsplit_1_ solve.
0 KSP preconditioned resid norm 2.155933535217e-01 true resid norm
2.155933535217e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.876369982498e-02 true resid norm
2.876369982498e-02 ||r(i)||/||b|| 1.334164497890e-01
2 KSP preconditioned resid norm 2.015316291302e-03 true resid norm
2.015316291302e-03 ||r(i)||/||b|| 9.347766331297e-03
3 KSP preconditioned resid norm 2.037356487804e-04 true resid norm
2.037356487804e-04 ||r(i)||/||b|| 9.449996739339e-04
4 KSP preconditioned resid norm 1.739275204864e-05 true resid norm
1.739275204864e-05 ||r(i)||/||b|| 8.067387869121e-05
5 KSP preconditioned resid norm 1.220766583893e-06 true resid norm
1.220766583895e-06 ||r(i)||/||b|| 5.662357229266e-06
Linear solve converged due to CONVERGED_RTOL iterations 5
1 KSP preconditioned resid norm 4.970924030301e-05 true resid norm
6.058440172041e-05 ||r(i)||/||b|| 6.386156670504e-07
Linear solve converged due to CONVERGED_RTOL iterations 1
3. Different RHS test: c(x) = 0 everywhere; g(x) = -0.2 in the sphere B, 0
everywhere else. f(x) = grad(g); boundary condition: ux = 5; other wall
velocities 0.
The solver converges;
Residual norms for fieldsplit_1_ solve.
0 KSP preconditioned resid norm 5.054223508734e+14 true resid norm
1.122436826478e+01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.749535352327e+14 true resid norm
6.107840938579e+00 ||r(i)||/||b|| 5.441589935838e-01
2 KSP preconditioned resid norm 1.505961910867e+14 true resid norm
3.348977798988e+00 ||r(i)||/||b|| 2.983667071487e-01
3 KSP preconditioned resid norm 5.174359504409e+13 true resid norm
1.162887407911e+00 ||r(i)||/||b|| 1.036038180928e-01
4 KSP preconditioned resid norm 2.080045117083e+13 true resid norm
4.956790811875e-01 ||r(i)||/||b|| 4.416097810535e-02
5 KSP preconditioned resid norm 7.755059044492e+12 true resid norm
2.492038982077e-01 ||r(i)||/||b|| 2.220204222892e-02
6 KSP preconditioned resid norm 2.466612045429e+12 true resid norm
1.883096820090e-01 ||r(i)||/||b|| 1.677686240925e-02
7 KSP preconditioned resid norm 8.579343124832e+11 true resid norm
1.811484167702e-01 ||r(i)||/||b|| 1.613885187094e-02
8 KSP preconditioned resid norm 2.905820616232e+11 true resid norm
1.802566230397e-01 ||r(i)||/||b|| 1.605940029652e-02
9 KSP preconditioned resid norm 1.050448005003e+11 true resid norm
1.801573367439e-01 ||r(i)||/||b|| 1.605055469439e-02
10 KSP preconditioned resid norm 3.403790595952e+10 true resid norm
1.801439682968e-01 ||r(i)||/||b|| 1.604936367439e-02
11 KSP preconditioned resid norm 1.299335081659e+10 true resid norm
1.801425947866e-01 ||r(i)||/||b|| 1.604924130579e-02
12 KSP preconditioned resid norm 4.493652072271e+09 true resid norm
1.801423944430e-01 ||r(i)||/||b|| 1.604922345681e-02
Linear solve converged due to CONVERGED_RTOL iterations 12
0 KSP preconditioned resid norm 6.298618568983e+01 true resid norm
9.494103433184e+01 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for fieldsplit_1_ solve.
0 KSP preconditioned resid norm 8.024336532177e+12 true resid norm
1.781762765346e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 4.365299472959e+12 true resid norm
9.692911986886e-02 ||r(i)||/||b|| 5.440068776497e-01
2 KSP preconditioned resid norm 2.390940384281e+12 true resid norm
5.308958077846e-02 ||r(i)||/||b|| 2.979609957678e-01
3 KSP preconditioned resid norm 8.215075861801e+11 true resid norm
1.824113496597e-02 ||r(i)||/||b|| 1.023769006780e-01
4 KSP preconditioned resid norm 3.302383114942e+11 true resid norm
7.332764204778e-03 ||r(i)||/||b|| 4.115454844715e-02
5 KSP preconditioned resid norm 1.231229515148e+11 true resid norm
2.733878712712e-03 ||r(i)||/||b|| 1.534367406191e-02
6 KSP preconditioned resid norm 3.916091815629e+10 true resid norm
8.695471611506e-04 ||r(i)||/||b|| 4.880263400170e-03
7 KSP preconditioned resid norm 1.362078326946e+10 true resid norm
3.024421449800e-04 ||r(i)||/||b|| 1.697432177068e-03
8 KSP preconditioned resid norm 4.613393408335e+09 true resid norm
1.024379137786e-04 ||r(i)||/||b|| 5.749245397361e-04
9 KSP preconditioned resid norm 1.667731887469e+09 true resid norm
3.703108691786e-05 ||r(i)||/||b|| 2.078339924825e-04
10 KSP preconditioned resid norm 5.404021874909e+08 true resid norm
1.199933903467e-05 ||r(i)||/||b|| 6.734532378861e-05
11 KSP preconditioned resid norm 2.062884538961e+08 true resid norm
4.580523903463e-06 ||r(i)||/||b|| 2.570782144823e-05
12 KSP preconditioned resid norm 7.134350475945e+07 true resid norm
1.584144235576e-06 ||r(i)||/||b|| 8.890881919785e-06
Linear solve converged due to CONVERGED_RTOL iterations 12
1 KSP preconditioned resid norm 5.821934702566e-04 true resid norm
1.801423947967e-01 ||r(i)||/||b|| 1.897413442612e-03
Linear solve converged due to CONVERGED_RTOL iterations 1
4. Same as 3, but with c(x) = 1 in the sphere B.
using user defined split
Residual norms for fieldsplit_1_ solve.
0 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.076313011714e+01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01
2 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01
3 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01
4 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01
5 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01
6 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01
7 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696886e+01 ||r(i)||/||b|| 9.993632755341e-01
8 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627696766e+01 ||r(i)||/||b|| 9.993632754221e-01
9 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627641906e+01 ||r(i)||/||b|| 9.993632244519e-01
10 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.075627067412e+01 ||r(i)||/||b|| 9.993626906904e-01
11 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.410873813881e+01 ||r(i)||/||b|| 1.310839689315e+00
12 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.215904758000e+01 ||r(i)||/||b|| 1.129694377720e+00
13 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.597693687523e+01 ||r(i)||/||b|| 1.484413614009e+00
14 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
2.266134843283e+01 ||r(i)||/||b|| 2.105460789399e+00
15 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
2.035308798305e+01 ||r(i)||/||b|| 1.891000829827e+00
16 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
2.407254169449e+01 ||r(i)||/||b|| 2.236574438152e+00
17 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
3.477754796008e+01 ||r(i)||/||b|| 3.231174164168e+00
18 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
3.156748925892e+01 ||r(i)||/||b|| 2.932928331755e+00
19 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
3.324768650679e+01 ||r(i)||/||b|| 3.089035080403e+00
20 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
3.488562155804e+01 ||r(i)||/||b|| 3.241215257863e+00
21 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
5.078895151790e+01 ||r(i)||/||b|| 4.718790069909e+00
22 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
4.939231093930e+01 ||r(i)||/||b|| 4.589028507669e+00
23 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
5.086042832143e+01 ||r(i)||/||b|| 4.725430963659e+00
24 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
6.737635727583e+01 ||r(i)||/||b|| 6.259922210597e+00
25 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
6.509813461374e+01 ||r(i)||/||b|| 6.048253055129e+00
26 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
6.471858831835e+01 ||r(i)||/||b|| 6.012989494133e+00
27 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
1.033462891313e+02 ||r(i)||/||b|| 9.601880494476e+00
28 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
7.685802125780e+01 ||r(i)||/||b|| 7.140861480008e+00
29 KSP preconditioned resid norm 5.174242038871e+13 true resid norm
8.469815519206e+01 ||r(i)||/||b|| 7.869286561646e+00
And it continues, it's not converging.
When c(x) is zero everywhere, it is easy to think of a compatibility
condition that total sum of g(x) should be zero if we enforce zero velocity
on all the walls. But when we release the incompressibilty condition in
some parts of the domain by putting non zero c(x), I think I do not need to
take care about the total sum of g(x) being zero. But I'm not sure what is
going wrong with this RHS.
> > It is the diagonal of this block corresponding to the domain B cells
> > that changes when I set c(x) = 1. Residual norms for fieldsplit_1_
> > solve. 0 KSP preconditioned resid norm 2.890735677419e+14 true resid
> > norm 1.265820125363e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.117075725063e+01 ||r(i)||/||b|| 8.824916768826e-01 2 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109990265188e+01 ||r(i)||/||b|| 8.768941518210e-01 3 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109895437193e+01 ||r(i)||/||b|| 8.768192375476e-01 4 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109892185160e+01 ||r(i)||/||b|| 8.768166684358e-01 5 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109889770615e+01 ||r(i)||/||b|| 8.768147609417e-01 6 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109886495655e+01 ||r(i)||/||b|| 8.768121737173e-01 7 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109886096749e+01 ||r(i)||/||b|| 8.768118585813e-01 8 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109885236160e+01 ||r(i)||/||b|| 8.768111787141e-01 9 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109882829958e+01 ||r(i)||/||b|| 8.768092778106e-01 10 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109882703198e+01 ||r(i)||/||b|| 8.768091776699e-01 11 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109882171007e+01 ||r(i)||/||b|| 8.768087572386e-01 12 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109882129003e+01 ||r(i)||/||b|| 8.768087240548e-01 13 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109880708177e+01 ||r(i)||/||b|| 8.768076016004e-01 14 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.109884240385e+01 ||r(i)||/||b|| 8.768103920501e-01 15 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.110342530238e+01 ||r(i)||/||b|| 8.771724417953e-01 16 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.668559859400e+01 ||r(i)||/||b|| 1.318165058342e+00 17 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 2.379573238198e+02 ||r(i)||/||b|| 1.879866807707e+01 18 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 7.475251038198e+02 ||r(i)||/||b|| 5.905460727331e+01 19 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 7.610489296073e+02 ||r(i)||/||b|| 6.012299175517e+01 20 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.162871674016e+03 ||r(i)||/||b|| 9.186705525659e+01 21 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 2.438552126509e+03 ||r(i)||/||b|| 1.926460227364e+02 22 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 2.112154724517e+03 ||r(i)||/||b|| 1.668605738047e+02 23 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 1.867895524623e+03 ||r(i)||/||b|| 1.475640564719e+02 24 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 2.760528221023e+03 ||r(i)||/||b|| 2.180821876435e+02 25 KSP
> > preconditioned resid norm 2.890735677419e+14 true resid norm
> > 2.585072583244e+03 ||r(i)||/||b|| 2.042211631374e+02 and it continues.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20131204/256f2334/attachment-0001.html>
More information about the petsc-users
mailing list