Problem in convergence after upgrade to petsc 3.0.0

Fri Jan 30 23:34:13 CST 2009

	Barry, trust me, these are *NOT* garbage. The reason you're seeing such
a jump in true residual is because in the case of ex10, I assume you're starting
with zero values. The nature of the physical system I'm solving has air layers with
electrical resistivity of 1e10 up against earth layers of resistivity 1.
A small change causes a large increase in the true residual. In my program,
I start with 1D boundary values which are much closer to the truth, and you
see only a slight increase in residual.

	I know from a numerical perspective you don't like these systems,
but that's what we deal with and they have been validated with direct
solvers and against 1D and analytic solutions for appropriate models.


	I will send the binaryoutput to petsc-maint. Regardless of whether or
not you like my numerical systems, there is obviously something wrong
in 3.0.0-p2 with the BCGSL and BCGS implementations.

	Randy
	
	
Barry Smith wrote:
> 
>    The convergence of both of those is GARBAGE. The true residual gets 
> hugely worse in the first
> step and then stays bad in both cases. The fact that the preconditioned 
> residual norm gets small
> better with the old PETSc then the new is kind of irrelevent. The 
> solution that KSP claims to give in
> both cases is frankly crap.
> 
>   Send me the output for both with -ksp_monitor_true_residual with 
> -ksp_type gmres with old and
> new PETSc.
> 
>    Barry
> 
> 
> On Jan 30, 2009, at 11:11 PM, Randall Mackie wrote:
> 
>>     Barry,
>>
>>     I've run ex10 from 2.3.3-p11 and from 3.0.0-p2 on the same 
>> binaryoutput
>> using the same parameters input to ex10 (ksp=bcgsl and pc=bjacobi, 
>> sub_pc=ilu)
>> and the convergence is different. Note, this is the binaryoutput from the
>> v 2.3.3-p11.
>>
>> Here is the convergence for ex10 from 2.3.3-p11:
>>
>> [rmackie 
>> ~/SPARSE/PETsc/petsc-2.3.3-p11/src/ksp/ksp/examples/tutorials] ./cmd_test
>>  0 KSP preconditioned resid norm 2.434324107505e-05 true resid norm 
>> 3.334159901684e-02 ||Ae||/||Ax|| 1.000000000000e+00
>>  2 KSP preconditioned resid norm 4.170258940641e-07 true resid norm 
>> 1.624814448703e+04 ||Ae||/||Ax|| 4.873234927583e+05
>>  4 KSP preconditioned resid norm 8.984963207977e-08 true resid norm 
>> 2.848180153976e+04 ||Ae||/||Ax|| 8.542422193181e+05
>>  6 KSP preconditioned resid norm 3.526062648105e-08 true resid norm 
>> 1.884591980560e+04 ||Ae||/||Ax|| 5.652374319564e+05
>>  8 KSP preconditioned resid norm 1.241153849592e-08 true resid norm 
>> 1.045401827970e+04 ||Ae||/||Ax|| 3.135427990247e+05
>> 10 KSP preconditioned resid norm 5.141701606491e-09 true resid norm 
>> 5.371087738119e+03 ||Ae||/||Ax|| 1.610926859088e+05
>> 12 KSP preconditioned resid norm 2.591522726108e-09 true resid norm 
>> 3.519633363751e+03 ||Ae||/||Ax|| 1.055628244456e+05
>> 14 KSP preconditioned resid norm 1.452300847705e-09 true resid norm 
>> 2.509415501645e+03 ||Ae||/||Ax|| 7.526380184636e+04
>> 16 KSP preconditioned resid norm 8.978785776773e-10 true resid norm 
>> 1.872610647707e+03 ||Ae||/||Ax|| 5.616439231845e+04
>> 18 KSP preconditioned resid norm 5.743601707920e-10 true resid norm 
>> 1.266421550645e+03 ||Ae||/||Ax|| 3.798322779917e+04
>> 20 KSP preconditioned resid norm 3.678705188041e-10 true resid norm 
>> 7.536064149571e+02 ||Ae||/||Ax|| 2.260258767363e+04
>> 22 KSP preconditioned resid norm 2.687340247327e-10 true resid norm 
>> 5.533061905955e+02 ||Ae||/||Ax|| 1.659507062981e+04
>> 24 KSP preconditioned resid norm 2.142070779181e-10 true resid norm 
>> 4.684485692902e+02 ||Ae||/||Ax|| 1.404997309978e+04
>> 26 KSP preconditioned resid norm 1.927583902818e-10 true resid norm 
>> 6.146993326148e+02 ||Ae||/||Ax|| 1.843640829296e+04
>> 28 KSP preconditioned resid norm 1.330050553611e-10 true resid norm 
>> 4.217422387701e+02 ||Ae||/||Ax|| 1.264913055181e+04
>> 30 KSP preconditioned resid norm 1.125778226987e-10 true resid norm 
>> 3.058702983977e+02 ||Ae||/||Ax|| 9.173834111652e+03
>> 32 KSP preconditioned resid norm 8.786382414490e-11 true resid norm 
>> 2.001904013306e+02 ||Ae||/||Ax|| 6.004223169665e+03
>> 34 KSP preconditioned resid norm 6.396029789854e-11 true resid norm 
>> 1.761414838880e+02 ||Ae||/||Ax|| 5.282934504701e+03
>> 36 KSP preconditioned resid norm 5.462116905743e-11 true resid norm 
>> 1.540860953921e+02 ||Ae||/||Ax|| 4.621436881724e+03
>> 38 KSP preconditioned resid norm 5.410616131680e-11 true resid norm 
>> 1.910613970561e+02 ||Ae||/||Ax|| 5.730420936308e+03
>> 40 KSP preconditioned resid norm 7.946054076400e-11 true resid norm 
>> 2.743761797072e+02 ||Ae||/||Ax|| 8.229244781230e+03
>> 42 KSP preconditioned resid norm 3.519925335531e-11 true resid norm 
>> 1.022381658728e+02 ||Ae||/||Ax|| 3.066384603244e+03
>> 42 KSP preconditioned resid norm 3.519925335531e-11 true resid norm 
>> 1.022381658728e+02 ||Ae||/||Ax|| 3.066384603244e+03
>> Number of iterations =  42
>>
>>
>> Here is the convergence for ex10 from 3.0.0-p2:
>>
>> rmackie ~/SPARSE/PETsc/petsc-3.0.0-p2/src/ksp/ksp/examples/tutorials] 
>> ./cmd_test
>>  0 KSP preconditioned resid norm 2.434324107505e-05 true resid norm 
>> 3.334159901684e-02 ||Ae||/||Ax|| 1.000000000000e+00
>>  2 KSP preconditioned resid norm 4.798562862178e-07 true resid norm 
>> 1.642225014976e+04 ||Ae||/||Ax|| 4.925453677690e+05
>>  4 KSP preconditioned resid norm 1.587355624039e-07 true resid norm 
>> 3.502938376404e+04 ||Ae||/||Ax|| 1.050620989904e+06
>>  6 KSP preconditioned resid norm 8.015528577103e-08 true resid norm 
>> 2.536595627800e+04 ||Ae||/||Ax|| 7.607900348506e+05
>>  8 KSP preconditioned resid norm 3.039306325625e-08 true resid norm 
>> 1.253662335500e+04 ||Ae||/||Ax|| 3.760054623856e+05
>> 10 KSP preconditioned resid norm 1.729327127129e-08 true resid norm 
>> 8.080934369743e+03 ||Ae||/||Ax|| 2.423679309940e+05
>> 12 KSP preconditioned resid norm 1.039787850500e-08 true resid norm 
>> 4.777791224801e+03 ||Ae||/||Ax|| 1.432982030162e+05
>> 14 KSP preconditioned resid norm 5.025780191774e-09 true resid norm 
>> 3.147651686153e+03 ||Ae||/||Ax|| 9.440614064618e+04
>> 16 KSP preconditioned resid norm 3.311781967950e-09 true resid norm 
>> 2.688053658600e+03 ||Ae||/||Ax|| 8.062161797467e+04
>> 18 KSP preconditioned resid norm 5.621276662229e-09 true resid norm 
>> 3.098810918425e+03 ||Ae||/||Ax|| 9.294128085637e+04
>> 20 KSP preconditioned resid norm 1.184533040734e-08 true resid norm 
>> 5.469874887175e+03 ||Ae||/||Ax|| 1.640555656737e+05
>> 22 KSP preconditioned resid norm 2.494642590524e-08 true resid norm 
>> 1.003335643955e+04 ||Ae||/||Ax|| 3.009260723962e+05
>> 24 KSP preconditioned resid norm 5.136091311727e-08 true resid norm 
>> 1.828432513826e+04 ||Ae||/||Ax|| 5.483937686680e+05
>> 26 KSP preconditioned resid norm 9.627430082715e-08 true resid norm 
>> 1.175348501769e+04 ||Ae||/||Ax|| 3.525171366783e+05
>> 28 KSP preconditioned resid norm 6.409712928943e-08 true resid norm 
>> 5.524687582334e+03 ||Ae||/||Ax|| 1.656995388716e+05
>> 30 KSP preconditioned resid norm 6.013091685526e-07 true resid norm 
>> 1.371019320496e+04 ||Ae||/||Ax|| 4.112038297274e+05
>> 32 KSP preconditioned resid norm 7.026562454712e-07 true resid norm 
>> 1.053982255306e+04 ||Ae||/||Ax|| 3.161162890759e+05
>> 34 KSP preconditioned resid norm 4.086784421188e-07 true resid norm 
>> 5.503180350963e+03 ||Ae||/||Ax|| 1.650544818856e+05
>> 36 KSP preconditioned resid norm 1.651444280250e-06 true resid norm 
>> 1.984011183420e+04 ||Ae||/||Ax|| 5.950557987388e+05
>> 38 KSP preconditioned resid norm 1.058319572456e-06 true resid norm 
>> 1.403784173466e+04 ||Ae||/||Ax|| 4.210308488073e+05
>> 40 KSP preconditioned resid norm 4.341084013969e-05 true resid norm 
>> 3.837773616917e+05 ||Ae||/||Ax|| 1.151046659453e+07
>> 42 KSP preconditioned resid norm 4.190225826231e-07 true resid norm 
>> 6.768382935039e+03 ||Ae||/||Ax|| 2.030011497535e+05
>> 44 KSP preconditioned resid norm 1.054511038261e-06 true resid norm 
>> 4.966771429542e+03 ||Ae||/||Ax|| 1.489662036615e+05
>> 46 KSP preconditioned resid norm 5.351004248086e-07 true resid norm 
>> 5.112611747101e+03 ||Ae||/||Ax|| 1.533403285343e+05
>> 48 KSP preconditioned resid norm 7.104477128923e-07 true resid norm 
>> 5.478002736962e+03 ||Ae||/||Ax|| 1.642993407183e+05
>> 50 KSP preconditioned resid norm 1.162050733932e-06 true resid norm 
>> 5.395393687747e+03 ||Ae||/||Ax|| 1.618216836278e+05
>>
>>
>> When I output binaryoutput using my code compiled under 3.0.0-p2 and 
>> run ex10, I get exactly
>> the same output as above. In other words, the matrix/rhs are exactly 
>> the same (as they
>> should be since I didn't change anything), and something about 
>> 3.0.0-p2 using BCGSL and ILU
>> is not working correctly.
>>
>> I changed ksptype to gmres, and I get the same convergence for both 
>> 2.3.3-p11 and for
>> 3.0.0-p2. Therefore, I conclude that bcgsl (PCBCGSL) is not working 
>> correctly under 3.0.0-p2.
>>
>> Randy
>>
>>
>> Barry Smith wrote:
>>>  Run the old code with -ksp_view_binary this will create a file 
>>> called binaryoutput; you can then
>>> run src/ksp/ksp/examples/tutorials/ex10.c using that input file. Use 
>>> the ex10 from the old version
>>> of PETSc and then the ex10 from the new version. Do they have they 
>>> same convergence?
>>> Now run the new code (that is build your code with petsc-3.0.) with 
>>> -ksp_view_binary and run that
>>> binaryoutput file with the old and new ex10 to see what happens.
>>>   Basically there are two possible changes with the change in the 
>>> version:
>>> 1) the matrix/right hand side has changed or
>>> 2) the solver has changed to behave differently.
>>>   By running the four cases you can start to get a handle on what has 
>>> actually changed, this will
>>> lead you to what needs to be investigated next.
>>>   Barry
>>> On Jan 30, 2009, at 8:01 PM, Randall Mackie wrote:
>>>> I just downloaded and compiled Petsc 3.0.0-p2, and after making some 
>>>> changes
>>>> in my code to specify the correct location of the include files, 
>>>> finally
>>>> got everything to compile okay.
>>>>
>>>> Now, I'm trying to run my test problem, and it's not converging. 
>>>> When I say
>>>> not converging, the first line (with ksp_monitor_true_residual) 
>>>> shows that
>>>> the true and preconditioned residuals are the same as before, but 
>>>> immediately
>>>> thereafter, the preconditioned residual fails to go below 1e-8 
>>>> whereas before
>>>> it quickly went down to 1e-15.
>>>>
>>>> The options in my command file are:
>>>>
>>>> -ksp_type bcgsl
>>>> -pc_type bjacobi
>>>> -sub_pc_type ilu
>>>> -sub_pc_factor_levels 3
>>>> -sub_pc_factor_fill 6
>>>>
>>>>
>>>> The only thing I see in the Change notes are that the ILU defaults 
>>>> to shifting
>>>> so that it's p.d. but I don't see an easy way to turn this off by 
>>>> the command
>>>> line to see if that's the problem. I tried to do it in my program, 
>>>> but it's unclear
>>>> if I did that correctly.
>>>>
>>>> Any suggestions?
>>>>
>>>> Thanks, Randy
>