[petsc-users] Troubles updating my code from PETSc-3.4 to 3.5 Using MUMPS for KSPSolve()

[Marc MEDALE]

Barry,

First of all I would like to thank you very much for the time you have spent on this problem.

>    Attached I plotted the solution (as a 1d vector for all the solvers), as you can see all the answers are very different.

Secondly I am well aware of most of the many limits in solving very ill-conditioned systems, however the reason that I keep going with that is I want to take advantage of direct solvers for computing bifurcation diagrams in incompressible fluid flow problems with a method that transforms the original non-linear problem into a set of linear ones, with the same operator but different rhs (further details can be found in [1]).

[1] B. Cochelin and M. Medale. Power series analysis as a major breakthrough to improve the efficiency of Asymptotic Numerical Method in the vicinity of bifurcations. J. Comput. Phys., Vol. 236, 594-607, 2013.

> Interesting when I ran with MUMPS on 1 and 2 processors using PETSc 3.5p2 (actually the maint branch) I get a solution very near your "old" solution. I do not know if the quad precision solution is "accurate" but I am pretty confident that all the other answers are equally valid and equally worthless.


I agree with you that except quad precision all other solutions can be worthless, but in our method it is only in the vicinity of the bifurcation point that the algebraic system degenerates and becomes close to singular, so I have to know what is the reliability limit to switch to an augmented system that recovers a more convenient condition number, but at more than twice the original cost.

> How do you know the "old" mumps solution is correct and the new mumps solution wrong?

After any changes or upgrade to new PETSc versions I use to run tests which compare the outputs between previous release and the new one. And I was very surprised to see such huge differences in outputs, for components that haven't been told to be changed in the changes_3.5 log file (neither KSP nor MUMPS were told to be changed in the changes from PETSc-3.4 to 3.5).

So, as longtime PETSc user (and aficionado), I would greatly appreciate you give me some hints about the following fundamental question: what are the changes that have been done in PETSc to change in such a way the solution computed by the MUMPS solver throughout the PETSc interface?

Thank you again for your time and pertinent explanations, as usual. Have a good day.

Marc MEDALE





Le 19 déc. 2014 à 00:29, Barry Smith <bsmith at mcs.anl.gov> a écrit :

> 
>   Marc,
> 
>    I have played around with your matrix using several "direct" solvers; all of them produced residual norms of around 10^-8 except Matlab which produced a residual norm of 10^-5 and a warning that the recond of the matrix was 10^-16
> I also solved with the PETSc LU factorization in quad precision and got a residual norm of 10^-26. UMFPACK ran out of memory.
> 
>    Attached I plotted the solution (as a 1d vector for all the solvers), as you can see all the answers are very different. Note that the figures are black inside the "envelop" of the solution because intermediate values of the vector are dense (ie. at a very fine scale the solution is oscillating a great deal). Interesting when I ran with MUMPS on 1 and 2 processors using PETSc 3.5p2 (actually the maint branch) I get a solution very near your "old" solution. 
> 
>   IThe matrix has 8,000 rows of the identity (1 on the diagonal) and the rest with entries of -10^8.  Don't put those trivial rows into the matrix and scale the matrix so it has positive diagonal entires. I've done this and it doesn't help the solver but it is still the right thing to do.
> 
> <Untitled.png>
> 
>    I do not know if the quad precision solution is "accurate" but I am pretty confident that all the other answers are equally valid and equally worthless. How do you know the "old" mumps solution is correct and the new mumps solution wrong?
> 
>   Can you try with the maint branch of PETSc and see if you get the solutions you want?
> 
> 
>   Barry
> 
> 
> 
>> On Dec 11, 2014, at 4:38 AM, Marc MEDALE <marc.medale at univ-amu.fr> wrote:
>> 
>> Dear PETSC Users,
>> 
>> I have just updated to PETSc-3.5 my research code that uses PETSc for a while but I'm facing an astonishing difference between PETSc-3.4 to 3.5 versions when solving a very ill conditioned algebraic system with MUMPS (4.10.0 in both cases).
>> 
>> The only differences that arise in my fortran source code are the following:
>> Loma1-medale% diff ../version_3.5/solvEFL_MAN_SBIF.F ../version_3.4/solvEFL_MAN_SBIF.F
>> 336,337d335
>> <       CALL MatSetOption(MATGLOB,MAT_KEEP_NONZERO_PATTERN,
>> <      &       PETSC_TRUE,IER)
>> 749,750c747,748
>> <       CALL KSPSetTolerances(KSP1,TOL,PETSC_DEFAULT_REAL,
>> <      &     PETSC_DEFAULT_REAL,PETSC_DEFAULT_INTEGER,IER)
>> ---
>>>      CALL KSPSetTolerances(KSP1,TOL,PETSC_DEFAULT_DOUBLE_PRECISION,
>>>     &     PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_INTEGER,IER)
>> 909c907,908
>> <           CALL KSPSetOperators(KSP1,MATGLOB,MATGLOB,IER)
>> ---
>>>          CALL KSPSetOperators(KSP1,MATGLOB,MATGLOB,
>>>     &                         SAME_NONZERO_PATTERN,IER)
>> 
>> When I run the corresponding program versions on 128 cores of our cluster with the same input data and the following command line arguments:
>> -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -mat_mumps_icntl_8 0
>> 
>> I get the following outputs:
>> a)  with PETSc-3.4p4:
>> 	L2 norm of solution vector: 7.39640E-02,
>> 
>> b)  with PETSc-3.5p1:
>> 	L2 norm of solution vector: 1.61325E-02
>> 
>> Do I have change something else in updating my code based on KSP from PETSc-3.4 to 3.5 versions?
>> Do any default values in the PETSc-MUMPS interface have been changed from PETSc-3.4 to 3.5?
>> Any hints or suggestions are welcome to help me to recover the right results (obtained with PETSc-3.4).
>> 
>> Thank you very much.
>> 
>> Marc MEDALE.
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20141219/e5fdf087/attachment.html>