[petsc-users] performance issue solving multiple linear systems of the same size with the different preconditioning methods

[Алексей Рязанов]

Hello!

Could you please help me to solve my performance problem.
I have two programs.

In 1st I solve one system with one method and one preconditioner and get
some performance numbers.
I run it 9 times with 9 different preconditioners.

In 2nd I solve the same system with the same one method but with 9 different
preconditioners consecutively one after another.
I run it once and also get some performance info.
In the 2nd case I have 2-5 times worse results, depending on used method.

Each KSPSolve procedure placed in its own stage of course, so I can compare
times, flops, messages an so..
I can see the difference but cant explain and eliminate it.

For example for -ksp_type cgs -pc_type asm -sub_pc_type jacobi -sub_ksp_type
preonly:
Summary of Stages:        ----- Time ------                 ----- Flops
-----       --- Messages ---  -- Message Lengths --  -- Reductions --
                                            Avg     %Total       Avg
%Total      counts   %Total        Avg         %Total   counts   %Total
    one stage frome 2nd:   5.5145e+00  14.9%  1.2336e+09  13.2%  2.494e+03
 22.4%  3.230e+03       22.2%  2.250e+03  18.5%
the once stage from 1st:  2.7541e+00  93.1%  1.2336e+09  99.8%  2.508e+03
 99.3%  1.470e+04       97.4%  2.266e+03  88.0%


My programs are pretty equivalent except the part with definition of
preconditioners and the number of called KSPSolve procedures.
I mean they use equal matrices, equally assemble them, use equal right hand
sides, equal convergence monitors.
Actually the 2nd one was made from the 1st.

In 1st i use KSPSetFromOptions(KSP); and then just set the -ksp_type
 -pc_type -sub_pc_type -sub_ksp_type keys from command line

In 2d i use for for nonblock PC:

  KSPGetPC(dKSP, &dPC);

  PCSetType(dPC, PCJACOBI);

and for block PC:

  PCSetType(dPC, PCASM);

  KSPSetUp(dKSP);

  PCSetUp(dPC);

  PCASMGetSubKSP(dPC, &n_local, &first_local, &ASMSubKSP);

  for (i=0; i<n_local; i++)

  {

    KSPGetPC(ASMSubKSP[i], &(SubPC[i]));

    PCSetType(SubPC[i], PCJACOBI);

  }


Im sure there is a mistake somewhere. Because 1st program compares Jacobi
and ASM-Jacobi preconditioners on my problem on the same KSP and tells me
that ASM-Jacobi is better and the 2nd shows otherwise results.

All tests I run on the 2cores machine.

Thank you in advance.

-- 
 Best regards,
Alexey Ryazanov
______________________________________
Nuclear Safety Institute of Russian Academy of Sciences
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110822/f523f14e/attachment.htm>