[petsc-users] I find slow performance of SNES

[Smith, Barry F.]

> On Sep 15, 2019, at 5:35 PM, Pedro Gonzalez via petsc-users <petsc-users at mcs.anl.gov> wrote:
> 
> Dear all,
> 
> I am working on a code that solves a nonlinear system of equations G(x)=0 with Gauss-Seidel method. I managed to parallelize it by using DMDA with very good results. The previous week I changed my Gauss-Seidel solver by SNES. The code using SNES gives the same result as before, but I do not obtain the performance that I expected:
> 1) When using the Gauss-Seidel method (-snes_type ngs) the residual G(x) seems not be scallable to the amplitude of x

   Do you simply mean that -snes_type ngs is not converging to the solution? Does it seem to converge to something else or nothing at all? 

   The SNES GS code just calls the user provided routine set with SNESSetNGS(). This means that it is expect to behave the same way as if the user simply called the user provided routine themselves.  I assume you have a routine that implements the GS on the DMDA since you write " I managed to parallelize it by using DMDA with very good results. " Thus you should get the same iterations for both calling your Gauss-Seidel code yourself and calling SNESSetNGS() and then calling SNESSolve() with -snes_type ngs.  You can check if they are behaving in the same (for simplicity) by using the debugger or by putting VecView() into code to see if they are generating the same values. Just have your GS code call VecView() on the input vectors at the top and the output vectors at the bottom.


> and I have to add the option -snes_secant_h in order to make SNES converge.

  Do you mean both   -snes_ngs_secant and  -snes_ngs_secant_h  ?  The second option by itself will do nothing.  Cut and paste the exact options you use.

> However, I varied the step from 1.E-1 to 1.E50 and obtained the same result within the same computation time.

   This would happen if you did not use -snes_ngs_secant  but do use -snes_ngs_secant_h but this doesn't change the algorithm

> Is it normal that snes_secant_h can vary so many orders of magnitude?

   That certainly does seem odd. Looking at the code SNESComputeNGSDefaultSecant() we see it is perturbing the input vector (by color) with h

-    for (j=0;j<size;j++) {
-      wa[idx[j]-s] += h;
-    }

It then does the secant step with 

        if (PetscAbsScalar(g-f) > atol) {
          /* This is equivalent to d = x - (h*f) / PetscRealPart(g-f) */
          d = (x*g-w*f) / PetscRealPart(g-f);
        } else {
          d = x;
        }

In PetscErrorCode SNESSetFromOptions_NGS(PetscOptionItems *PetscOptionsObject,SNES snes) one can see that the user provided h is accepted

ierr = PetscOptionsReal("-snes_ngs_secant_h","Differencing parameter for secant search","",gs->h,&gs->h,NULL);CHKERRQ(ierr);

You could run in the debugger with a break point in SNESComputeNGSDefaultSecant() to see if it is truly using the h you provided.




> 2) Compared to my Gauss-Seidel algorithm, SNES does (approximately) the same number of iterations (with the same convergence criterium) but it is about 100 times slower.

   I don't fully understand what you are running so cannot completely answer this.

-snes_ngs_secant   will be lots slower than using a user provided GS

By default SNES may be doing some norm computations at each iteration which are expensive and will slow things down. You can use SNESSetNormSchedule() or the command line form -snes_norm_schedule <none, always, initialonly, finalonly, initalfinalonly>  to turn these norms off. 

   
>  What can be the reason(s) of this slow performance of SNES solver? I do not use preconditioner with my algorithm so I did not add one to SNES.
> 
> The main PETSc subroutines that I have included (in this order) are the following:
> call DMDACreate3D
> call DMSetUp
> call DMCreateLocalVector
> call DMCreateGlobalVector
> call SNESCreate
> call SNESSetConvergenceTest
> call SNESSetDM
> call DMDASNESSetFunctionLocal
> call SNESSetFromOptions
> call SNESSolve
> 
> Thanks in advance for you help.
> 
> Best regards,
> Pedro
> 
>