[petsc-users] GMRES solver

[Barry Smith]

On Jan 8, 2012, at 9:33 PM, Mohamad M. Nasr-Azadani wrote:

>    How do you know it is diverging? Because it looks weird? You are comparing it to something?
> 
> While marching in time, it only takes 20-30 iterations to solve for pressure. 
> After a very long integrattion in time, all of a sudden the pressure lsys does not converge even in 10,000 iterations. So, that's why I am saying it is converging. 

   Are you using the exact same matrix at each time-step or does the matrix change?


> 
>    How accurately are you solving the linear system?
> I tried rtol = 1e-10 and 1e-12. 
> 
>     What strange behavior?  Is the right hand side for the pressure solution reasonable but the solution "strange"? How do you know you are not giving "bad stuff" to the pressure solve?
> For my case, I get huge pressure gradients close to the solid boundaries. That, indeed, causes very small delta_t's when I keep integrating. That should not happen cause at this stage of my simulations, nothing is really happening in the flow field. Very small velocities and velocity gradients. 
> I am trying to find the problem, since I have only seen this for such big problem size, i.e. 100 million grid points. That makes it really hard to see if the I am feeding back rhs into the pressure linear system. 
> 
> Best, 
> Mohamad
> 
> 
> 
> 
> 
> 
> On Sun, Jan 8, 2012 at 6:47 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> On Jan 8, 2012, at 5:13 PM, Mohamad M. Nasr-Azadani wrote:
> 
> > Thanks Barry and Matt,
> >
> > Barry,
> >   Also if you are really solving the Poisson problem you should use multigrid; if simple geometry then geometric multigrid if complicated geometry probably easier to use hypre BoomerAMG. No sane person solves Poisson problem with anything but a multigrid or FFT based solver.
> >
> > In my main code, I am actually doing what you suggested, i.e. GMRES + boomerAMG to solve for my Poisson equation. I have not used the KSPSetNullSpace() though.
> > The problem is that my code (CFD, incompressible flow 3D) diverges after a long time integration
> 
>    How do you know it is diverging? Because it looks weird? You are comparing it to something?
> 
> 
>    How accurately are you solving the linear system?
> 
> 
> > and I am trying to find out why.
> > The system that I have is a fairly big one, i.e. 100 million grid points and more.
> > I see that pressure solution (which is obviously coupled to the velocity field) starts showing strange behavior.
> 
>     What strange behavior?  Is the right hand side for the pressure solution reasonable but the solution "strange"? How do you know you are not giving "bad stuff" to the pressure solve?
> 
>   Barry
> 
> 
> > That's why I tried to first double check my pressure solver.
> >
> > Based on your experience, do you think that not using a nullspace() for the pressure solver for that linear system size could have caused it to diverge?
> >
> >
> > Matt,
> > 1) Matlab could be doing a lot of things. I am betting that they scale the problem, so -pc_type jacobi.
> >
> > That could be right. The reason that I relied on the MATLAB's gmres solver to behave exactly similar to PETSc was just their "help" saying that
> > ************
> >  X = GMRES(A,B,RESTART,TOL,MAXIT,M1,M2) use preconditioner M or M=M1*M2
> >     and effectively solve the system inv(M)*A*X = inv(M)*B for X. If M is
> >     [] then a preconditioner is not applied.
> > ************
> >
> > Best,
> > Mohamad
> >
> > On Sat, Jan 7, 2012 at 5:39 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> >
> > On Jan 7, 2012, at 4:00 PM, Mohamad M. Nasr-Azadani wrote:
> >
> > > Hi guys,
> > >
> > > I am trying to narrow down an issue with my Poisson solver.
> > > I have the following problem setup
> > >
> > > Laplace(f) = rhs(x,z,y)
> > > 0 <= x,y,z <= (Lx,Ly,Lz)
> > >
> > > I solve the Poisson equation in three dimensions with the analytical function f(x,y,z) defined by
> > >
> > > f(x,z,y) = cos(2*pi*x/Lx)*cos(2*pi*y/Ly)*cos(2*pi*z/Lz) + K
> > > where Lx = Ly =Lz = 1.0 and K is a constant I use to set f(Lx,Ly,Lz) = 0.0.
> > >
> > > Second order descritization is used for the Poisson equation.
> > > Also, Neumann boundary condition is used everywhere, but I set the top-right-front node's value to zero to get rid of the Nullspaced matrix manually.
> >
> >   Please don't do this. That results in a unnecessaryly huge condition number. Use KSPSetNullSpace.()
> >
> >   Also if you are really solving the Poisson problem you should use multigrid; if simple geometry then geometric multigrid if complicated geometry probably easier to use hypre BoomerAMG. No sane person solves Poisson problem with anything but a multigrid or FFT based solver.
> >
> >   Barry
> >
> > > I use 20 grid points in each direction.
> > >
> > > The problem is:
> > > I use GMRES(20) without any preconditioners (rtol = 1e-12) to solve the linear system.
> > > It takes 77,000 iterations to converge!!!!
> > >
> > > For the size of only 8,000 unknowns, even though the lsys is not preconditioned, I guess that is a LOT of iterations.
> > > Next, I setup the exact same problem in MATLAB and use their GMRES solver function.
> > > I set the same parameters and MATLAB tells me that it converges using only 3870 iterations.
> > >
> > > I know that there might be some internal differences between MATLAB and PETSc's implementations of this method, but given the fact that these two solvers are not preconditioned, I am wondering about this big difference?
> > >
> > > Any ideas?
> > >
> > > Best,
> > > Mohamad
> > >
> >
> >
> 
>