[petsc-users] Question about memory usage in Multigrid preconditioner

[Barry Smith]

  Frank,

    You can run with -ksp_view_pre to have it "view" the KSP before the solve so hopefully it gets that far.

     Please run the problem that does fit with -memory_info when the problem completes it will show the "high water mark" for PETSc allocated memory and total memory used. We first want to look at these numbers to see if it is using more memory than you expect. You could also run with say half the grid spacing to see how the memory usage scaled with the increase in grid points. Make the runs also with -log_view and send all the output from these options.

   Barry

> On Jul 5, 2016, at 5:23 PM, frank <hengjiew at uci.edu> wrote:
> 
> Hi,
> 
> I am using the CG ksp solver and Multigrid preconditioner  to solve a linear system in parallel.
> I chose to use the 'Telescope' as the preconditioner on the coarse mesh for its good performance.
> The petsc options file is attached.
> 
> The domain is a 3d box.
> It works well when the grid is  1536*128*384 and the process mesh is 96*8*24. When I double the size of grid and keep the same process mesh and petsc options, I get an "out of memory" error from the super-cluster I am using.
> Each process has access to at least 8G memory, which should be more than enough for my application. I am sure that all the other parts of my code( except the linear solver ) do not use much memory. So I doubt if there is something wrong with the linear solver.
> The error occurs before the linear system is completely solved so I don't have the info from ksp view. I am not able to re-produce the error with a smaller problem either.
> In addition,  I tried to use the block jacobi as the preconditioner with the same grid and same decomposition. The linear solver runs extremely slow but there is no memory error.
> 
> How can I diagnose what exactly cause the error?
> Thank you so much.
> 
> Frank
> <petsc_options.txt>