[petsc-users] FormFunction: high Memory requested

[Smith, Barry F.]

  If you need to use a direct solver than you need to start running in parallel and using MUMPS or SuperLU_DIST or Pastix as your direct solver. But direct solvers memory usage grows much more than linearly with problem size limiting how big you can go. 

  You should also investigate iterative solvers, their memory usage is generally order the problem size (or a tiny bit more like a log) so you can do much larger problems: see https://www.mcs.anl.gov/petsc/documentation/faq.html#newton and https://www.mcs.anl.gov/petsc/documentation/faq.html#kspdiverged for tips on figuring out why solvers are failing.

  Barry


> On Oct 4, 2019, at 3:26 PM, Amir via petsc-users <petsc-users at mcs.anl.gov> wrote:
> 
> Hi
> To test my FormFunction, running the code on a coarse mesh, the memory usage is normal. However, when I try to use higher mesh resolution, the memory behavior is abnormal. It goes up till the program finished with Memory requested message. To better see the memory usage, I run with -log_view -malloc_test  -malloc_dump. All of vectors created, were destructed. Can you help me through this. Is it really because I use higher mesh resolution? Thanks for your time.
> Amir
> 
> [0]PETSC ERROR: Memory requested 8589934260
> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.9.1, unknown
> [0]PETSC ERROR: ./main on a mpich-gcc-dbg named amir by amir Sat Oct  5 04:06:01 2019
> [0]PETSC ERROR: Configure options --download-mpich --download-fblaslapack --with-shared-libraries=1 --with-blaslapack-dir=/usr --with-x-dir=/ --download-mumps --download-superlu_dist --download-hypre --download-ml --download-hdf5 --download-cmake --download-scalapack -download-parmetis -download-metis --with-mpich=1 --with-fortran-kernels=generic --with-debugging=0
> [0]PETSC ERROR: #1 PetscFreeSpaceGet() line 11 in /home/amir/petsc/src/mat/utils/freespace.c
> [0]PETSC ERROR: #2 PetscTrMallocDefault() line 183 in /home/amir/petsc/src/sys/memory/mtr.c
> [0]PETSC ERROR: #3 PetscMallocA() line 390 in /home/amir/petsc/src/sys/memory/mal.c
> [0]PETSC ERROR: #4 PetscFreeSpaceGet() line 11 in /home/amir/petsc/src/mat/utils/freespace.c
> [0]PETSC ERROR: #5 MatLUFactorSymbolic_SeqAIJ() line 349 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> [0]PETSC ERROR: #6 MatLUFactorSymbolic() line 2975 in /home/amir/petsc/src/mat/interface/matrix.c
> [0]PETSC ERROR: #7 PCSetUp_LU() line 95 in /home/amir/petsc/src/ksp/pc/impls/factor/lu/lu.c
> [0]PETSC ERROR: #8 PCSetUp() line 923 in /home/amir/petsc/src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: #9 KSPSetUp() line 381 in /home/amir/petsc/src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: #10 KSPSolve() line 612 in /home/amir/petsc/src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: #11 SNESSolve_NEWTONLS() line 224 in /home/amir/petsc/src/snes/impls/ls/ls.c
> [0]PETSC ERROR: #12 SNESSolve() line 4312 in /home/amir/petsc/src/snes/interface/snes.c
> 
> [ 0]48 bytes PetscFreeSpaceGet() line 10 in /home/amir/petsc/src/mat/utils/freespace.c
> [ 0]1356241520 bytes PetscFreeSpaceGet() line 11 in /home/amir/petsc/src/mat/utils/freespace.c
> [ 0]48 bytes PetscFreeSpaceGet() line 10 in /home/amir/petsc/src/mat/utils/freespace.c
> [ 0]2227520 bytes MatLUFactorSymbolic_SeqAIJ() line 308 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> [ 0]23216 bytes PetscBTCreate() line 75 in /home/amir/petsc/include/petscbt.h
> [ 0]742512 bytes MatLUFactorSymbolic_SeqAIJ() line 306 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> [ 0]742512 bytes MatLUFactorSymbolic_SeqAIJ() line 301 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> [ 0]742512 bytes MatLUFactorSymbolic_SeqAIJ() line 300 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> [ 0]16 bytes PetscLayoutSetUp() line 141 in /home/amir/petsc/src/vec/is/utils/pmap.c
> [ 0]16 bytes PetscStrallocpy() line 189 in /home/amir/petsc/src/sys/utils/str.c
> [ 0]32 bytes PetscStrallocpy() line 189 in /home/amir/petsc/src/sys/utils/str.c
> [ 0]32 bytes PetscFunctionListAdd_Private() line 213 in /home/amir/petsc/src/sys/dll/reg.c
> [ 0]16 bytes ISCreate_General() line 740 in /home/amir/petsc/src/vec/is/is/impls/general/general.c
> [ 0]48 bytes PetscLayoutCreate() line 48 in /home/amir/petsc/src/vec/is/utils/pmap.c
> [ 0]784 bytes ISCreate() line 39 in /home/amir/petsc/src/vec/is/is/interface/isreg.c
> [ 0]742512 bytes ISInvertPermutation_General() line 205 in /home/amir/petsc/src/vec/is/is/impls/general/general.c
> 
>