[petsc-users] FormFunction: high Memory requested

Sun Oct 6 14:54:38 CDT 2019

How can we save memory during Jacobin computation? I use DMPlex for my DM layout. Is it better to set fill pattern of Jacobin matrix? can I use DASetBlockFills()?
Thanks
Amir

On Oct 6 2019, at 9:41 pm, Matthew Knepley <knepley at gmail.com> wrote:
> On Sat, Oct 5, 2019 at 1:12 AM Amir <mhbaghaei at mail.sjtu.edu.cn (https://link.getmailspring.com/link/D533A949-3DE7-40D6-8A83-7D4070DBD509@getmailspring.com/0?redirect=mailto%3Amhbaghaei%40mail.sjtu.edu.cn&recipient=cGV0c2MtdXNlcnNAbWNzLmFubC5nb3Y%3D)> wrote:
>
> > I tried to run with -pc_none or using other preconditioner than LU. The memory pattern remains the same, although the newton could not solve. For a Jacobian size of 180K*180K, do you think 5GB memory is much?
>
>
> 1.8e5 * 1.8e5 * 8 = 2.6e11 bytes so 5Gb is 2% fill in your matrix, which might be completely reasonable.
>
> Matt
>
> > Thanks
> > Amir
> > On Oct 5 2019, at 5:55 am, Matthew Knepley <knepley at gmail.com (mailto:knepley at gmail.com)> wrote:
> > > On Fri, Oct 4, 2019 at 4:27 PM Amir via petsc-users <petsc-users at mcs.anl.gov (https://link.getmailspring.com/link/6B28CA58-29C6-4D61-A0E9-873FD4DCB12A@getmailspring.com/0?redirect=mailto%3Apetsc-users%40mcs.anl.gov&recipient=a25lcGxleUBnbWFpbC5jb20%3D)> wrote:
> > >
> > > > Hi
> > > > To test my FormFunction, running the code on a coarse mesh, the memory usage is normal. However, when I try to use higher mesh resolution, the memory behavior is abnormal. It goes up till the program finished with Memory requested message. To better see the memory usage, I run with -log_view -malloc_test -malloc_dump. All of vectors created, were destructed. Can you help me through this. Is it really because I use higher mesh resolution? Thanks for your time.
> > >
> > >
> > > Probably yes. The memory for LU goes up superlinearly and can be quite large.
> > >
> > > Thanks,
> > >
> > > Matt
> > >
> > > > Amir
> > > >
> > > > [0]PETSC ERROR: Memory requested 8589934260
> > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
> > > > [0]PETSC ERROR: Petsc Release Version 3.9.1, unknown
> > > > [0]PETSC ERROR: ./main on a mpich-gcc-dbg named amir by amir Sat Oct 5 04:06:01 2019
> > > > [0]PETSC ERROR: Configure options --download-mpich --download-fblaslapack --with-shared-libraries=1 --with-blaslapack-dir=/usr --with-x-dir=/ --download-mumps --download-superlu_dist --download-hypre --download-ml --download-hdf5 --download-cmake --download-scalapack -download-parmetis -download-metis --with-mpich=1 --with-fortran-kernels=generic --with-debugging=0
> > > > [0]PETSC ERROR: #1 PetscFreeSpaceGet() line 11 in /home/amir/petsc/src/mat/utils/freespace.c
> > > > [0]PETSC ERROR: #2 PetscTrMallocDefault() line 183 in /home/amir/petsc/src/sys/memory/mtr.c
> > > > [0]PETSC ERROR: #3 PetscMallocA() line 390 in /home/amir/petsc/src/sys/memory/mal.c
> > > > [0]PETSC ERROR: #4 PetscFreeSpaceGet() line 11 in /home/amir/petsc/src/mat/utils/freespace.c
> > > > [0]PETSC ERROR: #5 MatLUFactorSymbolic_SeqAIJ() line 349 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> > > > [0]PETSC ERROR: #6 MatLUFactorSymbolic() line 2975 in /home/amir/petsc/src/mat/interface/matrix.c
> > > > [0]PETSC ERROR: #7 PCSetUp_LU() line 95 in /home/amir/petsc/src/ksp/pc/impls/factor/lu/lu.c
> > > > [0]PETSC ERROR: #8 PCSetUp() line 923 in /home/amir/petsc/src/ksp/pc/interface/precon.c
> > > > [0]PETSC ERROR: #9 KSPSetUp() line 381 in /home/amir/petsc/src/ksp/ksp/interface/itfunc.c
> > > > [0]PETSC ERROR: #10 KSPSolve() line 612 in /home/amir/petsc/src/ksp/ksp/interface/itfunc.c
> > > > [0]PETSC ERROR: #11 SNESSolve_NEWTONLS() line 224 in /home/amir/petsc/src/snes/impls/ls/ls.c
> > > > [0]PETSC ERROR: #12 SNESSolve() line 4312 in /home/amir/petsc/src/snes/interface/snes.c
> > > >
> > > > [ 0]48 bytes PetscFreeSpaceGet() line 10 in /home/amir/petsc/src/mat/utils/freespace.c
> > > > [ 0]1356241520 bytes PetscFreeSpaceGet() line 11 in /home/amir/petsc/src/mat/utils/freespace.c
> > > > [ 0]48 bytes PetscFreeSpaceGet() line 10 in /home/amir/petsc/src/mat/utils/freespace.c
> > > > [ 0]2227520 bytes MatLUFactorSymbolic_SeqAIJ() line 308 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> > > > [ 0]23216 bytes PetscBTCreate() line 75 in /home/amir/petsc/include/petscbt.h
> > > > [ 0]742512 bytes MatLUFactorSymbolic_SeqAIJ() line 306 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> > > > [ 0]742512 bytes MatLUFactorSymbolic_SeqAIJ() line 301 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> > > > [ 0]742512 bytes MatLUFactorSymbolic_SeqAIJ() line 300 in /home/amir/petsc/src/mat/impls/aij/seq/aijfact.c
> > > > [ 0]16 bytes PetscLayoutSetUp() line 141 in /home/amir/petsc/src/vec/is/utils/pmap.c
> > > > [ 0]16 bytes PetscStrallocpy() line 189 in /home/amir/petsc/src/sys/utils/str.c
> > > > [ 0]32 bytes PetscStrallocpy() line 189 in /home/amir/petsc/src/sys/utils/str.c
> > > > [ 0]32 bytes PetscFunctionListAdd_Private() line 213 in /home/amir/petsc/src/sys/dll/reg.c
> > > > [ 0]16 bytes ISCreate_General() line 740 in /home/amir/petsc/src/vec/is/is/impls/general/general.c
> > > > [ 0]48 bytes PetscLayoutCreate() line 48 in /home/amir/petsc/src/vec/is/utils/pmap.c
> > > > [ 0]784 bytes ISCreate() line 39 in /home/amir/petsc/src/vec/is/is/interface/isreg.c
> > > > [ 0]742512 bytes ISInvertPermutation_General() line 205 in /home/amir/petsc/src/vec/is/is/impls/general/general.c
> > > >
> > >
> > >
> > > --
> > > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> > > -- Norbert Wiener
> > >
> > >
> > > https://www.cse.buffalo.edu/~knepley/ (https://link.getmailspring.com/link/6B28CA58-29C6-4D61-A0E9-873FD4DCB12A@getmailspring.com/1?redirect=http%3A%2F%2Fwww.cse.buffalo.edu%2F~knepley%2F&recipient=a25lcGxleUBnbWFpbC5jb20%3D)
>
> --
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
>
>
> https://www.cse.buffalo.edu/~knepley/ (https://link.getmailspring.com/link/D533A949-3DE7-40D6-8A83-7D4070DBD509@getmailspring.com/1?redirect=http%3A%2F%2Fwww.cse.buffalo.edu%2F~knepley%2F&recipient=cGV0c2MtdXNlcnNAbWNzLmFubC5nb3Y%3D)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20191007/208c26f2/attachment.html>