memory check of /snes/example/tutorials/ex29.c
(Rebecca) Xuefei YUAN
xy2102 at columbia.edu
Mon Jul 27 17:14:32 CDT 2009
The memory status after running DMMGSetSNESLocal() is
Mem: 2033752k total, 1821800k used, 211952k free, 5944k buffers
then when it calls DMMGSolve(), the memory has been used up... till
corruption.
R
Quoting "(Rebecca) Xuefei YUAN" <xy2102 at columbia.edu>:
> Dear Matt,
>
> I ran the code on a 321*321 grid, with dof=4. The matrix is a sparse
> matrix with type aij.
>
> After set up user defined options calls, the memory status is
> Mem: 2033752k total, 607456k used, 1426296k free, 4832k buffers
>
> ierr = DMMGCreate(comm, parameters.numberOfLevels, &appCtx,
> &dmmg);CHKERRQ(ierr);
> ierr = DACreate2d(comm,DA_NONPERIODIC,DA_STENCIL_BOX, -5, -5,
> PETSC_DECIDE, PETSC_DECIDE, 4, 2, 0, 0, &da);CHKERRQ(ierr);
> ierr = DMMGSetDM(dmmg, (DM)da);CHKERRQ(ierr);
> ierr = DASetFieldName(DMMGGetDA(dmmg), 0, "phi");CHKERRQ(ierr);
> ierr = DASetFieldName(DMMGGetDA(dmmg), 1, "vz");CHKERRQ(ierr);
> ierr = DASetFieldName(DMMGGetDA(dmmg), 2, "psi");CHKERRQ(ierr);
> ierr = DASetFieldName(DMMGGetDA(dmmg), 3, "bz");CHKERRQ(ierr);
>
>
> before DAGetMatrix() called, the memory status is
> Mem: 2033752k total, 642940k used, 1390812k free, 4972k buffers
>
> ierr = DAGetMatrix(DMMGGetDA(dmmg), MATAIJ, &jacobian);CHKERRQ(ierr);
>
> In gdb, it uses around 500M memory after DAGetMatrix(), which I do not
> think it is right, since for a sparse matrix with 13 nonzeros per row,
> the memory it needs should be 321*321*4(dof)*13(nonzeros per
> row)*8(PetscReal) = 42865056 bytes ~ 40M. It is strange.
> i.e., after DAGetMatrix() call, the memory status is
>
> Mem: 2033752k total, 1152032k used, 881720k free, 5072k buffers
>
> Then when it goes the call of DMMGSetSNESLocal(), I found my memory is
> using till the message of corruption has appeared.
>
> ierr = DMMGSetSNESLocal(dmmg, FormFunctionLocal,
> FormJacobianLocal,0,0);CHKERRQ(ierr);
>
> The memory corruption happens. The error message is:
>
> 0 SNES Function norm 4.925849247379e-03
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: Out of memory. This could be due to allocating
> [0]PETSC ERROR: too large an object or bleeding by not properly
> [0]PETSC ERROR: destroying unneeded objects.
> [0]PETSC ERROR: Memory allocated 2134287012 Memory used by process 1630638080
> [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> [0]PETSC ERROR: Memory requested 327270824!
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 1, Thu Jan 1
> 13:54:27 CST 2009
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: /home/rebecca/linux/code/physics/qffxmhd/tests/qffxmhd3
> on a linux-gnu named YuanWork by rebecca Mon Jul 27 17:49:53 2009
> [0]PETSC ERROR: Libraries linked from
> /home/rebecca/soft/petsc-3.0.0-p1/linux-gnu-c-debug/lib
> [0]PETSC ERROR: Configure run at Mon Apr 20 16:41:56 2009
> [0]PETSC ERROR: Configure options
> --with-blas-lapack-dir=./externalpackages/fblaslapack-3.1.1/
> --download-mpich=1 --with-shared=0
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: PetscMallocAlign() line 61 in src/sys/memory/mal.c
> [0]PETSC ERROR: PetscTrMallocDefault() line 194 in src/sys/memory/mtr.c
> [0]PETSC ERROR: MatDuplicateNoCreate_SeqAIJ() line 3402 in
> src/mat/impls/aij/seq/aij.c
> [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line 1241 in
> src/mat/impls/aij/seq/aijfact.c
> [0]PETSC ERROR: MatILUFactorSymbolic() line 5243 in
> src/mat/interface/matrix.c
> [0]PETSC ERROR: PCSetUp_ILU() line 293 in src/ksp/pc/impls/factor/ilu/ilu.c
> [0]PETSC ERROR: PCSetUp() line 794 in src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: PCSetUp_MG() line 516 in src/ksp/pc/impls/mg/mg.c
> [0]PETSC ERROR: PCSetUp() line 794 in src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: KSPSolve() line 353 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: SNES_KSPSolve() line 2899 in src/snes/interface/snes.c
> [0]PETSC ERROR: SNESSolve_LS() line 191 in src/snes/impls/ls/ls.c
> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c
> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c
> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c
> [0]PETSC ERROR: Solve() line 318 in qffxmhd.c
> [0]PETSC ERROR: main() line 172 in qffxmhd.c
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0[unset]:
> aborting job:
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
>
> Program exited with code 01.
>
>
> 0 SNES Function norm 4.925849247379e-03
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: Out of memory. This could be due to allocating
> [0]PETSC ERROR: too large an object or bleeding by not properly
> [0]PETSC ERROR: destroying unneeded objects.
> [0]PETSC ERROR: Memory allocated 2134287012 Memory used by process 1630638080
> [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> [0]PETSC ERROR: Memory requested 327270824!
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 1, Thu Jan 1
> 13:54:27 CST 2009
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: /home/rebecca/linux/code/physics/qffxmhd/tests/qffxmhd3
> on a linux-gnu named YuanWork by rebecca Mon Jul 27 17:49:53 2009
> [0]PETSC ERROR: Libraries linked from
> /home/rebecca/soft/petsc-3.0.0-p1/linux-gnu-c-debug/lib
> [0]PETSC ERROR: Configure run at Mon Apr 20 16:41:56 2009
> [0]PETSC ERROR: Configure options
> --with-blas-lapack-dir=./externalpackages/fblaslapack-3.1.1/
> --download-mpich=1 --with-shared=0
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: PetscMallocAlign() line 61 in src/sys/memory/mal.c
> [0]PETSC ERROR: PetscTrMallocDefault() line 194 in src/sys/memory/mtr.c
> [0]PETSC ERROR: MatDuplicateNoCreate_SeqAIJ() line 3402 in
> src/mat/impls/aij/seq/aij.c
> [0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line 1241 in
> src/mat/impls/aij/seq/aijfact.c
> [0]PETSC ERROR: MatILUFactorSymbolic() line 5243 in
> src/mat/interface/matrix.c
> [0]PETSC ERROR: PCSetUp_ILU() line 293 in src/ksp/pc/impls/factor/ilu/ilu.c
> [0]PETSC ERROR: PCSetUp() line 794 in src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: PCSetUp_MG() line 516 in src/ksp/pc/impls/mg/mg.c
> [0]PETSC ERROR: PCSetUp() line 794 in src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: KSPSolve() line 353 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: SNES_KSPSolve() line 2899 in src/snes/interface/snes.c
> [0]PETSC ERROR: SNESSolve_LS() line 191 in src/snes/impls/ls/ls.c
> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c
> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c
> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c
> [0]PETSC ERROR: Solve() line 318 in qffxmhd.c
> [0]PETSC ERROR: main() line 172 in qffxmhd.c
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0[unset]:
> aborting job:
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
>
> Program exited with code 01.
>
> I thought it might because of the unfreed memory, so I picked up ex29.c
> as a comparision.
>
> Thanks,
>
> Rebecca
>
>
> Quoting Matthew Knepley <knepley at gmail.com>:
>
>> On Mon, Jul 27, 2009 at 4:34 PM, (Rebecca) Xuefei YUAN
>> <xy2102 at columbia.edu>wrote:
>>
>>> Those unfreed bytes cause "out of memory" when it runs at bigger grid
>>> sizes. So I have to find out those unfreed memory and free them... Any
>>> suggestions?
>>
>>
>> Not from what you mailed in. On that DA line, I see PetscHeaderCreate(). Is
>> that what you see?
>>
>> Matt
>>
>>
>>>
>>> Thanks,
>>>
>>> R
>>>
>>>
>>> Quoting Matthew Knepley <knepley at gmail.com>:
>>>
>>> On Mon, Jul 27, 2009 at 3:51 PM, (Rebecca) Xuefei YUAN
>>>> <xy2102 at columbia.edu>wrote:
>>>>
>>>> Hi,
>>>>>
>>>>> My own code has some left bytes still reachable according to valgrind,
>>>>> then
>>>>> I use two different version petsc (2.3.3-p15 and 3.0.0-p1) to compile and
>>>>> make the files, it gives me different number of bytes left still
>>>>> reachable.
>>>>> Moreover, I picked up the /snes/example/tutorials/ex29.c as another
>>>>> example,
>>>>> and found that some bytes are still reachable, what is the cause of it?
>>>>> It
>>>>> shows that it is from DACreate2D() and the I use -malloc_dump to get
>>>>> those
>>>>> unfreed informations.
>>>>>
>>>>> I understand that for those 5 loss record, the 2nd, 3rd and 4th are true
>>>>> for all examples, but where do 1st and 5th ones come from? Also, the
>>>>> -malloc_dump information shows that there are
>>>>> "[0]Total space allocated 37780 bytes",
>>>>> but valgrind gives the information as
>>>>> "==26628== still reachable: 132,828 bytes in 323 blocks"
>>>>>
>>>>> Why there is a big difference?
>>>>>
>>>>
>>>>
>>>> 1 is fine. It is from PMPI setup, which has some bytes not freed from
>>>> setting up the MPI
>>>> processes. The last one looks like an unfreed header for a DA, which is
>>>> strange.
>>>>
>>>> Matt
>>>>
>>>>
>>>>
>>>>> Thanks very much!
>>>>>
>>>>> Rebecca
>>>>>
>>>>> Here is the message from valgrind of running ex29:
>>>>> ==26628== 32 bytes in 2 blocks are still reachable in loss record 1 of 5
>>>>> ==26628== at 0x4022AB8: malloc (vg_replace_malloc.c:207)
>>>>> ==26628== by 0x86F9A78: MPID_VCRT_Create (mpid_vc.c:62)
>>>>> ==26628== by 0x86F743A: MPID_Init (mpid_init.c:116)
>>>>> ==26628== by 0x86D040B: MPIR_Init_thread (initthread.c:288)
>>>>> ==26628== by 0x86CFF2D: PMPI_Init (init.c:106)
>>>>> ==26628== by 0x8613D69: PetscInitialize (pinit.c:503)
>>>>> ==26628== by 0x804B796: main (ex29.c:139)
>>>>> ==26628==
>>>>> ==26628==
>>>>> ==26628== 156 (36 direct, 120 indirect) bytes in 1 blocks are definitely
>>>>> lost in loss record 2 of 5
>>>>> ==26628== at 0x4022AB8: malloc (vg_replace_malloc.c:207)
>>>>> ==26628== by 0x429B3E2: (within /lib/tls/i686/cmov/libc-2.7.so)
>>>>> ==26628== by 0x429BC2D: __nss_database_lookup (in /lib/tls/i686/cmov/
>>>>> libc-2.7.so)
>>>>> ==26628== by 0x4732FDB: ???
>>>>> ==26628== by 0x473413C: ???
>>>>> ==26628== by 0x4247D15: getpwuid_r (in /lib/tls/i686/cmov/libc-2.7.so
>>>>> )
>>>>> ==26628== by 0x424765D: getpwuid (in /lib/tls/i686/cmov/libc-2.7.so)
>>>>> ==26628== by 0x8623509: PetscGetUserName (fuser.c:68)
>>>>> ==26628== by 0x85E0CF0: PetscErrorPrintfInitialize (errtrace.c:68)
>>>>> ==26628== by 0x8613E23: PetscInitialize (pinit.c:518)
>>>>> ==26628== by 0x804B796: main (ex29.c:139)
>>>>> ==26628==
>>>>> ==26628==
>>>>> ==26628== 40 bytes in 5 blocks are indirectly lost in loss record 3 of 5
>>>>> ==26628== at 0x4022AB8: malloc (vg_replace_malloc.c:207)
>>>>> ==26628== by 0x429AFBB: __nss_lookup_function (in /lib/tls/i686/cmov/
>>>>> libc-2.7.so)
>>>>> ==26628== by 0x4732FFB: ???
>>>>> ==26628== by 0x473413C: ???
>>>>> ==26628== by 0x4247D15: getpwuid_r (in /lib/tls/i686/cmov/libc-2.7.so
>>>>> )
>>>>> ==26628== by 0x424765D: getpwuid (in /lib/tls/i686/cmov/libc-2.7.so)
>>>>> ==26628== by 0x8623509: PetscGetUserName (fuser.c:68)
>>>>> ==26628== by 0x85E0CF0: PetscErrorPrintfInitialize (errtrace.c:68)
>>>>> ==26628== by 0x8613E23: PetscInitialize (pinit.c:518)
>>>>> ==26628== by 0x804B796: main (ex29.c:139)
>>>>> ==26628==
>>>>> ==26628==
>>>>> ==26628== 80 bytes in 5 blocks are indirectly lost in loss record 4 of 5
>>>>> ==26628== at 0x4022AB8: malloc (vg_replace_malloc.c:207)
>>>>> ==26628== by 0x428839B: tsearch (in /lib/tls/i686/cmov/libc-2.7.so)
>>>>> ==26628== by 0x429AF7D: __nss_lookup_function (in /lib/tls/i686/cmov/
>>>>> libc-2.7.so)
>>>>> ==26628== by 0x4732FFB: ???
>>>>> ==26628== by 0x473413C: ???
>>>>> ==26628== by 0x4247D15: getpwuid_r (in /lib/tls/i686/cmov/libc-2.7.so
>>>>> )
>>>>> ==26628== by 0x424765D: getpwuid (in /lib/tls/i686/cmov/libc-2.7.so)
>>>>> ==26628== by 0x8623509: PetscGetUserName (fuser.c:68)
>>>>> ==26628== by 0x85E0CF0: PetscErrorPrintfInitialize (errtrace.c:68)
>>>>> ==26628== by 0x8613E23: PetscInitialize (pinit.c:518)
>>>>> ==26628== by 0x804B796: main (ex29.c:139)
>>>>> ==26628==
>>>>> ==26628==
>>>>> ==26628== 132,796 bytes in 321 blocks are still reachable in loss record
>>>>> 5
>>>>> of 5
>>>>> ==26628== at 0x4022AB8: malloc (vg_replace_malloc.c:207)
>>>>> ==26628== by 0x85EF3AC: PetscMallocAlign (mal.c:40)
>>>>> ==26628== by 0x85F049B: PetscTrMallocDefault (mtr.c:194)
>>>>> ==26628== by 0x81BCD3F: DACreate2d (da2.c:364)
>>>>> ==26628== by 0x804BAFB: main (ex29.c:153)
>>>>> ==26628==
>>>>> ==26628== LEAK SUMMARY:
>>>>> ==26628== definitely lost: 36 bytes in 1 blocks.
>>>>> ==26628== indirectly lost: 120 bytes in 10 blocks.
>>>>> ==26628== possibly lost: 0 bytes in 0 blocks.
>>>>> ==26628== still reachable: 132,828 bytes in 323 blocks.
>>>>> ==26628== suppressed: 0 bytes in 0 blocks.
>>>>>
>>>>>
>>>>> --
>>>>> (Rebecca) Xuefei YUAN
>>>>> Department of Applied Physics and Applied Mathematics
>>>>> Columbia University
>>>>> Tel:917-399-8032
>>>>> www.columbia.edu/~xy2102 <http://www.columbia.edu/%7Exy2102> <
>>>>> http://www.columbia.edu/%7Exy2102>
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments
>>>> is infinitely more interesting than any results to which their experiments
>>>> lead.
>>>> -- Norbert Wiener
>>>>
>>>>
>>>
>>>
>>> --
>>> (Rebecca) Xuefei YUAN
>>> Department of Applied Physics and Applied Mathematics
>>> Columbia University
>>> Tel:917-399-8032
>>> www.columbia.edu/~xy2102 <http://www.columbia.edu/%7Exy2102>
>>>
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their experiments
>> is infinitely more interesting than any results to which their experiments
>> lead.
>> -- Norbert Wiener
>>
>
>
>
> --
> (Rebecca) Xuefei YUAN
> Department of Applied Physics and Applied Mathematics
> Columbia University
> Tel:917-399-8032
> www.columbia.edu/~xy2102
--
(Rebecca) Xuefei YUAN
Department of Applied Physics and Applied Mathematics
Columbia University
Tel:917-399-8032
www.columbia.edu/~xy2102
More information about the petsc-users
mailing list