not enough memory?!

[Zhifeng Sheng]

Matthew Knepley wrote:

>The routine which calculates ICC requested 3G, so it seems that the matrix
>is too dense or large to be factored on this machine.
>
>  Matt
>
>On Nov 2, 2007 7:54 AM, Zhifeng Sheng <z.sheng at ewi.tudelft.nl> wrote:
>  
>
>>Dear all
>>
>>I have a problem for memory management.
>>
>>I implemented 3d FEM code and at this moment SeqSBAIJ is used to store
>>the system matrix, then I used CG method and ICC(k) preconditioner.
>>
>>The test configuration is not very big, I succeeded in constructing the
>>system matrix and construct the preconditioner. But when I need to setup
>>the KSP solver, I got the error message below ( I also dumped some
>>information about solver and system matrix):
>>
>>KSP Object:
>>  type: cg
>>  maximum iterations=10000, initial guess is zero
>>  tolerances:  relative=1e-09, absolute=1e-50, divergence=10000
>>  left preconditioning
>>PC Object:
>>  type: icc
>>    ICC: 2 levels of fill
>>    ICC: factor fill ratio allocated 1
>>  linear system matrix = precond matrix:
>>  Matrix Object:
>>    type=seqsbaij, rows=435450, cols=435450
>>    total: nonzeros=9470450, allocated nonzeros=21772580
>>        block size is 1
>>[0]PETSC ERROR: --------------------- Error Message
>>------------------------------------
>>[0]PETSC ERROR: Out of memory. This could be due to allocating
>>[0]PETSC ERROR: too large an object or bleeding by not properly
>>[0]PETSC ERROR: destroying unneeded objects.
>>[0]PETSC ERROR: Memory allocated 375435120 Memory used by process 428597248
>>[0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
>>[0]PETSC ERROR: Memory requested 3108507040!
>>[0]PETSC ERROR:
>>------------------------------------------------------------------------
>>[0]PETSC ERROR: Petsc Release Version 2.3.2, Patch 10, Wed Mar 28
>>19:13:22 CDT 2007 HG revision: d7298c71db7f5e767f359ae35d33cab3bed44428
>>[0]PETSC ERROR: See docs/changes/index.html for recent updates.
>>[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>[0]PETSC ERROR: See docs/index.html for manual pages.
>>[0]PETSC ERROR:
>>------------------------------------------------------------------------
>>[0]PETSC ERROR: bin/main on a linux-gnu named callisto by zhifeng Fri
>>Nov  2 13:26:39 2007
>>[0]PETSC ERROR: Libraries linked from
>>/u/01/01/zhifeng/install/lib/linux-gnu-c-debug
>>[0]PETSC ERROR: Configure run at Wed Aug  8 13:46:26 2007
>>[0]PETSC ERROR: Configure options --with-cc=gcc --with-fc=g77
>>--download-f-blas-lapack=1 --download-mpich=1
>>--prefix=/u/01/01/zhifeng/install --with-shared=0
>>[0]PETSC ERROR:
>>------------------------------------------------------------------------
>>[0]PETSC ERROR: PetscMallocAlign() line 61 in src/sys/memory/mal.c
>>[0]PETSC ERROR: PetscTrMallocDefault() line 194 in src/sys/memory/mtr.c
>>[0]PETSC ERROR: PetscFreeSpaceGet() line 14 in src/mat/utils/freespace.c
>>[0]PETSC ERROR: MatICCFactorSymbolic_SeqSBAIJ() line 1648 in
>>src/mat/impls/sbaij/seq/sbaijfact2.c
>>[0]PETSC ERROR: MatICCFactorSymbolic() line 4611 in
>>src/mat/interface/matrix.c
>>[0]PETSC ERROR: PCSetup_ICC() line 117 in src/ksp/pc/impls/factor/icc/icc.c
>>[0]PETSC ERROR: PCSetUp() line 801 in src/ksp/pc/interface/precon.c
>>[0]PETSC ERROR: KSPSetUp() line 234 in src/ksp/ksp/interface/itfunc.c
>>[0]PETSC ERROR: KSPSolve() line 338 in src/ksp/ksp/interface/itfunc.c
>>=====================the solver convergence info======================
>>Convergence in 0 iterations.
>>=====================================================================
>>writing solution to file temp_H.vtk ...
>>number of unknowns >>435450
>>finishing the numeric solver ...
>>LSFIM hybrid method for four domain problem
>>deallocating memory of domain class ...
>>
>>
>>I debug the code, and the error is dumped by the funciton KSPSetUp()
>>which should not take so much memory, but still 3108507040 memory was
>>requested .... I really could not figure out where I need them...
>>
>>Any help will be appreciated.
>>
>>Thank you all
>>Best regards
>>Zhifeng
>>
>>
>>    
>>
>
>
>
>  
>
So ... computation of the preconditioner is actually called by 
KSPSetUp() instead of PCGetFactoredMatrix(prec,&M)?

Could you please tell me when the system matrix be factorized 
(incompletely)? thank you.

Zhifeng