Question about memory used by PetsC Objects

Barry Smith bsmith at mcs.anl.gov
Fri Apr 13 21:31:58 CDT 2007 

  Laurent,

    It is a problem; we just don't have a good solution to the control of memory 
and measuring how much is used.

   Barry


On Sat, 14 Apr 2007, Laurent Nguyen wrote:

> Yes, I know, this is a problem on all computer systems.But, I hoped that your profiling functions could do something like that (this is the name of functions which give me this hope)
> 
> The reason why I need that is because of my computer system (IBM SP4). I've limits of memory and if I use more memory than allowed, the system break my program. For example, in a C code without allocation (only static memory), I can calculate the maximum of memory used by my program. With PetsC, I don't know all the mechanism behind Petsc Objects. This why I sent you theses mails ^^
> 
> I would have thought this is a problem for users who make computations on this sort of system (with limits etc...), but it seems this is not the case.
> 
> Sorry for the disturbance and thank you for your help
> 
> Laurent
> 
> On Fri, Apr 13, 2007 at 11:14:34AM -0500, Barry Smith wrote:
> > 
> >   Matt,
> > 
> >    I do not understand why one would want this, doesn't PETSc already keep
> > a current amount used (memory malloced but not freed) and the maximum current
> > amount used? 
> > 
> >   NGUYEN,
> > 
> >    Once PETSc frees memory (or any program) it is not returned to the operating
> > system, rather it is reused in a future malloc if possible. PETSc cannot give
> > the TOTAL memory used by the process since that includes pages of the program
> > and any memory your code is using and there is no good Unix command to get that
> > information on all computer systems.
> > 
> > On Fri, 13 Apr 2007, Matthew Knepley wrote:
> > 
> > > There is a way to get exactly what you want which is not really very hard.
> > > Petsc allows you to easily override PetscMalloc() and PetscFree() which
> > > we use for everything. You could just keep a running static tally of the sizes
> > > and then call the regular version. This would only take a few lines of code.
> > > Take a look at
> > > 
> > > http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Sys/PetscSetMalloc.html
> > > 
> > >   Matt
> > > 
> > > On 4/13/07, Laurent Nguyen <laurent.nguyen at idris.fr> wrote:
> > > > Yes, I've tested the argument -memory_info. I've seen the memory given
> > > > to me is the same memory given by a PetscMemoryGetCurrentUsage() before
> > > > PetscFinalize(). I didn't know it was the same ^^.
> > > > 
> > > > It is not the memory I need. I would like to have the real memory used
> > > > by my program (It is a little difficult to explain clearly): for
> > > > example, if I create 2 vectors of size 100, the program is consuming
> > > > 2*8*100 bytes. But later, when I destroy only 1 vector, the programs is
> > > > consuming 1*8*100 bytes. This is this kind of information that I need
> > > > (more precisely, the max). I tell you that because, in my context of
> > > > computation, I've only 1.5Go of memory per processors. And the memory
> > > > given by the argument -memory_info was 2.4Go. So I was perplex about
> > > > that. But now I understand how your function is running.
> > > > 
> > > > But thanks to yours functions, I can now better manage the memory of my
> > > > objects!
> > > > 
> > > > Regards,
> > > > 
> > > > **************************************
> > > > NGUYEN Anh-Khai Laurent
> > > > Equipe Support Utilisateur
> > > > 
> > > > Email    :    laurent.nguyen at idris.fr
> > > > T?l      :    01.69.35.85.66
> > > > Adresse  :    IDRIS - Institut du D?veloppement et des Ressources en
> > > >                Informatique Scientifique
> > > >                CNRS
> > > >                Batiment 506
> > > >                BP 167
> > > >                F - 91403 ORSAY Cedex
> > > > Site Web :    http://www.idris.fr
> > > > **************************************
> > > > 
> > > > Matthew Knepley a ?crit :
> > > > > On 4/13/07, Laurent Nguyen <laurent.nguyen at idris.fr> wrote:
> > > > >> Barry,
> > > > >>
> > > > >> I thank you for your help. So I tried to apply the functions. It seems
> > > > >> that PetscMemoryGetCurrentUsage() gives the total amount of memory
> > > > >> allocated at the point of the program where it is placed. But, if a
> > > > >> object is deallocated, PetscMemoryGetCurrentUsage() doesn't take it.
> > > > >>
> > > > >> I would like to try PetscMemoryGetMaximumUsage() at the end of the
> > > > >> program hoping it gives me the maximum memory used by my program but
> > > > >> there are problems. When I put PetscMemoryGetMaximumUsage(), I've this
> > > > >> error at the execution:
> > > > >>
> > > > >> [0]PETSC ERROR: --------------------- Error Message
> > > > >> ------------------------------------
> > > > >> [0]PETSC ERROR: Object is in wrong state!
> > > > >> [0]PETSC ERROR: To use this function you must first call
> > > > >> PetscMemorySetGetMaximumUsage()!
> > > > >>
> > > > >> So I put PetscMemorySetGetMaximumUsage() before, but at linking, it
> > > > >> didn't find PetscMemorySetGetMaximumUsage in the petsc libraries. I
> > > > >> thought I made a bad install but when I tried on another machine, I've
> > > > >> the same problem.
> > > > >
> > > > > The function is definitely there (check src/sys/memory/mem.c). You can try
> > > > > a short test:
> > > > >
> > > > >  cd src/ksp/ksp/examples/tutorials
> > > > >  make ex2
> > > > >  ./ex2 -memory_info
> > > > >  ./ex2 -malloc_log
> > > > >
> > > > > Also, I have fixed the specific logging errors you pointed out in the dev
> > > > > copy, but we are doing a code review to make sure no other omissions
> > > > > are lurking.
> > > > >
> > > > >  Matt
> > > > >
> > > > >> Best regards,
> > > > >>
> > > > >> **************************************
> > > > >> NGUYEN Anh-Khai Laurent
> > > > >> Equipe Support Utilisateur
> > > > >>
> > > > >> Email    :    laurent.nguyen at idris.fr
> > > > >> T?l      :    01.69.35.85.66
> > > > >> Adresse  :    IDRIS - Institut du D?veloppement et des Ressources en
> > > > >>                Informatique Scientifique
> > > > >>                CNRS
> > > > >>                Batiment 506
> > > > >>                BP 167
> > > > >>                F - 91403 ORSAY Cedex
> > > > >> Site Web :    http://www.idris.fr
> > > > >> **************************************
> > > > >>
> > > > >> Barry Smith a ?crit :
> > > > >> >   Laurent,
> > > > >> >
> > > > >> >    The memory usage printed with -log_summary is not correct.
> > > > >> > Much of the memory used is not printed out.
> > > > >> >
> > > > >> >    You can use PetscMemoryGetCurrentUsage() and
> > > > >> PetscMemoryGetMaximumUsage()
> > > > >> > in your code before and after creating a filling in PETSc objects
> > > > >> > to see how much memory they are used. You can also use
> > > > >> PetscMallocDump()
> > > > >> > at any point to see exactly where in PETSc and how much all of PETSc's
> > > > >> > memory was allocated.
> > > > >> >
> > > > >> >    Barry
> > > > >> >
> > > > >> >
> > > > >> > On Thu, 12 Apr 2007, Laurent Nguyen wrote:
> > > > >> >
> > > > >> >> Hi,
> > > > >> >>
> > > > >> >> I'm a Petsc user, but I'm beginning to use petsc in a supercomputation
> > > > >> >> context. So, I'm trying parallel versions of my old mono-processed
> > > > >> programs.
> > > > >> >> But I've some difficulties to determine how much memory my program
> > > > >> use. (I
> > > > >> >> give a little example). In this example on one processor (with
> > > > >> these command
> > > > >> >> line arguments: -mat_type mpiaij -pc_type none -log_summary), I
> > > > >> created one
> > > > >> >> mpiaij matrix (size = 500*500) and two vectors (size = 500). I've this
> > > > >> >> information from output:
> > > > >> >>
> > > > >> >>   Matrix Object:
> > > > >> >>     type=mpiaij, rows=250000, cols=250000
> > > > >> >>     total: nonzeros=749996, allocated nonzeros=1750000
> > > > >> >>
> > > > >> >> To me, there is an array of 1750000 double precision number of 8
> > > > >> bytes =
> > > > >> >> 14000000 bytes (14Mo).
> > > > >> >>
> > > > >> >> Same for vectors: 2*8*250000 = 4 Mo
> > > > >> >>
> > > > >> >> But in the log summary, I've:
> > > > >> >>
> > > > >> >> Memory usage is given in bytes:
> > > > >> >>
> > > > >> >> Object Type          Creations   Destructions   Memory
> > > > >> Descendants' Mem.
> > > > >> >>
> > > > >> >> --- Event Stage 0: Main Stage
> > > > >> >>
> > > > >> >>               Viewer     1              1          0     0
> > > > >> >>            Index Set     2              2        672     0
> > > > >> >>                  Map    16             16       4992     0
> > > > >> >>                  Vec    10             10    2006576     0
> > > > >> >>          Vec Scatter     1              1          0     0
> > > > >> >>               Matrix     3              3       1068     0
> > > > >> >>        Krylov Solver     1              1      17208     0
> > > > >> >>       Preconditioner     1              1          0     0
> > > > >> >> =============================================================
> > > > >> >>
> > > > >> >> I don't understand this output...If you have some documentation
> > > > >> about how
> > > > >> >> Petsc deals with memory or if you can explain, I would be very
> > > > >> interested.
> > > > >> >>
> > > > >> >> This is a problem because I'm working on IBM SP4, and I've
> > > > >> limitations of
> > > > >> >> memory (700mb for data and 700mb for stack). I think the objects
> > > > >> created by
> > > > >> >> Petsc is created in the stack memory, but as I cannot determine
> > > > >> exactly the
> > > > >> >> memory usage, I cannot verify.
> > > > >> >>
> > > > >> >> Thank you for your advices,
> > > > >> >>
> > > > >> >> Best regards
> > > > >> >>
> > > > >> >>
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >
> > > > >
> > > > 
> > > > 
> > > 
> > > 
> > > 
> 
>

More information about the petsc-users mailing list