Question about memory used by PetsC Objects
Laurent Nguyen
laurent.nguyen at idris.fr
Fri Apr 13 21:16:41 CDT 2007
Yes, I know, this is a problem on all computer systems.But, I hoped that your profiling functions could do something like that (this is the name of functions which give me this hope)
The reason why I need that is because of my computer system (IBM SP4). I've limits of memory and if I use more memory than allowed, the system break my program. For example, in a C code without allocation (only static memory), I can calculate the maximum of memory used by my program. With PetsC, I don't know all the mechanism behind Petsc Objects. This why I sent you theses mails ^^
I would have thought this is a problem for users who make computations on this sort of system (with limits etc...), but it seems this is not the case.
Sorry for the disturbance and thank you for your help
Laurent
On Fri, Apr 13, 2007 at 11:14:34AM -0500, Barry Smith wrote:
>
> Matt,
>
> I do not understand why one would want this, doesn't PETSc already keep
> a current amount used (memory malloced but not freed) and the maximum current
> amount used?
>
> NGUYEN,
>
> Once PETSc frees memory (or any program) it is not returned to the operating
> system, rather it is reused in a future malloc if possible. PETSc cannot give
> the TOTAL memory used by the process since that includes pages of the program
> and any memory your code is using and there is no good Unix command to get that
> information on all computer systems.
>
> On Fri, 13 Apr 2007, Matthew Knepley wrote:
>
> > There is a way to get exactly what you want which is not really very hard.
> > Petsc allows you to easily override PetscMalloc() and PetscFree() which
> > we use for everything. You could just keep a running static tally of the sizes
> > and then call the regular version. This would only take a few lines of code.
> > Take a look at
> >
> > http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Sys/PetscSetMalloc.html
> >
> > Matt
> >
> > On 4/13/07, Laurent Nguyen <laurent.nguyen at idris.fr> wrote:
> > > Yes, I've tested the argument -memory_info. I've seen the memory given
> > > to me is the same memory given by a PetscMemoryGetCurrentUsage() before
> > > PetscFinalize(). I didn't know it was the same ^^.
> > >
> > > It is not the memory I need. I would like to have the real memory used
> > > by my program (It is a little difficult to explain clearly): for
> > > example, if I create 2 vectors of size 100, the program is consuming
> > > 2*8*100 bytes. But later, when I destroy only 1 vector, the programs is
> > > consuming 1*8*100 bytes. This is this kind of information that I need
> > > (more precisely, the max). I tell you that because, in my context of
> > > computation, I've only 1.5Go of memory per processors. And the memory
> > > given by the argument -memory_info was 2.4Go. So I was perplex about
> > > that. But now I understand how your function is running.
> > >
> > > But thanks to yours functions, I can now better manage the memory of my
> > > objects!
> > >
> > > Regards,
> > >
> > > **************************************
> > > NGUYEN Anh-Khai Laurent
> > > Equipe Support Utilisateur
> > >
> > > Email : laurent.nguyen at idris.fr
> > > T?l : 01.69.35.85.66
> > > Adresse : IDRIS - Institut du D?veloppement et des Ressources en
> > > Informatique Scientifique
> > > CNRS
> > > Batiment 506
> > > BP 167
> > > F - 91403 ORSAY Cedex
> > > Site Web : http://www.idris.fr
> > > **************************************
> > >
> > > Matthew Knepley a ?crit :
> > > > On 4/13/07, Laurent Nguyen <laurent.nguyen at idris.fr> wrote:
> > > >> Barry,
> > > >>
> > > >> I thank you for your help. So I tried to apply the functions. It seems
> > > >> that PetscMemoryGetCurrentUsage() gives the total amount of memory
> > > >> allocated at the point of the program where it is placed. But, if a
> > > >> object is deallocated, PetscMemoryGetCurrentUsage() doesn't take it.
> > > >>
> > > >> I would like to try PetscMemoryGetMaximumUsage() at the end of the
> > > >> program hoping it gives me the maximum memory used by my program but
> > > >> there are problems. When I put PetscMemoryGetMaximumUsage(), I've this
> > > >> error at the execution:
> > > >>
> > > >> [0]PETSC ERROR: --------------------- Error Message
> > > >> ------------------------------------
> > > >> [0]PETSC ERROR: Object is in wrong state!
> > > >> [0]PETSC ERROR: To use this function you must first call
> > > >> PetscMemorySetGetMaximumUsage()!
> > > >>
> > > >> So I put PetscMemorySetGetMaximumUsage() before, but at linking, it
> > > >> didn't find PetscMemorySetGetMaximumUsage in the petsc libraries. I
> > > >> thought I made a bad install but when I tried on another machine, I've
> > > >> the same problem.
> > > >
> > > > The function is definitely there (check src/sys/memory/mem.c). You can try
> > > > a short test:
> > > >
> > > > cd src/ksp/ksp/examples/tutorials
> > > > make ex2
> > > > ./ex2 -memory_info
> > > > ./ex2 -malloc_log
> > > >
> > > > Also, I have fixed the specific logging errors you pointed out in the dev
> > > > copy, but we are doing a code review to make sure no other omissions
> > > > are lurking.
> > > >
> > > > Matt
> > > >
> > > >> Best regards,
> > > >>
> > > >> **************************************
> > > >> NGUYEN Anh-Khai Laurent
> > > >> Equipe Support Utilisateur
> > > >>
> > > >> Email : laurent.nguyen at idris.fr
> > > >> T?l : 01.69.35.85.66
> > > >> Adresse : IDRIS - Institut du D?veloppement et des Ressources en
> > > >> Informatique Scientifique
> > > >> CNRS
> > > >> Batiment 506
> > > >> BP 167
> > > >> F - 91403 ORSAY Cedex
> > > >> Site Web : http://www.idris.fr
> > > >> **************************************
> > > >>
> > > >> Barry Smith a ?crit :
> > > >> > Laurent,
> > > >> >
> > > >> > The memory usage printed with -log_summary is not correct.
> > > >> > Much of the memory used is not printed out.
> > > >> >
> > > >> > You can use PetscMemoryGetCurrentUsage() and
> > > >> PetscMemoryGetMaximumUsage()
> > > >> > in your code before and after creating a filling in PETSc objects
> > > >> > to see how much memory they are used. You can also use
> > > >> PetscMallocDump()
> > > >> > at any point to see exactly where in PETSc and how much all of PETSc's
> > > >> > memory was allocated.
> > > >> >
> > > >> > Barry
> > > >> >
> > > >> >
> > > >> > On Thu, 12 Apr 2007, Laurent Nguyen wrote:
> > > >> >
> > > >> >> Hi,
> > > >> >>
> > > >> >> I'm a Petsc user, but I'm beginning to use petsc in a supercomputation
> > > >> >> context. So, I'm trying parallel versions of my old mono-processed
> > > >> programs.
> > > >> >> But I've some difficulties to determine how much memory my program
> > > >> use. (I
> > > >> >> give a little example). In this example on one processor (with
> > > >> these command
> > > >> >> line arguments: -mat_type mpiaij -pc_type none -log_summary), I
> > > >> created one
> > > >> >> mpiaij matrix (size = 500*500) and two vectors (size = 500). I've this
> > > >> >> information from output:
> > > >> >>
> > > >> >> Matrix Object:
> > > >> >> type=mpiaij, rows=250000, cols=250000
> > > >> >> total: nonzeros=749996, allocated nonzeros=1750000
> > > >> >>
> > > >> >> To me, there is an array of 1750000 double precision number of 8
> > > >> bytes =
> > > >> >> 14000000 bytes (14Mo).
> > > >> >>
> > > >> >> Same for vectors: 2*8*250000 = 4 Mo
> > > >> >>
> > > >> >> But in the log summary, I've:
> > > >> >>
> > > >> >> Memory usage is given in bytes:
> > > >> >>
> > > >> >> Object Type Creations Destructions Memory
> > > >> Descendants' Mem.
> > > >> >>
> > > >> >> --- Event Stage 0: Main Stage
> > > >> >>
> > > >> >> Viewer 1 1 0 0
> > > >> >> Index Set 2 2 672 0
> > > >> >> Map 16 16 4992 0
> > > >> >> Vec 10 10 2006576 0
> > > >> >> Vec Scatter 1 1 0 0
> > > >> >> Matrix 3 3 1068 0
> > > >> >> Krylov Solver 1 1 17208 0
> > > >> >> Preconditioner 1 1 0 0
> > > >> >> =============================================================
> > > >> >>
> > > >> >> I don't understand this output...If you have some documentation
> > > >> about how
> > > >> >> Petsc deals with memory or if you can explain, I would be very
> > > >> interested.
> > > >> >>
> > > >> >> This is a problem because I'm working on IBM SP4, and I've
> > > >> limitations of
> > > >> >> memory (700mb for data and 700mb for stack). I think the objects
> > > >> created by
> > > >> >> Petsc is created in the stack memory, but as I cannot determine
> > > >> exactly the
> > > >> >> memory usage, I cannot verify.
> > > >> >>
> > > >> >> Thank you for your advices,
> > > >> >>
> > > >> >> Best regards
> > > >> >>
> > > >> >>
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >
> > > >
> > >
> > >
> >
> >
> >
--
**************************************
NGUYEN Anh-Khai Laurent
Equipe Support Utilisateur
Email : laurent.nguyen at idris.fr
Tel : 01.69.35.85.66
Adresse : IDRIS - Institut du Développement et des Ressources en
Informatique Scientifique
CNRS
Batiment 506
BP 167
F - 91403 ORSAY Cedex
Site Web: http://www.idris.fr
**************************************
More information about the petsc-users
mailing list