[petsc-users] [petsc-maint] Memory usage function: output for all ranks

[Matthew Knepley]

On Mon, Nov 30, 2015 at 5:20 PM, Andrey Ovsyannikov <aovsyannikov at lbl.gov>
wrote:

> Dear PETSc team,
>
> I am working on optimization of Chombo-Crunch CFD code for next-generation
> supercomputer architectures at NERSC (Berkeley Lab) and we use PETSc AMG
> solver. During memory analysis study I faced with a difficulty to get
> memory usage data from PETSc for all MPI ranks. I am looking for memory
> dump function to get a detailed information on memory usage (not only
> resident size and virtual memory but allso allocation by Vec, Mat, etc).
> There is PetscMallocDumpLog() function but it is a collective function and
> it always provides a log for 0 rank. I am wondering if it is possible to
> include in PETSc a modification of PetscMallocDumpLog() which dumps the
> similar log but for all MPI ranks.
>
> I am attaching an example of my own memory function which uses PETSc
> non-collective functions and it provides a resident set size and virtual
> memory for all ranks. Perhaps in a similar way it is possible to modify
> PetscMallocDumpLog.
>

You could walk the heap if you use the debugging malloc infrastructure in
PETSc. However, I would really recommend
trying out Massif from the valgrind toolset. Its designed for this and
really nice.

  Thanks,

    Matt


> Thank you,
>
> void petscMemoryLog(const char prefix[])
> {
>   FILE* fd;
>   char fname[PETSC_MAX_PATH_LEN];
>   PetscMPIInt rank;
>
>   MPI_Comm_rank(Chombo_MPI::comm,&rank);
>
>   PetscLogDouble allocated;
>   PetscLogDouble resident;
>   PetscMallocGetCurrentUsage(&allocated);
>   PetscMemoryGetCurrentUsage(&resident);
>   PetscSNPrintf(fname,sizeof(fname),"%s.%d",prefix,rank);
>   PetscFOpen(PETSC_COMM_SELF,fname,"a",&fd);
>
>   PetscFPrintf(PETSC_COMM_SELF,fd,"### PETSc memory footprint for rank %d
> \n",rank);
>   PetscFPrintf(PETSC_COMM_SELF,fd,"[%d] Memory allocated by PetscMalloc()
> %.0f bytes\n",rank,allocated);
>   PetscFPrintf(PETSC_COMM_SELF,fd,"[%d] RSS usage by entire process %.0f
> KB\n",rank,resident);
>   PetscFClose(PETSC_COMM_SELF,fd);
> }
>
> Best regards,
> Andrey Ovsyannikov, Ph.D.
> Postdoctoral Fellow
> NERSC Division
> Lawrence Berkeley National Laboratory
> 510-486-7880
> aovsyannikov at lbl.gov
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20151130/a7f379e3/attachment-0001.html>