[petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects

[Barry Smith]

> On Nov 19, 2014, at 2:36 PM, Abhyankar, Shrirang G. <abhyshr at mcs.anl.gov> wrote:
> 
> The DM keeps track of the local vectors created by DMGetLocalVector(). So, if your routine gets called a million times with exactly one DMGetLocalVector() followed by one call to DMRestoreLocalVector(), then only one vector is created.

   In other words the DMGet/RestoreXXVector() are designed EXACTLY to be used when you need work vectors so you do not need to manage keeping the work vectors around yourself. In other words you are using it exactly right.

   Barry

> 
> Shri
> 
> From: Miguel Angel Salazar de Troya <salazardetroya at gmail.com>
> Date: Wed, 19 Nov 2014 14:02:10 -0600
> To: barry smith <bsmith at mcs.anl.gov>
> Cc: "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
> Subject: Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects
> 
> You were right. I was forgetting a DMRestoreLocalVector(). Thanks. If we are creating local vectors every time we call functions such as the RHSFunction of a TS implementation, which is called many times in the TS integration, will this be a problem in terms of performance? I've noticed that it might be. This is the way it is implemented in some TS. I wanted to double check with you guys before I figure out a way to create just one local vector that is re used in functions such as RHSFunction.
> 
> Miguel
> 
> On Tue, Nov 18, 2014 at 12:54 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> > On Nov 18, 2014, at 11:54 AM, Miguel Angel Salazar de Troya <salazardetroya at gmail.com> wrote:
> >
> > PetscFinalize() is inside of the class destructor (the last line of the destructor), so when the object goes out of scope, the class destructor is called and PetscFinalize() as well. Is it better to have PetscFinalize() outside of the destructor and call the destructor explicitly before?
> 
>    It shouldn't really matter.
> 
>     My guess is you must be missing calling a destroy or restore on one of the PETSc objects.
> 
>   Barry
> 
> >
> > Miguel
> >
> > On Tue, Nov 18, 2014 at 11:32 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> >
> > > On Nov 18, 2014, at 11:19 AM, Miguel Angel Salazar de Troya <salazardetroya at gmail.com> wrote:
> > >
> > > Hi
> > >
> > > I'm implementing a problem using the TS. Most of my functions are methods inside of a class, except for the callbacks (to form the RHS and the TS monitor), which are outside of the class, although in the same .C file where the class methods are implemented. For these callbacks I followed the network example:
> > >
> > > https://bitbucket.org/petsc/petsc/src/a614f7369d93d476173b8fc6bf2463276dcbdb3a/src/snes/examples/tutorials/network/pflow/pf.c?at=master
> > >
> > > Therefore, the callbacks have the PetscFunctionBegin at the beginning and PetscFunctionReturn(0) at the end. My problems come when I run the program with -malloc_dump and I get a lot of unfreed memory. Inspecting the output I see that the line of my code where the memory is allocated corresponds with the line when PetscFunctionBegin is called.
> >
> >   This is normal. We cannot register the exact line the memory allocated, only the location of the PETScFunctionBegin;
> >
> >
> > > Later in the file, I see that the function DMGetLocalVector() is called within a petsc internal routine (at the file dmget.c). I also call this routine in my callback methods few lines after PetscFunctionBegin. The procedure that I follow to use the local vectors is as the one in the network example. For vectors that I want to modify this is:
> > >
> > >  ierr = DMGetLocalVector(networkdm,&localX);CHKERRQ(ierr);
> > >  ierr = DMGlobalToLocalBegin(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr);
> > >  ierr = DMGlobalToLocalEnd(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr);
> > >  ierr = VecGetArray(localX,&xarr);CHKERRQ(ierr);
> > >
> > > Modify values in xarr
> > >
> > >  ierr = VecRestoreArray(localX,&xarr);CHKERRQ(ierr);
> > >  ierr = DMLocalToGlobalBegin(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr);
> > >  ierr = DMLocalToGlobalEnd(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr);
> > >  ierr = DMRestoreLocalVector(networkdm,&localX);CHKERRQ(ierr);
> > >
> > > One last thing that I think it might be a issue here is how I destroy the petsc objects. I create the petsc objects within a class. For instance, the class has a petsc vector that later passes to the TS object to get the solution. To destroy the petsc objects, I use the class destructor, where at the end I call PetscFinalize() Inside the class I pass the callbacks to the TS routines that need them (e.g. TSSetRHSFunction() ) I can compile the code and run it, but many memory allocations are not freed. What can be the issue here? Do you know of an example using C++ classes to implement PETSc methods? Thanks in advance.
> >
> >    Do you call the class destructor yourself explicitly before the PetscFinalize()? You need to, otherwise the class may not be destroyed until after PetscFinalize() and hence the PETSc objects won't be freed when you call PetscFinalize().
> >
> >    You also need to make sure that you destroy ALL PETSc objects, if you miss even one PETSc object, since the objects have references to each other it may be that many PETSc objects do not get freed and hence -malloc_dump shows many objects still alive.
> >
> >   Barry
> >
> > >
> > > Miguel
> > >
> > > --
> > > Miguel Angel Salazar de Troya
> > > Graduate Research Assistant
> > > Department of Mechanical Science and Engineering
> > > University of Illinois at Urbana-Champaign
> > > (217) 550-2360
> > > salaza11 at illinois.edu
> > >
> >
> >
> >
> >
> > --
> > Miguel Angel Salazar de Troya
> > Graduate Research Assistant
> > Department of Mechanical Science and Engineering
> > University of Illinois at Urbana-Champaign
> > (217) 550-2360
> > salaza11 at illinois.edu
> >
> 
> 
> 
> 
> -- 
> Miguel Angel Salazar de Troya
> Graduate Research Assistant
> Department of Mechanical Science and Engineering
> University of Illinois at Urbana-Champaign
> (217) 550-2360
> salaza11 at illinois.edu
>