[petsc-dev] threadcomm memory leak

Sun Jul 15 16:48:53 CDT 2012

On Jul 15, 2012, at 3:27 PM, Barry Smith wrote:

> 
>  Shri,
> 
>    You need to test your code WITHOUT pthreads OR openmp configure.
That is what i am doing, no openmp or pthreads.

> You have the call to PetscThreadCommFinalize()  protected by PTHREADCLASSES

There is no PetscThreadCommFinalize(). There is a 'PetscThreadsFinalize()' call in PetscFinalize() that is protected by PTHREADCLASSES. PetscThreadsFinalize() is for finalizing the previous pthread pool model that i had added on which vec_type pthread and mat_type seqaijpthread are based. It has nothing to do with threadcomm. Once i add new vector and matrix code using threadcomm , i'll remove the old pthread code and the pthread vector and matrix class.
> but not PetscThreadCommInitializePackage() projected!   Also why do you call PetscThreadCommPackageInitialize() at the beginning but PetscThreadCommFinalize() at the end and not PetscThreadCommFinalizePackage()?

PetscThreadCommFinalizePackage is registerfinalized in PetscThreadCommInitializePackage() 

ierr = PetscRegisterFinalize(PetscThreadCommFinalizePackage);CHKERRQ(ierr);

So it is called on PetscRegisterFinalizeAll().
> 
>   BTW: If I #if def out the PetscThreadCommPackageInitialize() then nothing works (I have built without pthread classes and without openmp).  So there is some bad stuff going on.

PetscThreadCommPackageInitialize() attaches the threadcomm attribute on the MPI communicator and there are several places where the threadcomm attribute is requested (sys/objects/subcomm.c, src/sys/tagm.c). Hence, the errors. 

Thanks,
Shri
> 
>   Barry
> 
> On Jul 15, 2012, at 11:54 AM, Shri wrote:
> 
>> 
>> On Jul 14, 2012, at 7:55 PM, Satish Balay wrote:
>> 
>>> I see this with 32bit linux build - but not 64bit linux build.
>> 
>> I tried a fresh clone on my Mac and a linux cluster (both x86_64) but still did not get  any memory leaks. Perhaps this is a 32 bit build issue as Satish mentions. What configure options do i need to have a 32 bit build on x86_64? CC="gcc -m32" FC="gfortran -m32"?
>> 
>> Thanks,
>> Shri
>> 
>>> 
>>> also 32bit linux build gives errors with -fp_trap in ddot(). Haven't checked why.
>>> 
>>> Satish
>>> 
>>> 
>>> On Sat, 14 Jul 2012, Barry Smith wrote:
>>> 
>>>> 
>>>> Many of them with make test
>>>> 
>>>> Running test examples to verify correct installation
>>>> Using PETSC_DIR=/Users/barrysmith/Src/petsc-dev and PETSC_ARCH=arch-float128-opt
>>>> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process
>>>> See http://www.mcs.anl.gov/petsc/documentation/faq.html
>>>> lid velocity = Qg, prandtl # = Qg, grashof # = Qg
>>>> Number of SNES iterations = 2
>>>> [0]Total space allocated 6880 bytes
>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 260 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 259 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>> [ 0]48 bytes PetscThreadCommReductionCreate() line 258 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]272 bytes PetscThreadCommInitialize() line 692 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]16 bytes PetscThreadCommSetAffinities() line 331 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>> Completed test examples
>>>> barry-smiths-macbook-pro:petsc-dev barrysmith$ 
>>>> 
>>>> 
>>>> On Jul 14, 2012, at 6:36 PM, Shri wrote:
>>>> 
>>>>> Which example are you running? I don't see any memory leak for the examples i've tested (threadcomm examples sys/threadcomm/examples/tutorials/ and snes/examples/tutorials/ex19.c,ex5.c)
>>>>> On Jul 14, 2012, at 4:22 PM, Barry Smith wrote:
>>>>> 
>>>>>> 
>>>>>> Please always test for memory leaks before pushing
>>>>>> 
>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>  [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>  [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>  [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>  [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 
> 