[petsc-dev] threadcomm memory leak

Shri abhyshr at mcs.anl.gov
Mon Jul 16 09:44:56 CDT 2012 

Hong, 
  You need to do similar for your elemental grid to avoid memory leaks when using mpiuni.
http://petsc.cs.iit.edu/petsc/petsc-dev/rev/169c19e9ea54

Shri

On Jul 16, 2012, at 8:08 AM, Shri wrote:

> 
> On Jul 15, 2012, at 5:26 PM, Shri wrote:
> 
>> 
>> On Jul 15, 2012, at 4:57 PM, Barry Smith wrote:
>> 
>>> 
>>> On Jul 15, 2012, at 4:48 PM, Shri wrote:
>>> 
>>>> 
>>>> On Jul 15, 2012, at 3:27 PM, Barry Smith wrote:
>>>> 
>>>>> 
>>>>> Shri,
>>>>> 
>>>>> You need to test your code WITHOUT pthreads OR openmp configure.
>>>> That is what i am doing, no openmp or pthreads.
>>>> 
>>>>> You have the call to PetscThreadCommFinalize()  protected by PTHREADCLASSES
>>>> 
>>>> There is no PetscThreadCommFinalize(). There is a 'PetscThreadsFinalize()' call in PetscFinalize() that is protected by PTHREADCLASSES. PetscThreadsFinalize() is for finalizing the previous pthread pool model that i had added on which vec_type pthread and mat_type seqaijpthread are based. It has nothing to do with threadcomm. Once i add new vector and matrix code using threadcomm , i'll remove the old pthread code and the pthread vector and matrix class.
>>> 
>>> Why not just remove all the old pthread code now; it will make things clearer.
>> 
>> Don't want to hurl people who are using/testing it into darkness.
>> 
>>> 
>>>>> but not PetscThreadCommInitializePackage() projected!   Also why do you call PetscThreadCommPackageInitialize() at the beginning but PetscThreadCommFinalize() at the end and not PetscThreadCommFinalizePackage()?
>>>> 
>>>> PetscThreadCommFinalizePackage is registerfinalized in PetscThreadCommInitializePackage() 
>>>> 
>>>> ierr = PetscRegisterFinalize(PetscThreadCommFinalizePackage);CHKERRQ(ierr);
>>> 
>>> Sorry my mistake.
>>>> 
>>>> So it is called on PetscRegisterFinalizeAll().
>>>>> 
>>>>> BTW: If I #if def out the PetscThreadCommPackageInitialize() then nothing works (I have built without pthread classes and without openmp).  So there is some bad stuff going on.
>>>> 
>>>> PetscThreadCommPackageInitialize() attaches the threadcomm attribute on the MPI communicator and there are several places where the threadcomm attribute is requested (sys/objects/subcomm.c, src/sys/tagm.c). Hence, the errors. 
>>> 
>>> Try a build with mpiuni.
>> 
>>   Aah, i can see the memory leak now. Thanks. It looks like the threadcomm object is not being freed. Something to do with destroying the MPIUNI attribute perhaps? Will check.
> 
> Fixed. http://petsc.cs.iit.edu/petsc/petsc-dev/rev/169c19e9ea54
> The memory leak was due to the COMM_SELF and COMM_WORLD communicators being same for mpiuni. This was leaving an extra reference for the innercomm.
> 
> Shri
> 
>> 
>> Shri
>>> Maybe the memory leak is because that isn't handling stuff possible.   My build with memory leaks is with mpiuni.
>>> 
>>> Thanks
>>> 
>>> 
>>> Barry
>>> 
>>>> 
>>>> Thanks,
>>>> Shri
>>>>> 
>>>>> Barry
>>>>> 
>>>>> On Jul 15, 2012, at 11:54 AM, Shri wrote:
>>>>> 
>>>>>> 
>>>>>> On Jul 14, 2012, at 7:55 PM, Satish Balay wrote:
>>>>>> 
>>>>>>> I see this with 32bit linux build - but not 64bit linux build.
>>>>>> 
>>>>>> I tried a fresh clone on my Mac and a linux cluster (both x86_64) but still did not get  any memory leaks. Perhaps this is a 32 bit build issue as Satish mentions. What configure options do i need to have a 32 bit build on x86_64? CC="gcc -m32" FC="gfortran -m32"?
>>>>>> 
>>>>>> Thanks,
>>>>>> Shri
>>>>>> 
>>>>>>> 
>>>>>>> also 32bit linux build gives errors with -fp_trap in ddot(). Haven't checked why.
>>>>>>> 
>>>>>>> Satish
>>>>>>> 
>>>>>>> 
>>>>>>> On Sat, 14 Jul 2012, Barry Smith wrote:
>>>>>>> 
>>>>>>>> 
>>>>>>>> Many of them with make test
>>>>>>>> 
>>>>>>>> Running test examples to verify correct installation
>>>>>>>> Using PETSC_DIR=/Users/barrysmith/Src/petsc-dev and PETSC_ARCH=arch-float128-opt
>>>>>>>> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process
>>>>>>>> See http://www.mcs.anl.gov/petsc/documentation/faq.html
>>>>>>>> lid velocity = Qg, prandtl # = Qg, grashof # = Qg
>>>>>>>> Number of SNES iterations = 2
>>>>>>>> [0]Total space allocated 6880 bytes
>>>>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 260 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 259 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>>>> [ 0]48 bytes PetscThreadCommReductionCreate() line 258 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]272 bytes PetscThreadCommInitialize() line 692 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]16 bytes PetscThreadCommSetAffinities() line 331 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> Completed test examples
>>>>>>>> barry-smiths-macbook-pro:petsc-dev barrysmith$ 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Jul 14, 2012, at 6:36 PM, Shri wrote:
>>>>>>>> 
>>>>>>>>> Which example are you running? I don't see any memory leak for the examples i've tested (threadcomm examples sys/threadcomm/examples/tutorials/ and snes/examples/tutorials/ex19.c,ex5.c)
>>>>>>>>> On Jul 14, 2012, at 4:22 PM, Barry Smith wrote:
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Please always test for memory leaks before pushing
>>>>>>>>>> 
>>>>>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>

More information about the petsc-dev mailing list