[petsc-dev] threadcomm memory leak

Shri abhyshr at mcs.anl.gov
Mon Jul 16 08:08:07 CDT 2012 

On Jul 15, 2012, at 5:26 PM, Shri wrote:

> 
> On Jul 15, 2012, at 4:57 PM, Barry Smith wrote:
> 
>> 
>> On Jul 15, 2012, at 4:48 PM, Shri wrote:
>> 
>>> 
>>> On Jul 15, 2012, at 3:27 PM, Barry Smith wrote:
>>> 
>>>> 
>>>> Shri,
>>>> 
>>>> You need to test your code WITHOUT pthreads OR openmp configure.
>>> That is what i am doing, no openmp or pthreads.
>>> 
>>>> You have the call to PetscThreadCommFinalize()  protected by PTHREADCLASSES
>>> 
>>> There is no PetscThreadCommFinalize(). There is a 'PetscThreadsFinalize()' call in PetscFinalize() that is protected by PTHREADCLASSES. PetscThreadsFinalize() is for finalizing the previous pthread pool model that i had added on which vec_type pthread and mat_type seqaijpthread are based. It has nothing to do with threadcomm. Once i add new vector and matrix code using threadcomm , i'll remove the old pthread code and the pthread vector and matrix class.
>> 
>>  Why not just remove all the old pthread code now; it will make things clearer.
> 
> Don't want to hurl people who are using/testing it into darkness.
> 
>> 
>>>> but not PetscThreadCommInitializePackage() projected!   Also why do you call PetscThreadCommPackageInitialize() at the beginning but PetscThreadCommFinalize() at the end and not PetscThreadCommFinalizePackage()?
>>> 
>>> PetscThreadCommFinalizePackage is registerfinalized in PetscThreadCommInitializePackage() 
>>> 
>>> ierr = PetscRegisterFinalize(PetscThreadCommFinalizePackage);CHKERRQ(ierr);
>> 
>>  Sorry my mistake.
>>> 
>>> So it is called on PetscRegisterFinalizeAll().
>>>> 
>>>> BTW: If I #if def out the PetscThreadCommPackageInitialize() then nothing works (I have built without pthread classes and without openmp).  So there is some bad stuff going on.
>>> 
>>> PetscThreadCommPackageInitialize() attaches the threadcomm attribute on the MPI communicator and there are several places where the threadcomm attribute is requested (sys/objects/subcomm.c, src/sys/tagm.c). Hence, the errors. 
>> 
>>  Try a build with mpiuni.
> 
>    Aah, i can see the memory leak now. Thanks. It looks like the threadcomm object is not being freed. Something to do with destroying the MPIUNI attribute perhaps? Will check.

Fixed. http://petsc.cs.iit.edu/petsc/petsc-dev/rev/169c19e9ea54
The memory leak was due to the COMM_SELF and COMM_WORLD communicators being same for mpiuni. This was leaving an extra reference for the innercomm.

Shri

> 
> Shri
>> Maybe the memory leak is because that isn't handling stuff possible.   My build with memory leaks is with mpiuni.
>> 
>>  Thanks
>> 
>> 
>>  Barry
>> 
>>> 
>>> Thanks,
>>> Shri
>>>> 
>>>> Barry
>>>> 
>>>> On Jul 15, 2012, at 11:54 AM, Shri wrote:
>>>> 
>>>>> 
>>>>> On Jul 14, 2012, at 7:55 PM, Satish Balay wrote:
>>>>> 
>>>>>> I see this with 32bit linux build - but not 64bit linux build.
>>>>> 
>>>>> I tried a fresh clone on my Mac and a linux cluster (both x86_64) but still did not get  any memory leaks. Perhaps this is a 32 bit build issue as Satish mentions. What configure options do i need to have a 32 bit build on x86_64? CC="gcc -m32" FC="gfortran -m32"?
>>>>> 
>>>>> Thanks,
>>>>> Shri
>>>>> 
>>>>>> 
>>>>>> also 32bit linux build gives errors with -fp_trap in ddot(). Haven't checked why.
>>>>>> 
>>>>>> Satish
>>>>>> 
>>>>>> 
>>>>>> On Sat, 14 Jul 2012, Barry Smith wrote:
>>>>>> 
>>>>>>> 
>>>>>>> Many of them with make test
>>>>>>> 
>>>>>>> Running test examples to verify correct installation
>>>>>>> Using PETSC_DIR=/Users/barrysmith/Src/petsc-dev and PETSC_ARCH=arch-float128-opt
>>>>>>> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process
>>>>>>> See http://www.mcs.anl.gov/petsc/documentation/faq.html
>>>>>>> lid velocity = Qg, prandtl # = Qg, grashof # = Qg
>>>>>>> Number of SNES iterations = 2
>>>>>>> [0]Total space allocated 6880 bytes
>>>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 260 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 259 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>>> [ 0]48 bytes PetscThreadCommReductionCreate() line 258 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]272 bytes PetscThreadCommInitialize() line 692 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]16 bytes PetscThreadCommSetAffinities() line 331 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> Completed test examples
>>>>>>> barry-smiths-macbook-pro:petsc-dev barrysmith$ 
>>>>>>> 
>>>>>>> 
>>>>>>> On Jul 14, 2012, at 6:36 PM, Shri wrote:
>>>>>>> 
>>>>>>>> Which example are you running? I don't see any memory leak for the examples i've tested (threadcomm examples sys/threadcomm/examples/tutorials/ and snes/examples/tutorials/ex19.c,ex5.c)
>>>>>>>> On Jul 14, 2012, at 4:22 PM, Barry Smith wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Please always test for memory leaks before pushing
>>>>>>>>> 
>>>>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>

More information about the petsc-dev mailing list