[petsc-dev] threadcomm memory leak

Shri abhyshr at mcs.anl.gov
Sun Jul 15 17:26:05 CDT 2012 

On Jul 15, 2012, at 4:57 PM, Barry Smith wrote:

> 
> On Jul 15, 2012, at 4:48 PM, Shri wrote:
> 
>> 
>> On Jul 15, 2012, at 3:27 PM, Barry Smith wrote:
>> 
>>> 
>>> Shri,
>>> 
>>>  You need to test your code WITHOUT pthreads OR openmp configure.
>> That is what i am doing, no openmp or pthreads.
>> 
>>> You have the call to PetscThreadCommFinalize()  protected by PTHREADCLASSES
>> 
>> There is no PetscThreadCommFinalize(). There is a 'PetscThreadsFinalize()' call in PetscFinalize() that is protected by PTHREADCLASSES. PetscThreadsFinalize() is for finalizing the previous pthread pool model that i had added on which vec_type pthread and mat_type seqaijpthread are based. It has nothing to do with threadcomm. Once i add new vector and matrix code using threadcomm , i'll remove the old pthread code and the pthread vector and matrix class.
> 
>   Why not just remove all the old pthread code now; it will make things clearer.

Don't want to hurl people who are using/testing it into darkness.

> 
>>> but not PetscThreadCommInitializePackage() projected!   Also why do you call PetscThreadCommPackageInitialize() at the beginning but PetscThreadCommFinalize() at the end and not PetscThreadCommFinalizePackage()?
>> 
>> PetscThreadCommFinalizePackage is registerfinalized in PetscThreadCommInitializePackage() 
>> 
>> ierr = PetscRegisterFinalize(PetscThreadCommFinalizePackage);CHKERRQ(ierr);
> 
>   Sorry my mistake.
>> 
>> So it is called on PetscRegisterFinalizeAll().
>>> 
>>> BTW: If I #if def out the PetscThreadCommPackageInitialize() then nothing works (I have built without pthread classes and without openmp).  So there is some bad stuff going on.
>> 
>> PetscThreadCommPackageInitialize() attaches the threadcomm attribute on the MPI communicator and there are several places where the threadcomm attribute is requested (sys/objects/subcomm.c, src/sys/tagm.c). Hence, the errors. 
> 
>   Try a build with mpiuni.

    Aah, i can see the memory leak now. Thanks. It looks like the threadcomm object is not being freed. Something to do with destroying the MPIUNI attribute perhaps? Will check.

Shri
> Maybe the memory leak is because that isn't handling stuff possible.   My build with memory leaks is with mpiuni.
> 
>   Thanks
> 
> 
>   Barry
> 
>> 
>> Thanks,
>> Shri
>>> 
>>> Barry
>>> 
>>> On Jul 15, 2012, at 11:54 AM, Shri wrote:
>>> 
>>>> 
>>>> On Jul 14, 2012, at 7:55 PM, Satish Balay wrote:
>>>> 
>>>>> I see this with 32bit linux build - but not 64bit linux build.
>>>> 
>>>> I tried a fresh clone on my Mac and a linux cluster (both x86_64) but still did not get  any memory leaks. Perhaps this is a 32 bit build issue as Satish mentions. What configure options do i need to have a 32 bit build on x86_64? CC="gcc -m32" FC="gfortran -m32"?
>>>> 
>>>> Thanks,
>>>> Shri
>>>> 
>>>>> 
>>>>> also 32bit linux build gives errors with -fp_trap in ddot(). Haven't checked why.
>>>>> 
>>>>> Satish
>>>>> 
>>>>> 
>>>>> On Sat, 14 Jul 2012, Barry Smith wrote:
>>>>> 
>>>>>> 
>>>>>> Many of them with make test
>>>>>> 
>>>>>> Running test examples to verify correct installation
>>>>>> Using PETSC_DIR=/Users/barrysmith/Src/petsc-dev and PETSC_ARCH=arch-float128-opt
>>>>>> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process
>>>>>> See http://www.mcs.anl.gov/petsc/documentation/faq.html
>>>>>> lid velocity = Qg, prandtl # = Qg, grashof # = Qg
>>>>>> Number of SNES iterations = 2
>>>>>> [0]Total space allocated 6880 bytes
>>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 260 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 259 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>> [ 0]48 bytes PetscThreadCommReductionCreate() line 258 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]272 bytes PetscThreadCommInitialize() line 692 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]16 bytes PetscThreadCommSetAffinities() line 331 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>> Completed test examples
>>>>>> barry-smiths-macbook-pro:petsc-dev barrysmith$ 
>>>>>> 
>>>>>> 
>>>>>> On Jul 14, 2012, at 6:36 PM, Shri wrote:
>>>>>> 
>>>>>>> Which example are you running? I don't see any memory leak for the examples i've tested (threadcomm examples sys/threadcomm/examples/tutorials/ and snes/examples/tutorials/ex19.c,ex5.c)
>>>>>>> On Jul 14, 2012, at 4:22 PM, Barry Smith wrote:
>>>>>>> 
>>>>>>>> 
>>>>>>>> Please always test for memory leaks before pushing
>>>>>>>> 
>>>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>

More information about the petsc-dev mailing list