[petsc-dev] threadcomm memory leak

Barry Smith bsmith at mcs.anl.gov
Sun Jul 15 16:57:08 CDT 2012 

On Jul 15, 2012, at 4:48 PM, Shri wrote:

> 
> On Jul 15, 2012, at 3:27 PM, Barry Smith wrote:
> 
>> 
>> Shri,
>> 
>>   You need to test your code WITHOUT pthreads OR openmp configure.
> That is what i am doing, no openmp or pthreads.
> 
>> You have the call to PetscThreadCommFinalize()  protected by PTHREADCLASSES
> 
> There is no PetscThreadCommFinalize(). There is a 'PetscThreadsFinalize()' call in PetscFinalize() that is protected by PTHREADCLASSES. PetscThreadsFinalize() is for finalizing the previous pthread pool model that i had added on which vec_type pthread and mat_type seqaijpthread are based. It has nothing to do with threadcomm. Once i add new vector and matrix code using threadcomm , i'll remove the old pthread code and the pthread vector and matrix class.

   Why not just remove all the old pthread code now; it will make things clearer.

>> but not PetscThreadCommInitializePackage() projected!   Also why do you call PetscThreadCommPackageInitialize() at the beginning but PetscThreadCommFinalize() at the end and not PetscThreadCommFinalizePackage()?
> 
> PetscThreadCommFinalizePackage is registerfinalized in PetscThreadCommInitializePackage() 
> 
> ierr = PetscRegisterFinalize(PetscThreadCommFinalizePackage);CHKERRQ(ierr);

   Sorry my mistake.
> 
> So it is called on PetscRegisterFinalizeAll().
>> 
>>  BTW: If I #if def out the PetscThreadCommPackageInitialize() then nothing works (I have built without pthread classes and without openmp).  So there is some bad stuff going on.
> 
> PetscThreadCommPackageInitialize() attaches the threadcomm attribute on the MPI communicator and there are several places where the threadcomm attribute is requested (sys/objects/subcomm.c, src/sys/tagm.c). Hence, the errors. 

   Try a build with mpiuni. Maybe the memory leak is because that isn't handling stuff possible.   My build with memory leaks is with mpiuni.

   Thanks


   Barry

> 
> Thanks,
> Shri
>> 
>>  Barry
>> 
>> On Jul 15, 2012, at 11:54 AM, Shri wrote:
>> 
>>> 
>>> On Jul 14, 2012, at 7:55 PM, Satish Balay wrote:
>>> 
>>>> I see this with 32bit linux build - but not 64bit linux build.
>>> 
>>> I tried a fresh clone on my Mac and a linux cluster (both x86_64) but still did not get  any memory leaks. Perhaps this is a 32 bit build issue as Satish mentions. What configure options do i need to have a 32 bit build on x86_64? CC="gcc -m32" FC="gfortran -m32"?
>>> 
>>> Thanks,
>>> Shri
>>> 
>>>> 
>>>> also 32bit linux build gives errors with -fp_trap in ddot(). Haven't checked why.
>>>> 
>>>> Satish
>>>> 
>>>> 
>>>> On Sat, 14 Jul 2012, Barry Smith wrote:
>>>> 
>>>>> 
>>>>> Many of them with make test
>>>>> 
>>>>> Running test examples to verify correct installation
>>>>> Using PETSC_DIR=/Users/barrysmith/Src/petsc-dev and PETSC_ARCH=arch-float128-opt
>>>>> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process
>>>>> See http://www.mcs.anl.gov/petsc/documentation/faq.html
>>>>> lid velocity = Qg, prandtl # = Qg, grashof # = Qg
>>>>> Number of SNES iterations = 2
>>>>> [0]Total space allocated 6880 bytes
>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 260 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>> [ 0]16 bytes PetscThreadCommReductionCreate() line 259 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>> [ 0]48 bytes PetscThreadCommReductionCreate() line 258 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcommred.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommInitialize() line 698 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]176 bytes PetscThreadCommInitialize() line 694 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]272 bytes PetscThreadCommInitialize() line 692 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]16 bytes PetscThreadCommSetAffinities() line 331 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>> Completed test examples
>>>>> barry-smiths-macbook-pro:petsc-dev barrysmith$ 
>>>>> 
>>>>> 
>>>>> On Jul 14, 2012, at 6:36 PM, Shri wrote:
>>>>> 
>>>>>> Which example are you running? I don't see any memory leak for the examples i've tested (threadcomm examples sys/threadcomm/examples/tutorials/ and snes/examples/tutorials/ex19.c,ex5.c)
>>>>>> On Jul 14, 2012, at 4:22 PM, Barry Smith wrote:
>>>>>> 
>>>>>>> 
>>>>>>> Please always test for memory leaks before pushing
>>>>>>> 
>>>>>>> [ 0]48 bytes PetscThreadCommCreate() line 122 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>> [ 0]320 bytes PetscThreadCommCreate() line 118 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [0]  PetscThreadCommInitialize() line 685 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/threadcomm.c
>>>>>>> [0]  PetscThreadCommInitializePackage() line 53 in /Users/barrysmith/Src/petsc-dev/src/sys/threadcomm/interface/dlregisthreadcomm.c
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>

More information about the petsc-dev mailing list