[petsc-dev] Lower memory DMDA

[Barry Smith]

  Lisandro,

     Thanks. But we can we get this in master since all changes are made from there and show enough lines of the stack to see the line number of the DMDA code that calls the eventual culprit so we don't need to hunt for them manually?

   Barry

On May 8, 2013, at 3:51 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:

> On 8 May 2013 16:30, Barry Smith <bsmith at mcs.anl.gov> wrote:
>> 
>> On May 8, 2013, at 7:54 AM, Peter Brune <prbrune at gmail.com> wrote:
>> 
>>> DMGetCoordinateDM is, however, collective.  Looking deeper, it is created only when used, but setting uniform coordinates is using it.  We might want to be more upfront about this.  It's probably not their problem.
>> 
>>    You guys are speculating too much. Create a DMDA characteristic for their code with reasonably large array dimensions and then -malloc_dump it (or something similar) to find out where the memory actually is. Then given this table of sizes propose changes, not before.
>> 
> 
> $ python test2-dalcinl.py  -malloc_dump | grep "\[ 0\]" | cut -f 2 -d
> "]" | sort -r -n
> 12000000 bytes VecScatterCreate() line 1095 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 12000000 bytes VecScatterCreate() line 1095 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 12000000 bytes VecScatterCreate() line 1095 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 12000000 bytes VecScatterCreate() line 1095 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 12000000 bytes ISLocalToGlobalMappingCreate() line 211 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 12000000 bytes DMSetUp_DA_3D() line 1290 in
> /home/devel/petsc/3.3/src/dm/impls/da/da3.c
> 4000000 bytes ISLocalToGlobalMappingBlock() line 254 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 3856 bytes DMCreate() line 41 in /home/devel/petsc/3.3/src/dm/interface/dm.c
> 816 bytes DMCreate_DA() line 235 in
> /home/devel/petsc/3.3/src/dm/impls/da/dacreate.c
> 624 bytes PetscSFCreate() line 73 in /home/devel/petsc/3.3/src/sys/sf/sf.c
> 624 bytes PetscSFCreate() line 73 in /home/devel/petsc/3.3/src/sys/sf/sf.c
> 560 bytes VecScatterCreate() line 863 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 560 bytes VecScatterCreate() line 863 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 496 bytes ISLocalToGlobalMappingCreate() line 203 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 496 bytes ISLocalToGlobalMappingCreate() line 203 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 256 bytes DMCreate() line 41 in /home/devel/petsc/3.3/src/dm/interface/dm.c
> 112 bytes DMSetUp_DA_3D() line 694 in
> /home/devel/petsc/3.3/src/dm/impls/da/da3.c
> 64 bytes VecScatterCreate() line 863 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 64 bytes VecScatterCreate() line 863 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 64 bytes PetscSFCreate() line 73 in /home/devel/petsc/3.3/src/sys/sf/sf.c
> 64 bytes PetscSFCreate() line 73 in /home/devel/petsc/3.3/src/sys/sf/sf.c
> 64 bytes ISLocalToGlobalMappingCreate() line 203 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 64 bytes ISLocalToGlobalMappingCreate() line 203 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 64 bytes DMCreate() line 41 in /home/devel/petsc/3.3/src/dm/interface/dm.c
> 48 bytes VecScatterCreate() line 1094 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 48 bytes VecScatterCreate() line 1094 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 48 bytes VecScatterCreate() line 1094 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 48 bytes VecScatterCreate() line 1094 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 32 bytes DMSetUp_DA_3D() line 216 in /home/devel/petsc/3.3/src/dm/impls/da/da3.c
> 16 bytes VecScatterCreate() line 863 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 16 bytes VecScatterCreate() line 863 in
> /home/devel/petsc/3.3/src/vec/vec/utils/vscat.c
> 16 bytes PetscStrallocpy() line 186 in /home/devel/petsc/3.3/src/sys/utils/str.c
> 16 bytes PetscStrallocpy() line 186 in /home/devel/petsc/3.3/src/sys/utils/str.c
> 16 bytes PetscSFCreate() line 73 in /home/devel/petsc/3.3/src/sys/sf/sf.c
> 16 bytes PetscSFCreate() line 73 in /home/devel/petsc/3.3/src/sys/sf/sf.c
> 16 bytes PetscCommDuplicate() line 156 in
> /home/devel/petsc/3.3/src/sys/objects/tagm.c
> 16 bytes PetscCommDuplicate() line 156 in
> /home/devel/petsc/3.3/src/sys/objects/tagm.c
> 16 bytes ISLocalToGlobalMappingCreate() line 203 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 16 bytes ISLocalToGlobalMappingCreate() line 203 in
> /home/devel/petsc/3.3/src/vec/is/utils/isltog.c
> 16 bytes DMSetUp_DA_3D() line 329 in /home/devel/petsc/3.3/src/dm/impls/da/da3.c
> 16 bytes DMSetUp_DA_3D() line 316 in /home/devel/petsc/3.3/src/dm/impls/da/da3.c
> 16 bytes DMSetUp_DA_3D() line 304 in /home/devel/petsc/3.3/src/dm/impls/da/da3.c
> 
> 
> 
> --
> Lisandro Dalcin
> ---------------
> CIMEC (INTEC/CONICET-UNL)
> Predio CONICET-Santa Fe
> Colectora RN 168 Km 472, Paraje El Pozo
> 3000 Santa Fe, Argentina
> Tel: +54-342-4511594 (ext 1011)
> Tel/Fax: +54-342-4511169