[petsc-users] Can I eliminate all mallocs in parallel matrix assembly?

[Markus Berndt]

I am trying to minimize the memory footprint of an application by 
specifying exactly how many nonzeros are needed per row of the matrix. I 
am pretty sure that I am correctly calculating these memory requirements 
for the diagonal and off diagonal parts of the parallel matrix (I am 
using MatCreateAIJ).

When running the application with the -info option I see a fairly small 
number of mallocs:

[0] MatStashScatterBegin_Private(): No of messages: 2
[0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 13192
[0] MatStashScatterBegin_Private(): Mesg_to: 2: size: 14280
[0] MatAssemblyBegin_MPIAIJ(): Stash has 3432 entries, uses 0 mallocs.
[1] MatAssemblyBegin_MPIAIJ(): Stash has 2912 entries, uses 0 mallocs.
[2] MatAssemblyBegin_MPIAIJ(): Stash has 2616 entries, uses 0 mallocs.
[0] MatAssemblyEnd_SeqAIJ(): Matrix size: 692 X 692; storage space: 337 
unneeded,13896 used
[0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 25
[0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 27
[1] MatAssemblyEnd_SeqAIJ(): Matrix size: 718 X 718; storage space: 318 
unneeded,14830 used
[1] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 24
[1] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 27
[2] MatAssemblyEnd_SeqAIJ(): Matrix size: 787 X 787; storage space: 441 
unneeded,15763 used
[2] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 33
[2] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 27
[2] Mat_CheckInode(): Found 787 nodes out of 787 rows. Not using Inode 
routines
[0] Mat_CheckInode(): Found 692 nodes out of 692 rows. Not using Inode 
routines
[1] PetscCommDuplicate(): Using internal PETSc communicator 34964256 
64966848
[2] PetscCommDuplicate(): Using internal PETSc communicator 34964256 
66809232
[0] PetscCommDuplicate(): Using internal PETSc communicator 34964256 
53202336
[2] PetscCommDuplicate(): Using internal PETSc communicator 34964256 
66809232
[1] PetscCommDuplicate(): Using internal PETSc communicator 34964256 
64966848
[0] PetscCommDuplicate(): Using internal PETSc communicator 34964256 
53202336
[0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
[0] VecScatterCreate(): General case: MPI to Seq
[0] MatAssemblyEnd_SeqAIJ(): Matrix size: 692 X 363; storage space: 370 
unneeded,2091 used
[0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0
[0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 21
[1] MatAssemblyEnd_SeqAIJ(): Matrix size: 718 X 334; storage space: 344 
unneeded,1980 used
[1] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0
[1] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 21
[2] MatAssemblyEnd_SeqAIJ(): Matrix size: 787 X 301; storage space: 362 
unneeded,2093 used
[2] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0
[2] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 20

This example runs on three PEs.

I am wondering if I should be able to eliminate all mallocs? Instead 
should my goal be to eliminate not all but most mallocs with a small 
number of mallocs during matrix assembly being acceptable?

Thanks

- Markus


-- 
Markus Berndt - CCS-2 - LANL - 505-665-4711