[petsc-users] About Preallocation for MPI matrix

[E. Ekici]

Hi,

I would like to preallocate my highly sparse matrices to get speed up during assembling.

Up to 70000x70000, it works fine in parallel.

But when I exceed around 70000 rows/columns, I receive following error at preallocation line;

petsc4py.PETSc.Error: error code 4031
[0] MatMPIAIJSetPreallocation() at /home/ekrem/Dev/Venvs/dolfinxcomplex/installation/petsc/src/mat/impls/aij/mpi/mpiaij.c:4039
[0] MatMPIAIJSetPreallocation_MPIAIJ() at /home/ekrem/Dev/Venvs/dolfinxcomplex/installation/petsc/src/mat/impls/aij/mpi/mpiaij.c:2845
[0] MatSeqAIJSetPreallocation() at /home/ekrem/Dev/Venvs/dolfinxcomplex/installation/petsc/src/mat/impls/aij/seq/aij.c:3963
[0] MatSeqAIJSetPreallocation_SeqAIJ() at /home/ekrem/Dev/Venvs/dolfinxcomplex/installation/petsc/src/mat/impls/aij/seq/aij.c:4031
[0] PetscMallocA() at /home/ekrem/Dev/Venvs/dolfinxcomplex/installation/petsc/src/sys/memory/mal.c:401
[0] PetscMallocAlign() at /home/ekrem/Dev/Venvs/dolfinxcomplex/installation/petsc/src/sys/memory/mal.c:49
[0] MatSeqAIJSetPreallocation_SeqAIJ

I have attached my sparsity pattern as CSR.png.

I use petsc4py for matrix generation, and here are the commands that I am using to generate matrix;

row # global row numbers
col = # global column numbers
val = # flattened 2D nonzero array

mat = PETSc.Mat().create(PETSc.COMM_WORLD)
mat.setSizes([(local_size, global_size), (local_size, global_size)])
mat.setType('mpiaij')
ONNZ = len(row)*np.ones(local_size,dtype=np.int32)
mat.setPreallocationNNZ([ONNZ,ONNZ]) # This line throws the error above
mat.setOption(PETSc.Mat.Option.NEW_NONZERO_ALLOCATION_ERR, False)
mat.setUp()
mat.setValues(row, col, val, addv=PETSc.InsertMode.ADD_VALUES)
mat.assemblyBegin()
mat.assemblyEnd()

Many thanks for your response in advance,

Kind regards

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220801/edc32816/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CSR.png
Type: image/png
Size: 13851 bytes
Desc: CSR.png
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220801/edc32816/attachment-0001.png>