[petsc-users] Question on MatCreateSeqAIJ() and 32-bit integer

[Barry Smith]

  Lu,

    You need to switch to 64 bit indices. Even though the individual number of non zeros per row is less than the 32 bit limit the total number of non zeros in the matrix is higher than the limit; how much physical memory you have is not relevant. As you note it is because of internal counts inside of PETSc (actually the way we store the matrices) it requires 64 bit indices.

   Barry

Switching to 64 bit indices should not be difficult, please let us know if you have any problems.

On Jul 30, 2013, at 10:54 AM, Lu Qiyue <luqiyue at gmail.com> wrote:

> Dear All:
>  
> I am solving a huge system AX=b, A is a sparse matrix but its number of non-zeros is 3.29*10^9. I noticed that the 32-bit integer upper limit is ~2.15*10^9. 
> 
> A is in COO format here. When I prepare the input *.bin file for Petsc, the line
> 
> ierr = MatCreateSeqAIJ(PETSC_COMM_WORLD,m,n,0,cnt,&A);CHKERRQ(ierr);
>  
> can not pass.
> 
> m =n =39979380 here, cnt is an array holding the number of non-zeros per row. The error message is:
> 
> [0]PETSC ERROR: Out of memory. This could be due to allocating
> [0]PETSC ERROR: too large an object or bleeding by not properly
> [0]PETSC ERROR: destroying unneeded objects.
> [0]PETSC ERROR: Memory allocated 0 Memory used by process 492802048
> [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> [0]PETSC ERROR: Memory requested 18446744061813993472!
> 
> Calling MatCreateSeqAIJ() doesn't involve NNZ information there, m/n/cnt are all integers less than 32-bit integer limit.
> And the total size of data in COO format: nnz*8 bytes(values)+nnz*4 bytes(rows)+nnz*4 bytes(cols) are less than the memory limit of our system. 
> 
> The code works on a system with half size of this failed one. 
> 
> I am wondering, Does this because MatCreateSeqAIJ() might do some 'internal' counting things which exceeds the integer limit in this case?
> 
> Thanks
> 
> Qiyue Lu