[petsc-users] Question on MatCreateSeqAIJ() and 32-bit integer

[Barry Smith]

  Select a new PETSC_ARCH (so you don't over write the old install) and make a new install, use all the same other ./configure options. Why is an install difficult? You can run an install in your home directory without help from anyone else; an entire install takes about 5 minutes on my laptop so it should not be too time consuming. Send the configure.log and make.log to petsc-maint at mcs.anl.gov the new install fails.

   Barry

On Jul 30, 2013, at 2:05 PM, Lu Qiyue <luqiyue at gmail.com> wrote:

> Hello Barry:
> Switching to 64 bit, means to re-install PETSC and add 
> --with-64-bit-indices
> when do the configuration, right?
> 
> Because re-install PETSC is not easy here and I am wondering, is there any other way to solve this problem? Like add some ' -bigint' option while compiling the driver? Looks not enough for this case, because PETSC still at 32-bits.
> 
> From what you said that 'because of internal counts inside of PETSc (actually the way we store the matrices) it requires 64 bit indices', I am afraid that re-install is the only solution?
> 
> Thanks
> 
> Qiyue Lu
> 
> 
> 
> On Tue, Jul 30, 2013 at 1:38 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
>   Lu,
> 
>     You need to switch to 64 bit indices. Even though the individual number of non zeros per row is less than the 32 bit limit the total number of non zeros in the matrix is higher than the limit; how much physical memory you have is not relevant. As you note it is because of internal counts inside of PETSc (actually the way we store the matrices) it requires 64 bit indices.
> 
>    Barry
> 
> Switching to 64 bit indices should not be difficult, please let us know if you have any problems.
> 
> On Jul 30, 2013, at 10:54 AM, Lu Qiyue <luqiyue at gmail.com> wrote:
> 
> > Dear All:
> >
> > I am solving a huge system AX=b, A is a sparse matrix but its number of non-zeros is 3.29*10^9. I noticed that the 32-bit integer upper limit is ~2.15*10^9.
> >
> > A is in COO format here. When I prepare the input *.bin file for Petsc, the line
> >
> > ierr = MatCreateSeqAIJ(PETSC_COMM_WORLD,m,n,0,cnt,&A);CHKERRQ(ierr);
> >
> > can not pass.
> >
> > m =n =39979380 here, cnt is an array holding the number of non-zeros per row. The error message is:
> >
> > [0]PETSC ERROR: Out of memory. This could be due to allocating
> > [0]PETSC ERROR: too large an object or bleeding by not properly
> > [0]PETSC ERROR: destroying unneeded objects.
> > [0]PETSC ERROR: Memory allocated 0 Memory used by process 492802048
> > [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> > [0]PETSC ERROR: Memory requested 18446744061813993472!
> >
> > Calling MatCreateSeqAIJ() doesn't involve NNZ information there, m/n/cnt are all integers less than 32-bit integer limit.
> > And the total size of data in COO format: nnz*8 bytes(values)+nnz*4 bytes(rows)+nnz*4 bytes(cols) are less than the memory limit of our system.
> >
> > The code works on a system with half size of this failed one.
> >
> > I am wondering, Does this because MatCreateSeqAIJ() might do some 'internal' counting things which exceeds the integer limit in this case?
> >
> > Thanks
> >
> > Qiyue Lu
> 
>