[petsc-users] Question on MatCreateSeqAIJ() and 32-bit integer

[Lu Qiyue]

Hello Barry:
Switching to 64 bit, means to re-install PETSC and add
--with-64-bit-indices
when do the configuration, right?

Because re-install PETSC is not easy here and I am wondering, is there any
other way to solve this problem? Like add some ' -bigint' option while
compiling the driver? Looks not enough for this case, because PETSC still
at 32-bits.

>From what you said that 'because of internal counts inside of PETSc
(actually the way we store the matrices) it requires 64 bit indices', I am
afraid that re-install is the only solution?

Thanks

Qiyue Lu



On Tue, Jul 30, 2013 at 1:38 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
>   Lu,
>
>     You need to switch to 64 bit indices. Even though the individual
> number of non zeros per row is less than the 32 bit limit the total number
> of non zeros in the matrix is higher than the limit; how much physical
> memory you have is not relevant. As you note it is because of internal
> counts inside of PETSc (actually the way we store the matrices) it requires
> 64 bit indices.
>
>    Barry
>
> Switching to 64 bit indices should not be difficult, please let us know if
> you have any problems.
>
> On Jul 30, 2013, at 10:54 AM, Lu Qiyue <luqiyue at gmail.com> wrote:
>
> > Dear All:
> >
> > I am solving a huge system AX=b, A is a sparse matrix but its number of
> non-zeros is 3.29*10^9. I noticed that the 32-bit integer upper limit is
> ~2.15*10^9.
> >
> > A is in COO format here. When I prepare the input *.bin file for Petsc,
> the line
> >
> > ierr = MatCreateSeqAIJ(PETSC_COMM_WORLD,m,n,0,cnt,&A);CHKERRQ(ierr);
> >
> > can not pass.
> >
> > m =n =39979380 here, cnt is an array holding the number of non-zeros per
> row. The error message is:
> >
> > [0]PETSC ERROR: Out of memory. This could be due to allocating
> > [0]PETSC ERROR: too large an object or bleeding by not properly
> > [0]PETSC ERROR: destroying unneeded objects.
> > [0]PETSC ERROR: Memory allocated 0 Memory used by process 492802048
> > [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> > [0]PETSC ERROR: Memory requested 18446744061813993472!
> >
> > Calling MatCreateSeqAIJ() doesn't involve NNZ information there, m/n/cnt
> are all integers less than 32-bit integer limit.
> > And the total size of data in COO format: nnz*8 bytes(values)+nnz*4
> bytes(rows)+nnz*4 bytes(cols) are less than the memory limit of our system.
> >
> > The code works on a system with half size of this failed one.
> >
> > I am wondering, Does this because MatCreateSeqAIJ() might do some
> 'internal' counting things which exceeds the integer limit in this case?
> >
> > Thanks
> >
> > Qiyue Lu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130730/5855cec3/attachment.html>