[petsc-users] On the edge of 2^31 unknowns

[Matthew Knepley]

Sometimes when we do not have exact counts, we need to overestimate sizes.
This is especially true
in sparse MatMat.

   Matt

On Mon, Nov 16, 2015 at 9:38 AM, Eric Chamberland <
Eric.Chamberland at giref.ulaval.ca> wrote:

> Hi,
>
> we just tried this morning to solve a 1.7 billion dofs problem (on 1060
> processors task)  .
>
> Unfortunately, petsc-3.5.3 didn't succeed within a MatMatMult product with
> the following backtrace:
>
> [0]PETSC ERROR: Out of memory. This could be due to allocating
> [0]PETSC ERROR: too large an object or bleeding by not properly
> [0]PETSC ERROR: destroying unneeded objects.
> [0]PETSC ERROR: Memory allocated 0 Memory used by process 4126371840
> [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info.
> [0]PETSC ERROR: Memory requested 18446744070461249536
> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.5.3, Jan, 31, 2015
> [0]PETSC ERROR: /rap/jsf-051-aa/ericc/GIREF/bin/probGD.opt on a
> linux-gnu-intel named r101-n31 by
> [0]PETSC ERROR: Configure options PETSC_ARCH=linux-gnu-intel CFLAGS="-O3
> -xHost -mkl -fPIC -m64 -n
> [0]PETSC ERROR: #1 PetscMallocAlign() line 46 in
> /software6/src/petsc-3.5.3/src/sys/memory/mal.c
> [0]PETSC ERROR: #2 PetscLLCondensedCreate_Scalable() line 1327 in
> /software6/src/petsc-3.5.3/inclu
> [0]PETSC ERROR: #3 MatMatMultSymbolic_MPIAIJ_MPIAIJ() line 741 in
> /software6/src/petsc-3.5.3/src/m
> [0]PETSC ERROR: #4 MatMatMult_MPIAIJ_MPIAIJ() line 33 in
> /software6/src/petsc-3.5.3/src/mat/impls/
> [0]PETSC ERROR: #5 MatMatMult() line 8713 in
> /software6/src/petsc-3.5.3/src/mat/interface/matrix.c
>
> The amount of memory requested is, apparently, an overflow of a PetscInt
> which is a 32 bit signed int.
>
> Bad user I am, I didn't expected that to solve a problem with less than
> 2^31 unknowns, it could be mandatory to compile petsc with 64bit indices...
>
> Is compiling with "--with-64-bit-indices" the only solution to my problem?
>
> Is it a known limitation/bug with a "Petsc-32bit-indices"?
>
> Thanks,
>
> Eric
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20151116/41173a5b/attachment.html>