UMFPACK out of memory
Matthew Knepley
knepley at gmail.com
Tue Oct 27 11:00:41 CDT 2009
On Tue, Oct 27, 2009 at 10:56 AM, francois pacull <fpacull at fluorem.com>wrote:
> Thank you Matt for your quick reply.
>
> I will try to use MUMPS soon. About UMFPACK, the OS used is a 64-bit Linux
> one. Also, we do have 64-bit pointers from what I understand; the following
> lines are from the PETSc configure.log file:
>
> #ifndef PETSC_SIZEOF_VOID_P
> #define PETSC_SIZEOF_VOID_P 8
> #endif
>
Yes, then it appears that UMFPACK cannot be used for large memory. Did you
try MUMPS?
Matt
> And here is the umfpack_UMF_report_info from the subdomain that crashed:
>
>
> UMFPACK V5.4.0 (May 20, 2009), Info:
> matrix entry defined as: double
> Int (generic integer) defined as: int
> BLAS library used: Fortran BLAS. size of BLAS integer: 4
> MATLAB: no.
> CPU timer: POSIX times ( ) routine.
> number of rows in matrix A: 79002
> number of columns in matrix A: 79002
> entries in matrix A: 12030970
> memory usage reported in: 8-byte Units
> size of int: 4 bytes
> size of UF_long: 8 bytes
> size of pointer: 8 bytes
> size of numerical entry: 8 bytes
>
> strategy used: unsymmetric
> ordering used: colamd on A
> modify Q during factorization: yes
> prefer diagonal pivoting: no
> pivots with zero Markowitz cost: 0
> submatrix S after removing zero-cost pivots:
> number of "dense" rows: 0
> number of "dense" columns: 0
> number of empty rows: 0
> number of empty columns 0
> submatrix S square and diagonal preserved
> symbolic factorization defragmentations: 0
> symbolic memory usage (Units): 27435792
> symbolic memory usage (MBytes): 209.3
> Symbolic size (Units): 177636
> Symbolic size (MBytes): 1
> symbolic factorization CPU time (sec): 4.95
> symbolic factorization wallclock time(sec): 4.95
>
> symbolic/numeric factorization: upper bound actual
> %
> variable-sized part of Numeric object:
> initial size (Units) 31236744 -
> -
> peak size (Units) 597607658 -
> -
> final size (Units) 550474688 -
> -
> Numeric final size (Units) 550988250 -
> -
> Numeric final size (MBytes) 4203.7 -
> -
> peak memory usage (Units) 598718594 -
> -
> peak memory usage (MBytes) 4567.9 -
> -
> numeric factorization flops 1.63141e+12 -
> -
> nz in L (incl diagonal) 171352664 -
> -
> nz in U (incl diagonal) 346187947 -
> -
> nz in L+U (incl diagonal) 517461609 -
> -
> largest front (# entries) 15783705 -
> -
> largest # rows in front 2815 -
> -
> largest # columns in front 5607 -
> -
>
>
> UMFPACK V5.4.0 (May 20, 2009): ERROR: out of memory
>
> Thanks again,
> francois.
>
>
>
> Matthew Knepley a écrit :
>
> On Tue, Oct 27, 2009 at 10:12 AM, francois pacull <fpacull at fluorem.com<mailto:
>> fpacull at fluorem.com>> wrote:
>>
>> Dear PETSc team,
>>
>> I have a few questions... During the resolution of a linear system
>> in parallel, I am trying to apply a local LU solver to each of the
>> SEQaij diagonal blocks of a MPIaij matrix partitioned with PARMETIS.
>>
>> - I started with MUMPS but it seems that it only works with
>> unpartitioned aij matrices, is it really the case? Or could we use
>> MUMPS to build an additive Schwarz preconditioner for example?
>>
>>
>> You can use MUMPS for the subproblem solver.
>>
>> - Then I tried UMFPACK. This works fine when the diagonal blocks
>> (and the memory required to store the factors) are small but
>> crashes when they are a little bit larger. For example with a
>> "numeric final size" of 4203.7 MBytes, I got the following
>> message "ERROR: out of memory" while there was plenty of memory
>> left in the computer. I tried either with the UMFPACK version 5.2,
>> downloaded by PETSc, or with a manually installed version 5.4,
>> linked to PETSc. Is this a behavior from UMFPACK that you already
>> experienced?
>>
>>
>> Send all the error output. However, in oder to address more than 4G, you
>> will need 64-bit pointers.
>>
>> - Since UMFPACK seemed to have a memory limit around 4096 MB, I
>> tried to install a PETSc version with the option
>> "--with-64-bit-indices", however none of the partitioning packages
>> could be compiled with this option
>> (parmetis,chaco,jostle,party,scotch). Is there a way to compile
>> PETSc with 64 bit indices AND a partitioning package?
>>
>>
>> Not that I know of.
>>
>> - Finally, I tried to modify the PETSc source code umfpack.c so
>> that it would deal with 64 bit indices, but I only ended up so far
>> with a segmentation violation message at the execution... Is it
>> the only way I could use UMPACK with large sparse matrices?
>>
>>
>> Why not just upgrade to a 64-bit OS if you want to address so much memory
>> on a single machine?
>>
>> Matt
>>
>> Thank you,
>> Regards,
>> francois pacull.
>>
>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>
>
--
What most experimenters take for granted before they begin their experiments
is infinitely more interesting than any results to which their experiments
lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20091027/d9a011e3/attachment.htm>
More information about the petsc-users
mailing list