SBAIJ issue

Barry Smith bsmith at mcs.anl.gov
Mon Oct 12 17:30:16 CDT 2009


   Perhaps you have done enough memory preallocation for the  
multiprocessors and it is going very--very slowly with memory  
allocations?

   Barry

On Oct 12, 2009, at 8:03 AM, Andreas Grassl wrote:

> Hello,
>
> I'm trying to work with MUMPS-Cholesky as direct solver and CG as  
> iterative
> solver and have problems with SBAIJ-Matrices.
>
> I have a MatSetValues-line not regarding the symmetry but force it  
> by the
> command line option -mat_ignore_lower_triangular.
>
> The problems show up at "bigger" problems (~450000 DOF) with "more"  
> processes (1
> and 2 work fine, 4 gives a hangup) and/or trying to save the matrix.  
> The machine
> has 4 cores and 12 GB memory. NNZ per row is ~80.
>
> Attached you find the output of the program run with -info.
>
> Any hints where to search?
>
> Cheers,
>
> ando
>
> -- 
> /"\                               Grassl Andreas
> \ /    ASCII Ribbon Campaign      Uni Innsbruck Institut f. Mathematik
>  X      against HTML email        Technikerstr. 13 Zi 709
> / \                               +43 (0)512 507 6091
> same call as 2procinfo without -ksp_info_binary
>
> mpiexec -np 2 externalsolver
> [0] PetscInitialize(): PETSc successfully started: number of  
> processors = 2
> [1] PetscInitialize(): PETSc successfully started: number of  
> processors = 2
> [1] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [0] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [0] PetscCommDuplicate():   returning tag 2147483647
> [1] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [1] PetscCommDuplicate():   returning tag 2147483647
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483646
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483646
> [0] MatSetUpPreallocation(): Warning not preallocating matrix storage
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [0] PetscCommDuplicate():   returning tag 2147483647
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [1] PetscCommDuplicate():   returning tag 2147483647
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483646
> [0] PetscCommDuplicate():   returning tag 2147483646
> [0] MatSetOption_MPISBAIJ(): Option SYMMETRIC ignored
> [0] MatStashScatterBegin_Private(): No of messages: 1
> [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 30544064
> [0] MatStashScatterBegin_Private(): No of messages: 0
> [0] MatAssemblyBegin_MPISBAIJ(): Stash has 3818007 entries,uses 8  
> mallocs.
> [0] MatAssemblyBegin_MPISBAIJ(): Block-Stash has 3818007 entries,  
> uses 8 mallocs.
> [0] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 221545 X 221545, block  
> size 1; storage space: 16676676 unneeded, 5477824 used
> [0] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [0] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 62
> [1] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 225367 X 225367, block  
> size 1; storage space: 16793278 unneeded, 5743422 used
> [1] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [1] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 65
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483645
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483644
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483643
> [1] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483645
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483644
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483643
> [0] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate():   returning tag 2147483634
> [1] PetscCommDuplicate():   returning tag 2147483634
> [1] PetscCommDuplicate():   returning tag 2147483630
> [1] PetscCommDuplicate():   returning tag 2147483625
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483642
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483641
> [1] PetscCommDuplicate():   returning tag 2147483620
> [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
> [0] VecScatterCreate(): General case: MPI to Seq
> [0] PetscCommDuplicate():   returning tag 2147483630
> [0] PetscCommDuplicate():   returning tag 2147483625
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483642
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483641
> [0] PetscCommDuplicate():   returning tag 2147483620
> [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
> [0] VecScatterCreate(): General case: MPI to MPI
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483640
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDupli[1] PetscCommDuplicate(): Using internal PETSc  
> communicator 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483640
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483639
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483638
> [1] MatAssemblyEnd_SeqBAIJ(): Matrix size: 225367 X 0, block size 1;  
> storage space: 22536700 unneeded, 0 used
> [1] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [1] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 0
> [1] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 225367)/ 
> (num_localrows 225367) > 0.6. Use CompressedRow routines.
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483614
> [1] PetscCommDuplicate():   returning tag 2147483613
> [1]before KSPsetup
> [1] PetscCommDuplicate(): Using cate():   returning tag 2147483638
> [0] MatAssemblyEnd_SeqBAIJ(): Matrix size: 221545 X 6275, block size  
> 1; storage space: 22060506 unneeded, 93994 used
> [0] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [0] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 32
> [0] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 216534)/ 
> (num_localrows 221545) > 0.6. Use CompressedRow routines.
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483614
> [0] PetscCommDuplicate():   returning tag 2147483613
> [0]before KSPsetup
> [0] PCSetUp(): Setting up new PC
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483637
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483636
> [0] PetscCommDuplicate():   returning tag 2147483612
> [0] PetscCommDuplicate(): Using internainternal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483637
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483636
> [1] PetscCommDuplicate():   returning tag 2147483612
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483635
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483634
> l PETSc communicator 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483635
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483634
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483633
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483632
> [1] PetscCommDuplicate():   returning tag 2147483607
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483633
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483632
> [0] PetscCommDuplicate():   returning tag 2147483607
> [0] PetscCommDuplicate():   returning tag 2147483602
> [0] VecScatterCreate(): Special case: processor zero gets entire  
> parallel vector, rest get none
> [1] PetscCommDuplicate():   returning tag 2147483602
>
> DMUMPS 4.8.4
> L D L^T Solver for symmetric positive definite matrices
> Type of parallelism: Working host
>
> ****** ANALYSIS STEP ********
>
> Density: NBdense, Average, Median   =    0   49   50
> Ordering based on METIS
> ** Peak of sequential stack size (number of real entries)   :  
> 16763905.
> A root of estimated size         2965  has been selected for  
> Scalapack.
>
> Leaving analysis phase with  ...
> INFOG(1)                                       =               0
> INFOG(2)                                       =               0
> -- (20) Number of entries in factors (estim.) =       120657071
> --  (3) Storage of factors  (REAL, estimated) =       137364268
> --  (4) Storage of factors  (INT , estimated) =         6138804
> --  (5) Maximum frontal size      (estimated) =            3705
> --  (6) Number of nodes in the tree           =           21863
> --  (7) Ordering option effectively used      =               5
> ICNTL(6) Maximum transversal option            =               0
> ICNTL(7) Pivot order option                    =               7
> Percentage of memory relaxation (effective)    =             200
> Number of level 2 nodes                        =               1
> Number of split nodes                          =               0
> RINFO(1) Operations during elimination (estim) =   1.114D+11
> Distributed matrix entry format (ICNTL(18))    =               3
> ** Rank of proc needing largest memory in IC facto        :         0
> ** Estimated corresponding MBYTES for IC facto            :      2085
> ** Estimated avg. MBYTES per work. proc at facto (IC)     :      2044
> ** TOTAL     space in MBYTES for IC factorization         :      4088
> ** Rank of proc needing largest memory for OOC facto      :         1
> ** Estimated corresponding MBYTES for OOC facto           :       644
> ** Estimated avg. MBYTES per work. proc at facto (OOC)    :       630
> ** TOTAL     space in MBYTES for OOC factorization        :      1261
>
> ****** FACTORIZATION STEP ********
>
>
> GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
> NUMBER OF WORKING PROCESSES          =           2
> REAL SPACE FOR FACTORS               =   137364268
> INTEGER SPACE FOR FACTORS            =     6138804
> MAXIMUM FRONTAL SIZE (ESTIMATED)     =        3705
> NUMBER OF NODES IN THE TREE          =       21863
> Maximum effective relaxed size of S              =   239060000
> Average effective relaxed size of S              =   234725641
>
> REDISTRIB: TOTAL DATA LOCAL/SENT     =     9086109     2238491
> GLOBAL TIME FOR MATRIX DISTRIBUTION  =      0.7322
> ** Memory relaxation parameter ( ICNTL(14)  )            :       200
> ** Rank of processor needing largest memory in facto     :         0
> ** Space in MBYTES used by this processor for facto      :      2085
> ** Avg. Space in MBYTES per working proc during facto    :      2044
>
> ELAPSED TIME FOR FACTORIZATION       =     28.1724
> Maximum effective space used in S    (KEEP(67)   =    77516871
> Average effective space used in S    (KEEP(67)   =    75395654
> ** EFF Min: Rank of processor needing largest memory :         0
> ** EFF Min: Space in MBYTES used by this processor   :       741
> ** EFF Min: Avg. Space in MBYTES per working proc    :       720
>
> GLOBAL STATISTICS
> RINFOG(2)  OPERATIONS DURING NODE ASSEMBLY     = 2.565D+08
> ------(3)  OPERATIONS DURING NODE ELIMINATION  = 1.114D+11
> INFOG (9)  REAL SPACE FOR FACTORS              =   137364268
> INFOG(10)  INTEGER SPACE FOR FACTORS           =     6138809
> INFOG(11)  MAXIMUM FRONT SIZE                  =        3705
> INFOG(29)  NUMBER OF ENTRIES IN FACTORS        =   116259976
> INFOG(12) NB OF NEGATIVE PIVOTS          =           0
> INFOG(14)  NUMBER OF MEMORY COMPRESS           =           0
> [0]after KSPsetup
> [0]RHS setup
> [1]after KSPsetup
> [1]RHS setup
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483601
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483601
> [0] PetscCommDuplicate():   returning tag 2147483596
> [1] PetscCommDuplicate():   returning tag 2147483596
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483591
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483591
> [0] VecAssemblyBegin_MPI(): Stash has 225367 entries, uses 12 mallocs.
> [0] VecAssemblyBegin_MPI(): Block-Stash has 0 entries, uses 0 mallocs.
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483631
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483631
>
>
> ****** SOLVE & CHECK STEP ********
>
>
> STATISTICS PRIOR SOLVE PHASE     ...........
> NUMBER OF RIGHT-HAND-SIDES                    =           1
> BLOCKING FACTOR FOR MULTIPLE RHS              =           1
> ICNTL (9)                                     =           1
>  --- (10)                                     =           0
>  --- (11)                                     =           0
>  --- (20)                                     =           0
>  --- (21)                                     =           1
> ** Rank of processor needing largest memory in solve     :         0
> ** Space in MBYTES used by this processor for solve      :      2013
> ** Avg. Space in MBYTES per working proc during solve    :      1975
>
>
> LEAVING SOLVER WITH:  INFOG(1) ............ =           0
>                       INFOG(2) ............ =           0
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483630
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483629
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483630
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483629
> [0] PetscCommDuplicate():   returning tag 2147483590
> [1] PetscCommDuplicate():   returning tag 2147483590
> [1] VecScatterCreate(): General case: Seq to MPI
> [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
> [0] VecScatterCreate(): General case: Seq to MPI
> [0] PetscCommDuplicate():   returning tag 2147483586
> KSP Object:
>  type: preonly
>  maximum iterations=10000, initial guess is zero
>  tolerances:  relative=1e-06, absolute=1e-50, divergence=10000
>  left preconditioning
> PC Object:
>  type: cholesky
>    Cholesky: out-of-place factorization
>      matrix ordering: natural
>    Cholesky: factor fill ratio needed 0
>         Factored matrix follows
>        Matrix Object:
>          type=mpisbaij, rows=446912, cols=446912
>          package used to perform factorization: mumps
> [1] PetscCommDuplicate():   returning tag 2147483586
> [1] PetscCommDuplicate():   returning tag 2147483585
> [1] PetscCommDuplicate():   returning tag 2147483584
> [1] PetscCommDuplicate():   returning tag 2147483583
> [1] PetscCommDuplicate():   returning tag 2147483582
> [1] PetscCommDuplicate():   returning tag 2147483581
> [1] PetscCommDuplicate():   returning tag 2147483580
>          total: nonzeros=0, allocated nonzeros=893824
>            MUMPS run parameters:
>              SYM (matrix type):                  1
>              PAR (host participation):           1
>              ICNTL(1) (output for error):        6
>              ICNTL(2) (output of diagnostic msg):0
>              ICNTL(3) (output for global info):  6
>              ICNTL(4) (level of printing):       -1
>              ICNTL(5) (input mat struct):        0
>              ICNTL(6) (matrix prescaling):       0
>              ICNTL(7) (matrix ordering):         7
>              ICNTL(8) (scalling strategy):       77
>              ICNTL(9) (A/A^T x=b is solved):     1
>              ICNTL(10) (max num of refinements): 0
>              ICNTL(11) (error analysis):         0
>              ICNTL(12) (efficiency control):                         1
>              ICNTL(13) (efficiency control):                         0
>              ICNTL(14) (percentage of estimated workspace increase):  
> 200
>              ICNTL(18)[1] PetscCommDuplicate():   returning tag  
> 2147483579
> (input mat struct):                           3
>              ICNTL(19) (Shur complement info):                       0
>              ICNTL(20) (rhs sparse pattern):                         0
>              ICNTL(21) (solution struct):                            1
>              ICNTL(22) (in-core/out-of-core facility):               0
>              ICNTL(23) (max size of memory can be allocated locally):0
>              ICNTL(24) (detection of null pivot rows):               0
>              ICNTL(25) (computation of a null space basis):          0
>              ICNTL(26) (Schur options for rhs or solution):          0
>              ICNTL(27) (experimental parameter):                      
> -8
>              CNTL(1) (relative pivoting threshold):      0
>              CNTL(2) (stopping criterion of refinement): 1.49012e-08
>              CNTL(3) (absolute pivoting threshold):      0
>              CNTL(4) (value of static pivoting):         -1
>              CNTL(5) (fixation for null pivots):         0
>      RINFO(1) (local estimated flops for the elimination after  
> analysis):
>             [0] 5.16132e+10
> [0] PetscCommDuplicate():   returning tag 2147483585
>             [1] 5.97957e+10
>      RINFO(2) (local estimated flops for the assembly after  
> factorization):
>             [0]  1.33366e+08
> [0] PetscCommDuplicate():   returning tag 2147483584
>             [1]  1.23105e+08
>      RINFO(3) (local estimated flops for the elimination after  
> factorization):
>             [0]  5.16124e+10
> [0] PetscCommDuplicate():   returning tag 2147483583
>             [1]  5.9795e+10
>      INFO(15) (estimated size of (in MB) MUMPS internal data for  
> running numerical factorization):
>             [0] 2085
> [0] PetscCommDuplicate():   returning tag 2147483582
>             [1] 2003
>      INFO(16) (size of (in MB) MUMPS internal data used during  
> numerical factorization):
>             [0] 2085
> [0] PetscCommDuplicate():   returning tag 2147483581
>             [1] 2003
>      INFO(23) (num of pivots eliminated on this processor after  
> factorization):
>             [0] 236610
> [0] PetscCommDuplicate():   returning tag 2147483580
>             [1] 210302
>              RINFOG(1) (global estimated flops for the elimination  
> after analysis): 1.11409e+11
>              RINFOG(2) (global estimated flops for the assembly  
> after factorization): 2.56471e+08
>              RINFOG(3) (global estimated flops for the elimination  
> after factorization): 1.11407e+11
>              INFOG(3) (estimated real workspace for factors on all  
> processors after analysis): 137364268
>              INFOG(4) (estimated integer workspace for factors on  
> all processors after analysis): 6138804
>              INFOG(5) (estimated maximum front size in the complete  
> tree): 3705
>              INFOG(6) (number of nodes in the complete tree): 21863
>              INFOG(7) (ordering option effectively uese after  
> analysis): 5
>              INFOG(8) (structural symmetry in percent of the  
> permuted matrix after analysis): 100
>              INFOG(9) (total real/complex workspace to store the  
> matrix factors after factorization): 137364268
>              INFOG(10) (total integer space store the matrix factors  
> after factorization): 6138809
>              INFOG(11) (order of largest frontal matrix after  
> factorization): 3705
>              INFOG(12) (number of off-diagonal pivots): 0
>              INFOG(13) (number of delayed pivots after  
> factorization): 0
>              INFOG(14) (number of memory compress after  
> factorization): 0
>              INFOG(15) (number of steps of iterative refinement  
> after solution): 0
>              INFOG(16) (estimated size (in MB) of all MUMPS internal  
> data for factorization after analysis: value on the most memory  
> consuming processor): 2085
>              INFOG(17) (estimated size of all MUMPS internal data  
> for factorization after analysis: sum over all processors): 4088
>              INFOG(18) (size of all MUMPS internal data allocated  
> during factorization: value on the most memory consuming processor):  
> 2085
>              INFOG(19) (size of all MUMPS internal data allocated  
> during factorization: sum over all processors): 4088
>              INFOG(20) (estimated number of entries in the factors):  
> 120657071
>              INFOG(21) (size in MB of memory effectively used during  
> factorization - value on the most memory consuming processor): 741
>              INFOG(22) (size in MB of memory effectively used during  
> factorization - sum over all processors): 1441
>              INFOG(23) (after analysis: value of ICNTL(6)  
> effectively used): 0
>              INFOG(24) (after analysis: value of ICNTL(12)  
> effectively used): 1
>              INFOG(25) (after factorization: number of pivots  
> modified by static pivoting): 0
>  linear system matrix = precond matrix:
>  Matrix Object:
>    type=mpisbaij, rows=446912, cols=446912
>    total: nonzeros=11315240, allocated nonzeros=89382400
>        block size is 1
> [0] PetscCommDuplicate():   returning tag 2147483579
> [1] PetscCommDuplicate():   returning tag 2147483578
> [1] PetscCommDuplicate():   returning tag 2147483577
> [1] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483578
> [0] PetscCommDuplicate():   returning tag 2147483577
> [0] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784
> [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm 1140850689
> [0] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374783
> [0] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374783
> [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483576
> [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm 1140850689
> [1] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374783
> [1] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374783
> [0] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784
> [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm 1140850688
> [0] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784
> [0] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784
> [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm -2080374784
> [0] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784
> ************************************************************************************************************************
> ***             WIDEN YOUR WINDOW TO 120 CHARACTERS.  Use 'enscript - 
> r -fCourier9' to print this document            ***
> ************************************************************************************************************************
>
> ---------------------------------------------- PETSc Performance  
> Summary: ----------------------------------------------
>
> [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483576
> [1] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784
> [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm 1140850688
> [1] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784
> [1] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784
> [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm -2080374784
> [1] Petsc_DelViewer(): Deleting viewer data in an MPI_Comm -2080374784
> externalsolver on a linux32-i named mat1.uibk.ac.at with 2  
> processors, by csae1801 Mon Oct 12 12:08:38 2009
> Using Petsc Release Version 3.0.0, Patch 8, Fri Aug 21 14:02:12 CDT  
> 2009
>
>                         Max       Max/Min        Avg      Total
> Time (sec):           3.989e+01      1.00000   3.989e+01
> Objects:              3.600e+01      1.00000   3.600e+01
> Flops:                0.000e+00      0.00000   0.000e+00  0.000e+00
> Flops/sec:            0.000e+00      0.00000   0.000e+00  0.000e+00
> MPI Messages:         2.550e+01      1.08511   2.450e+01  4.900e+01
> MPI Message Lengths:  6.596e+07      1.00053   2.692e+06  1.319e+08
> MPI Reductions:       5.800e+01      1.00000
>
> Flop counting convention: 1 flop = 1 real number operation of type  
> (multiply/divide/add/subtract)
>                            e.g., VecAXPY() for real vectors of  
> length N --> 2N flops
>                            and VecAXPY() for complex vectors of  
> length N --> 8N flops
>
> Summary of Stages:   ----- Time ------  ----- Flops -----  ---  
> Messages ---  -- Message Lengths --  -- Reductions --
>                        Avg     %Total     Avg     %Total   counts    
> %Total     Avg         %Total   counts   %Total
> 0:      Main Stage: 2.8402e-01   0.7%  0.0000e+00   0.0%  1.000e 
> +00   2.0%  8.163e-02        0.0%  0.000e+00   0.0%
> 1:   decomposition: 4.7451e-01   1.2%  0.0000e+00   0.0%  0.000e 
> +00   0.0%  0.000e+00        0.0%  0.000e+00   0.0%
> 2:   Matrix Create: 7.2766e-01   1.8%  0.0000e+00   0.0%  0.000e 
> +00   0.0%  0.000e+00        0.0%  2.000e+00   3.4%
> 3:     Matrix fill: 2.5720e+00   6.4%  0.0000e+00   0.0%  1.600e+01   
> 32.7%  2.486e+06       92.4%  1.500e+01  25.9%
> 4:    Solver setup: 3.2841e+01  82.3%  0.0000e+00   0.0%  0.000e 
> +00   0.0%  0.000e+00        0.0%  8.000e+00  13.8%
> 5:       RHS setup: 3.2125e-02   0.1%  0.0000e+00   0.0%  4.000e 
> +00   8.2%  1.094e+05        4.1%  4.000e+00   6.9%
> 6:           Solve: 4.9751e-01   1.2%  0.0000e+00   0.0%  2.500e+01   
> 51.0%  5.944e+04        2.2%  6.000e+00  10.3%
> 7:        Postproc: 2.4653e+00   6.2%  0.0000e+00   0.0%  3.000e 
> +00   6.1%  3.679e+04        1.4%  0.000e+00   0.0%
>
> ------------------------------------------------------------------------------------------------------------------------
> See the 'Profiling' chapter of the users' manual for details on  
> interpreting output.
> Phase summary info:
>   Count: number of times phase was executed
>   Time and Flops: Max - maximum over all processors
>                   Ratio - ratio of maximum to minimum over all  
> processors
>   Mess: number of messages sent
>   Avg. len: average message length
>   Reduct: number of global reductions
>   Global: entire computation
>   Stage: stages of a computation. Set stages with  
> PetscLogStagePush() and PetscLogStagePop().
>      %T - percent time in this phase         %F - percent flops in  
> this phase
>      %M - percent messages in this phase     %L - percent message  
> lengths in this phase
>      %R - percent reductions in this phase
>   Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max  
> time over all processors)
> ------------------------------------------------------------------------------------------------------------------------
> Event                Count      Time (sec)      
> Flops                             --- Global ---  --- Stage ---    
> Total
>                   Max Ratio  Max     Ratio   Max  Ratio  Mess   Avg  
> len Reduct  %T %F %M %L %R  %T %F %M %L %R Mflop/s
> ------------------------------------------------------------------------------------------------------------------------
>
> --- Event Stage 0: Main Stage
>
> PetscBarrier           1 1.0 3.1269e-0328.4 0.00e+00 0.0 0.0e+00 0.0e 
> +00 0.0e+00  0  0  0  0  0   1  0  0  0  0     0
>
> --- Event Stage 1: decomposition
>
>
> --- Event Stage 2: Matrix Create
>
>
> --- Event Stage 3: Matrix fill
>
> MatAssemblyBegin       1 1.0 3.3235e-01 1.9 0.00e+00 0.0 6.0e+00 2.0e 
> +07 3.0e+00  1  0 12 92  5  10  0 38100 20     0
> MatAssemblyEnd         1 1.0 1.0536e+00 1.0 0.00e+00 0.0 1.0e+01 1.8e 
> +04 1.2e+01  3  0 20  0 21  41  0 62  0 80     0
>
> --- Event Stage 4: Solver setup
>
> MatCholFctrSym         1 1.0 4.0531e-06 2.1 0.00e+00 0.0 0.0e+00 0.0e 
> +00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
> MatCholFctrNum         1 1.0 3.2830e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e 
> +00 5.0e+00 82  0  0  0  9 100  0  0  0 62     0
> KSPSetup               1 1.0 6.9141e-06 1.4 0.00e+00 0.0 0.0e+00 0.0e 
> +00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
> PCSetUp                1 1.0 3.2841e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e 
> +00 8.0e+00 82  0  0  0 14 100  0  0  0100     0
>
> --- Event Stage 5: RHS setup
>
> VecAssemblyBegin       1 1.0 6.2160e-03 1.6 0.00e+00 0.0 4.0e+00 1.3e 
> +06 3.0e+00  0  0  8  4  5  16  0100100 75     0
> VecAssemblyEnd         1 1.0 8.4589e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e 
> +00 0.0e+00  0  0  0  0  0  26  0  0  0  0     0
>
> --- Event Stage 6: Solve
>
> MatSolve               1 1.0 4.9525e-01 1.0 0.00e+00 0.0 6.0e+00 4.8e 
> +05 4.0e+00  1  0 12  2  7 100  0 24 98 67     0
> MatView                2 1.0 1.4269e-03 1.0 0.00e+00 0.0 1.8e+01 2.7e 
> +03 2.0e+00  0  0 37  0  3   0  0 72  2 33     0
> VecSet                 1 1.0 8.1515e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e 
> +00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
> VecScatterBegin        2 1.0 6.7840e-03 1.3 0.00e+00 0.0 2.0e+00 1.3e 
> +06 0.0e+00  0  0  4  2  0   1  0  8 86  0     0
> VecScatterEnd          1 1.0 2.1610e-03 1.7 0.00e+00 0.0 0.0e+00 0.0e 
> +00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
> KSPSolve               1 1.0 4.9603e-01 1.0 0.00e+00 0.0 6.0e+00 4.8e 
> +05 4.0e+00  1  0 12  2  7 100  0 24 98 67     0
> PCApply                1 1.0 4.9526e-01 1.0 0.00e+00 0.0 6.0e+00 4.8e 
> +05 4.0e+00  1  0 12  2  7 100  0 24 98 67     0
>
> --- Event Stage 7: Postproc
>
> VecView                1 1.0 2.1809e+00 2.0 0.00e+00 0.0 2.0e+00 9.0e 
> +05 0.0e+00  4  0  4  1  0  66  0 67100  0     0
> ------------------------------------------------------------------------------------------------------------------------
>
> Memory usage is given in bytes:
>
> Object Type          Creations   Destructions   Memory  Descendants'  
> Mem.
>
> --- Event Stage 0: Main Stage
>
>   IS L to G Mapping     0              1     889756     0
>              Matrix     0              3  547690752     0
>                 Vec     0              6    3702888     0
>         Vec Scatter     0              2       1736     0
>              Viewer     0              1        544     0
>
> --- Event Stage 1: decomposition
>
>
> --- Event Stage 2: Matrix Create
>
>   IS L to G Mapping     1              0          0     0
>              Matrix     3              0          0     0
>
> --- Event Stage 3: Matrix fill
>
>           Index Set     4              4     152680     0
>                 Vec     7              1       1304     0
>         Vec Scatter     2              0          0     0
>
> --- Event Stage 4: Solver setup
>
>           Index Set     3              2       1008     0
>              Matrix     3              0          0     0
>                 Vec     2              1    1773664     0
>         Vec Scatter     1              0          0     0
>       Krylov Solver     1              0          0     0
>      Preconditioner     1              0          0     0
>
> --- Event Stage 5: RHS setup
>
>                 Vec     2              0          0     0
>              Viewer     1              0          0     0
>
> --- Event Stage 6: Solve
>
>           Index Set     2              2     947456     0
>                 Vec     1              0          0     0
>         Vec Scatter     1              0          0     0
>              Viewer     1              0          0     0
>
> --- Event Stage 7: Postproc
>
>           Index Set     0              1        504     0
>              Matrix     0              3    8870136     0
>                 Vec     0              4    7125104     0
>         Vec Scatter     0              2       1320     0
>       Krylov Solver     0              1        832     0
>      Preconditioner     0              1        728     0
>              Viewer     0              1        544     0
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> = 
> =================================================================[1]  
> PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [1] PetscCommDuplicate():   returning tag 2147483647
> [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm 1140850688
> [1] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784
> [1] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784
> [1] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm -2080374784
> ========================================
> Average time to get PetscTime(): 1.3113e-06
> Average time for MPI_Barrier(): 0.000144005
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [0] PetscCommDuplicate():   returning tag 2147483647
> Average time for zero size MPI_Send(): 6.54459e-05
> [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm 1140850688
> [0] PetscCommDestroy(): Deleting PETSc MPI_Comm -2080374784
> [0] Petsc_DelTag(): Deleting tag data in an MPI_Comm -2080374784
> [0] Petsc_DelComm(): Deleting PETSc communicator imbedded in a user  
> MPI_Comm -2080374784
> #PETSc Option Table entries:
> -dump_data false
> -info
> -ksp_compute_singularvalues
> -ksp_rtol 1e-6
> -ksp_type preonly
> -ksp_view
> -log_summary
> -mat_ignore_lower_triangular
> -mat_mumps_icntl_14 200
> -mat_mumps_sym 1
> -mat_type sbaij
> -monitorname errormonitor.dat
> -nz 100
> -options_left
> -pc_factor_mat_solver_package mumps
> -pc_type cholesky
> -preload false
> -verb false
> #End o PETSc Option Table entries
> Compiled without FORTRAN kernels
> Compiled with full precision matrices (default)
> sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8  
> sizeof(PetscScalar) 8
> Configure run at: Wed Oct  7 18:16:40 2009
> Configure options: --with-scalar-type=real --with-debugging=0 --with- 
> precision=double --with-cc=icc --with-fc=ifort --with-cxx=icpc -- 
> with-shared=0 --with-mpi=1 --with-external-packages-dir=/home/lux/ 
> csae1801/petsc/externalpackages --download-mpich=ifneeded --with- 
> scalapack=1 --download-scalapack=ifneeded --download-f-blas- 
> lapack=ifneeded --with-blacs=1 --download-blacs=ifneeded --with- 
> parmetis=1 --download-parmetis=ifneeded --with-mumps=1 --download- 
> mumps=ifneeded --with-hypre=1 --with-hypre-dir=/home/lux/csae1801/ 
> petsc/externalpackages/hypre-2.4.0b/src/hypre --with-spooles=1 -- 
> download-spooles=ifneeded --with-superlu_dist=1 --download- 
> superlu_dist=ifneeded PETSC_ARCH=linux32-intel-c-prod
> -----------------------------------------
> Libraries compiled on Mi 7. Okt 18:17:41 CEST 2009 on mat1.uibk.ac.at
> Machine characteristics: Linux mat1.uibk.ac.at 2.6.18-53.el5xen #1  
> SMP Wed Oct 10 16:48:44 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
> Using PETSc directory: /home/lux/csae1801/petsc/petsc-3.0.0-p8
> Using PETSc arch: linux32-intel-c-prod
> -----------------------------------------
> Using C compiler: /home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32- 
> intel-c-prod/bin/mpicc -O
> Using Fortran compiler: /home/lux/csae1801/petsc/petsc-3.0.0-p8/ 
> linux32-intel-c-prod/bin/mpif90 -O
> -----------------------------------------
> Using include paths: -I/home/lux/csae1801/petsc/petsc-3.0.0-p8/ 
> linux32-intel-c-prod/include -I/home/lux/csae1801/petsc/petsc-3.0.0- 
> p8/include -I/home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32-intel-c- 
> prod/include -I/home/lux/csae1801/petsc/externalpackages/ 
> hypre-2.4.0b/src/hypre/include
> ------------------------------------------
> Using C linker: /home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32- 
> intel-c-prod/bin/mpicc -O
> Using Fortran linker: /home/lux/csae1801/petsc/petsc-3.0.0-p8/ 
> linux32-intel-c-prod/bin/mpif90 -O
> Using libraries: -Wl,-rpath,/home/lux/csae1801/petsc/petsc-3.0.0-p8/ 
> linux32-intel-c-prod/lib -L/home/lux/csae1801/petsc/petsc-3.0.0-p8/ 
> linux32-intel-c-prod/lib -lpetscts -lpetscsnes -lpetscksp -lpetscdm - 
> lpetscmat -lpetscvec -lpetsc        -lX11 -Wl,-rpath,/home/lux/ 
> csae1801/petsc/petsc-3.0.0-p8/linux32-intel-c-prod/lib -L/home/lux/ 
> csae1801/petsc/petsc-3.0.0-p8/linux32-intel-c-prod/lib - 
> lsuperlu_dist_2.3 -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common  
> -lpord -lparmetis -lmetis -lscalapack -lblacs -Wl,-rpath,/home/lux/ 
> csae1801/petsc/externalpackages/hypre-2.4.0b/src/hypre/lib -L/home/ 
> lux/csae1801/petsc/externalpackages/hypre-2.4.0b/src/hypre/lib - 
> lHYPRE -lmpichcxx -lstdc++ -lspooles -lflapack -lfblas -lnsl -laio - 
> lrt -lPEPCF90 -ldl -L/home/lux/csae1801/petsc/petsc-3.0.0-p8/linux32- 
> intel-c-prod/lib -lmpich -lpthread -lrt -L/opt/intel/cmkl/10.1.1.019/ 
> lib/em64t -L/opt/intel/Compiler/11.0/074/ipp/em64t/lib -L/opt/intel/ 
> Compiler/11.0/074/mkl/lib/em64t -L/opt/intel/Compiler/11.0/074/tbb/ 
> em64t/cc4.1.0_libc2.4_kernel2.6.16.21/lib -L/opt/intel/Compiler/ 
> 11.0/074/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -limf - 
> lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s -lmpichf90 -lifport - 
> lifcore -lm -lm -lmpichcxx -lstdc++ -lmpichcxx -lstdc++ -ldl -lmpich  
> -lpthread -lrt -limf -lsvml -lipgo -ldecimal -lirc -lgcc_s -lirc_s - 
> ldl
> ------------------------------------------
> #PETSc Option Table entries:
> -dump_data false
> -info
> -ksp_compute_singularvalues
> -ksp_rtol 1e-6
> -ksp_type preonly
> -ksp_view
> -log_summary
> -mat_ignore_lower_triangular
> -mat_mumps_icntl_14 200
> -mat_mumps_sym 1
> -mat_type sbaij
> -monitorname errormonitor.dat
> -nz 100
> -options_left
> -pc_factor_mat_solver_package mumps
> -pc_type cholesky
> -preload false
> -verb false
> #End o PETSc Option Table entries
> There are no unused options.
>
>                     1olarr
>
>              before sol put
>              sol put succes
>              before fullsol
> /DIANA/AP/LS41    12:08:39     35.66-CPU    11.16-IO   SOLVE
> /DIANA/AP/LS41    12:08:39     35.66-CPU    11.18-IO   FILLUP
> /DIANA/AP/LS41    12:08:39     35.66-CPU    11.18-IO   FILLUP
> /DIANA/AP/LS41    12:08:39     35.66-CPU    11.18-IO   POST
> /DIANA/AP/LS41    12:08:41     36.79-CPU    12.27-IO   POST
> /DIANA/AP/LS41    12:08:41     36.79-CPU    12.27-IO   LINSTA
> /DIANA/DC/END     12:08:41     36.79-CPU    12.27-IO   STOP
> DIANA JOB 3446 finished
>
> 1 proc externalsolver stuck at 100% with 12%memory out of 12GB
> 1 proc externalsolver at 0%
>
> mpiexec -np 2 externalsolver
> [0] PetscInitialize(): PETSc successfully started: number of  
> processors = 2
> [0] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [1] PetscInitialize(): PETSc successfully started: number of  
> processors = 2
> [1] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [1] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [1] PetscCommDuplicate():   returning tag 2147483647
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [0] PetscCommDuplicate():   returning tag 2147483647
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483646
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483646
> [0] MatSetUpPreallocation(): Warning not preallocating matrix storage
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [1] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [1] PetscCommDuplicate():   returning tag 2147483647
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483646
> [0] PetscCommDuplicate():   returning tag 2147483647
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483646
> [0] MatSetOption_MPISBAIJ(): Option SYMMETRIC ignored
> [0] MatStashScatterBegin_Private(): No of messages: 1
> [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 30544064
> [0] MatStashScatterBegin_Private(): No of messages: 0
> [0] MatAssemblyBegin_MPISBAIJ(): Stash has 3818007 entries,uses 8  
> mallocs.
> [0] MatAssemblyBegin_MPISBAIJ(): Block-Stash has 3818007 entries,  
> uses 8 mallocs.
> [0] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 221545 X 221545, block  
> size 1; storage space: 16676676 unneeded, 5477824 used
> [0] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [0] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 62
> [1] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 225367 X 225367, block  
> size 1; storage space: 16793278 unneeded, 5743422 used
> [1] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483645
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483644
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483643
> [0] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate():   returning tag 2147483634
> [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
> [0] VecScatterCreate(): General case: MPI to Seq
> [0] PetscCommDuplicate():   returning tag 2147483630
> [0] PetscCommDuplicate():   returning tag 2147483625
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483642
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483641
> [0] PetscCommDuplicate[1] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros  
> blocks in any row is 65
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483645
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483644
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483643
> [1] PetscCommDuplicate():   returning tag 2147483639
> [1] PetscCommDuplicate():   returning tag 2147483634
> [1] PetscCommDuplicate():   returning tag 2147483630
> [1] PetscCommDuplicate():   returning tag 2147483625
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483642
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483641
> [1] PetscCommDuplicate():   returning tag 2147483620
> [1] PetscC():   returning tag 2147483620
> [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
> [0] VecScatterCreate(): General case: MPI to MPI
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483640
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483638
> [0] MatAssemblyEnd_SeqBAIJ(): Matrix size: 221545 X 6275, block size  
> 1; storage space: 22060506 unneeded, 93994 used
> [0] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [0] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 32
> [0] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 216534)/ 
> (num_localrows 221545) > 0.6. Use CompressedRow routines.
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetsommDuplicate(): Using internal PETSc communicator 1140850689  
> -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483640
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483639
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483638
> [1] MatAssemblyEnd_SeqBAIJ(): Matrix size: 225367 X 0, block size 1;  
> storage space: 22536700 unneeded, 0 used
> [1] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [1] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 0
> [1] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 225367)/ 
> (num_localrows 225367) > 0.6. Use CompressedRow routines.
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483614
> [1] PetscCommDuplicate():   returning tag 2147483613
> [1]before KSPsetup
> [1] PetscCommDuplicate(): Using internal PcCommDuplicate():    
> returning tag 2147483614
> [0] PetscCommDuplicate():   returning tag 2147483613
> [0]before KSPsetup
> [0] PCSetUp(): Setting up new PC
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483637
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483636
> [0] PetscCommDuplicate():   returning tag 2147483612
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483635
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483634
> ETSc communicator 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483637
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483636
> [1] PetscCommDuplicate():   returning tag 2147483612
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483635
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483634
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483633
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483632
> [0] PetscCommDuplicate():   returning tag 2147483607
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483633
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483632
> [1] PetscCommDuplicate():   returning tag 2147483607
> [0] PetscCommDuplicate():   returning tag 2147483602
> [0] VecScatterCreate(): Special case: processor zero gets entire  
> parallel vector, rest get none
> [1] PetscCommDuplicate():   returning tag 2147483602
>
> DMUMPS 4.8.4
> L D L^T Solver for symmetric positive definite matrices
> Type of parallelism: Working host
>
> ****** ANALYSIS STEP ********
>
> Density: NBdense, Average, Median   =    0   49   50
> Ordering based on METIS
> ** Peak of sequential stack size (number of real entries)   :  
> 16763905.
> A root of estimated size         2965  has been selected for  
> Scalapack.
>
> Leaving analysis phase with  ...
> INFOG(1)                                       =               0
> INFOG(2)                                       =               0
> -- (20) Number of entries in factors (estim.) =       120657071
> --  (3) Storage of factors  (REAL, estimated) =       137364268
> --  (4) Storage of factors  (INT , estimated) =         6138804
> --  (5) Maximum frontal size      (estimated) =            3705
> --  (6) Number of nodes in the tree           =           21863
> --  (7) Ordering option effectively used      =               5
> ICNTL(6) Maximum transversal option            =               0
> ICNTL(7) Pivot order option                    =               7
> Percentage of memory relaxation (effective)    =             200
> Number of level 2 nodes                        =               1
> Number of split nodes                          =               0
> RINFO(1) Operations during elimination (estim) =   1.114D+11
> Distributed matrix entry format (ICNTL(18))    =               3
> ** Rank of proc needing largest memory in IC facto        :         0
> ** Estimated corresponding MBYTES for IC facto            :      2085
> ** Estimated avg. MBYTES per work. proc at facto (IC)     :      2044
> ** TOTAL     space in MBYTES for IC factorization         :      4088
> ** Rank of proc needing largest memory for OOC facto      :         1
> ** Estimated corresponding MBYTES for OOC facto           :       644
> ** Estimated avg. MBYTES per work. proc at facto (OOC)    :       630
> ** TOTAL     space in MBYTES for OOC factorization        :      1261
>
> ****** FACTORIZATION STEP ********
>
>
> GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
> NUMBER OF WORKING PROCESSES          =           2
> REAL SPACE FOR FACTORS               =   137364268
> INTEGER SPACE FOR FACTORS            =     6138804
> MAXIMUM FRONTAL SIZE (ESTIMATED)     =        3705
> NUMBER OF NODES IN THE TREE          =       21863
> Maximum effective relaxed size of S              =   239060000
> Average effective relaxed size of S              =   234725641
>
> REDISTRIB: TOTAL DATA LOCAL/SENT     =     9086109     2238491
> GLOBAL TIME FOR MATRIX DISTRIBUTION  =      0.8155
> ** Memory relaxation parameter ( ICNTL(14)  )            :       200
> ** Rank of processor needing largest memory in facto     :         0
> ** Space in MBYTES used by this processor for facto      :      2085
> ** Avg. Space in MBYTES per working proc during facto    :      2044
>
> ELAPSED TIME FOR FACTORIZATION       =     28.2595
> Maximum effective space used in S    (KEEP(67)   =    77516871
> Average effective space used in S    (KEEP(67)   =    75395654
> ** EFF Min: Rank of processor needing largest memory :         0
> ** EFF Min: Space in MBYTES used by this processor   :       741
> ** EFF Min: Avg. Space in MBYTES per working proc    :       720
>
> GLOBAL STATISTICS
> RINFOG(2)  OPERATIONS DURING NODE ASSEMBLY     = 2.565D+08
> ------(3)  OPERATIONS DURING NODE ELIMINATION  = 1.114D+11
> INFOG (9)  REAL SPACE FOR FACTORS              =   137364268
> INFOG(10)  INTEGER SPACE FOR FACTORS           =     6138809
> INFOG(11)  MAXIMUM FRONT SIZE                  =        3705
> INFOG(29)  NUMBER OF ENTRIES IN FACTORS        =   116259976
> INFOG(12) NB OF NEGATIVE PIVOTS          =           0
> INFOG(14)  NUMBER OF MEMORY COMPRESS           =           0
> [0]after KSPsetup
> [0]RHS setup
> [1]after KSPsetup
> [1]RHS setup
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483601
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483601
> [1] PetscCommDuplicate():   returning tag 2147483596
> [0] PetscCommDuplicate():   returning tag 2147483596
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483591
> [0] PetscCommDuplicate():   returning tag 2147483591
> [0] VecAssemblyBegin_MPI(): Stash has 225367 entries, uses 12 mallocs.
> [0] VecAssemblyBegin_MPI(): Block-Stash has 0 entries, uses 0 mallocs.
> [0] PetscCommDuplicate():   returning tag 2147483590
> [1] PetscCommDuplicate():   returning tag 2147483590
> [1] PetscCommDuplicate():   returning tag 2147483589
> [0] PetscCommDuplicate():   returning tag 2147483589
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483631
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483630
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483631
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483630
>
> 4 processes of externalsolver stuck with 0% cpu
>
>
> mpiexec -np 4 externalsolver
> [1] PetscInitialize(): PETSc successfully started: number of  
> processors = 4
> [1] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [3] PetscInitialize(): PETSc successfully started: number of  
> processors = 4
> [3] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [0] PetscInitialize(): PETSc successfully started: number of  
> processors = 4
> [0] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [2] PetscInitialize(): PETSc successfully started: number of  
> processors = 4
> [2] PetscInitialize(): Running on machine: mat1.uibk.ac.at
> [3] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [3] PetscCommDuplicate():   returning tag 2147483647
> [1] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [1] PetscCommDuplicate():   returning tag 2147483647
> [2] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [2] PetscCommDuplicate():   returning tag 2147483647
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [2] PetscCommDuplicate():   returning tag 2147483646
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [3] PetscCommDuplicate():   returning tag 2147483646
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850688  
> -2080374784 max tags = 2147483647
> [0] PetscCommDuplicate():   returning tag 2147483647
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483646
> [0] MatSetUpPreallocation(): Warning not preallocating matrix storage
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483646
> [1] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [3] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [3] PetscCommDuplicate():   returning tag 2147483647
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483646
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [0] PetscCommDuplicate():   returning tag 2147483647
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483646
> [1] PetscCommDuplicate():   returning tag 2147483647
> [2] PetscCommDuplicate(): Duplicating a communicator 1140850689  
> -2080374783 max tags = 2147483647
> [2] PetscCommDuplicate():   returning tag 2147483647
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483646
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483646
> [0] MatSetOption_MPISBAIJ(): Option SYMMETRIC ignored
> [0] MatStashScatterBegin_Private(): No of messages: 3
> [0] MatStashScatterBegin_Private(): Mesg_to: 1: size: 11261544
> [0] MatStashScatterBegin_Private(): Mesg_to: 2: size: 10384840
> [0] MatStashScatterBegin_Private(): Mesg_to: 3: size: 393344
> [0] MatStashScatterBegin_Private(): No of messages: 0
> [0] MatAssemblyBegin_MPISBAIJ(): Stash has 2754963 entries,uses 8  
> mallocs.
> [0] MatAssemblyBegin_MPISBAIJ(): Block-Stash has 2754963 entries,  
> uses 8 mallocs.
> [3] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 112413 X 112413, block  
> size 1; storage space: 8404901 unneeded, 2836399 used
> [3] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [3] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 65
> [1] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 113299 X 113299, block  
> size 1; storage space: 8647559 unneeded, 2682341 used
> [1] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [1] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 62
> [0] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 112871 X 112871, block  
> size 1; storage space: 8670798 unneeded, 2616302 used
> [0] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [0] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 52
> [2] MatAssemblyEnd_SeqSBAIJ(): Matrix size: 108329 X 108329, block  
> size 1; storage space: 8247737 unneeded, 2585163 used
> [2] MatAssemblyEnd_SeqSBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [2] MatAssemblyEnd_SeqSBAIJ(): Most nonzeros blocks in any row is 52
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483645
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483644
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483643
> [1] PetscCommDuplicate():   returning tag 2147483639
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483645
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483644
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483643
> [3] PetscCommDuplicate():   returning tag 2147483639
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483645
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483644
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483643
> [2] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483645
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483644
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483643
> [0] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate():   returning tag 2147483634
> [2] PetscCommDuplicate():   returning tag 2147483634
> [1] PetscCommDuplicate():   returning tag 2147483634
> [3] PetscCommDuplicate():   returning tag 2147483634
> [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
> [0] VecScatterCreate(): General case: MPI to Seq
> [0] PetscCommDuplicate():   returning tag 2147483630
> [0] PetscCommDuplicate():   returning tag 2147483625
> [1] PetscCommDuplicate():   returning tag 2147483630
> [1] PetscCommDuplicate():   returning tag 2147483625
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483642
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483641
> [1] PetscCommDuplicate():   returning tag 2147483620
> [2] PetscCommDuplicate():   returning tag 2147483630
> [2] PetscCommDuplicate():   returning tag 2147483625
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483642
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483641
> [2] PetscCommDuplicate():   returning tag 2147483620
> [3] PetscCommDuplicate():   returning tag 2147483630
> [3] PetscCommDuplicate():   returning tag 2147483625
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483642
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483641
> [3] PetscCommDuplicate():   returning tag 2147483620
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483640
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483639
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483638
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483642
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483641
> [0] PetscCommDuplicate():   returning tag 2147483620
> [0] VecScatterCreateCommon_PtoS(): Using blocksize 1 scatter
> [0] VecScatterCreate(): General case: MPI to MPI
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483640
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483639
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483638
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483640
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483639
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483638
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483640
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483639
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483638
> [3] MatAssemblyEnd_SeqBAIJ(): Matrix size: 112413 X 0, block size 1;  
> storage space: 11241300 unneeded, 0 used
> [3] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [3] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 0
> [3] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 112413)/ 
> (num_localrows 112413) > 0.6. Use CompressedRow routines.
> [1] MatAssemblyEnd_SeqBAIJ(): Matrix size: 113299 X 2952, block size  
> 1; storage space: 11286236 unneeded, 43664 used
> [1] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [1] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 29
> [1] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 110902)/ 
> (num_localrows 113299) > 0.6. Use CompressedRow routines.
> [2] MatAssemblyEnd_SeqBAIJ(): Matrix size: 108329 X 9258, block size  
> 1; storage space: 10624060 unneeded, 208840 used
> [2] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [2] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 43
> [2] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 92865)/ 
> (num_localrows 108329) > 0.6. Use CompressedRow routines.
> [0] MatAssemblyEnd_SeqBAIJ(): Matrix size: 112871 X 17849, block  
> size 1; storage space: 10944569 unneeded, 342531 used
> [0] MatAssemblyEnd_SeqBAIJ(): Number of mallocs during MatSetValues  
> is 0
> [0] MatAssemblyEnd_SeqBAIJ(): Most nonzeros blocks in any row is 43
> [0] Mat_CheckCompressedRow(): Found the ratio (num_zerorows 92260)/ 
> (num_localrows 112871) > 0.6. Use CompressedRow routines.
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [0] PetscCommDuplicate():   returning tag 2147483614
> [0] PetscCommDuplicate():   returning tag 2147483613
> [0]before KSPsetup
> [0] PCSetUp(): Setting up new PC
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483637
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483636
> [0] PetscCommDuplicate():   returning tag 2147483612
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [1] PetscCommDuplicate():   returning tag 2147483614
> [1] PetscCommDuplicate():   returning tag 2147483613
> [1]before KSPsetup
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483637
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483636
> [1] PetscCommDuplicate():   returning tag 2147483612
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [2] PetscCommDuplicate():   returning tag 2147483614
> [2] PetscCommDuplicate():   returning tag 2147483613
> [2]before KSPsetup
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483637
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483636
> [2] PetscCommDuplicate():   returning tag 2147483612
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483635
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483634
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483635
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483634
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850688 -2080374784
> [3] PetscCommDuplicate():   returning tag 2147483614
> [3] PetscCommDuplicate():   returning tag 2147483613
> [3]before KSPsetup
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483637
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483636
> [3] PetscCommDuplicate():   returning tag 2147483612
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483635
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483634
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483635
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483634
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483633
> [1] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [1] PetscCommDuplicate():   returning tag 2147483632
> [1] PetscCommDuplicate():   returning tag 2147483607
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483633
> [2] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [2] PetscCommDuplicate():   returning tag 2147483632
> [2] PetscCommDuplicate():   returning tag 2147483607
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483633
> [0] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [0] PetscCommDuplicate():   returning tag 2147483632
> [0] PetscCommDuplicate():   returning tag 2147483607
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483633
> [3] PetscCommDuplicate(): Using internal PETSc communicator  
> 1140850689 -2080374783
> [3] PetscCommDuplicate():   returning tag 2147483632
> [3] PetscCommDuplicate():   returning tag 2147483607
> [0] PetscCommDuplicate():   returning tag 2147483602
> [1] PetscCommDuplicate():   returning tag 2147483602
> [2] PetscCommDuplicate():   returning tag 2147483602
> [3] PetscCommDuplicate():   returning tag 2147483602
> [0] VecScatterCreate(): Special case: processor zero gets entire  
> parallel vector, rest get none
>
> DMUMPS 4.8.4
> L D L^T Solver for symmetric positive definite matrices
> Type of parallelism: Working host
>
> ****** ANALYSIS STEP ********
>
> Density: NBdense, Average, Median   =    0   49   50
> Ordering based on METIS
> ** Peak of sequential stack size (number of real entries)   :  
> 16763905.
> A root of estimated size         2965  has been selected for  
> Scalapack.
>
> Leaving analysis phase with  ...
> INFOG(1)                                       =               0
> INFOG(2)                                       =               0
> -- (20) Number of entries in factors (estim.) =       120657071
> --  (3) Storage of factors  (REAL, estimated) =       137362626
> --  (4) Storage of factors  (INT , estimated) =         6167135
> --  (5) Maximum frontal size      (estimated) =            3705
> --  (6) Number of nodes in the tree           =           21863
> --  (7) Ordering option effectively used      =               5
> ICNTL(6) Maximum transversal option            =               0
> ICNTL(7) Pivot order option                    =               7
> Percentage of memory relaxation (effective)    =             200
> Number of level 2 nodes                        =               5
> Number of split nodes                          =               0
> RINFO(1) Operations during elimination (estim) =   1.114D+11
> Distributed matrix entry format (ICNTL(18))    =               3
> ** Rank of proc needing largest memory in IC facto        :         0
> ** Estimated corresponding MBYTES for IC facto            :      1112
> ** Estimated avg. MBYTES per work. proc at facto (IC)     :      1083
> ** TOTAL     space in MBYTES for IC factorization         :      4333
> ** Rank of proc needing largest memory for OOC facto      :         1
> ** Estimated corresponding MBYTES for OOC facto           :       465
> ** Estimated avg. MBYTES per work. proc at facto (OOC)    :       421
> ** TOTAL     space in MBYTES for OOC factorization        :      1684
>
> ****** FACTORIZATION STEP ********
>
>
> GLOBAL STATISTICS PRIOR NUMERICAL FACTORIZATION ...
> NUMBER OF WORKING PROCESSES          =           4
> REAL SPACE FOR FACTORS               =   137362626
> INTEGER SPACE FOR FACTORS            =     6167135
> MAXIMUM FRONTAL SIZE (ESTIMATED)     =        3705
> NUMBER OF NODES IN THE TREE          =       21863
> Maximum effective relaxed size of S              =   125678965
> Average effective relaxed size of S              =   122625031
>
> REDISTRIB: TOTAL DATA LOCAL/SENT     =     2497290     8939630
> GLOBAL TIME FOR MATRIX DISTRIBUTION  =      0.6324
> ** Memory relaxation parameter ( ICNTL(14)  )            :       200
> ** Rank of processor needing largest memory in facto     :         0
> ** Space in MBYTES used by this processor for facto      :      1112
> ** Avg. Space in MBYTES per working proc during facto    :      1083
>



More information about the petsc-users mailing list