Slow assembly

[John R. Wicks]

I have confirmed that I am calling MatSetValues() for local rows only and am
only setting each value exactly once.

Because of how the matrix was partitioned for another non-Petsc program,
each partition is partitioned (by columns) into 32 blocks (corresponding to
the row partitions).  I enter the data for each block one row at a time,
i.e., for any one SetValues call, the entries are sorted by increasing
column index.  Does that mean I can use
MatrixSetOption(A,MAT_COLUMNS_SORTED).  Should that help?  

P.S.: I tried it, and it still seems to be taking quite a long time.

> -----Original Message-----
> From: owner-petsc-users at mcs.anl.gov 
> [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay
> Sent: Friday, October 26, 2007 3:04 PM
> To: petsc-users at mcs.anl.gov
> Subject: Re: Slow assembly
> 
> 
> On Fri, 26 Oct 2007, John R. Wicks wrote:
> 
> > I am working on computing PageRank for a web scale graph 
> which uses a 
> > square matrix which is 1.2x10^8 dimensional with about 10^9 
> entries.  
> > I have partitioned the matrix for 32 processors myself into my own 
> > ascii format, and I know the memory allocation, so I:
> > 
> > 1) create the matrix with "A = MatCreateMPIAIJ(*n, *n, *N, 
> *N, 0, nnz, 
> > 0, onnz)",
> > 2) load the entries by repeatedly calling 
> > "MatSetValues(A,1,&row,links,cols,vals,INSERT_VALUES)", and
> > 
> > 3) call MatAssemblyBegin/End.
> > 
> > Steps 1 and 2 complete in a couple minutes, but step 3 is taking 
> > several hours.  What is going on?  Is there a way to speed 
> up matrix 
> > assembly?
> 
> Are you makeing sure that you call MatGetOwnershipRange() - 
> and calling MatSetValues() for mostly local rows only?
> 
> Also can you confirm that multiple processes [for eg: proc-0 
> and proc-1 etc..]  are not setting the same value [i.e both 
> of them calling MatSetValues(row=0,col=0)]
> 
> Satish
>