Slow assembly

[Satish Balay]

On Fri, 26 Oct 2007, John R. Wicks wrote:

> I am working on computing PageRank for a web scale graph which uses a square
> matrix which is 1.2x10^8 dimensional with about 10^9 entries.  I have
> partitioned the matrix for 32 processors myself into my own ascii format,
> and I know the memory allocation, so I:
> 
> 1) create the matrix with "A = MatCreateMPIAIJ(*n, *n, *N, *N, 0, nnz, 0,
> onnz)", 
> 2) load the entries by repeatedly calling
> "MatSetValues(A,1,&row,links,cols,vals,INSERT_VALUES)", and 
> 
> 3) call MatAssemblyBegin/End.
> 
> Steps 1 and 2 complete in a couple minutes, but step 3 is taking several
> hours.  What is going on?  Is there a way to speed up matrix assembly?

Are you makeing sure that you call MatGetOwnershipRange() - and
calling MatSetValues() for mostly local rows only?

Also can you confirm that multiple processes [for eg: proc-0 and
proc-1 etc..]  are not setting the same value [i.e both of them
calling MatSetValues(row=0,col=0)]

Satish