Slow assembly

[Barry Smith]

  The sorting does not matter.

  Under normal conditions the MatAssembly should take a fraction of a second.
The only cause that we know that slows it down to the extreme you have is that
it is sending a huge amount of data across processes (the -info option Satish 
suggested will tell us if that is true). 

  Are you only call MatAssemblyBegin/End() once? You should, don't call it 
multiple times. 

  The sorting is not important (in fact it takes advantage of it automatically
and does not need to be set).

   Barry


On Fri, 26 Oct 2007, John R. Wicks wrote:

> I have confirmed that I am calling MatSetValues() for local rows only and am
> only setting each value exactly once.
> 
> Because of how the matrix was partitioned for another non-Petsc program,
> each partition is partitioned (by columns) into 32 blocks (corresponding to
> the row partitions).  I enter the data for each block one row at a time,
> i.e., for any one SetValues call, the entries are sorted by increasing
> column index.  Does that mean I can use
> MatrixSetOption(A,MAT_COLUMNS_SORTED).  Should that help?  
> 
> P.S.: I tried it, and it still seems to be taking quite a long time.
> 
> > -----Original Message-----
> > From: owner-petsc-users at mcs.anl.gov 
> > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay
> > Sent: Friday, October 26, 2007 3:04 PM
> > To: petsc-users at mcs.anl.gov
> > Subject: Re: Slow assembly
> > 
> > 
> > On Fri, 26 Oct 2007, John R. Wicks wrote:
> > 
> > > I am working on computing PageRank for a web scale graph 
> > which uses a 
> > > square matrix which is 1.2x10^8 dimensional with about 10^9 
> > entries.  
> > > I have partitioned the matrix for 32 processors myself into my own 
> > > ascii format, and I know the memory allocation, so I:
> > > 
> > > 1) create the matrix with "A = MatCreateMPIAIJ(*n, *n, *N, 
> > *N, 0, nnz, 
> > > 0, onnz)",
> > > 2) load the entries by repeatedly calling 
> > > "MatSetValues(A,1,&row,links,cols,vals,INSERT_VALUES)", and
> > > 
> > > 3) call MatAssemblyBegin/End.
> > > 
> > > Steps 1 and 2 complete in a couple minutes, but step 3 is taking 
> > > several hours.  What is going on?  Is there a way to speed 
> > up matrix 
> > > assembly?
> > 
> > Are you makeing sure that you call MatGetOwnershipRange() - 
> > and calling MatSetValues() for mostly local rows only?
> > 
> > Also can you confirm that multiple processes [for eg: proc-0 
> > and proc-1 etc..]  are not setting the same value [i.e both 
> > of them calling MatSetValues(row=0,col=0)]
> > 
> > Satish
> > 
> 
>