Slow assembly

[Satish Balay]

If matassembly takes a long time - it usually means that lots of
values are set on the wrong processor [they get communicated to the
correct processor during assembly]. But several hours is a too long
for this operation [so I don't understand whats hapenning here]

Can you run the code with the option '-info' - and then send us this
log at petsc-maint at mcs.anl.gov

Satish

On Fri, 26 Oct 2007, John R. Wicks wrote:

> I have confirmed that I am calling MatSetValues() for local rows only and am
> only setting each value exactly once.
> 
> Because of how the matrix was partitioned for another non-Petsc program,
> each partition is partitioned (by columns) into 32 blocks (corresponding to
> the row partitions).  I enter the data for each block one row at a time,
> i.e., for any one SetValues call, the entries are sorted by increasing
> column index.  Does that mean I can use
> MatrixSetOption(A,MAT_COLUMNS_SORTED).  Should that help?  
> 
> P.S.: I tried it, and it still seems to be taking quite a long time.
> 
> > -----Original Message-----
> > From: owner-petsc-users at mcs.anl.gov 
> > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay
> > Sent: Friday, October 26, 2007 3:04 PM
> > To: petsc-users at mcs.anl.gov
> > Subject: Re: Slow assembly
> > 
> > 
> > On Fri, 26 Oct 2007, John R. Wicks wrote:
> > 
> > > I am working on computing PageRank for a web scale graph 
> > which uses a 
> > > square matrix which is 1.2x10^8 dimensional with about 10^9 
> > entries.  
> > > I have partitioned the matrix for 32 processors myself into my own 
> > > ascii format, and I know the memory allocation, so I:
> > > 
> > > 1) create the matrix with "A = MatCreateMPIAIJ(*n, *n, *N, 
> > *N, 0, nnz, 
> > > 0, onnz)",
> > > 2) load the entries by repeatedly calling 
> > > "MatSetValues(A,1,&row,links,cols,vals,INSERT_VALUES)", and
> > > 
> > > 3) call MatAssemblyBegin/End.
> > > 
> > > Steps 1 and 2 complete in a couple minutes, but step 3 is taking 
> > > several hours.  What is going on?  Is there a way to speed 
> > up matrix 
> > > assembly?
> > 
> > Are you makeing sure that you call MatGetOwnershipRange() - 
> > and calling MatSetValues() for mostly local rows only?
> > 
> > Also can you confirm that multiple processes [for eg: proc-0 
> > and proc-1 etc..]  are not setting the same value [i.e both 
> > of them calling MatSetValues(row=0,col=0)]
> > 
> > Satish
> > 
> 
>