[petsc-users] Customizeing MatSetValuesBlocked(...)

[Jinquan Zhong]

Satish,

I looked at
	http://www.mcs.anl.gov/petsc/petsc-current/src/snes/examples/tutorials/ex5s.c.html
where 
	MatCreateAIJ(PETSC_COMM_WORLD,rend-rstart,rend-rstart,N,N,5,0,0,0,&J);

was used to create J (N,N) with local J dimensioned as (rend-rstart,rend-rstart).

For my application, I have LDA=LDB=3 and N=9 in the following 

	ierr = MatCreateAIJ(PETSC_COMM_WORLD,LDA,LDB,N,N,LDA,0,0,0,&A);	CHKERRQ(ierr);

I got the following error message:

[0]PETSC ERROR: --------------------- Error Message ------------------------------------
[0]PETSC ERROR: Nonconforming object sizes!
[0]PETSC ERROR: Sum of local lengths 27 does not equal global length 9, my local length 3
  likely a call to VecSetSizes() or MatSetSizes() is wrong.


It does NOT appear PETSc  had the 9x9 matrix. It tried to build 27*27 since I am using 9 procs.  Is there a way to resolve this local size and global size issue?

Thanks,

Jinquan



-----Original Message-----
From: petsc-users-bounces at mcs.anl.gov [mailto:petsc-users-bounces at mcs.anl.gov] On Behalf Of Satish Balay
Sent: Wednesday, August 08, 2012 1:45 PM
To: PETSc users list
Subject: Re: [petsc-users] Customizeing MatSetValuesBlocked(...)

On Wed, 8 Aug 2012, Jed Brown wrote:

> On Wed, Aug 8, 2012 at 2:25 PM, Jinquan Zhong <jzhong at scsolutions.com>wrote:
> 
> > If I understand you correctly, ScaLAPACK blocks don't have anything 
> > to do with the sparse matrix structure.****
> >
> > ** **
> >
> >
> > ********************************************************************
> > ********************************************************************
> > **
> > ****
> >
> > ** **
> >
> > You are correct.  What I meant was how to define the diagonal and 
> > off-diagonal parts of each submatrix matrix  A (LDA, LDB).  For 
> > example, in the following matrix,****
> >
> > ** **
> >
> >                                   *Proc0* *Proc1*       *Proc2*****
> >
> >             1  2  0  |  0  3  0  |  0  4****
> >
> >     *Proc0*   0  5  6  |  7  0  0  |  8  0****
> >
> >             9  0 10  | 11  0  0  | 12  0****
> >
> >     -------------------------------------****
> >
> >            13  0 14  | 15 16 17  |  0  0****
> >
> >     *Proc3*   0 18  0  | 19 20 21  |  0  0      <=== owned by  Proc 5****
> >
> >             0  0  0  | 22 23  0  | 24  0****
> >
> >     -------------------------------------****
> >
> >     *Proc6*  25 26 27  |  0  0 28  | 29  0****
> >
> >                      30  0  0  | 31 32 33  |  0  34****
> >
> > ** **
> >
> > I am not sure how to fill out the values for d_nz, d_nnz,o_nz, o_nnz 
> > peoperly for the subblock (0 0 ;0 0; 24 0) owned by Proc 5 since it 
> > was based on diagonal and off-diagonal parts.
> >
> 
> Throw your 2D block cyclic nonsense out the window. These are sparse 
> matrices and that layout would be terrible. Logically permute your 
> matrices all you want, then define a global ordering and chunk it into 
> contiguous blocks of rows (no partition of columns). Work this out 
> with a pencil and paper. You should have a function that translates 
> row/column pairs from your ordering to our ordering. Now compute the 
> sparsity pattern in the new ordering. (Usually you can figure this out 
> on paper as well.) Then preallocate and call MatSetValues() with the new (row,column) locations.
> 

also check the 'example usage' at:

http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateAIJ.html

Satish