<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Sun, Mar 29, 2015 at 9:56 PM, Steena M <span dir="ltr"><<a href="mailto:stm8086@yahoo.com" target="_blank">stm8086@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Barry,<br>
<br>
I am trying to partition a 20 row and 20 col sparse matrix between two procs such that proc 0 has 15 rows and 20 cols and proc 1 has 5 rows and 20 cols. The code snippet:<br>
<br>
<br>
<br>
CHKERRQ(MatCreate(PETSC_COMM_WORLD,&A)); // at runtime: -matload_block_size 1<br>
<br>
<br>
if (rank ==0)<br>
{<br>
CHKERRQ( MatSetSizes(A, 15, 20, 20, 20) ); //rank 0 gets 75% of the rows<br>
CHKERRQ( MatSetType(A, MATMPIBAIJ) );<br>
CHKERRQ( MatLoad(A,fd) );<br>
}<br>
<br>
else<br>
{<br>
CHKERRQ( MatSetSizes(A, 5, 20, 20, 20) ); //rank 1 gets 25% of the rows<br>
CHKERRQ( MatSetType(A, MATMPIBAIJ) );<br>
CHKERRQ( MatLoad(A,fd) );<br>
}<br>
<br>
This throws the following error (probably from psplit.c):<br>
[1]PETSC ERROR: --------------------- Error Message ------------------------------------<br>
[1]PETSC ERROR: Nonconforming object sizes!<br>
[1]PETSC ERROR: Sum of local lengths 40 does not equal global length 20, my local length 20<br>
likely a call to VecSetSizes() or MatSetSizes() is wrong.<br>
See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#split" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#split</a>!<br>
<br>
This error printout doesn't quite make sense to me. I'm trying to specify a total matrix size of 20x20... I haven't yet figured out where the '40' comes from in the error message.<br>
<br>
Any thoughts on what might be going wrong?<br></blockquote><div><br></div><div>Its the column specification. Just use PETSC_DETERMINE for the local columns since all our sparse matrix</div><div>formats are row divisions anyway.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks in advance,<br>
Steena<br>
<br>
<br>
<br>
--------------------------------------------<br>
On Sun, 3/22/15, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
<br>
Subject: Re: [petsc-users] Unequal sparse matrix row distribution for MPI MatMult<br>
To: "Steena M" <<a href="mailto:stm8086@yahoo.com">stm8086@yahoo.com</a>><br>
Cc: <a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><br>
Date: Sunday, March 22, 2015, 3:58 PM<br>
<br>
<br>
<br>
Steena,<br>
<br>
I am<br>
a little unsure of your question. <br>
<br>
1) you can create a MPIBAIJ<br>
matrix with any distribution of block rows per process you<br>
want, just set the local row size for each process to be<br>
what you like. Use MatCreateVecs() to get correspondingly<br>
laid out vectors.<br>
<br>
or 2) if you have a MPIBAIJ<br>
matrix with "equal" row layout and you want a new<br>
one with uneven row layout you can simply use<br>
MatGetSubMatrix() to create that new matrix.<br>
<br>
Barry<br>
<br>
Unless you have another reason to have the<br>
matrix with an equal number row layout I would just generate<br>
the matrix with the layout you want.<br>
<br>
<br>
> On Mar 22, 2015, at 5:50 PM, Steena M<br>
<<a href="mailto:stm8086@yahoo.com">stm8086@yahoo.com</a>><br>
wrote:<br>
><br>
> Hello,<br>
><br>
> I need to distribute<br>
a sparse matrix such that each proc owns an unequal number<br>
of blocked rows before I proceed with MPI MatMult. My<br>
initial thoughts on doing this:<br>
><br>
> 1) Use MatGetSubMatrices() on the test<br>
MATMPIBAIJ matrix to produce a new matrix where each proc<br>
has an unequal number of rows.<br>
><br>
> 2) Provide scatter context for vector X<br>
(for MatMult )using IS iscol from MatGetSubMatrices() while<br>
creating the vector X.<br>
><br>
> 3) Call MatMult()<br>
><br>
> Will MatMult_MPIBAIJ continue to scatter<br>
this matrix and vector such that each proc will own an equal<br>
number of matrix rows and corresponding diagonal vector<br>
elements? Should I write my own MPIMatMult function to<br>
retain my redistribution of the matrix and vector?<br>
><br>
> Thanks in<br>
advance,<br>
> Steena<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>