<div dir="ltr">On Thu, Mar 31, 2011 at 12:38 PM, gouarin <span dir="ltr"><<a href="mailto:loic.gouarin@math.u-psud.fr">loic.gouarin@math.u-psud.fr</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div bgcolor="#ffffff" text="#000000">
On <a href="tel:31%2F03%2F2011%2019" target="_blank">31/03/2011 19</a>:06, Matthew Knepley wrote:
<blockquote type="cite">
<div>On Thu, Mar 31, 2011 at 10:10 AM, gouarin <span><<a href="mailto:loic.gouarin@math.u-psud.fr" target="_blank">loic.gouarin@math.u-psud.fr</a>></span>
wrote:<br>
<div>
<blockquote>
Thanks for the reply.
<div>
<div><br>
<br>
On <a href="tel:31%2F03%2F2011%2016" target="_blank">31/03/2011 16</a>:48,
Hong Zhang wrote:<br>
<blockquote>
Loic,<br>
<br>
<blockquote>
I want to do the same for MATMPI. This is what I do:<br>
<br>
MatGetLocalMat(A,MAT_INITIAL_MATRIX,&Aloc);<br>
MatGetRowIJ(Aloc, 1, PETSC_FALSE,
PETSC_FALSE,&(Acsr.n),&(Acsr.ia),<br>
&(Acsr.ja),&done);<br>
MatGetArray(Aloc,&(Acsr.a));<br>
<br>
Now, I need to know on what process the non local
points are.<br>
</blockquote>
Each process get its own local submatrix. See<br>
<a href="http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatCreateMPIAIJ.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatCreateMPIAIJ.html</a><br>
on data structure of MPIAIJ format. Applying your
statements above to<br>
the example listed on the page (a 8x8 matrix), Proc0
will get<br>
Aloc=A(0:2,:), ..., Porc2 gets Aloc=A(6:7,:).<br>
<blockquote>
Is there a petsc function that gives this partition
?<br>
</blockquote>
MatGetOwnershipRange()<br>
</blockquote>
</div>
</div>
I use this function to know the local points index. In the
same way, with this function I know the non local index.<br>
<br>
But this function doesn't give the process. I give you an
example. I have this matrix:<br>
<br>
1 0 | 0 1<br>
1 2 | 1 0<br>
-----------<br>
0 0 | 1 0<br>
0 2 | 0 1<br>
<br>
I want this kind of format<br>
<br>
for proc 0<br>
a = {1,1,1,2,1}<br>
ia = {0,2,5}<br>
ja = {0,3,0,1,2}<br>
rank = {0,0,1,1}<br>
<br>
for proc 1<br>
a = {1,2,1}<br>
ia = {0,1,3}<br>
ja = {0,2,1}<br>
rank = {1,1,0}<br>
<br>
rank give me the process for each non zero column. Local
rows should have numbers 1,...N and offdiagonal entries with
j>N.<br>
</blockquote>
<div><br>
</div>
<div>The ownership range is the row partition. However, MPIAIJ
matrices are not stored as you indicate above.</div>
<div><br>
</div>
<div>Different storage formats make the process you describe
fragile. WHy do you need the CSR form of the matrix? If you
really</div>
<div>do, use MatGetSubMatrix() to get a SEQAIJ matrix on each
process and pull out the values.</div>
<div><br>
</div>
</div>
</div>
</blockquote>
I want to use this format because I try to create a new multigrid PC
with this method:<br>
<br>
<a href="http://homepages.ulb.ac.be/~ynotay/AGMG/userguide/index.html" target="_blank">http://homepages.ulb.ac.be/~ynotay/AGMG/userguide/index.html</a><br>
<br>
I don't understand why it's not possible to have the process
partition. When you create a MPIAIJ matrix and you use a KSP to
solve the system, you know what values to exchange. No? Where is
this information ?<br></div></blockquote><div><br></div><div>1) It is possible to get the partition of rows across processes. This is EXACTLY what MatGetOwnershipRange() returns, as I said in the last email.</div><div>
<br></div><div>2) As I explained, matrices can be stored in a variety of formats. The MPIAIJ format stores 2 local matrices per partition in order to overlap</div><div>communication with computation.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div bgcolor="#ffffff" text="#000000">
I can also use MatGetOwnershipRange() and MPI_Allgather to have this
information.<br></div></blockquote><div><br></div><div><a href="http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRanges.html">http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRanges.html</a></div>
<div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div bgcolor="#ffffff" text="#000000">
Loic<br>
<br>
<br>
<br>
<br>
<blockquote type="cite">
<div>
<div>
<div> Matt</div>
<div> </div>
<blockquote>
I suppose that I have no information about the structure of
MPIAIJ.<br>
<br>
Loic<br>
<br>
</blockquote>
</div>
<br>
<br>
<br>
-- <br>
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to
which their experiments lead.<br>
-- Norbert Wiener<br>
</div>
</blockquote>
<br>
<br>
<pre cols="72">--
Loic Gouarin
Laboratoire de Mathématiques
Université Paris-Sud
Bâtiment 425
91405 Orsay Cedex
France
Tel: <a href="tel:%28%2B33%29%201%2069%2015%2060%2014" target="_blank">(+33) 1 69 15 60 14</a>
Fax: <a href="tel:%28%2B33%29%201%2069%2015%2067%2018" target="_blank">(+33) 1 69 15 67 18</a></pre>
</div>
</blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<br>
</div>