<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hei,</p>
<p>I am currently struggling a bit with my understanding of local
and global rows/columns in distributed dense matrices in PETSc. <br>
</p>
<p>The attached short program is creating a local matrix in every
thread with [matrix_size_rows, matrix_size_cols] as global size.
Afterwards, I flatten the matrix into a vector, and split this
vector using the boost-function split() when running using several
MPI threads. In addition I create a dense MPI-distributed matrix
based on PETSc, using the same input sizes, and set all values to
zero (after I'm not interested in the content itself, just the
shape). Finally, I retrieve the global and local values for the
number of rows and columns using MatGetLocalSize, and the number
of the first and last row in each matrix using
MatGetOwnershipRange() and print them. According to the
documentation I would expect that the PETSc-based matrix is split
up <br>
</p>
<p>Now, for an input size of [8, 8] and a single thread I get <br>
</p>
<p><i>Rank 0 has a total size of 8, 8<br>
Rank 0 has an initial size of 64<br>
Rank 0 has a local matrix size of 8, 8<br>
Rank 0 has a global matrix size of 8, 8<br>
Rank 0 has a matrix spanning from row 0 to 8<br>
Rank 0 has a vector size of 64</i><i></i></p>
<p>which is what I expect. For two threads, though, I get <br>
</p>
<i>Rank 0 has a total size of 8, 8</i><i><br>
</i><i>Rank 0 has an initial size of 64</i><i><br>
</i><i>Rank 0 has a local matrix size of 4, 4</i><i><br>
</i><i>Rank 0 has a global matrix size of 8, 8</i><i><br>
</i><i>Rank 0 has a matrix spanning from row 0 to 4</i><i><br>
</i><i>Rank 0 has a vector size of 32</i><i><br>
</i><i>Rank 1 has a total size of 8, 8</i><i><br>
</i><i>Rank 1 has an initial size of 64</i><i><br>
</i><i>Rank 1 has a local matrix size of 4, 4</i><i><br>
</i><i>Rank 1 has a global matrix size of 8, 8</i><i><br>
</i><i>Rank 1 has a matrix spanning from row 4 to 8</i><i><br>
Rank 1 has a vector size of 32</i>
<p>Here, most entries make sense, except the size of the local
matrices. Why do I get a size of [4, 4], and not a size of [4, 8]?
Each row should be contiguous in the local process, and therefore
each row should contain all columns, not only a part of it. <br>
</p>
<p>Is there a misunderstanding about how to use MatGetLocalSize(),
or something else?</p>
<p>Thanks!</p>
<p>Regards,</p>
<p>Roland Richter<br>
</p>
<p><i><br>
</i></p>
</body>
</html>