<div dir="ltr"><div><div>Dear <span style class="">PETSc</span> users and developers,<br></div>I am assembling a PC matrix in parallel. For instance, with just one element in the mesh the structure of the matrix is<br><br>
</div> <font><span style="color:rgb(0,0,255)"><b>[5x5 nonzero] [5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</b></span> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]<b> <span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b><span style="color:rgb(0,0,255)"> </span>[5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]</font><font size="1"><font><b><span style="color:rgb(0,0,255)"><br>
[5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</span> </b>[5x5 <span style class="">ZEROs</span>] <span style="color:rgb(0,0,255)"><b>[5x5 nonzero]</b> </span> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]<b> <span style="color:rgb(0,0,255)">[5x5 nonzero] </span></b>[5x5 <span style class="">ZEROs</span>]<br>
<b><span style="color:rgb(0,0,255)">[5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>] <span style="color:rgb(0,0,255)"><b>[5x5 nonzero]</b></span><span style="color:rgb(0,0,255)"></span> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>] <b> <span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b><br>
<b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> [5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</span></b><b><span style="color:rgb(0,0,255)"> [5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]<br>
[5x5 <span style class="">ZEROs</span>] <b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> </span></b><b><span style="color:rgb(0,0,255)"> [5x5 nonzero] [5x5 nonzero] [5x5 nonzero] </span></b>[5x5 <span style class="">ZEROs</span>] <b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>]<br>
[5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> [5x5 nonzero]</span></b><b><span style="color:rgb(0,0,255)"> </span></b><b><span style="color:rgb(0,0,255)"> [5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>] <b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b><br>
<b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> [5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> </span></b><b><span style="color:rgb(0,0,255)"> [5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</span></b><br>
[5x5 <span style class="">ZEROs</span>] <b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> </span></b>[5x5 <span style class="">ZEROs</span>] <b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> </span></b><b><span style="color:rgb(0,0,255)"> [5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</span></b><br>
[5x5 <span style class="">ZEROs</span>]<b><span style="color:rgb(0,0,255)"> </span></b>[5x5 <span style class="">ZEROs</span>] <b><span style="color:rgb(0,0,255)">[5x5 nonzero]</span></b> [5x5 <span style class="">ZEROs</span>] [5x5 <span style class="">ZEROs</span>] <b><span style="color:rgb(0,0,255)">[5x5 nonzero] [5x5 nonzero] [5x5 nonzero]</span></b><b><span style="color:rgb(0,0,255)"> [5x5 nonzero]</span></b></font> </font><div>
<br><br></div><div>With multiple elements in the mesh, the above matrix will be the structure of the diagonal blocks of the PC matrix. If I call the block above [A], then the full PC matrix (assemble through all the processors) will look like<br>
<br></div><div> [A] 0 0 0 0 0 0 0 <br></div><div> 0 [A] 0 0 0 0 0 0<br></div><div> 0 0 [A] 0 0 0 0 0<br></div><div> 0 0 0 [A] 0 0 0 0<br>
</div><div> PC= 0 0 0 0 [A] 0 0 0 -> PC matrix if I have 8 element in my mesh.<br></div><div> 0 0 0 0 0 [A] 0 0<br></div><div> 0 0 0 0 0 0 [A] 0<br></div>
<div> 0 0 0 0 0 0 0 [A]<br></div><div><br>Now in my code I have constructed the <span style class="">CSR</span> format which gives the position of the blue nonzero blocks in the PC matrix constructed by each processor.<br>
<br></div><div>I am calling the following <span style class="">PETSc</span> functions:<br></div><div><br>call <span style class="">MatCreate</span>(<span style class="">PETSC</span>_COMM_WORLD,<span style class="">petsc</span>_<span style class="">pc</span>_mat,i_err)<br>
call check_<span style class="">petsc</span>_error(i_err)<br><br>call <span style class="">MatSetType</span>(<span style class="">petsc</span>_<span style class="">pc</span>_mat,<span style class="">MATMPIBAIJ</span>,i_err)<br>
call check_<span style class="">petsc</span>_error(i_err)<br><br>call <span style class="">MatSetSizes</span>(<span style class="">petsc</span>_<span style class="">pc</span>_mat,<span style class="">nprows</span>,<span style class="">nprows</span>,<span style class="">ngrows</span>,<span style class="">ngrows</span>,i_err) ! <span style class="">nprows</span> is the number of local rows which is equal to the number of local columns<br>
call check_<span style class="">petsc</span>_error(i_err)<br><br>call <span style class="">MatSetFromOptions</span>(<span style class="">petsc</span>_<span style class="">pc</span>_mat,i_err)<br>call check_<span style class="">petsc</span>_error(i_err)<br>
<br>call <span style class="">MatSetUp</span>(<span style class="">petsc</span>_<span style class="">pc</span>_mat,i_err)<br>call check_<span style class="">petsc</span>_error(i_err)<br><br>block_size = <span style class="">nequations</span><br>
call <span style class="">MatMPIBAIJSetPreallocationCSR</span>(<span style class="">petsc</span>_<span style class="">pc</span>_mat,block_size,<span style class="">ia</span>, <span style class="">ja</span>, <span style class="">PETSC</span>_NULL_SCALAR i_err)<br>
call check_<span style class="">petsc</span>_error(i_err)<br><br></div><div>1) In the latter call, should <span style class="">ia</span> and <span style class="">ja</span> be the <span style class="">CSR</span> vectors which gives the position of the blue blocks in the matrix or the <span style class="">ia</span> and <span style class="">ja</span> be the <span style class="">CSR</span> vectors which gives the position of the <span style class="">nnz</span> scalar terms in the PC matrix?<br>
<br></div><div>2) My blue block are stored in an array of 3 dimensions: a (1:5,1:5, Number_of_blocks). I am thinking to try to use 2 approaches:<br> a) Looping over the number of blocks and at each time set a single block in the matrix. However, I have seen that I should flatten the array before to pass the values. Is there a way to avoid to flatten the array and just pass it as a(1:5,1:5,block_index)?<br>
<br></div><div> b) If I am not wrong, having the <span style class="">CSR</span> structure I could call just 1 time <span style class="">MatSetValueBlocked</span> and set the entire matrix owned by one process. Am I right?<br>
<br></div><div>Thank you.<br><br>Best Regards<br></div><div><br><br></div><div><br><br><br><br clear="all"></div><div><div><div><br>-- <br><div dir="ltr">Matteo<br></div>
</div></div></div></div>