<div dir="ltr">
<div>Dear PETSc team,</div><div><br></div><div>I would like to ask you a question about the matrix preallocation. <br></div><div>I am using the routine MatMPIAIJSetPreallocation(). <br></div><div><br></div><div>Example: The matrix A has the size 18 x 18 with 168 nonzeros:<br></div><div>A =    <br></div><div> 106.21      -91.667       26.042            0            0            0            0            0            0      -2704.2       2862.5      -1354.2       260.42            0            0            0            0            0<br>      -91.667       132.25      -91.667       26.042            0            0            0            0            0       2862.5      -4058.3       3122.9      -1354.2       260.42            0            0            0            0<br>       26.042      -91.667       132.25      -91.667       26.042            0            0            0            0      -1354.2       3122.9      -4058.3       3122.9      -1354.2       260.42            0            0            0<br>            0       26.042      -91.667       132.25      -91.667       26.042            0            0            0       260.42      -1354.2       3122.9      -4058.3       3122.9      -1354.2       260.42            0            0<br>            0            0       26.042      -91.667       132.25      -91.667       26.042            0            0            0       260.42      -1354.2       3122.9      -4058.3       3122.9      -1354.2       260.42            0<br>            0            0            0       26.042      -91.667       132.25      -91.667       26.042            0            0            0       260.42      -1354.2       3122.9      -4058.3       3122.9      -1354.2       260.42<br>            0            0            0            0       26.042      -91.667       132.25      -91.667       26.042            0            0            0       260.42      -1354.2       3122.9      -4058.3       3122.9      -1354.2<br>            0            0            0            0            0       26.042      -91.667       132.25      -91.667            0            0            0            0       260.42      -1354.2       3122.9      -4058.3       2862.5<br>            0            0            0            0            0            0       26.042      -91.667       106.21            0            0            0            0            0       260.42      -1354.2       2862.5      -2704.2<br>       9.3542      -8.3333       2.6042            0            0            0            0            0            0       106.21      -91.667       26.042            0            0            0            0            0            0<br>      -8.3333       11.958      -8.3333       2.6042            0            0            0            0            0      -91.667       132.25      -91.667       26.042            0            0            0            0            0<br>       2.6042      -8.3333       11.958      -8.3333       2.6042            0            0            0            0       26.042      -91.667       132.25      -91.667       26.042            0            0            0            0<br>            0       2.6042      -8.3333       11.958      -8.3333       2.6042            0            0            0            0       26.042      -91.667       132.25      -91.667       26.042            0            0            0<br>            0            0       2.6042      -8.3333       11.958      -8.3333       2.6042            0            0            0            0       26.042      -91.667       132.25      -91.667       26.042            0            0<br>            0            0            0       2.6042      -8.3333       11.958      -8.3333       2.6042            0            0            0            0       26.042      -91.667       132.25      -91.667       26.042            0<br>            0            0            0            0       2.6042      -8.3333       11.958      -8.3333       2.6042            0            0            0            0       26.042      -91.667       132.25      -91.667       26.042<br>            0            0            0            0            0       2.6042      -8.3333       11.958      -8.3333            0            0            0            0            0       26.042      -91.667       132.25      -91.667<br>            0            0            0            0            0            0       2.6042      -8.3333       9.3542            0            0            0            0            0            0       26.042      -91.667       106.21<br></div><div><br></div><div>I am wondering how the d_nz and o_nz values should be set.</div><div>I read the docs: <a href="https://petsc.org/release/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html">https://petsc.org/release/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html</a></div><div><br></div><div><b>For example, for 2 processes P1, P2: </b><br></div><div>mi - denote the number of rows which belong to process Pi<br></div><div>P0: m0 = 9<br></div><div>P1: m1 = 9 <br><br></div><div>The dn_nz and o_nz should be: <br></div><div>P0: d_nz = 5; o_nz = 7<br></div><div>P1: d_nz = 5; o_nz = 7<br><div><br></div><div>
Therefore, we are allocating m*(d_nz+o_nz) storage locations for every process:<br></div><div>P0: 9*(5+7) = 108</div><div>
P1: 9*(5+7) = 108



<br></div><div>So, in total, we have 216 storage locations to store 168 nonzeros. <br></div><div><br></div><div>But I am doing more experiments and I use the various number of processes - e.g. 2, 4, 8, 19, 32, etc. <br><br></div><div><b>For example, if we take 3 processes - P0, P1, P2:</b><br>
<div>P0: m0 = 6<br></div><div>P1: m1 = 6</div><div>P2: m2 = 6</div><div><br></div><div>
<div>The dn_nz and o_nz should be: <br></div><div>P0: d_nz = 5; o_nz = 7 
 (I took the maximum number of o_nz)

</div><div>P1: d_nz = 3; o_nz = 8  (I took the maximum number of o_nz)<br></div><div>P2: d_nz = 5; o_nz = 7 
 (I took the maximum number of o_nz)

</div><div><div><br></div><div>
Therefore, we are allocating m*(d_nz+o_nz) storage locations for every process:<br></div><div>P0: 6*(5+7) = 72<br></div><div>
P1: 6*(3+8) = 66



<br></div><div>P2: 6*(5+7) = 72<br></div><div>So, in total, we have 210 storage locations to store 168 nonzeros. <br></div><div><br></div><div><b>My questions are:</b> <br></div><div>1) How should I set the d_nz and o_nz values when the number of processes changes in every experiment? <br></div><div>-- I suppose, there has to be some "general" settings, so you do not have to change these values for every experiment, right? <br></div><div>
-- For better illustration, I am sending you figures of the matrix A in the attachment.

<br><br></div><div>2) If I run the code sequentially, then the 
MatMPIAIJSetPreallocation() routine will crash? <br></div><div><br></div><div>3) If I need to generate a large identity matrix, is this a correct approach, please? <br>//create matrix I (diagonal matrix)<br> ierr = MatCreate(comm, &Ione);CHKERRQ(ierr);<br>      ierr = MatSetType(Ione, MATMPIAIJ);CHKERRQ(ierr);<br>     ierr = MatSetSizes(Ione, PETSC_DECIDE, PETSC_DECIDE, data_size, data_size);CHKERRQ(ierr);<br><br>   // parallel sparse - preallocation for better performance<br>     ierr = MatMPIAIJSetPreallocation(Ione, 1, PETSC_NULL, 0, PETSC_NULL);CHKERRQ(ierr);<br><br> // create vector with ones<br>    ierr = VecCreate(PETSC_COMM_WORLD, &diagVec);CHKERRQ(ierr);<br>       ierr = VecSetType(diagVec, VECMPI);CHKERRQ(ierr);<br>     ierr = VecSetSizes(diagVec, PETSC_DECIDE, data_size);CHKERRQ(ierr);<br>   ierr = VecSet(diagVec, 1);CHKERRQ(ierr);<br>      // set the diagonal of matrix I using the vector diagVec<br>      ierr = MatDiagonalSet(Ione, diagVec, INSERT_VALUES);CHKERRQ(ierr);<br><br>  ierr = MatAssemblyBegin(Ione, MAT_FINAL_ASSEMBLY);<br>    ierr = MatAssemblyEnd(Ione, MAT_FINAL_ASSEMBLY);</div><div><br>

</div><div><br></div><div>Thank you very much in advance. <br></div><div>Kind regards<br></div><div>Gabi<br></div><div><br></div></div>

 </div>

</div><div>  <br></div>

</div></div>