<div dir="ltr"><div>Can you please send your code?</div><div><br></div><div>It would greatly help Matt (and others looking at this thread) </div><div>to understand the performance you see and to offer advice on how (if possible) to improve it.</div><div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 17 Jul 2019 at 16:00, Matthew Knepley via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, Jul 17, 2019 at 8:51 AM Mohammed Mostafa <<a href="mailto:mo7ammedmostafa@gmail.com" target="_blank">mo7ammedmostafa@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Sorry for the confusion <div>First I fully acknowledge that setting Matrix non-zeros or copying in general is not cheap and memory access pattern can play an important role.</div><div>So to establish a baseline to compare with, I tried setting the same matrix but in an Eigen Sparse Matrix and the timings are as follows</div><div><span style="color:rgb(204,0,0)">FillPetscMat_with_MatSetValues 100 1.0 3.8820e+00 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 23 0 0 0 0 </span><b style="color:rgb(204,0,0)"><span style="background-color:rgb(244,204,204)"> 96</span> </b><span style="color:rgb(204,0,0)"> 0 0 0 0 0</span> </div><div>FilEigenMat 100 1.0 2.8727e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 18 0 0 0 0 88 0 0 0 0 0 </div></div></blockquote><div><br></div><div>Great. This helps. Two things would help me narrow down what is happening.</div><div><br></div><div> 1) Are you timing only the insertion of values, or computation and insertion?</div><div><br></div><div> 2) Can you tell me how many values are inserted?</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I used the same code but simply filled a different Matrix something like</div><div><br></div><div>for ( i =0; i < nRows;i++)</div><div>{</div><div>//</div><div>.......</div><div>// Some code to get j_index, coefvalues</div><div>// Method1</div><div>MatSetValues(A, 1, &cell_global_index, nnz_per_row, j_index, coefvalues, INSERT_VALUES); </div><div><br></div><div>//Method2</div><div>for ( int k = 0;k < nnz_per_row; k++)</div><div> EigenMat.coeffRef(i, j_index[k] ) = coefvalues[k];</div><div><br></div><div> }<br></div><div>Please note that only one of the two methods is being used at a time. Also, I separately time the code section used to < j_index, coefvalues> but simpling disabling both Method1 and Method2.</div><div>I found the cost to be trivial in comparison to when either one of the methods is used.</div><div>I used Eigen out of convenience since I used for some vector and tensor arithmetics somewhere else in the code and it may not be the best choice.</div><div>Since in PetscMatrix we technically fill two matrices: diagonal and off-diagonal so I expected some difference but is that normal or am I missing something. ?</div><div>Maybe some setting or MatOption I should be using so far this what I have been using</div><div><br></div><div>MatCreateMPIAIJWithArrays(PETSC_COMM_WORLD, local_size, local_size, PETSC_DETERMINE,<br>PETSC_DETERMINE, ptr, j , v, A);<br> MatSetOption(A,MAT_NO_OFF_PROC_ENTRIES,PETSC_TRUE);<br>MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>MatSetOption(A,MAT_NEW_NONZERO_LOCATION_ERR,PETSC_TRUE);<br>MatSetOption(A,MAT_NEW_NONZERO_LOCATIONS,PETSC_FALSE);<br>MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>MatSetUp(A);<br>MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY);<br>MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); <br></div><div><br></div><div>Thanks,</div><div>Kamra</div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_4985737615019044776gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div></div>