<div class="gmail_quote">On Tue, Jun 21, 2011 at 07:34, khalid ashraf <span dir="ltr"><<a href="mailto:khalid_eee@yahoo.com">khalid_eee@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>In order to find where the extra time is consumed, I started from ksp/ksp/example/tutorials/ex22.c and changed it one line at a time. I found that the time is consumed in the call: </div><div><br></div><div><div> ierr = MatSetValuesStencil(B,7,row,7,col,&val[0][0],INSERT_VALUES);CHKERRQ(ierr);</div>
</div></blockquote><div><br></div><div>You must have changed more than just this because the arrays just aren't that big. This is a finite difference example. It inserts one row at a time. Inserting 7 rows at a time doesn't make sense.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div></div></div><div style="font-family:arial, helvetica, sans-serif;font-size:12pt"><div style="font-family:arial, helvetica, sans-serif;font-size:10pt">
<div style="font-family:arial, helvetica, sans-serif;font-size:10pt"> This is because I expected the computation time to be 7*1e-2 by this repetitive call. However, I find that the execution time is ~1e-2 only for l=3. For any other value of l, it is ~1e0. I can see that the only speciality of l=3 is that it corresponds to row=(i,j,k). Any other combination like (i+1,j,k) causes the call to</div>
<div style="font-family:arial, helvetica, sans-serif;font-size:10pt"> MatSetValuesBlockedStencil(B,1,&row[l],7,col,&val[0][0],INSERT_VALUES);CHKERRQ(ierr);}</div><div>to be slower by 2 orders of magnitude. Could you please suggest why the performance goes down so drastically. </div>
</div></div></blockquote><div><br></div><div>You have to preallocate correctly to get high performance. When you change the insertion code to set values arbitrarily, the preallocation becomes incorrect. This is all explained in the link I gave you earlier as well as the Users Manual.</div>
<div><br></div><div> <a href="http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#efficient-assembly">http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#efficient-assembly</a></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="font-family:arial, helvetica, sans-serif;font-size:12pt"><div style="font-family:arial, helvetica, sans-serif;font-size:10pt"><div style="font-family:arial, helvetica, sans-serif;font-size:10pt">I want to assemble a matrix for FEM application. I started from the example ksp/ksp/examples/tutorials/ex3.c</div>
<div style="font-family:arial, helvetica, sans-serif;font-size:10pt">This example shows the similar problem stated above when m>=120. </div></div></div></blockquote></div><br><div>For simplicity, this example did not preallocate. I have added very naive preallocation to this example in petsc-dev. If you add the two lines below, assembly will be fast for large sizes.</div>
<div><br></div><div><div>changeset: b961a1bfd123</div><div>user: Jed Brown <jed@59A2.org></div><div>date: Tue Jun 21 11:25:30 2011 +0200</div><div>summary: Add naive preallocation to example</div>
<div><br></div><div>diff --git a/src/ksp/ksp/examples/tutorials/ex3.c b/src/ksp/ksp/examples/tutorials/ex3.c</div><div>--- a/src/ksp/ksp/examples/tutorials/ex3.c</div><div>+++ b/src/ksp/ksp/examples/tutorials/ex3.c</div><div>
@@ -64,6 +64,8 @@</div><div> ierr = MatCreate(PETSC_COMM_WORLD,&A);CHKERRQ(ierr);</div><div> ierr = MatSetSizes(A,PETSC_DECIDE,PETSC_DECIDE,N,N);CHKERRQ(ierr);</div><div> ierr = MatSetFromOptions(A);CHKERRQ(ierr);</div>
<div>+ ierr = MatSeqAIJSetPreallocation(A,9,PETSC_NULL);CHKERRQ(ierr);</div><div>+ ierr = MatMPIAIJSetPreallocation(A,9,PETSC_NULL,5,PETSC_NULL);CHKERRQ(ierr); /* More than necessary */</div><div> start = rank*(M/size) + ((M%size) < rank ? (M%size) : rank);</div>
<div> end = start + M/size + ((M%size) > rank); </div></div><div><br></div>