<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Apr 22, 2010, at 7:33 AM, Parker, Andrew (UK Filton) wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"> <font face="Arial"><font size="2"> </font></font><div> <div><span class="825482612-22042010"><font face="Arial" size="2">Hi,</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2">I'm new to these postings. On very large cases, regardless of where I stop the debugger it is always copying data. I've put it into a debugger because on smaller cases it runs fine, on larger it takes a while (very long time) to get going. The stack trace always gives something like:</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2">memcpy</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2">PetscMemCpy</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2">MatSetValues_SeqAIJ</font></span></div></div></blockquote><div><br></div> It is using SeqAIJ matrices</div><div><br><blockquote type="cite"><div> <div><span class="825482612-22042010"><font face="Arial" size="2">MatSetValues</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2">my own wrapper to add values to a location within the matrix.</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2">I'm using Seq_BAIJ. My bets are that I've got the sparsity wrong, or the preallocation wrong, but I'm not sure why. I know this could be anything, but has anybody got any thoughts, remember stopping the debugger at random, regardless of the frequency always gives the above....</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2">I set up the matrix like this</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2"> MatCreate(PETSC_COMM_SELF,&_storage);<br> MatSetSizes(_storage,PETSC_DECIDE,PETSC_DECIDE,numVars*numLocs,numVars*numLocs);<br> MatSetFromOptions(_storage);<br> MatSeqBAIJSetPreallocation(_storage,numVars,PETSC_NULL,sparsityStart);</font></span></div></div></blockquote><div><br></div> Unless you use the argument -mat_type seqbaij this will use SeqAIJ while you preallocate SeqBAIJ.</div><div><br><blockquote type="cite"><div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2">However, using this makes zero difference to the speed</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2"> MatCreateSeqBAIJ(PETSC_COMM_SELF,<br> numVars,<br> numVars*numLocs,<br> numVars*numLocs,<br> 0, </font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"> sparsityStart,</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"> &_storage);</font></span></div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2">It is so slow that it has not even completed one cycle of the solver....</font></span></div></div></blockquote><div><br></div> Your sparseityStart is wrong. Run with -info and grep the results for malloc and you'll see it is mallocing many times to get enough space.</div><div><br></div><div> Barry</div><div><br><blockquote type="cite"><div> <div><span class="825482612-22042010"><font face="Arial" size="2"></font></span> </div> <div><span class="825482612-22042010"><font face="Arial" size="2">Cheers again,<br>Andy</font></span></div> <br> ********************************************************************<br> This email and any attachments are confidential to the intended<br> recipient and may also be privileged. If you are not the intended<br> recipient please delete it from your system and notify the sender.<br> You should not copy it or use it for any purpose nor disclose or<br> distribute its contents to any other person.<br> ********************************************************************<br> <br> </div></blockquote></div><br></body></html>