<div dir="ltr">For icc and cholesky factorizations, MatSolve_SeqSBAIJ() is used for<div>both aij and sbaij. Why it gives different flops?</div><div><br></div><div>Hong<br><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Mar 26, 2017 at 2:44 PM, Bodhisatta Pramanik <span dir="ltr"><<a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><span style="font-size:12.8px">*The flop rate for the MatMult and MatSolve are much lower than I would expect. Try using the AIJ matrix instead of SBAIJ.*</span><br><div><span style="font-size:12.8px">SeqAIJ: </span></div><div><span style="font-size:12.8px"> MatMult = 213 Mflop/s</span></div><div><span style="font-size:12.8px"> MatSolve = 204 Mflop/s</span></div><div><span style="font-size:12.8px">AIJ: </span></div><div><span style="font-size:12.8px"> MatMult = 211 Mflop/s</span></div><div><span style="font-size:12.8px"> MatSolve = 198 Mflop/s</span></div><div><span style="font-size:12.8px">SBAIJ:</span></div><div><span style="font-size:12.8px"> MatMult = 179 Mflop/s</span></div><div><span style="font-size:12.8px"> MatSolve = 167 Mflop/s</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Shouldn't the SBAIJ provide the best performance since it only works with the upper triangular part of the whole large sparse symmetric matrix? </span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Thanks,</span></div><div><span style="font-size:12.8px">Bodhi</span></div></div><div class="gmail-HOEnZb"><div class="gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Mar 26, 2017 at 12:30 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
The flop rate for the MatMult and MatSolve are much lower than I would expect. Try using the AIJ matrix instead of SBAIJ.<br>
<div><div class="gmail-m_-5080726389703827270h5"><br>
<br>
> On Mar 26, 2017, at 6:25 AM, Bodhisatta Pramanik <<a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a>> wrote:<br>
><br>
> *First set a new PETSC_ARCH say arch-opt and ./configure with the additional argument --with-debugging=0 then recompile PETSc with the new PETSC_ARCH and recompile your code then send the new -log_view output.*<br>
> I have attached the new log summary with this file. The computation time is a lot faster now. It takes around 23 seconds to compute the first 3 eigenvectors of a 200K sized matrix.<br>
><br>
> Thanks,<br>
> Bodhi<br>
><br>
> On Sun, Mar 26, 2017 at 3:14 AM, Bodhisatta Pramanik <<a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a>> wrote:<br>
> *The error is in EPSComputeError(), not in EPSSolve(). Did you modify the state of the EPS object between EPSSolve() and EPSComputeError()?*<br>
><br>
> I am not modifying the eps object. I have copied a part of the code where I am computing the eigenvectors.<br>
><br>
> EPSCreate(PETSC_COMM_WORLD,&e<wbr>ps);<br>
> EPSSetOperators(eps,m.Lpl,NUL<wbr>L);<br>
> EPSSetProblemType(eps,EPS_<wbr>HEP);<br>
> EPSSetFromOptions(eps);<br>
><br>
> EPSSolve(eps); //Solving for the eigenvalues<br>
><br>
> EPSGetIterationNumber(eps,&it<wbr>s);<br>
> PetscPrintf(PETSC_COMM_WORLD,<wbr>"Number of iterations of the method: %D\n",its);<br>
> EPSGetType(eps,&type);<br>
> PetscPrintf(PETSC_COMM_WORLD,<wbr>"Solution Method: %s\n\n",type);<br>
> EPSGetTolerances(eps,&get_<wbr>tol,&maxit);<br>
> PetscPrintf(PETSC_COMM_WORLD,<wbr>"Stopping condition: tol=%.4g, maxit=%D\n",(double)get_tol,ma<wbr>xit);<br>
> EPSGetConverged(eps,&nconv);<br>
> PetscPrintf(PETSC_COMM_WORLD, "Number of converged eigenpairs: %D\n\n",nconv);<br>
> if(nconv>0)<br>
> {<br>
> PetscPrintf(PETSC_COMM_WORLD,<br>
> " k ||Ax-kx||/||kx||\n"<br>
> "-------------- ----------------\n");<br>
> for(i=0;i<nconv;i++)<br>
> {<br>
> EPSGetEigenpair(eps,i,&kr,&<wbr>ki,xr,xi);<br>
> EPSComputeError(eps,i,EPS_ERR<wbr>OR_RELATIVE,&error);<br>
> re = PetscRealPart(kr);<br>
> im = PetscImaginaryPart(kr);<br>
><br>
> re = kr;<br>
> im = ki;<br>
> if(im!=0.0) {<br>
> PetscPrintf(PETSC_COMM_WORLD,<wbr>" %9f%+9fi %12g\n",(double)re,(double)im,<wbr>(double)error);<br>
> }<br>
> else {<br>
> PetscPrintf(PETSC_COMM_WORLD,<wbr>" %12f %12g\n",(double)re,(double)err<wbr>or);<br>
> }<br>
> }<br>
> PetscPrintf(PETSC_COMM_WORLD,<wbr>"\n");<br>
> }<br>
><br>
> EPSDestroy(&eps);<br>
> VecDestroy(&xr);<br>
> VecDestroy(&xi);<br>
> }<br>
><br>
> This is what I pass through my command line:<br>
> ./RunPart -eps_type jd -eps_nev 3 -st_ksp_type cg -st_ksp_rtol 0.001 -eps_tol 0.001 -st_pc_type bjacobi -eps_smallest_real<br>
><br>
><br>
> *In graph partitioning, I would strongly recommend deflating the eigenvector corresponding to the zero eigenvalue, as is done in ex11.c:*<br>
> I have tried doing this but for some reason the eigenvalues fail to converge.<br>
><br>
><br>
><br>
> On Sun, Mar 26, 2017 at 2:41 AM, Jose E. Roman <<a href="mailto:jroman@dsic.upv.es" target="_blank">jroman@dsic.upv.es</a>> wrote:<br>
><br>
> > El 26 mar 2017, a las 6:08, Bodhisatta Pramanik <<a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a>> escribió:<br>
> ><br>
> > Send all the output in the error message:<br>
> ><br>
> > [0]PETSC ERROR: --------------------- Error Message ------------------------------<wbr>------------------------------<wbr>--<br>
> > [0]PETSC ERROR: Object is in wrong state<br>
> > [0]PETSC ERROR: Must call EPSSolve() first: Parameter #1<br>
> > [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/d<wbr>ocumentation/faq.html</a> for trouble shooting.<br>
> > [0]PETSC ERROR: Petsc Release Version 3.7.5, Jan, 01, 2017<br>
> > [0]PETSC ERROR: ./RunPart on a arch-linux2-c-debug named <a href="http://research-5.ece.iastate.edu" rel="noreferrer" target="_blank">research-5.ece.iastate.edu</a> by bodhi91 Sat Mar 25 22:33:56 2017<br>
> > [0]PETSC ERROR: Configure options --with-cc=gcc --with-css=g++ -with-fc=gfortran --download-fblaslapack --download-mpich<br>
> > [0]PETSC ERROR: #10468 EPSComputeError() line 643 in /tmp/Bodhi/slepc-3.7.3/src/eps<wbr>/interface/epssolve.c<br>
> > 0.000000 2.0795e-317<br>
> ><br>
><br>
> The error is in EPSComputeError(), not in EPSSolve(). Did you modify the state of the EPS object between EPSSolve() and EPSComputeError()?<br>
><br>
> In graph partitioning, I would strongly recommend deflating the eigenvector corresponding to the zero eigenvalue, as is done in ex11.c:<br>
> <a href="http://slepc.upv.es/documentation/current/src/eps/examples/tutorials/ex11.c.html" rel="noreferrer" target="_blank">http://slepc.upv.es/documentat<wbr>ion/current/src/eps/examples/<wbr>tutorials/ex11.c.html</a><br>
><br>
> Jose<br>
><br>
><br>
><br>
><br>
> --<br>
> Bodhisatta Pramanik,<br>
> Graduate Student,<br>
> Department of Electrical and Computer Engineering,<br>
> 301 Durham,<br>
> Iowa State University,<br>
> Ames,Iowa 50011,<br>
> <a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a><br>
> <a href="tel:515-735-6300" value="+15157356300" target="_blank">515-735-6300</a><br>
><br>
><br>
><br>
> --<br>
> Bodhisatta Pramanik,<br>
> Graduate Student,<br>
> Department of Electrical and Computer Engineering,<br>
> 301 Durham,<br>
> Iowa State University,<br>
> Ames,Iowa 50011,<br>
> <a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a><br>
> <a href="tel:515-735-6300" value="+15157356300" target="_blank">515-735-6300</a><br>
</div></div>> <New Log Summary><br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail-m_-5080726389703827270gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><b><font size="2" face="times new roman, serif">Bodhisatta Pramanik,</font></b><div><i><font size="2" face="times new roman, serif">Graduate Student,</font></i></div><div><i><font size="2" face="times new roman, serif">Department of Electrical and Computer Engineering,</font></i></div><div><i><font size="2" face="times new roman, serif">301 Durham,</font></i></div><div><i><font size="2" face="times new roman, serif">Iowa State University,</font></i></div><div><i><font size="2" face="times new roman, serif">Ames,Iowa 50011,</font></i></div><div><font size="2" face="times new roman, serif"><a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a></font></div><div><u><font size="2" face="times new roman, serif"><a href="tel:(515)%20735-6300" value="+15157356300" target="_blank">515-735-6300</a></font></u></div></div></div></div></div></div></div></div></div>
</div>
</div></div></blockquote></div><br></div></div></div>