<div dir="ltr"><span style="font-size:12.8px">*The flop rate for the MatMult and MatSolve are much lower than I would expect. Try using the AIJ matrix instead of SBAIJ.*</span><br><div><span style="font-size:12.8px">SeqAIJ: </span></div><div><span style="font-size:12.8px"> MatMult = 213 Mflop/s</span></div><div><span style="font-size:12.8px"> MatSolve = 204 Mflop/s</span></div><div><span style="font-size:12.8px">AIJ: </span></div><div><span style="font-size:12.8px"> MatMult = 211 Mflop/s</span></div><div><span style="font-size:12.8px"> MatSolve = 198 Mflop/s</span></div><div><span style="font-size:12.8px">SBAIJ:</span></div><div><span style="font-size:12.8px"> MatMult = 179 Mflop/s</span></div><div><span style="font-size:12.8px"> MatSolve = 167 Mflop/s</span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Shouldn't the SBAIJ provide the best performance since it only works with the upper triangular part of the whole large sparse symmetric matrix? </span></div><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">Thanks,</span></div><div><span style="font-size:12.8px">Bodhi</span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Mar 26, 2017 at 12:30 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
The flop rate for the MatMult and MatSolve are much lower than I would expect. Try using the AIJ matrix instead of SBAIJ.<br>
<div><div class="h5"><br>
<br>
> On Mar 26, 2017, at 6:25 AM, Bodhisatta Pramanik <<a href="mailto:bodhi91@iastate.edu">bodhi91@iastate.edu</a>> wrote:<br>
><br>
> *First set a new PETSC_ARCH say arch-opt and ./configure with the additional argument --with-debugging=0 then recompile PETSc with the new PETSC_ARCH and recompile your code then send the new -log_view output.*<br>
> I have attached the new log summary with this file. The computation time is a lot faster now. It takes around 23 seconds to compute the first 3 eigenvectors of a 200K sized matrix.<br>
><br>
> Thanks,<br>
> Bodhi<br>
><br>
> On Sun, Mar 26, 2017 at 3:14 AM, Bodhisatta Pramanik <<a href="mailto:bodhi91@iastate.edu">bodhi91@iastate.edu</a>> wrote:<br>
> *The error is in EPSComputeError(), not in EPSSolve(). Did you modify the state of the EPS object between EPSSolve() and EPSComputeError()?*<br>
><br>
> I am not modifying the eps object. I have copied a part of the code where I am computing the eigenvectors.<br>
><br>
> EPSCreate(PETSC_COMM_WORLD,&<wbr>eps);<br>
> EPSSetOperators(eps,m.Lpl,<wbr>NULL);<br>
> EPSSetProblemType(eps,EPS_HEP)<wbr>;<br>
> EPSSetFromOptions(eps);<br>
><br>
> EPSSolve(eps); //Solving for the eigenvalues<br>
><br>
> EPSGetIterationNumber(eps,&<wbr>its);<br>
> PetscPrintf(PETSC_COMM_WORLD,"<wbr>Number of iterations of the method: %D\n",its);<br>
> EPSGetType(eps,&type);<br>
> PetscPrintf(PETSC_COMM_WORLD,"<wbr>Solution Method: %s\n\n",type);<br>
> EPSGetTolerances(eps,&get_tol,<wbr>&maxit);<br>
> PetscPrintf(PETSC_COMM_WORLD,"<wbr>Stopping condition: tol=%.4g, maxit=%D\n",(double)get_tol,<wbr>maxit);<br>
> EPSGetConverged(eps,&nconv);<br>
> PetscPrintf(PETSC_COMM_WORLD, "Number of converged eigenpairs: %D\n\n",nconv);<br>
> if(nconv>0)<br>
> {<br>
> PetscPrintf(PETSC_COMM_WORLD,<br>
> " k ||Ax-kx||/||kx||\n"<br>
> "-------------- ----------------\n");<br>
> for(i=0;i<nconv;i++)<br>
> {<br>
> EPSGetEigenpair(eps,i,&kr,&ki,<wbr>xr,xi);<br>
> EPSComputeError(eps,i,EPS_<wbr>ERROR_RELATIVE,&error);<br>
> re = PetscRealPart(kr);<br>
> im = PetscImaginaryPart(kr);<br>
><br>
> re = kr;<br>
> im = ki;<br>
> if(im!=0.0) {<br>
> PetscPrintf(PETSC_COMM_WORLD," %9f%+9fi %12g\n",(double)re,(double)im,<wbr>(double)error);<br>
> }<br>
> else {<br>
> PetscPrintf(PETSC_COMM_WORLD," %12f %12g\n",(double)re,(double)<wbr>error);<br>
> }<br>
> }<br>
> PetscPrintf(PETSC_COMM_WORLD,"<wbr>\n");<br>
> }<br>
><br>
> EPSDestroy(&eps);<br>
> VecDestroy(&xr);<br>
> VecDestroy(&xi);<br>
> }<br>
><br>
> This is what I pass through my command line:<br>
> ./RunPart -eps_type jd -eps_nev 3 -st_ksp_type cg -st_ksp_rtol 0.001 -eps_tol 0.001 -st_pc_type bjacobi -eps_smallest_real<br>
><br>
><br>
> *In graph partitioning, I would strongly recommend deflating the eigenvector corresponding to the zero eigenvalue, as is done in ex11.c:*<br>
> I have tried doing this but for some reason the eigenvalues fail to converge.<br>
><br>
><br>
><br>
> On Sun, Mar 26, 2017 at 2:41 AM, Jose E. Roman <<a href="mailto:jroman@dsic.upv.es">jroman@dsic.upv.es</a>> wrote:<br>
><br>
> > El 26 mar 2017, a las 6:08, Bodhisatta Pramanik <<a href="mailto:bodhi91@iastate.edu">bodhi91@iastate.edu</a>> escribió:<br>
> ><br>
> > Send all the output in the error message:<br>
> ><br>
> > [0]PETSC ERROR: --------------------- Error Message ------------------------------<wbr>------------------------------<wbr>--<br>
> > [0]PETSC ERROR: Object is in wrong state<br>
> > [0]PETSC ERROR: Must call EPSSolve() first: Parameter #1<br>
> > [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/<wbr>documentation/faq.html</a> for trouble shooting.<br>
> > [0]PETSC ERROR: Petsc Release Version 3.7.5, Jan, 01, 2017<br>
> > [0]PETSC ERROR: ./RunPart on a arch-linux2-c-debug named <a href="http://research-5.ece.iastate.edu" rel="noreferrer" target="_blank">research-5.ece.iastate.edu</a> by bodhi91 Sat Mar 25 22:33:56 2017<br>
> > [0]PETSC ERROR: Configure options --with-cc=gcc --with-css=g++ -with-fc=gfortran --download-fblaslapack --download-mpich<br>
> > [0]PETSC ERROR: #10468 EPSComputeError() line 643 in /tmp/Bodhi/slepc-3.7.3/src/<wbr>eps/interface/epssolve.c<br>
> > 0.000000 2.0795e-317<br>
> ><br>
><br>
> The error is in EPSComputeError(), not in EPSSolve(). Did you modify the state of the EPS object between EPSSolve() and EPSComputeError()?<br>
><br>
> In graph partitioning, I would strongly recommend deflating the eigenvector corresponding to the zero eigenvalue, as is done in ex11.c:<br>
> <a href="http://slepc.upv.es/documentation/current/src/eps/examples/tutorials/ex11.c.html" rel="noreferrer" target="_blank">http://slepc.upv.es/<wbr>documentation/current/src/eps/<wbr>examples/tutorials/ex11.c.html</a><br>
><br>
> Jose<br>
><br>
><br>
><br>
><br>
> --<br>
> Bodhisatta Pramanik,<br>
> Graduate Student,<br>
> Department of Electrical and Computer Engineering,<br>
> 301 Durham,<br>
> Iowa State University,<br>
> Ames,Iowa 50011,<br>
> <a href="mailto:bodhi91@iastate.edu">bodhi91@iastate.edu</a><br>
> <a href="tel:515-735-6300" value="+15157356300">515-735-6300</a><br>
><br>
><br>
><br>
> --<br>
> Bodhisatta Pramanik,<br>
> Graduate Student,<br>
> Department of Electrical and Computer Engineering,<br>
> 301 Durham,<br>
> Iowa State University,<br>
> Ames,Iowa 50011,<br>
> <a href="mailto:bodhi91@iastate.edu">bodhi91@iastate.edu</a><br>
> <a href="tel:515-735-6300" value="+15157356300">515-735-6300</a><br>
</div></div>> <New Log Summary><br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><b><font size="2" face="times new roman, serif">Bodhisatta Pramanik,</font></b><div><i><font size="2" face="times new roman, serif">Graduate Student,</font></i></div><div><i><font size="2" face="times new roman, serif">Department of Electrical and Computer Engineering,</font></i></div><div><i><font size="2" face="times new roman, serif">301 Durham,</font></i></div><div><i><font size="2" face="times new roman, serif">Iowa State University,</font></i></div><div><i><font size="2" face="times new roman, serif">Ames,Iowa 50011,</font></i></div><div><font size="2" face="times new roman, serif"><a href="mailto:bodhi91@iastate.edu" target="_blank">bodhi91@iastate.edu</a></font></div><div><u><font size="2" face="times new roman, serif">515-735-6300</font></u></div></div></div></div></div></div></div></div></div>
</div>