On Tue, Mar 3, 2009 at 9:17 AM, STEPHANE TCHOUANMO <span dir="ltr"><<a href="mailto:tchouanm@msn.com">tchouanm@msn.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<font style="font-size: 12pt;" size="3"><font face="Courier New">Ok Matt you're right. The SNES Solve is definitely at fault.<br>But still there's something i dont understand in the log summary i get. Take for example the one for the unsteady heat equation right after:<br>
<br></font></font><div class="im">Event Count Time (sec) Flops/sec --- Global --- --- Stage --- Total<br> Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s<br>
------------------------------------------------------------------------------------------------------------------------<br><br>--- Event Stage 0: Main Stage<br><br></div><font color="#ff0000">VecMDot 22 1.0 5.0474e-04 1.0 5.48e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 6 0 0 0 0 6 0 0 0 548</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecNorm 62 1.0 8.8694e-03 1.0 4.72e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 9 0 0 0 0 9 0 0 0 47</font><font color="#ff0000"><br></font><font color="#ff0000">VecScale 32 1.0 3.8212e-04 1.0 2.83e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 283</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecCopy 81 1.0 1.1948e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br></font><font color="#ff0000">VecSet 88 1.0 8.4816e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecAXPY 10 1.0 1.8910e-04 1.0 3.57e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 357</font><font color="#ff0000"><br></font><font color="#ff0000">VecWAXPY 10 1.0 2.6472e-04 1.0 1.27e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 127</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecMAXPY 32 1.0 1.0271e-03 1.0 4.14e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 9 0 0 0 0 9 0 0 0 414</font><font color="#ff0000"><br></font><font color="#ff0000">VecAssemblyBegin 40 1.0 8.7160e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecAssemblyEnd 40 1.0 7.5617e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br></font><font color="#ff0000">VecScatterBegin 39 1.0 1.5163e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecNormalize 32 1.0 3.8553e-03 1.0 6.65e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 5 0 0 0 67</font><br><font color="#00b050">MatMult 22 1.0 1.5831e-02 1.0 2.16e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 72 0 0 0 0 72 0 0 0 216</font><font color="#00b050"><br>
</font><font color="#00b050">MatAssemblyBegin 30 1.0 6.5176e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#00b050"><br></font><font color="#00b050">MatAssemblyEnd 30 1.0 1.2829e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#00b050"><br>
</font><font color="#00b050">MatZeroEntries 9 1.0 1.8313e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><br><font color="#17365d">SNESSolve 10 1.0 1.7674e+01 1.0 2.69e+05 1.0 0.0e+00 0.0e+00 3.0e+01 93100 0 0 0 94100 0 0 75 0</font><font color="#17365d"><br>
</font><font color="#17365d">SNESLineSearch 10 1.0 3.7443e+00 1.0 4.51e+04 1.0 0.0e+00 0.0e+00 1.0e+01 20 4 0 0 0 20 4 0 0 25 0</font><font color="#17365d"><br></font><font color="#17365d">SNESFunctionEval 20 1.0 7.2693e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+01 38 0 0 0 0 39 0 0 0 50 0</font><font color="#17365d"><br>
</font><font color="#17365d">SNESJacobianEval 10 1.0 1.0367e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 55 0 0 0 0 55 0 0 0 25 0</font><br><font color="#974806">KSPGMRESOrthog 22 1.0 1.4277e-03 1.0 3.88e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 12 0 0 0 0 12 0 0 0 388</font><font color="#974806"><br>
</font><font color="#974806">KSPSetup 10 1.0 1.3128e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#974806"><br></font><font color="#974806">KSPSolve 10 1.0 2.8431e-02 1.0 1.57e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 94 0 0 0 0 94 0 0 0 157</font><font color="#974806"><br>
</font><font color="#974806">PCSetUp 10 1.0 2.5831e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#974806"><br></font><font color="#974806">PCApply 32 1.0 5.7973e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><div class="im">
<br>------------------------------------------------------------------------------------------------------------------------<br><br>Memory usage is given in bytes:<br><br>Object Type Creations Destructions Memory Descendants' Mem.<br>
<br>--- Event Stage 0: Main Stage<br><br></div> Index Set 40 40 548800 0<br> Vec 167 153 4198932 0<br> Vec Scatter 40 40 0 0<div class="im">
<br> Matrix 1 0 0 0<br></div> SNES 10 9 1116 0<br> Krylov Solver 10 9 151920 0<br> Preconditioner 10 9 0 0<br>
Viewer 1 0 0 0<br>========================================================================================================================<br><br><font style="font-size: 12pt;" size="3"><font face="Courier New">Now it says SNESSolve takes 93% of the main stage, right? <br>
In that case what does is mean the 20% for SNESLinesearch, 38% for SNESFunctionEval and 55% for SNESJacobianEval? It cant be percentages of the main stage or of the SNESSolve. Do you have an idea?</font></font></div></blockquote>
<div><br>1) There is not strict separation or nesting for events. For instance, line search, function eval, and Jacobian eval all happen inside SNESolve. In addition,<br> function evaluation happens inside line search.<br>
<br>2) However, clearly jac eval + func eval = solve roughly. Thus, nothing else is taking any time. This is confirmed by looking at KSPSolve, which takes no time.<br><br>3) If you say that the first step (of 10) takes most of the time, the conclusion for me is inescapable. LibMesh is making a whole bunch of allocation calls<br>
during the first time step, which is very very slow. After that, it has the memory and everything runs fine.<br><br>I suggest talking to the LibMesh developers.<br><br> Matt<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><font style="font-size: 12pt;" size="3"><font face="Courier New"><br>Actually to answer your question, what is long is the first Newton iteration in the first time step and a debugging in DDD shows it too. So with the log summary i get, its obviously due to SNESSolve with Residual and Jacobian evaluations.<br>
Here is a part of my Jacobian and Residual computation routine in LibMesh for the basic Laplacian (</font></font><font style="font-size: 12pt;" size="3" face="Courier New">- \Delta u = f </font><font style="font-size: 12pt;" size="3"><font face="Courier New">). Its called 'compute_jacobian' and 'compute_residual' respectively. Could you please look at it quickly and tell me if you see at first look something strange?<br>
<br><br><b><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> void compute_jacobian (const NumericVector<Number>& soln,</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> SparseMatrix<Number>& jacobian)</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> {</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> EquationSystems &es = *_equation_system;</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> const MeshBase& mesh = es.get_mesh();</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> NonlinearImplicitSystem& system = </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> es.get_system<NonlinearImplicitSystem>("dc");</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> const DofMap& dof_map = system.get_dof_map();</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#ff0000" size="2" face="Garamond"> // Define the finite volume </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> FV fv;</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> MeshBase::const_element_iterator el = mesh.active_local_elements_begin();</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> const MeshBase::const_element_iterator end_el = mesh.active_local_elements_end();<br><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><font color="#ff0000"> // The loop on every simplex</font><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> for ( ; el != end_el; ++el)</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> {</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> const Elem* elem = *el;</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> dof_map.dof_indices (elem, dof_indices);</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> fv.reinit(elem);</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> n_dofs = dof_indices.size(); <font color="#ff0000">// = 4 for a hex</font></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> Ke.resize (n_dofs, n_dofs);</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br> <font color="#ff0000">// Assemble the elementary matrix for the Laplacian problem (size 4*4)</font> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> Ke=fv.elmmat(perm);</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> dof_map.constrain_element_matrix (Ke, dof_indices);<br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> <font color="#ff0000">// Adds the small matrix Ke to the Jacobian</font><br> jacobian.add_matrix (Ke, dof_indices);</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> }</font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"> }<br>
<br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> void compute_residual (const NumericVector<Number>& soln,</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> NumericVector<Number>& residual)</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> {</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> EquationSystems &es = *_equation_system;</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> const MeshBase& mesh = es.get_mesh();</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> NonlinearImplicitSystem& system = </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> es.get_system<NonlinearImplicitSystem>("dc");</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> const DofMap& dof_map = system.get_dof_map();</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> <font color="#ff0000">// Define the finite volume </font></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> FV fv;</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> residual.zero();</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> MeshBase::const_element_iterator el = mesh.active_local_elements_begin();</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> const MeshBase::const_element_iterator end_el = mesh.active_local_elements_end();<br><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font></b></font></font><font style="font-size: 12pt;" size="3"><font face="Courier New"><b><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><font color="#ff0000"> // The loop on every simplex</font></font></b></font></font><font style="font-size: 12pt;" size="3"><font face="Courier New"><b><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> for ( ; el != end_el; ++el)</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> {</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> const Elem* elem = *el;</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> dof_map.dof_indices (elem, dof_indices);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> fv.reinit(elem);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> n_dofs = dof_indices.size(); </font></b></font></font><font style="font-size: 12pt;" size="3"><font face="Courier New"><b><font style="font-size: 10pt;" color="#002060" size="2" face="Garamond"><font color="#ff0000">// = 4 for a hex</font></font></b></font></font><font style="font-size: 12pt;" size="3"><font face="Courier New"><b><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> Se.resize (n_dofs);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> <br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> <font color="#ff0000"> // Compute the solution from the previous Newton iterate</font></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> for (unsigned int l=0; l<n_dofs; l++)</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> Se(l) = soln(dof_indices[l]);</font><br>
<br><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> Re.resize (n_dofs);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> elmMat=fv.elmmat(perm);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> for (unsigned int i=0; i<n_dofs; i++)</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> {</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> vol=fv.elmvolume(i); </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> xyz=elem->point(i);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> Re(i) = vol*(xyz(0)-0.5) ; </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> for (unsigned int j=0; j<n_dofs; j++)</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> Re(i) += elmMat(i,j)*Se(j) ;</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> } </font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> dof_map.constrain_element_vector (Re, dof_indices);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> residual.add_vector (Re, dof_indices);</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> }</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"> }</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br>
</font><font style="font-size: 10pt;" color="#1f497d" size="2" face="Garamond"><br></font></b><br>and thats it!<br><br>What amazes me is that i always get the right solution after resolution.<br><br>Thanks a lot.<br><br>Stephane<br>
</font></font><br><br><br><br><br><br><hr>Date: Tue, 3 Mar 2009 08:15:15 -0600<br>Subject: Re: petsc-users Digest, Vol 2, Issue 33<br>From: <a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a><br>To: <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
CC: <a href="mailto:tchouanm@msn.com" target="_blank">tchouanm@msn.com</a><div><div></div><div class="h5"><br><br>On Tue, Mar 3, 2009 at 7:54 AM, STEPHANE TCHOUANMO <span dir="ltr"><<a href="mailto:tchouanm@msn.com" target="_blank">tchouanm@msn.com</a>></span> wrote:<br>
<div><blockquote style="padding-left: 1ex;">
<div>
<font style="font-size: 12pt;" size="3" face="Courier New">Hi all,</font><font style="font-size: 12pt;" size="3" face="Courier New"><br><br></font><font style="font-size: 12pt;" size="3" face="Courier New">thank you Barry for the indication you gave me.</font><font style="font-size: 12pt;" size="3" face="Courier New"><br>
<br></font><font style="font-size: 12pt;" size="3" face="Courier New">As a matter of fact, i verified my jacobian and function evaluation again and again but i really dont see anything wrong in it. </font><font style="font-size: 12pt;" size="3" face="Courier New"><br>
</font><font style="font-size: 12pt;" size="3" face="Courier New">So i came back to the basic Laplacian problem (- \Delta u = f ) in the unit cube discretized in regular hexes. The numerical scheme i use is a vertex-centred finite volume scheme.</font><font style="font-size: 12pt;" size="3" face="Courier New"><br>
</font><font style="font-size: 12pt;" size="3" face="Courier New">The solution i get is correct compared to the exact solution (of second order) and i know my jacobian and residual evalutions are correct. But here is the log out i get.</font><font style="font-size: 12pt;" size="3" face="Comic Sans MS"><br>
<br></font><br>Event Count Time (sec) Flops/sec --- Global --- --- Stage --- Total<br> Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s<br>
------------------------------------------------------------------------------------------------------------------------<br><br>--- Event Stage 0: Main Stage<br><br><font color="#ff0000">VecMDot 71 1.0 2.9587e-02 1.0 6.23e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 25 0 0 0 0 25 0 0 0 623</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecNorm 77 1.0 3.3638e-02 1.0 4.24e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 42</font><font color="#ff0000"><br></font><font color="#ff0000">VecScale 74 1.0 2.1052e-03 1.0 3.26e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 326</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecCopy 80 1.0 3.4863e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br></font><font color="#ff0000">VecSet 9 1.0 2.0776e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecAXPY 5 1.0 2.3208e-04 1.0 3.99e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 399</font><font color="#ff0000"><br></font><font color="#ff0000">VecWAXPY 1 1.0 6.6995e-05 1.0 1.38e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 138</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecMAXPY 74 1.0 3.8138e-02 1.0 5.18e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 27 0 0 0 0 27 0 0 0 518</font><font color="#ff0000"><br></font><font color="#ff0000">VecAssemblyBegin 4 1.0 9.8636e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecAssemblyEnd 4 1.0 6.9494e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br></font><font color="#ff0000">VecScatterBegin 3 1.0 3.0706e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#ff0000"><br>
</font><font color="#ff0000">VecNormalize 74 1.0 3.4648e-02 1.0 5.88e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 59</font><br><font color="#00b050">MatMult 73 1.0 1.4618e-01 1.0 2.22e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 45 0 0 0 0 45 0 0 0 222</font><font color="#00b050"><br>
</font><font color="#00b050">MatAssemblyBegin 2 1.0 6.9899e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#00b050"><br></font><font color="#00b050">MatAssemblyEnd 2 1.0 6.1999e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><br>
<font color="#17365d">SNESSolve 1 1.0 6.7333e+01 1.0 1.08e+06 1.0 0.0e+00 0.0e+00 3.0e+00 99100 0 0100 99100 0 0100 1</font><font color="#17365d"><br></font><font color="#17365d">SNESLineSearch 1 1.0 5.1989e-01 1.0 8.91e+04 1.0 0.0e+00 0.0e+00 1.0e+00 1 0 0 0 33 1 0 0 0 33 0</font><font color="#17365d"><br>
</font><font color="#17365d">SNESFunctionEval 2 1.0 1.0441e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 2 0 0 0 67 2 0 0 0 67 0</font><font color="#17365d"><br></font><font color="#17365d">SNESJacobianEval 1 1.0 6.6026e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 97 0 0 0 33 97 0 0 0 33 0</font><br>
<font color="#974806">KSPGMRESOrthog 71 1.0 6.5884e-02 1.0 5.60e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 51 0 0 0 0 51 0 0 0 560</font><font color="#974806"><br></font><font color="#974806">KSPSetup 1 1.0 2.2203e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#974806"><br>
</font><font color="#974806">KSPSolve 1 1.0 2.6036e-01 1.0 2.80e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0100 0 0 0 0100 0 0 0 280</font><font color="#974806"><br></font><font color="#974806">PCSetUp 1 1.0 7.9495e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><font color="#974806"><br>
</font><font color="#974806">PCApply 74 1.0 3.6445e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0</font><br>------------------------------------------------------------------------------------------------------------------------<br>
<br>Memory usage is given in bytes:<br><br>Object Type Creations Destructions Memory Descendants' Mem.<br><br>--- Event Stage 0: Main Stage<br><br> Index Set 3 3 111792 0<br>
Vec 44 3 223596 0<br> Vec Scatter 3 3 0 0<br> Matrix 1 0 0 0<br> SNES 1 0 0 0<br>
Krylov Solver 1 0 0 0<br> Preconditioner 1 0 0 0<br> Viewer 2 0 0 0<br> Draw 1 0 0 0<br>
========================================================================================================================<br>Average time to get PetscTime(): 1.60268e-06<br><br><br><font style="font-size: 12pt;" size="3" face="Courier New">This shows that the Jacobian evaluation takes 97% of time and the residual just 2% in the SNESSolve</font><font style="font-size: 12pt;" size="3" face="Courier New">. But if you look at the total MFlops, you can see that its null(i guess very low) for these phases. What seems to be long is the part in red concerning Vector manips. You can even see at the end that the most memory use is in Index set and Vec.</font><font style="font-size: 12pt;" size="3" face="Courier New"></font></div>
</blockquote><div><br>This analysis does not make sense. If you add all the time spent in the Vec operations (in red), it is less than 1/100 of the time in the<br>SNES Solve. There is obviously a problem in that routine, if there is indeed a problem. Do you have a model of the computation that<br>
says that this time is too long?<br><br> Matt<br> </div></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br><br></div></div><hr>Get news, entertainment and everything you care about at Live.com. <a href="http://www.live.com/getstarted.aspx" target="_blank">Check it out!</a></div>
</blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<br>