<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi,<br>
<br>
I have combined the momentum linear eqns involving x,y,z into 1
large matrix. The Poisson eqn is solved using HYPRE strcut format
so it's not included. I run the code for 50 timesteps (hence 50
kspsolve) using 96 procs. The log_summary is given below. I have
some questions:<br>
<br>
1. After combining the matrix, I should have only 1 PETSc matrix.
Why does it says there are 4 matrix, 12 vector etc? <br>
<br>
2. I'm looking at the stages which take the longest time. It seems
that MatAssemblyBegin, VecNorm, VecAssemblyBegin, VecScatterEnd
have very high ratios. The ratios of some others are also not too
good (~ 1.6 - 2). So are these stages the reason why my code is
not scaling well? What can I do to improve it?<br>
<br>
Btw, I insert matrix using:<br>
<br>
<i>do ijk=ijk_xyz_sta+1,ijk_xyz_end</i><i><br>
</i><i><br>
</i><i> II = ijk - 1</i><i> !Fortran shift to 0-based</i><i><br>
</i><i> </i><i><br>
</i><i> call
MatSetValues(A_semi_xyz,1,II,7,int_semi_xyz(ijk,1:7),semi_mat_xyz(ijk,1:7),INSERT_VALUES,ierr)</i><i><br>
</i><i><br>
</i><i>end do</i><br>
<br>
where ijk_xyz_sta/ijk_xyz_end are the starting/end index<br>
<br>
int_semi_xyz(ijk,1:7) stores the 7 column global indices<br>
<br>
semi_mat_xyz has the corresponding values.<br>
<br>
and I insert vectors using:<br>
<br>
call
VecSetValues(b_rhs_semi_xyz,ijk_xyz_end_mz-ijk_xyz_sta_mz,(/ijk_xyz_sta_mz:ijk_xyz_end_mz-1/),q_semi_vect_xyz(ijk_xyz_sta_mz+1:ijk_xyz_end_mz),INSERT_VALUES,ierr)<br>
<br>
Thanks!<br>
<br>
<i><br>
</i><br>
<pre class="moz-signature" cols="72">Yours sincerely,
TAY wee-beng</pre>
On 30/9/2012 11:30 PM, Jed Brown wrote:<br>
</div>
<blockquote
cite="mid:CAM9tzS=D2gzSy6VENT83fRgTuU5V5c-nB5433dy8ysn1gZS42A@mail.gmail.com"
type="cite">
<p>You can measure the time spent in Hypre via PCApply and
PCSetUp, but you can't get finer grained integrated profiling
because it was not set up that way.</p>
<div class="gmail_quote">On Sep 30, 2012 3:26 PM, "TAY wee-beng"
<<a moz-do-not-send="true" href="mailto:zonexo@gmail.com">zonexo@gmail.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>On 27/9/2012 1:44 PM, Matthew Knepley wrote:<br>
</div>
<blockquote type="cite">On Thu, Sep 27, 2012 at 3:49 AM, TAY
wee-beng <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:zonexo@gmail.com" target="_blank">zonexo@gmail.com</a>></span>
wrote:<br>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"> Hi,<br>
<br>
I'm doing a log summary for my 3d cfd code. I have
some questions:<br>
<br>
1. if I'm solving 3 linear equations using ksp, is the
result given in the log summary the total of the 3
linear eqns' performance? How can I get the
performance for each individual eqn?<br>
</blockquote>
<div><br>
</div>
<div>Use logging stages: <a moz-do-not-send="true"
href="http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Profiling/PetscLogStagePush.html"
target="_blank">http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Profiling/PetscLogStagePush.html</a></div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"> 2.
If I run my code for 10 time steps, does the log
summary gives the total or avg performance/ratio?<br>
</blockquote>
<div><br>
</div>
<div>Total.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"> 3.
Besides PETSc, I'm also using HYPRE's native geometric
MG (Struct) to solve my Cartesian's grid CFD poisson
eqn. Is there any way I can use PETSc's log summary to
get HYPRE's performance? If I use boomerAMG thru
PETSc, can I get its performance?</blockquote>
<div><br>
</div>
<div>If you mean flops, only if you count them yourself
and tell PETSc using <a moz-do-not-send="true"
href="http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Profiling/PetscLogFlops.html"
target="_blank">http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Profiling/PetscLogFlops.html</a></div>
<div><br>
</div>
<div>This is the disadvantage of using packages that do
not properly monitor things :)</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
</div>
</blockquote>
So u mean if I use boomerAMG thru PETSc, there is no proper
way of evaluating its performance, beside using
PetscLogFlops?<br>
<blockquote type="cite">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"> <span><font
color="#888888"><br>
-- <br>
Yours sincerely,<br>
<br>
TAY wee-beng<br>
<br>
</font></span></blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
What most experimenters take for granted before they begin
their experiments is infinitely more interesting than any
results to which their experiments lead.<br>
-- Norbert Wiener<br>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
<br>
</body>
</html>