<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<div class="moz-cite-prefix">On 1/3/2018 10:07 AM, Matthew Knepley
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAMYG4GnbZc4RjadOxoKjVKFZfjX045awmahw9srJyXyVSmuRmg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Feb 28, 2018 at 9:01 PM, TAY
wee-beng <span dir="ltr"><<a
href="mailto:zonexo@gmail.com" target="_blank"
moz-do-not-send="true">zonexo@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> <br>
<div class="m_8518435943966620320moz-cite-prefix">On
1/3/2018 12:10 AM, Matthew Knepley wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Feb 28, 2018 at
10:45 AM, TAY wee-beng <span dir="ltr"><<a
href="mailto:zonexo@gmail.com"
target="_blank" moz-do-not-send="true">zonexo@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">Hi,<br>
<br>
I have a CFD code which uses PETSc and HYPRE.
I found that for a certain case with grid size
of 192,570,048, I encounter scaling problem
when my cores > 600. At 600 cores, the code
took 10min for 100 time steps. At 960, 1440
and 2880 cores, it still takes around 10min.
At 360 cores, it took 15min.<br>
<br>
So how can I find the bottleneck? Any
recommended steps?<br>
</blockquote>
<div><br>
</div>
<div>For any performance question, we need to
see the output of -log_view for all test
cases.</div>
</div>
</div>
</div>
</blockquote>
Hi,<br>
<br>
To be more specific, I use PETSc KSPBCGS and HYPRE
geometric multigrid (entirely based on HYPRE, no PETSc)
for the momentum and Poisson eqns in my code.<br>
<br>
So can log_view be used in this case to give a
meaningful? Since part of the code uses HYPRE?<br>
</div>
</blockquote>
<div><br>
</div>
<div>Make an event to time the HYPRE solve. It only takes a
few lines of code.</div>
</div>
</div>
</div>
</blockquote>
Hi,<br>
<br>
I check PETSc and found some routines which can be used to time the
HYPRE solve, like PetscGetTime and PetscGetCPUTime.<br>
<br>
And then using:<br>
<br>
PetscLogDouble t1, t2;<br>
<br>
ierr = PetscGetCPUTime(&t1);CHKERRQ(ierr);<br>
... code to time ...<br>
ierr = PetscGetCPUTime(&t2);CHKERRQ(ierr);<br>
printf("Code took %f CPU seconds\n", t2-t1);<br>
<br>
Are these 2 routines suitable? Which one should I use?<br>
<blockquote type="cite"
cite="mid:CAMYG4GnbZc4RjadOxoKjVKFZfjX045awmahw9srJyXyVSmuRmg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> I also program
another subroutine in the past which uses PETSc to solve
the Poisson eqn. It uses either HYPRE's boomeramg,
KSPBCGS or KSPGMRES.<br>
<br>
If I use boomeramg, can log_view be used in this case?<br>
</div>
</blockquote>
<div><br>
</div>
<div>Yes, its automatic.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> Or do I have to use
KSPBCGS or KSPGMRES, which is directly from PETSc?
However, I ran KSPGMRES yesterday with the Poisson eqn
and my ans didn't converge.<br>
</div>
</blockquote>
<div><br>
</div>
<div>Plain GMRES is not good for Poisson. You would be
better off with GMRES/GAMG.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> Thanks.<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex"> I must also mention
that I partition my grid only in the x and y
direction. There is no partitioning in the z
direction due to limited code development. I
wonder if there is a strong effect in this
case.</blockquote>
<div><br>
</div>
<div>Maybe. Usually what happens is you fill up
memory with a z-column and cannot scale
further.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex"><span
class="m_8518435943966620320HOEnZb"><font
color="#888888"><br>
-- <br>
Thank you very much<br>
<br>
Yours sincerely,<br>
<br>
==============================<wbr>==================<br>
TAY Wee-Beng 郑伟明 (Zheng Weiming)<br>
Personal research webpage: <a
href="http://tayweebeng.wixsite.com/website"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://tayweebeng.wixsite.com/<wbr>website</a><br>
Youtube research showcase: <a
href="https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://www.youtube.com/channe<wbr>l/UC72ZHtvQNMpNs2uRTSToiLA</a><br>
linkedin: <a
href="http://www.linkedin.com/in/tay-weebeng"
rel="noreferrer" target="_blank"
moz-do-not-send="true">www.linkedin.com/in/tay-weeben<wbr>g</a><br>
==============================<wbr>==================<br>
<br>
</font></span></blockquote>
</div>
<br>
<br clear="all">
<span class="HOEnZb"><font color="#888888">
<div><br>
</div>
-- <br>
<div
class="m_8518435943966620320gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>What most experimenters take for
granted before they begin their
experiments is infinitely more
interesting than any results to
which their experiments lead.<br>
-- Norbert Wiener</div>
<div><br>
</div>
<div><a
href="http://www.caam.rice.edu/%7Emk51/"
target="_blank"
moz-do-not-send="true">https://www.cse.buffalo.edu/~<wbr>knepley/</a><br>
</div>
</div>
</div>
</div>
</div>
</font></span></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>What most experimenters take for granted before
they begin their experiments is infinitely more
interesting than any results to which their
experiments lead.<br>
-- Norbert Wiener</div>
<div><br>
</div>
<div><a href="http://www.caam.rice.edu/%7Emk51/"
target="_blank" moz-do-not-send="true">https://www.cse.buffalo.edu/~knepley/</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>