<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Jed,<br>
<br>
On 9/1/2012 5:07 PM, Jed Brown wrote:
<blockquote
cite="mid:CAM9tzS=HkJDba0gEPuUG88bwBwr3KYvru6aJWePzTCz1e9H6mw@mail.gmail.com"
type="cite">
<div class="gmail_quote">On Mon, Jan 9, 2012 at 09:57, TAY
wee-beng <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:zonexo@gmail.com">zonexo@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">I only compute rows
which the processor own. Can it be the memory allocation?
I'll check on that. <br>
</div>
</blockquote>
<div><br>
</div>
<div>Usually memory allocation mistakes cost much more.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div class="im">
<blockquote type="cite">
<div class="gmail_quote">
<div><br>
</div>
<div>Most of your solve time is going into PCSetUp()
and PCApply, both of which are getting more
expensive as you add processes. These are more than
10x more than spent in MatMult() and MatMult() takes
slightly less time on more processes, so the
increase isn't entirely due to memory issues.</div>
<div><br>
</div>
<div>What methods are you using?</div>
</div>
</blockquote>
<br>
</div>
What do you mean methods? I am doing Cartesian grid 3d CFD,</div>
</blockquote>
<div><br>
</div>
<div>Then just use a regular partition. If your meshes have
boundary layers, you might want to adjust the subdomain aspect
ratios so that strongly coupled cells tend to reside on the
same process, but don't bother with general graph partitions
(like ParMETIS) because you will end up writing most of an
unstructured CFD code in order to use those partitions
efficiently.</div>
</div>
</blockquote>
<br>
Can you explain a bit more about how to adjust the subdomain aspect
ratios so that strongly coupled cells tend to reside on the same
process?
<blockquote
cite="mid:CAM9tzS=HkJDba0gEPuUG88bwBwr3KYvru6aJWePzTCz1e9H6mw@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> using fractional mtd
which solves the momentum and Poisson eqns. I construct the
linear eqn matrix and insert them in PETSc matrix/vectors.
Then I solve using Bicsstab and hypre AMG respectively. Why
is PCSetUp() and PCApply using more time?</div>
</blockquote>
</div>
<br>
<div>It is expensive because BoomerAMG setup and apply is
expensive.</div>
</blockquote>
<br>
So this is normal? Is there any suggestion to improve performance?<br>
<br>
Thanks!<br>
</body>
</html>