general question on speed using quad core Xeons

[amjad ali]

Hello all, specially Dr. Matt,

On 4/16/08, Matthew Knepley <knepley at gmail.com> wrote:
>
> On Tue, Apr 15, 2008 at 7:19 PM, Randall Mackie <rlmackie862 at gmail.com>
> wrote:
> > I'm running my PETSc code on a cluster of quad core Xeon's connected
> >  by Infiniband. I hadn't much worried about the performance, because
> >  everything seemed to be working quite well, but today I was actually
> >  comparing performance (wall clock time) for the same problem, but on
> >  different combinations of CPUS.
> >
> >  I find that my PETSc code is quite scalable until I start to use
> >  multiple cores/cpu.
> >
> >  For example, the run time doesn't improve by going from 1 core/cpu
> >  to 4 cores/cpu, and I find this to be very strange, especially since
> >  looking at top or Ganglia, all 4 cpus on each node are running at 100%
> > almost
> >  all of the time. I would have thought if the cpus were going all out,
> >  that I would still be getting much more scalable results.
>
> Those a really coarse measures. There is absolutely no way that all cores
> are going 100%. Its easy to show by hand. Take the peak flop rate and
> this gives you the bandwidth needed to sustain that computation (if
> everything is perfect, like axpy). You will find that the chip bandwidth
> is far below this. A nice analysis is in
>
> http://www.mcs.anl.gov/~kaushik/Papers/pcfd99_gkks.pdf
>
> >  We are using mvapich-0.9.9 with infiniband. So, I don't know if
> >  this is a cluster/Xeon issue, or something else.
>
> This is actually mathematics! How satisfying. The only way to improve
> this is to change the data structure (e.g. use blocks) or change the
> algorithm (e.g. use spectral elements and unassembled structures)


Would you please explain a bit about "unassembled structures"?
Does Discontinuous Galerkin Method falls into this category?

Thanks and Regrads,
Amjad Ali.

Matt
>
> >  Anybody with experience on this?
> >
> >  Thanks, Randy M.
> >
> >
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20080523/eb482d9c/attachment.htm>