<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta name=Title content=""><meta name=Keywords content=""><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Arial;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:Arial;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body bgcolor=white lang=EN-US link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:Arial'>What's your problem size (number of elements and polynomial order)?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Arial'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Arial'>Let's assume t_MPI << t (this holds if your problem size is reasonably large). Even in this limit you don't get a linear intra-node speedup simply because Nek5000 is not purely compute bound and the cumulative memory bandwidth is saturated with N cores (N < total number of cores).<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Arial'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Arial'>Cheers,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Arial'>Stefan<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Arial'><o:p> </o:p></span></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-family:Calibri;color:black'>From: </span></b><span style='font-family:Calibri;color:black'><nek5000-users-bounces@lists.mcs.anl.gov> on behalf of <nek5000-users@lists.mcs.anl.gov><br><b>Reply-To: </b><nek5000-users@lists.mcs.anl.gov><br><b>Date: </b>Monday, December 5, 2016 at 6:12 AM<br><b>To: </b><nek5000-users@lists.mcs.anl.gov><br><b>Subject: </b>[Nek5000-users] Parallel speedup on supercomputer Tianhe-2<o:p></o:p></span></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><p class=MsoNormal>Dear Neks,<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I'm using Nek5000 to simulate turbulent Rayleigh-Benard convection, which is governed by the coupled Navier-Stokes equations and convective heat equation. I'm running the code on a supercomputer, Tianhe-2, located in Guangzhou, China. Each computer node in Tianhe-2 has 24 cores (2 Xeon E5 12-core CPUs) and 64GB memory. I find the speedup curve is not linear on a single node. For example, a 24-task job is only 8 times faster than the serial one. However, the performance with an increasing number of nodes is quite good. I don't know whether there is any parameter in nek500 that I can change in order to improve the speedup performance of the individual nodes.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thanks in advance!<o:p></o:p></p></div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>Best regards,<o:p></o:p></p></div><div><p class=MsoNormal>Wei XU<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><p class=MsoNormal>----------<o:p></o:p></p></div><div><p class=MsoNormal>Wei XU<o:p></o:p></p></div><div><p class=MsoNormal>Ph.D. Candidate<o:p></o:p></p></div><div><p class=MsoNormal>Nano Science and Technology Program<o:p></o:p></p></div><div><p class=MsoNormal>The Hong Kong University of Science and Technology<o:p></o:p></p></div></div></div><p class=MsoNormal>_______________________________________________ Nek5000-users mailing list Nek5000-users@lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users <o:p></o:p></p></div></body></html>