<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On , 2021Jan28, at 10:35, Jed Brown <<a href="mailto:jed@jedbrown.org" class="">jed@jedbrown.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">
<blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
Why plots like this are not _absolutely standard_ on all HPC sites' webpages is a source of continuing mystery to me.<br class="">
</blockquote>
<br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">I've
been asking for it for years.</span></div>
</div>
</blockquote>
<div><br class="">
</div>
Curious. Why? What would it tell you? I mean, other than “42”.</div>
<div><br class="">
</div>
<div>I’ll ask around how this is done in TX. Considering all the tinkering Intel does to their MPI and what with all the UCX crap, we keep a close eye on MPI performance, especially collectives. </div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">latency
is super variable (depending on the partition you get and what other jobs are running elsewhere on the machine; Blue Gene famously didn't have that problem).</span></div>
</div>
</blockquote>
</div>
<br class="">
<div class="">The harsh truth of fat-trees with static routing:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://arxiv.org/abs/1909.12195" class="">https://arxiv.org/abs/1909.12195</a></div>
<div class=""><br class="">
</div>
<div class="">Btw, I think we’ve turned off dynamic routing for now because of too many problems.</div>
<div class=""><br class="">
</div>
<div class="">Victor.</div>
</body>
</html>