[petsc-users] How to confirm the performance of asynchronous computations

[Jed Brown]

Lawrence Mitchell <wencel at gmail.com> writes:

>> On 27 Jan 2021, at 16:30, Matthew Knepley <knepley at gmail.com> wrote:
>> 
>> This is very important to do _first_. It would probably only take you a day to measure the Allreduce time on your target, say the whole machine you run on.
>
> Why plots like this are not _absolutely standard_ on all HPC sites' webpages is a source of continuing mystery to me.

I've been asking for it for years. They say if you care, you should just go run it. Never mind how wasteful that is, and the time commitment to doing so. I think they often avoid making a commitment because latency is super variable (depending on the partition you get and what other jobs are running elsewhere on the machine; Blue Gene famously didn't have that problem).

Meanwhile, latency on cloud providers keeps dropping and they're sure to beat conventional HPC centers to publishing a dashboard of expected latency for different configurations.  

This page illustrates how hardware reductions scale much better than log(P).

https://www.mcs.anl.gov/~fischer/gop/

> Although I guess Figure 2 from here https://www.mcs.anl.gov/papers/P5347-0515.pdf probably gives me a clue.
>
> Viet, I suspect that Matt thinks you should try and produce a figure like Figure 3 from that linked paper.
>
> Lawrence