[hpc-announce] Call for Participation: Deep500 BOF at Supercomputing 2018

[Tal Ben-Nun]

SC18 BOF on Deep500: An HPC Deep Learning Benchmark and Competition
https://www.deep500.org/

Wednesday, November 14, 2017
5:15 PM - 6:45 PM
Room D221
Kay Bailey Hutchinson Convention Center
Dallas, TX

Short summary and topics discussed:
Machine Learning (ML), and particularly Deep Learning, will soon become necessary tools in the scientific computing toolbox. The recent success of ML has sparked research into scaling up the underlying problems and mechanisms in use. The goal of this BOF is to standardize a new benchmark, which necessitates a collaborative brainstorming of the entire HPC community. 

The BOF aims to define a standard set of competitions and metrics for Deep Learning in scientific computing. In particular, a panel of experts will discuss the following topics:
    * What are the problems that we should test? In particular, is it possible to maintain a set of relevant models and training algorithms, which are in use by the scientific computing community?
    * Given the multitude of datasets, performance metrics, and robustness to accuracy (e.g., mixed-precision solutions), how should the competitors be ranked? How should the unit of measurement ("AI Operations") be defined?
    * How can we adequately represent all the layers that constitute deep neural network training? Such layers include benchmarking different hardware, software stacks, distributed communication mechanisms, etc.
    * As a scientific computing problem, how does reproducibility of the results come into play?

Organizers: Tal Ben-Nun (ETH Zurich), Torsten Hoefler (ETH Zurich)
Panel Members: Pradeep Dubey (Intel), Todd Gamblin (LLNL), Tom Gibbs (NVIDIA), Torsten Hoefler (ETH Zurich), Thorsten Kurth (LBL), Satoshi Matsuoka (Tokyo Institute of Technology), Jidong Zhai (Tsinghua University)

Agenda:
5:15 - 6:00     State of the Practice (short talks by panel members)
6:00 - 6:15     Topic 1: Set of problems and problem sizes, including models and datasets
6:15 - 6:30     Topic 2: Metrics and ranking
6:30 - 6:45     Topic 3: Reproducibility and infrastructure