[hpc-announce] CFP: Computational Reproducibility at Exascale (CRE2019) @ SC19

[Keyrouz, Walid (Fed)]

Call for Participation
Computational Reproducibility at Exascale Workshop (CRE2019)


Where           In cooperation with SC19, Dallas, TX
When            Sunday, November 17, 2019, 9:00 am–12:30 pm
Web             http://www.cs.fsu.edu/~cre
Submit          http://bit.ly/CRE2019-submit
Deadline        Monday, August 26, 2019
Notifications   Monday, September 16, 2019
Full Papers     Monday, September 30, 2019
Organized by    Walid Keyrouz (NIST) and Michael Mascagni (FSU & NIST)


Overview
========

Experimental reproducibility is a cornerstone of the scientific
method.  As computing has grown into a powerful tool for scientific
inquiry, computational reproducibility has been one of the core
assumptions underlying scientific computing.  With "traditional"
single-core CPUs, documenting a numerical result was relatively
straightforward.  However, hardware developments over the past several
decades have made it almost impossible to ensure computational
reproducibility or to even fully document a computation without
incurring a severe loss of performance.  This loss of reproducibility
started when systems combined parallelism (e.g., clusters) with
non-determinism (e.g., single-core CPUs with out-of-order execution).
It has accelerated with recent architectural trends towards platforms
with increasingly large numbers of processing elements, namely
multicore CPUs and compute accelerators (GPUs, Intel Xeon Phi, FPGAs).

Programmers targeting these platforms rely on tools and libraries to
produce codes or execute them efficiently.  As a result, codes can run
efficiently, but have execution details that can be impossible to
predict and are often very difficult to understand after execution.
Furthermore, parallel implementations often result in code with
varying execution orders between runs, leading to non-reproducible
computations.  The underlying reasons are that (1) the hardware and
system software allocate parallel work in ways that are not always
specifiable at compile time and (2) the execution often proceeds in an
opportunistic manner with the execution order changing between runs.
As such, floating-point computations, which are not commutative and
associative, can have different execution orders and execute on
different processing elements between runs, leading to runs with
varying results as a matter of fact.  The predictability of systems is
further complicated by two issues that are becoming more critical as
systems grow in scale: (1) interconnect systems with latencies that
are often outside the control of programmers and (2) reliability as
the mean time between failure (MTBF) is now measured in hours on large
systems.  This situation seriously affects the ability to rely on
scientific computations as a metrological substitute for
experimentation!


Previous Offerings
==================

This workshop combines the Numerical Reproducibility at Exascale
workshops (conducted in 2015 and 2016 at SC) and the panel on
Reproducibility held at SC'16 (originally a BOF at SC'15) to address
several different issues in reproducibility that arise when computing
at Exascale.  The first combined workshop, Computational
Reproducibility at Exascale 2017 (CRE2017), took place at SC17 and was
a success; its web page can be found http://www.cs.fsu.edu/
cre/cre2017here; the following edition, CRE2018, took place at SC18
and its web page can be found at http://www.cs.fsu.edu/
cre/cre2018here.  Previous editions of the workshop were NRE2015 and
NRE2016; their web pages can be found
http://www.nist.gov/itl/ssd/is/numreprod2015.cfmhere and
http://www.cs.fsu.edu/ cre/nre-2016.htmlhere respectively.  We began
to offer the International version of this workshop in 2019 as NRE2019
at the ISC-HPC conference in Germany; its website is
http://www.cs.fsu.edu/ cre/cre-2019/index.htmlhere.


Workshop Scope
==============

The workshop is meant to address issues of numerical reproducibility
as well as approaches and best practices to sharing and running code
and the reproducible dissemination of computational results.  The
workshop is meant to address the scope of the problems of
computational reproducibility in HPC in general, and those anticipated
as we scale up to Exascale machines in the next decade.  The
participants of this workshop will include government, academic, and
industry stakeholders; the goals of this workshop are to understand
the current state of the problems that arise, what work is being done
to deal with these issues, and what the community thinks the possible
approaches to these problems are.

The workshop is meant to address the scope of the problems of
numerical reproducibility in HPC in general and those anticipated as
we scale to Exascale machines in the next decade.  We initially seek
contributions of extended abstracts (two pages) in the areas of
computational reproducibility in HPC from academic, government, and
industry stakeholders.  Areas of interest include, but are not limited
to:

- Case studies of reproducibility or the lack of it
- Reproducibility issues in current HPC
- System-level solutions
- Algorithmic solutions
- Software solutions
- Uncertainty quantification in computational reproducibility
- Fundamental numerical analysis of reproducibility
- Future prospects


Workshop Format
===============

The workshop will have: (1) two plenary talks (25 min + 5 min QA
each), (2) four contributed talks (15 min + 5 min QA each), and (3)
a 40-min panel discussion to summarize the problem, current research,
and prospects on long-term solutions.  The table below gives the
workshop’s schedule:

Start      Speaker   Title
9:00 am    TBD       Plenary I
9:30 am    Talk 1    TBD
9:50 am    Talk 2    TBD
10:10 am   Talk 3    TBD
10:30 am   Break     Food and coffee provided by SC
11:00 am   TBD       Plenary II
11:30 am   Talk 4    TBD
11:50 am   Panel     Panel Discussion on Computational Reproducibility

Papers submitted to the workshop will be reviewed.  The referees will
select the papers that will be presented in the workshop.  In
addition, a group of papers will be published by IEEEin a special issue of
the International Journal of High-Performance Computing and
Applications (IJHPCA) devoted to Computational Reproducibility.
Please note that papers submitted to the IJHPCA for the CRE2017
special issue must fall within the IJHPCA’s editorial scope.  This
primarily means that all papers for the special issue have relevance
to high-performance computing..


Submissions
===========

Submissions of two-page extended abstracts are sought.  The format for
the abstracts and full papers should follow the IEEE Conference
Proceedings format.  Templates are available at
https://www.ieee.org/conferences_events/conferences/publishing/templates.htmlIEEE
- Manuscript Templates for Conference Proceedings.  The full papers
should be 12 pages or less.

The abstracts are to be submitted as a PDF document using Easychair at http://bit.ly/CRE2019-submit.


Important Dates
===============

Mon., Aug. 26, 2019:   submission deadline for two page abstracts via
                       http://bit.ly/CRE2019-submit

Mon., Sep. 16, 2019:   notification of authors about their submissions
                       based on rejection, acceptance as a paper,
                       acceptance as a paper and presentation

Mon., Sep. 30, 2019:   submission deadline for full papers for
                       refereeing; site will be announced later.


Organizers and Co-Editors
=========================

Walid Keyrouz, National Institute of Standards and Technology (NIST), USA

Michael Mascagni, National Institute of Standards and Technology (NIST) and Florida State University, USA


Steering Committee
==================

Dong H. Ahn, Lawrence Livermore National Lab, USA
David Bailey, UC Davis, USA
Xuebin Chi, Supercomputing Center, Chinese Academy of Sciences, China
Mike Heroux, Sandia National Laboratory, USA
Torsten Hoefler, ETH-Zurich, Switzerland
Lisa Kempler, MathWorks
Walid Keyrouz (co-organizer), NIST, USA
Xiaoye Sherry Li, Lawrence Berkeley National Laboratory, USA
Yaohang Li, Old Dominion University, USA
Michael Mascagni (co-organizer), FSU/NIST, USA
Junji Nagano, Insitute of Statistical Mathematics, Japan
Nathalie Revol, INRIA/ENS-Lyon, France
Siegfried Rump, University of Hamburg, Germany
Michela Taufer, University of Delaware


Contact
=======

E-mail: numerical.reproducibility.at.nist.gov (replace “.at.” by "@")


                -Walid

------

Walid Keyrouz, PhD
Research Scientist
NIST | ITL | SSD | ISG
+1 (301) 975-4031