[hpc-announce] Call for Papers - First International Workshop on Distributed infrastructures for Foundation Models

[Vatche Isahagian]

-----------------------------------------------------------------------------------------------
Please accept our apologies if you receive multiple copies of this CFP
-----------------------------------------------------------------------------------------------

The DIFM workshop is co-located with ACM/IFIP Middleware 2023, which takes
place from December 11-15 in Bologna, Italy.

Following the innovations in deep learning, foundation models (FM) are the
next evolution in machine learning. Foundation models, including large
language models, are driving the recent breakthroughs around conversational
AI chatbots (such as ChatGPT), image generation (such as Stable Diffusion),
and code assistants (such as GitHub CoPilot).

Foundation models across text, speech, and vision domains can be trained at
scale using self-supervision techniques and applied to a broad set of
downstream tasks. They address a limitation of deep learning models which
typically required large task-specific labeled datasets. There is
tremendous activity in this space from large enterprises, such as Google,
Microsoft, Amazon, and IBM; and startups, such as OpenAI,Anthropic,
Stability AI, and Cohere. As well there have been a flurry of papers in
this space from academia and the open source community, resulting in models
such as Alpaca, and efforts such as OpenAssistant.

Much of the attention has been on the models and datasets, with academic
communities relying on the training and serving infrastructure provided by
large cloud providers. Innovations on the infrastructure aspects have
largely been taking place in closed enterprises and startups. This workshop
will serve as a venue for academics and practitioners to share their
findings, visions, and ideas around these infrastructure challenges and
concerns. There are still challenging open problems that need attention.
One piece of evidence of these challenges is OpenAI’s GPT-4 technical
report. While this report is light on technical details, it includes an
extensive Acknowledgements section that listed large dedicated teams
focused on infrastructure aspects such as “Compute cluster scaling”,
“Distributed training infrastructure”, “Hardware correctness”, “Training
run babysitting”, “Deployment & post-training”, “Data infrastructure”,
“Acceleration forecasting”, “Inference research”, “Inference
infrastructure”, and “Reliability engineering”. This suggests the
importance of middleware infrastructure to train and serve FMs, and the
need for research in this space.

The scope of this workshop includes, but is not limited to:

Resource scheduling algorithms and optimizations for FM serving workloads,
including batch, streaming, and synchronous invocation patterns.
Novel techniques to train large FMs.
Frameworks for fine-tuning FMs.
Programming model abstractions for FMs (such as LangChain).
Case studies of FM middleware.
Novel debugging and logging techniques, both in cases of black-box FMs
available through an API, and locally available FMs.
Deployment of FMs in resource constrained environments (such as edge
platforms, web browsers, and mobile devices).

Dates and location
Paper submissions: September 25, 2023
Notification to authors: October 20, 2023
Camera-ready copy due: October 27, 2023

Papers and Submissions
We are looking for the following types of submissions:
Research and industry papers (up to 8 pages): Reports on original results
including novel techniques, significant case studies or surveys. Authors
may include extra material beyond the six pages as a clearly marked
appendix, which reviewers are not obliged to read but could read.
Position papers (up to 4 pages): Reports identifying unaddressed problems
and research challenges.
Abstracts (up to 1 page): An extended abstract on a preliminary or ongoing
work.
Papers must be written in English and submitted in PDF format. All papers
should follow ACM formatting instructions, specifically the ACM SIG
Proceedings Standard Style. The author kit containing the templates for the
required style can be found at
http://www.acm.org/publications/proceedings-template.

Submissions should not be blinded for review. Please submit your papers via
the submission site: https://difm23.hotcrp.com/

All accepted papers will appear in the Middleware 2023 companion
proceedings, available in the ACM Digital Library. All accepted papers will
also be presented at the workshop, and at least one author of each paper
must register for the workshop.

Workshop Co-chairs
Bishwaranjan Bhattacharjee, IBM Research
Vatche Isahagian, IBM Research
Vinod Muthusamy, IBM Research
Program Committee (Tentative)
Parag Chandakkar, Walmart Labs
Ian Foster, Argonne National Laboratory and the University of Chicago
Matthew Hill, Dataminr
Mayoore Jaiswal, Nvidia
Gauri Joshi, Carnegie Mellon University
Jayaram K. R., IBM Research
Ruben Mayer, Technical University of Munich
Pietro Michiardi, Eurecom
Phuong Nguyen, eBay
Peter Pietzuch, Imperial College
Chuan Wu, University of Hong Kong