Baolin Li PhD Proposal Review

Title:
Making Machine Learning on HPC Systems Cost-Effective and Carbon-Friendly

Date:
10/24/2023

Time:
1:00:00 PM

Committee Members:
Prof. Devesh Tiwari (Advisor)
Prof. Ningfang Mi
Dr. Vijay Gadepally

Abstract:
The end users want the machine learning (ML) training/inference services to be lightning-fast. However, the cost, incurred by service providers, to support these lighting-fast ML services is often prohibitively high. Large-scale HPC and data centers are struggling to keep their cost low as they provide faster ML services, while the excessive demand for these services is already negatively impacting our environment due to the large carbon footprint of ML services. Therefore, this dissertation focuses on better understanding the complex trade-off among performance, cost, and environmental footprint of ML models.

This dissertation asks three simple questions: (1) Is slower hardware always worse? (2) Is more expensive hardware always better? (3) Should we always strive to design and train ML models with the highest possible accuracy? As this dissertation reveals, the answers to these questions are more complex than what the conventional wisdom suggests. In fact, simplistic answers — based on first-order intuitions — can lead to missed opportunities in terms of performance efficiency, cost-effectiveness, and carbon footprint.

In this dissertation, we build multiple novel frameworks to demonstrate that mixing slower-and-cheaper hardware with faster-and-expensive hardware can unlock much higher performance- and cost-effectiveness than using only faster-and-expensive hardware configurations. But, unlocking this potential requires a careful design — a design that carefully exploits the diversity in ML inference workload characteristics and adapts to varying ML inference request loads. Next, this dissertation demonstrates that while the highest-possible-accuracy ML models are desirable, using such models can have a severe negative environmental impact. To mitigate this challenge, this dissertation builds an experimental framework to reduce the carbon footprint of ML inference services. The key idea, behind this framework, is to mix the lower-quality ML models with higher-quality ML models intelligently and share the hardware resources during inference query execution to reduce the excessive carbon footprint of high-quality ML model inference, esp. during the periods when a data center’s energy source has high carbon intensity. The extensive experimental evaluation confirms that significant carbon emission reductions can be achieved with transient, very minimal, and configurable loss in accuracy.

As we make rapid advances in the era of large-language models (LLMs) and foundation models, the novel methods and open-source tools presented in this dissertation will enable us to build ML services faster but cheaper and in an environmentally sustainable manner