- This event has passed.
Yuezhou Liu’s PhD Proposal Review
May 26, 2023 @ 9:00 am - 10:00 am
Prof. Edmund Yeh (Advisor)
Prof. Stratis Ioannidis
Prof. Lili Su
Prof. Carlee Joe-Wong
Significant advances in edge and mobile computing capabilities enable machine learning to occur at geographically diverse locations in networks, e.g., cloud, edge, and mobile devices. The training data needed in those learning tasks may not be fully generated locally. Moreover, some promising distributed learning paradigms enable devices to collaboratively train a model, which requires communication among the devices for exchanging necessary information. Thus, optimizing network strategies for the transmission/exchange of ML/AI ingredients (e.g., input data, model parameters, gradients) is important for facilitating efficient in-network distributed ML. While there exist many works that use ML to optimize network operation strategies, few works study optimized networks that boost ML performance. This dissertation tries to fill the gap by studying several network optimization problems for distributed ML. Different from classic network optimization problems for data delivery or edge computing that optimize energy consumption, delay, throughput, etc., we also pay attention to ML-related metrics such as model accuracy and training convergence time.
We first propose an experimental design network paradigm, wherein learner nodes train possibly different ML models via consuming data streams generated by data source nodes over a network. We formulate this problem as a social welfare optimization problem in which the global objective is defined as the sum of experimental design objectives of individual learners, and the decision variables are the data transmission strategies subject to network constraints. We show that, assuming Bayesian linear regression models and Poisson data streams in steady state, the global objective is continuous DR-submodular, which enables the design of efficient approximate algorithms with approximation guarantees. We will further extend our framework to incorporate more practical ML applications, such as ML with arbitrary nonlinear models.
The second half of this dissertation studies network optimization for Federated learning (FL), a distributed paradigm for collaboratively learning models without having clients disclose their private data. We propose to use caching for improving FL efficiency concerning the total model training time for convergence. Instead of having all clients download the latest global model from a parameter server, we select a subset of clients to access, with smaller delays, a somewhat stale global model stored in caches. We propose CacheFL — a cache-enabled variant of FedAvg, and provide theoretical convergence guarantees in the general setting where the local data is imbalanced and heterogeneous. With this result, we determine the caching strategies that minimize total wall-clock training time at a given convergence threshold for both stochastic and deterministic communication/computation delays.