Huan Wang’s PhD Proposal Review

Title:

Towards Efficient Deep Learning in Computer Vision via Sparsity and Distillation

Committee Members:

Prof. Yun Fu (Advisor)

Prof. Octavia Camps

Prof. Zhiqiang Tao

Abstract:

AI, empowered by deep learning, has been profoundly transforming the world. However, the excessive size of these models remains a central obstacle that limits their broader utility. Modern neural networks commonly consist of millions of parameters, with foundation models extending to billions. The rapid expansion in model size introduces many challenges including training cost, sluggish inference speed, excessive energy consumption, and negative environmental implications such as increased CO2 emissions.

Addressing these challenges necessitates the adoption of efficient deep learning. This thesis focuses on two overarching approaches, network sparsity and knowledge distillation, to enhance the efficiency of deep learning models in the context of computer vision. Network sparsity focuses on eliminating redundant parameters in a model while preserving the performance. Knowledge distillation aims to enhance the performance of the target model, referred to as the “student,” by leveraging guidance from a stronger model, known as the “teacher”. This approach leads to performance improvements in the target model without reducing its size. In the proposal, I will start with the background and major challenges of leveraging these techniques towards efficient deep learning. Then, I shall present the potential solutions in various tasks (e.g., image classification, image super-resolution, neural rendering, and text-to-image generation), with preliminary results to justify the efficacy of the proposed approaches. Finally, a comprehensive outlook of the future work will conclude this proposal.