Northeastern University College of Engineering

Graduate Student Fellowship Writing Club-Hosted by NU CommLab and NetSI
2024-07-31 15:00 - 16:00, 2024-08-07 15:00 - 16:00, 2024-08-14 15:00 - 16:00 (America/New_York)
Join the NU CommLab and NetSI sponsored weekly graduate student fellowship writing club for support in writing your fellowship application! The fellowship writing club meets virtually on Wednesdays from 3-4pm from July 10- August 21. We will offer you an opportunity to ask questions to faculty, staff and students who have reviewed, mentored or applied and received fellowships. We will provide fellowship writing tips and guidance as well as offer writing and draft review sessions. Register to join our Zoom Sessions.

LinkedIn, CV, Resume: CommLab Drop-In Workshops
2024-07-31 16:00 - 17:00
Join the CommLab’s empowering LinkedIn, CV, and Resume Drop-In Workshops any Wednesday from 4 pm to 5 pm ET. This collaborative space offers valuable advice and peer feedback to enhance your online profile and professional presence. Join this workshop series through Zoom.

Mock Interview: CommLab Drop-In Workshops
2024-08-01 16:00 - 17:00, 2024-08-08 16:00 - 17:00, 2024-08-15 16:00 - 17:00
Join the CommLab any Thursday from 4-5pm ET, we’ll delve into the intricacies of interviews, unveiling effective preparation strategies for any interview scenario. Engage in an interactive setting as we dissect the overall interview experience, discuss common interview scenarios, and share insights on what to do during critical moments. Join this hybrid workshop series through Zoom.

Poster Design and Presentation: CommLab Drop-In Workshops
2024-08-01 17:00 - 18:00, 2024-08-08 17:00 - 18:00, 2024-08-15 17:00 - 18:00
The CommLab will host drop-in workshops for poster design and presentation to focus on crafting the best visual communication of your research and telling your research story! We will discuss techniques and implement communication strategies to successfully showcase your work. No matter where you are in the process, whether it is just in the idea phase or you are trying to polish your final poster, we are happy to help you. Join us any Thursday from 5-6pm, on Zoom.

CommLab Drop-In Writing Hours
2024-08-02 11:00 - 12:30, 2024-08-09 11:00 - 12:30, 2024-08-16 11:00 - 12:30 | 13 International Village, 360 Huntington Ave, 13 INV, Boston, MA, 02115, United States
Graduate students, are you looking for a place for focused research writing time? Join the CommLab drop-in writing hours any Friday from 11 am-12:30 pm ET. Drop in any Friday and stay for a short time or the whole hour and a half. CommLab Fellows will be available to provide feedback on your writing. We will be meeting in 13 International Village.

Graduate School of Engineering Campus Tour - In Person
2024-08-03 13:00 - 14:00
Want to learn more about Northeastern’s Boston campus? Then we welcome you to sign up for a Graduate School of Engineering campus tour! Led by one of our expert Graduate Student Ambassadors, we’ll show you key locations on campus, in addition to resources specific to Engineering, and answer your questions about Boston. Please complete the registration form linked below to select the date and time that works best for you. Tours are open to all students interested in learning more about the Graduate School of Engineering. We can’t wait to meet you!

Joshua Groen PhD Proposal Review
2024-08-05 14:00 - 15:00
Name:
Joshua Groen

Title:
Optimizing and Securing Open RAN with Experimental System Validation
Date:
8/5/2024
Time:
2:00:00 PM
Location:
ISEC232;
Committee Members:
Prof. Kaushik Chowdhury (Advisor)
Prof. Stratis Ioannidis
Prof Engin Kirda
Dr. Christopher Morrell
Abstract:
5G and beyond cellular networks promise remarkable advancements in bandwidth, latency, and connectivity, with the emergence of Open Radio Access Network (Open RAN) representing a pivotal direction. O-RAN inherently supports machine learning (ML) for network operation control, with RAN Intelligence Controllers (RICs) utilizing ML models developed by third-party vendors based on key performance indicators (KPIs) from geographically dispersed base stations or user equipment (UE). Realistic and robust datasets are crucial for developing these ML models. We collect a comprehensive 5G dataset using real-world cell phones across diverse scenarios and replicate this traffic within a full-stack srsRAN-based O-RAN framework on Colosseum, the world’s largest radio frequency (RF) emulator. This process produces a robust, O-RAN compliant KPI dataset reflecting real-world conditions, enabling the training of ML models for traffic slice classification with high accuracy.
The O-RAN paradigm introduces cloud-based, multi-vendor, open, and intelligent architectures, enhancing network observability and reconfigurability. However, this also expands the threat surface, exposing components and ML infrastructure to cyberattacks. We examine O-RAN security, focusing on specifications, architectures, and intelligence proposed by the O-RAN Alliance. We identify threats, propose solutions, and experimentally demonstrate their effectiveness in defending O-RAN systems against cyberattacks, offering a holistic and practical perspective on O-RAN security.
We investigate the impact of encryption on two key O-RAN interfaces: the E2 interface and the Open Fronthaul, using a full-stack O-RAN ALLIANCE compliant implementation within the Colosseum network emulator and a production-ready Open RAN and 5G-compliant private cellular network. Our findings provide quantitative insights into the latency and throughput impacts of encryption protocols, and we propose four fundamental principles for security by design within Open RAN systems.
Finally, we address the security of Time-Sensitive Networking (TSN) in O-RAN. The O-RAN framework encourages multi-vendor solutions but increases the exposure of the open fronthaul (FH) to security risks, especially when deployed over third-party networks. Synchronization is crucial for reliable 5G links, with attacks on synchronization mechanisms posing significant threats. We demonstrate the impact of spoofing and replay attacks on Precision Time Protocol (PTP) synchronization, causing catastrophic failures in a production-ready O-RAN and 5G-compliant private cellular network. To counter these threats, we design an ML-based monitoring solution detecting various malicious attacks with over 97.5% accuracy, and outline additional security measures for the O-RAN environment.

Malith Jayaweera PhD Dissertation Defense
2024-08-06 10:00 - 11:00
Name:
Malith Jayaweera
Title:
Energy-Aware Transformations for Affine Programs on GPUs
Date:
8/6/2024
Time:
10:00:00 AM
Committee Members:
Prof. David Kaeli (Co-advisor)
Prof. Yanzhi Wang (Co-advisor)
Dr. Norman Rubin
Prof. Martin Kong (Ohio State University)
Abstract:
Graphics Processing Units (GPUs) have been increasingly used to accelerate workloads ranging from high performance computing to machine learning. Development of high-level programming languages, improved compilers, and runtime drivers have helped to accelerate the widespread adoption of GPUs. Given the wider adoption and ever-increasing computing capabilities, the power consumption of GPUs is quickly becoming a critical factor. Furthermore, the GPU micro-architecture differs from vendor to vendor, and even between hardware generations of the same vendor. Also, program variants with similar performance could differ in energy consumption due to the difference in utilization of GPU resources such as Streaming Multiprocessors (SMs) or memory. Despite performance improvements in compilation techniques, energy-aware code generation for heterogeneous GPUs has not been aggressively explored.
In this dissertation, we first identify the potential for energy-aware compilation techniques for GPUs. Next, we use these insights to study loop tiling, which is a popular loop transformation that has been successfully applied to computational domains such as linear algebra, deep neural networks and iterative stencils. We then propose an energy-aware tile size selection for affine programs to generate energy-efficient code targeting GPUs.
We also investigate the challenging problem of optimizing the scheduling of complex sparse tensor algebra and expressions on GPUs, with a focus on maximizing parallelism utilization to unlock optimal performance. We perform a comprehensive examination of the search space for sparse tensor expression scheduling, seeking to characterize the intricate inter-relationships between kernel characteristics, GPU architecture, and hardware constraints such as memory bandwidth limitations, to inform optimal scheduling decisions.

LeetCode Mock Interviews – CommLab Drop-In Workshops
2024-08-06 16:00 - 17:00, 2024-08-13 16:00 - 17:00
Join the CommLab any Tuesday from 4-5 PM for our weekly LeetCode Mock Interview Workshop via Zoom. This workshop is tailored towards programming jobs and prior coding knowledge is expected. Boost your LeetCode problem-solving confidence for interviews by building your speaking skills while solving programming problems.

Kubra Alemdar PhD Dissertation Defense
2024-08-07 11:00 - 12:00
Name:
Kubra Alemdar
Title:
Overcoming and Engineering Wireless Signals for Communication and Computation
Date:
8/7/2024
Time:
11:00:00 AM
Committee Members:
Prof. Kaushik Chowdhury (Advisor)
Prof. Josep Jornet
Prof. Marvin Onabajo
Abstract:
The phenomenal growth of connected devices, especially rapid expansion of IoT networks and the increasing demand for wireless services are the main driving forces for the evolution of wireless technologies. However, the realization of such technologies requires a radical transformation of existing infrastructures to satisfy the needs of changing wireless environments. The main limitation in delivering these systems stems from a vast diversity in their demands and constraints. To address this limitation, this dissertation shows how wireless signals and their interaction with and within the wireless propagation domain can be used as communication or computational tools that enable us to achieve certain novel tasks. Specifically, we build i) cross-functionality architectures to engineer the wireless channel to a) enable the operation of emerging technologies, and b) demonstrate a new paradigm for computing with wireless signals, and ii) intelligently shape the wireless channel to create reliable communication links. This dissertation presents an experimentally validated software-hardware systems with thorough analysis, delivering the following key advancements with distinct contributions:
First, We present an innovative physical layer solution for distributed networks that provides over-the-air (OTA) clock synchronization, known as RFCLOCK, to overcome the hurdle of implementing fine-grained synchronization for emerging technologies. We first develop the theory for such precision synchronization, and second implement it in a custom-design, compatible with commercial-off-the-shelf (COTS) software-defined radios (SDRs). We compare the performance of RFClock with popular wired and GPS-based hardware solutions, both in terms of clock performance as well as impact on distributed beamforming.
Next, we propose two novel approaches, utilizing reconfigurable intelligent surfaces (RISs) to ensure reliable connectivity in wireless networks by controlling the propagation environment: i) we present RIS-based spatio-temporal approach to enhance the link reliability for IoTs where sensors are small-factor designs with single-antenna in a rich multipath environment. We demonstrate the design of RIS and how it can effectively perturb the environment, generating multiple wireless propagation channels and achieving the performance of a multi-antenna receiver in a Single-Input Single-Output (SISO) link. We compare the performance of the system with a multi-antenna receiver in terms of channel hardening and outage probability. ii) We introduce REMARKABLE, an online learning based adaptive beam selection strategy for robot connectivity that trains kernelized multi-armed bandit (MAB) model directly in real-world settings of a factory floor. We show how RISs with passive reflective elements can create beamforming towards target robots, and provide a solution to the problem of adaptive beam selection in dynamic channel conditions. We experimentally demonstrate that REMARKABLE can achieve a significant reduction in beam selection time compared to classical approaches and adaptive beam selection in mobility settings.
Finally, we introduce AirFC, a system harnessing the capability of OTA computation to run inference on a neural network (NN) consisting of a set of fully connected layers (FC) by leveraging multi-antenna systems. We experimentally demonstrate and validate that such computation is accurate enough when compared to its digital counterpart.

Cobra Alemdar PhD Dissertation Defense
2024-08-07 11:00 - 12:00

Northeast Regional Higher Education Neurodiversity Coalition Conference
2024-08-08 - 2024-08-09 | Interdisciplinary Science and Engineering Complex (ISEC), 805 Columbus Ave, Boston, MA, 02115, United States
Improving educational and professional outcomes for neurodivergent students
This conference is both timely and relevant. The neurodiverse population is increasing, while students have lower academic performance and higher college dropout and underemployment rates.
Join Together to Make a Difference

Increase your understanding of neurodiversity
Learn trends in neurodiversity employment
Understand best practices for neuroinclusive pedagogy and curriculum
Gain knowledge to create a neuroinclusive workplace and campus
Explore and share best practices with a community of higher education and industry professionals

Who Should Attend
First-year engineering faculty, deans, department chairs, teaching faculty, research assistants, academic and career advisors, DEI professionals, employers, and other college and workforce professionals.
Featured Keynote Speakers
Dr. Jeff Karp, a renowned biomedical engineer at Harvard Medical School and MIT grew up being “written off” for his learning differences. He consequently developed “Life Ignition Tools,” a process for embracing life that resulted from years of iteration and tinkering to make his unique thought patterns and behavior work for him.
John Elder Robison, a well-known author of Switched On: A Memoir of Brain Change and Emotional Intelligence, Look Me in the Eye, Be Different, and Raising Cubby, which details his life with Asperger syndrome. He’s a leading voice for autism and neurodiversity, imploring audiences to find strengths where others see weaknesses based on societal standards.
Learn More and Register
https://coe.northeastern.edu/nehenc.
Presented in partnership by Northeastern University, University of Connecticut, and the University of Rhode Island

Peiyan Dong PhD Dissertation Defense
2024-08-08 15:00 - 16:00
Name:
Peiyan Dong
Title:
Software-Hardware Co-Design: Towards Ultimate Efficiency in Deep Learning Acceleration
Date:
8/8/2024
Time:
3:00:00 PM
Committee Members:
Prof. Yanzhi Wang (Advisor)
Prof. David R. Kaeli
Prof. Devesh Tiwari
Prof. Cheng Tan
Abstract:
As AI techniques continue to advance, the efficient deployment of deep neural networks on resource-constrained devices becomes increasingly appealing yet challenging. Simultaneously, the proliferation of powerful AI technologies has raised significant concerns about sustainability and fairness, demanding increased attention from the community. This talk presents two novel software-hardware co-designs for improving the efficiency and sustainability of deep learning models. The first part introduces a hardware-efficient adaptive token pruning framework for Vision Transformers (ViTs) on embedded FPGA, HeatViT, which achieves significant speedup under similar model accuracy compared to the state-of-the-art. HeatViT is the first end-to-end accelerator for ViT on embedded FPGA and also achieve practical speedup by data-level compression for the first time. The second presents PackQViT and Agile-Quant, a paradigm of the efficient implementation for transformer-based models by sub-8-bit packed quantization and SIMD-based optimization for computing kernels. Our framework can achieve better task performance than state-of-the-art ViTs and LLMs with significant acceleration on edge processors, such as mobile CPU, Raspberry Pi and RISC-V. This work not only marks the first successful implementation of the LLM on the edge but also addresses the previous limitation where edge processors struggled to efficiently handle sub-8-bit computations. At the conclusion of the presentation, the speaker will discuss today’s challenges related to AI sustainability and fairness and outline her research plans aimed at addressing these issues.

Galante Program Virtual Info Sessions
2024-08-08 16:30 - 17:00, 2024-08-12 08:30 - 09:00
Learn how the Galante Engineering Business Program and Engineering Business Certificate can complement your technical engineering education with essential business skills.
Join us for an informational session to learn more about the Galante Engineering Business Program on one of the following dates:

Thursday, August 8 at 4:30 p.m. EDT – Virtual
Monday, August 12 at 8:30 a.m. EDT – Virtual
Tuesday, August 20 at 12:00 p.m. EDT – Virtual
Thursday, August 22 at 9:00 a.m. EDT – Virtual

RSVP Here
During the session, we will cover the details of the Galante Program and Engineering Business Certificate, including the application process and eligibility requirements.

Yifan Gong PhD Dissertation Defense
2024-08-09 11:00 - 12:00
Name:
Yifan Gong
Title:
Towards Energy-Efficient Deep Learning for Sustainable AI
Date:
8/9/2024
Time:
11:00:00 AM
Committee Members:
Prof. Yanzhi Wang (Advisor)
Prof. David R. Kaeli
Prof. Xue Lin
Prof. Huaizu Jiang
Prof. Stratis Ioannidis
Abstract:
The rapid advancements in deep learning (DL) and artificial intelligence (AI) have led to transformative applications across various domains, such as community virtual reality experiences, autonomous systems, and climate change prediction. Edge devices including mobile and embedded systems play a vital role in carrying these applications, facilitating the widespread adoption of machine intelligence. Along with the great success of DL and AI is the huge energy consumption for both training and inference. With the breakthrough of large-scale models for AI-generated content (AIGC) such as large language models and diffusion models, the energy consumption issue intensifies, causing the urgent need for sustainable AI solutions. In this talk, I will talk about how to facilitate deep learning on various edge devices in an energy-efficient manner for the goal of sustainable AI. Specifically, I will start by introducing my two system-level approaches to tackling the challenge. The first approach is named bottom-up, which conducts AI algorithm-aware efficient system design. The second approach is a top-down approach that achieves hardware-driven efficient AI algorithm design. Then, I will share my recent works addressing the efficiency issues for large-scale models. Finally, I will show the applications of my methods and pointers to the future direction.

Gözde Özcan PhD Dissertation Defense
2024-08-12 10:00 - 11:00
Name:
Gözde Özcan
Title:
Learning and Optimizing Set Functions
Date:
8/12/2024
Time:
10:00:00 AM
Location:
EXP 601
Committee Members:
Prof. Stratis Ioannidis (Advisor)
Prof. Jennifer Dy
Prof. Evimaria Terzi
Abstract:
Learning and optimizing set functions play a crucial role in the artificial intelligence research as various problems of interest can be characterized with set inputs and/or outputs. Submodular functions, i.e., set functions with a diminishing returns property, are an important subcategory of such functions. They naturally present themselves in applications such as sensor placement, data summarization, feature selection, influence maximization, hyper-parameter optimization, and facility location, to name a few. In a lot of these compelling problems, the objective is to maximize a submodular function subject to matroid constraints, which is known to be NP-hard. For problems of this nature, the continuous greedy algorithm provides a (1 − 1/e)-approximation guarantee in polynomial-time. It does so by estimating the gradient of the so-called multilinear relaxation of the objective function via sampling. However, for the general class of submodular functions, the number of samples required to achieve this theoretical guarantee can be computationally prohibitive.
In this dissertation, we address deterministic submodular maximization problems with matroid constraints, specifically those with objectives expressed through compositions of analytic and multilinear functions. We introduce a novel polynomial series estimator to approximate the multilinear relaxation of such functions and demonstrate that the sub-optimality introduced by our polynomial expansion can be minimized by increasing the polynomial order. By utilizing this estimator, a variant of the continuous greedy algorithm achieves an approximation ratio close to (1 − 1/e) ≈ 0.63 through deterministic gradient estimation. In numerical experiments, our polynomial estimator outperforms the sampling estimator, offering reduced errors in less time.
We extend our study to the stochastic submodular maximization setting with general matroid constraints, where objectives are defined as expectations over submodular functions with an unknown distribution. Adapting polynomial estimators to this context reduces the variance of the gradient estimation while introducing a controlled bias term. For several notable stochastic submodular maximization problems, we demonstrate that this bias decays exponentially with the degree of our polynomial approximators. Furthermore, for monotone functions, a stochastic variant of the continuous greedy algorithm attains an approximation ratio (in expectation) close to (1 − 1/e) ≈ 0.63 using these polynomial estimators. Our experimental results validate the advantages of our approach across synthetic and real-life datasets.
Finally, we turn our attention to the learning set functions under a so-called optimal subset oracle setting. A recent approach approximates the underlying utility function with an energy-based model. Approximating this energy-based model yields iterations of fixed-point update steps during mean-field variational inference. However, these fixed-point iterations are not guaranteed to converge and as the number of iterations increases, automatic differentiation quickly becomes computationally prohibitive due to the size of the Jacobians that are stacked during backpropagation. We address these challenges by examining the convergence conditions for the fixed-point iterations and utilizing implicit differentiation over automatic differentiation. We empirically demonstrate the efficiency of our method on synthetic and real-world subset selection applications.

Yufei Feng MS Thesis Defense
2024-08-13 14:00 - 15:00
Name:
Yufei Feng
Title:
Beam Management in Operational 5G mmWave Networks
Date:
8/13/2024
Time:
2:00:00 PM
Committee Members:
Prof. Dimitrios Koutsonikolas (Advisor)
Prof. Josep Jornet
Prof. Mallesham Dasari
Abstract:
Due to the directional nature of mmWave signal propagation, beam management plays a critical role in the performance of 5G mmWave deployments. However, the details of beam management in commercial deployments and its performance in real-world scenarios remain largely unknown. In this paper, we fill this gap by performing a comparative measurement study of the beam management procedure of two major US operator in Boston, MA. We study a number of beamforming parameters including beamwidth, number of beams, beam switching delay, and their impact on performance, and we explore the interplay between beam management and rate adaptation. We also investigate for first time Rx beam management on the UE side. Finally, we study the beam tracking performance and the quality of the selected beams for two operators.

Gordon Institute Virtual Information Session
2024-08-14 17:30
Learn how you can earn a Graduate Certificate in Engineering Leadership as a stand-alone certificate or in combination with one of twenty-three Master of Science degrees offered through Northeastern’s College of Engineering, College of Science, or Khoury College of Computer Sciences.
The National Academy of Engineering recognized The Gordon Institute of Engineering Leadership (GIEL) for its innovative curriculum that combines technical education, leadership capabilities, and the “Challenge Project”: an opportunity for students to receive master’s level credit while working in industry.
By aligning technical proficiency with leadership capabilities, GIEL accelerates the development of high-potential engineers and prepares them to lead complex projects early in their careers. Upon completing the program, more than 88% of the 2022 class reported increased leadership responsibility, while more than 50% of the 2022 class reported being promoted within one year of graduation.
Our Director of Admissions will answer your application questions for Fall 2024.
You will have the opportunity to hear from Alumni on how The Gordon Institute propelled their engineering careers. Program professors will also be present to answer curriculum questions.

Faruk Volkan Mutlu PhD Dissertation defense
2024-08-16 10:30 - 11:30
Name:
Faruk Volkan Mutlu
Title:
Cost-aware Joint Caching and Forwarding in Networks with Diverse Cache Resources
Date:
8/16/2024
Time:
10:30:00 AM
Committee Members:
Prof. Edmund Yeh (Advisor)
Prof. Stratis Ioannidis
Prof. Elif Uysal
Abstract:
The rapid growth of data-intensive applications is testing the limitations of today’s data distribution networks. Caching is a crucial tool for high performance in such networks, and a core principle in emerging paradigms like information-centric networking (ICN). In this dissertation, motivated by the needs of a landmark initiative addressing the networking challenges faced by large-scale scientific research programs, we focus on the key issue of expanding cache capacities in a cost-effective manner. While DRAM is still the standard cache device today due to its high transfer rates, its capacity is very limited and subject to contention by other networking functions. Large DRAM modules are also expensive, making wide area networks with cache-enabled routers costly to deploy. On the other hand, technologies like flash storage offer larger capacities at lower costs; observing recent advancements in this domain, we expect devices like NVMe SSDs to feature as additional cache tiers in networks supporting data-intensive applications. However, the slower transfer rates of such devices and the added operational costs they introduce pose some challenges.
This dissertation primarily focuses on the open problem of developing cost-aware caching policies that can effectively manage multiple types of cache available to routers. We begin by introducing an object-level multi-tiered caching model that incorporates the diverse characteristics of cache devices, namely their transfer rates and utilization costs. We then integrate this model with an established optimization framework to develop a joint caching and forwarding policy that uses caching resources available in the network intelligently to improve performance, minimize costs of cache utilization and avoid congestion. To highlight the advantages of our approach against adapted baselines, we conduct an exhaustive experimental evaluation of this policy under a large variety of simulation settings; we also provide a discussion of the event-driven object-level ICN simulator we built to facilitate this evaluation and support future research. Lastly, we also present a variation on our strategy that attempts to improve its efficiency under certain conditions by introducing new control variables into the aforementioned optimization framework.
This dissertation also discusses our work on the effective use of transmission and cache resources in wireless networks. While this work investigates a different context than that outlined above, it serves to complement our primary scope with a perspective on how caching can be leveraged in settings where resource constraints have a different type of interplay with network performance. Specifically, in the context of multi-hop wireless networks with arbitrary topologies and interfering transmissions, we study the open problem of delay minimization via caching subject to transmission power limitations. We cast this scenario as an optimization problem and highlight its analytical properties. We identify the challenges in finding a global optimum to the problem under general conditions, and propose an algorithm that converges to a local optimum. We conclude this discussion with numerical results that demonstrate the effectiveness of our approach.

Yanyu Li PhD Dissertation Defense
2024-08-16 11:00 - 13:00
Name:
Yanyu Li
Title:
Accelerating Large Scale Generative AI: a Comprehensive Study
Date:
8/16/2024
Time:
11:00:00 AM
Committee Members:
Prof. Yanzhi Wang (Advisor)
Prof. David Kaeli
Prof. Kaushik Chowdhury
Abstract:
We have witnessed the great success of deep learning in various domains, such as the emerging large language models (LLMs) and Artificial General Intelligence (AGI), diffusion models for image and video generation, and classic vision tasks including classification, segmentation, detection, etc. Built with linear, convolution, and attention blocks, Deep Neural Networks (DNNs) play a vital role in the performance revolution. However, powerful DNNs often call for tremendous computation and storage size, which hinders their wide adoption. For instance, LLMs and diffusion models generally have billions of parameters and hundreds of GMACs, which is prohibitive for edge deployment. As a result, Efficient AI has become a hot research area. In this work, with algorithm optimizations and co-designs with hardware platform, we pursue the appealing features of edge or user-end AI, where we cut down energy consumption, shorten response latency, shrink model storage size, eliminate the need for cloud server access and protect user privacy. Firstly, we systematically investigate quantization, pruning, and architecture search techniques for efficient vision backbones. We do a comprehensive study on quantization number system and precision, and propose a novel mix-scheme mix-precision quantization technique to maximize hardware utilization and minimize performance loss. Regarding network pruning, we propose a novel indicator-based approach, named Pruning-as-Search, that is fully differentiable and automatically decides pruning policies, outperforming human tuning methods in terms of performance and efficiency. Further, we address the long-existing issue of rigid network width design, proposing a family of flexible-width pruned networks with minimal per-layer redundancy. As for architecture search, we formulate a joint optimization objective of both size and latency, releasing a series of efficient Vision Transformers, named EfficientFormer (V1 and V2), to serve as strong vision backbones with MobileNet-level size and millisecond-level latency on mobile phones.
Secondly, we make dedicated optimizations for large-scale generative tasks, i.e., Stable Diffusion (SD) for text-to-image generation, which serves as pioneer work to enable their mobile deployment. With the proposed efficient architecture design and novel step distillation, we shrink the generation latency of SD by a magnitude, from more than 1 minute to generate a 512$times$512 image to 1~2 seconds, while preserving the stunning generative quality. We extend our work to the even more challenging video generation task, enabling 2-bit inference and single step adversarial distillation to speedup video diffusion models by a magnitude.
