ECE PhD Dissertation Defense: Yulun Zhang

PhD Dissertation Defense: Deep Convolutional Neural Network for Image Restoration and Synthesis

Yulun Zhang

Location: Zoom Link

Abstract: Image restoration and synthesis with deep learning play a fundamental role in the computer vision community. They are widely used on mobile devices (e.g., smartphones) or lead to billion-dollar startups. However, how to design efficient deep convolutional neural networks (CNNs) to extract higher-quality deep CNN features for better image restoration and synthesis is still challenging. In this dissertation talk, I will describe my recent works to enhance CNN features in the channel dimension or/and the spatial dimensions. First, for image restoration, I will briefly introduce our proposed residual dense network. Then, I will introduce the residual in residual (RIR) structure to train very deep super-resolution networks. Such an RIR structure could also make the network learn more high-frequency information, being critical for high-resolution output. Attention mechanism (e.g., channel attention and spatial attention) is further explored to highlight the features. Second, for image synthesis, I will introduce multimodal style transfer via graph cuts. I visualize the deep features and find the multimodal style representation. I then formulate the style matching problem as an energy minimization one, which could be solved via graph cuts. As a result, the transferred features contain spatially semantic information, providing more visually pleasing stylized results. Besides, we investigate image synthesis about texture hallucination with large scaling factors. We propose an efficient high-resolution hallucination network for very large scaling factors.