PhD Dissertation Defense: Kunpeng Li

PhD Dissertation Defense: Visual Learning with Limited Supervision

Kunpeng Li

Location: Zoom Link

Abstract: Deep learning models have achieved remarkable success in many computer vision tasks. However, they typically rely on large amounts of carefully labeled training data whose annotating process is usually expensive, time-consuming and even infeasible when considering the task complexity and scarcity of expert knowledge.
In this dissertation talk, I will discuss several explorations along the direction of visual learning with limited supervision. They are mainly about learning from data with weak forms of annotations and learning from multi-modal data pairs. Specifically, I will first present a guided attention learning framework to conduct semantic segmentation by mainly using image-level labels, as such weak form of annotation can be collected much more efficiently than pixel-level labels. Under mild assumptions, our framework can also be used as a plug-in to existing convolutional neural networks to improve their generalization performance. This is achieved by guiding the network to focus on correct things when learning concepts from a limited set of training samples. Besides, I will also introduce models that can effectively learn from multi-modal data pairs without relying on dense annotations of visual semantic concepts. Our models incorporate relational reasoning ability into the visual representation learning process so that it can be better aligned with the supervision from corresponding text descriptions.