Shuangjun Liu’s PhD Dissertation Defense

Location: 532 ISEC

“United Human Pose: Integrating Domain Knowledge and Machine Learning”

Abstract:

Deep learning (DL) approaches have been rapidly adopted across a wide range of fields because of their accuracy and flexibility, but require large labeled training data. This presents a fundamental problem for applications with limited, expensive, or private data (i.e. Small Data Domains). There are two basic approaches to reduce data needs during model training: (1) incorporate domain knowledge in the learning pipeline through the use of data-driven or simulation-based generative models, and (2) decrease inference model learning complexity via data-efficient machine learning. This PhD research is unfolded around addressing small data relevant problems in the context of human pose estimation by leveraging the existing research and filling in key research gaps with original work. We started with introducing a specific human pose estimation problem, in-bed pose estimation and present our solutions to this problem in an increasing order of feasibility, that make use of (1) conventional non-deep inference models, (2) fine-tuning already trained deep model with limited data, and (3) building and training a pose estimation model from scratch using a novel dataset.

This practical application also introduced us new challenges such as 3D human pose estimation when no 3D pose data is available in the target domain (e.g. in-bed pose domain) and dense physical signal sensing from vision signals (e.g. contact pressure estimation).
In order to address the small data problem in a more general way,

we also explored estimating 3D human poses without using any real 3D pose data but only easy-to-get synthetic human models. We introduced a semi-supervised data augmentation approach via the use of 3D graphical engines and tested its effectiveness in training pose inference models against real human pose data.

Committee:

Prof. Sarah Ostadabbas (Advisor)

Prof. Raymond Fu

Prof. Octavia Camps