ECE MS Thesis Defense: Kathan Vyas

MS Thesis Defense: Data-Efficient analysis of Human Behavior by Spatio-Temporal Pose Generation and Inference

Kathan Vyas

Location: Zoom Link 

Passcode: 474462

Abstract: Identifying human pose over time provides critical information towards understanding human behavior and their physical interaction with the environment surrounding them. In the past few decades, the human pose estimation topic has witnessed groundbreaking research in the computer vision field thanks to the powerful deep learning models. These models are trained using several thousands of labeled sample images if not more. Such extensive data requirement posed a fundamental problem for domains (i.e. Small Data domains), in which data collection or labeling is expensive or limited due to privacy or security concerns such as healthcare. In this thesis, we present a data-efficient learning pipeline to address small data problem in a healthcare-related human pose estimation application. In particular, we infer spatio-temporal human poses to analyze typical vs. atypical behaviors in children with Autism spectrum disorder (ASD). To mitigate data limitation, we propose two thrusts in our learning pipeline. The first thrust is a data-efficient machine learning approach, in which a pre-trained (on adult pose images) pose estimation model with deep structure is fine-tuned on a small set of children pose videos, provided to us by our collaborators. We implement a non-linear particle filter interpolation to deal with any missing body keypoints in the estimated poses and employ a novel PoTion (pose motion) based temporal aggregation technique to evaluate poses over time. The second thrust is a synthetic data augmentation approach, in which we build a framework to create synthetic 3D humans with articulated bodies in order to render more pose images/videos in our application contexts. We use a novel 3D registration approach based on RANSAC and implement iterative closest point (ICP) to obtain 3D meshes from the scanned point clouds from both adult and kid mannequins, which is then rigged and articulated in the Blender to generate our human avatars. We then infuse these avatars in various synthetic environments to create contexts similar to the target application, which is a kid with both typical and atypical behaviors in a home-like environment.