Balaji Sundareshan’s MS Thesis Defense

“Cross-View Action Recognition using Transformers”

Committee Members:
1. Prof. Octavia Camps (Advisor)
2. Prof. Mario Sznaier
3. Prof. Huaizu Jiang

Abstract:
Cross-view action recognition (CVAR) seeks to recognize a human action when observed from a previously unseen viewpoint. This is a challenging problem since the appearance of action changes significantly with the viewpoint. Applications of CVAR include surveillance and monitoring of assisted living facilities where is not practical or feasible to collect large amounts of training data when adding a new camera. In this thesis, we propose a method to perform cross-view action recognition from 2D skeleton data using Transformers. First, we understand the interpretability of the basline network and its submodules by visualizing the saliency map. Next, we integrate Transformers at different parts of the network for both single-clip and multi-clip and understand the impact on the performance. In the end, we also discuss the necessity of pretraining sub-modules in the network and their impact on the performance.