Sadjad Asghari Esfeden’s PhD Dissertation Defense

“Spatiotemporal Localization of Object Handover for Human Robot Collaboration”

Committee Members:

Prof. Deniz Erdogmus (Advisor)

Prof. Taskin Padir

Prof. Eugene Tunik

Prof. Mathew Yarossi

Abstract:

Human-robot interaction in a physical world like handover of objects requires perception systems to be efficient in localizing the object of interest. We propose an approach to estimate the location of the object with a low-cost RGB camera in a real-time inference for human-robot handover. While handover can take place in a short amount of time, it is important for a robot to keep track of the object and fill in the gaps of missing detections in the perception module, especially when the object is partially or completely occluded. A robot needs to proactively detect and track the object since the human decides where and when to transfer the object to the robot in a human to robot object handover.  In order to develop a perception system for robot to be capable of constantly localizing the object and predict its location and time of transfer, we integrate an object detection algorithm with a tracking framework. The evaluation of this pipeline shows promising results for the goal of localization and tracking of the handover object and can help its location prediction in future.