David Femi Lamptey MS Thesis Defense

Name:
David Femi Lamptey

Title:
Coordinating Camera and Millimeter-Wave Imaging Systems to Detect Concealed Threats in Public Spaces

Date:
4/15/2024

Time:
2:00:00 PM

Location:
Snell Library CoLab J

Committee Members:
Prof. Carey Rappaport (Advisor)
Prof. Octavia Camps
Prof. Sarah Ostadabbas

Abstract:
This paper tackles the problem of uniquely identifying and tracking targets for the purposes of concealed threat detection in public spaces. Cameras, computer vision techniques, and deep neural networks have made the task of detecting and tracking people in videos almost trivial but provide no means for the detection of otherwise concealed threats a target may be carrying, while millimeter-wave radars provide a means to perform accurate scanning for concealed objects on a target, but do not provide enough information for tracking and unique identification of a target, particularly one with a concealed threat or contraband. This paper proposes a method utilizing a video camera stream and millimeter-wave multi-beam radar fusion in order to identify people in a public space, track them, and identify the best beam in a multi-beam radar to refer to at any given point in time in order to obtain the best scan of a particular target from the millimeter-wave radar, which will then enable an effective determination of a concealed threat. We focus on the computer vision aspects of this challenge, implementing a tracker and an algorithm to look up the best beam in the radar to associate with a target at a point in time. This algorithm uses the properties of the camera, such as the video resolution, field of view of the camera, internal parameters of the camera, and elevation of the camera, in order to perform an estimation of the distance of a person from the camera and perform a determination of the optimal beam to look at for a clear view of the target. This approach was optimized using an efficient spatial indexing lookup technique based on the R-tree data structure. The results from this paper show that this technique is robust, accurate, and versatile for a wide variety of scenarios and that the real-time tracking and association between targets and millimeter-wave beams can be performed accurately. We conclude that this technique is a fitting solution to the problem of camera and millimeter-wave multi-beam radar fusion in order to identify concealed threats on targets in public spaces.