Batool Salehihikouei Phd Proposal Review

Title:
Leveraging Deep Learning on Multimodal Sensor Data for Wireless Communication: From mmWave Beamforming to Digital Twins

Committee Members:
Prof. Kaushik Chowdhury (Advisor)
Prof. Hanumant Singh
Prof. Josep Jornet
Dr. Mark Eisen

Abstract:
With the widespread Internet of Things (IoT) devices, a wide variety of sensors are now present in different environments. For example, self-driving vehicles and automated warehouses depend on sensor information for navigation and management of the robots, respectively. In this dissertation, we present a paradigm, where these sensors are re-purposed to assist network management in wireless communication, especially when classic approaches fall short to provide the required quality of service (QoS). This thesis presents data-driven and AI-based methods, where the multimodal sensor information is used for beamforming at the mmWave band, and envisions a systematic framework for joint optimization of the navigation and network management in factory floor environments. In particular, the contributions in this dissertation are as follows. First, we present deep learning fusion algorithms, where the inputs from a multitude of sensor modalities such as GPS (Global Positioning System), camera, and LiDAR (Light Detection and Ranging) are combined towards predicting the optimum beam at the mmWave band. We prove that fusing the multimodal sensor data improves the prediction accuracy compared to using single modalities. Second, we study the trade-off between the accuracy and cost of different learning strategies for multimodal beamforming. In this regard, we make a case for using federated learning for beamforming at the mmWave band and demonstrate that it is the most successful learning strategy, with respect to the communication overhead. Finally, we take measures to further optimize the computation and communication overhead, by incorporating a pruning strategy tailored to the disturbed nature of the federated learning systems. In the proposed research work, we suggest using digital twins to overcome the challenges of scarcity of data and close-world assumption in deep learning algorithms. A digital twin is a replica of a real world entity, which is typically used for studying the impact of any configuration settings in a safe, digital environment. In this dissertation, we propose using digital twins for generating training data for multimodal beamforming, in unseen scenarios. Moreover, we study a robotic industrial setting, where the path planning policy is continuously updated by monitoring the dynamics of the real world, constructing the digital twin, and updating the policy.