Sumegha Singhania MS Thesis Defense

Title: Exploring Log of RGB Space as a Better Input for Computer Vision Tasks

Committee Members:
Prof. Bruce Maxwell (Advisor)
Prof. Hanumant Singh
Prof. David Rosen
Prof. Mahdi Imani

Abstract:
There are specific, physics-based rules that govern the interaction of light and matter. Though studied extensively in the greater computer vision community, these rules are largely broken by common image processing techniques like JPEG compression and sRGB conversion. While the reliability and usability of color and intensity found in RAW images might better train networks to successfully complete vision-based tasks, these smaller, more heavily-processed formats have become the standard input for training sets. As a result, many of the images used to train neural networks do not retain the inherent structure that would enable neural networks to learn more general rules that exist in the natural world.

We hypothesize that using linear RGB or log RGB images, which preserve the physics of reflection, can simplify the learning process for certain vision tasks, enhance overall robustness and performance, and provide invariance to visual variations that exist in real-world vision applications. Our research demonstrates that employing linear and log RGB images to train deep networks for the task of object detection improves their performance when using the same network architecture and the same set of training images. Additionally, we also show that the networks trained on linear and log RGB show greater resilience to variations in intensity and color balance. Specifically, the network trained on linear and log RGB inputs shows invariance to intensity and color balance variations that were not encountered during training, while the network trained on the same images in sRGB JPEG format experiences significant performance degradation. To understand the reasons behind this disparity, we analyze and visualize low-level features in log RGB, linear RGB, and JPEG data. Our findings reveal that the log space preserves certain relevant features across variations in intensity and color balance.