ECE PhD Proposal Review: Bahare Azari

PhD Proposal Review: Circular-Symmetric Correlation Layer based on FFT

Bahare Azari

Location: Zoom Link

Abstract: Planar convolutional neural networks, widely known as CNNs, have been exceptionally successful in many computer vision and machine learning tasks, such as object detection, tracking, and classification. The convolutional layers in CNN are characterized by pattern-matching filters that can identify motifs in the signal residing on a 2D plane. However, there exists various applications in which we have signals lying on a curved manifold or an arbitrary collection of coordinates, e.g., temperature and climate data on the surface of the (spherical) earth, and 360-panoramic images acquired from LiDAR. In these applications, we usually need our network to be equivariant/invariant to various transformations of the input, i.e., as we transform the input according to a certain action of a group, the output is respectively transformed (equivariance), or remains unchanged (invariance). The convolution layers are empirically known to be invariant to small translations of their input image, but they are not completely immune to relatively large translations Hence, they may fail on the tasks that requires invariance to a specific transformation, and and on the data that includes a wide range of that transformation.

In this work we consider equivariant/invariant tasks on 360-panoramic data. For a systematic treatment of analyzing the 360-panoramic data, we propose a circular-symmetric correlation Layer (CCL) based on the formalism of roto-translation equivariant correlation on the continuous group constructed of the unit circle and the real line. We implement this layer efficiently using the well-known Fast Fourier Transform (FFT) and discrete cosine transform (DCT) algorithm. We discuss how the FFT yields the exact calculation of the correlation along the panoramic direction due to the circular symmetry and guarantees the invariance with respect to circular shift. The DCT provides an improved approximation with respect to transnational symmetry compared to what we observe in CNNs. We demonstrated the invariance analysis of networks built with CCL on two benchmark datasets comparing the equivariance of neural networks adopting CCL layers and regular CNN. Then, we showcase the performance analysis of a general network equipped with CCL on recognition and classification tasks, such as panoramic scene change detection, 3D object classification, LIDAR Semantic Segmentation.