Research Projects

Lensless imaging is a novel imaging modality that replaces a lens in a conventional camera with an optical mask. This leads to a significant reduction in the size and weight of the camera as the lens contributes heavily to both size and weight. Moreover, because of the optical mask, lensless cameras have the ability to perform computations on the incident light prior to data capture. These advantages in form factor, weight, and the ability to perform computation optically lead to numerous promising applications in augmented and virtual reality (AR/VR), imaging for healthcare, and visual privacy. Due to the absence of the focusing element, the measurement a lensless camera records is a globally multiplexed version of the scene. Therefore, to obtain a photograph of the scene from a lensless measurement, we need to solve an inverse image reconstruction problem. Existing works have mostly addressed this problem of image reconstruction. However, very little has been explored in terms of using lensless cameras for solving various computer vision inference tasks. Also, optimal mask design for lensless imaging systems has not been explored. In this work, we plan to explore three promising applications of lensless imaging in 1) AR/VR/XR, 2) visual privacy, and 3) 3D endoscopy, where lensless cameras can have a significant impact. We propose to develop visual inference algorithms and design optimal optical masks for these applications. Our objective is to develop a theoretical framework for direct inference (without full image reconstruction) from lensless captures and optimal mask design (single and double masks) for lensless imaging systems. Using the above theoretical framework, we will explore the following applications: 1) Lensless for AR/VR: We will perform direct inference for computer vision tasks such as eye tracking and ego-motion tracking from optical flow. 2) Lensless for Visual privacy: We will design single or double masks for privacy preserving lensless cameras. 3) Lensless for 3D endoscopy: We will design single and double masks for extracting both depth and all-in-focus image from lensless captures. We will also super-resolve the depth and all-in-focus image to obtain better image and depth resolution. On completion of the above tasks, we plan to develop industrial prototypes for each of the three applications with the goal of licensing and commercializing them.