Research Projects

Imaging inverse problems are routinely encountered in medical image reconstruction (e.g., X-ray computed tomography (CT)) and low-level computer vision tasks (such as deblurring, inpainting, etc.). The reconstructed image is often utilized for a downstream task, for example, disease classification or tumor segmentation in CT; and object detection or classification in computer vision applications. With the growing amount of training data and computing power, deep learning (DL) has delivered excellent results for image recovery and downstream image analysis tasks in various real-world applications. However, the existing DL algorithms for image reconstruction are agnostic to the downstream task, meaning that the DL approaches for image recovery and subsequent analysis are optimized separately. The objective of this project is to build end-to-end DL pipelines wherein the image reconstruction and task operators (both based on deep neural networks (DNNs)) are jointly learned, thereby leading to image reconstruction approaches that are adapted to the downstream task. The key motivation behind this is the hypothesis that while the measured raw data (e.g., the noisy sinogram in CT) could be incomplete for reconstructing a high-fidelity image, it might still have enough information to solve the task at hand (such as tumor delineation in CT), which could be exploited by tailoring the image reconstruction operator to the underlying task. For joint learning of reconstruction and task operators, we will use the framework of bilevel learning, which enables a principled way of information sharing between image recovery and the downstream task. The lower-level problem in the bilevel approach would seek to learn a DNN-based regularizer for image reconstruction, whereas the upper-level loss would represent the performance of the learned reconstruction at the lower level in terms of the task under consideration (e.g., segmentation). In the context of clinical CT, we would demonstrate that such a task-adapted reconstruction approach can potentially lead to a reduction in X-ray dose for patients without compromising performance in the subsequent medical diagnosis. We will also demonstrate the flexibility of the bilevel learning framework to adapt the image-sensing operator as well, together with the reconstruction, considering the use case of compressive imaging. We will develop novel theoretical and algorithmic solutions for bilevel learning to tackle the issues of convergence, scalability, and robustness of the overall DL pipeline.