Research Projects

The primary purpose of this project is to develop a diffusion-based hand gesture interaction understanding system that can understand and interpret occluded or blurred gestures from an image or video. The real-world hand gestures are challenging to interpret since they are heavily occluded when hand interactions are involved in real-world environments. The key points in this project are to investigate the occluded regions to investigate the area of interest, leverage conditional denoising diffusion probabilistic models (DDPMs) for learning a latent distribution over 3D hand poses, and synthesize realistic hand gestures. This project addresses the dual challenge of advancing the theoretical understanding of generative diffusion model for 3D hand gesture interaction and applying it to socially impactful scenarios. In particular, this project will demonstrate its applicability by deploying a gesture-based emergency detection system aimed at improving safety for women, children, and disabled individuals in India. Such a system allows users to silently trigger alerts using predefined gestures when they cannot speak or access a phone. The proposed system will take advantage of affordable hardware (CCTV, mobile cameras) and run lightweight inference models for wide accessibility in India.