Research Projects

The proposed project addresses a critical challenge in utilizing recent developments of foundation models for ground root societal problems, particularly on the development of robust, privacy-preserving, and explainable AI systems for sensor-rich environments in public health and safety. As India witnesses rapid deployment of multimodal sensing infrastructures, from rural health clinics to disaster-prone zones and urban surveillance systems, AI must evolve to operate across heterogeneous data streams (e.g., vision, audio, biosignals, environmental sensors), under constrained compute environments, while ensuring privacy, security, and interpretability. We hypothesize that current foundation models are not natively suited for real-world Indian contexts, which involve sparse, noisy sensor data, edge-level resource constraints, and complex privacy requirements. This project proposes a unified AI architecture that is sensor-aware, adversarially robust, edge-compatible, and policy-compliant. The key innovation in this project lies in integrating modular embeddings for diverse modalities, federated learning with differential privacy, and large language model (LLM)-based explainability, tailored for mission-critical deployment. The overall project is structured into five interrelated Work Packages (WPs): WP1 - Design and adapt multimodal foundation models using embedding schemes and lightweight architectures suitable for edge deployment. WP2 - Simulate cross-modal adversarial threats (e.g., spoofed biosignals, tampered video/audio) and develop defense strategies including out of distribution (OOD) detection and cross-modal consistency checks. WP3 - Create privacy-preserving training pipelines using federated learning and secure inference methods aligned with Indian data protection laws (e.g., DPDP Act, NDHM). WP4 - Build uncertainty estimation and human-in-the-loop (HITL) frameworks using LLMs to generate sensor-specific explanations and trigger manual validation for risky predictions. WP5 - Integrate all components into lab-scale proof-of-concept deployments that simulate real-world applications like smart surveillance and rural health kiosks. A distinctive strength of this proposal is the strategic use of IIT Kharagpur’s geographical location, which offers direct access to rural areas such as Hijli, Salua, Gopali, and Prembazar. These health centers present a realistic test environment and field study for deploying and evaluating the developed AI systems. The proximity allows for iterative field testing, user feedback, and rapid system refinement. This will not only validate the robustness and usability of the proposed solutions but also directly contribute to strengthening rural healthcare infrastructure in eastern India through AI-assisted diagnostics, early disease detection, and improved medical workflows. Experiments will involve data collection in both lab and rural environments, simulation of adversarial sensor threats, federated training over distributed edge nodes, and explainability evaluation via human studies. The system will be benchmarked on accuracy, latency, resilience, privacy compliance, and interpretability. Successful completion of this project will significantly advance our understanding of how foundation models can be customized for edge-based, multimodal, societal applications. In summary, the project has the potential to transform India’s public AI infrastructure, empowering local health workers, strengthening emergency response, and enabling transparent decision-making systems. It will also produce open-source tools, privacy-aware model architectures, and simulation platforms to catalyze broader innovation, aligned with India’s national digital and health missions.