Research Projects

Generative AI models such as large language models (LLMs) have the potential to transform various fields. However, an adversary can use the model to reconstruct the data they were trained on. This significant privacy risk hinders their wider deployment, particularly in sensitive domains like healthcare and finance. This project aims to bridge the gap between theoretical privacy guarantees and practical scalability in generative AI by developing a new framework for provable protection against data extraction. Existing approaches to AI privacy suffer from limitations. While the standard approach of differential privacy offers strong theoretical guarantees, enforcing this worst-case guarantee requires a 10-25x increase in computational cost. This makes it impractical for large-scale models. On the other hand, heuristic methods lack provable guarantees and can fail unexpectedly. This project addresses the urgent need for a privacy solution that is both theoretically sound and practically applicable to large generative models. My key idea is that enforcing a minimum level of ambiguity in the probabilistic reconstruction of training data from the model can provide provable protection against extraction attacks. The key objectives and the methodology include: 1. Provable Protection: Establish a rigorous definition of what it means for a generative model to be provably protected against training data extraction. This definition will be grounded in information theoretic concepts like conditional entropy to quantify the ambiguity in “guessing” individual data points using the trained model. 2. Scalable Algorithms: Develop efficient learning algorithms that can provide provable privacy guarantees while maintaining scalability. These will be variants of noisy stochastic gradient descent, where the noise level scales inversely with the difficulty of “guessing” a specific data point from a trained model. 3. Practical Effectiveness: Evaluate the proposed framework by fine-tuning LLMs on sensitive healthcare or personal conversation data. Demonstrate that the approach provides meaningful semantic guarantees against data extraction while preserving the model's utility for downstream tasks. Successful completion of this project will have significant implications for both the fundamental understanding and practical application of AI privacy. First, it will provide a novel theoretical framework for analyzing and quantifying privacy in generative AI models. Second, it will enable the deployment of AI in privacy-sensitive domains like healthcare and finance, unlocking its transformative potential for societal benefit. Further, the framework will promote greater trust in AI technologies by providing robust and provable privacy guarantees. By developing a scalable and provably secure approach to protecting against data extraction, this project will pave the way for the responsible and ethical development of generative AI technologies.