Research Projects

With the emergence of generative Artificial Intelligence (AI), the Large Language Models (LLMs) like Gemini, LLaMa, GPT have shown remarkable abilities in understanding and producing new content like human beings. However, this massive computing and processing capabilities come at a substantial energy cost leading to carbon emissions and significant environmental damage. For instance, in a study it was found that the Bloom LLM model produced 24.7 tonnes of CO2 emissions during its training. This underscores the critical need for developing more energy-efficient AI hardware and software to mitigate the environmental impact of these powerful technologies. Our proposed research “GreenLLM: An Ecosystem for sustainable LLM” aims to tackle several fundamental challenges associated with scaling the training and deployment of Large Language Models (LLMs) by answering the following questions: Q1: Do different LLM models exhibit some common pattern training and Inference? Q2: Can we leverage the knowledge of these patterns to apply the energy management techniques like Dynamic Voltage and Frequency Scaling and Power Gating? Q3: To deploy the feasible solution in the real systems, what are the different overheads (time, hardware and software) involved? (a)Scientific rationale and Technical Impact: This research advocates the need for sustainable AI computing. Since many of the big companies like Google, Facebook, Microsoft etc are in the AI race and are focusing on building faster LLMs, it is the need of the hour to focus on the energy efficiency of the LLMs deployed on domain specific accelerators. With the end of Moore's law and Dennard scaling, we need to make sure that the LLMs continue to scale up and meet the performance demand. The proposed research shall help to design energy efficient and scalable LLMs. (b) National Ecosystem Impact: This alligns with the government of India initiatives on “national mission on supercomputing” and “national mission on artificial Intelligence”. It will contribute to developing sustainable AI technologies, supporting Digital India, and fostering domestic product and technology development. By driving innovation in the semiconductor industry, this research aims to boost India's technology sector. (c) Education Impact: The PI is involved in teaching Undergraduate and Graduate level courses on Computer Architecture and GPUs. This project will enrich the educational experience and prepare the students as well as the researchers to be industry 5.0 ready. (d) Preliminary Work: The Principal Investigator (PI) has made significant contributions to optimizing the energy efficiency of matrix multiplication, a core component of Large Language Model (LLM) self-attention mechanisms. They were instrumental in pioneering the use of CUDA graphs to streamline LLM deployment by reducing CPU overhead. Additionally, the PI possesses a deep understanding of Deep Neural Network (DNN) profiling for various hardware platforms.