Assistant Professor Wei Niu alongside Professor and School Director Gagan Agrawal have recently been funded by National Science Foundation (NSF) with a new 3-year, 600K grant. Their project addresses the growing need for running powerful Artificial Intelligence (AI) models on mobile phones, specifically targeting the transformer architectures (used by models like ChatGPT and Gemini). This project is based on the premise that features of these workloads and characteristics of mobile devices require not only the application of existing techniques from compiler literature but also the development of new methods and it turns out that optimization related to memory hierarchy are extremely important for the efficient execution of transformer-based models on modern mobile devices. The project’s novelties are in considering such workload and architecture combination and proposing techniques related to choosing new layouts, removing redundant layout changes that slow down execution, performing memory allocation judiciously to improve performance, and dealing with the newest accelerators.
