Multi-resource-cost Optimization of Neural Network Models

This project will tackle the conceptual challenge of how to measure and the engineering challenge of how to optimize multiple resource costs of neural network models. We will develop (1) ways to quantify the costs of space, time, energy, and data of neural network models and (2) differentiable objectives that enable efficient joint minimization of the costs of these multiple resources. More practically, the project will bring members of our labs together to build two open-source Python libraries, one for measurement and one for optimization of the resource requirements of neural network models implemented in PyTorch. The methods will help us understand biological neural mechanisms that emerge from particular profiles of resource costs and behavioral affordances. It will also empower us to engineer more efficient AI for resource-limited devices. The outcomes will be two papers and associated open-source Python libraries for measurement and optimization, respectively, of multiple resource costs. These contributions will benefit other projects in our ARNI Working Group and also the broader communities of Cognitive Computational Neuroscience and NeuroAI.