Animal Behavior Video Analysis Working Group

Title: Brain Decodes Deep Nets
Presenter: Jianbo Shi, PhD
GRASP Laboratory
Computer and Information Science
University of Pennsylvania

Abstract: We developed a surprising usage of brain encoding: using a brain fMRI prediction model to draw a picture of how a deep net processes information onto a brain.  Our tool provides a detailed analysis of large pre-trained vision models by mapping them onto the brain, thus exposing their hidden layers and channels.   Our results show how different training methods matter: they lead to remarkable differences in hierarchical organization and scaling behavior. It also provides insight into finetuning: how large pre-trained models change when adapting to new datasets.