CTN: Guillaume Hennequin

Abstract:  When faced with a novel situation, humans often spend substantial periods of time contemplating possible futures. For such planning to be rational, the benefits to behaviour must compensate for the time spent thinking. I will show how we recently captured these features of human behaviour by developing a neural network model where planning itself is controlled by prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences from its own policy, which we call ‘rollouts’. The agent learns to plan when planning is beneficial, explaining empirical variability in human thinking times. Additionally, the patterns of policy rollouts employed by the artificial agent closely resemble patterns of rodent hippocampal replays recently recorded during spatial navigation. Our work provides a new theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by – and adaptively affect – prefrontal dynamics. This is joint work with Kristopher Jensen and Marcelo Mattar.