CTN: Wei Ji Ma

Abstract: Two of the greatest triumphs of computational neuroscience have been efficient coding accounts of tuning properties of sensory neurons and reinforcement learning accounts of dopaminergic neurons in the midbrain. At first glance, these theories seem to have no connection, but I will argue that they do. One can apply efficient coding principles to derive the optimal population of neurons to encode rewards drawn from a probability distribution. Similar to this optimal population, dopaminergic reward prediction error neurons in the mouse have a
broad distribution of thresholds. We can make further predictions: that neurons with higher thresholds have higher gain and that the asymmetry of their responses depends on the
threshold. We also derive learning rules that can approximate the efficient code. Finally, we apply the theory to monkey data. Taken together, efficient coding might provide a normative underpinning to distributional reinforcement learning.