Self-supervised pre-training for the embodied Turing test

PI: Blake Richards
Co-PI: Bence Ölveczky, Sean Escola, James Kozloski

Abstract

For the first time in history, artificial intelligence (AI) models can effectively pass the Turing test, i.e. they can pass for human in a text-based conversation. However, when it comes to basic embodied tasks that we consider “lower”, like running, jumping, and navigation, animals still greatly outperform AI and robots. This gap in natural and artificial intelligence is one of the greatest impediments to the adoption of AI in wide swaths of the physical economy, and implies a missing link between AI and neuroscience. To close this gap, the goal of our project is to develop AI models that can pass the “embodied Turing test”, i.e. be indistinguishable from an animal in sensorimotor tests in real-world settings (or sophisticated simulations of them). We are pursuing this goal using a virtual rodent, which allows us to take existing recordings of real animal behavior, and apply 3D pose estimation to the recordings in order to register the poses with the virtual rat. We then train a neural network to take recorded trajectories and output the actions by the virtual rat that would produce these trajectories and we use this model to generate large amounts of simulated, but realistic, motor sequences. We will use these virtual sensorimotor sequences to pre-train an auto-regressive transformer model, and then attach our sensory and motor models to a reinforcement learning module, providing a behavioral policy in latent space. This is analogous to having a basal ganglia circuit operating off of thalamo-cortical representations and sending motor commands to a low-level controller in the brain stem – a novel, brain-inspired architecture. We believe that this novel architecture will be able to pass the embodied Turing test.

Publications

In progress

Resources

In progress