Speaker: Hadi Vafaii ARNI WG Multi-resource-cost optimization of neural network models
2026-04-07 13:00 - 15:00 (America/New_York) | Zuckerman Institute – L3-079, 3227 Broadway, New York, NY, 10027, United States
Location: ZI L3-079
Time: 1:00pm
Title: Metabolic cost of information processing in Poisson variational autoencoders
Abstract:Computation in biological systems is fundamentally energy-constrained, yet standard theories of computation treat energy as freely available. Here, we argue that variational free energy minimization under a Poisson assumption offers a principled path toward an energy-aware theory of computation. Our key observation is that the Kullback-Leibler (KL) divergence term in the Poisson free energy objective becomes proportional to the prior firing rates of model neurons, yielding an emergent metabolic cost term that penalizes high baseline activity. This structure couples an abstract information-theoretic quantity — the coding rate — to a concrete biophysical variable — the firing rate — which enables a trade-off between coding fidelity and energy expenditure. Such a coupling arises naturally in the Poisson variational autoencoder (P-VAE; a brain-inspired generative model that encodes inputs as discrete spike counts and recovers a spiking form of sparse coding as a special case) but is absent from standard Gaussian VAEs. To demonstrate that this metabolic cost structure is unique to the Poisson formulation, we compare the P-VAE against GReLU-VAE, a Gaussian VAE with ReLU rectification applied to latent samples, which controls for the non-negativity constraint. Across a systematic sweep of the KL term weighting coefficient β and latent dimensionality, we find that increasing β monotonically increases sparsity and reduces average spiking activity in the P-VAE. In contrast, GReLU-VAE representations remain unchanged, confirming that the effect is specific to Poisson statistics rather than a byproduct of non-negative representations. These results establish Poisson variational inference as a promising foundation for a resource-constrained theory of computation.

Speaker: Mengye Ren - ARNI Continual Learning Working Group Meeting
2026-04-09 15:00 - 16:00 | CEPSR 620, Schapiro 530 W. 120th St
Mengye Ren
Mengye will also be giving a talk on continual learning at the Zemel group meeting an hour prior (at 2pm) that working group attendees are welcome to join if interested. Here’s the abstract of his talk:

Today’s AI models primarily acquire knowledge through offline, i.i.d. learning. While in-context learning offers some capacity for online adaptation, enabling models to continue learning at deployment or even from scratch through experiential streams remains a crucial question. In this talk, I will introduce approaches toward always-learning machines, beginning with video streams. We find that event segmentation—clustering event concepts in lifelong video—enables effective visual representation learning and grouping from scratch. We also show that pretrained VLMs can introspect over past memory and form event clusters through attention, building hierarchical episodic memory for video question answering. Lastly, I will discuss my recent theory linking continual learning and world modeling to self-consciousness.

ARNI Biological Learning Working Group
2026-04-10 16:00 - 17:00 | Virtual
Continuation of prior meetings.
Zoom: Upon request @arni@columbia.edu

CTN: Jack Lindsey (Anthropic)
2026-04-14 14:00 - 15:00 | Zuckerman Institute – L5-084, 3227 Broadway, New York, NY, United States
Title: The inner lives of language models
Abstract: In recent years, LLMs have evolved from bad text completion engines, to decent chatbots, to digital genies that work miracles on your computer (while making the occasional catastrophic error). The increasing sophistication of AI models’ behavior has been accompanied by a commensurate enrichment of their internal representations and computations. In this talk, I’ll give an overview of what’s known about LLM cognition, and the ways in which it emulates components of human psychology: emotional reactions, strategic manipulation, and forms of introspection. I’ll also cover aspects of LLM behavior that are fundamentally un-human-like, owing to features of their architecture and training process, and how these give rise to odd failure modes—for instance, a weakly anchored sense of self. Finally, I’ll discuss the urgency of addressing pathologies, both human-like and alien, of LLM psychology, and some ideas for doing so.
The talk is in-person. If you do not have card access to the Jerome L. Greene Science center building, you can email Arianna Pepin <ap4287@columbia.edu> to be added to the guest list for the seminar.

ARNI Continual Learning Working Group Meeting
2026-04-23 15:00 - 16:00 | CEPSR 620, Schapiro 530 W. 120th St
Continuation from prior meeting.
Zoom link: upon request @ arni@columbia.edu

ARNI Distinguished Seminar Series: Ellie Pavlick (Brown University)
2026-04-24 15:00 - 16:30 | Zuckerman Institute- Kavli Auditorium 9th Fl, 3227 Broadway, NY
Ellie Pavlick
(Assistant Professor of Computer Science and Linguistics, Brown University and Director, NSF Institute on Interaction for AI Assistants (ARIA))
Location: ZI Kavli Auditorium 9th Floor
Time: 3:00pm
Title: (How) Does AI Think?
Abstract: The increasingly human-like behavior of AI has led to a fascination with ascribing it human-like internal properties — notions like thinking, understanding, and reasoning. In this talk, I will take a step back and discuss a range of results from the past several years of interpretability which present an increasingly consistent view of AI’s internal processing. Specifically, I will discuss how AI represents a surprising amount of internal structure, and yet remains primarily idiosyncratic and context-sensitive. I use the opportunity to talk about the philosophical nature of human-AI comparison–when such analogies are fruitful for scientific and technological progress, and when they mislead.
Zoom: Upon request @arni@columbia.edu

Speaker: Ziwei (Sara) Gong - ARNI Language and Vision Working Group
2026-04-27 15:00 - 16:00 | Virtual
Title: Decoding Human Emotions: From Psychological Theories to Multimodal NLP Models
Abstract: Understanding and modeling human emotions is essential for natural language processing (NLP) applications, from conversational AI to mental health assessment. This talk explores the intersection of emotion theory, dataset development, and multimodal machine learning, highlighting key challenges and innovations in emotion recognition. We discuss the alignment of psychological emotion frameworks with computational models, strategies for improving multimodal emotion recognition, and advances in self-supervised learning for low-resource languages. Additionally, we examine how multimodal signals enhance model performance and interpretability.

ARNI Continual Learning Working Group Meeting
2026-04-30 15:00 - 16:00 | NY
We will feature a team of speakers from Blank Slate Technologies, a startup investigating issues related to continual learning and catastrophic forgetting in human operatives on mission-critical tasks. The team will be led by CEO Matt Trevithick. The title of the talk is the following:

Quantifying Cognitive Performance in the Wild: Measurement, Modeling, and Operational Outcomes

Speaker: Hubert Banville, Meta – ARNI Frontier Models for Neuroscience and Behavior Working Group
2026-05-11 15:00 - 16:00 | Virtual
Hubert Banville, Meta

Title: A foundation model of vision, audition, and language for in-silico neuroscience

Abstract: Cognitive neuroscience is fragmented into specialized models, each tailored to specific experimental paradigms, hence preventing a unified model of cognition in the human brain. Here, we introduce TRIBE v2, a tri-modal (video, audio and language) foundation model capable of predicting human brain activity in a variety of naturalistic and experimental conditions. Leveraging a unified dataset of over 1,000 hours of fMRI across 720 subjects, we demonstrate that our model accurately predicts high-resolution brain responses for novel stimuli, tasks and subjects, superseding traditional linear encoding models, delivering several-fold improvements in accuracy. Critically, TRIBE v2 enables in silico experimentation: tested on seminal visual and neuro-linguistic paradigms, it recovers a variety of results established by decades of empirical research. Finally, by extracting interpretable latent features, TRIBE v2 reveals the fine-grained topography of multisensory integration. These results establish artificial intelligence as a unifying framework for exploring the functional organization of the human brain. We will be hosting Hubert Banville from Meta who will discuss their latest TRIBE fMRI foundation model.

Memory, Neuroscience and AI: Zuckerman Institute’s Local Circuits Symposium
2026-05-12 13:00 - 19:00 | Zuckerman Institute- Kavli Auditorium 9th Fl, 3227 Broadway, NY
Register Here!
How are memories formed, organized, and used to guide behavior? And what can artificial intelligence teach us about how the brain remembers?
Join faculty and early-career researchers from across Columbia for the Local Circuits symposium, exploring the science of memory across biological and artificial systems. Talks will span systems and cognitive neuroscience, machine learning, and theoretical modeling, examining how brain circuits encode and retrieve memories and how AI is helping researchers probe these processes in new ways.
Part of the Zuckerman Institute’s Local Circuits series, this symposium brings together researchers from across the university to spark collaboration around mind, brain, and behavior.
All Columbia ID holders are welcome. Registration is required.
Presented by the Alan Kanzer Center for Cognition and Reasoning
Opening Remarks:
Angela V. Olinto, Provost of the University; Professor of Astronomy and of Physics, Columbia University
Speakers include:
Chris Baldassano, PhD, Associate Professor of Psychology, Columbia University
Christine Denny, PhD, Associate Professor of Clinical Neurobiology (in Psychiatry), Columbia University Irving Medical Center
Stefano Fusi, PhD, Professor of Neuroscience, Principal Investigator in the Zuckerman Institute, Columbia University
Scott Small, MD, Boris and Rose Katz Professor of Neurology, Director of the Alzheimer’s Disease Research Center, Columbia University Irving Medical Center
Kim Stachenfeld, PhD, Senior Research Scientist at Google DeepMind in NYC and Adjunct Assistant Professor at the Center for Theoretical Neuroscience, Columbia University
Richard Zemel, PhD, Trianthe Dakolias Professor of Engineering and Applied Science; Professor of Computer Science; Director of the NSF AI Institute for Artificial and Natural Intelligence (ARNI), Columbia University
Moderator:
Daphna Shohamy, PhD, Kavli Professor of Brain Science; Director of the Zuckerman Institute; Co-director of the Kavli Institute for Brain Science, Columbia University

ARNI Biological Learning Working Group
2026-05-22 16:00 - 17:00 | Virtual
Continuation from prior meetings.
Zoom link: Upon request @ arni@columbia.edu

ARNI Continual Learning Working Group
2026-06-03 11:00 - 12:00 | CEPSR 620, Schapiro 530 W. 120th St
Continuation of prior meeting!
Zoom: request @arni@columbia.edu

ARNI Biological Learning Working Group
2026-06-05 16:00 - 17:00 | Virtual
Continuation from prior meetings
Zoom: request @ arni@columbia.edu

Speaker: Mia Dai – ARNI Frontier Models for Neuroscience and Behavior Working Group
2026-06-08 15:00 - 16:00 | Virtual
Speakers: Mia Dai, PhD student in the Paninski Group at Columbia University
Title: Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding
Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior. Yet, capturing behavior from video in a form suitable for scientific analysis remains a fundamental challenge. Many prior studies represent behavior via pose estimation or nonlinear video embeddings. However, pose tracking discards rich information beyond predefined keypoints, while nonlinear video embeddings lack interpretability. We address this limitation with SABLE (Sparse-view Animal Behavior Latent Embeddings), a self-supervised framework that leverages geometric inductive biases to learn behavior representations. By augmenting a multi-view transformer with priors from monocular depth and pose estimation, SABLE reconstructs 3D animal behavior from extremely sparse views while learning explicit 3D latent structure. Without ground-truth 3D labels, it reliably recovers 3D behavior from two-view videos, whereas state-of-the-art methods fail or yield degenerate solutions. Through comprehensive evaluation, we demonstrate that SABLE learns 3D representations that match or exceed prior SOTA performance in neural encoding and decoding. Our method establishes 3D-aware video embeddings that capture complex behavior, opening new avenues for studying brain-behavior relationships.
Zoom: Upon request @ arni@columbia.edu

CTN: Bence Olveczky (Harvard University) - June 26, 2026
2026-06-26 11:30 - 13:00 | Zuckerman Institute- Kavli Auditorium 9th Fl, 3227 Broadway, NY
Bence Olveczky
Location: ZI
Date: June 26, 2026
Time: 11:30am
Zoom: Upon Request @ arni@Columbia.edu
Title: Neural circuits underlying learned motor sequences.
Abstract: Our ability to sequence movements and actions in response to unpredictable environmental events underlies our rich and adaptive behavioral repertoire. Such flexible behaviors contrast with overtrained, or automatic, motor sequences directed at specific tasks and executed the same way every time. We probed how neural circuits underlie these distinct forms of motor sequence execution by training rats on a ‘piano task’, in which the same motor sequence can be generated in response to unpredictable cues or overtrained to the point of automaticity. By measuring and manipulating neural activity in motor cortex and sensorimotor striatum, we delineate the logic by which these circuits combine to generate both flexible and automatic motor sequences.

Speaker: Konstantinos Barmpas (Imperial College London) and Jarod Lévy (Meta FAIR Paris and Inria MIND) – ARNI Frontier Models for Neuroscience and Behavior Working Group
2026-06-29 15:00 - 16:00 | Virtual
Konstantinos Barmpas (Postdoc Imperial College London)
Title: NeuroRVQ – Multi-Scale Biosignal Tokenization for Generative Foundation Models
Abstract: Biosignals such as electroencephalography (EEG), electrocardiography (ECG), and electromyography (EMG) encode physiological activity across multiple temporal and spectral scales, yielding representations that are rich but challenging for machine learning. Foundation models trained to predict masked signal tokens have shown promise in learning generalizable biosignal representations, yet their performance depends on the tokenizer’s ability to preserve high-frequency dynamics and reconstruct signals with high fidelity. We introduce NeuroRVQ, a modality-adaptive biosignal tokenizer family designed for high-fidelity signal reconstruction. To capture the full frequency spectrum, NeuroRVQ decomposes biosignals into frequency-specific representations via multi-scale temporal convolutions, each encoded into hierarchical RVQ codebooks to preserve high-frequency detail, combined with a novel phase-aware training loss that respects the circular topology of Fourier phase. By tuning the temporal resolution, number and size of temporal kernels and RVQ depth, this design adapts to the spectro-temporal characteristics of each biosignal modality. To validate that tokenizer quality drives downstream performance, we train a simple masked-token foundation model for each modality (NeuroRVQ-FM) using the corresponding NeuroRVQ tokenizer. The NeuroRVQ-FM family achieves competitive or superior downstream performance compared to existing modality-specific foundation models, demonstrating that high-fidelity tokenization is a critical factor for effective biosignal modeling.

Jarod Lévy (PhD Student Meta FAIR/Inria)
Title: DANCE – Detect and Classify Events in EEG
Abstract: Event identification in continuous neural recordings is a critical task in neuroscience. Decoding in EEG is dominated by classifying windows aligned to known event onsets. However, while available in controlled experiments, such onsets are absent in continuous real-world monitoring. Here, we introduce DANCE, a deep learning pipeline that frames neural decoding as a set-prediction problem and jointly detects and classifies events directly from raw, unaligned signals. Evaluated separately on ten datasets curated from the literature with a wide variety of event types (ranging from milliseconds to minutes in duration), our model outperforms existing methods on a broad range of cognitive, clinical and BCI tasks. This single architecture establishes a new state of the art in the competitive task of seizure monitoring and matches the accuracy of onset-informed models for BCI tasks. Overall, our method marks a step towards end-to-end asynchronous neural decoding models.

Zoom: upon request @ arni@columbia.edu

ARNI Continual Learning Working Group
2026-06-30 15:00 - 16:00 | CSB 453, Mudd Building, 500 W 120th Street
Continuation from prior meetings.
Zoom: Upon request @ arni@columbia.edu

CCN 2026 Satelite Event: Modeling and Understanding Human Brain Computation at Scale
2026-08-02 10:00 - 18:00 | To Be Determined
Location: Zuckerman Institute (room TBD)
Date and Time: August 2nd, 10am to 6pm
About:
Recent advances in deep learning and improvements in the quantity and quality of available human brain-activity data (including functional MRI, EEG, MEG, and intracranial recordings) have made it possible to build accurate encoding models of the human brain that can predict neural activity for new visual and auditory stimuli in individual people, even with generalization to new individuals. In parallel, recent decoding models leverage prior information from generative multimodal models to extract rich perceptual and semantic content from brain activity with increasing fidelity. It remains unclear, however, how these technical advances can best be translated into theoretical advances (a better scientific understanding of human brain computation) and impactful applications for the benefit of humanity.
One important goal is to build human brain foundation models that are constrained simultaneously by rich stimulus data, large-scale diverse brain-activity data, and task performance requirements, so as to capture the computations performed by the human brain. This satellite event “Modeling and Understanding Human Brain Computation at Scale” brings together researchers who build neural network models that capture shared structure in neural responses across the human population at scale and use the models to drive theoretical progress on the computations underlying human cognition and perception. A central theme is methodology: What mapping functions, architectures, and training regimes achieve strong generalization and enable interpretation? The event aims to foster dialogue between those collecting and modeling large-scale human brain data and those asking what such models can tell us about how the brain works and how human brain foundation models might be applied for human benefit.

Speaker:Jarod Lévy and Lucy Zhang– ARNI Frontier Models for Neuroscience and Behavior Working Group
2026-09-07 15:00 - 16:00 | Virtual
Title: Brain2Qwerty: Noninvasive decoding of typed sentences from human brain activity
Abstract: Restoring communication for people who have lost the ability to speak or move after a brain injury is a major challenge. While intracranial implants now enable high-performing brain-computer-interfaces, non-invasive alternatives are still lagging behind. Here, we present Brain2Qwerty v2, a model that can decode the production of natural sentences solely from real-time magnetoencephalography (MEG) recordings. By collecting 22,000 sentences typed by nine subjects, each recorded for 10 hours, our model leverages character, word and sentence-level representations to achieve an average word error rate (WER) of 39%. For our best participant, the model accurately decodes half of the sentences with one word error or less. Critically, decoding accuracy log-linearly improves with data volume, suggesting that the performance gap with intracranial approaches could be partially bridged through data scaling. We show that AI enables this performance in three main ways: the substitution of hand-crafted pipelines for event detection with deep learning, the finetuning of large language models to extract semantic representations, and the deployment of AI agents to iteratively refine our decoding pipeline via automated code development. Together, these results show that non-invasive brain-to-text decoding starts to operate at a level of accuracy previously thought exclusive to surgical implants, opening a path toward safe and efficient brain-computer-interfaces.
Zoom: Upon request @arni@columbia.edu

Speaker: Alexandre Pouget - ARNI Distinguish Seminar Series
2026-09-09 15:00 - 16:00 | Zuckerman Institute- Kavli Auditorium 9th Fl, 3227 Broadway, NY
Alexandre Pouget
Date and Time: May 21st at 3pm
Location: Zuckerman Institute Kavli Auditorium 9th Floor
Title: Neural Models of Compositionality
Abstract: Compositionality is widely regarded as one of the cornerstones of general intelligence. It refers to the ability to rapidly generate or learn new concepts by combining simpler ones according to an underlying syntax, as exemplified in natural language. Compositionality was long thought to be primarily a human capacity and widely considered incompatible with artificial neural networks. Recent neural models, however, have begun to challenge this view. I will present two such models: one focused on simple cognitive tasks, the other on the control of complex motor trajectories. In both cases, few-shot learning emerges through the discovery of compositional solutions. Remarkably, the latter approach captures key, and often counterintuitive, aspects of rodent behavior in escape tasks, precisely the kind of setting in which animals exhibit near zero-shot learning. I will also discuss how these findings connect naturally to more sophisticated forms of compositionality in humans, particularly the use of language to support zero-shot learning and inference.
Zoom link: Upon request @ arni@columbia.edu
