Speaker Josue Ortega Caro: ARNI Frontier Models for Neuroscience and Behavior Working Group

Title: Large scale models for spatiotemporal data.

Speaker: Josue Ortega Caro https://josueortc.github.io/

Abstract:  Spatiotemporal and multimodal datasets contain structured variability distributed across space, time, and measurement modality, motivating modeling approaches that can learn representations directly from large-scale data. Inspired by video foundational models, we study how the masked autoencoder training objective can learn shared structure across heterogeneous observations while preserving modality-specific information, and how training these models requires multiple engineering methods for scaling. Furthermore, we show that self-attention supports the emergence of interpretable structure by decomposing them based on the variability across samples. These results suggest that large-scale self-supervised learning provides a unified approach for modeling high-dimensional dynamical systems while enabling interpretation of the learned representations.