Summary/Abstract, Core D: Behavioral Analysis and Modeling This proposal’s overarching goal is to understand how internal states influence decisions and to identify the underlying neural mechanisms. The Behavioral Analysis and Modeling Core’s development, testing, and application of statistical tools to rigorously characterize behavioral states is critical to achieving this goal. This collaboration will study behavioral state changes defined on three different time scales: those arising spontaneously with engagement and disengagement in a task, those resulting from changing expectations during the task, and those resulting from learning within and across days. The goals of this core are to develop and extend novel open-source analytical tools for extracting state information from behavioral and video data over these three timescales. First, the investigators will identify latent states governing choice behavior, which vary across trials within an experimental session, using tools based on a hidden Markov model with generalized linear model outputs. In addition, they will develop a hierarchical extension of the model to take statistical advantage of the vast behavioral dataset produced by the proposed experimental projects. Next, they will infer behavioral states that vary within a single trial using cutting-edge video analysis methods. In particular, they will apply state-of-the-art markerless tracking methods to extract the position of animal features (paws, tongue, nose, etc.) from behavioral video, and extend these methods to obtain estimates of animal pose in three dimensions (fusing multiple camera views). They will then combine the markerless tracking output with nonlinear autoencoder compression methods to obtain a more informative semi-supervised, low-dimensional data representation of the video data. Using machine learning methods, they will temporally segment the resulting representation to obtain interpretable behavioral states within each trial (e.g., “rest,” “groom,” “reach”), suitable for further downstream analyses. Finally, they will develop new tools to track the dynamics of behavior over the course of learning. Decision-making strategies evolve during training, both within and across sessions, and continue to vary even in well-trained animals. To characterize these state changes, the investigators will develop and apply novel statistical models that combine state-space modeling and reinforcement learning approaches to analyze the learning curves observed in individual animals as they are trained to perform the International Brain Laboratory decision-making task. The resulting framework will quantify how much of the pronounced differences in learning curves across animals can be attributed to differences in identified learning rules, and will help identify neural correlates of inferred learning dynamics in brainwide recordings. All software tools that are developed will be fully open source and will be shared via a public, parallelized cloud implementation for maximal scalability and reproducibility.