PROJECT SUMMARY/ABSTRACT Blunted or flat facial expression is characteristic of schizophrenia (Sz) spectrum disorders and their clinical high risk (CHR) states, and is associated with negative symptoms, social impairment, and poor outcome. Currently, there is no objective test to quantify blunted face expression in Sz. However, with advancements in computational methods, we can begin to operationalize blunted face expression, necessary for both methodological studies and clinical trials, especially as there are no evidence-based treatments. We propose computational analyses of time series of video frame-based estimates of movements of individual face muscles, based on Ekman and Friesen’s Facial Action Coding System or FACS, obtained during ecologically valid open-ended interview on a remote platform (and during a standard viewing paradigm of valenced stimuli). Face action units (AUs) index movement of individual face muscles, which have known physiology and circuitry. Our preliminary data are consistent with early small video coding and electromyography studies of face AUs in Sz, in replicating decreased mean amplitude of face AUs, especially of AU7 (“lid tightener”) or orbicularis oculi, which is involved in smiling, but also social signaling more broadly. Further, in generating matrix and distance profiles for face AU time series data, we find indices of decreased repertoire of face expression in Sz, also associated like amplitudes in CHR with ratings of blunted affect and poor function. Aim 1 is to assess the correlates and psychometric properties of these face expression amplitude and repertoire metrics in a large international cohort of early course Sz spectrum, CHR and healthy individuals well-characterized for demographics, symptoms, cognition and function, (and in whom effects of medications and chronicity may be less). These metrics are expected to differentiate psychosis from the norm, have convergent validity with ground truth ratings of “blunted affect”, and be correlated with social and role impairment, and also face processing ability. Variation by demographics (age, sex, ethnicity) will be assessed, as well as test-retest reliability at one year. Aim 2 is to align video and audio time series to create per frame sets of features and envelope metrics in segments in participants to test for differences in facial dynamics when individuals are speaking vs. listening. We hypothesize that global face expression will decrease significantly during pauses in individuals with Sz, consistent with preliminary data. Based on a cognitive model, we hypothesize decreased face expression and pause behavior will be correlated and associated with slowed processing speed. Aim 3 assesses synchrony of face expression between interviewers and participants, with patients hypothesized to have decreased synchrony and alignment across modalities. Overall, this large rich dataset of multimodal time series raw data will be archived and available for analyses, including more complex nonlinear time series analyses.

PROJECT NARRATIVE Blunted or flat facial expression is characteristic of schizophrenia spectrum disorders, associated with negative symptoms and social impairment. However, there are no evidence-based treatments for decreased face expression, which is typically assessed through clinical observation and ratings. We will use computational analyses to quantify and operationalize face expression deficits during open-ended interview, including temporal patterns with speech and pauses, and nonverbal synchrony between participants and interviewers.