PROJECT SUMMARY The proposed research and career development plan aim to provide the candidate with the knowledge, experience, and resources necessary to become an independent neurosurgeon-scientist whose research reduces stroke and neurologic disability after neurosurgery through the design and implementation of machine learning (ML) and computer vision (CV) systems that provide surgeons with feedback to improve surgical performance. After formal training, practicing neurosurgeons receive little feedback and instead learn by experience accrued during procedures. However, most do not accrue sufficient case volume to achieve optimal outcomes in every procedure. Ultimately, >16,000 patients are harmed by preventable neurosurgical errors each year, resulting in stroke and neurologic disability in up to 70% and death in up to 16% of affected patients. Unfortunately, the study of harmful adverse events during surgery is obstructed by a lack of datasets containing surgical actions leading up to adverse events or outcomes. The candidate proposes to overcome this limitation by using advanced CV and ML methods to analyze a previously unstudied, multimodal dataset combining pituitary surgical video and clinical data. Pituitary surgery is performed >10,000 times annually in the U.S., can be recorded for analysis, and its steps, errors, and adverse events were recently standardized in an international consensus statement. Specifically, the candidate will test the central hypothesis that the interaction of visible surgeon skill factors with visualized features, including patient anatomy and disease pathology, produces identifiable step-specific surgical errors that result in postoperative stroke, neurologic disability, and other adverse events. Specific Aims: 1) Use CV to identify step-specific errors (defined by tool usage, step progression, and phase characteristics) preceding adverse events; 2) Train ML models to predict upcoming adverse events from prior metrics of surgeon skill and current visual features of the surgical field. Methods to identify high-risk neurosurgical actions, predict upcoming adverse events from a surgeon’s movements, and retrospectively highlight critical timepoints and visual features associated with adverse events are necessary to rationally design and implement interventions to reduce stroke, neurologic disability, and other adverse events. The feasibility and success of this work will be facilitated by the candidate’s outstanding mentoring team, including a surgeon-scientist with experience conducting CV-based surgical performance assessments from procedural visual data and experts in ML using medical image and clinical data, biomedical applications of deep learning in complex prediction models, and multi-institutional pituitary surgical research. In the final year of the award, the candidate will apply for an R01 award to prospectively implement generalizable predictive models (developed in Aim 2) using a larger dataset of videos from several surgical procedures and to develop methods to collect and act upon data from operative video in real-time.

PROJECT NARRATIVE Preventable, intraoperative technical errors harm 16,000 patients in the United States each year, resulting in neurologic injury, stroke, and other disabilities in up to 70% and death in up to 16% of affected patients. Without a systematic data source and method describing intraoperative actions, interventions to prevent intraoperative errors during neurosurgery cannot be rationally designed and implemented. The proposed research seeks to address this problem by constructing a clinical and video dataset of neurosurgical video and patient data, using advanced machine learning and computer vision techniques to ‘watch’ recorded surgical video, automatically detect intraoperative actions, and predict upcoming adverse events using a model which integrates clinical data with surgeon movement metrics and visual features of the surgical field.