PROJECT SUMMARY / ABSTRACT Artificial Intelligence (AI) algorithms have demonstrated success in numerous medical applications. In particular, they excel at analyzing raw medical data formats (such as medical imaging) which has been driven largely by a category of algorithms called deep neural networks (DNN) analyzing single diagnostic tests or modalities indi- vidually. However human disease physiology is rarely confined to one organ system or wholly captured by one diagnostic test. For many of the most clinically-relevant medical applications, the limitation of only being able to analyze one data modality places a major barrier on the types and complexity of medical problems that can be solved. Nearly all important medical decisions require consideration of multiple pieces of information simultane- ously. Current state-of-the-art DNN architectures commonly used in medicine do not readily accept multiple raw data modalities and do not perform highly-complex tasks requiring differential consideration of multi-modal data. To achieve higher-level complex medical reasoning, medical AI algorithms will require fundamental physiologi- cally-focused innovation. The overall objective of this application is to establish a novel physiologically-focused approach to train AI algorithms in medicine that, if successful, will supersede existing AI approaches by func- tioning similarly to the way physician experts triangulate information from multiple sources to arrive at conclu- sions. This research is significant because it addresses the two largest methodologic barriers confronting the real-world application of AI in medicine and that are relevant to all medical specialties: data pre-processing and DNN algorithm architecture. This proposal innovates at the intersection of AI and physiology by developing a new DNN architecture that can accept and learn from multi-modality data in a manner that accommodates a priori medical and physiologic knowledge. Through this novel DNN architecture, resulting algorithms will be able to draw complementary information from multiple inter-related data modalities, similar to how an expert physician considers multiple sources of information to derive a diagnosis or treatment plan. In addition to algorithmic inno- vation, medical AI lacks a scalable approach to perform large-scale data pre-processing, given the heterogeneity of real-world medical data across a wide range of hardware manufacturers and healthcare institutions. Solving this is critical since training data is so important to developing high-performing AI algorithms. This project will also develop an automated approach to perform data pre-processing and harmonization of medical data that is modality-agnostic. To develop and refine these innovations in real-world data, both the data pre-processing pipeline and multi-modal DNN architecture will be applied and prospectively validated to identify heart failure- related phenotypes in a large multi-modality cohort of heart failure patients. The expected outcome of this pro- posal is a general-purpose AI platform that will enable multi-modal medical AI algorithms to be developed that will achieve better performance for existing tasks while also expanding the scope of medical tasks can be ac- complished through AI.

PROJECT NARRATIVE The proposed research is relevant to public health because it will develop a new and broadly-applicable platform to develop multi-modal AI algorithms that achieve higher-level physiologically-focused problem solving across all medical specialties. This would not only improve AI performance overall, but also greatly expands the scope of possible tasks that AI algorithms can accomplish in medicine, while also supporting AI-enabled data-driven discovery of physiology and mechanism. This proposal’s fundamental physiologically-focused innovation in AI algorithm architecture is expected to shift the paradigm by which medical AI is developed and integrates alongside clinicians into the clinical work flow.