The frequency of diagnostic errors in emergency departments (ED) is largely unknown but likely to be significant. There is a compelling need to create measurement methods that provide diagnostic safety data to clinicians and leaders who in turn can act upon these data to prevent diagnostic harm. Electronic trigger (e- trigger) tools mine vast amounts of clinical and administrative data to identify signals for likely adverse events and have demonstrated capability to identify diagnostic errors. Such tools are more efficient and effective than other methods and can reduce the number of records requiring human review to those at highest risk of harm. In prior work, we used rules-based e-trigger algorithms to identify patterns of care suggestive of missed or delayed diagnoses in primary care and inpatient settings. For instance, a clinic visit followed several days later by an unplanned hospitalization could be indicative of potential problems with the diagnostic process at the clinic visit. We also proposed a knowledge discovery framework, the Safer Dx Trigger Tools Framework, to enable health care organizations (HCOs) to develop and implement e-trigger tools to measure diagnostic errors using comprehensive electronic health record (EHR) data. Review and analysis of these cases can uncover safety concerns and provide information on diagnostic process breakdowns and related contributory factors, which in turn could generate learning and feedback for improvement purposes. Sophisticated techniques from machine learning (ML) and data science could help inform ‘second generation’ e-trigger algorithms that better identify diagnostic errors and/or harm than rules-based e-triggers that require substantial manual effort and chart reviews. In contrast to rules-based systems, ML techniques could help learn from examples and accurately retrieve charts with diagnostic error without the need for “hand crafting” of an e-trigger. We will apply e-triggers to comprehensive EHRs that contain longitudinal patient care data (progress notes, tests, referrals) that provide an extensive picture of patients’ diagnostic journeys. Using national VA data, including data from 9 million veterans, and data from Geisinger health system, a pioneer HCO that serves approximately 3 million patients, we propose the following aims: Aim 1 – To develop, refine, test, and apply Safer Dx e-triggers to enable detection, measurement, and learning from diagnostic errors in diverse emergency department (ED) settings. We will calculate the frequency of diagnostic errors in the ED based on these e-triggers and describe the burden of preventable diagnostic harm. Aim 2 - To explore machine learning techniques that yield robust, accurate models to predict diagnostic errors using EHR-enriched data derived from expert-labeled patient records containing diagnostic errors (from Aim 1). To our knowledge this is the first ML application in diagnostic error measurement, which could help scale up expert-driven e-trigger development and refinement. Newly developed e-triggers can be pilot tested and implemented at other HCOs, enabling them to create actionable safety-related insights from digital data.

Narrative The process of making a diagnosis in emergency departments can be vulnerable to error. To study the risks involved, we will develop strategies to better identify electronic medical records of patients who may have had a diagnostic error in their care. We will test the use of human-derived rule-based algorithms and data-driven machine learning methods that can more accurately and efficiently identify these medical records than other currently available methods used in measurement of patient safety.