PROJECT SUMMARY Implementation science aims to close the science-to-practice gap, and progress has been made toward consensus on implementation strategies and outcomes. Implementation studies are primarily focused on direct outcomes of implementation strategies, despite evidence that implementation efforts in complex settings such as children’s mental health services lead to a host of multilevel outcomes. Multilevel outcomes include constructs that are not the primary focus; they are indirect, unplanned, or more salient to other stakeholders than the researcher. A systematic review of children’s mental health implementation strategies found only one study systematically investigated unplanned outcomes. The long-term goal of this study is to improve implementation science in children’s mental health services by developing a taxonomy and decision tool for incorporating multilevel outcome measures, and develop one pragmatic measure as relevant to consumers. Including multilevel outcomes may identify important and unexpected results to inform quality improvement, usability, and the effectiveness of interventions as well as learning about negative downstream. Many implementation research projects lack the awareness of or resources to measure multilevel outcomes, which we believe is caused by the current absence of existing guidelines for selecting the most appropriate multilevel outcomes for common implementation strategies. Through use of theory, expert consensus, and stakeholder participation, we will identify a taxonomy of multilevel outcomes that are linked to implementation strategies and are most relevant to four stakeholder roles (consumers/families, providers, policy makers, and researchers). This will be done through three rounds of surveys with a pool of expert participants representing each of these roles, who will rate implementation strategies and associated outcomes as relevant to their stakeholder role. Results will be used to create a decision making tool to identify and measure the full balance of outcomes of implementation strategies in children’s mental health services, which would have several benefits. It would provide a more complete and democratic picture of the cascading and occasionally surprising impacts of implementation efforts, and contribute to theory modification. It would help implementers and policy makers increase the usability and effectiveness of interventions, make better cost-benefit choices, provide opportunities for the prevention of undesirable outcomes, enable innovators to highlight positive multilevel outcomes, and improve precision in predicting implementation.

PROJECT NARRATIVE Implementation science aims to close the science-to-practice gap in children’s mental health services, and has made great progress operationalizing implementation strategies and outcomes to help scientists study the causal determinants of implementation and intervention usability, effectiveness, and sustainability. However, multilevel outcomes (outcomes which are indirect, unplanned, unanticipated, or salient to other stakeholders besides researchers) have received less attention, but are likely to occur during implementation in complex systems such as children’s mental health care. We propose to use a participatory process to develop a taxonomy and decision making resource to assist researchers, intermediaries, and communities to identify and measure the full balance of outcomes of implementation strategies in children’s mental health, and to develop one pragmatic multilevel outcome measure as relevant to youth consumers, in preparation for a larger study.