Project Summary / Abstract As scientific practice evolves in response to exponential increases in data volume, availability of rapid computational statistics, and the so-called “reproducibility crisis”, researchers are developing new methods for collecting and analyze data in rigorous and responsible fashion. The aim of this proposal is to develop three training units for researchers in the neurosciences that will bring learners up to speed on these developments, improving the rigor and quality of their scientific research by deepening their understanding of the role of statistics in biomedical research. Each unit, developed iteratively in a cycle of testing, evaluation, and revision will be designed for online or classroom use suitable for diverse learning styles. Units will comprise a series of short video segments and interactive exercises that lead learners in a process of guided discovery and self-reflection as they move toward a set of well-specified learning goals. Our units will teach neuroscientists to avoid common pitfalls in designing and analyzing data. In the first unit, we address a set of easy-to-make mistakes wherein a researcher alters her plans midway through the process of data analysis. The practice of HARKing—hypothesizing after the results are known—involves testing hypotheses that are formulated after viewing research outcomes. Outcome switching occurs when a study yields negative results based on the pre- specified outcome measures, but other measures are reported instead. The Garden of Forking Paths refers to the latitude that researchers have in shaping a statistical analysis as they go along. The second unit addresses the problem of publication bias, which arises when authors and journals prefer to publish positive results in favor of negative one, and can lead researchers to reduplicate efforts or draw mistaken inferences from published data. The aim of this unit is to make students aware of problem, teach them how to adjust when reading the literature, and suggest strategies for avoiding publication bias in their own work. The third unit will train students how to figure out whether when a statistical analysis rigorous and reliable. Students will learn how to ask “Are the data appropriate what we want to learn?” “Is the choice of statistical test reasonable?” “Are the inferences supported by the evidence?” By developing this set of units, to be included in a broader neuroscience curriculum, we can train a new generation of biomedical scientists who are well-equipped to work with the vast datasets that are becoming available thanks to new research tools and technologies. These scientists will be able to work more accurately, make new discoveries more efficiently, and advance our knowledge in the health and life sciences at a faster rate than ever before.

Project Narrative Biomedical science increasingly relies upon sophisticated statistical methods to draw conclusions about biological processes and health outcomes. The aim of this proposal is to train the next generation of biomedical and neuroscience researchers to identify and avoid common pitfalls in designing and analyzing data, by developing three education units intended to be included in a larger curriculum. In particular, we will address the following three core issues: changing one's approach midway through a study, accounting for gaps in the scientific literature that arise from selectively publishing successful trials, and learning to question statistical analyses from a study.