Field Experiments in Education Research
How effective is tutoring for improving the mathematics performance of students from economically disadvantaged backgrounds? Does participating in mindfulness training during school improve the mental health and wellbeing of adolescents? Does peer observation of teaching have beneficial effects on student learning? What are the benefits and costs of reducing class sizes in the elementary grades? Such questions focus on understanding the causal effects of an intervention (program, practice, or policy) on educational outcomes, and are often of central interest to educational leaders and policy-makers, program developers, and researchers. Randomized experiments are one way to investigate questions about causal impacts, and indeed, are often accorded a special status due to their potential for clearly identifying causal relationships. Although once rare in education research, randomized experiments conducted in naturalistic educational settings have grown increasingly common and prominent over the past two decades. Planning and conducting such studies involve a unique set of challenges, and their findings have a distinctive profile of advantages and limitations.
This course will cover the design, analysis, and interpretation of randomized field experiments in education research. We will examine the theoretical, statistical, and pragmatic considerations involved in such studies, from articulating a theory of change and selecting outcome measurements, to choosing an experimental design and analytic strategy, to addressing questions of generalizability. Our focus will be on designs most commonly used in educational settings, such as block-randomized or cluster-randomized experiments, and the implementation challenges that arise in such settings, such as non-compliance, attrition, and interference. By the end of the course, students should be able to contribute to the design and planning of a randomized experiment in a realistic field setting, to critically assess reported findings from field experiments, and to make informed judgments about the role of randomized field experiments within education research.