This year, Dr. Laura Dunne and I are serving as program co-chairs for the AERA special interest group on Systematic Reviews and Meta-Analysis, which is a great group of scholars interested in the methodology and application of research synthesis to questions in education and the broader social sciences. We had a strong batch of submissions to the SIG and (since we’re new and still a fairly small group) only a few sessions to fill with them. In assembling this year’s program, Laura and I noted a few common themes that stood out to us. In this post, I’ll highlight a few of them and hopefully whet your appetite to hear more during our sessions at this year’s convention. And if you want to skip the details for now, just take a look at our handy pdf with the full SIG program.
First, two of this year’s presentations deal with network meta-analysis, an approach that goes beyond a single intervention-control comparison, to instead synthesize evidence on the comparative effects of multiple alternative interventions (not just red pill vs blue pill, but also red versus green, green versus blue, etc.). Network meta-analysis is increasingly important in clinical medicine (for example, here’s a recent synthesis examining the relative efficacy of 21 different anti-depressant drugs) but it is still relatively rare in education and other social science meta-analyses. Not in this year’s SIG program though! Both our Sunday morning paper session and Monday round table feature applications of network meta-analysis: one on distance and face-to-face learning, and one on interventions for treatment of post-traumatic stress disorder.
Second, publication bias and other forms of outcome reporting bias remain one of the most vexing challenges for meta-analysis. Our Sunday morning paper session includes an innovative methodological study on how to detect selective outcome reporting in multi-level meta-analyses—an important setting where publication bias techniques have yet to be explored. Even with very sophisticated statistical tools, though, the best way to address publication bias is probably to try and prevent it in the first place. To that end, our Monday round table session includes a presentation on locating unreported outcome data for use in meta-analysis.
Third, the Systematic Reviews and Meta-Analysis SIG has always included a mix of theory and practice. In this year’s program, we’ve tried to preserve that mix within each of our sessions, so that our Sunday paper session and Monday round table each include both methodological research and substantive applications of meta-analysis. We hope that this will promote interesting and valuable dialogues within our community.
Finally, I am very excited that our business meeting will feature an address by Dr. Rebecca Maynard, who is the University Trustee Chair Professor of Education and Social Policy at the University of Pennsylvania Graduate School of Education, and an influential voice in the use of research synthesis methods to inform education and social policy. She’ll be speaking on Expanded Roles for Meta-Analysis in Supporting Evidence-Based Policy and Practice.
Be sure to check out the SIG program for more details and other sessions of interest. I look forward to seeing everyone in Toronto!