James E. Pustejovsky

I am a statistician and assistant professor in the College of Education at the University of Texas at Austin, where I teach in the Educational Psychology Department and the graduate program in Quantitative Methods. My research involves developing statistical methods for problems in education, psychology, and other areas of social science research, with a focus on methods related to research synthesis and meta-analysis.

Interests

  • Meta-analysis
  • Causal inference
  • Robust statistical methods
  • Education statistics
  • Single case experimental designs

Education

  • PhD in Statistics, 2013

    Northwestern University

  • BA in Economics, 2003

    Boston College

Recent Posts

2020-06-27

What do meta-analysts mean by 'multivariate' meta-analysis?

If you’ve ever had class with me or attended one of my presentations, you’ve probably heard me grouse about how statisticians are mostly awful about naming things.1 A lot of the terminology in our field is pretty bad and ineloquent.

2020-06-09

Weighting in multivariate meta-analysis

One common question about multivariate/multi-level meta-analysis is how such models assign weight to individual effect size estimates. When a version of the question came up recently on the R-sig-meta-analysis listserv, Dr.

2020-05-03

An update on code folding with blogdown + Academic theme

About a year ago I added a code-folding feature to my site, following an approach developed by Sébastien Rochette. I recently updated my site to work with the latest version of the Academic theme for Hugo, and it turns out that this broke my code-folding implementation.

2019-09-30

Simulating correlated standardized mean differences for meta-analysis

As I’ve discussed in previous posts, meta-analyses in psychology, education, and other areas often include studies that contribute multiple, statistically dependent effect size estimates. I’m interested in methods for meta-analyzing and meta-regressing effect sizes from data structures like this, and studying this sort of thing often entails conducting Monte Carlo simulations.

2019-07-02

Sometimes, aggregating effect sizes is fine

In meta-analyses of psychology, education, and other social science research, it is very common that some of the included studies report more than one relevant effect size. For example, in a meta-analysis of intervention effects on reading outcomes, some studies may have used multiple measures of reading outcomes (each of which meets inclusion criteria), or may have measured outcomes at multiple follow-up times; some studies might have also investigated more than one version of an intervention, and it might be of interest to include effect sizes comparing each version to the no-intervention control condition; and it’s even possible that some studies may have all of these features, potentially contributing lots of effect size estimates.

See all posts

Working papers

Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes

Meta-analysis is a set of statistical tools used to synthesize results from multiple studies evaluating a common research question. Two methodological challenges when conducting meta-analysis include …
Preprint Supplementary materials

Systematic review and meta-analysis of stay-play-talk interventions for improving social behaviors of young children

Stay-play-talk (SPT) is a peer-mediated intervention which involves training peer implementers to stay in proximity to, play with, and talk to a focal child who has disabilities or lower social …
Preprint Supplementary materials

Recent Publications

The impact of response-guided designs on count outcomes in single-case experimental design baselines

In single-case experimental design (SCED) research, researchers often choose when to start treatment based on whether the baseline data collected so far are stable, using what is called a …
Code Journal

Psychosocial interventions for cancer survivors: A meta-analysis of effects on positive affect

Purpose Positive affect has demonstrated unique benefits in the context of health-related stress and is emerging as an important target for psychosocial interventions. The primary objective of this …
Journal

An examination of measurement procedures and characteristics of baseline outcome data in single-case research

There has been growing interest in using statistical methods to analyze data and estimate effect size indices from studies that use single-case designs (SCDs), as a complement to traditional visual …
Preprint Code Dataset Journal

Interventions to enhance self-efficacy in cancer patients and survivors: A meta-analysis of randomized controlled trials

Objective: Self-efficacy expectations are associated with improvements in problematic outcomes widely considered clinically significant (i.e., emotional distress, fatigue, pain), related to positive …
Dataset Journal

Examining the effects of social stories on challenging behavior and prosocial skills in young children: A systematic review and meta-analysis

Social stories are a commonly used intervention practice in early childhood special education. Recent systematic reviews have documented the evidence-base for social stories, but findings are mixed. …
Preprint Code Dataset Journal
See all publications

Recent Presentations

A generalized excess significance test for selective outcome reporting with dependent effect sizes

2019-07-22 1:30 PM Society for Research Synthesis Methodology annual meeting Chicago, IL
Slides

Log response ratio effect sizes: Rationale and methods for single case designs with behavioral outcomes

2019-05-26 5:00 PM Association for Behavior Analysis International Annual Convention Chicago, IL
Slides

Evaluating meta-analytic methods to detect outcome reporting bias in the presence of dependent effect sizes

2019-04-08 12:00 AM American Educational Research Association annual convention Toronto, Ontario
PDF

An examination of measurement procedures and baseline behavioral outcomes in single-case research

2019-04-07 12:00 AM American Educational Research Association annual convention Toronto, Ontario
PDF

The impact of response-guided designs on count outcomes in single-case design baselines

2019-04-07 12:00 AM American Educational Research Association annual convention Toronto, Ontario
PDF
See all talks

Software

lmeInfo

Information Matrices for ‘lmeStruct’ and ‘glsStruct’ Objects

simhelpers

Helper package to assist in running simulation studies

ARPobservation

Simulate systematic direct observation data

clubSandwich

Cluster-robust variance estimation

scdhlm

Between-case SMD for single-case designs

SingleCaseES

Single-case design effect size calculator

Teaching

At UT Austin

Contact

Topics

AERA alternating renewal process behavioral observation cancer causal inference correlation deep-learning delta method demo dependent effect sizes design-comparable SMD distribution theory econometrics effect size experimental design fixed effects generalized linear model hierarchical models hypothesis testing instrumental variables inter-rater reliability meaning and purpose mental health meta-analysis meta-regression missing data multivariate nlme non-linear model non-overlap measures panel data physical health positive affect pre-registration programming psycho-social intervention publication bias question-order R regression discontinuity religion/spirituality response ratio response-guided design robust variance estimation Rstats sandwiches self-efficacy simulation single-case design small-sample social health social networks standardized mean difference Stata systematic review TACC transportation weighting weights