James E. Pustejovsky

I am a statistician and associate professor in the School of Education at the University of Wisconsin-Madison, where I teach in the Educational Psychology Department and the graduate program in Quantitative Methods. My research involves developing statistical methods for problems in education, psychology, and other areas of social science research, with a focus on methods related to research synthesis and meta-analysis.

Interests

  • Meta-analysis
  • Causal inference
  • Robust statistical methods
  • Education statistics
  • Single case experimental designs

Education

  • PhD in Statistics, 2013

    Northwestern University

  • BA in Economics, 2003

    Boston College

Recent Posts

Variance component estimates in meta-analysis with mis-specified sampling correlation

\[ \def\Pr{{\text{Pr}}} \def\E{{\text{E}}} \def\Var{{\text{Var}}} \def\Cov{{\text{Cov}}} \] In a recent paper with Beth Tipton, we proposed new working models for meta-analyses involving dependent effect sizes. The central idea of our approach is to use a working model that captures the main features of the effect size data, such as by allowing for both between- and within-study heterogeneity in the true effect sizes (rather than only between-study heterogeneity).

Implications of mean-variance relationships for standardized mean differences

I spend more time than I probably should discussing meta-analysis problems on the R-SIG-meta-analysis listserv. The questions that folks pose there are often quite interesting—especially when they’re motivated by issues that they’re wrestling with while trying to complete meta-analysis projects in their diverse fields.

Inverting partitioned matrices

There’s lots of linear algebra out there that’s quite useful for statistics, but that I never learned in school or never had cause to study in depth. In the same spirit as my previous post on the Woodbury identity, I thought I would share my notes on another helpful bit of math about matrices.

Standardized mean differences in single-group, repeated measures designs

I received a question from a colleague about computing variances and covariances for standardized mean difference effect sizes from a design involving a single group, measured repeatedly over time.

Finding the distribution of significant effect sizes

In basic meta-analysis, where each study contributes just a single effect size estimate, there has been a lot of work devoted to developing models for selective reporting. Most of these models formulate the selection process as a function of the statistical significance of the effect size estimate; some also allow for the possibility that the precision of the study’s effect influences the probability of selection (i.

Working papers

Comparison of competing approaches to analyzing cross-classified data: Random effects models, ordinary least squares, or fixed effects with cluster robust standard errors

Cross-classified random effects modeling (CCREM) is a common approach for analyzing cross-classified data in education. However, when the focus of a study is on the regression coefficients at level …

Cluster wild bootstrapping to handle dependent effect sizes in meta-analysis with a small number of studies

The most common and well-known meta-regression models work under the assumption that there is only one effect size estimate per study and that the estimates are independent. However, meta-analytic …

Multi-level meta-analysis of single-case experimental designs using robust variance estimation

Single-case experimental designs (SCEDs) are used to study the effects of interventions on the behavior of individual cases, by making comparisons between repeated measurements of an outcome under …

Recent Publications

Meta-Analysis with robust variance estimation: Expanding the range of working models

In prevention science and related fields, large meta-analyses are common, and these analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to …

Evaluating the Transition to College Mathematics Course in Texas high schools: Examining heterogeneity across schools and student characteristics

Texas House Bill 5 introduced requirements that school districts partner with institutions of higher education to provide college preparatory courses in mathematics and English language arts for high …

A systematic review and meta‐analysis of effects of psychosocial interventions on spiritual well‐being in adults with cancer

Objective Spiritual well‐being (SpWb) is an important dimension of health‐related quality of life for many cancer patients. Accordingly, an increasing number of psychosocial intervention studies have …

Systematic review and meta-analysis of stay-play-talk interventions for improving social behaviors of young children

Stay-play-talk (SPT) is a peer-mediated intervention which involves training peer implementers to stay in proximity to, play with, and talk to a focal child who has disabilities or lower social …

Evaluating the Transition to College Mathematics Course in Texas high schools: Findings from the second year of implementation

Texas House Bill 5 introduced requirements that school districts partner with institutions of higher education to provide college preparatory courses in mathematics and English for high school seniors …

Recent Presentations

Four things every quantitative social scientist should know about meta-analysis

Meta-analysis is a set of statistical tools for synthesizing results across multiple sources of evidence. Meta-analyses of intervention research are often taken as a gold standard for informing …

Synthesis of dependent effect sizes: Robust variance estimation with clubSandwich

Large meta-analyses often involve dependent effect sizes, but where the exact form of the dependence is unknown. Meta-analysis with robust variance estimation handles this problem through …

Statistical frontiers for selective reporting and publication bias

This workshop will cover methods to investigate selective reporting in meta-analysis of statistically dependent effect sizes, which are a common feature of systematic reviews in psychology. The workshop is organized into two sections.

Synthesis of dependent effect sizes: Versatile models through metafor and clubSandwich

Across scientific fields, large meta-analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to include all dependent effect sizes in a single …

A generalized excess significance test for selective outcome reporting with dependent effect sizes

Software

lmeInfo

Information Matrices for ‘lmeStruct’ and ‘glsStruct’ Objects

simhelpers

Helper package to assist in running simulation studies

ARPobservation

Simulate systematic direct observation data

clubSandwich

Cluster-robust variance estimation

scdhlm

Between-case SMD for single-case designs

SingleCaseES

Single-case design effect size calculator

Students

Current Advisees

Avatar

Young Ri Lee

Graduate student

Avatar

Man Chen

Graduate student

Alumni

Avatar

Megha Joshi

Quantitative Researcher

Avatar

Christopher Runyon

Measurement Scientist

Mascot

Avatar

Whatev's Donkey

Lab mascot

Contact