Peter Halpin
University of North Carolina at Chapel Hill, NC·Education Research
When researchers test whether a new curriculum or teaching method helps students, the answer often depends on which test they used to measure learning. Reviews of science, technology, engineering and math (STEM) education research keep finding the same pattern: assessments built by the people running the intervention tend to show larger effects than independently developed assessments such as standardized tests.
- Fields
Education Research
Mathematics & Statistics
Social Science
- Topics
STEM education research
item response theory
differential item functioning
- Grants
- 1 known grant, through Jun 2027
What they work on
- Develop estimators of treatment effects that are robust to item-level heterogeneity, by combining robust statistics with item response theory, the framework linking student ability to the probability of answering each test question correctly.
- Construct a general-purpose specification test that detects whether conventional treatment effect estimates are biased by variation in effects across individual assessment items.
- Partition the bias in naive treatment effect estimates into components explained by item-level covariates versus idiosyncratic sources tied to a specific outcome measure.
- Evaluate the new procedures at sample sizes and research designs typical of STEM education impact evaluations, and apply them across a sample of such studies to synthesize empirical findings about measure dependence.
More about the research
That gap raises hard questions about how transparent and replicable education research — and social science generally — really is. The problem is not only which test is chosen but that the effect of a program can differ from one test question to the next.
Why it matters
Decisions about curricula and instructional practices rest on impact evaluations whose conclusions can hinge on the test chosen to measure learning. Statistical tools that flag when an estimated effect depends on particular test items could make these evaluations more trustworthy and more replicable, giving researchers, funders and educators a firmer basis for judging what actually works. The same methods may also reveal why studies disagree, which can inform STEM education theory itself.
Known grants
From NIH and NSF research grants. Labs often have other funding we can't see.
Closest labs
Hok Chio Lai
University of Southern California, CA·Psychology·Early-career
Develop a robust optimization algorithm for data harmonization that puts different instruments measuring the same concept on a common scale while also correcting for measurement bias across groups.
Xueying Tang
University of Arizona, AZ·Mathematics & Statistics
Develop a data-driven method to extract informative features from response process data, the logs of test-takers' actions recorded during computer-based assessment items.
Celia Karp
Johns Hopkins University, MD·Public Health & Epidemiology
Develop gender-specific versions of the Desire to Avoid Pregnancy (DAP) scale with construct validity in Uganda, translating items into local languages for use with women and men.
The school
University of North Carolina at Chapel Hill: 72 labs on PI Pages, $1.6B a year on research, 523 research doctorates a year.
Reach out
A start for your email. Fill in the brackets.