Most impact studies in education and other policy areas are designed to estimate the average effects of the program. However, average effects can often mask variation in effects across individual units — variation that can often be more relevant to policy makers. Further, focusing on average effects can mask variation in effects across sites that may be useful for intervention developers and program implementers to understand. Abt Associates researchers are developing cutting-edge approaches to understand both the overall and more specific effects of educational interventions — to learn more than just what works, but also for whom, and under what conditions. Abt researchers are developing:

• Systematic approaches to measuring implementation fidelity so that our evaluations can better answer questions about whether variation in fidelity is related to program impacts;
• Methods for mediational analysis in impact studies using instrumental variables; and
• Methods for moderation analysis in impact studies when subgroups are only observed for treatment groups, such as implementation, fidelity, dosage, and receipt of particular intervention components.

Read journal articles about Abt’s work in education evaluation: Subgroup Analysis in Social Experiments: Measuring Program Impacts Based on Post-Treatment Choice On Analysis of Symmetrically Predicted Endogenous Subgroups Part One of a Method Note in Three Parts On Overfitting in Analysis of Symmetrically Predicted Endogenous Subgroups from Randomized Experimental Samples: Part Three of a Method Note in Three Parts Bias and Bias Correction in Multi-Site Instrumental Variables Analysis of Heterogeneous Mediator Effects