Internal versus External Validity in Rigorous Policy Impact Evaluations: Do We Have to Choose?

Can researchers give policymakers the right information about what is and is not working for the nation as a whole, especially when research is limited to select pockets of the country? This is not as impossible as it sounds.

Read More >


Black Boxes, the Counterfactual, and Bringing Order to RCTs

Abt evaluation experts are engaged in discussions to advance the leading edge of evaluation methods. Recently, Laura Peck and Allan Porowski shared insights on the American Evaluation Association blog AEA365.

Read More >


Getting Meaningful Technical Assistance from Webinars: We Can Evaluate That

The field of technical assistance is changing rapidly. Many organizations that provide national and local technical assistance have moved toward the use of "virtual" TA. How can TA providers evaluate webinar-based TA?

Read More >


Learning Together: Building Stronger Practitioner-Researcher Partnerships

Promoters of evidence-based policies and practices are seeking to engage practitioners more fully in developing and carrying out technically challenging evaluations — notably randomized controlled trials (RCTs).

Read More >


How Can We Measure and Evaluate the Racial Wealth Gap?

The current political cycle has prompted renewed interest in the nation's distribution of wealth - highlighting a disparity that is often called the nation's "wealth gap." The gap has been described as a chasm between the riches of the few set against the struggles of the poor and middle class. Yet a deeper social disparity has challenged public policy since the Civil Rights movement: even today, the fortunes of people of color remain radically different from those of white people.

Read More >


Data without Design: Don’t Do It!

In a recent blog post, Jacob Klerman and I argued that having administrative data available for answering a question about the impact of a program or intervention won't be successful unless paired with a good research design. Here is an all-too-typical example of why relying on administrative data, even where it includes the primary outcome of interest, is insufficient when a participant's entry into a program cannot be explained.  Since my purpose is general and not about the particular study, I’ve anonymized its description.

Read More >


The Long Reach of Inclusive Education

Across the globe, many students with disabilities face challenges in accessing high-quality education. Longstanding misconceptions exist regarding the capabilities of children with intellectual, physical, sensory, and learning disabilities to benefit from formal education. These misunderstandings have, for generations, driven educators to deny these students access to formal schooling.

Read More >

Categories: Education

Sometimes, the Story is in the Subgroups

Program evaluators are often asked to determine whether a given program or program had its intended effects. Getting answers to the question "did it work?" remains the primary goal of many evaluations. But recently, many policy makers and program leaders also want to know "for whom and under what circumstances did the program work?"

Read More >


Want Better Evaluations? First Do This

The ultimate goal of policy analysis is to identify programs that work. Policymakers need to know: Does this program work? For whom does this program work? When does this program work? And would some variant of this program work better? To answer these questions, we need estimates of program impact; i.e., outcomes with the program relative to what outcomes would have been without the program. The "gold standard" approach to estimating impact is random assignment, but other methods are often appropriate.

Read More >


Administrative Data: When Is It Useful for Estimating Impact?

In early October, we had the pleasure of participating in a meeting sponsored by the U.S. Department of Health and Human Services entitled "The Promises and Challenges of Administrative Data in Social Policy Research." Consistent with the title of the meeting, the presentations emphasized both the promise of using administrative data for policy analysis and the real challenges of doing so: getting access to the data, understanding what it means, verifying that it is sufficient for the intended purpose.

Read More >