Spotlight On: Impact Evaluation

Abt’s impact evaluation work is powered by a small handful of deceptively simple questions: Does this program or policy work? Who does it work for? And how do we know? Even as Abt has expanded its reach across the globe over the last half-century—and the scope and complexity of our projects has increased dramatically in turn—our researchers have remained relentlessly focused on their one core task: developing ever-more sophisticated ways to answer those few key questions.

Along the way, we’ve developed cutting-edge innovations in the design and analysis of social programs and driven real change in the way policy works for the people who need it most. Using state-of-the-art methods and flexible and powerful experimental and non-experimental designs we’ve delivered key insights into homeless assistance systems, international agricultural management, and career pathways—to name just a few! Today’s world is too complicated to navigate without a thorough understanding of what works, for whom, and why. Understanding that is what we do.

• Randomized Experiments
• Quasi-experimental Designs, including:
  • Regression Discontinuity
  • Comparison Group Designs that use Propensity Score Matching
  • Comparative Interrupted Time Series

No two projects are alike, and no single method will work equally well in every situation. Our staff have decades of experience determining the right evaluation approach for an organization’s particular needs—and then executing with excellence on that approach thereafter. The key lies is understanding what isn’t known, and planning for it.

Our researchers are deeply committed to moving their fields forward, and participating in a national community of scholarship around high-quality impact evaluation. When we’re not working on our client projects, our team also publishes cutting-edge new research on the practice of evaluation itself. Some of our favorite recent work can be found here.

Relevant Expertise
 

The Family Options Study
Client: U.S. Department of Housing and Urban Development (HUD)

More than 150,000 families experience homelessness each year and are forced to seek emergency shelter or face life on the street. What are the best ways to help? The Family Options Study is a multi-site random assignment experiment designed to study the impact of various housing and services interventions for homeless families. While the primary outcome of interest is housing stability (and preventing families from returning to homelessness), additional outcomes studied include family preservation, adult & child well-being, and self-sufficiency.

AgResults
Client: U.K. Department for International Development (DFID)

AgResults is a $110 million multilateral initiative funded by Australia, Canada, the United Kingdom, the United States, and the Bill & Melinda Gates Foundation to test the use of “pull mechanisms,” which use prizes to incentivize private companies to develop and disseminate high-impact agricultural innovations that promote food security and benefit smallholder farmers. As external evaluators, we’re assessing the effectiveness of these mechanisms in AgResults pilot projects in Nigeria, Kenya, Zambia, Uganda, Vietnam, and India—to name a few.

Benefit Offset National Demonstration (BOND)
Client: U.S. Social Security Administration (SSA)

Can retaining disability benefits spur beneficiaries to work? That’s the question we’re trying to answer for the Social Security Administration through our implementation and evaluation of BOND. This nine-year project involves randomly assigning more than 1 million disability beneficiaries in 10 randomly selected U.S. sites. Our findings promise critical insights as to whether allowing beneficiaries to keep more of their benefits motivates them to increase their work effort and earnings.

The Summer Electronic Benefits Transfer for Children (SEBTC)
Client: The Food and Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA)

More than 21 million U.S. children received a free or reduced-price lunch each school day in 2014. But accessing these meals is a challenge during the summer, when school is out and summer food assistance operates on a limited scale. Abt and its partners evaluated two levels of monthly benefit amounts—$30 and $60 per child per month—to test their respective impact on food insecurity, nutrition, and more general measures of child well-being.

Pathways for Advancing Careers and Education (PACE)
Client: The Administration for Children and Families (ACF) of the U.S. Department of Health & Human Services (HHS)

In recent years, career pathways have emerged as an innovative framework for improving education, training, and skills, and improving economic self-sufficiency. PACE is the first-ever randomized controlled trial of career pathways programs, featuring nine of the country’s leading and innovative programs. This study will present new evidence on the efficacy of career pathways programs as they work to improve the prospects for low-opportunity Americans.

Contact Us