Abt Associates: Bold thinkers driving real-world impact
Olsen: Yes, we can. It is sometimes possible to conduct social experiments in truly representative samples so that we can learn about broad populations of policy interest. It is always possible to design studies with the goal of obtaining findings that apply to a broader population. Some social experiments have chosen sites with this goal in mind. And more could do so.
Q2: How can we select sites to obtain more representative samples?
Olsen: We can select sites randomly. If some sites “opt out”, the remaining sites may not be representative. But replacement sites can also be selected randomly. Reluctant sites can be recruited harder and offered incentives to participate. When evaluation sponsors insist on representative samples to inform policy, researchers will work hard to achieve this goal.
Q3: What if despite our best efforts, the study sample is not perfectly representative?
Olsen: I think the effort to obtain a more representative sample is worthwhile, even if it not completely successful. When it’s not, we can make statistical adjustments to generalize the study findings to broader populations. As long as the researchers can obtain data on factors that are associated with the effects of the program, they can use the data to make adjustments. These adjustments are designed to make the study findings more informative to policymakers.
Abt Associates is a mission-driven, global leader in research, evaluation and program implementation in the fields of health, social and environmental policy, and international development. Known for its rigorous approach to solving complex challenges, Abt Associates is regularly ranked as one of the top 20 global research firms and one of the top 40 international development innovators. The company has multiple offices in the U.S. and program offices in more than 40 countries.