When deciding how to allocate limited funds for social programs, policymakers and program managers increasingly ask for evidence of effectiveness based on studies that rely on solid methodology, providing credible scientific evidence. The basic claim for the “social experiment”—that the “coin flip” of randomization creates two statistically equivalent groups that do not diverge except through an intervention’s effects—makes resulting estimates unbiased. Despite the transparency and conceptual strength of the experimental strategy for revealing the causal connection between an intervention and the outcomes of its participants, the wisdom or feasibility of conducting social experiments is often questioned on a variety of grounds.

In this article, Abt's Stephen Bell and Laura Peck define fifteen common concerns about the viability and policy reliability of social experiments, in order to assess how much these issues constrain the use of the method in providing policy evidence. The authors use their experience designing and conducting dozens of social experiments to examine the basis for and soundness of each concern. 

The authors conclude that none of the fifteen concerns precludes substantially extending the use of randomized experiments as a means of evaluating the impacts of government and foundation social policies and programs.

Access the full text here.