IN PURSUIT OF strongBALANCEstrong RANDOMIZATION IN PRACTICE IN.pdf

IN PURSUIT OF BALANCE: RANDOMIZATION IN PRACTICE # IN DEVELOPMENT FIELD EXPERIMENTS Miriam Bruhn, World Bank Email: mbruhn@ David McKenzie, World Bank, BREAD, CReAM and IZA Email: dmckenzie@ Abstract We present new evidence on the randomization methods used in existing experiments, and new simulations comparing these methods. We find that many papers do not describe the randomization in detail, implying that better reporting is needed. Our simulations suggest that in samples of 300 plus, the different methods perform similarly. However, for very persistent outcome variables and in smaller samples pair-wise matching and stratification perform best and appear to dominate the re-randomization methods commonly used in practice. The simulations also point to specific recommendations for which variables to balance on and for which controls to include in the ex-post analysis. Keywords: Randomized experiment; Program evaluation; Development. JEL codes: C93, O12. # We thank the leading researchers in development field experiments who participated in our short survey, as well as colleagues who have shared their experiences with implementing randomization. We thank Angus Deaton, Esther Duflo, David Evans, Xavier Gine, Guido Imbens, Ben Olken and seminar participants at the World Bank for helpful comments, We are also grateful to Radu Ban for sharing his pair-wise matching Stata code, Jishnu Das for the LEAPS data, and to Kathleen Beegle and Kristen Himelein for providing us with their constructed IFLS data.. All views are of course our own.