Call Us: US - +1 845 478 5244 | UK - +44 20 7193 7850 | AUS - +61 2 8005 4826

A stochastic dominance approach to program evaluation

Over 602,000 individual child MUAC measurements were taken in 128 sublocations in 10 arid and semiarid ALRMP II districts between June 2005 and August 2009. Table 1 presents the sample sizes and selected MUAC Z-score statistics for intervention and control sublocations for the two years used in the analysis. We classified sublocations into intervention and control groups according to the cumulative ALRMP II investment data provided by the ALRMP district data managers.3 The distribution of project investments suggests a natural cut-off point with sublocations without any sublocation specific investment forming the control locations and sublocations with some investment the intervention locations. Table 1 Sample size and selected MUAC Z-score statistics for intervention and control sublocations year Sample size Median MUAC Zscore 25th percentile MUAC Z-score 10th percentile MUAC Z-score intervention control intervention control intervention control intervention control 2005/06 71,315 63,863 -1.22 -1.12 -1.80 -1.67 -2.31 -2.14 2008/09 30,046 25,570 -1.15 -1.07 -1.70 -1.64 -2.16 -2.12 Overall levels of nutrition are very low. All summary statistics lie below minus one standard deviations, indicating mild malnutrition on average in this population. The severity of malnutrition is particularly evident when looking at the 25th and 10th percentile of the distribution. For example, more than 10 percent of children fall below the threshold of severe malnutrition of minus two standard deviations. Children in intervention sublocations are worse off than those in control sites. This indicates a placement effect of the ALRMP interventions. Our stochastic dominance difference-in-difference helps control for this non-randomness and avoids the negative bias of simple evaluation methods that fail to control for initial differences between control and treatment groups. These summary statistics suggest that child nutrition improved from 2005/06 to 2008/09 with larger improvements in intervention communities and among the least well-off. To estimate changes over time and compare them between intervention and control sublocations more rigorously we need to construct a panel. Our child-level observations are unsuitable for this for three reasons. First, individual child identifiers are not consistent across time in the data set. Second, MUAC data are not available for all children in all months. Third, and most importantly, the sample of children will necessarily change over time