Chlorine Vouchers

Summary

Published: June 2023

What is the problem?

The World Health Organization (WHO) estimates that contaminated drinking water causes 485,000 deaths from diarrhea per year and that at least two billion people use drinking water contaminated with feces.1 The Global Burden of Disease (GBD) estimates that in countries with a low Socio-Demographic Index (SDI), unsafe water is responsible for 7.4% of all deaths, and 10.9% of deaths in children under five.2

What is the program?

Chlorination is a well-established method of water treatment. (See our water quality intervention report.)

One potential approach to increasing take-up of chlorination is to offer vouchers (also called coupons) for free chlorine treatment. Implementation details can vary, but in the trials we reviewed, this involved distributing to households yearly printed calendars where each month includes a voucher for a month’s supply of chlorine treatment (e.g., WaterGuard) for an average household size.3 Vouchers could then be redeemed in a variety of locations, including one-stop shops and health clinics (i.e., someone from the household would take the voucher to the one-stop shop every month to receive their free chlorine bottle).4

Researchers hypothesize that offering vouchers for free redemption of chlorine treatment could increase household chlorine use while reducing wastage by introducing a “hassle cost” for those who would otherwise not use chlorine distributed directly to their homes.5

Vouchers programs can target areas with relatively high all-cause mortality rates and low water chlorination rates to achieve higher counterfactual chlorination usage and mortality reductions. They may also target households with children under two or under five, or pregnant women, given the hypothesis that a large share of under-five deaths averted from chlorination is clustered in the neonatal to 12-month age range.6 Given the sizable share of the population that is under five in low SDI countries,7 this targeting strategy could plausibly make vouchers more cost-effective than other water-quality interventions such as Dispensers for Safe Water and in-line chlorination.

Does the program have strong evidence of effectiveness?

In our cost-effectiveness analysis, we model chlorination vouchers as leading to an increase in chlorination rates, which in turn leads to reductions in under-five mortality and over-five mortality, as well as reduced diarrheal morbidity, lower medical costs, and higher income later in life.

To estimate the effect of chlorination vouchers on chlorination rates, we rely on two RCTs (Dupas et al. 2023 in Malawi and Dupas et al. 2016 in Kenya). Dupas et al. 2023 found that an offer of vouchers led to a 26.5 percentage point increase in chlorine use8 at 18-month follow-up, and Dupas et al. 2016 found that receipt of vouchers led to a 32.9% chlorination rate.9 We view this evidence as moderately strong. We have multiple uncertainties regarding the internal validity of these two studies, including lack of pure control group, short-term follow-up, and lack of measurement of health outcomes. We are also uncertain about the extent to which the effect of chlorination vouchers on chlorination rates will generalize to future implementation settings. (more)

These trials did not measure effects on child mortality. To translate the chlorination effect into mortality, we rely on two meta-analyses (Kremer et al. 2022 and GiveWell’s meta-analysis of a subset of five RCTs included in Kremer et al. 2022) of the effect of water chlorination on all-cause mortality rates among children under five. Our best guess is that chlorination interventions similar to those we are interested in implementing (i.e., Dispensers for Safe Water, in-line chlorination, and vouchers) reduce all-cause mortality in children under five by about 12% in low-income settings.10 We adjust this effect to account for differences in uptake of chlorination from vouchers, relative to chlorination interventions in the GiveWell and Kremer et al. 2022 meta-analyses. This estimate is imprecise, and we have significant uncertainty about the magnitude of the effect, potential publication bias, and reliance on interventions that were different from those we’d consider funding. (more)

We do not have direct evidence on chlorine’s effect on mortality reduction in people over five, but because enteric infections kill people in this age range, we find it plausible that chlorine interventions reduce mortality in people over five as well. We estimate this effect is 39% as large as the effect on children under five11 in low-SDI countries. We are moderately uncertain about whether vouchers result in mortality reductions in people over five given the lack of direct evidence that we are aware of. (more)

There is also direct evidence of moderate quality on chlorine’s effect on diarrhea morbidity, and indirect evidence of varying quality on chlorine’s effect on development and on averting medical costs. (more) Our main uncertainties are about the magnitude of the effect of chlorination on child morbidity, its development effects, and to what extent it averts medical costs.

Evidence of vouchers on water chlorination

Overall, we believe there is moderately strong but limited evidence that chlorine vouchers increase water chlorination rates, and limited evidence that chlorine vouchers reduce diarrheal illness. We rely on evidence from two RCTs of chlorine vouchers: Dupas et al. 2023 in Malawi and Dupas et al. 2016 in Kenya.

Dupas et al. 2023

Dupas et al. 2023 was an RCT of chlorine vouchers that took place in Malawi between 2018 and 2019.12 The trial randomized 2,313 households with children under six.13 We focus on the subset of the study where households were randomly assigned to one of three treatment arms:14 (i) a treatment arm that received 18 months of coupons for free chlorine redeemable at local shops; (ii) a treatment arm that received free monthly chlorine deliveries by community health workers (CHWs); or (iii) a control group. Participating shop owners received a monthly stipend and signed a contract agreeing to participate in the redemption program.15 The average redemption rate in the vouchers arm was 48%,16 with suggestive evidence of gradual fade out over time.17

The study found the following effects:

• 26.5pp increase in chlorine use (30.4% T vs. 3.8% C, p0.01)18 at 18-month follow-up
• 4.5pp reduction in caregiver-reported diarrhea in the prior month for under-five children (11.9% T vs. 16.4% C, p0.01)19

We view this study as being generally well-conducted with moderately high internal validity due to key study limitations. It was pre-registered,20 had decent baseline equivalence,21 measured chlorine use via water quality tests during unannounced visits,22 and had moderate but non-differential attrition at the latest follow-up point.23 Our primary concerns are short-term follow-up24 and reliance on self-reports for caregiver-reported diarrhea,25 which introduces the possibility of social desirability bias and of overstating vouchers’ benefits on diarrhea.

Dupas et al. 2016

Dupas et al. 2016 was an RCT of chlorine vouchers that took place in Kenya in 2007 and 2008.26 The trial targeted parents of children 6 to 12 months old, recruited from waiting rooms of maternal and child health clinics.27 A sample of 1,118 participants were randomized to one of three treatment arms: (i) a treatment arm that was offered chlorine for immediate purchase at a 50% discount, and individuals could buy up to five 150-mL bottles;28 (ii) a treatment arm that was offered 12 vouchers (on a calendar to track their expiration), each redeemable for one 150-mL bottle of chlorine in a local shop or clinic;29 or (iii) a treatment group that was offered two 500-mL bottles of chlorine for free, one immediately and the second three to five months later.30 There was no pure control group. Among those in the vouchers group, 85% redeemed at least one voucher, and an average of 40% of the 12 vouchers were redeemed per participant.31

The study found the following effects:

• 32.9% of the vouchers group had a positive chlorine test at three- to five-month follow-up, compared to 12.4% of the cost-sharing group and 34% of the free delivery group.32

We view this study as being moderately well-conducted with moderate internal validity. It was pre-registered,33 had good baseline equivalence,34 measured chlorine use via water-quality tests35 (though it is not clear whether these home visits were announced, which could result in households purifying water in advance of the visit due to social desirability bias), and low and non-differential attrition.36 Our primary concerns are short-term follow-up,37 lack of a pure control group,38 no measurement of health outcomes (e.g., caretaker-reported child diarrhea),39 and lack of baseline water quality testing.40

External validity

We are very uncertain about the extent to which these findings will generalize to future implementation contexts. Specific external validity concerns from Dupas et al. 2023 and Dupas et al. 2016 include:

• Significant research involvement and oversight in the RCT. In Dupas et al. 2023, shop owners were given monthly stipends to incentivize their participation in vouchers redemption, and no shop ever ran out of WaterGuard.41 In Dupas et al. 2016, redemption locations were monitored to ensure stockouts would not prevent voucher redemption.42 In at-scale contexts, we expect stockouts to occur, which could reduce redemption rates and overall cost-effectiveness.
• Variations in baseline diarrhea prevalence. Survey data from Malawi around the time of Dupas et al. 2023 showed that approximately 20% of children had a case of diarrhea in the previous two weeks.43 Though health outcomes were not measured in Dupas et al. 2016, they note that the region had the second highest prevalence of diarrhea in Kenya.44 In contexts with higher diarrhea prevalence rates or higher all-cause mortality rates, cost-effectiveness could increase (and it could decrease if the rates were lower).
• Variations in baseline chlorination rates. Baseline chlorination rates were low in Dupas et al. 2023 at 5%45 and not measured in Dupas et al. 2016, though they were likely higher than in Malawi given higher chlorination rates in Kenya generally. Higher baseline chlorination rates could lead to lower cost-effectiveness, and lower baseline chlorination rates could lead to higher cost-effectiveness.
• Variations in chlorine acceptability and use. Baseline survey data from Dupas et al. 2023 indicate that 71% of respondents believe chlorine makes their water safe, and 46% believe chlorine makes their water taste bad.46 In areas with more familiarity with chlorination benefits and greater acceptability of chlorine, vouchers take-up could be greater, increasing overall cost-effectiveness. In areas with less familiarity and less acceptability, take-up of vouchers could be lower, decreasing overall cost-effectiveness.
• Variations in redemption rates between shops and health clinics. In Dupas et al. 2016, researchers partnered with three shops near the health clinics where participants were recruited, and one health clinic that was located far from any shop, to serve as vouchers redemption sites.47 Dupas et al. 2016 does not report the percentage of vouchers that were redeemed at shops as opposed to clinics but notes that omitting the clinic-redeeming subsample does not affect results.48 This provides suggestive but weak evidence that, within this trial context, the redemption site did not affect voucher redemption or downstream health effects.

Evidence of water chlorination on under-five mortality

The two RCTs on chlorination vouchers do not measure effects on mortality, which is the primary outcome we’re interested in. To measure the effect on mortality, we rely on two main sources of evidence: (i) a meta-analysis by Kremer et al. 2022 on the effect of water chlorination on under-five mortality, and (ii) a meta-analysis by internal GiveWell staff of a subset of five RCTs included in Kremer et al. 2022. The latter informs the effect size used in our cost-effectiveness analysis. We then adjust that estimate based on the increase in chlorination caused by vouchers and the share of deaths due to enteric infection occurring in settings where vouchers would be implemented in the future.

Main effect from water chlorination meta-analysis

Our best guess, based on the GiveWell meta-analysis, is that programs to increase chlorination reduce all-cause mortality in children under five by 12% in low-income settings.49 Our water quality intervention report contains more detail on how we derived our impact estimate of water chlorination interventions on all-cause mortality in children under five in settings without safe water. A brief summary is below:

• Kremer et al. 2022—the only published meta-analysis of RCTs that quantifies the impact of water quality interventions in children under five in low-income settings—is a meta-analysis of 17 published and unpublished water quality RCTs. Fifteen trials with child mortality data were included in its main analysis.50 Twelve of these 15 trials involved chlorination interventions,51 although only four of the trials involving chlorination used interventions that were limited to chlorination.52
• Depending on the method used to pool results, Kremer’s meta-analysis found that water quality interventions reduce the odds of all-cause mortality in children under five by 25% (95% confidence interval, 8% to 45%) or 30% (95% credible interval, 8% to 51%).53
• A comprehensive discussion of our internal and external validity concerns regarding Kremer et al. 2022 is included here. Key limitations include possible publication bias,54 bundled treatments,55 and limited external validity.56
• To try to account for these limitations, GiveWell conducted an alternate meta-analysis that pooled data from a subset of five RCTs included in Kremer et al. 2022 that tested chlorination interventions similar to those we are interested in implementing (e.g., Dispensers for Safe Water, in-line chlorination, vouchers) and which we believe are the least susceptible to publication bias and other study limitations.57
• Our current internal estimate suggests that chlorination interventions reduce all-cause mortality in children under five by about 12% in low-income settings (95% confidence interval, 31% reduction to 13% increase).58 This is the primary mortality estimate we model in our cost-effectiveness analysis.
• We apply a 25% downward internal validity adjustment to account for bundling of interventions included in the meta-analysis, to come up with a rough guess of 9% reduction in all-cause mortality from chlorination (post-internal validity adjustment) in these trials.59
• This estimate is imprecise and not statistically significant by conventional standards. We do not view the evidence underlying it as strong, but we nevertheless believe it is the most informative estimate available due to its directness.

Adjustments for vouchers

We make the following adjustments to extrapolate this effect of vouchers programs to other settings:

• Adherence. We apply a downward adjustment to account for vouchers programs increasing chlorination rates by less than the trials included in the Kremer et al. meta-analysis. We estimate this parameter by calculating the ratio of the estimated overall increase in water chlorination due to vouchers to the estimated increase in water chlorination as a result of the interventions included in Kremer et al. 2022.
• Share of deaths addressable by chlorination. We apply an upward adjustment to account for our assumption that vouchers programs we would consider funding would be implemented in countries with a higher share of deaths due to enteric infection, relative to settings studied in the Kremer et al. meta-analysis.

Together, these adjustments form our external validity adjustment used for under-five mortality. Given these adjustments, our best guess is that vouchers programs reduce all-cause child mortality by roughly 8%.60

Our cost-effectiveness analysis of chlorination interventions is very sensitive to adherence. Our estimate of adherence to vouchers is particularly uncertain because we rely mostly on chlorination usage data from one trial (Dupas et al. 2023) because Dupas et al. 2016 did not measure baseline chlorination rates and did not have a pure control group.61 We plan to further refine this adjustment as we receive additional evidence on vouchers and investigate opportunities to fund direct implementation of vouchers.

Effect of water chlorination on over-five mortality

Kremer et al. 2022 does not measure the effect on mortality of individuals over five. However, because diarrhea causes death in individuals in other age groups, we guess that the same effect applies.

To estimate the effect on over-five mortality, we apply a scaling factor on the under-five effects that accounts for a lower share of deaths in over-fives due to enteric infection.62 Based on this, our best guess is that the percentage reduction in all-cause mortality is roughly 39% as high as under-five mortality.63

We discuss this approach further in our water quality intervention report.

Evidence of water chlorination on additional benefits

We think chlorination may lead to additional benefits as well, including increased income later in life for individuals that experience better health early in life due to chlorination (development effects), reduced diarrhea morbidity, and lower medical costs. We discuss these further in our water quality intervention report.

Potential additional benefits of vouchers

Vouchers might have a number of other benefits outside of mortality, morbidity, development effects, and medical costs averted. A few examples of potential additional benefits are:

• Increase in child preventive healthcare utilization or treatment: If vouchers are distributed or redeemed at health clinics, it’s plausible that they could incentivize households to bring children into clinics more regularly than they would otherwise, inducing more health-seeking behavior and downstream health impacts.64 This could improve child health outcomes in ways not currently accounted for in our cost-effectiveness model. While plausible, we think this is unlikely to make up a significant share of benefits given that we’re already modeling reductions in all-cause mortality for children and diarrhea morbidity.
• Reduction in illness from indoor air pollution: Water treatment could both reduce the need to boil water and reduce childhood bouts of illness that require children to stay indoors65 —plausibly indirectly reducing exposure to and illness from indoor air pollution. This is speculative and we have not investigated this hypothesis in detail.

Potential offsetting/negative effects of vouchers

We are not aware of any significant adverse effects of household water chlorination, or vouchers specifically. We discuss this further in our water quality intervention report. We plan to investigate context-specific negative effects as we consider vouchers grant opportunities.

How cost-effective is the program?

We conducted a preliminary cost-effectiveness analysis. As of March 2023, we estimate that this program is in the range of cost-effectiveness of programs to which we expect to direct funding.

Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

This cost-effectiveness analysis is at an early stage, and we think it’s likely that our bottom line cost-effectiveness estimate will change with further review and consideration of specific implementation contexts.

Household chlorination is generally inexpensive and easily accessible, and can lead to a substantial decline in all-cause mortality, primarily in children under five. This leads to high cost-effectiveness.

A sketch of the cost-effectiveness model is below:

• Prevalence of under-five mortality: We use data from low SDI countries for illustration. Based on Global Burden of Disease 2019 data, annual all-cause mortality in children under five is estimated to be 1.6%, and 0.5% in over-fives.66
• Effect of vouchers on under-five all-cause mortality: Our initial estimate for the effect of water chlorination on all-cause mortality—a 12% reduction—is based on data from GiveWell’s meta-analysis of a subset of five RCTs included in Kremer et al. 2022.67 Our best guess, which incorporates adjustments for speculative concerns about the internal and external validity of the study, is that vouchers reduce under-five mortality by 7.6% in low-SDI countries.68 These are highly subjective estimates that we use for the purposes of comparing the cost-effectiveness of different interventions as directly as possible to each other. Our best guess is that 51% of vouchers’ benefits come from averting under-five mortality.69
• Effect of vouchers on over-five mortality: Our best guess is that vouchers reduce over-five all-cause mortality by 3% in low-SDI countries.70 This estimate incorporates downward adjustments for speculative concerns about internal and external validity of the study. Kremer et al. 2022’s evidence for mortality reduction applies to under-five children. To estimate the degree of mortality reduction in people over five, we adjusted the under-five percent mortality reduction figure using the relative percent of all-cause mortality attributed to diarrhea in under-fives vs. over-fives.71 We estimate that 11% of vouchers’ benefits come from averting over-five mortality.72
• Effect of vouchers on other outcomes (diarrhea morbidity, development effects, medical costs): We estimate that 1%, 27%, and 10% of vouchers’ benefits come from averting diarrhea morbidity, development effects, and averting medical costs, respectively.73
• Cost of the program: We estimate that vouchers distributed via health clinics to women with children under five costs $1.19 USD annually per person served.74
• Cost-effectiveness: Using an estimate of $1.19 per person served, our best guess is that vouchers averts mortality for $2,860.75 As of March 2023, our best guess is that vouchers in low-SDI countries are 14 times as cost-effective as cash transfers.76 This ranges from 9x in Kenya to 41x in Nigeria.77

However, we have high uncertainty about intervention costs, baseline under-five mortality rates, baseline chlorination rates, mortality reduction in under-fives, implementation locations, intervention take-up and adherence, duration of effects, and magnitude of additional benefits.

• Intervention costs. We have high uncertainty about the cost estimate that we use. Specific considerations that could affect costs include (i) distribution method (e.g., whether vouchers are distributed via door-to-door delivery or via health clinics during routine antenatal care appointments); (ii) redemption method (e.g., in Dupas et al. 2023, local one-stop-shops were given monthly incentives to participate in chlorine redemption78 ); and (iii) whether behavior change is incorporated into the program to increase chlorine take-up and adherence, which would likely increase costs. While we have limited confidence in these estimates, we believe they are the best ones at our disposal at this stage of our investigation. We expect to be able to refine our estimates as we investigate a specific funding opportunity.
• Baseline under-five mortality rates. We are interested in funding opportunities to implement vouchers in settings with high baseline rates of under-five mortality. Based on Global Burden of Disease data, we estimate the under-five mortality rate to be around 1.6% in low-SDI countries,79 which are likely roughly representative of settings in which we would consider funding vouchers in the future, though it is possible that baseline mortality rates would be higher or lower in specific populations where vouchers would be well-suited. This rate varies considerably across and within countries (e.g., 2.3% in Nigeria and 0.83% in Kenya).80
• Baseline water chlorination rates. Our model currently takes into account baseline chlorination rates and effects on chlorination take-up (both taken from Dupas et al. 2023 and Dupas et al. 2016),81 and adjusts for the fact that adherence is lower in the two vouchers trials than the trials in the Kremer et al. meta-analysis.82 We expect that the baseline mortality rate among children targeted by vouchers will depend on the country of implementation and target population.
• Mortality reduction in under-fives. We currently rely on GiveWell’s internal meta-analysis estimate of chlorine’s impact on under-five mortality, which comes from a subset of five component RCTs included in Kremer et al. 2022. This estimate is noisy and imprecise, and we hope to improve its precision by funding future research that measures chlorine’s effects on under-five mortality.
• Implementation locations. We currently model the average effects of vouchers in low-SDI countries and are very uncertain about the specific contexts in which we’d consider funding vouchers at scale. These locations matter considerably as they’re likely to vary in important parameters such as baseline water chlorination rates, baseline mortality rates, vouchers take-up rates, and adherence rates. It’s likely that our model will change over time to better reflect a specific funding opportunity.
• Intervention take-up and adherence. Our cost-effectiveness analysis is very sensitive to the adherence rate for vouchers, which we estimate using trial data from Dupas et al. 2023 and Dupas et al. 2016. However, Dupas et al. 2016 did not measure baseline chlorination rates—a key input into the adherence calculation—and did not have a pure control group, so we impute baseline chlorination rates from Dispensers for Safe Water and in-line chlorination in Kenya. Given the high levels of researcher oversight in trial contexts and the lack of data on voucher adherence in non-trial contexts, we are unsure about the extent to which these adherence rates are representative of future implementation contexts we’d consider funding. We are very uncertain about this estimate and plan to further refine it as we investigate specific opportunities.
• Duration of effects. Unlike Dispensers for Safe Water and in-line chlorination, vouchers require more behavior change on behalf of the voucher recipient since recipients must treat their water on an ongoing basis to derive the benefits. Given the short-term follow-up periods in Dupas et al. 2016 and Dupas et al. 2023, we are uncertain about the extent to which chlorination rates are sustained over a longer period. Dupas et al. 2023 provided suggestive evidence of chlorine redemption fadeout over time,83 which would imply waning chlorination rates and mortality effects as well.
• Additional benefits (e.g., over-five mortality, development effects, medical costs averted). We think it is likely that vouchers yield mortality reductions in individuals over five, as well as reductions in medical costs and improvements in long-term earnings resulting from exposure to vouchers in childhood. We are very uncertain about the assumptions used in our modeling of these outcomes, and expect that they may change with future work.

Is there room for more funding?

We are not aware of any organizations or governments implementing chlorine vouchers at a significant scale.

We speculatively estimate that voucher programs have total room for more funding of approximately $152 million per year above 10x cash. This is a highly speculative estimate for a general vouchers program; the specific amount of available room for more funding will likely depend on key implementation details such as distribution method (e.g., whether households are targeting at home or via health clinics) and target population (e.g., pregnant women, households with children under five or under ten), as well as the existence of other safe water interventions such as Dispensers for Safe Water or in-line chlorination.

How feasible is implementation of the program?

Based on a shallow review of the feasibility of implementing vouchers, our impression is that vouchers would be reasonable to implement with high quality on a large scale. The main advantages of vouchers are that they are low-cost and low-tech, and can therefore be implemented in low-resource settings and potentially capitalize on existing distribution channels such as public health clinics. However, the program requires behavior change from end users (i.e., continuously chlorinating household water on an ongoing basis), and prior research and conversations we’ve had with potential implementers and researchers suggests that there are potential barriers to adoption.

Barriers to adoption

Based on limited conversations with potential implementers and evaluators of vouchers, as well as Dupas et al. 2023 and Dupas et al. 2016, we identified the following barriers to adoption:

• Behavior change on behalf of voucher recipients. Unlike passive safe water interventions such as in-line chlorination, which automates chlorine dosing, vouchers require participants to travel in order to redeem their vouchers and to treat their water on an ongoing basis. Our impression is that sustained behavior change is difficult to achieve, and it’s plausible that chlorination rates will decline over time.
• Chlorine acceptability. To a greater extent than in-line chlorination, chlorine acceptability is key to take-up of vouchers.84 In-line chlorination can operate in environments where chlorine acceptance is low because in-line chlorination devices are calibrated to dose an effective amount of chlorine solution without leaving a taste.85 We have heard anecdotal evidence that Aquatab has less of a taste issue but is more expensive than Waterguard, a common chlorination solution.86 We plan to investigate chlorine acceptability as part of our investigation into specific funding opportunities.
• Coordination across voucher distributors and redemption locations. Some level of buy-in is likely required on behalf of entities participating in either voucher distribution or redemption (e.g., health clinics and one-stop shops). We plan to investigate the extent to which this is a barrier as we consider specific funding opportunities.
• Likelihood of stockouts. Without significant researcher oversight ensuring stockouts do not occur, it is unclear how reliable chlorination supply chains are and whether stockouts are common, which would reduce voucher redemption and water treatment rates. We plan to investigate this—specifically partnerships between implementers and chlorination suppliers—as we consider specific funding opportunities.

Focus of further investigation

We are interested in funding additional research on vouchers or other water chlorination interventions (Dispensers for Safe Water or in-line chlorination) to (i) improve the precision of water chlorination's mortality effect size in the Kremer et al. 2022 and GiveWell meta-analyses, and (ii) better understand the overall cost-effectiveness of vouchers relative to Dispensers for Safe Water and in-line chlorination. Our next step is to investigate an RCT of vouchers and to better understand the program's key implementation details in a trial context and at scale.

Key questions for further investigation

Questions regarding vouchers implementation

• How much do vouchers cost?
• Where and how can vouchers be feasibly implemented?
• What is the optimal implementation environment for vouchers vs. Dispensers for Safe Water vs. in-line chlorination?
• What are priority countries for implementation, taking into account factors like under-five mortality rates, baseline water chlorination rates, and public sector interest in supporting scale-up?
• How will chlorine vouchers look when implemented in particular settings? For example:
  • Who is the target population (e.g., pregnant women, households with children under five or under two, households in rural or peri-urban areas)?
  • How will the vouchers be distributed (e.g., through door-to-door visits by community health workers or at health clinics during routine antenatal care visits)?
  • How will the vouchers be redeemed (e.g., at one-stop shops or health clinics)?
  • Will the vouchers be paper vouchers or e-vouchers? If the former, will they be distributed as calendars that remind users to pick up their monthly allotment of chlorine? If the latter, how difficult will the technology be to operationalize?
• How will implementers mitigate potential chlorine stockouts and wastage?
• Would incorporating an additional behavior change component improve counterfactual take-up and adherence rates, and overall cost-effectiveness?

Questions regarding the link between chlorination and benefits modeled

• How will the targeting of vouchers and various selection effects affect overall chlorine uptake and mortality reduction? For example, will targeting pregnant women in health clinics lead to lower mortality reductions than targeting pregnant women who are not receiving antenatal care, since the women visiting health clinics may be more likely to treat their water anyway?
• What is the long-term effect of vouchers on water chlorination? Do the effects fade out over time?
• What is the effect of chlorination on under-five all-cause mortality?
• What is the effect of chlorination on over-five all-cause mortality?
• What is the effect of chlorination on household health expenditures?
• What is the effect of childhood chlorination on long-term income?

Our process

• We reviewed Dupas et al. 2023, Dupas et al. 2016, and Kremer et al. 2022 in detail.
• We conducted a literature review of vouchers’ impact on chlorination. To find literature on vouchers, we searched Google Scholar and followed the citation trails of Dupas et al. 2023 and Dupas et al. 2016 to find trials of vouchers conducted after these two RCTs were completed. We did not identify any additional studies for inclusion.
• We drew from previous work GiveWell has done on water quality interventions,87 including incorporating GiveWell's alternate meta-analysis of a subset of Kremer et al. 2022 component trials.
• We had conversations with potential implementing partners and evaluators. We spoke with a few organizations that are interested in implementing vouchers, as well as researchers who have evaluated vouchers, to understand what potential implementation details could look like and in which specific geographies they would be most appropriate.
  

  Sources

  Document	Source
  AEA RCT Registry, "Community health workers, subsidies and safe drinking water: Experimental evidence from Malawi," 2018	Source (archive)
  AEA RCT Registry, "Water Treatement Promotion at Health Clinics in Kenya," 2016	Source (archive)
  Boisson et al. 2013	Source (archive)
  Crider et al. 2018	Source
  Dupas et al. 2016	Source
  Dupas et al. 2016, Supplementary materials	Source
  Dupas et al. 2020 (working paper)	Source
  Dupas et al. 2023	Source
  Dupas et al. 2023, Online Appendix	Source (archive)
  GiveWell, Vouchers water-quality CEA, 2023	Source
  GiveWell, Water quality cost-effectiveness analysis, 2022	Source
  GiveWell, "Water quality interventions," 2022	Source
  GiveWell, Water vouchers value of information BOTEC, 2023	Source
  GiveWell, "Why we can’t take expected value estimates literally (even when they’re unbiased)," 2016	Source
  Haushofer et al. 2021 (working paper)	Source
  Institute for Health Metrics and Evaluation, Global Burden of Disease 2019 results, Deaths from unsafe water source, Low SDI countries, 2019	Source
  Institute for Health Metrics and Evaluation, "Socio-demographic Index (SDI)"	Source (archive)
  Kirby et al. 2019	Source (archive)
  Kremer et al. 2011	Source
  Kremer et al. 2022 (working paper)	Source (archive)
  Lantagne and Clasen 2012	Source
  Null et al. 2018	Source (archive)
  Peletz et al. 2012	Source (archive)
  World Health Organization, "Drinking-water," 2022	Source (archive)

  • 1
    • “Microbiologically contaminated drinking water can transmit diseases such as diarrhoea, cholera, dysentery, typhoid and polio and is estimated to cause 485,000 diarrhoeal deaths each year.” World Health Organization, "Drinking-water," 2022.
    • "Globally, at least 2 billion people use a drinking water source contaminated with faeces." World Health Organization, "Drinking-water," 2022.

  • 2

    Institute for Health Metrics and Evaluation, Global Burden of Disease 2019 results, Deaths from unsafe water source, Low SDI countries, 2019.

  • 3
    • “Households could redeem one coupon per month for 18 months. Each coupon corresponded to one unique month (e.g., the coupon for March 2018 could only be redeemed in March 2018). Coupons were attached to a calendar which was also given to control households and which many households displayed on their wall (see Figure A1 in the online appendix). Each coupon was exchangeable for one 150-milliliter bottle of WaterGuard, enough to treat a standard 20-liter jerry can of water approximately 30 times. One bottle, if used correctly, is enough for roughly one month’s supply of treated water (for drinking and cooking) for a family of five." Dupas et al. 2023, p. 282-833.
    • "The average household has just over five members." Dupas et al. 2023, p. 282.

  • 4
    • "Households assigned to the coupon intervention received coupons that could be
      redeemed for a free bottle of WaterGuard at a local shop." Dupas et al. 2023, p. 282.
    • "Twelve vouchers, each redeemable for one 150 ml bottle of water treatment solution at either
      a local shop or the clinic itself were provided." Dupas et al. 2016, p. 4.

  • 5

    “One mechanism proposed to reduce wastage is to require that households pay some hassle cost in order to get the free water treatment product. Dupas et al. (2016) show that coupons that required retrieval of free chlorine from a local shop achieved the same level of chlorine use as free home delivery at a much lower cost, since households not interested in using the product do not bother to redeem coupons but do not refuse the home delivery.” Dupas et al. 2023, p. 277.

  • 6

    This is based on unpublished conversation notes with Michael Kremer at the Development Innovation Lab.

  • 7

    See our estimate here.

  • 8
    • For treatment group rate, see Dupas et al. 2023, p. 289, Table 2, column: (1) Positive Chlorine Test, row: Pooled Coupon Effect.
    • For control group rate, see Dupas et al. 2023, p. 279, Table 1, section: Chlorine Beliefs and Use, row: Positive Chlorine Test.
    • The chlorination vouchers led to an increase from 4.7% chlorination to 31.2% chlorination.

  • 9

    “In the vouchers group, 32.9% of households had water testing positive for residual chlorine. . . . The 95% confidence intervals for the vouchers . . . groups are [25.3, 40.5].” Dupas et al. 2016, p. 891. Note that Dupas et al. 2016 did not have a pure control group, so we cannot determine counterfactual water treatment in absence of the voucher intervention.

  • 10

    See Water quality cost-effectiveness analysis [Nov 2022], "Mortality effect size" tab, "Pooled relative risk" row.

  • 11

    See our preliminary cost-effectiveness analysis, “Vouchers” tab. We estimate a 3.0% reduction in mortality for those over five and a 7.6% reduction in mortality for those under five (3.0/7.6=0.395).

  • 12

    Dupas et al. 2023, Figure 1, p. 282.

  • 13
    • “A total of 2,313 households were successfully surveyed and incorporated into the study.” Dupas et al. 2023, p. 281.
    • “Among those, households with a child under the age of 6 were considered eligible for the study, and a subset of those eligible were sampled for the study.” Dupas et al. 2023, p. 281.

  • 14

    "In a region with a well-established NGO-supported CHW program, in which CHWs work under the government's Health Surveillance Assistants (HSAs) and are assigned 20 to 30 households each to monitor, we randomly assign households to receive either 18 months of coupons for free chlorine solution redeemable at local shops, free monthly chlorine deliveries by CHWs, or to be in a control group. We cross randomize CHWs to incorporate active water, sanitation, and hygiene (WASH) education into their monthly household visits to test whether this increases chlorine take-up or helps sustain chlorine use over 18 months. In a neighboring region with only the HSA program (no NGO-supported CHWs), we randomize households between the coupon program and a control group.” Dupas et al. 2023, p. 273.

  • 15

    “All participating shop owners received a monthly stipend of MKW10,000 (roughly $13) and signed a contract agreeing to (1) accept coupons in exchange for WaterGuard bottles with no additional fee, (2) only accept coupons in the month for which the coupon was specified, (3) record each coupon redemption with a coupon serial number in a record book, (4) retain the coupon until collected by the study team, and (5) pay the cost of any missing bottles not recorded in the record book (approximately $0.50).” Dupas et al. 2023, p. 283.

  • 16

    “Across the 872 households that received coupons an average of 419 coupons were redeemed per month.” Dupas et al. 2023, p. 302.

  • 17

    See Dupas et al. 2023, Figure 3, p. 291.

  • 18
    • For treatment group rate, see Dupas et al. 2023, p. 289, Table 2, column: (1) Positive Chlorine Test, row: Pooled Coupon Effect
    • For control group rate, see Dupas et al. 2023, p. 289, Table 2. Column 1 "Positive Chlorine Test", row: Neno Control Group Mean.

  • 19

    See Dupas et al. 2023, Online Appendix, p. 16, Table B4, column: (1) Diarrhea, row: Pooled Coupon Effect and Neno Control Group Mean.

  • 20

    It is registered in the AEA RCT registry (AEARCTR-0002893).

  • 21
    • We estimate equation (1) using data from the baseline survey to assess balance on our main outcomes prior to the start of the interventions. The results in Table A2 in the online Appendix show some imbalance. Neno households assigned to the coupon arm without WASH promotion were 3.4 percentage points (p = 0.100) less likely to use chlorine at baseline than control households. Neno households assigned to a WASH-trained CHW were 2.9 percentage points more likely to use chlorine at baseline if assigned to coupons than no coupons (sum of Coupon and Coupon × WASH coefficients, p = 0.041). These differences are statistically significant but economically very small and will be dwarfed by our estimates of the coupon treatment effects below, so the small imbalance does not affect our ability to estimate causal impacts, and controlling for baseline levels in the analysis does not change the results (Table B2 in the online Appendix). Households assigned to coupons and the WASH intervention were balanced with control households on chlorine use (adding the coefficients and interaction terms gives −0.034 − 0.024 + 0.063 = 0.005 with p = 0.808). Also note that when pooling all arms that received coupons, the coefficient on coupon assignment at baseline was trivial (pooled coupon effect of 0.4 percentage points, p = 0.717). Study arm assignment is not associated with the probability of a CHW visit in the four weeks prior to the baseline survey. The likelihood of any child illness is 5 percent lower in the pooled coupon group (−3.5 percentage points, p = 0.044) and number of illnesses is 6 percent lower (0.07 illnesses, p = 0.038)." Dupas et al. 2023, p. 277-78.
    • See also Dupas et al. 2023, Online Appendix, Table A2, p. 7.

  • 22
    • "Table 2 shows the impacts on our primary outcomes of interest: child health and whether we could detect chlorine in the household’s drinking water during unannounced follow-up visits." Dupas et al. 2023, p. 288.
    • “Household chlorine use was measured using colorimetric tests. Respondents were asked to provide a cup of water from their drinking water reserve, and enumerators added reagent powder which turned a shade of pink if residual chlorine was present. Enumerators compared this shade against a provided color wheel to determine the concentration of chlorine in the water.” Dupas et al. 2023, p. 285-86.

  • 23
    • “From our baseline sample of 2,313 households, we completed follow-up surveys for 2,105 [91%] households in the first round and 1,731 [74.8%] during the second round.” Dupas et al. 2023, p. 285.
    • “Table A5 in the online Appendix regresses the number of follow-up visits per household on their treatment assignment. This table shows that the probability of attrition was similar across study arms. . . . Table A6 shows baseline characteristics of attriters (the 608 households that had less than two follow-up visits). Differences among attriters between arms were small and mostly insignificant.” Dupas et al. 2023, p. 300-301.
    • See Dupas et al. 2023, Online Appendix, Table A5 and Table A6, pp. 10-11.

  • 24

    Dupas et al. 2023 had an 18-month study period, but the time between baseline and endline was closer to 14 months. See Dupas et al. 2023, Figure 1, p. 282.

  • 25

    “One limitation of our study is that our health outcomes are not measured objectively but reported by
    households, and such self-reports can be subject to bias (Wolf et al., 2018).” Dupas et al. 2023, p. 274.

  • 26

    “This study took place from November 2007 to September 2008 in western Kenya, a region with the second highest prevalence of child diarrhea in Kenya.” Dupas et al. 2016, p. 890.

  • 27

    “Parents of children age 6 to 12 months, an age group at high risk of mortality due to diarrheal disease, were recruited from the waiting rooms of four maternal and child health clinics in Busia District.” Dupas et al. 2016, p. 890.

  • 28

    “Water treatment solution was made available for immediate purchase at a 50% discount off the retail price. Participants could purchase up to five 150-ml bottles of the solution (enough to last approximately 5 to 8 months), at 10 Ksh per bottle.” Dupas et al. 2016, p. 891.

  • 29

    “Twelve vouchers, each redeemable for one 150-ml bottle of water treatment solution at either a local shop or the clinic itself, were provided. Each voucher was marked for a specific month, for the next 12 consecutive months, and participants were given a calendar to track the expiration of vouchers.” Dupas et al. 2016, p. 891.

  • 30

    “Two 500-ml bottles of water treatment solution were provided, one immediately and the second given during the follow-up survey conducted at the participant’s home, 3 to 5 months later. At the time they received the first bottle, participants were informed that they would receive a second bottle later.” Dupas et al. 2016, p. 891.

  • 31

    See Dupas et al. 2016, Table 1, p. 891. Section: Vouchers, row: “Redeemed at least one voucher” and “Proportion of 12 vouchers redeemed.”

  • 32

    See Dupas et al. 2016, Table 2, p. 892. Column: “Vouchers.”

  • 33

    It is registered in the AEA RCT registry (AEARCTR-0001076).

  • 34
    • See Dupas et al. 2016, Supplementary materials, Table S1, p. 6.
    • “Table S1 presents tests of balance across the three experimental groups described in the paper. Characteristics generally appear balanced across treatments, with the exception that those in the VOUCHER and FREE SAMPLE groups are approximately one year older than those in the COST SHARING group. One of the 33 comparisons are significant at the five percent level, no more than one would expect if treatment were random. There are also differences across treatments in the proportion of respondents for whom the clinic visit was prompted by illness versus routine care, and whether the
      respondent walked to the clinic, but neither of these are large in magnitude or significant at the 5% level.” Dupas et al. 2016, Supplementary materials, p. 4.

  • 35

    “By testing households’ stored water for chlorine residual, we assess actual use of the product, and thus compare the extent to which each mechanism generates errors of inclusion (by providing the product to households who will not use it to treat water) or of exclusion (by preventing households who would use the product to treat water from obtaining the product.)" Dupas et al. 2016, p. 890.

  • 36

    “Attrition in the follow-up survey was non trivial, given that respondents recruited through clinics were traced at their homes, but not differential by treatment arms. Including both respondents who were not interviewed (9.8% of the sample), and those who had no water stored at the time of the survey and whose usage could therefore not be verified (a further 3.4%), and controlling for baseline covariates shown in Table S1 and stratification variables, attrition was 12.8% in the COST SHARING group, 11.8% in the
    VOUCHERS group, and 13.4% in the FREE DELIVERY group; the p-values are 0.586 and 0.736 respectively for the test of equality in attrition rates between COST SHARING and each of the other groups. Unadjusted rates (p-values) for the VOUCHERS and FREE DELIVERY groups are 12.0% (0.7504) and 14.0% (0.633). This suggests that selection bias is unlikely to account for the differential outcomes across groups.” Dupas et al. 2016, Supplementary materials, p. 2.

  • 37

    “A follow-up was conducted at participants’ homes 3 to 5 months after enrollment.” Dupas et al. 2016, p. 891.

  • 38

    This makes it impossible to know what water chlorination rates were absent of any intervention.

  • 39

    “We note three limitations of this study. First, we do not report on health outcomes. Self-reported diarrhea in the context of a trial through which recipients were provided free water
    treatment solution may be subject to bias, and collection of observational data on this
    outcome was beyond the available budget.” Dupas et al. 2016, p. 894.

  • 40

    “Note that since the baseline survey was administered in waiting rooms of clinics and not at respondents’ homes, no water test could be performed at baseline, and therefore we cannot test for baseline balance on the primary endpoint of interest (presence of chlorine in drinking water at home).” Dupas et al. 2016, p. 891.

  • 41

    “Shop owners received an initial WaterGuard supply and were restocked each month as needed. (No shop ever ran out.)” Dupas et al. 2023, p. 283.

  • 42

    “Redemption points were monitored to ensure stock-outs would not prevent voucher redemption.” Dupas et al. 2016, Supplementary materials, p. 2.

  • 43

    "A representative survey in 2015 shows that over 20% of children in the southern region of Malawi had a case of diarrhea within the prior two weeks (National Statistical Office 2017).” Dupas et al. 2023, p. 281.

  • 44

    “The study took place . . . in western Kenya, a region with the second highest prevalence of child diarrhea in Kenya.” Dupas et al. 2016, p. 889.

  • 45

    “At baseline in 2017, [current usage rate of chlorine among households] was only 5% in our study context of southern Malawi.” Dupas et al. 2023, p. 276.

  • 46

    See Dupas et al. 2023, Table 1, p. 41.

  • 47

    “The research team partnered with three local shops near the clinics where respondents were recruited, and staff at one of the clinics (which was located too far from any shop), to provide WaterGuard in exchange for vouchers.” Dupas et al. 2016, Supplementary Materials, p. 2.

  • 48

    “Omitting this subsample from the analysis does not affect the pattern or statistical significance of the results.” Dupas et al. 2016, p. 894.

  • 49

    See Water quality cost-effectiveness analysis [Nov 2022], "Mortality effect size" tab, "Pooled relative risk" row.

  • 50

    “The sample of 17 studies with mortality data is summarized in Table 1, based on information from manuscripts, aggregation of microdata, and correspondence with authors. Two of them were excluded from the main analysis due to contamination in the control group.” Kremer et al. 2022 (working paper), p. 6.

  • 51

    “Twelve examined water chlorination, two examined water filtration, and one examined spring protection.” Kremer et al. 2022 (working paper), p. 7.

  • 52
    • The following four studies involved interventions that were limited to chlorination: Boisson et al. 2013, Null et al. 2018, Haushofer et al. 2021 (working paper), Dupas et al. 2020 (working paper).
    • Table 1 of Kremer et al. 2022 lists study characteristics, including intervention type. Chlorination interventions often include additional elements such as flocculation or provision of safe storage vessels. Kremer et al. 2022 (working paper), table 1, pp. 35-43.

  • 53
    • The initial estimate from an earlier version of Kremer et al. 2022 was 28 to 30%: "In the full set of 15 studies we estimated an average reduction in odds of all-cause child mortality of 28% (Peto OR 0.72; CI 95% 0.55, 0.92) or 30% (Bayes OR 0.70; CrI 95% 0.49, 0.92), depending on the model (see Figure 2, Table S3)." Kremer et al. 2022 (working paper), p. 8. These were the figures we had access to at the time we began our own analysis, and the ones we cite in our water quality intervention report.
    • In a more recent version of the meta-analysis, Kremer et al. revised their estimate downward to account for the greater uncertainty of implementation in a new context: "Sensitivity to choice of model: under a fixed effect Bayesian logit model the reduction in odds was 25% (OR 0.75, 95% CrI 0.61, 0.91), compared to 30% under the random effects model. Using the inverse weighting method (with random effects) and excluding studies with no events in one of the arms the reduction was 26% (OR 0.74, 95% CI 0.59 to 0.93), compared to 28% under the Peto OR model." (p. 8) In other words, the new estimate is meant to be a prediction of the results one would expect from conducting a new trial, not simply a statistical description of the findings of previous RCTs.

  • 54

    “An adjusted estimate of the OR obtained using Andrews and Kasy method was OR = 0.83 (CI 95% 0.74, 0.92). Since the power of these tests may be limited when applied to our sample of 15 studies, we also consider a post hoc simulation-based exploration of small-study bias; see Discussion.” Kremer et al. 2022 (working paper), p. 7.

  • 55

    See our preliminary cost-effectiveness analysis, “Internal validity adjustment” tab, "Adjustment for bundled interventions in Kremer et al." row.

  • 56

    For example, Peletz et al. 2012 and Kirby et al. 2019 used water filtration, and Kremer et al. 2011 used spring protection. See Kremer et al. 2022 (working paper), Table 1, pp. 38, 39, 41.

  • 57
    • “To limit these concerns and generate an estimate that is most applicable to the specific interventions we are evaluating, we developed an alternative meta-analysis method in consultation with our external reviewers. We pool the findings of a subset of the trials identified by Kremer et al. 2022 (working paper) that have the following characteristics:
      • The water treatment method is chlorination, without additional treatments like flocculation or filtration. This excludes water quality interventions that were less similar to those we are evaluating.
      • Follow-up length of one year or greater. This tends to exclude small trials that are more susceptible to publication bias.

      We also exclude Haushofer et al. 2021 (working paper), a follow-up study of a RCT of chlorine dispensers, because we believe the effect size it reports is implausibly large, and it has a substantial impact on the pooled estimate.

    • This leaves five trials, Reller et al. 2003, Boisson et al. 2013, Luby et al. 2018, Null et al. 2018, and Humphrey et al. 2019. We weight them using inverse variance, a common meta-analysis method that minimizes variance of the mean, and also by their similarity to the simple chlorination interventions we are evaluating. We then pool individual trial estimates of mortality reduction using these weights.” GiveWell, "Water quality interventions," 2022.

  • 58

    See “Water quality cost-effectiveness analysis [Nov 2022], "Mortality effect size" tab, "Pooled relative risk" row.

  • 59
    • 9% = (1-0.88)*0.75
    • See GiveWell, Vouchers water-quality CEA, 2023

  • 60

    7.6% = (1-0.88)*0.75*0.88 GiveWell, Vouchers water-quality CEA, 2023, "Vouchers" tab, "Percent reduction in under-5 all-cause mortality, initial estimate" row.

  • 61

    For our adherence estimate, see our preliminary cost-effectiveness analysis, “Adherence adjustment” sheet, “Vouchers for water treatment adherence adjustment” section.

  • 62

    See our calculations here. GiveWell, Water quality cost-effectiveness analysis, 2022, "Over-5 mortality scaling factor" tab.

  • 63

    See our preliminary cost-effectiveness analysis, “Vouchers” tab. We estimate a 3.0% reduction in mortality for those over five and a 7.6% reduction in mortality for those under five (3.0/7.6=0.395).

  • 64

    “Such a program, if implemented through the health care system (i.e., vouchers distributed during well-baby checkups), would also provide at least some increased incentive for households to bring children into clinics.” Dupas et al. 2016, p. 893.

  • 65
    • “Water-borne pathogens common in our setting do not have coughing as a symptom, but coughing is a symptom of indoor air pollution, which is high in rural Malawi, where wood burning is the primary source of cooking fuel (91 percent of households in our sample). There are two potential indirect channels through which water treatment can reduce indoor air pollution exposure: by reducing the need to boil water, and by reducing illness spells during which children stay indoors.” Dupas et al. 2023, p. 286.
    • “Specifically, the incidence of coughing—very high in our sample due to indoor air pollution (46.4 percent of children under ten experienced coughing in the previous four weeks in the control group)—reduces by 17 percent.” Dupas et al. 2023, p. 301.

  • 66

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Under-5 deaths averted” and “Over-5 deaths averted” sections.

  • 67

    See our preliminary cost-effectiveness analysis, “Mortality effect size” sheet, “Pooled relative risk” row.

  • 68

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Percent reduction in under-5 all-cause mortality, final estimate” row.

  • 69

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Percent of cost-effectiveness coming from under-5 mortality reduction” row.

  • 70

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Percent of cost-effectiveness coming from over-5 mortality reduction” row.

  • 71

    See our preliminary cost-effectiveness analysis, “Adult mortality scaling factor” sheet, “Low SDI countries” column.

  • 72

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Percent of cost-effectiveness coming from over-5 mortality reduction” row.

  • 73

    See our preliminary cost-effectiveness analysis, "Vouchers" sheet, "Contribution of each outcome to overall cost-effectiveness" section.

  • 74

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Costs” section.

  • 75

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Cost per life saved” section.

  • 76

    See our preliminary cost-effectiveness analysis, "Vouchers" sheet, "NGO vs. cash" row.

  • 77

    See our preliminary cost-effectiveness analysis, “RFMF (inputs) for all countries” sheet.

  • 78

    "All participating shop owners received a monthly stipend of MKW10,000 (roughly $13) and signed a contract agreeing to (1) accept coupons in exchange for WaterGuard bottles with no additional fee, (2) only accept coupons in the month for which the coupon was specified, (3) record each coupon redemption with a coupon serial number in a record book, (4) retain the coupon until collected by the study team, and (5) pay the cost of any missing bottles not recorded in the record book (approximately USD $0.50)." Dupas et al. 2023, p. 283.

  • 79

    See our preliminary cost-effectiveness analysis, “Vouchers” sheet, “Under-5 deaths averted” section.

  • 80

    See our preliminary cost-effectiveness analysis, “Vouchers locations” sheet, “Under-5 all-cause mortality rate (%)” row.

  • 81

    For vouchers, we rely on baseline water treatment data from Dupas et al. 2023 in Malawi, which conducted direct chlorine tests. We rely on this estimate rather than self-reported chlorine usage due to concerns about social-desirability bias, though we have considerable uncertainty given that it’s only one RCT and may not be representative of future implementation contexts. We calculate our adherence adjustment for vouchers by comparing the estimated overall increase in water treatment due to vouchers (which incorporates the above baseline water treatment rate) to the estimated overall increase in water treatment as a result of the interventions in Kremer et al. 2022’s meta-analysis. See our data and calculations on the “Adherence adjustments” sheet in our preliminary cost-effectiveness analysis.

  • 82

    See our preliminary cost-effectiveness analysis, “Adherence adjustment” sheet, “Adherence adjustment for vouchers” row.

  • 83

    See Dupas et al. 2023, Figure 3, p. 291.

  • 84
    • A recent study in Bangladesh found that acceptable chlorine levels were below the recommended dose for household treatment of water with chlorine.
    • “Among residents of an urban low income community in Bangladesh, we found the median chlorine detection threshold (0.71 mg/L for all respondents across both sodium hypochlorite and NaDCC) and median acceptability threshold (1.25 mg/L) were well below the approximately 2.0 mg/L free chlorine target dose for most household treatment products. Our findings suggest that current household water treatment products may be poorly received by the majority of target users in Dhaka, and also by the average consumer in the United States and France. Our results indicate that lower target doses for chlorine treatment products could potentially increase acceptability, usage, and demand for such products.” Crider et al. 2018, pp. 845-46.

  • 85

    Conversation with Amy Pickering, February 8, 2023 (unpublished).

  • 86
    • Conversation with researchers at Development Innovation Lab, November 2, 2023 (unpublished).
    • Lantagne and Clasen 2012 reported that taste and smell were the main reason for disuse of WaterGuard in Nepal, but a lack of product was the main reason that people stopped using Aquatab. Table 3, p. 5.

  • 87

    See GiveWell, "Water quality interventions," 2022.