Evidence Action's Dispensers for Safe Water program - December 2018 Version

Published: November 2017

We spoke with Arjun Pant and Grace Morgan about updates on Dispensers for Safe Water on June 20, 2018.

Summary

What do they do? The Dispensers for Safe Water program (https://www.evidenceaction.org/dispensers/) provides chlorine dispensers for decontamination of drinking water to prevent diarrhea and associated deaths of young children. (More)

Does it work? We believe that there is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death. The program carries out surveys to gather data on the implementation and impact of the program, including whether dispensers are correctly installed, what the quality of the water is at baseline, and whether chlorine is present in water at the household level. We have not thoroughly evaluated Dispensers for Safe Water's monitoring. (More)

What do you get for your dollar? Our rough cost-effectiveness analysis of Dispensers for Safe Water suggests that the program is in a similar range of cost-effectiveness as unconditional cash transfer programs. (More)

Is there room for more funding? We have not thoroughly evaluated Dispensers for Safe Water's room for more funding. Evidence Action told us that it could use up to $4.7 million per year to maintain current operations and $7.2 million to expand the program in 2018-2020. (More)

Our Process

In 2012, we conducted a site visit of Dispensers for Safe Water, then operated by Innovations for Poverty Action (IPA).1 In the years since, we have had ongoing conversations with Dispensers for Safe Water.2 In 2013, we completed an intervention report on water quality interventions. We have also written a blog post on water quality interventions. In 2016-2017, we evaluated Dispensers for Safe Water's cost-effectiveness relative to our top charities (more). The review of the evidence of effectiveness, together with our cost-effectiveness estimates, resulted in the decision to recommend Dispensers for Safe Water as a standout charity.

Evidence Action as an organization

We believe that Evidence Action as a whole is an organization with a strong track record of self-evaluation, communication, and transparency. Evidence Action's Deworm the World Initiative has been a GiveWell top charity since 2013, and we recommended Evidence Action's No Lean Season program as a top charity in 2017. We removed No Lean Season from our list of top charities in 2018; see this blog post for more detail.

What do they do?

Evidence Action was formed in 2013 and at that time managed two programs: the Deworm the World Initiative (formerly Deworm the World) and Dispensers for Safe Water, both previously managed by Innovations for Poverty Action (IPA).3

Dispensers for Safe Water provides chlorine dispensers at wells and other water sources in rural and remote communities of Kenya, Uganda, and Malawi.4 Community members can dispense a pre-measured amount of diluted chlorine into their water container prior to collecting water from the source.5 Chlorine disinfects water by chemically reacting with microorganisms over a period of time, such as during the walk home from the water source (more). Dispensers for Safe Water believes that water treated in this way remains disinfected for 2-3 days.6 We have not evaluated water storage techniques or storage duration in communities served by Dispensers for Safe Water.

Dispensers for Safe Water states that it installs and repairs dispensers, provides refills of chlorine, engages with the community, and conducts monitoring and evaluation of the program.7

Breakdown of Dispensers for Safe Water's spending

We summarize Dispensers for Safe Water's 2016 spending in this spreadsheet.

In short:

• Dispensers for Safe Water spent approximately $4.7 million in 2016.
• Of that spending, approximately 38% was on staffing costs (including field officers who deliver chlorine and maintain dispensers, monitoring and evaluation staff, and others), 25% was on supplies such as chlorine and costs of vehicles to deliver chlorine,8 22% was on other costs in countries of operation such as field office expenses (rent, insurance, IT, etc.) and staff travel for meetings and trainings, and 15% was on US-based finance, fundraising, donor reporting and relations, and communications.9

Dispensers for Safe Water expects its spending in 2017 to be about the same as in 2016.10

Does it work?

We have focused our evaluation on evidence of whether programs like Dispensers for Safe Water reduce diarrhea and death in children under five. There is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death.

Does water chlorination kill microorganisms that cause diarrhea?

Chlorination is a well-established means of disinfecting drinking water. When dilute sodium hypochlorite or calcium hypochlorite is added to water, it reacts with organic matter,11 destroying the membranes of microorganisms.12 The rate and efficacy of inactivation of microorganisms varies, and depends on pH, chlorine concentration, and chlorine demand.13 Additionally, microorganisms may be protected from inactivation by chlorine if they are attached to or within particulate matter in the water; therefore chlorination is less effective in turbid water.14 Some microorganisms are resistant to chlorine, including the cysts and eggs of protozoa and helminths.15 After disinfection, excess chlorine in the water may remain available to prevent re-contamination, depending on storage conditions.16

Do people use chlorine dispensers?

We discuss what we know about how Dispensers for Safe Water monitors dispenser use below. In brief, it tests for both total chlorine residual and free chlorine residual in the water of randomly sampled households during unannounced visits.17 The results of these visits are presented on the Dispensers for Safe Water webpage: Dispenser Data as Total Dispenser Adoption Rates, showing around 50% adoption. Total chlorine residual is the sum of free chlorine, chlorine which remains available for disinfection, and combined chlorine, a product of chlorine's chemical reactions.18 Therefore, measuring total chlorine residual measures whether the water was ever chlorinated (adoption rate) but does not necessarily indicate whether the water is disinfected. Our understanding is that levels of free chlorine residual (chlorine which remains available for disinfection) were, at least in 2015, slightly lower than levels of total chlorine residual.19 However, due to the interim status of our investigation into Dispensers for Safe Water, we have not closely analyzed the methodology or results of monitoring of dispenser use.

Does water chlorination prevent under-5 diarrhea?

Our best guess is that water chlorination interventions prevent diarrhea in children under five years of age. However, a number of factors complicate the interpretation of this body of evidence: self-report of diarrhea, risk of bias from lack of blinding, and variable adherence to treatment.20 There are also plausible mechanisms by which diarrheal rates may not be closely dependent on household water quality.21 We have very recently been made aware of a recent unpublished meta-analysis of the effect of water treatment on all-cause child mortality, including three studies of the effect of water chlorination on all-cause child mortality. We have not incorporated this information into our view of water chlorination interventions, but will do so in our next update on this work.

One recent randomized controlled trial, WASH Benefits, investigated the effect of chlorine dispensers on self-reported diarrhea in Kenya and Bangladesh.22 Results from this trial are forthcoming. This is the only study we know of that assesses the relationship between chlorine dispensers and diarrheal outcomes, or on water chlorination at the water source ("point of collection") rather than in the household ("point of use").

The most recent (2015) Cochrane review of interventions to improve water quality identified 15 randomized controlled trials (RCTs) of the effect of point-of-use water chlorination on diarrhea in children under five.23 We summarize these studies in GiveWell's summary of studies of point-of-use chlorination on under-5 diarrhea. Two of the fifteen studies take place in east Africa, the same region where Dispensers for Safe Water operates. Four of the fifteen studies were published since 2007 (within the last 10 years). The Cochrane meta-analysis of these fifteen studies finds that water chlorination reduces the risk of under-5 diarrhea by 23% (statistically significant) among populations offered the intervention.24 This estimated effect size includes both households that adopted and households that did not adopt the offered treatment25 and therefore likely underestimates the impact in adopting households. The effects found in individual studies are heterogeneous, and hence may reflect sensitivity to local conditions.26 Of 15 studies, the Cochrane review classified five as having low risk of bias due to blinding, and 10 as having high risk of bias due to blinding.27 Considering all studies and subgroups of water chlorination studies (not just outcomes on children under five), the Cochrane meta-analysis finds no effect in the studies at low risk of bias by blinding of participants, but finds a statistically significant effect in the studies at high risk of bias by blinding of participants.28 We have not reviewed whether this pattern holds for studies of subgroups of children under five, the methodology for assigning risk of bias, or the methodology of blinding in individual studies. We note that there were substantial methodological limitations for several of the blinded studies, such as small sample sizes and, in one case, a placebo that was partially effective at improving water quality.29 The Cochrane review also notes that the primary outcome in most studies is self-reported diarrhea.30 Self-reported outcomes may be particularly prone to bias (e.g. participants reporting what they believe researchers want to hear) when participants are not blinded to the intervention. We are uncertain of the likely magnitude of this potential bias.31

As might be expected, adherence to the treatment appears to affect the effect of the treatment on diarrhea. Considering all studies and subgroups of water chlorination studies (not just outcomes on children under five) which report on adherence, the Cochrane meta-analysis finds statistically significant effects of water chlorination on diarrhea in studies with over 50% adherence, but not in studies with under 50% adherence or in studies which do not report adherence.32 This pattern is supportive of the theory that water chlorination is effective at reducing diarrhea rates. We have not reviewed whether this pattern holds for studies of subgroups of children under five, nor have we reviewed the methodology of adherence measurement in individual studies.

We are aware of one RCT of water chlorination published since the most recent Cochrane review. A three-armed study in Bangladesh comparing home water chlorination and safe storage, safe storage alone, and control found a significant reduction in under-5 diarrhea in both treatment arms (36% in the chlorine plus safe storage arm, 31% in the safe storage arm). However, there was no statistical difference between the two treatment arms, suggesting that the benefits were predominantly due to the safe storage intervention rather than the chlorination intervention.33

Dispensers for Safe Water recently conducted its own updated review of the evidence for the effect of water quality interventions, available at Dispensers for Safe Water's 2017 updated review of the evidence, concluding that "we continue to have confidence that Dispensers for Safe Water is backed by strong evidence and is highly cost-effective."

What monitoring does the Dispensers for Safe Water program collect?

For this standout charity review, we asked Dispensers for Safe Water to share high-level information about the types of monitoring it conducts to ensure its program is working as intended. We have not attempted to analyze how compelling this monitoring is at this interim stage of our investigation; rather, we present a descriptive account of Dispensers for Safe Water's monitoring processes and the data they generate.

Dispensers for Safe Water provided an overview of the types of monitoring information it collects.34 We focused on three types of monitoring that seemed most likely to be useful to us in understanding program performance:

• Installation surveys. Dispensers for Safe Water field officers conduct checks after installation for every dispenser to verify that dispensers are installed consistent with its requirements, for example, that the dispenser is located next to the pathway that people use for accessing the source.35 5-10% of these surveys are re-done by monitoring and evaluation field associates as a quality check.36
• Water quality baseline surveys. Dispensers for Safe Water conducts a microbiological assessment of drinking water safety in a random sample of households before its intervention begins.37 The survey occurs after dispenser installation but before chlorine is delivered to the community.38 The list of households to survey is generated by first randomly selecting a sample of water points and then randomly selecting 8-10 households that use those water points.39 Of the surveyed households, surveyors are instructed to collect water samples from:40
  • the first household at each waterpoint that mentioned not treating their water.
  • all households that mentioned treating water with chlorine but which did not test positive for free chlorine residual in an initial test at the household.
  • all households that mentioned using non-chlorine methods to purify their water.
• Household/community surveys. Dispensers for Safe Water conducts household/community surveys with users of a 1.5% random sample of installed dispensers every two months, surveying 8 randomly selected households per sampled water point, while also checking whether the water point's dispenser remains functional and has chlorine available.41 When the program expands to a new region, it collects more intensive data, surveying a random sample of 2% of all households within the catchment areas of dispensers during the first three months following installation.42 The survey asks, among other things:
  • How many children under 5 live in the household and whether or not those children have had diarrhea in the past 2 weeks.43
  • For a sample of drinking water and for information about the source, storage, and treatment of the household's drinking water.44 The surveyor uses a portable tool to test the water sample for total chlorine and free chlorine.45

  When aggregating the results of these surveys, Dispensers for Safe Water assumes that households that test positive for chlorine residuals in their drinking water treated their water with chlorine from the Dispensers for Safe Water dispenser.46 Dispensers for Safe Water notes that it only tests water in households where (a) the dispenser promoter, an individual tasked with championing the dispenser to the community, claims that the household has been using the chlorine dispenser and (b) when asked by the surveyor, the household member indicates that they use the chlorine dispenser.47 As with installation surveys, 5-10% of household/community surveys are re-done by monitoring and evaluation field associates to verify results.48

Dispensers for Safe Water collects other monitoring data, including data about attendance at community education meetings, chlorine deliveries, hardware spot checks, and surveys with promoters.49 More information about these data is available in Dispensers for Safe Water, Monitoring and Evaluation Primer. We also note that Dispensers for Safe Water publishes a dashboard with real-time data on number of people with access to dispensers, adoption rate, and dispenser outages.

What do you get for your dollar?

Based on GiveWell's CEA of Dispensers for Safe Water, 2017, our best estimate is that Dispensers for Safe water is in a similar range of cost-effectiveness as unconditional cash transfer programs. As with all of our cost-effectiveness models, there are several parameters about which we are significantly uncertain. We are especially uncertain about the expected impact of the program on under-5 diarrhea: although we use a summary effect based on existing research, we are concerned about the heterogeneity of results in existing studies. Our cost-effectiveness analysis also makes a significant judgment about how to treat carbon credit revenues earned by the program: we treat part of the revenue generated by carbon credit sales (the market value on the compliance market) as value generated by the program, offsetting costs of the program. We treat the remainder of the revenue generated by carbon credit sales (in excess of compliance market value) as an additional source of voluntary funding for the program, which does not factor into the cost-effectiveness of the program.

What is our best guess of Dispensers for Safe Water's health impact?

Due to a number of complicating factors in the evidence base for the effect of home water chlorination on under-5 diarrhea, and the lack of evidence about the effect of chlorine dispensers specifically, we are very uncertain about our best-guess estimate of the effect of Dispensers for Safe Water on under-5 diarrhea.

The studies of the effect of point-of-use chlorination on under-5 diarrhea identified in Clasen et al. 2015 measure the intervention's effect on the population targeted. Because we wish to include the proportion of the population's water that is in fact chlorinated ("adherence", or "adoption", or "uptake") as an input in our model of Dispensers for Safe Water, we attempt to estimate the effect of water chlorination on those who use water chlorination (a "treatment on the treated" effect). To do this, we adjust the effect sizes of the studies included in the Cochrane review by reported rate of chlorine use and weight them by study size to get an average effect. Then, we use the adoption rate measured by Dispensers for Safe Water to arrive at an estimate of the program's effect on the Dispensers for Safe Water population.50

Of 15 studies of water chlorination on under-5 diarrhea, the Cochrane review identified a reported adherence rate in 10.51 We excluded two of these studies because we were unable to identify the level of adherence in these studies.52 Additionally, we believe it may be appropriate to exclude Jain et al. 2010 from this analysis due to particularly low diarrheal rates reported in both the treatment and control group, and due to some methodological concerns; this does not have a large effect on our conclusions.53 Additionally, we are aware of two forthcoming studies relevant to the effect of chlorination on under-5 diarrhea: a working paper by Null et al., and the WASH Benefits54 study. WASH Benefits is a large study, and one of its trials takes place in Kenya, a context highly relevant to the Dispensers for Safe Water program.55 We believe these studies are highly relevant to this investigation, and we have some limited information from these studies which contributes to our view of the effect of water chlorination on under-5 diarrhea. However, we are not at liberty to discuss the findings from these studies.

The summary effect of water chlorination on under-5 diarrhea in a population which adopts the intervention is sensitive to which studies we include or exclude in our analysis. Our best guess is based on results from the studies identified in the Cochrane review, and adjusts for our expectation of results from the Null et al. working paper. Due to our limited information about results from WASH Benefits, and the sensitivity of discussing such results prior to publication, we are not formally incorporating expectations about results from that study into this analysis.

Details of our calculations and results are in GiveWell's CEA of Dispensers for Safe Water, 2017.56 Note that we have not closely reviewed the methodologies of the studies included in the Cochrane review, and we are particularly concerned about the heterogeneity of the evidence and differences between study contexts and the Dispensers for Safe Water context.

How do we treat Dispensers for Safe Water's carbon credit revenue?

Dispensers for Safe Water is partially funded by revenue the program earns by being awarded, and selling, carbon credits.57 Dispensers for Safe Water is accredited to generate such carbon credits because the promotion of water chlorination is expected to avert greenhouse gas emissions compared to a counterfactual program that would promote water purification via boiling.58 Our understanding is that water boiling is not highly prevalent in areas where Dispensers for Safe Water works, and so the carbon credit accreditation is based on displacing possible counterfactual water boiling promotion rather than on converting current water treatment from boiling to chlorination. We have not reviewed the details of Dispensers for Safe Water's carbon credit accreditation.

Our understanding of carbon credit markets is based on a small number of conversations with Dispensers for Safe Water. Carbon credits are bought on carbon markets by companies with greenhouse gas reduction targets in order to offset their emissions. Carbon credits were sold on a compliance market for roughly $0.34 in the year that Dispensers for Safe Water negotiated the sale price of their carbon credits.59 There is also a voluntary market, where buyers may seek to voluntarily offset their carbon emissions at a higher price, and often also seek to buy carbon credits from organizations that generate additional environmental or humanitarian benefits.60 Our understanding is that Dispensers for Safe Water sells carbon credits at a price higher than market value because (a) it sells on the voluntary market rather than the compliance market, and (b) it receives a higher than usual price on the voluntary market, possibly due to the perceived additional environmental, humanitarian, and development value of the Dispensers for Safe Water program.

GiveWell generally seeks to include funding provided by other funders in our cost-effectiveness analyses.61 We consider Dispensers for Safe Water's accreditation to sell carbon credits to represent a commodity produced as a benefit of the program. We believe it is appropriate to treat revenue generated by the market sale of a commodity produced by a program as offsetting the costs of that program. However, to the extent that buyers are paying above-market price for carbon credits, we consider the difference between market price and sale price to be in effect a philanthropic contribution of funding to the Dispensers for Safe Water program. We believe it is important to count the contributions of other funders such as buyers of Dispensers for Safe Water's carbon credits both because GiveWell seeks the most cost-effective giving opportunities when all costs are considered, and because we value the counterfactual use of those funds which are directed towards programs with humanitarian and development effects.

GiveWell's CEA of Dispensers for Safe Water, 2017 includes the options to subtract all, none, or a specific amount of the carbon credit revenue from the program's costs. Fully subtracting carbon credit revenue from costs rather than partially subtracting this revenue (keeping all else fixed) moves the median estimate of the cost-effectiveness of the Dispensers for Safe Water program from 1.2 times as cost-effective as GiveDirectly to 3.2 times as cost-effective as GiveDirectly.62 Dispensers for Safe Water notes that it updates its carbon revenue projections to incorporate changes in the carbon market and carbon verification methods as new information becomes available, and it expects to share revisions to its carbon projections in the future.63

The way we account for carbon credit revenue has a significant effect on the cost-effectiveness analysis. However, we feel confident that our decision in this case is consistent with our general approach to alternative funding sources.

Is there room for more funding?

We asked Dispensers for Safe Water to give a brief, high-level overview of its priorities for additional funding. It told us that its highest priority for additional funding is to maintain existing operations across its 28,000 dispensers in Kenya, Uganda, and Malawi, at a total cost of $4.65 million per year. Given other sources of revenue, Dispensers for Safe Water expects to have a funding gap for this work of $7.2 million in 2018-2020.64

Sources

• 1

  GiveWell's notes from a site visit of Dispensers for Safe Water, 2012

• 2

  We have not produced notes from all conversations. Our page of research conversation notes includes GiveWell's non-verbatim summary of a conversation with Dispensers for Safe Water, July 24, 2014.

• 3

  "Two IPA initiatives that touch millions of people in Africa and Asia - Dispensers for Safe Water and the Deworm the World Initiative - will spin off from IPA to be managed by Evidence Action." IPA press release: Evidence Action launch, published in August 2013.

• 4
  • Dispensers for Safe Water webpage, section "Where Dispensers for Safe Water Operates".
  • Dispensers for Safe Water webpage: Dispenser Data shows number of people served in each country over time, and a map of locations where Dispensers for Safe Water operates.

• 5

  "Community members go to their water source to fetch water, place their bucket or jerrican under the dispenser, turn the valve to dispense a correctly measured amount of diluted chlorine, and then fill the bucket with water." Dispensers for Safe Water webpage, section "What is Dispensers for Safe Water?"

• 6

  "The chlorine disinfects the water as a community member is walking home, and by the time he or she arrives, much of the chlorine smell has dissipated and they are left with clean, safe water that stays safe for 2-3 days." Dispensers for Safe Water webpage, section "What is Dispensers for Safe Water?"

• 7
  Dispensers for Safe Water webpage, section "Five Reasons Why Dispensers Work:"
  • "Additionally, in every community with a dispenser, there is an elected, local community promoter who is responsible for the dispenser and educates villages on how to use it."
  • "Dispensers have to be functional, so we have circuit riders who maintain and repair dispensers in even the most remote locations. They have to be full, so our circuit riders deliver chlorine to our local promoters in regular intervals (usually every three months)."
  "Performance monitoring of Dispensers for Safe Water is done on a monthly basis by MLIS data monitors across Kenya, Uganda and Malawi." Dispensers for Safe Water webpage, section "Results at a Glance"

• 8

  This category is named "Program management and material." The notes for this category describe various supplies used for the program: "Includes chlorine procurement, dispenser parts, fuel and vehicle costs for delivery, community and promoter engagement materials, etc."

• 9

  Further detail provided by Dispensers for Safe Water in response to a draft of this page in November 2017.

• 10

  "The difference between the 2016 total budget amount you will see in the attached ($4.73M) vs. the $4.65M we project for 2017 and beyond (referenced in the paragraph above) is due to cost saving measures we undertook at the end of 2016." Arjun Pant, Evidence Action Chief of Staff, email to GiveWell, November 7, 2017.

• 11
  • "When chlorine is added to water, it is involved in three types of reaction. These affect the availability of chlorine and its efficiency as a disinfectant.

    First, substances such as manganese, iron, and hydrogen sulphide dissolved in the water will react irreversibly with chlorine. This reaction removes these substances, thereby improving water quality and taste. Chlorine, which reacts in this way is, however, lost and does not contribute to disinfection.

    Secondly, chlorine may react reversibly with organic matter and ammonia in water. The compounds formed are weak disinfectants. The products are referred to as combined chlorine or residual combined chlorine.

    Thirdly, the chlorine may react with and dissociate in water. The products are efficient disinfectants unless the water is alkaline and are referred to as free chlorine or free residual chlorine.

    The total amount of chlorine which will react both with compounds like iron and manganese and with organics and ammonia is referred to as the chlorine demand. The chlorine demand of different waters can vary widely.

    Chlorine demand is therefore the difference between the amount of chlorine added to the water (the chlorine dose) and the free chlorine detectable in the water.

    The chlorine demand for some waters, for instance some river waters, can increase dramatically, particularly after heavy rain. Measurement of chlorine demand is important for control of water treatment processes and is detailed in Fact Sheet 2.31."

    WHO fact sheet: Inactivation of microbes by chlorine, pp. 117-118.

  • "What is chlorination?
    "Chlorination is the process of adding chlorine to drinking water to disinfect it and kill germs. Different processes can be used to achieve safe levels of chlorine in drinking water. Chlorine is available as compressed elemental gas, sodium hypochlorite solution (NaOCl) or solid calcium hypochlorite (Ca(OCl)2."
    CDC, Disinfection with Chlorine, Frequently Asked Questions

• 12

  "When chlorine is added to water, it destroys the membrane of microorganisms and kills them." WHO Measuring chlorine levels in water supplies, p. 1.

• 13
  • "The rate of inactivation varies widely, but is more rapid when more chlorine is present in the water... The efficiency of inactivation of microbes by chlorine is affected by a number of factors including pH, contact time and the reactions of chlorine with the water." WHO fact sheet: Inactivation of microbes by chlorine, p. 117.
  • On chlorine demand of water: "First, substances such as manganese, iron and hydrogen sulphide dissolved in the water will react irreversibly with chlorine. This reaction removes these substances, thereby improving water quality and taste. Chlorine, which reacts in this way is, however, lost and does not contribute to disinfection... The total amount of chlorine which will react both with compounds like iron and manganese and with organics and ammonia is referred to as the chlorine demand. The chlorine demand of different waters can vary widely." WHO fact sheet: Inactivation of microbes by chlorine, pp. 117-118.
  • "The time taken for different types of microbes to be killed varies widely. In general, amoebic cysts are very resistant and require most exposure. Bacteria, including free-living Vibrio cholerae are rapidly inactivated by free chlorine under normal conditions. For example, a chlorine residual of 1mg/l after 30 minutes will kill schistosomiasis cercariae, while 2mg/l after 30 minutes may be required to kill amoebic cysts. This it is important to ensure that adequate contact time is available before water enters a distribution system or is collected for use." WHO fact sheet: Inactivation of microbes by chlorine, p. 119.

• 14

  "Nevertheless, microbes may be protected from chlorine if they are attached to or within particles in the water. For this reason, water to be chlorinated must be clear. It should always have a turbidity of less than five turbidity units and ideally less than one turbidity unit." WHO fact sheet: Inactivation of microbes by chlorine, p. 117.

• 15

  "For practical purposes, cysts and eggs of protozoa and helminths my be considered resistant to disinfection with chlorine. They are killed at high doses or after prolonged contact times, but these are often impractical. Cysts and eggs of protozoa and helminths should be removed by filtration prior to disinfection or, in the case of groundwaters (springs and wells), excluded by source protection." WHO fact sheet: Inactivation of microbes by chlorine, p. 117.

• 16

  "Chlorine persists in water as residual chlorine after dosing and this helps to minimize the effects of re-contamination by killing or inactivating microbes which may enter the water supply after chlorination. It is important to take this into account when estimating requirements for chlorination in order to ensure that residual chlorine is always present.

  The level of chlorine residual required varies with the type of water supply and local conditions. In water supplies which are chlorinated there should always be a minimum of 0.5 mg/l residual chlorine after 30 minutes contact time in water.

  ... Chlorine residual is readily and rapidly lost, particularly in open or regularly opened storage containers. Good household storage and handling practice are therefore vital to ensure good quality water in the home, and reliance should not be placed on residual disinfectant effect." WHO fact sheet: Inactivation of microbes by chlorine, pp. 120, 122.

• 17
  • "The adoption rate is an important measure of the use of dispensers by the community. It is the percentage of randomly sampled households that tested positive for residual chlorine ('Total Chlorine Residual') in their drinking water during an unannounced household visit. 1.5% of all dispensers that we have installed are evaluated that way every month. For the first three months of evaluation in a new geographic area, we increase that to 2% of dispensers monitored." Dispensers for Safe Water webpage: Dispenser Data
  • "SECTION 8: TEST FOR CHLORINE. Please explain to the respondent that you are testing to see if she has chlorine in her water. Please test the water for total chlorine using the total chlorine 5mL sachet." Dispensers for Safe Water, Community Survey
  • "Community_Survey", "c803_tcr_reading What was the total chlorine reading on the color wheel?" Dispensers for Safe Water, Community Survey
  • "Community_Survey", "c806_fcr_reading What was the free chlorine reading on the color wheel?" Dispensers for Safe Water, Community Survey

• 18

  "Three types of chlorine residual can be measured:

  1. Free chlorine - which kills microorganisms most effectively.
  2. Combined chlorine - formed when free chlorine reacts with other chemicals in the water.
  3. Total chlorine - the sum of free and combined chlorine."

  WHO fact sheet: Chlorine testing, p. 181.

• 19

  "Additionally, the program did indeed collect both free and total chlorine residuals for 2015. Average TCR was 55 percent program-wide, and average FCR was 47 percent." Dispensers for Safe Water, email to GiveWell, May 15, 2017.

• 20

  For more information, see our intervention report on water quality interventions.

• 21
  • Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, and first author of Clasen et al. 2015, the most recent Cochrane review of water quality interventions, wrote: "Based on epidemiological modeling, even occasional consumption of untreated water can vitiate the impact of HWTS [household water treatment and safe storage]. This may be an almost insurmountable challenge of household-based solutions when children and adults are regularly consuming water in places outside the home” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, p. 1.
  • For more discussion, see our water quality intervention report.

• 22
  • "The Kenya study installs chlorine dispensers within the cluster boundary at public water sources used by study participants. All community members will be able to use the dispensers. After filling their water collection container (typically a 20 L plastic jerry can) at the source, users can place the container under the dispenser and turn a knob to release 3mL of 1.25% sodium hypochlorite, an amount designed to yield 2 mL/L of free chlorine residual after 30 min for 20 L of water. The Kenya study also includes community level promotion of dispenser use and all households in the study compound receive bottles of sodium hypochlorite (6 months’ supply) to facilitate householders’ water treatment during periods when they rely on rainwater harvesting (common during the rainy season) or if they use a water source in which a dispenser has not been installed. In both countries, the behaviour change strategies target the consistent provision of treated water to all children living in the household." Arnold et al. 2013, p. 7.
  • See also the WASH Benefits website.

• 23

  Clasen et al. 2015, p. 116, Analysis 3.2. Comparison 3 POU: water chlorination versus control, Outcome 2 Diarrhoea: cluster-RCTs: subgrouped by age. Subgroup 2 ("<5 years") contains 15 RCTs.

• 24

  Clasen et al. 2015, p. 116, Analysis 3.2. Comparison 3 POU: water chlorination versus control, Outcome 2 Diarrhoea: cluster-RCTs: subgrouped by age. The meta-analysis result for subgroup 2 ("<5 years") is presented at the bottom of p. 116 as 0.77 with a 95% confidence interval of 0.64 - 0.92.

• 25

  Our understanding is that this analysis was done on an intent to treat basis, comparing households which were randomly assigned to a water chlorination intervention to households which were randomly assigned to be part of studies' control groups. "Analysis 3.2 Comparison 3 POU: water chlorination versus control, Outcome 2 Diarrhoea: cluster-RCTs: subgrouped by age" compares "intervention" to "control" and does not note any adjustments based on adherence (Clasen et al. 2015, p. 116). As the review authors note, it can be challenging to determine the degree to which households adopt or adhere to water chlorination in these studies:

  "Some studies of POU water treatment undertook indirect assessments of adherence by measuring residual chlorine levels in stored water, comparing microbiological water quality of intervention and control groups, conducting periodic or post-study surveys, or counting the amount of intervention product used. Most other studies measured adherence only by occasional observation, while eight cluster-RCTs did not report on adherence.

  "The studies of chlorine residuals reported adherence ranging from a high of 95% (Doocy 2006 LBR) to a low of 11% (Opryszko 2010a AFG). Even among these studies, however, investigators acknowledged that it was not possible to know to what extent intervention group participants actually consumed treated water or avoided consuming untreated water. For those studies that reported on adherence, three took the additional step of investigating and reporting on continued consumption of untreated water (Boisson 2010 DRC; Peletz 2012 ZMB; Boisson 2013 IND) a source of exposure that could be masked by less direct metrics of adherence." Clasen et al. 2015, p. 12.

• 26
  • Clasen et al. 2015, p. 116, Analysis 3.2. Comparison 3 POU: water chlorination versus control, Outcome 2 Diarrhoea: cluster-RCTs: subgrouped by age. See subgroup 2. Of 15 studies, 4 found an effect statistically significant at 95% confidence. Two studies found effects very close to significant at 95% confidence: Crump 2005a 0.83 [0.66, 1.04] and Mahfouz 1995 0.55 [0.30, 1.00].
  • The Cochrane review highlights heterogeneity in its discussion of results: "On average, POU chlorination in cluster RCTs reduced the risk of diarrhoea episodes by around a quarter, both for all ages (RR 0.77, 95% CI 0.65 to 0.91; 14 trials, 30,746 participants; Analysis 3.2) and for children under five years of age (RR 0.77, 95% CI 0.64 to 0.92; Analysis 3.2). However, there was substantial heterogeneity in the size of the effect which was not well explained by a series of subgroup analyses (Analysis 3.2 to Analysis 3.9)." Clasen et al. 2015, p. 15.
  • The Cochrane review states elsewhere, "In the end, however, the effectiveness of HWTS [household water treatment and safe storage] to prevent diarrhea is likely to depend largely on the prevailing conditions—a conclusion that is consistent with the heterogeneity observed in pooled estimates. A single, generalizable estimate of effect is unlikely to emerge, given important differences in pathogens circulating in the population, transmission dynamics, seasons, compliance, and other factors." Clasen 2015, pp. 70-71.

• 27

  For each study of under-5 diarrhea listed in Clasen et al. 2015, p. 116, Analysis 3.2. Comparison 3 POU: water chlorination versus control, Outcome 2 Diarrhoea: cluster-RCTs: subgrouped by age, subgroup 2 ("<5 years"), we recorded the study's risk of bias by blinding of participants according to p. 119, Analysis 3.4. Comparison 3 POU: water chlorination versus control, Outcome 4 Diarrhoea: cluster-RCTs by risk of bias by blinding of participants. We recorded the risk of bias in GiveWell's summary of studies of point-of-use chlorination on under-5 diarrhea.

• 28

  Clasen et al. 2015, p. 119, Analysis 3.4. Comparison 3 POU: water chlorination versus control, Outcome 4 Diarrhoea: cluster-RCTs by risk of bias by blinding of participants. Risk ratio for diarrhea in 5 studies judged at low risk of bias by blinding of participants is 1.07 [0.97, 1.17]. Risk ratio for diarrhea in 11 studies judged at high risk of bias by blinding of participants is 0.68 [0.56, 0.83].

• 29

  For more information, see the methodological limitations noted in the relevant section of our water quality intervention report.

• 30

  "The primary outcome in most studies was self-reported diarrhoea, which is at high risk of bias due to the lack of blinding in over 80% of the included studies." Clasen et al. 2015, p. 2, referring to all studies included in the review.

• 31
  • An adjustment for non-blinding in Clasen et al. 2015 changes the estimated risk ratio for chlorination from 0.72 to 0.80. See Table 23, Clasen et al. 2015, p. 195. We have not vetted this method of adjusting for non-blinding and note that the authors urge caution in relying on this adjusted estimate.
    • "We conducted a sensitivity analysis to investigate the robustness of the results to each of the ’Risk of bias’ components by including only studies that were at low risk of bias. We used this information to guide our judgments on the quality of the evidence. In addition, we explored the impact of non-blinding of POU interventions using a Bayesian meta-analysis with bias correction. For this purpose, we assumed the true log relative risks from non-blinding studies are subject to a multiplicative bias that results in the observed relative risks being inflated in magnitude. We assumed the bias is normally distributed with a mean 1.48 or 1.65 and a corresponding standard deviation (SD) of 0.17 or 0.13. These values were derived from the additive bias correction employed in Wood 2008 and Savovi 2012. While we believe an attempt to adjust for non-blinding is appropriate, we urge caution in relying on these adjusted estimates since the basis for the adjustment is from clinical (mainly drug) studies that may not be transferable to field studies of environmental interventions and because methodology for the adjustment has not been validated." Clasen et al. 2015, p. 9.

• 32

  Clasen et al. 2015, pp. 117-118, Analysis 3.3. Comparison 3 POU: water chlorination versus control, Outcome 3 Diarrhoea: cluster-RCTs; subgrouped by adherence. Four subgroups are considered:

  • Residual chlorine in 86 to 100% of samples: risk ratio [95% CI]: 0.78 [0.73, 0.83]
  • Residual chlorine in 51 to 85% of samples: risk ratio [95% CI]: 0.60 [0.40, 0.91]
  • Residual chlorine in ≤ 50% of samples: risk ratio [95% CI]: 0.90 [0.76, 1.06]
  • Residual chlorine not reported: risk ratio [95% CI]: 0.85 [0.65, 1.12]

• 33
  Ecrumen et al. 2015
  • "We conducted a randomized controlled trial to assess whether improving the microbiological quality of tubewell drinking water by household water treatment and safe storage would reduce diarrhea in children <2 years in rural Bangladesh." p. 1.
  • "Both interventions had high uptake. Safe storage, alone or combined with chlorination, reduced heavy contamination of stored water. Compared to controls, diarrhea in index children was reduced by 36% in the chlorine plus safe storage arm (prevalence ratio, PR = 0.64, 0.55-0.73) and 31% in the safe storage arm (PR = 0.69, 0.60-0.80), with no difference between the two intervention arms. One limitation of the study was the non-blinded design with self-reported outcomes. However, the prevalence of health outcomes not expected to be impacted by water interventions did not differ between study arms, suggesting minimal reporting bias." p. 1.
  • "Safe storage significantly improved drinking water quality at the point of use and reduced child diarrhea in rural Bangladesh. There was no added benefit from combining safe storage with chlorination." p. 2.

• 34

  Dispensers for Safe Water, Monitoring and Evaluation Primer.

• 35
  • "Through the installation form, the program checks and confirms the quality of installation (eg, the access and location of the dispenser in relation to the path to the waterpoint and the height of the dispenser), dispenser and waterpoint geocodes for future management of the dispenser. This data forms the dispenser database that we actively update and maintain to reflect the functional status of the dispenser." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 2.
  • "SECTION 1: SURVEY INFORMATION Details required in this section should be provided to the field officer before leaving the office" and "Installation Checklist, IC.3, "Is the dispenser located next to the pathway that people use for accessing the source?" Dispensers for Safe Water, Installation form, Pg. 1.

• 36

  "Additionally we strive to conduct back-check surveys on 5-10% of key surveys such as installation, spot check, and community surveys. These back-checks are done by monitoring and evaluation field associates and provide a clear check on the performance and results recorded in the initial data collection activity." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 3.

• 37

  "Between installation of the dispenser and the community sensitization meeting (that is, before the delivery of the first batch of chlorine to the community), a random sample of waterpoints are chosen and a sub-sample of households selected, for a microbiological assessment of the safety of their drinking water to serve as a baseline." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 2.

• 38

  "Between installation of the dispenser and the community sensitization meeting (that is, before the delivery of the first batch of chlorine to the community), a random sample of waterpoints are chosen and a sub-sample of households selected, for a microbiological assessment of the safety of their drinking water to serve as a baseline." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 2.

• 39
  • "[A] random sample of waterpoints are chosen and a sub-sample of households selected, for a microbiological assessment of the safety of their drinking water to serve as a baseline." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 2.
  • An example Dispensers for Safe Water shared with us:
    • "The water point list with all 1,073 waterpoints was imported to Stata from the excel and random number assigned to the them using the runiform() command. The list was then sorted using the generated random numbers in ascending order and assigned numbers 1 to 1073. From the list of first 30 water points that that had been randomly selected, two (7020425_Dambyo borehole & 7020425_Bulumbi) were replaced by picking the next two in line(7020018_ Iganga and 7020261_ Lyada) i.e water point number 31 and 32 to keep the number at 30." Dispensers for Safe Water, Water Quality Baseline Sampling Methodology, Pg. 1.
    • "The list of all compounds in a waterpoint was then sorted by the assigned random number and at most the first 30 household [sic] selected to allow for replacement." Dispensers for Safe Water, Water Quality Baseline Sampling Methodology, Pg. 1.
    • "Continue [visiting listed households] in this fashion until: All households have been visited, Or the day is over, Or you have collected the requested number of household visits (usually 8-10)." Dispensers for Safe Water, Water Quality Baseline Sampling Methodology, Pg. 2.

• 40

  "Water samples are collected from drinking water at the different households that meet any of the following criteria:

  • The first household at each waterpoint that mentioned not treating their water.
  • Any household that mentioned treating their drinking water with a chlorine-based product e.g. WaterGuard, Pur, AquaGuard etc. but turned negative when tested for Free Chlorine Residual.
  • Any household that mentioned either filtering, boiling, Solar Disinfection (SODIS) etc. i.e. non-chlorine based methods."

  Dispensers for Safe Water, Water Quality Baseline Sampling Methodology, Pg. 3.

• 41
  • "Sampling 1.5% of all dispensers across 2 months, MLIS field officers visit 8 households per water point per day to determine the presence of chlorine residual in a random sample of household drinking water. … Also done at this time is a “spot check” of dispenser functionality and chlorine availability in the dispenser tank." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 2.
  • "[t]he field officers actually sample 1.5% of all dispensers every 2 months now. They do not return for multiple days to the same water point; your characterization below is correct that they visit a target of 8 randomly selected households that use a given dispenser, targeting 1 water point (i.e. 8 households) per day in addition to the promoter survey that should be conducted at the beginning of the day before visiting community members for the community survey – these households are randomly selected through the “in-field randomization” methodology described in the sampling methodology document for water quality baseline." Arjun Pant, Evidence Action Chief of Staff, email to GiveWell, November 7, 2017.

• 42

  "Household/Community Surveys: For the first three months of evaluation in a new region, households in 2% of all dispenser areas are monitored. Following that, 1.5% of installed dispensers are sampled and monitored over every 2 month period. For community surveys, we employ 2-stage cluster sampling. 1.5% of the dispensers are selected at field office cluster level. At each randomized waterpoint, 8 households are randomly selected for the surveys and chlorine residual testing." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 3.

• 43

  "Community_Survey", "c305b_hhold_child How many children under 5 live in this HOUSEHOLD?" and "c311a_chld1_drhea_past2wks Has this child had diarrhea in the past 2 weeks (past 14 days) including and up to today?" Dispensers for Safe Water, Community Survey

• 44

  See, for example, the following questions:

  • "c401b_giv_glass_watr_normal ASK: "Could you give me a glass of drinking water?"
  • "c401d_drink_confirm Confirm: Is this the water used for drinking by household members?"
  • "c402_watr_drawn OBSERVE or ASK: Where does the respondent take the water from?"
  • "c403_cont_type OBSERVE or ASK: What type of water storage container is used to store the drinking water?"
  • "c410_watr_treatd Did you do anything to the water in this glass to make it safer to drink?"

  Dispensers for Safe Water, Community Survey

• 45
  • "SURVEY DETAILS Details for section 0 and 1 should be provided to the monitor before heading to the field. SECTION 0: SURVEY INFORMATION … c008_cwheel_id Color Wheel ID." Dispensers for Safe Water, Community Survey
  • "SECTION 8: TEST FOR CHLORINE. Please explain to the respondent that you are testing to see if she has chlorine in her water. Please test the water for total chlorine using the total chlorine 5mL sachet." Dispensers for Safe Water, Community Survey
  • "Community_Survey", "c803_tcr_reading What was the total chlorine reading on the color wheel?" Dispensers for Safe Water, Community Survey
  • "Community_Survey", "c806_fcr_reading What was the free chlorine reading on the color wheel?" Dispensers for Safe Water, Community Survey

• 46

  "Water samples from each of these households are then tested for the presence chlorine residuals using the color wheel test kit. Those households testing positive are assumed to have treated their water with chlorine from the Dispenser." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 3.

• 47

  Comment provided by Dispensers for Safe Water in response to a draft of this page in November 2017.

• 48

  "Additionally we strive to conduct back-check surveys on 5-10% of key surveys such as installation, spot check, and community surveys. These back-checks are done by monitoring and evaluation field associates and provide a clear check on the performance and results recorded in the initial data collection activity." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pg. 3.

• 49
  • "Process M&E: inputs required to achieve program goals … Village Community Sensitization, Community Education Meetings, Chlorine Delivery & Hardware 'Spot Check'", Dispensers for Safe Water, Monitoring and Evaluation Primer, Pgs. 1-2.
  • "Performance Monitoring: the target population reached by the program. Promoter Survey." Dispensers for Safe Water, Monitoring and Evaluation Primer, Pgs. 2-3.

• 50

  For details, see GiveWell's CEA of Dispensers for Safe Water, 2017, tab "Diarrhea reduction by adherence".

• 51

  Compare the studies reporting on under-5 diarrhea listed in Analysis 3.2. Comparison 3 POU: water chlorination versus control, Outcome 2 Diarrhoea: cluster-RCTs: subgrouped by age, Clasen et al. 2015, pp. 115-116 with the studies grouped by adherence in Analysis 3.3. Comparison 3 POU: water chlorination versus control, Outcome 3 Diarrhoea: cluster-RCTs; subgrouped by adherence. Clasen et al. 2015, pp. 117-118. Of the 15 studies reporting on under-5 diarrhea, 5 of them appear in the adherence subgroup "Residual chlorine not reported", while the other 10 appear in some other adherence subgroup.

• 52

  Clasen et al. 2015, p. 117 subgroup analysis by adherence categorizes "Austin 1993a" as finding residual chlorine in ≤ 50% of samples and "Austin 1993b" as finding residual chlorine in 51 to 85% of samples. We were not able to identify more precise adherence levels in these study, as the data is unpublished (c.f. the references in Clasen et al. 2015, p. 21, are to "Austin 1993a GMB {unpublished data only}" and "Austin 1993b GMB {unpublished data only}").

• 53
  Jain et al. 2010
  • Self-reported rates of diarrhea were particularly low in this study context. The study designers expected weekly under-5 diarrheal prevalence around 15%: "To calculate household sample size, we assumed a 15% weekly diarrhea prevalence among children under 5-years-old based on Ghana’s 2003 Demographic and Health Survey (DHS)..." (p. 17). However, reported diarrhea rates for children under 5 in the study were much lower than expected in both the treatment group and the control group, with no significant difference detected between the groups: "Among children under 5-years-old, there was no significant difference in diarrhea rates between the two groups with 185 episodes (5.3%) in the intervention and 156 episodes (4.7%) in the control group (Table 3). Diarrhea rates decreased in both groups over time (Figure 2)." p. 18.
  • Chlorination effects: Detected chlorine was higher in the treatment group than the control group. "Over the study period, the percentage of households with free chlorine residuals ≥ 0.2 mg/L in stored drinking water ranged from 74–89% in the intervention group and 0–7% in the control group (Figure 3)." p. 18.
  • It is possible that water quality in this study was high in both treatment and control groups: "Second, although the study took place during the rainy season (when waterborne diarrhea outbreaks, including cholera, have occurred in the past), diarrhea rates were much lower than anticipated, particularly in children under 5-years-old. This finding suggests that enteric pathogens were not circulating widely at that time in the population. Third, source water at the time of the study had very low levels of contamination, which suggests that waterborne transmission of diarrheal pathogens may have been low. Previous research has suggested that diarrhea risk increases with the intensity of E. coli contamination in source water, with a significantly increased risk occurring above a threshold of 1,000 colonies/ 100 mL; in this study, the percentage of stored water samples that exceeded this threshold was low and decreased over time" (p. 20). Note that "For 93% of samples, the water source was tap water; ≤ 1% of household water samples were from surface water sources during the study period." p. 18.
  • The authors speculate that it is possible that aspects of the study other than chlorination affected drinking water quality in both the treatment and the control group of this study:
    • Improved water storage: "Finally, the improved water storage vessels, by protecting stored water from the introduction of potential contaminants, may have served as a water quality intervention independent of tablet use and contributed to decreased diarrhea rates in both the intervention and control groups. This possibility is supported by the finding that stored water quality in the control group at the end of the study was significantly better than at baseline and mid-point. Previous research has shown that safe water storage can decrease diarrhea risk." (p. 20). Note that "All study households were given a standard 20-liter plastic vessel with a plastic lid and metal spigot for drinking water storage to assure that participants treated the appropriate volume of water." (p. 16)
    • Frequent home visits by study staff: "...frequent home visits by the field officers may have influenced water handling and household hygienic practices in both the intervention and control groups." (p. 21). Note that "After distribution of the intervention, we initiated active diarrheal surveillance through twice weekly visits (every 3 to 4 days) to all households over a 12-week period; each household had two visits in each calendar week." (p. 17) and "A separate team of technicians visited each household twice weekly to measure free chlorine residuals in stored water using digital Colorimeters..." (p. 17) for a total of 4 visits per household per week.
  • Participants appeared to be adequately blinded, and were far more likely to believe they were receiving treatment than to believe they were receiving placebo regardless of actual assignment. "Of 238 respondents, 127 (53%) believed they received NaDCC tablets whereas only 4 (2%) thought they received placebo tablets; 107 (45%) did not know. When those who did not know were asked to make a guess, 37 (35%) believed they received NaDCC tablets; the rest were unsure but none believed they received placebo tablets. There were no differences between intervention and control groups in beliefs about which group they were in. The Blinding Index was 0.65." p. 19.
  • We have explored the effect of including or excluding Jain et al. 2010 from our summary analysis: Because Jain et al. 2010 found no effect (non-significantly slightly more reported diarrhea in the treatment group compared to the control group) but was not a large study, including this study in our summary effect calculation slightly decreases the estimated effect of water chlorination on under-5 diarrhea. See GiveWell's CEA of Dispensers for Safe Water, 2017, tab "Diarrhea reduction by adherence", rows 24-29: for example, including only studies identified in the Cochrane review, our estimated effect is a 32% reduction in diarrhea for households that adhere to a chlorination intervention when excluding Jain et al. 2010, and a 29% reduction when including Jain et al. 2010.

• 54

  WASH Benefits website

• 55

  "The WASH Benefits Study provides rigorous evidence on the health and developmental benefits of water quality, sanitation, handwashing, and nutritional interventions during the first years of life. The study includes two, cluster-randomized controlled trials to measure the impact of intervention among newborn infants in rural Bangladesh and Kenya. The studies are large in scope (> 5,000 newborns per country) and will measure primary outcomes after two years of intervention." WASH Benefits website, Overview

• 56

  See GiveWell's CEA of Dispensers for Safe Water, 2017, tab "Diarrhea reduction by adherence", for illustration of our methodology. We have excluded Jain et al. 2010 (as discussed above) and adjusted for our expectation of results from the Null et al. unpublished working paper. Due to our limited information about results from WASH Benefits (discussed above), and the sensitivity of discussing such results prior to publication, we are not formally incorporating expectations about results from that study into this analysis.

• 57

  "We sell carbon credits on carbon markets to generate revenue and offset our costs of running Dispensers for Safe Water.
  "A carbon credit represents one ton of carbon dioxide or equivalent of greenhouse gases removed or not emitted into the atmosphere." Dispensers for Safe Water webpage: Dispenser Data

• 58

  "Dispenser for Safe Water is certified by 3rd party auditors to generate such carbon credits by averting the greenhouse gas emissions from a carbon-intensive activity like boiling water in favor of a cleaner safe water technology like chlorine dispensers." Dispensers for Safe Water webpage: Dispenser Data

• 59

  Dispensers for Safe Water negotiated the sale price of their carbon credits in 2013. (GiveWell conversation with Dispensers for Safe Water, unpublished.) We use data from Quandl ICE CER Emission Futures to calculate an average price of carbon credits on the compliance market in 2013, in GiveWell's calculations of 2013 carbon credit price. See cell L1121 for our calculation of an average price of €0.44 in 2013. We convert this to US dollars using the United States Internal Revenue Service's yearly average currency exchange rate of 0.783 US dollars per euro in 2013 for a result of $0.34 per carbon credit.

• 60
  • "Evidence Action expects to be able to receive funding for Dispensers for Safe Water via carbon credits. It could sell a very large number of carbon credits through the compliance market at lower prices, but it expects to receive higher prices by selling credits on the voluntary market." GiveWell's non-verbatim summary of a conversation with Dispensers for Safe Water, July 24, 2014, p. 1.
  • "Companies with greenhouse gas reduction targets purchase carbon credits from “offset” projects in developing countries, provided that those projects also contribute to sustainable development priorities." Dispensers for Safe Water webpage: Dispenser Data

• 61

  We wrote a blog post in 2011 about leverage in philanthropy, including "When we do cost-effectiveness estimates (e.g., “cost per life saved”) we consider all expenses from all sources, not just funding provided by GiveWell donors." Some of our current views may differ from those expressed in that blog post.

• 62

  GiveWell's CEA of Dispensers for Safe Water, 2017, "Parameters" tab, row 40: "Subtract carbon revenues from costs?"

• 63

  Comment provided in response to a draft of this page in November 2017

• 64

  "Dispensers for Safe Water’s highest priority for additional funding is to maintain existing operations across our 28,000 dispensers in Kenya, Uganda, and Malawi. It costs $4.65 million to operate our network, translating to $0.99 per person served per year. This includes the costs of chlorine procurement, delivery of refills and maintenance of dispensers, monitoring & evaluation of the program, continuous engagement of communities to sustain adoption, and all other operating costs. Based on projections as of November 2017, the program could absorb $7.2 million in additional funding for 2018-2020." Arjun Pant, Evidence Action Chief of Staff, email to GiveWell, November 7, 2017.