We have published a more recent version of this page. See our most recent version of this page.

Published: November 2013

What are the diseases targeted by the intervention?

Waterborne diseases are diseases commonly transmitted through contaminants in water. Bathing, washing or drinking contaminated water as well as eating or preparing food with contaminated water can cause a range of diseases, including diarrheal disease, hepatitis A, hepatitis E, poliomyelitis, gastroenteritis and typhoid fever.1 The Global Burden of Disease estimated that in 2010, diarrheal disease alone caused about 10% of child deaths in developing countries.2

What is the intervention?

Many different interventions address water and sanitation in developing countries. Their emphasis ranges from the proper disposal of excreta to promoting hygiene to increasing supply (see our page on water supply interventions). Water quality interventions aim to remove or disable the pathogens found in water, usually through one of the following methods3 :

• Chemical treatment, most commonly with chlorine
• Pathogen removal, such as filtration, absorption or sedimentation
• Heat or UV disinfection, such as boiling or sun exposure
• Protection from recontamination, through, for example, piped distribution and safe storage

The central treatment and direct supply of safe drinking water in high-income countries requires significant startup costs and ongoing maintenance.4 Lower cost interventions in developing countries provide wells to protect groundwater, ceramic filters to remove pathogens, chlorine tablets to disinfect water, or sealable containers for safe storage among other approaches that disinfect water in the home or improve it at its source.5

Does the intervention work?

There is little debate that contaminated water causes diarrhea and that the methods used in interventions to improve water quality, chlorination for example, reduce contamination.6 However, the lack of sanitation, hygiene or access to water presents practical challenges. For example, previously clean water can get recontaminated on the walk home from an improved source, participants may fail to regularly use chlorine tablets, or other pathways for pathogens, such as hands and food, may counteract gains from cleaner water. 7 While we have not reviewed the literature on the biological mechanisms through which pathogens cause diarrhea, some scientists believe it is plausible that water quality interventions will have no effect at all on diarrhea unless they are used consistently or unless all sources of pathogens are eliminated.8 For these reasons, the evidence that real-world programs in developing countries have improved health is less clear.9

Meta-analyses of developing world RCTs

A Cochrane review examining 20 randomized controlled trials (RCTs) enrolling about 30,000 participants in developing countries, found that interventions to improve water quality were effective at preventing diarrhea.10 On average, ten studies reporting odds ratios for all ages showed a 35% reduction in the odds of getting diarrhea with individual study results ranging from showing no effect to showing a 70% reduction.11

The studies included in this analysis share at least one potential methodological weakness. In most water quality studies, researchers measure diarrhea rates through participant reports and study participants know whether or not they have received an intervention. This knowledge may influence reporting of diarrhea cases. For example, participants who know that their water is being treated may be less likely to report diarrhea, inflating estimates of the treatment effect. This knowledge may also influence the investigators surveying participants. Because of this potential bias, some studies conceal, or "blind", who has received the intervention by distributing a placebo (e.g. a sham filter) to the control group. None of the studies included in the above meta-analysis of effects on odds ratios are blinded, so this result may be biased.

Blinded studies

The 5 blinded studies of water quality interventions in developing countries did not find a statistically significant effect.12 All 5 blinded studies examine household interventions (4 evaluate chlorination and 1 evaluates a water filter), which the Cochrane review finds to be more effective than interventions at the water source.13 Four of the blinded studies report longitudinal prevalence ratios (LPRs) as their primary outcome and one of the studies reports rate ratios. Pooling the 4 blinded studies that report LPRs, we calculate a 5% reduction with a 95% confidence interval ranging from a 6% increase to a 14% reduction in the percent of days with diarrhea for children under 5 and no reduction with a 95% confidence interval ranging from a 12% increase to a 11% reduction for all ages. These 5 studies include 2 studies (Kirchhoff et al 1985 and Austin 1993) with less than 200 participants, 1 study with 1,144 participants (Boisson et al 2010), and 1 study with 2,986 participants (Boisson et al 2013). Jain et al 2010 had a sample size of 3,240.14

The lack of effect found in these studies may not be entirely due to blinding. These studies also may have other weaknesses:

• Kirchhoff et al 1985 only enrolled 112 participants, the smallest of all trials included in the Cochrane review, and Austin 1993 only enrolled 287 participants.15
• At baseline, Jain et al 2010 and Boisson et al 2013 reported levels of water contamination that the World Health Organization (WHO) classifies as moderate risk as well as low levels of diarrheal disease, which may have made it harder to detect an effect16
• Boisson et al 2010 reported that the sham filter used for the control group actually reduced water contamination and that blinding was not completely successful.17
• Boisson et al 2013 reported low levels of compliance with the intervention18
• Austin 1993 and Jain et al 2010 offered alternative, water quality interventions to the control group that may have reduced the number of diarrhea cases in the control group and thus reduced the difference between the intervention and control group19
• Boisson et al 2013, Jain et al 2010 and Kirchhoff et al 1985 surveyed participants once a month, twice a week and 3 times a week respectively, which may have led to safer water, sanitation and hygiene practices in the control group and thus reduced the difference between the intervention and control group.20 Zwane et al 2011 finds a 15% decrease in reported diarrhea cases in study participants surveyed every 2 weeks compared to participants surveyed every 6 months.21 The trial also finds evidence that the frequent surveys served as a reminder for households to chlorinate their water.22 All 3 studies reported declines in diarrhea in the control group over the study period though rates also vary seasonally.23

Though these methodological weaknesses may explain why the blinded studies have not found an effect, we have examined these studies in much greater depth than we have examined the large number of unblinded studies. Some of the unblinded studies also have similar weaknesses.

Objectively measured outcomes

Few studies have evaluated the success of interventions based on objectively measured outcomes.

Mortality

One RCT reported on mortality, 14 of the 2,124 in the intervention group receiving flocculant-disinfectant died, compared to 28 out of the 2,277 in the control group, but this study was not designed primarily to look at death as an outcome, so we worry about selective reporting of positive results and the association between the deaths and diarrheal disease.24

Presence of pathogens in stool samples

One RCT analyzed samples of stool for contaminants that cause diarrheal disease.25 This study reported an 80% reduction (95% CI: 20%-97%) in the odds ratio for the presence of Campylobacter and no significant reduction in enteric parasitic infections.26

Nutritional status and anthropometrics

Water quality interventions may prevent malnutrition and stunting.27 3 studies examined changes in anthropometrics related to malnutrition, such as weight-for-age.28 We have not been able to obtain the papers for 2 of these studies.29 . At least 1 study found a positive effect and at least 1 study found no statistically significant effect.30

Historical improvements in water quality

Two observational studies have compared historical changes in mortality in communities that had improved water quality to mortality at the same time in otherwise similar communities:

• Cutler and Miller 2005 attributed a 16% decline in mortality to the adoption of chlorination and filtration in a sample of major cities in the United States from 1900 to 193631
• Watson 2005 finds a 2.5% decrease in infant mortality for every 10% increase in the fraction of homes on Indian reservations in the U.S. receiving sanitation improvements in the 1960s32

We have not examined these studies closely, but we note that Watson 2005 considered more intensive interventions than those described on this page. The intervention on Indian reservations included “digging wells, providing latrines or septic tanks, building or improving water or sewer treatment plants, extending water and sewer lines and connecting individual homes to those lines.”33 . We have not looked closely enough at the historical context for Cutler and Miller 2005 to evaluate whether sanitation and hygiene conditions were similar at the advent of chlorination in the United States to the conditions found in developing countries now.

The bottom line

Several RCTs have shown large effects for water quality interventions, but they have been unblinded and relied on self-reported diarrhea to measure a program’s success. The few blinded studies that have been conducted find no effect but, like some of the unblinded studies, also have methodological weaknesses. It is unclear how large a bias is introduced by self-reporting of diarrhea when respondents know whether or not they have received an intervention. Given the results of the blinded studies, it is possible that this bias accounts for all the effect reported in unblinded studies.

The strength of the evidence relies on how one chooses to weigh a few, blinded studies with some methodological weaknesses that report no effect against a large number of unblinded studies, some of which may also have some methodological weaknesses, that, on average, report large effects. It should be noted that if the intervention were effective, we would guess that a carefully designed and well-executed program would be slightly below the range of cost-effectiveness of opportunities that we expect to direct marginal donations to (about 10x cash or higher, as of 2021).34

Overall, we are ambivalent about the effect of water quality interventions on diarrhea. We find plausible theories grounded in the available evidence for both believing that water quality interventions reduce diarrhea and for the more pessimistic conclusion that these interventions do not have an effect. We also do not see any analysis that may lead to a more definitive answer without a significant, additional investment of time. We therefore summarize our work so far and leave questions for further investigation.

Feedback from scholars

We asked Dr. Alix Zwane, the Executive Director of Evidence Action, and Dr. Thomas Clasen, one of the authors of the Cochrane review and a professor with appointments at Emory University and the London School for Tropical Hygiene and Medicine, to read and comment on this page. See their comments here:

• Comments from Dr. Alix Zwane35
• Comments from Dr. Thomas Clasen36

Questions for water quality charities

• Does the intervention take place in the household, at the water source, or both?
• Does the intervention combine multiple approaches (e.g. hygiene education and chlorination) or use a single approach?
• Can you assure a high level of compliance with the intervention?
• Can you prevent recontamination of the water?
• Does the intervention require upkeep?
• Where does the community usually get its water?
• What is the baseline water quality in the communities you serve?
• What are the sanitation conditions in the communities you serve?
• What is the baseline diarrhea morbidity and mortality in the communities you serve?
• Does the intervention affect the quantity of water consumed by recipients?
• Does the intervention affect additional health behaviors related to safe water consumption?
• What ages is the intervention targeting?

Questions for further investigation

Two additional randomized controlled trials, funded by the Gates Foundation, are currently being conducted that will measure more objective outcomes in addition to diarrhea morbidity and also seek to answer some of the questions in this section, such as whether an approach combining multiple different types of interventions works better than an approach aimed at one pathway for pathogens.37

• How much does the ambient water quality and the level of sanitation and hygiene in the communities where the intervention takes place matter? The Cochrane review finds no evidence that interventions work better in settings with improved sanitation, but we have not carefully vetted these claims.38
• Does an intervention that combines multiple approaches, such as hygiene education and chlorination, work better? Again, Cochrane finds no evidence of this difference, but individual studies need to be examined.39
• How large a bias should we expect in unblinded studies with subjectively assessed outcomes? One meta-epidemiological analysis suggests a 7% to 39% increase in the odds ratio, but we have not carefully vetted this claim.40
• Many studies see declines in diarrhea in both the control and the intervention group. Is this decline due to the effect of frequent surveying on the control or just seasonal variation in diarrhea or is there some other explanation?
• Why did water quality decline to such low levels in the control group in Jain et al 2010?
• How correlated are different levels of indicators of water quality with diarrhea? There may be reasons to think that at certain levels there is little correlation.41
• Did the United States at the advent of piped, treated water start out at a better point than the developing world now in terms of hygiene practices and sanitation levels?
• Are quasi-RCTs strong enough to be included in the analysis? It may affect sub-analyses more than the overall analysis.
• Many of the comparisons have to be made separately by outcome measure. What information does each trial give if we decided on a common outcome?
• How does cost-effectiveness vary for different types of water quality interventions? With more robust inputs, how does the cost-effectiveness change?
• In what contexts, do chlorine-resistant pathogens undermine the impact of chlorination interventions on diarrhea? A recent study shows that the chlorine-resistant cryptosporidium is an important cause of severe diarrhea.42
• What is the evidence on the need for consistent use of safe water, or even exclusive use, to successfully reduce diarrhea? There are some studies showing with epidemiological models that occasional consumption of contaminated water may counteract the gains of an intervention.43

Sources

Document	Source
Austin 1993	Unpublished
Boisson et al 2010	Source (archive)
Boisson et al 2013	Source (archive)
Checkley et al 2008	Source (archive)
Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013	Source
Clasen 2009	Source (archive)
Clasen et al 2005	Source (archive)
Clasen et al 2006	Source (archive)
Clasen et al 2007	Source (archive)
Clasen, Brown and Collin 2006	Source (archive)
Colford et al 2002	Source (archive)
Colford et al 2005a	Source (archive)
Colford et al 2005b	Source (archive)
Colford et al 2009	Source (archive)
Condran and Crimmins-Gardner 1978	Source (archive)
Crump et al 2005	Source (archive)
Cutler and Miller 2005	Source (archive)
GBD Compare	Source (archive)
GiveWell, Cost-effectiveness analysis, 2020, Version 2	Source
GiveWell, Water quality CEA, 2021	Source
GiveWell, Zinc/ORS moral weights, 2021	Source
Gorchev and Ozolins 1984	Source (archive)
Jain et al 2010	Source (archive)
Kirchhoff et al 1985	Source (archive)
Kremer et al 2011	Source (archive)
Mahfouz et al 1995	Source (archive)
Meeker 1972	Source (archive)
Moe et al 1991	Source (archive)
Quick et al 1999	Source (archive)
Rose et al 2005	Source (archive)
Universidad Rafael Landivar 1995	Source (archive)
Watson 2005	Source (archive)
WASH Benefits	Source (archive)
Wood et al 2008	Source (archive)
Alix Zwane, Executive Director of Evidence Action, email to GiveWell, November 22, 2013	Source
Zwane et al 2011	Source (archive)

• 1
  • “Many of these diarrhoegenic agents are potentially waterborne-transmitted through the ingestion of contaminated water. However, most of the same pathogens are also transmitted by ingestion of contaminated food and other beverages, by person-to-person contact, and by direct or indirect contact with infected faeces.” Clasen et al 2006, pg. 3.
  • "Other important diseases associated with drinking water, such as hepatitis A and E, poliomyelitis, gastroenteritis and typhoid fever, may not cause diarrhoea but are nevertheless associated with potentially waterborne microbes of faecal origin” Clasen et al 2006, pg. 3.

• 2

  Diarrhea accounted for roughly 10% of deaths of children under 5 in 2010. GBD Compare

• 3

  “A number of interventions have been developed to improve the microbiological quality of water and can be grouped into four main categories.

  • Physical removal of pathogens (e.g. filtration, adsorption, or sedimentation).
  • Chemically treating water to kill or deactivate pathogens, most commonly with chlorine.
  • Disinfection by heat (e.g. boiling or pasteurization) and ultraviolet (UV) radiation, either using the sun (solar disinfection) or an artificial UV lamp.
  • Combination of these approaches (e.g. filtration or flocculation combined with disinfection).

  Water quality can also be enhanced by protecting it from recontamination, for example, by residual disinfection, piped distribution, and safe storage. A combination approach is also common in conventional systems since individual approaches are not effective against the full range of microbial pathogens under all water conditions. Mechanical removal of viruses, for example, presents a challenge to most filters due to their submicron size. Similarly, certain encysted protozoa are resistant to chemical disinfection. The microbiological performance of these approaches may also be impacted by the temperature, pH, turbidity, chemical content, and other characteristics of the water.” Clasen et al 2006, pg. 4.

• 4

  “In higher income countries, and in many urban settings worldwide, drinking water is treated centrally at the source of supply and is distributed to consumers through a network of pipes and household taps. However, such conventional systems involve significant upfront investment and continued maintenance.” Clasen et al 2006, pg. 4.

• 5
  • “In remote and low-income settings, water quality may nevertheless be improved at the source by, for example, providing protected groundwater (springs, wells and bore holes) or harvested rainwater as an alternative to surface sources (rivers and lakes) that are more susceptible to faecal contamination. Microbial water quality may also be improved at the source or other point in the distribution system by chlorination, filtration, and other means.” Clasen et al 2006, pg. 4.
  • “Interventions aimed at improving the microbiological quality of drinking water, including steps to improve water quality by removing or inactivating microbiological pathogens (eg filtration, sedimentation, chemical treatment, heat or UV radiation) and protecting the microbiological integrity of water prior to consumption (eg residual disinfection, protected distribution, or improved storage).” Clasen et al 2006, pg. 5.

• 6

  “Health authorities generally accept that microbiologically safe water plays an important role in preventing outbreaks of waterborne disease (Hunter 1997). Accordingly, the most widely accepted guidelines for water quality allow no detectable level of harmful pathogens at the point of distribution (WHO 2004).” Clasen et al 2006, pg. 3.

• 7
  • “For those who have access to sufficient quantities of water but whose water is of poor microbiological quality, an alternative is to treat water at the household or other point of use. Such household treatment may minimize recontamination in the home, a well-known cause of water quality degradation (Wright 2004).” Clasen et al 2006, pg. 4.
  • “Compliance with the intervention (ie consumption of the improved quality water) is an important factor in assessing potential impact of the intervention…The trials of chorine residuals report compliance ranging from a high of 95% (Doocy 2004) to a low of 27% (Reller 2003-i).” Clasen et al 2006, pg. 9.
  • "Many of these diarrhoegenic agents are potentially waterborne-transmitted through the ingestion of contaminated water. However, most of the same pathogens are also transmitted by ingestion of contaminated food and other beverages, by person-to-person contact, and by direct or indirect contact with infected faeces...Because of the multiple pathways of diarrheogenic infection, improvements in water quality alone may not necessarily interrupt transmission (Briscoe 1984)” Clasen et al 2006, pg. 3.

• 8

  “Based on epidemiological modeling, even occasional consumption of untreated water can vitiate the impact of HWTS. This may be an almost insurmountable challenge of household-based solutions when children and adults are regularly consuming water in places outside the home” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, pg. 1

• 9

  “In settings that are not served by reliable water treatment and distribution systems, diarrhoeal disease is often endemic, that is, present or usually prevalent in the population at all times. In such settings much of the epidemiological evidence for increased health benefits following improvements in the quality of drinking water has been equivocal (Esrey 1986; Lindskog 1987; Cairncross 1989)” Clasen et al 2006, pg. 3.

• 10
  • Clasen et al 2007 reviewed 33 studies in total. 22 were RCTs and 11 were quasi-randomized trials, where assignment to an intervention group or control group was based on a pseudo-random sequence, such as date of birth. Two of the randomized trials (Colford et al 2002 and Colford et al 2005a) took place in the United States. We restrict our analysis to the RCTs in developing countries. We exclude quasi-randomized controlled trials, because these studies may be prone to bias. Note that Clasen et al 2007 included 3 studies that Clasen et al 2006 had not previously included (See Clasen et al 2005, Colford et al 2005a and Rose et al 2005 in Table 1, Clasen et al 2007). All 3 studies were unblinded RCTs and one of the studies, Colford et al 2005a, took place in the United States. The other 2 studies, Clasen et al 2005 and Rose et al 2005 had sample sizes of 680 and 200 respectively. See Table 1, Clasen et al 2007, pg. 3, but note that source w6 is incorrectly assigned to “Clasen 2006” (Clasen, Brown and Collin 2006), instead of “Clasen 2005” (Clasen et al 2005).
  • An update of this Cochrane review is due out in 2014: “One is an update of my 2006 Cochrane review which is due out next year” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, pg. 1.
  • ”The 19 randomized controlled trials included at least 29,920 participants (median 607, range 112 to 6650…” Clasen et al 2006, pg. 8. Because some studies randomized at the village level and the behavior of participants within a village may be correlated, the effective sample size of the 19 RCTs is less than the number of participants in all studies Clasen et al 2006, pg. 24-37.
  • "In general, the evidence suggests that interventions to improve microbiological quality of drinking water are effective in preventing diarrhea both for populations of all ages and children less than five years old.” Clasen et al 2006, pg. 1.

• 11
  • Table 2 shows pooled estimates Clasen et al 2007, pg. 4. Pooling 10 studies reporting on the odds ratios, all of which were RCTs in developing countries, shows a 35% reduction (95% CI: 24%-44%). For studies reporting on other outcome measures, we calculated pooled estimates using the random effects model employed in the Cochrane review, because Clasen et al 2007 does not restrict their analysis to RCTs in developing countries (the authors include quasi-RCTs and 2 trials conducted in the U.S.). See spreadsheet. The longitudinal prevalence, or the percent of days with diarrhea, is the only outcome for which the pooled estimate is not statistically significant Clasen et al 2007, pg. 4. It is also the outcome “reported to be more closely associated with mortality” Boisson et al 2013, pg. 5. But “excluding Doocy 2004, which reported a very large and statistically significant effect for both age groups and is a possible outlier” Clasen et al 2006, pg. 11, we find that the pooled estimate goes from a 43% reduction with a 95% confidence interval ranging from a 25% increase to a 74% reduction to a 29% reduction with a 95% confidence interval ranging from a 12% reduction to 43% reduction. See spreadsheet.
  • Figure 2 shows individual study estimates Clasen et al 2007, pg. 6.

• 12
  • The Cochrane review cites 2 of the 5 studies in developing countries: Kirchhoff et al 1985 and Austin 1993. We found 3 others that were published since the review: Jain et al 2010, Boisson et al 2010, and Boisson et al 2013.
  • We exclude RCTs that took place in high-income countries, including Colford et al 2002 and Colford et al 2005a, which were included in the Cochrane review, as well as Colford et al 2005b and Colford et al 2009, because the context in these studies differs so much from the developing world settings of primary interest.
  • "Only three of the 19 randomized controlled trials were blinded, and in each case the intervention had no statistically significant protective effect. This must give pause to any definitive conclusion about the potential value of water quality interventions in the prevention of diarrhoea” Clasen et al 2006, pg. 15.
  • "We conducted a randomized, placebo-controlled, triple-blinded trial to determine the health impact of daily use of sodium dichloroisocyanurate (NaDCC)...Results of this study suggest that the use of NaDCC tablets, despite contributing to marked improvement in stored water quality, had no impact on diarrheal diseases in the study population.” Jain et al 2010, pg. 1, 19.
  • "Our study was designed to overcome the shortcomings of previous double-blinded trials of household water treatment in low-income settings. The sample size was larger, the follow-up period longer, both urban and rural populations were included, and adherence and water quality were monitored extensively over time. These results provide no evidence that the intervention was protective against diarrhea.” Boisson et al 2013, pg. 1.
  • “We undertook a randomized, double-blinded placebo-controlled trial among 240 households (1,144 persons) in rural Democratic Republic of Congo to asses the field performance, use and effectiveness of a novel filtration device in preventing diarrhea…While the filter was effective in improving water quality, our results provide little evidence that it was protective against diarrhea.” Boisson et al 2010, pg. 1.

• 13

  “Subgroup analyses suggest that household interventions are more effective in preventing diarrhoea than interventions at the water source” Clasen et al 2006, pg. 1.

• 14

  We calculated pooled estimates using the random effects model employed in the Cochrane review. We extracted data on outcomes and their standard errors from the Cochrane review for studies included in that review. For Boisson et al 2010 and Boisson et al 2013, we extracted the longitudinal prevalence ratio and a 95% confidence interval from the paper and then approximated the standard error of the log longitudinal prevalence ratio based on those inputs and assuming a log-normally distributed outcome. See spreadsheet.

• 15
  • "Kirchhoff 1985, though a pioneering trial of a potentially important household intervention, had a study population of only 112 persons (smallest of all the included trials)” Clasen et al 2006, pg. 15.
  • “Characteristics of included studies”, Clasen et al 2006, pg. 24.

• 16
  • "At baseline, median estimated E. coli MPN per 100 mL was 93 in the intervention group and 219 in the control group. At the end of the study, the median estimated E. coli MPN per 100 mL was 0 in the intervention group and 1 in the control group." Jain et al 2010 Table 4, pg. 20. WHO has defined a count of 1-10 thermotolerant (fecal) coliforms or E. coli per 100 mL in the water as low risk, a count of 10-100 per 100mL as intermediate risk and a count of 100-1000 per 100 mL as high risk Gorchev and Ozolins 1984 Table 5.2 pg. 78.
  • "Another potential explanation for the lack of health effect was the comparatively modest improvement in water quality among intervention households. With a mean of 50 TTC/100 mL, even water sampled from intervention households would be classified as ‘moderate risk’ using WHO nomenclature. Other studies of household water treatment have reported higher baseline levels of contamination and larger reduction in faecal contamination of drinking water from the intervention” Boisson et al 2013, pg. 10.
  • Jain et al 2010 and Boisson et al 2013 show a decline in the percentage of days with diarrhea in both the intervention and control groups over the study period from approximately 1.7% at baseline to about 0.5% at the end of the trial Jain et al 2010 Figure 2, pg. 19.; Boisson et al 2013 Figure 2,pg. 6. As a point of comparison, a multi-country review of the effect of diarrhea on childhood stunting found an average longitudinal prevalence of diarrhea of 8.06 for children under 2 years of age with individual study estimates of prevalence ranging from 2.4 to 16.33 Checkley et al 2008, pg. 821.

• 17
  • “Moreover, the placebo was not microbiologically neutral, as it removed about 90% of faecal bacteria from the source water used by control households. The reasons for this apparent effectiveness are not clear. Field staff responsible for water quality testing were extensively trained and supervised through the study, thereby minimizing the risk of measurement errors. One of the most plausible explanations is the formation of a biofilm resulting from adhesion of suspended solid particles and bacteria on the inner surface of the plastic pipe forming the placebo cartridge. The effectiveness of the placebo rendered our trial a comparison between a 1-log filter and a 3-log filter.” Boisson et al 2010, pg. 9
  • “The blinding of the intervention was not successful. In both treatment groups, the vast majority of survey respondents believed that they had received the active filter, although this proportion was significantly lower in the placebo group…The fact that large proportion of control households remained blinded throughout the trial suggests that respondents’ bias may have at least been partly reduced.” Boisson et al 2010, pg. 9

• 18
  • "While 51% of intervention households reported their child’s drinking water to be treated with the tablets at the time of visit, only 32% of the water samples tested positive for RFC” Boisson et al 2013, pg. 9. Other studies had higher levels of compliances “Appendix 4: Interventions”, Clasen et al 2006, pg. 111.
  • "The low level of uptake was unanticipated...A number of chlorine-based interventions have achieved compliance in excess of 80%...Uptake did increase over time, and it is possible that the low uptake at the start was due in part to challenges in scaling up the promotional campaign” Boisson et al 2013, pg. 10.

• 19
  • "Austin 1993-i also suggested possible methodological issues and used dilute sodium hypochlorite in the control group, an approach that probably improved their water quality thus resulting in an understatement of the intervention’s effectiveness” Clasen et al 2006, pg. 15-16.
  • "We randomized 240 households (3,240 individuals) to receive either NaDCC or placebo tablets. All households received a 20-liter safe water storage vessel” Jain et al 2010, pg. 16.

• 20
  • ”Trained fieldworkers visited households every month for 12 mo (between late December 2010 and December 2011)” Boisson et al 2013, pg. 5.
  • "To improve recall, we visited homes twice a week...frequent home visits by the field officers may have influence water handling and household hygienic practices in both the intervention and control groups” Jain et al 2010, pg. 21.
  • "Information on gastro-intestinal illnesses was collected by three medical students on thrice-weekly visits to each of the participating households” Kirchhoff et al 1985, pg. 176.
  • There is at least one study of a water quality intervention that has surveyed substantially less frequently than other studies and found positive effects. A randomized controlled trial conducted after the Cochrane review that surveyed participants once every 6 months found that "spring protection significantly reduces diarrhea for children under age three at baseline or born since the baseline survey” and, in particular, that diarrhea incidence falls by 4.5 percentage points” due to the intervention Kremer et al 2011, pg. 168. We found that survey frequencies for other studies ranged from once every 6 weeks to 3 times a week with a median frequency of once per week (See spreadsheet). Less frequent surveying requires larger sample sizes to power the study for the same effect size, which may be one reason it is more rare. Kremer et al 2011 enrolled a larger sample size. The study was also unblinded and relied on self-reports of diarrhea.

• 21

  Column 2, Table 1, Zwane et al 2011, pg. 1822.

• 22

  "Chlorine use fell in both groups, but column 4 shows that usage was 5 percentage points higher (P = .06) in the biweekly survey group. This smaller treatment effect (compared with round 9) is consistent with the hypothesis that a survey serves as a reminder to chlorinate, with an effect that falls over time in the absence of reminders” Zwane et al 2011, pg. 1823.

• 23
  • Figure 2, pg. 19. Jain et al 2010
  • Figure 2, pg. 6. Boisson et al 2013
  • Figure 1, pg. 177. Kirchhoff et al 1985

• 24
  • "There were 28/2277 deaths in the control group, 14/2124 in the flocculant-disinfectant group (relative risk of death 0.53, P=0.052 compared with the control), and 17/2249 in the sodium hypochlorite group (0.61, P=.108).” Crump et al 2005, pg. 3.
  • "Two trials reported on death. Crump 2004-I reported a risk ratio of 0.53 (P = 0.052) for the flocculant and disinfectant arm of a household-based water treatment and 0.61 (P = 0.108) for the disinfectant-only arm. No physical or verbal autopsies were performed, and no association between deaths and diarrhoea was established. Messou 1997, which involved a combination of source water improvement with an oral rehydration intervention and hygiene instruction, reported an 85% reduction (from 27% to 4%) in the proportion of deaths related to diarrhoea in the villages with the intervention (P = 0.04) compared with no reduction in control villages. The trial also reported an 85% reduction (from 5.3% to 0.8%) in the death rate associated with diarrhoea morbidity among intervention villages (P = 0.04) with no correspondingly decline in control villages. We emphasize that neither trial was primarily designed to investigate the impact of the intervention on death, and that such studies may require important differences in study design, sample size, and other parameters” Clasen et al 2006, pg. 13.

• 25
  • Quick et al 1999, a RCT, and Mahfouz et al 1995, a quasi-randomized controlled trial, both analyzed stool samples. “Rectal swabs were obtained from 36 (43%) of 83 diarrhoea patients in 22 household in the intervention group and from 60 (41%) of 148 patients in 28 control households” Quick et al 1999, pg. 5.
  • Mahfouz et al 1995 reported declines in the prevalence of intestinal parasites in the intervention group: “Table 3 shows that there was no significant difference between the groups before the start of the programme regarding the prevalence of Entamoeba coli, E. histolytica, and Giardia lamblia. Stool samples taken at the end of the study period from Group I children showed a significant decrease in the prevalence of intestinal parasites.” Mahfouz et al 1995, pg. 129. “Low response rate figures”, including the numbers for the control group after chlorination, “were omitted from statistical analysis” and not reported Mahfouz et al 1995, pg. 129.

• 26
  • “Campylobacter was isolated from stool specimens from 2 (9%) of 22 intervention households and 12 (43%) of 28 control households (OR 0.2, 95% confidence interval 0.03-0.8). Stool specimens from 8 households yielded ETEC, 5 of which were obtained from patients in intervention households (P = 0.28). Only one (1%) stool specimen yielded salmonella and one (1%) yielded shigella; both were from control group patients.” Quick et al 1999, pg. 7.
  • “ELISA tests for rotavirus were conducted on 65 (68%) of 96 specimens; 4 (6-2 %) were positive, three of these from intervention group patients. Of 91 stool samples examined microscopically, Ascaris lumbricoides was identified in 38 (42%). Giardia lamblia in 21 (23%). Enmrnoeba histolytka in 1 (1%). and other parasites in 25 (27%). Thirty-three (35%) specimens were negative and 27 (30%) samples had multiple parasite species. There was no significant difference in the number of enteric parasitic infections in intervention and control group patients.” Quick et al 1999, pg. 7.

• 27

  “And since diarrhoeal diseases inhibit normal ingestion of foods and adsorption of nutrients, continued high morbidity is an important cause of malnutrition, leading to impaired physical growth and cognitive function (Guerrant 1999), reduced resistance to infection (Baqui 1993), and potentially long-term gastrointestinal disorders (Schneider 1978).” Clasen et al 2006, pg. 3

• 28
  • Both Austin 1993 and Universidad Rafael Landivar 1995 reported on anthropometrics. “Characteristics of studies” Clasen et al 2006, pg. 24-38
  • “Weight-for-age z-score (WAZ), school absenteeism, and health care expenditure for diarrhea were included as secondary outcomes.” Boisson et al 2013, pg. 5.

• 29

  We could not obtain the papers for Austin 1993 and Universidad Rafael Landivar 1995.

• 30
  • “In one of the few blinded trials of HWTS, anthropometrics showed that intervention group members were less likely to suffer from malnutrition than those in the control group (Austin, 1992).” Clasen 2009, pg. 4.
  • “Each additional day of diarrhoea resulted in a decrease of 0.0249 in WAZ (95% CI 20.0365 to 20.0133). However, the intervention had no effect on WAZ (mean WAZ of 21.589 among control children versus 21.586 among intervention). The regression coefficient after adjusting for baseline WAZ was 0.0003 (20.0347; 0.0354).” Boisson et al 2013, pg. 9.

• 31

  "On average, filtration reduced typhoid fever mortality by 46%, total mortality by 16%, infant mortality (ages 0-1) by 43%, and child mortality (ages 1-4) by 46%. These are large effects. The second row shows estimates of the chlorination effects, suggesting that chlorination alone had no detectable effect on mortality. The third row shows the coefficient estimates for the interaction between filtration and chlorination. These coefficients are positive for typhoid fever mortality and total mortality, suggesting that filtration and chlorination were substitute technologies” Cutler and Miller 2005, pg. 11.

• 32

  "The empirical analysis shows that sanitation interventions have large effects. A 10 percentage point increase in the fraction of homes receiving sanitation improvement reduced Indian infant mortality by 0.51 per 1000 births, or by 2.5 percent.” Watson 2005, pg. 5.

• 33

  Watson 2005, pg. 8.

• 34

  This guess is based on a rough cost-effectiveness model we put together in Nov 2013 (using approximate costs from Dispensers for Safe Water) and updated with our most recent moral weights as of February 2021.

• 35Alix Zwane, Executive Director of Evidence Action, email to GiveWell, November 22, 2013
  
• 36Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013
• 37WASH Benefits
  
• 38

  "...water quality interventions appear to be effective in preventing diarrhoea regardless of whether they are deployed in settings where sanitation is ‘improved (ie connection to a public sewer or septic system, pour-flush latrine, simple pit latrine, or ventilated improved pit latrine) or ‘unimproved’” Analysis 17.1, Analysis 18.1, Analysis 19.1 Clasen et al 2006, pg. 15.

• 39"The subgroup analyses did not demonstrate that the effectiveness of a water quality intervention to prevent diarrhoea is enhanced by adding hygiene instruction, a separate vessel to treat or store water, or by improving sanitation or water supply” Analysis 20-23, Clasen et al 2006, pg. 15.

• 40

  “…bias associated with lack of blinding seemed restricted to trials with subjectively assessed outcomes (ratio of odds ratios 0.75 (0.61 to 0.93))…” Wood et al 2008, pg. 4.

• 41

  “Little difference was observed between the illness rates of children drinking water good quality water ([Less than] 1 E. coli per 100 ml) and those drinking moderately contaminated water (2 – 100 E. coli per 100 ml)” Moe et al 1991

• 42

  “Chlorine has been widely promoted by HWT advocates, partly because of its accessibility (local availability and up front affordability), despite challenges in it acceptability and use. Given the recent evidence from the GEMS study on the role of chlorine resistant crypotsporidium as the aetiological agent for severe diarrohea (it was ranked 2 or 3 in most settings) (Kotloff 2013), I think we should refrain from promoting chlorine-only solutions (sodium hypochlorite, NaDCC, etc.), just as we should ceramic filters that are ineffective against viruses.” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, pg. 1.

• 43

  “Paul Hunter pointed to this in a paper in 2009; others have done more work on this (Brown & Clasen 2012; Enger 2013). Based on epidemiological modeling, even occasional consumption of untreated water can vitiate the impact of HWTS. This may be an almost insurmountable challenge of household-based solutions when children and adults are regularly consuming water in places outside the home.” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, pg. 1.