Published: 2009

Basics of the program

What is the program? What problem does it target?

This page focuses on water supply programs focused on health issues (though there are other potential benefits to improving the water supply, such as saving time and labor). Relevant diseases include diarrhea, trachoma, and schistosomiasis, all of which are transmitted through water or can be alleviated through improved hygiene.1 Of these, diarrhea has by far the largest potential burden of disease averted by improved access to clean water.2

What are the components required to implement this program - how does it work?

There are many types of improved water supply programs. A key distinction is between house connections, which provide water directly to a user's home, and public water points, which provide water at a shared, communal location, such as a standpost, borehole, or dug well.3

This page focuses on "source" interventions - improving the water supply - as opposed to "point of use" interventions such as treating water with chlorine.

Program track record

Micro evidence: Has this program been rigorously evaluated and shown to work?

While it may seem intuitive that improving the supply of water would reduce diarrhea, we have seen it argued that the effects may be negligible when there are other major sanitation issues. 4

The best review of the relevant evidence we've seen is a Cochrane review, "Interventions to improve water quality for preventing diarrhoea" (Clasen et al. 2009).

The review distinguishes between "source" interventions (which this report focuses on) and "household/point of use" interventions (treating water with chlorine, filters, etc.)5 Concerning the former, it does not find any randomized controlled trials;6 it instead seeks "quasi-randomized" studies in which participants and non-participants appear similar on relevant measures prior to the intervention.7 It analyzes six such studies, and finds ambiguous/mixed results.8 Results from household-based interventions are more encouraging, finding statistically significant drops in diarrhea episodes by most measures.9

The review notes some major concerns in interpreting these results:

• The vast majority of the studies reviewed were not fully blinded ("blinding" refers to the use of a placebo, dummy device, or other technique for ensuring that the investigators and participants don't know whether they're in the treatment group10 ), and the few studies that were fully blinded found no statistically significant impact.11 This is particularly concerning because the studies nearly all relied on self-reported data to assess impact on diarrhea episodes.12
• The review notes that the studies seemed to be of programs conducted explicitly for the purposes of the study; it raises the concern that the results may not generalize to real-world conditions.13
• The review also raises some flags regarding publication bias, though the issues it notes are both highly inconclusive.14

The review concludes:

More encouraging evidence comes from a relatively recent study that uses a randomized rollout of spring protection to gauge the effect on water quality and health.16 The study attributed a 66% reduction in water contamination17 and a 24% reduction in diarrhea incidence18 to the intervention.

Overall, we feel that improving water supply may have substantial potential health benefits, but we also see reason to believe the benefits depend heavily on the details of the project and context. We don't believe there is any approach to improving the water supply that has the same level of evidence support that our priority programs have.

(More on our interpretation of "micro evidence" and evaluation quality.)

An additional concern: the question of maintenance

According to Kremer and Zwane (2006):

A recent survey conducted by a UNICEF representative estimates the percentage of handpumps that are nonfunctioning by country, concluding, "It has been estimated that between 20% and 70% of installed handpumps in Sub-Saharan Africa are not functioning."20

We do not know what methodology was used to arrive at the estimates discussed above; we have not vetted these estimates and are not confident in them, but provide them as an illustration that there is a legitimate potential concern regarding this issue.

Macro evidence: Has this program played a role in large-scale success stories?

We know of no such large-scale success stories.

Recommendations and concerns

What are the potential downsides of the intervention?

We have not identified any widely recognized downsides.

Cost-effectiveness

We have not done thorough cost-effectiveness analysis of water infrastructure programs. Because such analysis is highly time-consuming - and because the results can vary significantly depending on details of the context - we generally do not provide cost-effectiveness analysis for an intervention unless we find what we consider to be a strong associated giving opportunity.

We provide some preliminary figures based on the Disease Control Priorities in Developing Countries report, which we previously used for cost-effectiveness estimates until we vetted its work in 2011, finding major errors that raised general concerns.

The report estimates that water supply programs can cost $159 per disability-adjusted life-year (DALY) averted when implemented in areas without existing access to water, though they cost far more ($1,974-6,396 per DALY) when implemented in areas with some existing infrastructure.21 (More on the DALY metric.)

Using a simple conversion calculation,22 we estimate that ~$5,000 prevents a death from diarrhea and ~2,100 less severe diarrhea episodes.

The high-quality study of spring protection discussed above comes to a significantly more optimistic estimate: $16.75 per DALY, implying ~$544 per death averted.23

Note that these estimates assume successful implementation in an area without previous access to clean water/infrastructure. We also note that cost-effectiveness may be diminished when water infrastructure is not properly maintained, something that (as we discuss above) we feel is a legitimate concern.

Sources

• Clasen, TF, et al. 2009. Interventions to improve water quality for preventing diarrhoea (Review) (PDF).
• Esrey, Steven, et al. 1991. Effects of improved water supply and sanitation on ascariasus, diarrhoea, dracunculiasis, hookworm infection, schistosomiasis, and trachoma (PDF). Bulletin of the World Health Organization 69: 609-621.
• Harvey, Peter, UNICEF Zambia, Rural Water Supply Network. Handpump data. http://www.rwsn.ch/prarticle.2005-10-25.9856177177/prarticle.2005-10-26…
  (accessed November 1, 2012). Archived by WebCite® at http://www.webcitation.org/6BrEdjgVG.
• Jamison, Dean T., et al., eds. 2006. Disease control priorities in developing countries (PDF). 2nd ed. New York: Oxford University Press.
• Kremer, Michael, and Alix Peterson Zwane. 2006. Cost-effective prevention of diarrheal diseases: A critical review (PDF).
• Kremer, Michael, et al. 2009. Spring cleaning: A randomized evaluation of source water quality improvement (PDF).
• Whittington, Dale, et al. 2008. Copenhagen Consensus 2008 challenge paper: Sanitation and water (PDF).
• World Health Organization. Disease and injury regional estimates for 2004: DALYs for WHO regions (XLS).
• World Health Organization. Disease and injury regional estimates for 2004: Deaths for WHO regions (XLS).
• World Health Organization. Disease and injury regional estimates for 2004: Incidence for WHO regions (XLS).

• 1

  Jamison et al. 2006, Pg 775, Table 41.1.

• 2

  "Because the effect on diarrheal disease accounts for the vast majority of the effect, no effort is made to apportion the costs between their effectiveness in preventing the other diseases affected by water supply, sanitation, and hygiene." Jamison et al. 2006, Pg 789.

• 3

  "The report treated the following technologies as improved: household connection, public stand- pipe, borehole, protected (lined) dug well, protected spring, and rainwater collection.the user's dwelling...Within the broad category of those with reasonable access to an improved water supply, two significantly different levels of service can be distinguished:

  • house connections
  • public or community sources.

  In most settings, these subcategories correspond to very different levels of water consumption, different amounts of time spent collecting water, and as discussed in later sections, different health benefits." Jamison et al. 2006, Pg 772.

• 4

  "Providing a public water point appears to have little effect on health, even where the water provided is of good quality and replaces a traditional source that was heavily contaminated with fecal material. By contrast, moving the same tap from the street corner to the yard produces a substantial reduction in diarrheal morbidity. How is this pattern to be understood?

  The first step to an explanation is an understanding that most endemic diarrheal disease is transmitted by water-washed routes and is not waterborne. Although waterborne epidemics of diarrheal diseases such as cholera and typhoid have been notorious in the history of public health, the endemic pattern of transmission seems to be different, particularly in poor communities. Five types of evidence support this view:

  • Negative health impact studies. As mentioned earlier, Esrey
    and Habicht (1985) and Esrey and others (1991) cite a number of studies of the health impact of water supplies in
    which water quality improvements have failed to have a significant effect on diarrheal disease incidence.
  • Foodmicrobiology. Studies of the microbiology of foods in
    developing countries - particularly the weaning foods fed
    to children in the age group most susceptible to diarrheal
    disease - have shown such food to be far more heavily contaminated with fecal bacteria than is drinking water (Lanata 2003), even when the water has been stored in
    open pots.
  • Seasonality of diarrhea. In countries with a seasonal variation in temperature, bacterial diarrheas peak in the warmer
    season, whereas viral diarrheas peak in the winter. This pattern suggests that the bacterial pathogens show environmental regrowth at some stage in their transmission route,
    which means that they must have a nutritional substrate.
    Water is, thus, a less likely vehicle than food.
  • Fly-control studies. Trials in rural Asia and Africa have
    shown that fly control can reduce diarrheal disease incidence by 23 percent (Chavasse and others 1999).
  • Hand-washing studies. A recent systematic review of the
    effect of hand washing with soap has shown that this simple
    measure is associated with a reduction of 43 percent in diarrheal disease and 48 percent in diarrheas with the more life-threatening etiologies (Curtis and Cairncross 2003).

  Those five types of evidence suggest that domestic Hygiene - particularly food and hand hygiene - is the principal determinant of endemic diarrheal disease rates and not drinking water quality.

  The second step is an understanding of how the level of service and convenience of a water supply influence such hygiene practices in the home.Taking the amount of water used per capita as an indicator of hygiene changes, other things being equal, one finds that providing a source of water closer to the home - and therefore more convenient to use - has very little effect on water consumption unless the old source was more than 1 kilometer (30 minutes - roundtrip journey) away from the user's dwelling (Feachem and others 1978).

  However, water consumption doubles or triples when house connections are provided (White, Bradley,and White 1972), and reason exists to believe that much of the additional consumption is used for hygiene purposes. For example, Curtis and others (1995) found that provision of a yard tap nearly doubled the odds of a mother washing her hands after cleaning her child's anus and more than doubled the odds that she would wash any fecally soiled linen immediately. In conclusion, water supplies are likely to have an effect on diarrheal disease when they lead to hygiene behavior change - that is, when the old source of water was more than 30 minutes' roundtrip away or when house connections are provided. " Jamison et al. 2006, Pgs 777-778.

• 5

  "In higher income countries, and in many urban settings worldwide, drinking water is treated centrally at the source of supply and is distributed to consumers through a network of pipes and household taps. However, such conventional systems involve significant upfront investment and continued maintenance. In remote and low-income settings, water quality may nevertheless be improved at the source by, for example, providing protected groundwater (springs, wells, and bore holes) or harvested rainwater as an alternative to surface sources (rivers and lakes) that are more susceptible to faecal contamination. Microbial water quality may also be improved at the source or other point in the distribution system by chlorination, filtration, and other means. Improving water at the source is also frequently accompanied by improvements in quantity or access to water by increasing the volume or frequency of water delivery or reducing the time spent in collecting water. This may result in significant benefits not only in health but also in economic and social welfare (Hutton 2004). For purposes of this review, any form of treatment at the water source or otherwise prior to the point of use will be referred to collectively as 'source' water treatment.

  For those who have access to sufficient quantities of water but whose water is of poor microbiological quality, an alternative is to treat water at the household or other point of use. Such household treatment may minimize recontamination in the home, a well-known cause of water quality degradation (Wright 2004). At the same time, certain household water filters have been associated with adverse health impacts (Payment 1991a). A review commissioned by the World Health Organization (WHO) identified a wide variety of options for household-based water treatment and assessed the available evidence on their microbiological effectiveness, health impact, acceptability, affordability, sustainability, and scalability (Sobsey 2002). Research on the economics of such interventions also suggests that where adequate quantities of water are already available, household-based water treatment is among the most cost-beneficial and cost-effective approaches in preventing diarrhoeal disease (WHO 2002; Hutton 2004). There is also evidence that the vulnerable population to whom such household-based interventions have been targeted will pay all or a portion of the cost of household water treatment products (Clasen 2004c)." Clasen et al. 2009, Pg 4.

• 6

  "Nineteen trials were randomized and 11 were quasi-randomized. Study design varied with the type of intervention: 19 of 23 trials of household interventions were randomized controlled trials; and the all seven trials of interventions at the water source or other point prior to distribution used quasi-randomization. Most randomized controlled trials used households as the unit of randomization, while some used neighbourhoods or other clusters of households (Chiller 2004; Doocy 2004; Luby 2004b-i), or villages or other communities (Austin 1993-i)." Clasen et al. 2009, Pg 7.

• 7

  "Additionally, we assessed quasi-randomized controlled trials using the following criteria:
  1. Comparability of characteristics between intervention and control groups with respect to relevant baseline characteristics such as water quality, diarrhoeal morbidity, age, socioeconomic status, access to water, hygiene practices, and sanitation facilities. We classified this as 'adequate' if no substantial differences were present, 'unclear' if not reported or not known whether substantial differences exist, or 'inadequate' if one or more substantial difference exists.
  2. Data collection for intervention and control groups at the same time. We classified this as 'adequate' if data were collected at similar points in time, 'unclear' if the relative timing was not reported or not clear from trial, or 'inadequate' if data were not collected at similar points in time." Clasen et al. 2009, Pg 6.

• 8

  "The six trials reporting on interventions at the water source used three different effect measures. There was no difference in diarrhoea episodes when measured using rate ratios (all ages, 4 trials, Analysis 1.1.2; under fives, 3 trials, Analysis 1.2.2). The interventions were favoured in the trials that reported a risk ratio (all ages 0.45, 95% CI 0.43 to 0.47, 1 trial, Analysis 2.1.2) or a longitudinal prevalence ratio (all ages 0.56, 95% CI 0.37 to 0.84, 1 trial, Analysis 3.1.2; under fives 0.63, 95% CI 0.49 to 0.81, 1 trial, Analysis 3.2.2)." Clasen et al. 2009, Pg 11.

• 9

  "Thirty-two trials reported on household-based interventions, which included chlorination, filtration, solar disinfection, combined flocculation and disinfection, and improved storage. Overall, the household interventions significantly reduced diarrhoea episodes amongst people of all ages and in children under five years as measured with rate ratios (all ages 0.56, 95% CI 0.42 to 0.74, 6 trials, Analysis 1.1.3; under fives 0.42, 95% CI 0.19 to 0.95, 3 trials, Analysis 1.2.3), risk ratios (all ages 0.43, 95% CI 0.27 to 0.70, 6 trials, Analysis 2.1.3; under fives 0.54, 95% CI 0.43 to 0.69, 5 trials, Analysis 2.2.2), odds ratios (all ages 0.68, 95% CI 0.59 to 0.79, 9 trials, Analysis 4.1.2; under fives 0.70, 95% CI 0.50 to 0.99, 6 trials, Analysis 4.2.2), and means ratios (all ages 0.57, 95% CI 0.52 to 0.62, 1 trial, Analysis 5.1; under fives 0.75, 95% CI 0.65 to 0.86, 1 trial, Analysis 5.2). The longitudinal prevalence ratios ( Analysis 3.1.3 and Analysis 3.2.3) only reached statistical significance when a possible outlier, Doocy 2004, was excluded from the analysis for all age groups (0.70, 95% CI 0.56 to 0.88, 9 trials) and for the under fives (0.76, 95% CI 0.66 to 0.88, 9 trials). As mentioned above, caution must be taken when interpreting these results because some of the analyses use the control arms more than once (URL 1995-i; Reller 2003-i; Luby 2004a-i)." Clasen et al. 2009, Pg 11.

• 10

  "We classified blinding − whether the participant or outcome assessor is blind to the intervention group − as 'double blind' if the trial uses a placebo or double-dummy technique such that neither the participants nor the assessor knows whether or not the participants receive the intervention; 'single blind' if the participant or the assessor knows whether or not the participant receives the intervention; or 'open' if both participant and assessor know whether or not the participant receives the intervention." Clasen et al. 2009, Pg 6.

• 11

  "Only three of 19 randomized controlled trials were blinded, and in each case the intervention had no statistically significant protective effect. This must give pause to any definitive conclusion about the potential value of water quality interventions in the prevention of diarrhoea. The authors of each of these trials suggested possible explanations for their findings. Colford 2002 was the only trial conducted in a developed country setting, and the water there already complied with US standards. Kirchhoff 1985, though a pioneering trial of a potentially important household intervention, had a study population of only 112 persons (smallest of all the included trials) and was rated low on three other criteria of methodological quality. Austin 1993-i also suggested possible methodological issues and used dilute sodium hypochlorite in the control group, an approach that probably improved their water quality thus resulting in an understatement of the intervention's effectiveness." Clasen et al. 2009, Pg 15.

• 12

  "The method of diarrhoea surveillance and assessment also varied. In most cases, participants were visited on a periodic basis, either weekly (13 trials), biweekly (five trials), or more infrequently (four trials), and were asked to recall and report on cases of diarrhoea during a previous period, usually seven days (16 trials) or 14 days (six trials). The other trials asked each participant or a designated householder to keep a log or record to indicate days with or without diarrhoea (Austin 1993-i; Colford 2002; du Preez 2004), procured data on diarrhoea from family records and disease registries (Mahfouz 1995), or used paediatricians to assess the participants during regular medical checkups (Gasana 2002). Only one trial did not report the method (Xiao 1997)." Clasen et al. 2009, Pg 9.

• 13

  "Finally, it appears that many if not most of the trials were undertaken primarily for the purpose of investigating the effectiveness of the intervention, and not as an assessment of an ongoing programme. This seems particularly true of the household interventions. While investigators often took special steps to minimize the effect that such a research focus may have had on the study results, the continuous onsite participation of investigators, many of whom were foreign to the study settings, in implementing and assessing the intervention could be a source of possible bias. It may also raise questions about whether the results obtained would be representative of the effectiveness of the interventions outside a research context. Future trials should include assessments of ongoing programmes implemented outside a research context." Clasen et al. 2009, Pg 16.

• 14
  • "Many of the trials reported results on selected age groups, and not on all ages of persons who would have been affected by the intervention. It is common for research on diarrhoeal disease to specifically target children less than five years of age, a group that is particularly vulnerable to diarrhoea, and many of the trials did provide data for this group. Many others, however, reported results only for a subset of this group, or for some other segment of the population. In most cases, it appears that results were reported for the full population on which data were collected. It is possible, that by omitting a portion of the population affected by the intervention, the design or reporting of results is a source of bias." Clasen et al. 2009, Pg 16.
  • "Figure 1 presents a funnel plot of the estimate of effect of the trials and the standard error of such estimate of effect (reflecting the study size). The asymmetrical shape of such funnel plot is suggestive of publication bias. It is noted, however, that funnel plot asymmetry may also be due to clinical and methodological heterogeneity. Since we found substantial evidence of such heterogeneity, we cannot conclude that the funnel plot demonstrates evidence of publication bias in this case." Clasen et al. 2009, Pg 13.

• 15

  Clasen et al. 2009, Pg 14.

• 16

  "The NGO planned for the water quality improvement intervention to be phased in over four years due to their financial and administrative constraints. Although all springs were eventually protected, for our analysis the springs protected in round 1 (January-April 2005) and round 2 (August-November 2005) are called the treatment springs and those that were protected later are the comparison group...A representative sample of households that regularly used each sample spring was selected at baseline...Water quality was measured at all sample springs and households using protocols based on those used at the U.S. Environmental Protection Agency. The water quality measure we use is contamination with E. coli, an indicator bacteria that is correlated with the presence of fecal matter. The household survey gathered baseline information about child diarrhea and anthropometrics, mothers' hygiene knowledge and behaviors (hand washing), household water collection and treatment behavior, and socioeconomic status." Kremer et al. 2009, Pgs 6-7.

• 17

  "Spring protection dramatically reduces fecal contamination of source water. The average reduction in In E. coli across all four rounds of data is -1.07, corresponding to a 66% reduction." Kremer et al. 2009, Pg 10.

• 18

  "Spring protection leads to statistically significant reductions in diarrhea for children under age 3 at baseline or born since the baseline survey. In the simplest specification taking advantage of the experimental design, diarrhea incidence falls by -4.5 percentage points...On a comparison group average of 19% of children with diarrhea in the past week, this is a drop of one quarter. We conclude that the moderate reductions in household water contamination caused by spring protection were sufficient to significantly reduce diarrhea incidence." Kremer et al. 2009, Pgs 12-13.

• 19

  Kremer and Zwane 2006, Pg 17.

• 20

  Peter Harvey, UNICEF Zambia, Rural Water Supply Network, "Handpump Data."

• 21

  Jamison et al. 2006, Pg 72, Table 2.B.2.

• 22
  • We took data on 2004 (a) incidence (total cases) from World Health Organization, "Disease and Injury Regional Estimates for 2004: Incidence for WHO Regions"; (b) deaths from World Health Organization, "Disease and Injury Regional Estimates for 2004: Deaths for WHO Regions"; (c) DALYs from World Health Organization, "Disease and Injury Regional Estimates for 2004: DALYs for WHO Regions" (using 3% discounting and no age weights, as the Disease Control Priorities in Developing Countries report does - see Jamison et al. 2006, Pg 29).
  • There were a total of ~67 cases per DALY, and ~2136 cases per death.
  • $159 per DALY averted thus implies $2.37 (159/67) per case averted. Assuming one would have to avert 2,136 cases to avert a single death implies a cost of ~$5,000 per death averted.

• 23

  "Using the household time values derived from our surveys, the bound on the value of ... avoiding a child diarrhea death is $769 ... Using a standard conversion from diarrhea to disability-adjusted life-years (DALYs), this corresponds to an upper bound on the value of averting one DALY of about $23.68 ... For comparison, we estimate that the cost per DALY averted for this intervention is $16.75." Kremer et al. 2009, Pg 21. In order to "replicate" the unspecified DALY-to-death conversion used by the authors, we applied the ratio between the estimated cost per DALY and the "value of averting one DALY" ($16.75 / $23.68) to the "value of avoiding a child diarrhea death" ($769). Also see the discussion of this study on our blog.