Evidence Action's Dispensers for Safe Water program – General Support (January 2022)

Note: This page summarizes the rationale behind a GiveWell grant to Evidence Action's Dispensers for Safe Water program. Evidence Action staff reviewed this page prior to publication. The page reflects our rationale at the time the grant was recommended.

Published: October 2022

The organization

Dispensers for Safe Water is run by Evidence Action, an organization with which we have a long-standing relationship and to which we have provided significant support.1 Dispensers for Safe Water started operations in Uganda and Kenya in 2012 and in Malawi in 2013. They expanded their operations until early in 2016. Since then, they have focused on maintaining existing dispensers and increasing adoption rates among current beneficiaries.2 In 2019, Dispensers for Safe Water maintained roughly 18,000 dispensers in Kenya, 5,300 dispensers in Uganda, and 3,700 dispensers in Malawi. Using Dispensers for Safe Water data, we estimate that 2.3 million people (including 300,000 children) drank water chlorinated with Dispensers for Safe Water's dispensers in 2019.3

The intervention

Dispensers to chlorinate water

Chlorine dispensers are tanks containing liquid chlorine. Chlorine is dispensed by turning a valve, which is calibrated to dispense a certain amount of chlorine per turn.4 Dispensers are installed near communal water points in rural areas. Users of a given water point are advised to dispense chlorine in the water container before fetching water, so that the water and chlorine mix automatically once the water is poured in the container.5 In addition to installing dispensers, Dispensers for Safe Water staff train community members on the importance of chlorination and how to use the dispensers. Each water point with a chlorine dispenser has two community volunteers, known as promoters, who monitor dispenser use and troubleshoot in case there are issues with the technology or take-up in the community.6

Dispensers for Safe Water activities

Dispensers for Safe Water activities include:

• Site and water point selection
• Installation, refilling, and troubleshooting of dispensers
• Facilitation of community sensitization and education meetings
• Training for Dispensers for Safe Water promoters

For full details, see here.

Do dispensers work?

Our full analysis of the benefits of water chlorination on mortality can be found in our water quality report. In brief, we estimate that the benefits of water chlorination derive from three primary sources:

• Reducing mortality: approximately 50% of benefits
• Developmental effects: approximately 25% of benefits
• Medical costs averted: approximately 25% of benefits

Additional details are linked in the following footnote.7

The grant

We recommended a $64.7 million grant from Open Philanthropy to Evidence Action's Dispensers for Safe Water. We recommended a portion of the grant, $48.8 million, to be disbursed unconditionally, to support Dispensers for Safe Water's scale up and maintenance. We recommended the remainder of this grant, $15.9 million, to be disbursed as an exit grant. The exit grant will be released if we decide to end our support of the program and that leads to Evidence Action deciding to discontinue Dispensers for Safe Water's operations. The purpose of the exit grant is to mitigate the potential negative effects of a rapid exit from the program.

This grant covers:

• Extension of Dispensers for Safe Water's current footprint in Kenya, Malawi, and Uganda: The grant covers maintenance of the current footprint for four years in Malawi and Uganda and three years in Kenya ($19 million), and exit grants for Malawi and Uganda ($5.9 million), for a total of $24.9 million.8

  Maintenance activities include chlorine supply provision, repairs and replacements for dispensers, and community engagement.9 See above for more details on Dispensers for Safe Water activities. The $19 million would allow Dispensers for Safe Water to maintain roughly 18,000 dispensers in Kenya, 5,300 in Uganda, and 3,700 in Malawi; adoption rates indicate that an estimated 2.3 million people in the areas served by Dispensers for Safe Water treat their water with chlorine from Dispensers for Safe Water dispensers.10

  We are recommending an exit grant to be released in the case that GiveWell terminates its support of Dispensers for Safe Water, conditional on Evidence Action deciding to discontinue Dispensers for Safe Water operations. Details of the activities an exit grant would support are in the following footnote.11
• Expansion of Dispensers for Safe Water's footprint in Malawi and Uganda: This grant covers scale-up costs ($8.6 million), four years of maintenance ($20.4 million), and an exit grant ($10 million) for three new offices in Malawi and three in Uganda, for a total of $39 million.12

  Scale-up costs include the costs of set-up activities to be performed before installation (e.g., establishing field offices and staff recruitment, securing local government permission, identifying suitable water points through water point verification, and conducting baseline surveys) as well as installation itself (including initial community engagement and physical installation).13 Dispensers for Safe Water estimates that scale-up would last roughly 15 months.

  We are recommending expansion that involves creating three new offices in Malawi and three in Uganda. Each new office will install and maintain 4,000 dispensers. We estimate this will result in roughly 1.9 million additional people in Uganda and 1.4 million additional people in Malawi chlorinating their water using Dispensers for Safe Water's dispensers.14

• Scoping grant: This grant covers two years of costs associated with scoping new countries for potential introduction of Dispensers for Safe Water and in-line chlorination, for a total of $0.8 million.15 In-line chlorination is described in our water quality intervention report here. Evidence Action is currently piloting in-line chlorination in Kenya.16 We are considering recommending a separate grant to Evidence Action to scale in-line chlorination.

Budget for grant activities and expected revenue

Evidence Action has proposed a total budget of $77,416,486 for the activities described above. We expect that they will raise revenue from other sources totaling $12,693,106, so GiveWell is recommending a total grant for the difference, $64,723,380. See here for a full breakdown of Dispensers for Safe Water's proposed budget by country and cost type. See here for a breakdown of the revenue we expect Dispensers for Safe Water will raise from 2021 to 2025.

Grant duration

The proposed grant duration for Malawi and Uganda (five-and-a-half to seven years) is longer than is usual for grants we recommend, both because it covers four years of maintenance, rather than the usual three, and because it includes additional support for scale-up (new geographies) and exit (all geographies).

The main reasons we believe this to be appropriate in Dispensers for Safe Water's case are:

• The costs to scale up and scale down Dispensers for Safe Water infrastructure. Our impression is that entering communities involves substantial infrastructure investment and community engagement, and exiting requires both community notification as well as infrastructure drawdown.17
• Absorbing risks faced by Dispensers for Safe Water. Our understanding is that expanding Dispensers for Safe Water would pose risks for Evidence Action. It is difficult to fundraise for Dispensers for Safe Water (see below for details). Scaling up Dispensers for Safe Water as proposed here would imply that the Dispensers for Safe Water program would be roughly one-third of Evidence Action's total organizational revenue. Accordingly, if we stopped directing funding to Dispensers for Safe Water, it would be unlikely to fill that gap. Such a funding cliff could have a destabilizing effect on Evidence Action as an organization overall. It seems appropriate to absorb some of that risk by committing to a longer time period for funding and ensuring that exit funding is available.
• Enabling expansion of other Evidence Action programs. Evidence Action has said that in addition to setup and exit costs, it wants to ensure that Dispensers for Safe Water has a minimum of four years of maintenance funding (the amount of time that would justify the scale up and down costs described in the first bullet of this section).

  If we provided fewer than four years of maintenance funding, Evidence Action would fundraise for Dispensers for Safe Water to ensure its maintenance for four years. Evidence Action has said that this would reduce its ability to expand other, non-Dispensers for Safe Water programs, as its financial and organizational capacity would be stretched thinner as more resources were directed toward fundraising.

  We're considering funding other, non-Dispensers for Safe Water programs at Evidence Action, so a reduction in Evidence Action's appetite for expanding those programs could limit our ability to support potentially cost-effective programs.

The case for the grant

Cost-effectiveness

A note on how we discuss cost-effectiveness on this page

We use GiveDirectly's unconditional cash transfers as a benchmark for comparing the cost-effectiveness of different funding gaps, which we describe in multiples of "cash." Thus, if we estimate that a funding gap is "10x cash," this means we estimate it to be ten times as cost-effective as unconditional cash transfers. At the time we made this grant, we typically funded opportunities that met or exceeded a relatively high bar: 8x cash. We also considered funding opportunities that were between 5x and 8x cash.

Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes to other grants we have made or considered making, and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

Cost-effectiveness by country

We estimate that the cost-effectiveness of this program across all three countries is roughly 6x to 7x cash. We estimate that the cost-effectiveness of the program is 3.3x to 4.4x cash in Kenya, 5.4x to 7.0x cash in Uganda, and 6.4x to 8.6x cash in Malawi.18

Note: lower cost-effectiveness in Kenya

We still recommend $8 million (12% of the total grant) to support three years of maintenance for the current program in Kenya. The Kenya program hosts a number of activities that benefit Dispensers for Safe Water's global work and are needed to support global expansion for both Dispensers for Safe Water and in-line chlorination.20

Moreover, we may recommend funding for an in-line chlorination program with Evidence Action later this year. In-line chlorination may be more cost-effective than Dispensers for Safe Water in Kenya21 , and will likely substantially leverage Dispensers for Safe Water infrastructure, increasing the cost-effectiveness of the Dispensers for Safe Water Kenya program via shared costs. Furthermore, the Dispensers for Safe Water infrastructure will facilitate a more rapid expansion of in-line chlorination outside of Kenya.

For more information on how we estimated cost effectiveness, see our water quality intervention report.

There are two reasons we think we might be underestimating Dispensers for Safe Water's cost-effectiveness:

Cost sharing with in-line chlorination and/or in-line chlorination leveraging Dispensers for Safe Water's platform. Dispensers for Safe Water estimates that cost-sharing with in-line chlorination might lead to a decrease in costs by 6 to 10 percent.22 Beyond that, it's possible we should think of in-line chlorination and Dispensers for Safe Water as a single "water program," and that this program would be more cost-effective than Dispensers for Safe Water alone due to shared infrastructure and potentially higher average cost-effectiveness of in-line chlorination compared to Dispensers for Safe Water.
• Further leveraging of Dispensers for Safe Water platform. Dispensers for Safe Water maintains a platform of promoters (volunteers who refill dispensers and promote their use). It's possible this platform could be leveraged to distribute other goods or services, such as eyeglasses or safe birth kits.

Funding gap

We believe that there is a funding gap for this work. Dispensers for Safe Water has struggled to fundraise in the last few years and expects that some of its current funders will reduce or discontinue their support. Dispensers for Safe Water halted expansion in 2016, and another funder has not stepped in to support scale-up.23 Because of this, our understanding is that Dispensers for Safe Water falls outside the sphere of interest of most large funders, who tend to focus on water infrastructure rather than treatment.24

Room for more funding

We speculatively estimate Dispensers for Safe Water has a total room for more funding of approximately $170 million per year above 8x cash.

Ability to measure and evaluate impact

We expect to receive high-quality monitoring and evaluation data as part of this grant. We expect this to improve the precision of our cost-effectiveness estimate of Dispensers for Safe Water in the future.

Monitoring and evaluation data overview

In brief, Dispensers for Safe Water plans to collect data on:

• Average number of dispensers maintained a year.
• Average number of people and average number of under-five children per household among those using a water point with a dispenser.
• Average chlorination rates among people using a water point with a dispenser.
• Average number of households per water source with dispensers.25
• Baseline water treatment rates and baseline diarrhea burden (in expansion areas only).

While Dispensers for Safe Water already collects strong monitoring and evaluation data, the plan for this grant improves on current practices in the following ways:

• More precise and representative data of baseline water treatment and under-five diarrhea. Current baseline data is based on a limited, non-randomly selected sample. It is also somewhat dated, which might make it unrepresentative of current water treatment rates.26 Expansion of the program enables Dispensers for Safe Water to obtain a current-day baseline in the expansion areas that is likely more representative of water treatment and under-five diarrhea rates today than the ones we currently use. During the grant period, Dispensers for Safe Water will use a randomly selected sample, large enough to provide district-level estimates with a margin of error of approximately 5% and 95% confidence level.27 Data on baseline treatment rates will be especially valuable because our model is very sensitive to this input.
• More precise and representative estimates of the number of people per household. Current estimates aggregate data from 2017 to 2019, and the estimates are not precise enough to detect significant differences at the district level.28 During the grant period, Dispensers for Safe Water will use a sample large enough to provide district-level yearly estimates with a margin of error of approximately 5%, and a 90% confidence level.29 The number of people per household is a crucial input in our costing analysis, so this will help us make progress on one of our major uncertainties (see below).
• Accurate estimates of households per water point. Dispensers for Safe Water collects survey data on households per water point. However, they lack confidence that the questionnaire meaningfully distinguishes between households per water point and households per dispenser.30 We therefore use the average households per dispenser as a proxy.31 Dispensers for Safe Water plans to collect higher quality data on the average households per water point going forward.32 This will allow us to use a more direct estimate of the input we are ultimately interested in.

We currently lack data on medical costs and baseline consumption for the population served by Dispensers for Safe Water. We asked Dispensers for Safe Water if it's feasible to add questions on these inputs in their baseline surveys: they are open to doing so in principle, but have suggested it might be best to use existing data.33 We plan to discuss this with Dispensers for Safe Water after the grant is recommended—although it would be helpful to collect this data, we do not consider it essential to our future estimates, since we might be able to make some progress through further desk research.34

Risks and reservations

Limitations in evidence included in our cost-effectiveness estimate

We have a number of uncertainties on the evidence informing our cost-effectiveness estimate, which are discussed in our water quality report.

Longer grant time period

This grant covers a longer timeline than we typically recommend. We discuss why we think this is appropriate here, but we don't have generalizable principles guiding exit grants and longer funding timelines for other grantees, so we remain somewhat uncertain about the proposal.

Case for the Kenya grant

Due to lower expected cost-effectiveness, we are recommending funding a shorter timeline for the existing footprint in Kenya (three years of maintenance, no exit grant). It's also possible that we should fund only part of the current footprint to further reduce spending in the country, or not fund the Kenya Dispensers for Safe Water program at all. The main reason we are recommending the grant is that it's possible that the Kenya program would be cost-effective when coupled with in-line chlorination, and because the Kenya office hosts a number of activities that benefit Dispensers for Safe Water's work across all three countries and that are needed to support further global expansion of Dispensers for Safe Water and in-line chlorination.

Plans for follow-up

We plan to ask Dispensers for Safe Water to regularly report on a number of indicators on the program's success.35

Internal forecasts

By April 2023, we think there is a 60% chance that our best guess cost-effectiveness across all countries funded under this grant (including Kenya) will be equal to or greater than 6x to 8x cash. We think there's a 70% chance that we will have identified one or more countries (outside of those funded under this grant) where we expect cost-effectiveness for Dispensers for Safe Water would be greater than 6 to 8x, resulting in total room for more funding at scale of more than $30 million per year by December 2022, and a 70% chance we will have identified more than $60 million per year by December 2023.

Our process

• We visited Dispensers for Safe Water in 2019. During that visit, we had extensive conversations with Dispensers for Safe Water's team about the program and M&E practices.36
• We engaged with Dispensers for Safe Water during our grant investigation. We sent Dispensers for Safe Water a list of questions and data requests. Dispensers for Safe Water shared the data we required, and we had conversations to discuss the program, monitoring and evaluation, room for more funding, and grant size.
• We spoke with local stakeholders from Malawi, Uganda, and Kenya and with representatives from UNICEF Uganda and Malawi. We also briefly reviewed strategies of WASH global funders to understand the risk of funging.
• We updated our cost-effectiveness model for water quality interventions. See here for the cost-effectiveness model and here for the report. The model received sign-off from four internal reviewers and the water report received sign off from five internal reviewers.
• We commissioned two external expert reviews of Kremer et al. (a statistical review and an epidemiological review). As a result, we decided not to use the result reported in the meta-analysis in our model, but rather to exclude (a) trials in which the intervention did not involve chlorination and (b) trials with follow-up of less than one year.
• GiveWell Content Editors vetted the cost-effectiveness model, as well as estimates of costs, room for more funding, and grant size.
• Program Officers reviewed the cost-effectiveness model and signed off on the grant.

Sources

• 1Evidence Action's Deworm the World Initiative has been one of GiveWell's top charities from 2013 to 2022. We have also supported the Evidence Action Accelerator, an incubator portfolio within Evidence Action focused on GiveWell-aligned, evidence-backed, and cost-effective interventions.
  
• 2

  GiveWell, Dispensers for Safe Water Organization basics, based on field notes, 2022

• 3See here for calculations.
  
• 4"When the dispenser valve is turned one full revolution, the valve dispenses 3 ml of chlorine, which is the quantity of chlorine needed to treat 20L of water." GiveWell, Dispensers for Safe Water Organization basics, based on field notes, 2022, p. 2.

• 5
  • "Dispensers for Safe Water obtains from government officials a list of water sources in the sub-counties it plans to operate in. All listed water points in a sub-county are visited and verified against a set criteria for dispenser eligibility. Verification includes water points found on the ground, even if they were not provided by the government administrative chains (Department of water, assistant chiefs, village elders, CHVs). A water point is selected to host a chlorine dispenser if it meets a list of criteria, including requirements pertaining # HHs using the water point, and whether the water is used for drinking." GiveWell, Dispensers for Safe Water Organization basics, based on field notes, 2022, p. 2.
  • "Training includes the following guidelines . . . Users should first wash the jerrican, then dispense the chlorine, then fetch the water. By dispensing the chlorine before fetching water, the water and chlorine mix automatically once the water is poured in the container." GiveWell, Dispensers for Safe Water Organization basics, based on field notes, 2022, p. 2.

• 6"The community votes to elect two promoters…Promoters are volunteers who refill the chlorine dispensers and promote the use of the dispensers (including reaching out to community members who did not participate in the community sensitization meeting and attending local meetings to sensitize communities about the use of chlorine). Promoters are trained during the community education meeting, after their election."Promoters are also asked to report faults (they are provided with a phone number they can use to report problems). Promoters and staff delivering chlorine are trained to address basic functionality problems; engineers are mobilised for more serious faults." GiveWell, Dispensers for Safe Water Organization basics, based on field notes, 2022, p. 2.
  
• 7

  For each of these inputs, see the relevant section of our model as well as our intervention report:

  • Reducing mortality: model, intervention report
  • Developmental effects: model, intervention report
  • Medical costs averted: model, intervention report

• 8

  See here for calculations. Here and below, we report the grant amount rather than budgeted costs. Grant amounts are lower than budgeted costs, since they account for Dispensers for Safe Water expected revenue. We assume the revenue is split equally among different buckets. Note that this is a simplification, since in the absence of the grant Dispensers for Safe Water would use all its revenue to cover costs for the current footprint.
  

• 9
  • "Maintenance phase; continued community engagement, chlorine supply, repairs and replacements" Dispensers for Safe Water, Scale-up model & Budget: Assumptions, Risks & Milestones, September 2021.
  • See below for details on the budget.

• 10See here for details.

• 11
  • "Months 25-35: Government, community, and partner notification and exit planning: We would undertake phased engagements with the national government, local governments, and with each community. The engagement with local governments and communities would involve regular quarterly meetings to inform of, plan for, and implement the transition. We'd seek to give communities sufficient notice to undertake responsible wind-down before beginning dispenser removals, allowing them to consider alternative options for safe water and to allow for engagement with partners on alternative approaches. We would not engage communities on closure discussions prior to a firm decision on shutdown- an inconsistent message to communities would erode the government and community-level trust which is so important to high adoption rates and program execution. For communities that could transition to another funding source,this would jeopardize the ability to continue operating effectively." Dispensers for Safe Water, email to GiveWell, November 16, 2021 (unpublished)
  • "Months 36 through closure: Physical exit, contract terminations, and staff layoffs: In this phase we will be focused on ensuring that every community is responsibly exited and that the physical transition is complete, from asset distribution to staff release. We will wind down contracts and engagement with local manufacturers of dispenser parts to ensure a smooth and cost-effective transition. We will undertake final government and community-level meetings. If no other community, NGO or government plan is in place for dispenser maintenance, we'd enact activities to remove, relocate and/or recycle dispensers We'd also use this funding to cover mandatory severance in Uganda and Malawi, which can only be provided following employee exit. We anticipate this period will last 9-10 months, though the monthly budget expenditures will be winding down during this period." Dispensers for Safe Water, email to GiveWell, November 16, 2021 (unpublished)
  • See this section of this grant page for details on the budget.

• 12See here for information about how the grant funding will be allocated.
  
• 13
  • "Milestones:
    • Day 1: Funding decision
    • 1-6 months: Establishment of required field office(s)
    • 1-6 months: Secure relevant permissions from local government
    • 1-6 months: Identification of DSW suitable waterpoints in new geographical areas
    • 1-6 months: Recruitment of majority of leadership team for DSW expansion
    • 1-6 months: Recruitment of scale-up teams (Lead Officers and Masons)
    • 6-9 months: Recruitment of on-going maintenance teams
    • 6-15 months: Dispenser installations, community engagement, supply chain set-up
    • 15 months: Discontinuation of scale-up teams" Dispensers for Safe Water, Scale-up model & Budget: Assumptions, Risks & Milestones, September 2021.
  • Information on water point verification and conducting baseline surveys from Dispensers for Safe Water, email to GiveWell, June 15, 2022 (unpublished)

• 14A larger number of people would have access to dispensers (and some would choose not to use them). See here for details.
  
• 15
  • "For the expansion budget, we're still discussing, but to make sure we're on the right track: we'd be lookingat ~3 full time staff focused on new geographies for ILC [in-line chlorination] and DSW for ~2 years. It would include travel;government engagement to ensure interest; initial water point surveys; and operational scoping to ensure feasibility of us working there. This wouldn't include the initiation of scaling activities, like country registration;setting up finances; hiring full time staff; HH surveys or baselines; signing MOUs, etc." Dispensers for Safe Water, email to GiveWell, November 16, 2021 (unpublished).
  • See below for details on the budget.

• 16

  "We are currently designing and testing an in-line chlorination intervention to reach populations in urban and peri-urban areas in western Kenya, leveraging our Dispensers for Safe Water network that serves nearby rural communities." Evidence Action, "Evidence Action Accelerator"

• 17
  • "When entering communities, DSW (i) coordinates with local government and community leaders to select water points that meet internal criteria, (ii) seeks government permission to enter new areas, (iii) sets up new supply chains and manufacturing capacity, (iv) hires and trains new staff, (v) conducts monitoring and evaluation surveys, (vi) procures and physically installs dispensers, and (vii) runs community sensitization meetings and selects promoters (volunteers who refill dispensers)." Dispensers for Safe Water, email to GiveWell, June 15, 2022 (unpublished)
  • See above for details.
  • When exiting communities, Dispensers for Safe Water has to physically uninstall dispensers. Dispensers for Safe Water also has a strong preference for providing communities with one year of notice, to allow them to find alternative financing for dispensers or alternative sources of clean water. See the footnote in this section for details.

• 18
  • The above estimates refer to long-term cost-effectiveness (including 10 years of maintenance and 1.5 years of exit for all areas, and 1.25 years of scale-up for new areas). In the shorter term (i.e., in the timeline supported by this grant), we estimate the cost-effectiveness will be lower for Malawi and Uganda, since start-up and exit costs will be amortized over a shorter period. We estimate the cost-effectiveness in the shorter term will be higher in Kenya because the current grant does not include exit costs. Accordingly, we expect the cost-effectiveness over the course of the seven-year grant to be 3.4x to 4.7x in Kenya, 4.8x to 6.3x in Uganda and 5.8x to 7.8x in Malawi.
  • We provide a range, rather than a point estimate, due to internal disagreement on how to account for benefits deriving from medical costs averted. The upper bound assumes our best guess, that medical costs averted make up approximately 25% of total benefits. The lower bound assumes medical costs averted make up 6% of total benefits, and while this is not our current best guess, it allows for comparability with top charities. This is important, because our cost-effectiveness estimates for these charities strongly inform our funding bar.

• 19
  • We estimate that the program results in a 5.6% reduction in under-five mortality in Kenya, 6.1% in Uganda, and 10.9% in Malawi. See here in GiveWell, Water Quality CEA (ILC and DSW), 2022, "DSW short term" tab, "Percent reduction in under-5 all-cause mortality, final estimate" row.
  • We also estimate that the program results in a 1% reduction in over-five mortality in Kenya, 3% in Uganda, and 4% in Malawi. See here in GiveWell, Water Quality CEA (ILC and DSW), 2022, "DSW short term" tab, "Implied reduction in over-5 all-cause mortality, for illustration only" row.
  • We estimate that each dispenser in Kenya serves approximately 119 people, each dispenser in Uganda serves approximately 260 people, and each dispenser in Malawi serves approximately 181 people. See here. The cost per person in Kenya ($1.68) is roughly similar to Malawi's ($1.86), but significantly higher than Uganda's ($1.22).

• 20"The Kenya program is an important hub for DSW's programs across the region.
  • DSW's regional office is in Nairobi, so that's where much of its back office support functions are located, including finance, HR, operations, procurement, and M&E.
  • Dispenser production, warehousing, and distribution also run through Kenya right now, though DSW aims to set up these functions in Malawi and Uganda as well.
  • DSW's engineering staff are based in Nairobi.
  • DSW often conducts program innovation testing within its Kenya program, so it's important for research. ILC [in-line chlorination] is a good example.
  • DSW believes that moving these functions out of Kenya would be costly and time-consuming and that recruitment would likely be more difficult in Uganda or Malawi.

  DSW expects that Kenya would still serve as an important hub office, even if it were to expand its program outside of Africa.

  • The majority of DSW's M&E staff and staff that could do scoping work are based in Kenya.
  • The Kenya program has provided mentorship for Malawi and Uganda, and would probably do so for programs outside of Africa as well.
  • DSW would work closely with its offices in other potential expansion countries and may set up hub offices in other regions, such as West Africa or Southeast Asia, but these offices would probably remain reliant on leadership and mentorship from Kenya."

  Dispensers for Safe Water, call with GiveWell, October 7, 2021 (unpublished)

• 21For example, we estimate the cost-effectiveness of Dispensers for Safe Water in Kenya (before adjustments for leverage or funging) at 5.3x cash, and we estimate the cost-effectiveness of in-line chlorination in Kenya before adjustments at 7.6x cash. See GiveWell, Water Quality CEA (ILC and DSW), 2022, "DSW_short term" and "ILC Kenya" tabs.
  
• 22
  • "% share of DSW/ILC cost sharing for existing DSW costs (best guess of potential DSW/ILC cost savings)Evidence Action: Given the fact that ILC is still designing their implementation approach and budget, we cannot determine a precise division of costs that could be shared by DSW & ILC. However, we know that there will be efficiencies between the programs, and a good idea of where those efficiencies will be found. For the 3 year funding scenario, our rough estimate is that DSW costs would be 4.5-6.5% lower; and 6-10% lower for the 3 year plus exit grant. Also note that it may take some time (a year or two) to achieve our most efficient state, given the newness of ILC's scaling efforts." Dispensers for Safe Water, email to GiveWell, October 18, 2021 (unpublished).
  • "DSW can provide estimates of the costs that it will share with ILC, but they are unable to estimate ILC-only costs…DSW doesn't think that it's necessarily appropriate to assume that ILC-only costs will be roughly similar to DSW-only costs." Dispensers for Safe Water, call with GiveWell, October 7, 2021 (unpublished).

• 23GiveWell, Dispensers for Safe Water Organization basics, based on field notes, 2022, p. 2.
  
• 24“GiveWell: My understanding is that DSW stopped expansion in 2016 due to lack of funding. Is this right? Evidence Action: That's correct.” Dispensers for Safe Water, email to GiveWell, December 3, 2021 (unpublished)
  
• 25"GiveWell: In the last email, you mentioned you'd like to discuss what major factors we'd like you to be considering as we move forward. Broadly, the key MLE statistics we are interested in are:
  
  1. HH per water source
  2. People per HHs using water source with dispensers
  3. U5 per HHs using water source
  4. % of HHs whose water sample tests positive for chlorine
  5. % of HH whose water sample was positive for chlorine who report using dispensers

  Because we use district-level mortality in the CEA, ideally we'd want to be able to estimate these, with a reasonable level of precision, at the district level. However, I don't have a good sense of feasibility/costs -and we'd definitely want to find alternatives if the above would require a lot of staff capacity, or be very costly. It would be great to have your thoughts on this."
  "Evidence Action: 1. HH per water source--This is collected using program data and it's a census of all water points, so you will get the data for this without an issue. For indicators 2-5 below, we collect this data from a monitoring sample. For this type of data available at district level with 'reasonable level of precision', we suggest collecting data from 41 water points per district per monitoring period. This would provide estimates at a district level with ~10% MoE and 80% CL each monitoring period. If we combine the data collected over the year to create a yearly average, we'd get district-level estimates with ~5% MoE and 90% CL. The cost for this would be included in the revised MLE budget (shown above)." Evidence Action, email to GiveWell, November 1, 2021 (unpublished)
  

• 26
  • "We collected baseline data for 19 out of 25 installation rounds in Kenya. . . . All results below and in the baseline analysis workbook are for 2014-2015 data only, and comes from a total of 143 water points and 1,067 households from 15 installation rounds."
  • "Uganda dispensers were primarily installed between 2012 and 2016 in 8 installation rounds across 11 districts. . . . We only collected baseline data for two installation rounds, Manafwa and Tororo."
  • "Malawi dispensers were primarily installed between 2013 and 2016 in one district, Zomba. Dispensers were installed in nine installation rounds. . . . We only collected baseline data for one installation round, ZMB2, located in the Likangala cluster in 2014."
  • "After discussion with the Malawi program team, we understand the ZMB2 area was/is known to be a cholera hotspot, relative to other areas in the district, but is generally a good representation of our catchment areas in Zomba."
  • Dispensers for Safe Water, Baseline Data Report, August 27, 2021

• 27

  "If we were to adopt a baseline survey that could produce estimates with roughly 5% MoE at the district-level, it would cost an additional $103.9k per office in Uganda, and $87.3k in Malawi." Evidence Action, email to GiveWell, November 1, 2021 (unpublished)
  

• 28

  "In order to standardize estimates and reduce standard errors, we use community monitoring survey data for the last 3 years to calculate the average number of people and children under 5 per household. We don't expect to see significant change in the HH size during that period. Prior to 2019, we were using data from inception, but restricted the data to the last 3 years in 2019." Dispensers for Safe Water, 2019 monitoring data analysis, shared with GiveWell in July and Oct 2021 (public)

• 29

  "To detail further, we suggest collecting data from 41 water points per district per monitoring period. This would provide estimates at a district level with approximately 10% MoE and 80% CL each monitoring period. If we combine the data collected over the year to create a yearly average, we'd get district-level estimates with approximately 5% MoE and 90% CL." Evidence Action, email to GiveWell, November 1, 2021 (unpublished)
  

• 30
  • Dispensers for Safe Water does collect data on households per water point during MLIS surveys (during which it also collects chlorination rates, which we use in our estimate of benefits). However, it is not confident that the current questionnaire meaningfully distinguishes between households per water point and households per dispenser. Since the average households per dispenser is estimated on the basis of a larger census, collected during dispensers spot checks (which cover all dispensers 2-4 times a year), Dispensers for Safe Water prefers using it for the purpose of the current analysis.
  • "The MLIS surveys are conducted at a sample of dispensers each 2-month monitoring period, and cover ~9% of all dispensers throughout the year (pre-COVID). During MLIS surveying, we ask promoters how many households use the water point (where the dispenser is installed) then ask them to list the households so the MLE Field Officer can randomly select 8 households for the MLIS Community survey using the "Infield Randomization" form." Dispensers for Safe Water, email to GiveWell, November 23, 2021 (unpublished)
  • "Unfortunately, we haven't necessarily designed the surveys to look at the difference between those numbers. . . . We also have questions about how we've been asking the questions about HH per waterpoint. . . .Given our confidence in the HH per dispenser numbers, we'd prefer to use those for now, though moving forward, we'd like to discuss how to shift to HH per waterpoint, and would like to have that discussion with you in the beginning of the year as we prepare for the launch of the expansion." Dispensers for Safe Water, email to GiveWell, November 24, 2021 (unpublished)

• 31
  • We would guess the number of households per water point is larger than the number of households per dispenser, since not all households who use the water point use the dispenser. However, Dispensers for Safe Water reported not observing a large difference between HH per water point and HH per dispenser.
  • "Unfortunately, we haven't necessarily designed the surveys to look at the difference between those numbers, and we don't actually see large and consistent differences between the two. We also have questions about how we've been asking the questions about HH per waterpoint." Dispensers for Safe Water, email to GiveWell, November 24, 2021 (unpublished)

• 32"Given our confidence in the HH per dispenser numbers, we'd prefer to use those for now, though moving forward, we'd like to discuss how to shift to HH per waterpoint, and would like to have that discussion with you in the beginning of the year as we prepare for the launch of the expansion." Dispensers for Safe Water, email to GiveWell, November 24, 2021 (unpublished)
  
• 33
  • question from GiveWell: "Collecting consumption and medical costs data at baseline: the current version of our CEA model implies roughly one quarter of benefits come from averting medical costs. To estimate those, we need to estimate baseline consumption and the magnitude of the medical costs averted. At the moment, we do so by using respectively WorldBank national data and literature from LIC/LMIC. However, we'd prefer to use data from DSW's program participants.Would it be possible to add questions on baseline consumption and medical costs to the baseline survey?"
  • Evidence Action response: "We're very open to more discussion around how to add some of these questions to surveys.Overall, we think there are quite a few nuances and complications to getting to a total cost of illness averted that could go beyond the scope of what we could collect through our surveys. There are some high quality studies out there that have been designed to capture these types of costs in very similar communities to which we work. We're open to having more conversations on what the limitations of those studies are or what data points in particular GiveWell is looking for further validation on so that we can best hone in on the most useful data points to help validate the existing evidence base. We'd love to set up a few conversations early in the year as we finalize our surveys to make sure we're all aligned on questions." Dispensers for Safe Water, email to GiveWell, December 3, 2021 (unpublished).

• 34"Health expenditures. We have general estimates for the cost of medical treatment for diarrhea in low- and lower-middle-income countries, but we do not have direct estimates for the specific populations benefited by the interventions we are evaluating. Further desk research may allow us to refine these estimates somewhat, but direct estimates from implementing organizations would be ideal." GiveWell, "Water Quality Interventions," 2022.
  
• 35
  • Budget spent, yearly
  • Baseline surveys
    • % of households who report treating water
    • % of households who report chlorinating water
    • % of households whose water sample tests positive for chlorine
    • Under 5 diarrhea rates
    • Number of eligible water points in expansion areas
  • Ongoing M&E data, bi-annaully
    • Households per water sources with dispensers
    • People per household
    • Under-five per household
    • % of households whose water sample tests positive for chlorine
    • % of households whose water sample was positive for chlorine who report using dispensers

• 36GiveWell, Dispensers for Safe Water Organization basics, based on field notes, 2022