Wageningen University — Add-ons to Sierra Leone RCT on Health Service Delivery to Remote Communities (May 2024)

Note: This page summarizes the rationale behind a GiveWell grant to Wageningen University. Wageningen University staff reviewed this page prior to publication.

Published: July 2024

Summary

This grant will support a team of researchers to extend a bundle of services delivered to communities located far from public health facilities in Sierra Leone and evaluate the intervention with a randomized controlled trial (RCT). The project is led by researchers affiliated with Wageningen University and the Yale Research Initiative on Innovation and Scale (Y-RISE) (see the full research team below). The researchers are partnering with the Expanded Programme on Immunization (EPI) at the Ministry of Health in Sierra Leone, which will provide vaccines, vitamin A supplements (VAS), and deworming pills, and Concern Worldwide, which will coordinate with the Ministry of Health on managing the delivery of the health services.

Prior to this grant, the researchers had funding to deliver vaccines, VAS, and deworming pills, and for one round of surveys at the time of delivery. This grant will be used to purchase chlorine, for disinfecting drinking water, and oral rehydration solution (ORS) with zinc, for treating dehydration due to diarrhea, and add them to the bundle of services. It will also add a second round of service delivery and surveys, in order to measure usage of chlorine and ORS/zinc after three months. (More below)

GiveWell made this grant mainly to increase the amount of RCT evidence available on how much free, household-level provision of chlorine and ORS/zinc increases usage of those products, particularly when delivered household-to-household in remote communities. Chlorine and ORS/zinc are low cost and have the potential to prevent a meaningful share of the large number of deaths from diarrhea each year in low- and middle-income countries. (More in our reports on water quality interventions and ORS/zinc.) GiveWell is considering supporting programs that deliver vaccines to communities far from public health facilities, as one strategy to increase vaccination coverage in areas with high vaccine-preventable disease burdens. Chlorine and ORS/zinc, if usage rates are reasonably high, have the potential to increase the cost-effectiveness of such programs. (More below)

One of the primary researchers working on the project is Mushfiq Mobarak, who GiveWell has worked with on several RCTs that yielded decision-relevant results (more below). (Disclosure: Mushfiq Mobarak is a member of GiveWell's Research Council, a group of advisors.)

Our top reservations about this grant are:

• Sierra Leone might not be the “best” place to fund this study. This grant is opportunistic. We were approached by the researchers about the possibility of layering additional commodities onto their planned RCT. We have not yet conducted a scoping exercise on where mobile vaccination campaigns or distribution of ORS/zinc and chlorine might be most promising, so it’s not clear how closely Sierra Leone resembles the contexts where we might want to fund programs in the future.
• The study results may not be as helpful as we think. We did not do a deep review of the existing literature on mobile vaccination campaigns/chlorine/ORS during this grant investigation. Instead, we relied on our impressions of chlorine and ORS/zinc provision from existing GiveWell work and conducted a relatively shallow literature review. It's possible that a deeper review would make us more comfortable extrapolating results from the existing literature, or would make us hesitant to extrapolate the results from this Sierra Leone study, either of which might mean that the study results would not be as informative for future funding decisions as we anticipate.
• We have not created a cost-effectiveness analysis for this grant. For research grants, we sometimes create models of the value of the information we expect the research to produce. This process can help us understand and explain what we'd need to believe to make it worthwhile for GiveWell to fund a piece of research. We skipped this step in this case in order to limit the time we spent on this grant investigation.

The intervention

The study will be a cluster-randomized RCT in which the treatment communities receive household-to-household visits from health teams offering routine childhood and HPV vaccines to those who have not received all of the vaccines in the vaccination schedule. Households with children under the age of 5 will also be offered VAS, deworming pills, chlorine to treat drinking water, and ORS/zinc to treat future diarrhea episodes.1 Control communities will not receive any interventions.2 The health teams will also provide information on when young children are due for subsequent vaccines.3 The health teams will return after three months and provide the same bundle of services.4

Data on vaccination, VAS, and deworming uptake will be collected at the same time as delivery at both the initial visit and the 3-month follow-up visit, and simultaneously in control communities.5 At the 3-month visit, households will also be asked about use of chlorine and ORS/zinc, and the survey team will test household water for chlorine.6 Chlorine and ORS/zinc usage will be measured just at the 3-month visit, as unlike the other commodities, take-up of these products cannot be measured right away.7

The grant

This grant adds to existing funding for an RCT due to take place in Sierra Leone in 2024 in partnership with the Ministry of Health in Sierra Leone and Concern Worldwide. The grant will be routed through Wageningen University, where one of the principal investigators of the RCT is affiliated.

The study is a cluster RCT. The clusters are communities in Sierra Leone that are located far from health facilities.8 The control group will receive no intervention and is surveyed when the treatment group first receives the health bundle and then again after 3 months.9 In treatment villages, a health team goes household to household to provide:10

• Health information
• Any missed vaccines for children under 5, and information on how to receive subsequent vaccinations in the schedule
• HPV vaccines for girls age 10-17
• Vitamin A supplements
• Deworming pills
• ORS/zinc sachets
• Chlorine

Budget for grant activities

The grant total is $676,857.11 The grant funds:

• An additional round of data collection 3 months after the first round: $275,462.12
  • The study design prior to this grant would have measured vaccine, deworming, and VAS uptake at the time of delivery with no subsequent follow-up. This grant funds a second visit after 3 months to deliver the same bundle of services and to ask about usage of ORS/zinc and chlorine, and about whether any eligible children in the household received vaccinations between the visits. The health team will tell households at the first visit that they will return in 3 months to offer the same bundle of services again.13
• Purchase and distribution of chlorine in two rounds: $209,01214
• Purchase and distribution of ORS/zinc in two rounds: $120,00315
• Increasing the number of communities from 375 in total (125 in the control group) to 450 (200 in control)16 : $72,380.17 This increases statistical power and decreases, by a small amount, the effect size that the study can detect.18

We talked to the research team about funding additional study activities, but ultimately decided not to fund them in this grant. Those include:

• $332,11719 to add a third data collection round. This could potentially be added later and could be used to answer a variety of questions, depending on the time between intervention and survey. More in footnote.20
• $52,000 for Y-RISE policy outreach staff time to disseminate the results with the government of Sierra Leone, funders, and NGOs.21 This could potentially be added later depending on the results of the study.

Research team

The research team includes Mushfiq Mobarak (Professor of Management and Economics at Yale University), Maarten Voors (Associate Professor at the Development Economics Group at Wageningen University), Niccòlo F. Meriggi (Postdoctoral Research Fellow in Economics at University of Oxford), Dr. Desmond Kangbai (Sierra Leone Ministry of Health), Uday Wadehra (Research Manager at Wageningen University), and Sara Gómez Mesa (Economics Pre-Doctoral Fellow at Yale University).

The case for the grant

We are recommending this grant because:

• We expect this study to have a decent chance of affecting our future grant decisions. The existing RCT evidence base for the effect of interventions similar to this one is thin. This study is expected to add meaningfully to that evidence base and inform a significant amount of future GiveWell grantmaking. It may also be helpful to policymakers in low- and middle-income countries who are determining how to best deliver health services to communities that are far from health facilities. (More below.)
• We have relatively high confidence in the research team. (More below.)

We expect this study to have a decent chance of affecting our future grant decisions.

In short:

• Distribution of ORS/zinc and chlorine are programs that are high priorities for GiveWell due to potential for high cost-effectiveness in averting deaths among children under 5 in areas with high child mortality. This is due to diarrhea being a leading cause of death for children under 5, and ORS/zinc and chlorine being inexpensive and moderately effective at preventing death attributed to diarrhea and other causes.
• For both programs, we have fairly limited evidence for which distribution methods lead to what increase in uptake, and the distribution model of this study (integrated with a bundle of other health services) differs in ways that could affect uptake.
• The study seems reasonably well-powered to detect decision-relevant effect sizes.
• There is a high amount of potentially highly cost-effective room for more funding for distributing ORS/zinc and chlorine. Vaccine outreach to remote communities is a theoretically promising way to deliver it, since such a program reaches families with young children (the age group most at risk of mortality due to diarrhea) who are particularly disconnected from public health services.
• There's also a benefit of testing ORS/zinc and chlorine usage when they are distributed together, as they may be in a scaled up version of the program. It's possible that households will see the products as substitutes, e.g. using less chlorine because they have ORS/zinc available for diarrhea episodes. Usage of each might be lower than if we assumed usage of each when they are distributed separately.

More detail for ORS/zinc

What is the existing evidence base?

• We searched for evidence on the impact of free, proactive distribution of ORS/zinc (i.e. distributing widely so households have it on hand for later diarrhea episodes) as part of writing our intervention report. We identified one RCT conducted in Uganda.22 This study found large increases in the share of children treated with ORS when they have diarrhea and ORS is provided for free to households. Adjusting for baseline ORS usage and internal and external validity adjustments, this implies a 15-16 percentage point increase in that share in the settings we model.23
• We funded CHAI to implement campaigns to distribute ORS/zinc to households in Bauchi, Nigeria. The program will be rolled out randomly and studied with an RCT.
• We have been discussing with New Incentives (one of GiveWell's top charities) the idea of distributing ORS/zinc to caregivers when they bring their children to health clinics for routine vaccinations, and comparing ORS/zinc usage rates before and after rollout of the program.

How will this study add to the evidence base?

• This study would contribute to the evidence base on household-to-household delivery of ORS/zinc and may allow us to better assess the effectiveness of this delivery method in particular. Furthermore, the distribution model in this study differs from that of previous studies we have seen in that the commodities are being distributed as part of a bundle with other health services, and distribution is focused on remote communities in particular. We think results from this model may be informative despite existing evidence on household-to-household distribution of ORS/zinc because we think it's plausible that usage might be different in this specific context. For example:
  • Recipients might be overwhelmed by the quantity of health services they’re receiving (childhood vaccines, HPV vaccines, VAS, deworming) and not process instructions on ORS/zinc usage
  • Recipients might not use either ORS/zinc or chlorine if they view these commodities as substitutes for each other
  • People in areas with very little contact with health systems might react to health information or commodities differently than people in other areas
• The researchers' primary research question is "What is the increase in DALYs [averted] per dollar spent delivering a bundle of health services to remote, rural communities in Sierra Leone?" More detail, from the proposal: "The trial will produce data on the increase in immunizations and take-up of deworming pills in treated communities relative to control - both immediately, and after 3 months. We will also have data on increase in water chlorination, and take-up of vitamin A and ORS after 3 months. [...] The take-up of immunizations will be observed directly and measured immediately, and our SSRC-funded project was designed for that. With additional GiveWell funds we will conduct a second visit after 3 months to measure the take-up of chlorination and other health services not directly administered on day 1."24
• Estimated minimal detectable effect for ORS/zinc usage is about 8 percentage points,25 from an estimated control group usage rate of 43%.26

How much future funding could better evidence affect?

We wrote previously, "We have not yet completed a full analysis of room for more funding for ORS and zinc provision programs. We believe there may be substantial room for more funding based on conversations with charities working in child health programs, including in areas with a high burden of mortality from diarrhea."

In particular, we have had a few conversations with New Incentives about distributing ORS/zinc in the health facilities in which its staff meet with caregivers of young children to provide cash stipends for vaccinating their children. New Incentives has extensive reach in nine states in northern Nigeria, and enrolled approximately 1,500,000 infants in its program in 2023.27

More detail for chlorination

What is the existing evidence base?

• There are a fairly large number of RCTs of water chlorination, some of which, if they collected and shared data on child mortality, are included in a Kremer et al. meta-analysis. These include studies from Uzbekistan, Guatemala, Kenya, Pakistan, India, Bangladesh, and Zimbabwe.28 We haven't done a systematic search for studies of chlorination uptake in low- and middle-income countries.
• We are interested in studies of chlorination uptake from rural Sierra Leone specifically because we expect that willingness to use chlorine can differ by context in hard-to-predict ways. Our analysis of risk factors that we use as proxies for the impact of chlorination in a country suggests that a chlorination program in Sierra Leone may be highly cost-effective.29 A light search30 did not uncover any previous RCTs of chlorination uptake in Sierra Leone.
• The study may also contribute to our estimates of chlorine uptake in countries with significant shared characteristics, but we haven't yet traced how we would update on this evidence for other locations.

How will this study add to the evidence base?

See above for a description of the intervention and primary research question.

The study seems to be well-powered for measuring chlorine uptake. The estimated minimal detectable effect for chlorine use is estimated to be very low (5 percentage points31 ) for this study due to very low estimated current usage (8%32 ). The study being powered to detect even a very small effect size means it's more likely that if there is a large enough effect that we would want to fund this further, the study would be able to detect that.

How much future funding could better evidence affect?

If the main use of the study results is to estimate chlorine uptake from free distribution in Sierra Leone only, room for more funding is somewhat limited, given an estimated non-urban population of 4.5 million.33 However, there is also the possibility that this evidence will impact our estimates for chlorination uptake in other countries. We previously recommended a grant of up to $64.7 million over seven years for chlorine dispensers in Kenya, Malawi, and Uganda, and at the time of recommending that grant believed that there was likely room for more funding for that program in other locations.

It's also possible that this research will influence policy makers and funders beyond GiveWell to put resources toward a highly cost-effective intervention. The proposal notes, "Our primary hope is that if the trial data shows that this last-mile delivery of a bundle of health services cost-effectively improves population health in remote communities, then we will inspire the Sierra Leone MoHS and its international development partners to deploy this strategy on a larger scale."34 This possibility was not a major reason for our decision to fund the study due to our uncertainty about whether this research will be useful and compelling to policymakers.

We have relatively high confidence in the research team

GiveWell has either funded or relied for major decisions on several RCTs led by Mushfiq Mobarak, including of the No Lean Season program, a former GiveWell top charity, and of the impact of mask distribution and promotion to reduce COVID-19 cases. The No Lean Season case is notable because the program was studied in a series of RCTs. Smaller scale versions of the program yielded positive results, while the evaluation of the scaled-up program found minimal impact.35 The program was closed, allowing the funding to flow to programs with a higher impact. The RCT of masks was notable for breaking new ground on a policy-relevant question in unusual conditions (during a pandemic).

Most of the same researchers also led a recent prior study that was very similar—delivering COVID-19 vaccines in similarly remote communities in Sierra Leone. They worked with Concern Worldwide and the government of Sierra Leone for that study as well.

Risks and reservations

Our main reservations about this grant are:

• Sierra Leone might not be the “best” place to fund this study. This grant is opportunistic. We were approached by the researchers about the possibility of layering additional commodities onto their planned RCT. We have not yet conducted a scoping exercise on where mobile vaccination campaigns or distribution of ORS/zinc and chlorine might be most promising, so it’s not clear how closely Sierra Leone resembles the contexts where we might want to fund programs in the future.
• The study results may not be as helpful as we think. We did not do a deep review of the existing literature on mobile vaccination campaigns/chlorine/ORS during this grant investigation. Instead, we relied on our impressions of chlorine and ORS/zinc provision from existing GiveWell work and conducted a relatively shallow literature review. It's possible that a deeper review would make us more comfortable extrapolating results from the existing literature, or would make us hesitant to extrapolate the results from this Sierra Leone study, either of which might mean that the study results would not be as informative for future funding decisions as we anticipate. Relatedly, there are some differences between the researchers' goals and GiveWell's goals that may limit the study's usefulness for our work. For example, we're interested in how an intervention like this would perform at scale, and a study focused on evaluating an intervention for scale-up would build in a period in which the intervention is implemented and the details of how it will be carried out at scale are ironed out. Instead, this project has a very short piloting period (~2 weeks).36
• We haven't quantified the value of information for this grant. We've found it a helpful exercise in the past to do some basic calculations to understand what we'd need to believe to make it worthwhile for GiveWell to fund a piece of research. This process makes our assumptions explicit about how much a piece of evidence could change our decisions and how much more impact would be achieved by the change. We skipped this step in this case in order to limit the time we spent on this grant investigation, due to this grant being a small proportion of our vaccines team's total grantmaking for the year, and an opportunistic grant (i.e. not squarely in our strategic priorities).
• Potential risks of ORS/zinc and chlorination. Overall, ORS/zinc and chlorine are widely recommended and used products,37 so we think these risks are small. Risks include:
  • Dosages of zinc and chlorine that exceed recommended levels. In particular, WHO recommends different zinc dosages for children under 6 months due to potential risks of overdosing zinc.38 The dosage of zinc in a standard co-pack is higher than the recommended dose for children under 6 months and caregivers may not adjust the dosage properly. We write in our report on water quality interventions, "Chlorine itself is toxic to humans in concentrated form, but at lower concentrations water chlorination has a long history of apparently safe municipal use. [...] WHO states that “the risks to health from disinfection by-products are extremely small in comparison with the risks associated with inadequate disinfection.”
  • There are different recommendations for how to provide ORS/zinc to children with severe acute malnutrition.39 We haven't asked the study team whether this will be part of the health information the program provides.
  • Programs that provide health goods for free may disincentivize local retailers from supplying these goods. This program only lasts three months, which we'd guess is not long enough to significantly undermine private markets.
  • Households that receive health goods for free may become accustomed to receiving the services for free and may be less willing to purchase them after the program ends.This program only lasts three months, which we'd guess is not long enough to change behavior longer term.

Plans for follow up

The primary follow-up from this grant will be reviewing the research results and descriptions of methods, asking for any additional analysis or clarifications that would be helpful to us, and determining how to update our cost-effectiveness estimates for the interventions.

We plan to request semi-annual check-in calls with the research team until the analysis is complete.

Internal forecasts

For this grant, we are recording the following forecasts:

Our process

• We talked to the study team several times to find areas of mutual interest for adding onto this RCT. From the team's initial list of potential interventions to add to the project, GiveWell suggested a focus on ORS/zinc and chlorine based on prior research.
• We requested a budget and proposal from the study team, which they provided.
• A GiveWell senior program officer and a program officer reviewed the case for and against the grant before it was approved.

Relationship disclosures

Mushfiq Mobarak, one of the researchers involved with the study funded by this grant, is a member of the GiveWell Research Council.

Sources

• 1

  "In both treatment conditions, we will deliver the same bundle of health services and vaccines. This
  bundle includes:
  1. Child immunization for children under 5, including the following vaccines
  a. BCG
  b. Pneumococcal
  c. Rotavirus
  d. IPTi
  e. MCV
  f. Yellow fever
  g. Malaria RTS vaccine
  h. IPV vaccine
  2. HPV vaccines for girls aged 10-17
  3. Chlorination tablets for every household in the community to treat drinking and cooking
  water. If used correctly, the tablets delivered should be enough to treat water for 3 months.
  4. Health services for children under 5:
  a. Vitamin A drops
  b. Deworming pills
  c. ORS/Zinc sachets to treat cases of diarrhea" YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 4.

• 2

  "The health teams visit treatment communities for 3 consecutive days. Control villages will not host any mobile health services delivery team." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 5.

• 3

  "The vaccinators will also talk to the parents about when the next vaccine dose is
  due and where and when they should take their children to receive the required doses of
  immunization (i.e. the CHC closest to their community)." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 6.

• 4

  "With additional GiveWell funds we will conduct a second visit after 3 months to measure the take-up of chlorination and other health services not directly administered on day 1." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 6.
  "Secondary research questions
  1. What are the marginal gains from visiting a community a second time?
  If we cover most households in the community with required vaccinations during the initial
  visit, it is possible that a second visit would have relatively low value-add. Perhaps those
  resources are better redirected towards initial visits for an entirely new set of communities.
  On the other hand, restocking of chlorine tablets and ORS packets may be very important
  for sustained use." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 7.

• 5

  "The trial will produce data on the increase in immunizations and take-up of deworming pills in
  treated communities relative to control - both immediately, and after 3 months." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 6.

• 6

  "We will conduct “spot check” chlorine tests to measure whether the drinking water
  in the household at the time of our unannounced visit has any chlorine." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 9.

• 7

  "The take-up of immunizations will be observed directly and measured immediately, and our
  SSRC-funded project was designed for that. With additional GiveWell funds we will conduct a
  second visit after 3 months to measure the take-up of chlorination and other health services not
  directly administered on day 1." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 6.

• 8

  "The main objective of our study is to calculate the effectiveness and cost-effectiveness of an
  intervention in which we deploy mobile healthcare services delivery (including vaccination) teams to
  remote communities in Sierra Leone that are currently not well served by an existing government
  and NGO health services delivery programs." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 3.

• 9

  "The trial will produce data on the increase in immunizations and take-up of deworming pills in
  treated communities relative to control - both immediately, and after 3 months." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 6.

• 10

  "They start the visit by privately discussing with residents about the efficacy and safety of the health services included in the bundle. This will include information on the importance of drinking clean water, the dangers of diarrhea for children under 5, the importance of vaccines, and any concerns about the bundle that the household residents may have. The vaccinator will check the vaccination cards to determine which vaccines the children already have, and what doses are due. If the residents agree, the team will vaccinate the children under 5, girls aged 10 to 17 with the HPV vaccine, administer deworming pills, supply Vitamins A drops, and distribute appropriate dosages of ORS+Zinc sachets, chlorination tablets, and additional deworming pills and Vitamin A drops, enough to last to treat drinking water for 3
  months or more. " YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, pp. 5-6.

• 11

  See the budget for grant activities here.

• 12

  $213,862 in data collection costs (assuming 375 communities; bullet below accounts for cost of moving to 450 communities) + $118,800 in Yale research costs - $57,200 in policy outreach ($52,000 for policy outreach + $5,200 in indirect costs (10% of the $52,000)) =$275,462. We subtract policy outreach costs because we have elected not to fund those activities in this grant. See bullet below in this section.

• 13

  "Other ancillary benefits of the second visit include (a) measuring the effects of the vaccination
  conversations and reminders to see whether it induces any follow-up actions by parents in the
  interim period, and (b) informing households that we will return in 3 months to re-stock chlorine
  and ORS may change adherence." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 9

• 14

  $329,015 in implementation costs, of which $153,990 is procurement costs. Thus $175,025 is delivery costs, which we split evenly between chlorine and ORS/zinc. $60,750 for one round of chlorine delivery * 2 rounds = $121,500 is chlorine procurement. $121,500 + $175,025 / 2 = $209,012

• 15

  See prior footnote. $329,015 - $209,012 = $120,003 is the ORS/zinc portion.

• 16

  The prior funding for the study was on the basis of having a split treatment group (125 communities in each of two arms) to measure the effect of having higher level health officials contact local officials to promote the intervention. Hence a treatment group that is double the size of the control group.

• 17

  $734,057 for 450 communities minus $661,677 for 375 communities, assuming 2 rounds of data collection.

• 18

  "Given our current assumptions, with 125 control communities we are powered to detect effect sizes
  of 7.33-8.60 pp for all of the health services included in the bundle. These are reasonable guesses on
  the effect sizes we expect, in light of findings from our previous work. However, given the current
  gaps in immunization in Sierra Leone, we have constructed Budget B that adds 75 control
  communities to detect even smaller effect sizes in the range of 6.43-7.59 pp." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 14.

• 19

  $1,066,174 total cost with three data rounds - $734,057 total cost with two data rounds = $332,117 additional cost for a third data collection round.

• 20

  "A third round of data collection will allow us to study the longer-term effects from the delivery of
  the bundle. If child immunization, life expectancy, and overall health in the household improves, we
  may observe downstream changes in two dimensions. First, households may form habits and choose
  to procure chlorine and ORS/Zinc on their own, or they may increase trips to health facilities.
  Second, households may change their investment in children's human capital. We would like to
  measure changes in children’s height and weight than middle upper arm circumference. Some of
  these biometrics are very quickly sensitive to changes in the nutritional and disease environment, so
  we hope to do the 3rd visit somewhere between month 7 and month 18 post-intervention." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 14.

• 21

  "Policy outreach - $52,000. Our goal is to work on disseminating and communicating results
  with Sierra Leone MoHS high-level decision-makers, their international development
  partners like UNICEF, UNDP, WHO, UK and US government, and international NGOs
  with a local presence in Sierra Leone, To encourage replication and scale up depending on
  the trial results." YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 16.

• 22

  "The effect of the program on ORS and zinc usage. Our estimates rely on evidence from a single RCT and it is very possible that further RCT evidence could update our assumption." GiveWell, Oral Rehydration Solution (ORS) and Zinc.

• 23
  • Intervention arms included "free delivery of ORS prior to illness" and "free ORS upon retrieval using voucher," with both ORS (first arm) and vouchers (second arm) provided by community health workers. Our report focuses on the latter, while the former is closer to the study discussed on this page. We're not making a distinction in the discussion here because the two intervention arms found similar results on ORS/zinc use.
    • "In March of 2017, we randomized CHWs to one of four methods of ORS distribution: (1) free delivery of ORS prior to illness (free and convenient); (2) home sales of ORS prior to illness (convenient only); (3) free ORS upon retrieval using voucher (free only); and (4) status quo CHW distribution, where ORS is sold and not delivered (control).... instructing CHWs to provide free and convenient distribution increased ORS coverage by 19 percentage points relative to the control group (95% CI 13–26; P < 0.001), 12 percentage points relative to convenient only (95% CI 6–18; P < 0.001), and 2 percentage points (not significant) relative to free only (95% CI −4 to 8; P = 0.38)." Wagner et al. 2019.
  • "To estimate the effect of free provision of ORS/zinc to caregivers on the usage of ORS/zinc, we rely on one randomized controlled trial (RCT) that studied the link between ORS provision and usage in Uganda. Our best guess is that free provision of ORS can decrease the share of children not using ORS when they have diarrhea by 27%. This is equivalent to a 15-16 percentage point increase in the share of children treated with ORS when they have diarrhea in settings we model. We make a simplifying assumption that this also applies to zinc usage." GiveWell, Oral Rehydration Solution (ORS) and Zinc, "Effect of provision of commodities on ORS/zinc usage" section.
  • "We apply an 80% adjustment to the estimate of the reduction of non-ORS usage to account for internal validity factors, including a lack of masking of data analysts and self-reporting by participants. We also apply an 85% adjustment to the effect size to account for external validity factors. In particular, community health workers in the study visited households and provided redeemable vouchers to parents, many of whom redeemed them for ORS prior to their child being sick." GiveWell, Oral Rehydration Solution (ORS) and Zinc, "Effect of provision of commodities on ORS/zinc usage" section.

• 24

  YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 6.

• 25

  YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 12.

• 26

  YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 19.

• 27

  See more information on New Incentives' 2023 enrollments in the "Highlights from our top charities section" of our February 2024 updates blog post.

• 28

  See Table 1: Summary of studies included, "Country" column, Kremer et al. 2023.

• 29

  These proxies include under-5 all-cause mortality rate and % of mortality from enteric infection. For more details, see this row of our cost-effectiveness analysis of chlorine vouchers to improve water quality.

• 30

  In particular, we looked at the first two pages of results on Google Scholar for “sierra leone, chlorination OR chlorine”.

• 31

  YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 11.

• 32

  YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 20.

• 33

  See this cell of our cost-effectiveness analysis of chlorine vouchers to improve water quality.

• 34

  YRISE, Scaling bundled health services in rural Sierra Leone, GiveWell research proposal, 2024, p. 7.

• 35

  "The program has been evaluated in several randomized controlled trials (RCTs) in Bangladesh. Two RCTs found strong positive results of seasonal migration subsidies on household income and consumption. Two RCTs, including one conducted in 2017 (which was the first to evaluate the program as it operated at a large scale), found that the program did not significantly increase migration rates relative to the control group." GiveWell, Evidence Action's No Lean Season.

• 36

  Call with Y-RISE, May 8, 2024 (unpublished).

• 37
  • WHO recommends ORS for treatment of diarrhea: "Diarrhoeal diseases remain one of the major causes of mortality among children aged under 5 years, accounting for 9% of deaths among children worldwide. Most of the deaths in children from diarrhoea could be averted by using ORS and zinc supplementation during episodes of diarrhoea, and basic interventions to improve drinking water, sanitation and hygiene (WASH)." WHO, "Children <5 years with diarrhoea receiving oral rehydration solution (ORS)."
  • ORS is often used in low-income countries: "We conducted a comparative analysis of two Demographic Health Survey (DHS) cycles to examine changes in ORS coverage in Zimbabwe, Zambia and Malawi… Crude ORS coverage increased from 21.0% (95% CI: 17.4–24.9) in 1st Period to 40.5% (36.5–44.6) in 2nd Period in Zimbabwe; increased from 60.8% (56.1–65.3) to 64.7% (61.8–67.5) in Zambia; and decreased from 72.3% (68.4–75.9) to 64.6% (60.9–68.1) in Malawi." Gona et al. 2020.
  • WHO recommends chlorine for disinfecting drinking water: “the risks to health from disinfection by-products are extremely small in comparison with the risks associated with inadequate disinfection.” World Health Organization, Principles and practices of drinking-water chlorination, 2017, Pg. 18.
  • Chlorine is widely used: "Chlorine was first used in the United States as a major disinfectant in 1908 in Jersey City, New Jersey. Chlorine use became more and more common in the following decades, and by 1995 about 64% of all community water systems in the United States used chlorine to disinfect their water." CDC, "Water Disinfection with Chlorine and Chloramine."

• 38

  WHO integrated case management module 2014 (p. 31): “Zinc is only given to children 2 months up to 5 years…
  GIVE ZINC SUPPLEMENTS (one tablet is 20 mg zinc) [...] Tell the caregiver how much zinc to give
  Up to 6 months: ½ tablet per day, for 14 days
  6 months or older: 1 tablet per day, for 14 days”

• 39

  WHO recommendation: "Children who are under 5 years of age with severe acute malnutrition who present with some dehydration or severe dehydration but who are not shocked should be rehydrated slowly, either orally or by nasogastric tube, with either ReSoMal, or half-strength standard WHO low-osmolarity oral rehydration solution with added potassium and glucose* at a rate of 5–10 mL/kg/h, for up to a maximum of 12 hours… standard WHO low-osmolarity oral rehydration solution (75 mmol/L sodium) should not be used."

• 40

  Note that during review of this page, Wageningen University staff suggested a completion date of October 2024 instead of August 2024. We have kept August 2024 here to reflect our original forecast at the time this grant was made.