Clinton Health Access Initiative — Oral Rehydration Solution and Zinc Distribution in Bauchi, Nigeria (September 2023)

Note: This page summarizes the rationale behind a GiveWell grant to CHAI as of September 2023, when we made the grant. In January 2024, we increased the total budget for this grant, as we explain in the Addendum. CHAI staff reviewed this page prior to publication.

Addendum added: March 2024

Published: January 2024

Summary

What we think this grant will do

Diarrhea is a significant cause of illness and death in Nigeria among children aged under five. Oral rehydration solution (ORS) is a type of fluid replacement, often administered alongside short-term zinc supplementation, to treat dehydration due to diarrhea.

This grant will support CHAI to design and implement a program in Bauchi, Nigeria, to distribute free ORS and zinc to all households with children under age 5. We think that free provision of ORS and zinc to caregivers could increase use of ORS and zinc and, in turn, decrease diarrhea-related mortality among children under five.

Why we made this grant

• We estimate that an ORS and zinc provision program in Bauchi, Nigeria, is cost-effective because it is cheap to deliver and can avert childhood mortality.

  We estimate that this grant is approximately 17 times as cost-effective as cash transfers, which is above our current funding bar of 10 times as cost-effective as cash. We think this grant is highly cost-effective because:

  • Providing ORS causes additional children to be treated with ORS when they have diarrhea. Our best guess is that convenient and free provision of ORS directly to caregivers will increase the share of children in Bauchi, Nigeria, treated with ORS when they have diarrhea by 17 percentage points.
  • We think that using ORS is likely to reduce diarrhea-related mortality. Our impression is that it is widely accepted that ORS is effective at reducing mortality due to diarrhea. Our best guess is that treating children with ORS reduces diarrhea mortality by 60%.
  • The cost per person is quite low. CHAI has estimated that the program would cost around $2.70 per child aged under 5. Cost per child estimates are based on CHAI's budgets and adjusted population estimates from GeoPoDe.2

A sketch of our cost-effectiveness analysis and level of uncertainty is in the table below. This analysis requires taking a stand on several uncertain parameters that could change cost-effectiveness substantially. These key parameters are in orange.

You can see our cost-effectiveness analysis for the program here and a simple version here. More on our cost-effectiveness estimate is below.

• We think the RCT will allow us to learn about the impact of the program. This grant has additional value in allowing us to reduce our uncertainty about the cost-effectiveness of this program and informing our decisions of whether to direct more or less funding to similar programs in the future. We model the total grant including these learning benefits as 24 times as cost-effective as cash transfers.

  We believe this learning is highly cost-effective because:

  • During the RCT we expect to learn more about key uncertainties such as program costs and changes in ORS usage.
  • If we conclude that the program is above our funding bar after completing the evaluation, we could direct more funding to ORS and zinc provision across several countries in the future.
  • We also think there is some value from learning that this grant is below our funding bar. This is because we think there is currently some chance that we would fund this program without an evaluation (given its high estimated cost-effectiveness), so if we learn it is below our funding bar we will gain value by redirecting funding to other more cost-effective opportunities.

You can see our “value of information” analysis of the RCT here and a simple version here.

We also think that this program could open up options to either (a) add other commodities to the same delivery platform; or (b) add ORS/zinc provision to other programs. More on our estimate of learning value is below.

Factors outside the model

• We have a strong qualitative impression of CHAI as an implementer. This is discussed more below, and in our grant page for the CHAI Incubator here as well.
• CHAI plans to work with an external evaluator for the RCT, which alleviates concerns we had regarding the independence of the analysis. The evaluator will fully and independently conduct the data analysis based on a pre-analysis plan.3 More here.

Main reservations

• We are relying on quasi-experimental evidence on the effect of ORS on mortality. The effect of ORS on diarrhea-related mortality is the largest driver of benefits in the CEA, and it is possible that we are overestimating this effect. We don’t plan to power a study for mortality effects at this stage given the size of study required, the biological plausibility of an effect, and what we view as limited broader benefit of such a study. More below.
• This is a new program, and it is possible that we are underestimating the difficulty of implementation. This is an ambitious program requiring training and supervision of many people in order to distribute ORS to a very large number of households. It is possible that there may be context-specific operational difficulties or higher costs that we haven’t accounted for. More below.
• We will still have uncertainties about several factors that could affect the cost-effectiveness of ORS and zinc provision which we don’t expect to resolve through the evaluation. If we update our estimate following the evaluation to think that the program is closer to our cost-effectiveness bar, then those factors could become more relevant to future grant decisions and we still may be very uncertain about them. More below.
• The program might incentivize children to increase their intake of poor quality water, and we may be missing an opportunity to also provide chlorine to households. Because ORS is mixed with water, there is some risk that young children who are exclusively breastfed could be induced to drink poor-quality water. We believe this risk is small, and hope to learn more about this qualitatively during the trial. More below.

The organization

The Clinton Health Access Initiative (CHAI) is an international public health non-profit founded in 2002, which operates in 35 countries.4 GiveWell is the sole funder of the CHAI Incubator, a program within CHAI that exists to find, assess, and implement programs that have high cost-effectiveness and potential to scale. CHAI has developed this oral rehydration solution (ORS) and zinc distribution program through the Incubator.

CHAI has experience scaling the availability of ORS and zinc for children with diarrhea in other programs: since 2012, CHAI has implemented programs to scale up ORS and zinc coverage in Ethiopia, India, Kenya, Nigeria, and Uganda.5

The intervention

Diarrheal disease is the frequent passing of loose or liquid stools,6 usually caused by an infection of the intestinal tract.7 Diarrhea generally causes death via severe dehydration and fluid loss.8 The Institute for Health Metrics and Evaluation (IHME) estimated that diarrhea killed about 130,000 children under five years old in Nigeria in 2019.9 We estimate that around 1.3% of children not receiving ORS with zinc will die over the course of the year from diarrhea-associated causes in Bauchi.10

ORS is a type of fluid replacement used to prevent and treat dehydration due to diarrhea.11 Therapeutic zinc supplementation (generally in the form of dispersible tablets) is also often provided alongside ORS, because it is understood to reduce the duration and severity of diarrhea episodes.12

We have moderate confidence that free ORS and zinc provision will increase usage of ORS and zinc, and that using ORS and zinc will reduce child mortality caused by diarrhea.

For more information, see our intervention report on ORS and zinc here.

The grant

This $6.6 million grant,13 funded by GiveWell and Effektiv Spenden, will support CHAI to run a mass distribution campaign of ORS and zinc in Bauchi, Nigeria, and to study the impact of the program through an RCT.

The program will involve:14

• Recruiting and training Community Nutrition Volunteers (CNVs)15 to map every household with children aged under 5, and distribute free ORS and zinc to those households.16 We think there are around 1.5 million children under age 5 living in Bauchi.17
• Providing educational materials to caregivers on using ORS/zinc and managing and preventing diarrhea18

The RCT will estimate the effect of free and convenient provision of ORS and zinc on subsequent usage by children with diarrhea.19 CHAI will work with an external research team in order to ensure an independent evaluator is conducting the study.20 The ORS and zinc distribution activities will be done in two stages so that a treatment and control group can be compared.21 In stage one, the treatment group will receive ORS and zinc, and data will be collected on both the treatment and control groups. In stage two, the control group will receive ORS and zinc to ensure equitable distribution.22

Budget for grant activities

The $6.6 million budget includes:23

• $4,147,815 for implementation of the program, of which:
  • $1,293,214 covers commodity costs
  • $1,956,821 covers CHAI's technical assistance and coordination work
  • $525,441 covers campaign implementation (including recruitment and training)
  • $372,339 covers CHAI's indirect costs
• $2,446,000 for the impact evaluation, of which:
  • $1,731,000 covers costs for an external data collection agency
  • $715,000 covers costs for an external evaluator

The case for the grant

We are recommending this grant because:

• This program is likely to be highly cost-effective. We estimate this grant is 17 times as cost-effective as unconditional cash transfers, which is above our current threshold for funding a program. As of mid-2023, our bar for directing funding is about 10 times as cost-effective as unconditional cash transfers. More below.
• We believe we will learn valuable information from the RCT that will inform our future decision-making. We estimate that the value of information derived from this grant is around 24 times as cost-effective as unconditional cash transfers. More below.
• We have a strong qualitative impression of CHAI as an implementer. More below.
• CHAI plans to work with an external evaluator for the RCT, which increases our trust in the data collected and analyzed through the trial. More below.

Cost-effectiveness

Our current CEA for this grant suggests that the program may be about 17 times as cost-effective as unconditional cash transfers over the course of the grant period.24 This is equivalent to an estimated cost per life saved of around $2,100. In future years (where costs would be lower because one campaign could cover the entire region, rather than having a separate campaign for the treatment and control group) we expect that the cost-effectiveness of the program could increase.25

ORS and zinc are very inexpensive commodities, and we expect that using these commodities leads to a substantial decline in mortality caused by diarrhea. Usage of ORS and zinc is estimated to be below 50% in Bauchi, while the mortality burden of diarrhea is high.26

A sketch of the cost-effectiveness model is below:

• We assume that there are around 1.5 million children aged under five in Bauchi, Nigeria. We guess that without this intervention around 46% of children would receive ORS with zinc during diarrhea episodes during the year and that providing these commodities increases this to 63%, or a 17 percentage point increase in use of ORS with zinc.27
• We estimate that around 1.3% of children who do not receive ORS will die over the course of the year from diarrhea-related causes in Bauchi. This includes indirect deaths associated with diarrhea.
• We estimate ORS with zinc lowers mortality from diarrhea by around 60%.28
• As a result, we think this program would avert around 2,000 deaths.29
• We estimate that CHAI’s program costs around $2.70 per child aged under 5.30
• This implies a cost of around $2,100 per death averted, which is around 16 times as cost-effective as unconditional cash transfers.
• We then incorporate additional benefits from development effects31 and medical costs averted.32 We estimate that mortality benefits account for around 90% of the total benefits of the program.33 We also incorporate an adjustment for excluded effects,34 as well as adjustments for the possibility of funging other funders.35 We do not think this program is likely to meaningfully leverage government costs.36 Our bottom line is that the program is around 17 times as cost-effective as unconditional cash transfers.

Learning value from an RCT

• We will directly estimate the treatment effect of the program on ORS usage among children with diarrhea. The RCT will allow us to directly estimate the effect of free and convenient provision of ORS and zinc on usage, which is a key parameter affecting our assessment of cost-effectiveness.37
• Additional learning about important model parameters. There are several other important model parameters which we hope to learn about through data collection conducted as part of the RCT. This includes:
  • The share of children treated for diarrhea at baseline
  • The program cost per child reached
  • Use of ORS packets by different age groups and/or loss of packets
  • Diarrhea prevalence38

  We also hope to learn about some other factors that indirectly affect our assessment of the program, including any adverse events, diarrhea incidence over the trial period, and barriers to ORS and zinc usage. The evaluation will include baseline and endline household surveys as part of the RCT, but may also be supplemented with focus groups and other qualitative data collection.39

• Monitoring check on implementation. Enumerators conducting data collection for the RCT will measure the share of treatment group households who are actually visited by CNVs during the trial. This will serve as an additional independent check on the fidelity of implementation, over and above the checks that CHAI will put in place as part of programmatic activities.40
• High modeled cost-effectiveness of learning. When we include the value of information from an RCT, we model the cost-effectiveness of this grant to be around 24x.

  A sketch of the “value of information” model is below:

  • We assume there is some chance we fund the program in the absence of RCT results because we currently believe it is cost-effective in Bauchi.41 We roughly estimate around $28 million in annual room for more funding of this program, and think that we might direct between $6 and $14 million to ORS and zinc provision in the absence of an RCT.42
  • We map out a range of possible effect sizes for the impact of the program on ORS and zinc usage and assign a guess for the likelihood that the effect is each size. We then estimate the associated cost-effectiveness for these given effect sizes across a range of countries, and derive the following probabilities for three potential funding scenarios43 :

  	8x future bar for funding	10x future bar for funding	12x future bar for funding
  Probability of scenario 1: We update our view and direct funding to this program	28%	25%	18%
  Probability of scenario 2: We update our view and redirect funding away from this program	17%	17%	14%
  Probability of scenario 3: We make no change to our funding	55%	58%	68%

  • In scenario 1, we would learn that the program is cost-effective and would redirect funding from other opportunities to ORS and zinc distribution.44
  • In scenario 2, we would learn that ORS and zinc distribution programs are less cost-effective than expected. In this case, we would redirect funding away from this program towards other more cost-effective opportunities.45
  • In scenario 3, our funding allocation would not change, so there is no value from this scenario in our model.46
  • There is also value from the delivery of the program itself, as discussed above.
  • We also incorporate some ad hoc adjustments.47
  • Bottom line: Our bottom line cost-effectiveness of the total grant including learning value is around 24 times as cost-effective as unconditional cash transfers.
• We think that learning more about this program could also open up optionality to layer other commodities onto the platform, or layer ORS/zinc provision onto other programs. It is possible that there are some commodities that could make sense to provide alongside ORS and zinc, which could make a given commodity provision program more cost-effective (assuming we conclude the program is highly cost-effective following the RCT). Funding this study now could give us optionality in the future to provide additional funding to programs to layer ORS and zinc distribution.
• Note: As of the writing of this grant page (November 2023), the pre-analysis plan for the RCT is not yet finished and details of the RCT as described above are subject to change. We intend to publish the pre-analysis plan on this page once it becomes available and note key differences if applicable.

Qualitative considerations beyond our model

In theory, our cost-effectiveness analysis intends to capture the total impact of a program per dollar spent. But we recognize that our cost-effectiveness calculations are not able to capture every factor that could make a program more or less impactful. Focusing only on our cost-effectiveness model may mean we’re missing things that are difficult to quantify.

As a result, we think it’s helpful to look at other perspectives and types of evidence that may not be captured in our bottom line cost-effectiveness number. Such considerations include:

• We have a strong qualitative impression of CHAI as an implementer. CHAI stands out as a strong partner due to its successful track record scaling very large, evidence-based programs, its extensive experience working in low- and middle-income countries, and its global footprint.48 Our regular collaboration through the CHAI Incubator has strengthened our view of the organization. We have found the CHAI team answer our questions with exceptional rigor and provide high-quality feedback, causing us to update and correct our work.

• Independent analysis. CHAI plans to work with an external evaluator for the RCT. The evaluator will take a leading role in the evaluation design (protocol, planning, tool development) and implementation (randomization, survey sample selection, data collection agency selection and training, and field management). The evaluator will also fully and independently conduct the data analysis based on a pre-analysis plan.49 This alleviates concerns we had regarding the independence of the analysis.

Risks and reservations

Our main reservations about this grant are:

• We are relying on quasi-experimental evidence on the effect of ORS on mortality. There is a lack of high-quality randomized evidence on the effect of ORS at reducing mortality in community settings, so we are uncertain about the magnitude of the expected effect of this program.50 The effect of ORS on diarrhea-related mortality is the largest driver of benefits in our model, and it is possible that we are overestimating this effect. We discount the effect size reported in the literature (a 93% reduction in diarrhea mortality among children who are treated with ORS) to account for this low evidence quality.51 We don’t currently plan to fund a study that is powered to detect the effect of ORS on mortality, because such a study would be very large and expensive, we believe such a study would be unlikely to have an effect on other donors' decision-making, and we believe it is biologically plausible that ORS would reduce diarrhea-related mortality.
• This is a new program, and it is possible that we are underestimating the difficulty of implementation. This is an ambitious program requiring training and supervision of many people in order to distribute ORS and zinc to a very large number of households. Although CHAI has experience both working in Bauchi.52 and working on similar mass distribution programs,53 it is possible that there may be context-specific operational difficulties or higher costs that we haven’t accounted for.
• We will still have uncertainties about several factors that could affect the cost-effectiveness of ORS/zinc following the evaluation, and don’t expect to resolve these through the evaluation. If we update our estimate following the evaluation to think that the program is closer to our cost-effectiveness bar, then various factors could become more relevant to the bottom line. Because of these uncertainties, we may still feel uncertain about future funding of the program. These uncertainties include:
  • The marginal benefit of zinc
  • Non-diarrheal mortality averted
  • The risk of crowding out of private providers
  • The importance of the timing of treatment.54

  We also think it is possible that the findings of the RCT may not generalize to contexts with different existing community health worker models.55

• The program might incentivize children to increase their intake of poor quality water. Because ORS is mixed with water and water quality in many areas of Bauchi is poor,56 we think there is some risk that children who are exclusively breastfeeding could be induced to drink poor-quality water. We conducted a short literature review, and found very limited discussion of this. We think this risk is limited because (a) our understanding is that ORS is still recommended during exclusive breastfeeding57 and where water quality is poor,58 (b) we believe that infants who are exclusively breastfeeding are at lower risk of diarrhea-related morbidity,59 and (c) CNVs during the program will be trained to educate caregivers on the importance of boiling water or using clean water.60 We hope to learn more about this during the trial by asking whether children are exclusively breastfeeding, and collecting some information on household water quality and how caregivers prepared the solution if feasible.61
  • We considered whether chlorine tablets or vouchers (which could improve home water quality)62 should be layered onto the program, but ultimately decided against it (although we may consider layering this in future). This is for several reasons:
    • We think that the RCT results would be more broadly applicable to other (mass commodity distribution) program types without the inclusion of chlorine.
    • In the case of vouchers, we think this would likely require a materially different operational program structure and add complexity.63
    • In the case of direct distribution, we’d guess that this might be less cost-effective if encouraging people to first chlorinate water introduces effort to administering ORS and therefore decreases usage.

Plans for follow up

We plan to have periodic check-ins with the CHAI team during the planning phase of the RCT. We will review the pre-analysis plan when complete.

We will have regular meetings with the CHAI team to discuss progress with the program and RCT over the course of the grant period. We expect to receive preliminary results of the RCT by mid-to-late 2025, and will consider this program for renewal or expansion once we receive those results.

Internal forecasts

For this grant, we are recording the following forecasts:

Our process

• We published an intervention report on ORS and zinc in August 2023.
• CHAI shared potential program geographies, program and study design options, and budgets. We had many discussions with CHAI to determine program specifics and study design.
• Researchers on other teams reviewed our internal investigation plan and case for the grant and raised additional questions.
• We spoke to external experts about ORS programmatic details, including the authors of Wagner et al 2019.
• Using our cost-effectiveness analysis of CHAI’s ORS/zinc distribution program as a base, we created a value of information analysis of the proposed RCT.

Relationship disclosures

Neil Buddy Shah was hired in April 2022 as CHAI’s CEO. Previously, he was GiveWell’s Managing Director. Dr. Shah was not involved in any discussions GiveWell had with CHAI related to this grant.

Sources

• 1

  This increase includes:

  • $865,500 for the data collection agency to return to the enumeration areas for a second wave of data collection.
  • $254,336 for CHAI to extend activities until December 2025 to complete the campaign in control villages, which will receive ORS/zinc 12 months later than originally planned.
  • $60,000 for the external evaluator to extend staffing to December 2025 to analyze and write the manuscript for the study.

  Budget calculations:

  • $865,500 + $254,336 + $60,000 = $1,179,836 total additional grant.
  • $6,593,815 + $1,179,836 = $7,773,651 total updated grant.

• 2

  $4.15 million / 1.51 million.

• 3

  CHAI, email to GiveWell, Aug 22, 2023 (unpublished).

• 4

  "CHAI was founded in 2002 with a transformational goal: help save the lives of millions of people living with HIV/AIDS. Today, along with HIV, we work with governments and partners to prevent and treat other deadly infectious diseases, including COVID-19, accelerate the rollout of lifesaving vaccines, reduce maternal, infant and child mortality, make assistive technology available to those who need it, and strengthen health systems. We operate in over 35 countries around the world and more than 125 countries have access to CHAI-negotiated deals on medications, diagnostics, vaccines, and other health tools." CHAI, "About Us"

• 5

  "In 2012, CHAI began working with governments, donors, and other partners to rapidly increase ORS and zinc use in India, Kenya, Nigeria, and Uganda. After only four years, ORS and zinc use increased rapidly from less than 1 percent receiving both ORS and zinc to 24 percent, reaching more than 55 million children and averting up to 76,000 deaths." CHAI, "How a simple diarrhea treatment can save thousands of children’s lives".
  CHAI began working in Ethiopia in 2016 on a range of essential child health commodities including ORS/zinc. CHAI, comments on a draft of this page.

• 6

  "Diarrhoea is defined as the passage of three or more loose or liquid stools per day (or more frequent passage than is normal for the individual). Frequent passing of formed stools is not diarrhoea, nor is the passing of loose, 'pasty' stools by breastfed babies." WHO, "Diarrhoeal disease," 2017.

• 7

  "Diarrhoea is usually a symptom of an infection in the intestinal tract, which can be caused by a variety of bacterial, viral and parasitic organisms. Infection is spread through contaminated food or drinking-water, or from person-to-person as a result of poor hygiene." WHO, "Diarrhoeal disease," 2017.

• 8

  "Diarrhoeal disease is the second leading cause of death in children under five years old, and is responsible for killing around 525 000 children every year [estimate as of 2017]. Diarrhoea can last several days, and can leave the body without the water and salts that are necessary for survival. In the past, for most people, severe dehydration and fluid loss were the main causes of diarrhoea deaths. Now, other causes such as septic bacterial infections are likely to account for an increasing proportion of all diarrhoea-associated deaths. Children who are malnourished or have impaired immunity as well as people living with HIV are most at risk of life-threatening diarrhoea." WHO, "Diarrhoeal disease," 2017

  We have been unable to verify this claim about the increasing share of diarrhea-associated deaths that are caused by septic bacterial infection. If we learn more about this, we may update our best guess about the share of diarrhea-related mortality that using ORS and zinc will avert.

• 9

  See IHME data.

• 10

  This includes indirect deaths associated with diarrhea. We expect that this may decline as rotavirus vaccine coverage increases in Bauchi because evidence suggests that rotavirus is responsible for a significant share of diarrhea-related illness among children under 5 in Nigeria. See Cohen et al. 2022, Supplementary Table 3, "Rotavirus" column, "Nigeria" row.

• 11
  • "Dehydration from diarrhoea can be prevented by giving extra fluids at home, or it can be treated simply, effectively, and cheaply in all age-groups and in all but the most severe cases by giving patients by mouth an adequate glucose-electrolyte solution. This way of giving fluids to prevent or treat dehydration is called oral rehydration therapy (ORT)." WHO, Oral Rehydration Salts report, 2006, p. 1.
  • The WHO defines oral rehydration therapy (ORT) as "The administration of fluid by mouth to prevent or correct the dehydration that is a consequence of diarrhoea" versus oral rehydration salt solution (ORS), defined in the same report as "Specifically, the complete, new WHO/UNICEF formula." WHO, Oral Rehydration Salts report, 2006, p. iv.

• 12
  • “Evidence shows that zinc is beneficial in managing acute or persistent diarrhoea in children ages 6–59 months, showing clinically important reductions in illness duration and severity.” UNICEF, Pneumonia and diarrhoea: Tackling the deadliest diseases for the world’s poorest children, 2012, p. 22.
  • “Dosage and duration
    • Child under 6 months: 10 mg (½ tablet) once daily for 10 days
    • Child from 6 months to 5 years: 20 mg (1 tablet) once daily for 10 days

  Place the half-tablet or full tablet in a teaspoon, add a bit of water to dissolve it, and give the entire spoonful to the child.” MSF medical guidelines, 'ZINC SULFATE Oral'.

• 13

  GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "VOI" sheet, "Total grant cost (millions)" row.

• 14

  "The ORS/zinc distribution would be using community members as campaigners to distribute ORS/zinc and provide educational information" CHAI, email to GiveWell, Sep 20, 2023 (unpublished).

• 15
  • “CNVs are generally women (due to cultural and religious sensitivities to enter households and sit with other women to provide education), have previous experience supporting similar campaigns, have some level of education, and live within the communities that they will be serving. CNVs are not a formal health cadre in Nigeria, but the role was created for the ANRIN project.” CHAI, comments in Running document - Givewell and CHAI - ORS and zinc (unpublished)
  • Note that despite being called “volunteers”, these workers receive payment. In a previous program, CHAI implemented "a performance-based system of payment whereby each CNV was paid NGN40 per intervention delivered, and we used a system of verification and validation to calculate their payment…. this is likely the same model we would use to hire and pay the distributors" CHAI, responses in "Running document - Givewell and CHAI - ORS and zinc" (unpublished)

• 16
  • "Once the distributors are recruited, their first activity is to map the households and register them into the program. The distributors conduct a walkthrough census whereby they visit each household, obtain bio-data for the entire household if there are any children under-5, and mark the house with paint or chalk/charcoal." CHAI, responses in "Running document - Givewell and CHAI - ORS and zinc" (unpublished)
  • "We are planning to do door-to-door distribution to every household with a child under-5." CHAI, responses in "Running document - Givewell and CHAI - ORS and zinc" (unpublished)
  • "We didn’t consider a voucher system. We are unsure this is going to be more effective than free distribution but believe it will introduce more complexity and potentially costs." CHAI, responses in "Running document - Givewell and CHAI - ORS and zinc" (unpublished)

• 17

  Based on adjusted GeoPoDe estimates. See our calculations here.

• 18

  "Will educational materials provided alongside ORS and zinc, and/or will distributors talk to caregivers at the point of distribution Or will there be educational sessions in community-wide meetings (if so, how will CHAI ensure that a sufficiently large number of caregivers turn up)?

  • Yes, distributors are trained on interpersonal communication and counseling skills to educate caregivers at the time of visit. For ANRIN, the distributors were assisted by talking points located on the reporting app. The talking points included how to prepare, use, and store ORS/zinc, benefits of the treatments, recommended health behaviors such as seeking care, counseling on how to prevent outbreaks of cholera, encouraging basic handwashing and hygiene practices.
  • Additionally, each household was given a flyer translated in the local language with these messages.
  • ANRIN also engaged community leaders to hold community-wide meetings to encourage care-seeking and acceptance of the intervention.
  • During the monthly re-supply and reconciliation meetings, CNVs would raise questions that came up during interpersonal meetings for group discussion.
  • We will very likely replicate these activities in Bauchi as well." CHAI, responses in "Running document - Givewell and CHAI - ORS and zinc" (unpublished)

• 19

  Learning from CHAI's Z/ORS pilot in Bauchi (unpublished)

• 20

  “[The external evaluator] would take a leading role in the evaluation design (protocol, planning, tool development) and implementation (randomization, survey sample selection, data collection agency selection and training, and field management). CHAI would remain informed and provide input.
  [The external evaluator] would fully and independently conduct the analysis based on a pre-analysis plan” CHAI, email to GiveWell, 22 August 2023 (unpublished).

• 21

  "We expect to incur some additional/redundant costs in order to roll out the campaign in only half the wards at a time for each study arm. For example, recruitment and training will need to occur twice – prior to each rollout in the study setting; however, for the base campaign scenario that reflects actual implementation, these costs would only be incurred once" CHAI, responses in "Running document - Givewell and CHAI - ORS and zinc" (unpublished).

• 22

  CHAI, slides for GiveWell, July 2023, "Design 2: clustered RCT with one campaign" slide.

• 23

  CHAI ORS Budget (Oct 25, 2023) (unpublished).

• 24
  • See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "CEA" sheet, "Cost-effectiveness in multiples of cash transfers, after all adjustments" row, "Bauchi, Nigeria" "Program costs during RCT period" column.
  • Note that a) our cost-effectiveness analyses are simplified models that are highly uncertain, and b) our cost-effectiveness threshold for directing funding to particular programs changes periodically. See GiveWell’s Cost-Effectiveness Analyses webpage for more information about how we use cost-effectiveness estimates in our grantmaking.

• 25
  • Our current CEA suggests that the program may be around 21 times as cost-effective as unconditional cash transfers at scale. See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "CEA" sheet, "Cost-effectiveness in multiples of cash transfers, after all adjustments" row, "Bauchi, Nigeria" "Program costs at scale" column.
  • In the program funded by this grant, distribution will happen in two staggered campaigns for research purposes (the control group will receive zinc and ORS six months after the treatment group). This staggered distribution means higher implementation costs due to (a) needing to recruit and train community nutrition volunteers twice, and (b) extended CHAI staff time required to support the planning and implementation recruitment and training. "1) Campaign: We expect to incur some additional/redundant costs in order to roll out the campaign in only half the wards at a time for each study arm. For example, recruitment and training will need to occur twice – prior to each rollout in the study setting; however, for the base campaign scenario that reflects actual implementation, these costs would only be incurred once. 2) CHAI TA: In addition, there is extended staff time required to support the planning and implementation (12 vs. 20 months for the base campaign and cRCT, respectively)." CHAI, Running document - Givewell and CHAI - ORS and zinc (unpublished).

• 26
  • ORS usage for children with diarrhea in Bauchi is estimated at 45.6%. See UNICEF, 2021 MICS survey, Table TC.3.3, percentage of children with diarrhea who received any ORS in Bauchi, p. 213
  • See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "CEA" sheet, “Effective direct diarrhea mortality rate per 1,000 children aged under-5” row.

• 27

  For more information on this, see our IR. We use a different study treatment arm (“free and convenient”, rather than “free”) for the CHAI program, because CHAI has proposed a model of broadly distributing ORS and zinc to all households in a region.

• 28

  For more information on this, see our IR.

• 29

  1,510,000 x 0.17 x 1.3% x 60%

• 30

  $4.15 million / 1.51 million.

• 31

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "CEA" sheet, "Units of value from development effects generated by hypothetical donation" row for calculations.

• 32

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "CEA" sheet, "Units of value from costs averted" row for calculations.

• 33

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "Simple CEA" sheet, "Child mortality benefits" row for calculations.

• 34

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "CEA" sheet, "Total adjustment factor for excluded effects" row for calculations.

• 35

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "CEA" sheet, “Leverage/Funging adjustment" section for calculations. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

• 36

  Our understanding is that minimal costs would be paid by the government in this case (comments from CHAI 29 August 2023). This is because the program would fund training, supervision, salaries of distributors, and commodity procurement. CHAI, Running document - Givewell and CHAI - ORS and zinc (unpublished).

• 37
  • "Our recommendation would be for the primary outcome to be ORS usage as this is the life-saving component and simpler to measure on its own, and to assess ORS+Zinc usage as a secondary outcome." CHAI, ORS-Zinc AT-HOME Trial Design - Discussion Points around Potential Updates (unpublished).
  • Our current estimates are based on one RCT in Uganda, that measured uptake one month after providing ORS: “One month after the interventions were implemented, we conducted an endline survey, in which we visited the 80 closest households to the CHW’s home on the list.” Wagner et al 2019. The implementation model in this program was different to ours, where in Uganda distribution was layered onto existing community health worker visits. In addition, the outcome measure in that trial was self-reported usage, and we hope that the research team can identify an objective measure in addition to self-reported outcomes during this trial.

• 38

  Learning from CHAI's Z/ORS pilot in Bauchi (unpublished).

• 39

  Learning from CHAI's Z/ORS pilot in Bauchi (unpublished).

• 40

  Learning from CHAI's Z/ORS pilot in Bauchi (unpublished).

• 41

  50% probability in the 8x scenario, 33% probability in the 10x scenario and 20% probability in the 12x scenario. See here.

• 42

  We assume for this model that there is around $28 million in annual room for more funding for ORS and zinc distribution programs across some other states in Nigeria, Cameroon, the Democratic Republic of the Congo, Senegal and Ethiopia. We have not investigated these locations in depth, but expect that there would be a number of cost-effective opportunities for funding support in other countries with low ORS usage.

• 43

  See here for full calculations. These estimates include a 10% assumption that the trial will “fail” in some way.

• 44

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "VOI" sheet, “Scenario 1: Value from directing more funding to program" section for calculations.

• 45

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "VOI" sheet, “Scenario 2: Value from directing less funding to program" section for calculations.

• 46

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "VOI" sheet, “Scenario 3: Value from no change to program funding" section for calculations.

• 47

  See GiveWell, Cost-effectiveness analysis of CHAI ORS/zinc distribution, 2023, "VOI" sheet, "Ad hoc adjustments” section for calculations. These adjustments are subjective, and account for the risk that we learn this program is above/below our bar and are wrong, the risk that this study would be funded without us, and the benefit to the broader research community.

• 48

  See our grant page for the CHAI Incubator here for more details.

• 49

  CHAI, email to GiveWell, Aug 22, 2023 (unpublished).

• 50

  We are not aware of any randomized studies of ORS in community settings. Munos, Walker, and Black 2010 is a frequently cited meta-analysis of non-randomized studies that estimates that ORS reduces mortality by around 93% in community settings:

  • The pooled estimate for the effectiveness of ORS interventions (from three studies reporting mortality, 68 total deaths in the studies) found a ~69% decrease in diarrhea mortality (95% CI 51% to 80%) with ~74% ORS coverage: "Diarrhea Mortality (n=3; 68 events). ORS reduces mortality by 69% (95% CI: 51-80%) given mean coverage of 74% (range 52-96%) (93% reduction with 100% coverage)" Munos, Walker, and Black 2010, Pg i80.
  • The authors estimate that 100% coverage would cause a 93% reduction in diarrhea mortality. "The mean and median coverage levels in the intervention arms of the diarrhoea mortality studies were 74%; assuming a linear relationship between coverage and mortality reduction, at 100% coverage a 93% relative reduction in diarrhoea mortality would be expected (Figure 2)." Munos, Walker, and Black 2010, Pg i78.

  See our discussion in our public report for more information.

• 51

  We make a 20% downward adjustment for internal validity because of the non-randomized nature of the studies and the limited available evidence. We also apply a 20% downward adjustment for external validity because we are very uncertain about how generalizable the studies’ results are. Overall, we estimate ORS with zinc lowers mortality from diarrhea by around 60%. See our discussion in our public report for more information.

• 52

  “CHAI has a strong and long standing relationship with the SMOH in Bauchi state. In 2015, CHAI worked closely with the SMOH in Bauchi on a ORS/zinc program which worked directly with the SMOH to forecast ORS/zinc demand, include ORS/zinc into state DRFs, and conduct demand generation and marketing activities to spur uptake of ORS/zinc.
  Currently, CHAI has a team based in the state who are providing support to the SMOH on several programs:

  • An oxygen project to establish an oxygen unit and a technical working group to develop policies and strategies focused on increasing oxygen access
  • An immunization program with the Primary Health Care Management Board
  • A community-based health management information systems project”

  CHAI and GiveWell, comments in Running document - Givewell and CHAI - ORS and zinc (unpublished).

• 53

  E.g., with the ANRIN project: “Local community members will be recruited to work as distributors. In the ANRIN project, we called them Community Nutrition Volunteers (CNVs). CNVs are generally women (due to cultural and religious sensitivities to enter households and sit with other women to provide education), have previous experience supporting similar campaigns, have some level of education, and live within the communities that they will be serving. CNVs are not a formal health cadre in Nigeria, but the role was created for the ANRIN project.” CHAI, comments in Running document - Givewell and CHAI - ORS and zinc (unpublished).

• 54

  See more on these uncertainties in our public report on ORS and zinc distribution.

• 55

  The RCT will aim to tell us the combined effect of (i) provision of ORS and zinc, and (ii) receiving a visit and some education from a community nutrition volunteer (CNV). We believe that this is the relevant model in many contexts, but it is possible that in other geographies it would be more appropriate to layer ORS and zinc provision onto existing community health worker visits.

• 56
  • Garba et al 2021 tested drinking water in Bauchi Metropolis, and found that “The calculated WQI [water quality index] indicates that 25% of water samples are excellent for drinking. 46.9% of the samples fall in good class of WQI.”
  • Jagaba et al 2020 tested water quality in hand dug wells in Rafin Zurfi, and found that “Based on the hand-dug well WQI’s, 6.67% of the samples indicated excellent water, 46.67% under the category for good water while 20% in the poor water category. Similar to excellent water, 6.67% signified very poor water while 20% of the samples were also under the water unsuitable for drinking purposes.”

• 57

  The WHO’s recommendation for exclusive breastfeeding includes exceptions for medications including ORS: “ “Exclusive breastfeeding” is defined as giving no other food or drink – not even water – except breast milk. It does, however, allow the infant to receive oral rehydration salts (ORS), drops and syrups (vitamins, minerals and medicines).” WHO, “Breastfeeding”, 2015.

• 58

  Based on unpublished conversations that GiveWell and CHAI have had with experts and clinicians, our current understanding is that providing ORS to children who need it, even when water quality may be poorer than recommended, is still the preferred choice. For instance: "Something to note that one of our clinical advisors who worked as a clinician in LMICs says: “in practice I would always give the ORS to a child who needs it, using the best water available—even water in facilities can be pretty yucky—on its own or because of the vessels. But little children with bad diarrhea can decline so quickly so giving ORS using the available water is the common choice in practice I would say.” CHAI, Comments in Running document - Givewell and CHAI - ORS and zinc (unpublished).

• 59

  See our public report on breastfeeding promotion for more information.

• 60

  “CNVs are trained to communicate specific talking points when distributing ORS/zinc. This includes demonstrating how to mix ORS and stressing the importance of using clean water, such as boiling water and letting it cool or buying water in a sachet. This message is communicated to caregivers of all children, including those who are exclusively breastfed. We also include messaging on prevention/danger signs/when to seek for care/referrals. CHAI linked CNVs to public health staff such as medical officers during the training sessions so that they could connect sick patients to the nearest public facility for referral.” CHAI, comments in Running document - Givewell and CHAI - ORS and zinc (unpublished)

• 61

  GiveWell: “Could we ask at endline: i) whether children under 6 months are EBF, ii) what water source they used if they gave ORS to the child (would it be easy to code a pre-set list of answer options that would allow us to distinguish safe from unsafe water sources?), iii) ask about proper use (e.g., did they boil water first)?”...

  CHAI: “This is certainly feasible and pretty standard in household questions. The only additional idea that comes to mind is whether we could do any on-spot water testing. Not sure if such tools exist to detect contaminated water or if we’d need to collect samples and send them to a lab for testing. It could be done just once for every cluster rather than every household…” Comments in Running document - Givewell and CHAI - ORS and zinc (unpublished).

• 62

  See our public report on chlorine vouchers for more information.

• 63

  CHAI: “We didn’t consider a voucher system. We are unsure this is going to be more effective than free distribution but believe it will introduce more complexity and potentially costs, such as printing vouchers, ensuring they are authentic and not photocopied when redeemed, etc..” CHAI, comments in Running document - Givewell and CHAI - ORS and zinc (unpublished).

• 64

  90% (sum of probabilities of 10% or higher) multiplied by a 10% risk of failure.

• 65

  69% (sum of probabilities of 20% or higher) multiplied by a 10% risk of failure.

• 66

  41% (sum of probabilities of 30% or higher) multiplied by a 10% risk of failure.

• 67

  Roughly based on estimated child population of Bauchi (1.54 million) x 60.5% share of children reached.

• 68

  Due to either lower diarrhea incidence since modeled estimates were last available, seasonality affecting measurement, some effect of self-reporting, or random chance.