Helen Keller International — Vitamin A Supplementation (January 2023)

Funding landscape

Helen Keller's requests for funding for the countries we considered for this grant have generally been growing over time, which we think is because of:

• Fewer opportunities for VAS campaigns to piggyback onto polio campaigns. VAS can be delivered more cheaply when it piggybacks on another campaign, and historically polio campaigns were the chosen vehicle.26 As polio campaigns decreased in frequency over the last few years, costs of VAS campaigns increased.27 Polio campaigns may increase again due to an increase in polio cases, which may again change VAS implementation possibilities.28
• Uncertain and/or limited funding from other funders for existing programs. Based on conversations with funders and implementers working on VAS, our understanding is that the primary sources of funding for VAS other than GiveWell are domestic governments (generally in the form of in-kind contributions), Global Affairs Canada, and UNICEF. Global Affairs Canada, via grants to UNICEF, seems to be the primary other funder of VAS in countries with the highest child mortality rates, and we do not expect its future support to be significantly higher than its past support, though this is uncertain.29 We also expect funding for VAS from UNICEF, beyond its funding from Global Affairs Canada, to be limited, though we don't have a strong understanding of how UNICEF's country offices decide where and when to support VAS.30
• Limited funds from other funders available for expanding VAS campaigns. Most of this grant will fund programs that GiveWell previously funded,31 but Helen Keller will also be expanding significantly in DRC as a result of this grant, going from serving 6 provinces, which are home to 6.4 million children, in 2022, to 13 provinces, home to 11 million children, in 2025.32 Our understanding is that funding for VAS in DRC is very limited, with no funding outside of Helen Keller for large-scale VAS campaigns.33

  Based on what we learned about the funding landscape, we updated the values we use in our cost-effectiveness analysis for each country program for the likelihood that this grant funding will crowd out funding from other sources in the near-term. Previously, the likelihoods were in the range of 10-35% depending on the program, and after this update, most were decreased by about 5%.34 It is possible, though, that by continuing to fund VAS, we are setting expectations for other funders in the long-term that their support for VAS isn't necessary (see risks and reservations below).

Risks and reservations

• Risks of bias in monitoring. Helen Keller commissions post-event coverage surveys for VAS campaigns. We reviewed recent surveys when investigating this grant,35 and overall, both the coverage survey results and the survey methods were very similar to the last time we conducted this analysis.36 However, after reviewing Helen Keller's program monitoring and discussing it with them,37 two risks of bias have come up that we may not have thoroughly understood in the past:
  1. Helen Keller does a survey for only one of the two rounds of VAS each year, and surveys are more frequently occurring during the non-rainy season.38 Coverage rates may be substantially lower in the rainy season.
  2. Various decision-makers for campaigns know whether or not a campaign will have a coverage survey, and may know which districts will be surveyed.39 Both of these facts could bias coverage results upwards.

  We have brought up these concerns with Helen Keller. In response, Helen Keller has decided not to inform regional authorities in several countries about whether or not there will be a coverage survey in their region until after the campaign, and it may expand this practice to additional locations.40 We remain concerned that Helen Keller has typically conducted coverage surveys following one of the two VAS campaigns each year41 and decision-makers may be told or be able to predict which campaign will and will not have a coverage survey. We plan to continue to discuss mitigating these risks of bias (see plans for follow up below).

• Uncertainties about the impact of VAS in current contexts – Our current cost-effectiveness estimate relies heavily on a review of VAS studies, Imdad et al. 2017. The most heavily-weighted study in that analysis was done in 2013 and did not find a statistically significant effect on mortality.42 The five other studies that receive the most weight in the analysis are about 30 years old.43 Both these facts raise questions about the impact of VAS in current contexts. We have an external validity adjustment in our cost-effectiveness analysis to adjust for differences between trial contexts and the contexts in which these VAS programs will be implemented, and refining our external validity adjustments has been an ongoing area of research. We estimated new best-guess vitamin A deficiency rates (see above). We have taken some very early steps to consider funding a new trial of VAS and we think it is unlikely that we'll find a trial design that would be good value for money (i.e. a large enough chance of a major update in our views to be worth spending what it would take to get a sufficient sample size); that work is ongoing.
• Potential underestimation of VAS coverage through routine health services – We may be underestimating what coverage for VAS is or could be achieved through delivery through routine health visits. If more VAS is being delivered routinely than we think, we would be overestimating the extent to which Helen Keller is adding additional VAS coverage. We have updated our adjustment to account for this possibility (see above), but it is difficult to determine accurately what coverage rates would be without Helen Keller's intervention, and we are thus uncertain about this adjustment.
• Possible shift to hybrid campaigns may change cost-effectiveness. According to Helen Keller, the Ministries of Health in several countries that Helen Keller supports are interested in making changes to the way VAS is delivered, including putting more emphasis on delivering VAS to children when they visit health facilities and/or delivering VAS through established community health worker networks. There is a possibility that this could result in lower coverage (and also a possibility that this would reduce the cost of the program). Our best guess currently is that lower coverage is not a major concern. Helen Keller believes that, until health facilities can deliver high coverage, governments that have Helen Keller support will continue to implement twice annual door-to-door delivery to catch children who didn't receive VAS at a facility.44 We are planning to learn more about how delivery of VAS is changing through future conversations with Helen Keller and reviewing data on costs and coverage rates (see our plans for follow up).
• Risk of future crowding out – It is possible that by making this grant, we are setting the expectation for other VAS funders and for domestic governments that GiveWell funding will be available to fill future funding needs. This expectation may lead them to direct funding that they would have directed toward VAS to other programs and services.

Plans for follow up

• Annually, we will continue to receive narrative progress reports, coverage survey reports, and spending reports from Helen Keller.45
• We will also continue our monthly calls with Helen Keller to get informal updates. We will discuss strategies for mitigating risks of bias in campaign monitoring surveys and follow up about potential changes in government VAS implementation plans.
• We will request annual room for more funding updates from Helen Keller, which we'll use, in combination with conversations, to understand more about how the funding landscape is changing.

Internal forecasts

Our process

Our process for this grant relied heavily on (a) our prior work on modeling the cost-effectiveness of VAS, and (b) our monthly discussions with Helen Keller. For this particular grant, we also:

• Made updates to our cost-effectiveness analysis (see above).
• Spoke to representatives from:
  • The Global Affairs Canada Nutrition Team
  • The UNICEF Nutrition Team

For internal review, a Senior Researcher, Program Officer, and Senior Program Associate who were not otherwise involved in the grant investigation gave feedback on the plan for investigating the grant. A Program Officer and Senior Program Associate reviewed the final case for the grant and gave feedback prior to final grant approval by a Senior Program Officer and GiveWell's CEO.

Sources

• 1
  • "Deficiency of sufficient duration or severity can lead to disorders that are common in vitamin A deficient populations such as xerophthalmia (xeros = dryness; -ophthalmia = pertaining to the eye), the leading cause of preventable childhood blindness, anaemia, and weakened host resistance to infection, which can increase the severity of infectious diseases and risk of death." World Health Organization, "Global prevalence of vitamin A deficiency in populations at risk," WHO Global Database on Vitamin A Deficiency, 2009, p. 1.
  • "The term xerophthalmia encompasses the clinical spectrum of ocular manifestations of VAD, from milder stages of night blindness and Bitot’s spots, to potentially blinding stages of corneal xerosis, ulceration and necrosis (keratomalacia). . . . The stages of xerophthalmia are regarded both as disorders and clinical indicators of VAD, and thus can be used to estimate an important aspect of morbidity and blinding disability as well as the prevalence of deficiency. As corneal disease is rare, the most commonly assessed stages are night blindness, obtainable by history, and Bitot’s spots, observable by handlight examination of the conjunctival surface. Standard procedures exist for assessing xerophthalmia. Although night blindness and Bitot’s spots are considered mild stages of eye disease, both represent moderate-to-severe systemic VAD, as evidenced by low serum retinol concentrations, and increased severity of infectious morbidity (i.e. diarrhoea and respiratory infections) and mortality in children and pregnant women." World Health Organization, "Global prevalence of vitamin A deficiency in populations at risk," WHO Global Database on Vitamin A Deficiency, 2009, pp. 2-3.
  • "Vitamin A deficiency (VAD) impairs body functions and may cause death. Adverse health consequences may also include xerophthalmia (dry eyes), susceptibility to infection, stunting, and anaemia (Sommer 1996; Rice 2004)."Imdad et al. 2022, p. 7.

• 2
  • In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation). The quality of the available evidence for all-cause mortality was high, whereas for all other critical outcomes it was moderate to very low. The quality of the available evidence for outcomes in human immunodeficiency virus (HIV)- positive children was moderate for all-cause mortality." World Health Organization, "Guideline: Vitamin A supplementation in infants and children 6-59 months of age," 2011, p. 1.
  • One dose of 100,000 IU of vitamin A is recommended for infants aged 6 to 11 months of age, and a 200,000 IU dose of vitamin A is recommended for children 12 to 59 months of age every four to six months. Table 1, World Health Organization, "Guideline: Vitamin A supplementation in infants and children 6-59 months of age," 2011, p. 5.
  • WHO defines vitamin A deficiency to be of mild public health importance when rates of vitamin A deficiency (defined as a measure of serum or plasma retinol 0.70 µmol/l) among preschool-aged children or pregnant women are between 2% and 10%, moderate public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are between 10% and 20%, and severe public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are greater than or equal to 20%. World Health Organization, "Global prevalence of vitamin A deficiency in populations at risk," WHO Global Database on Vitamin A Deficiency, 2009, p. 8, Table 5.

• 3

  For more on Helen Keller's role in VAS programs, see here.

• 4We are adding 18 months of funding, rather than 1 year, to bring our funding in line with Helen Keller's budgeting process, which uses July-June fiscal years. Helen Keller International, conversation with GiveWell, October 11, 2022 (unpublished).
  
• 5

  See this summary in our room for more funding analysis, and a full discussion in Helen Keller International, Room for more funding narrative report, 2022.

• 6

  Table 14. Target children 6 to 59 months in Helen Keller supported regions in DRC, Helen Keller International, Room for more funding narrative report, 2022, p. 12.

• 7

  See here in our room for more funding analysis.

• 8We currently expect to direct funding to programs that are 10 or more times as cost-effective as unconditional cash transfers, or "10x."
  
• 9

  See our cost-effectiveness estimates here.

• 10See our cost-effectiveness estimate for Kenya here.
  
• 11We also expect to incorporate lower cost per child covered per supplementation round for Kenya, but Kenya still falls below our bar. See the updates we expect to make here, row 'Cost per child covered per supplementation round'
  
• 12 See the updates we expect to make here, row 'Treatment costs averted from prevention'
  
• 13See cost-effectiveness estimates with these edits incorporated here.
  
• 14For more information on the changes we made to our cost-effectiveness model, see our 2023 changelog here.
  
• 15More information about this parameter in the cell note here.
  
• 16For more details on this change, see our changelog here.
  
• 17See the impact of this change on our cost-effectiveness estimates here.
  
• 18The biggest uncertainty with our cost per supplement estimates are our estimates for DRC, Côte d'Ivoire, Kenya and Nigeria. Rather than use the cost per supplement data as calculated, we have decided to use the weighted average cost per supplement from the five other countries in the analysis, because we have uncertainties about the cost and coverage data in those countries. See here. We expect to update these estimates soon (see note above).
  
• 19For a full introduction to our approach to leverage and funging adjustments, see this blog post.
  
• 20Our country-specific funging estimates are based primarily on information from Helen Keller International, Room for more funding narrative report, 2022.
  
• 21The counterfactual value of spending by other philanthropic actors is our estimate of how valuable other actors' spending would be if they were not contributing to this program. See our estimates here.
  
• 22For more information about these changes, see our changelog here.
  
• 23See these parameters here.
  
• 24See the impact of our changes to the monitoring parameters on cost-effectiveness here.
  
• 25Vitamin A supplementation may be delivered through both routine (facility-based) systems and mass campaigns. We have heard of cases where coverage may be relatively high for 6-12 month-olds who may receive vitamin A supplementation when they visit clinics for routine vaccinations. This rough value is attempting to account both for (a) the possibility that some children receive vitamin A through campaigns supported by Helen Keller International when they recently received it through visits to clinics, and (b) that the children most likely to receive double treatment are the youngest in the age range (because 6-12 month-olds are more likely to visit clinics as part of receiving routine vaccinations) and this may be a group where vitamin A supplementation has a higher than average impact on mortality (since mortality tends to be highest in the youngest children). Examples:
  • Helen Keller International notes, "Vitamin A supplements in Kenya are administered to children aged 6-59 months through various delivery approaches. Throughout the year, children can access VAS in primary health care facilities, but this routine coverage only accounts for around 20 percent of children, essentially children below 12 months, as many caregivers do not bring their children to the health facilities after the end of the immunization contact points at one year of age." Helen Keller International, Room for more funding report, 2021, p. 21.
  • The 2021 Demographic and Health Survey in Madagascar reports (translated by Google), "Among last-born children aged 6–23 months living with their mother, 40% have received vitamin A supplements in the last 6 months [...] Between 2008–09 and 2021, the percentage of children aged 6–59 months who received vitamin A supplements decreased significantly, from 72% to 40%. This is mainly due to the change in mode of distribution of vitamin A, which went from 'campaign' mode to routine mode since 2021." Institut National de la Statistique (INSTAT) Antananarivo, Madagascar, Enquête Démographique et de Santé à Madagascar (EDSMD-V) 2021, 2022, p. 224.
  • In 2022, Helen Keller conducted coverage surveys in parts of three countries where there was no external support for VAS campaigns and where Helen Keller may support in the future. These were locations that GiveWell had not yet funded because they were thought to have stronger routine delivery of VAS—they are therefore not representative of all countries supported by Helen Keller. Preliminary coverage rates found in these surveys were 42%, 46%, and 52%. Helen Keller International, email to GiveWell, January 9, 2023 (unpublished).
  • Our understanding from Helen Keller is that most of the countries it works in to deliver VAS through campaigns have particularly weak health systems, so may have lower rates of health facility-delivered VAS than the examples above.

• 26
  • “When polio campaigns take place, the World Health Organization and the Global Polio Eradication Initiative cover the cost of the core teams, and VAS stakeholders 'only' have to support the addition of one distributor. When there is no polio campaign, VAS partners usually support two distributors per campaign, resulting in significantly higher costs.” Helen Keller International, Room for more funding report, 2021, p. 7.
  • For past Helen Keller VAS campaigns that were linked with polio campaigns, see "Table 2. Polio campaign organized in Helen Keller supported countries between 2018 and 2021 and links with VAS distribution" Helen Keller International, Room for more funding report, 2021, p. 7.

• 27"The frequency of polio campaigns organized in Helen Keller countries continues to decline (see Table 2). … In situations where VAS could not be “piggy-backed” on polio campaigns, we had to increase funding support for VAS in these countries." Helen Keller International, Room for more funding report, 2021, p. 7.
  
• 28"The Covid-19 pandemic continues to significantly impact health systems and the health status of populations. In the last two years, access to routine immunization for children has reduced dramatically, leaving millions of children unprotected. As a result, cases of wild polio have resurfaced in Sub-Saharan Africa, and cases of measles are increasing rapidly. Polio campaigns are likely to be organized by WHO in 2022 and 2023, once again changing the campaign landscape in many countries." Helen Keller International, Room for more funding narrative report, 2022, p. 2.
  
• 29This understanding comes from unpublished calls with UNICEF and Global Affairs Canada.
  
• 30"In 2021, Helen Keller International significantly increased its support for Vitamin A Supplementation (VAS) in multiple countries following the large reduction in funds available from one of the main VAS supporters--UNICEF. With support from GiveWell, Helen Keller was able to close part of the funding gap and distributed ~50 million capsules through semi-annual VAS campaigns in 2021. More recently, reductions in UNICEF VAS funding in 2022 were less severe overall, but remain significant in countries such as Kenya, Nigeria or Niger. Although UNICEF thought it could provide greater support to Guinea, Mali and Côte d’Ivoire in 2022, their actual level of support remains lower than in the previous years. The likelihood of increased UNICEF support in these countries in the near future is low. Filling UNICEF’s funding gaps has led Helen Keller to spend higher amounts of GiveWell funds than initially budgeted, causing estimated funding gaps in each country beginning in 2024 and creating 'room for more funding' for these countries.” Helen Keller International, Room for more funding narrative report, 2022, p. 2.
  
• 31See last year's grant to Helen Keller here: GiveWell, "Helen Keller International — Vitamin A Supplementation (January 2022)".
  
• 32Table 14. Target children 6 to 59 months in Helen Keller supported regions in DRC, Helen Keller International, Room for more funding narrative report, 2022, p. 12.
  
• 33
  • UNICEF is currently the only funder for VAS in DRC and its support is limited to a few thousand dollars per round; thus there is no support for large-scale campaigns: "In 2016, after Helen Keller pulled out of DRC, UNICEF remained the sole VAS support partner of the PRONANUT [DRC's National Program for Nutrition]; however, their funding limited the geographic coverage area for VAS. About this time, UNICEF and the PRONANUT started to integrate VAS into the primary health care system; however, VAS coverage through this approach remains extremely low (approximately 20 percent). The health system remains very weak in most parts of the country, with limited infrastructure, insufficient work force, supplies and equipment. Support provided by UNICEF to each province is limited to a few thousand US dollars per distribution round." Helen Keller International, Room for more funding report, 2021, p. 18.
  • “In February 2022, a workshop was organized by Helen Keller and UNICEF and the National Program for Nutrition (PRONANUT) to reflect on the campaigns and plan for the future of VAS in DRC. It was agreed between partners to continue to organize mop-up events at the end of each semester to ensure high coverage of children, while identifying and implementing cost effective solutions to increase routine coverage. PRONANUT officially requested Helen Keller to extend its support to 10 more provinces that are not currently being supported.” Helen Keller International, Room for more funding narrative report, 2022, p. 12.

• 34See previous likelihoods here and updated likelihoods here.
  
• 35See our most recent review of monitoring here.
  
• 36See our previous analysis here.
  
• 37See our most recent review of monitoring here.
  
• 38Helen Keller International, conversation with GiveWell, September 6, 2022 (unpublished).

• 39Helen Keller International, conversation with GiveWell, September 6, 2022 (unpublished).
  
• 40

  Helen Keller International, email to GiveWell, January 9, 2023 (unpublished).

• 41

  Most campaigns generally have coverage surveys after only one round of VAS, though in some years surveys were conducted for both rounds for some countries. See this spreadsheet, sheet "Comprehensiveness (2018-22)," column "Have we seen a coverage survey?"

• 42
  • The DEVTA 2013 study received approximately 62% of the weight in the analysis in Imdad et al. 2017. See Figure 3, p. 17.
  • The weight of each study in the analysis can be seen in Imdad et al. 2017, Figure 3, pg. 17.
  • The Deworming and Enhanced Vitamin A (DEVTA) was a large VAS trial in India, with results published in 2013. The results of DEVTA were published in Awasthi et al. 2013, which estimates that VAS reduced child mortality by 4% and cannot rule out the possibility that VAS did not affect child mortality at all (the 95% confidence interval ranged from a 3% increase in child mortality to an 11% decrease).
  • “Deaths per child-care centre at ages 1.0–6.0 years during the 5-year study (the primary trial endpoint) were 3.01 retinol versus 3.15 control (absolute reduction 0.14 [SE 0.11], mortality rate ratio [RR] 0.96, 95% CI 0.89–1.03, p=0.22), suggesting absolute risks of death between ages 1.0 and 6.0 years of approximately 2.5% retinol versus 2.6% control. Although this finding suggests that overall child mortality was 4% lower in vitamin A than in control blocks, this 4% reduction includes the possibility of no benefit and the possibility of appreciable benefit (95% confidence limit for reduction 11%).” Awasthi et al. 2013, p. 1473.

• 43
  • The five studies are: Ross et al. 1993, West et al. 1991, Herrera et al. 1992, Daulaire et al. 1992, and Sommer et al. 1986. For more details on these studies, see our page on VAS here.
  • The weight of each study in the analysis can be seen in Imdad et al. 2017, Figure 3, pg. 17.

• 44"Another significant change between 2021 and 2022 is the deliberate push from most country governments towards identifying, testing and implementing more sustainable and integrated delivery models for VAS, and moving away from a reliance on traditional campaigns that are often organized in silos. Hybrid delivery models that better integrate VAS in the health system’s components (i.e., information systems, supply chains) are being designed and tested in several countries in the coming years, including in Guinea, Mali and Niger. Such systems are already in place in Democratic Republic of Congo and Cote d’Ivoire. Building evidence for this transition is essential to ensure that a high proportion of at-risk children are being reached at an affordable cost. Effective VAS delivery often relies on a functioning community health worker system which is lacking in many countries. In addition, evidence is needed on how to reduce the costs of campaigns. To this end, Helen Keller conducted cost-effectiveness studies in Kenya and Burkina Faso in 2022. Based on these experiences, Helen Keller will develop a simple, user- friendly toolkit to more easily assess costs of campaigns on a regular basis. This information will allow countries to identify main cost drivers, examine their impact on coverage, and explore ways to reduce costs without significant effects on VAS coverage.” Helen Keller International, Room for more funding narrative report, 2022, p. 2.
  
• 45

  For recent examples, see Helen Keller International, Annual narrative report, April 2022, Helen Keller International, Annual coverage report, April 2022, and Helen Keller International, Expenditure report, April 2022.