Helen Keller International — Vitamin A Supplementation (January 2022)

Note: This page summarizes the rationale behind a GiveWell grant to Helen Keller International. Helen Keller International reviewed this page prior to publication.

Summary

In January 2022, GiveWell recommended a grant from several funding sources1 for $46.7 million to Helen Keller International for vitamin A supplementation (VAS). Helen Keller International expects to use this funding to support VAS campaigns in nine countries. These are countries where Helen Keller is currently supporting VAS campaigns, covering some but not all regions in each country. The grant would both extend the funding runway for this work out to 2024 and allow Helen Keller to expand to additional regions in 2022-2024. Helen Keller International is one of GiveWell's top charities.

We recommended this grant because we believe that the work that the grant will support will be cost-effective. VAS is among the most cost-effective programs we know of and we believe that there are funding gaps for supporting VAS in these nine countries.

Published: April 2022; Last Updated: October 2022

Table of Contents

Planned activities and budget

The size of this grant is based on Helen Keller's estimates of potential funding gaps for VAS in nine countries: Burkina Faso, Cameroon, Côte d’Ivoire, Democratic Republic of Congo, Guinea, Kenya, Mali, Niger, Nigeria.2 We have supported Helen Keller's work on VAS in each of these countries before.3 For each country, this grant would both extend funding through 2024 for the regions Helen Keller has supported previously and give Helen Keller the option to expand to additional regions within the country if there are additional funding gaps in 2022-2024.4 (With the exception of Nigeria, where the grant extends funding for current states to 2024 but not to additional states.)5 More details are in this spreadsheet.

For more on Helen Keller's role in VAS programs see here.

The case for the grant

The case for this grant is based largely on our estimate that this grant will meet our bar for cost-effectiveness, including after we have adjusted for the possibility of crowding out some funding from other sources. Our confidence in this assessment is increased by the fact that we have worked with Helen Keller International for several years on VAS and seen results from their work on this program (more in our review of Helen Keller's VAS program).6

A note on how we use cost-effectiveness estimates in our grantmaking

After assessing a potential grantee's room for more funding, we may then choose to investigate potential grants to support the spending opportunities that we do not expect to be funded with the grantee's available and expected funding, which we refer to as "funding gaps." The principles we follow in deciding whether or not to fill a funding gap are described on this page.

The first of those principles is to put significant weight on our cost-effectiveness estimates. We use GiveDirectly's unconditional cash transfers as a benchmark for comparing the cost-effectiveness of different funding gaps, which we describe in multiples of "cash." Thus, if we estimate that a funding gap is "10x cash," this means we estimate it to be ten times as cost-effective as unconditional cash transfers. As of this writing, we have typically funded opportunities that meet or exceed a relatively high bar: 8x cash. We also consider funding opportunities that are between 5 and 8x cash.

Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes to other grants we have made or considered making, and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

Cost-effectiveness

Based on our cost-effectiveness analysis of VAS in these nine countries, we believe the program is in the range of cost-effectiveness of programs we expect to direct funding to, as of 2022. Our cost-effectiveness estimates, in terms of X times as cost-effective as GiveDirectly's program:

Country Cost-effectiveness7
Burkina Faso 20
Cameroon 26
Côte d’Ivoire 11
DRC 16
Guinea 12
Mali 31
Niger 58
Nigeria 26
Kenya 8

We discuss our model of VAS cost-effectiveness on this page. The main questions we asked about that model as we considered this grant were:

  • What is the probability that another funder would fill these funding gaps if we do not? We discuss what we did and what we learned below. We made some small updates to these values. All values remained in a similar range as before: 10% to 35%.8
  • How might we make our external validity adjustment more robust? This parameter in the model seeks to capture differences in the characteristics of populations targeted by this grant relative to populations who participated in trials where the impact of VAS was estimated. To calculate the adjustment, we compare child mortality rates, vitamin A deficiency rates, and proportion of deaths due to measles, diarrhea and infectious diseases in current contexts and trial contexts.9 Steps we have taken recently to revisit this parameter:
    • We spoke with a researcher at the Institute of Health Metrics and Evaluation (IMHE) about how it generates the estimates of vitamin A deficiency that we use in our model.
    • We asked Megan Higgs, a statistical consultant, to provide a brief high-level evaluation of the Cochrane meta-analysis of VAS studies that we use in our model with an eye to two questions: the use of fixed vs. random effects models,10 and the relationship between the level of vitamin A deficiency and the impact of VAS on mortality. Dr. Higgs agreed that the random effects approach was preferable given expected heterogeneity in effects among the studies included (the effect of VAS on mortality is likely to vary with characteristics of a population such as the contribution of certain causes to the overall mortality rate), and suggested that we could possibly improve predictions of the effect in new contexts by combining individual study results in a way that puts more weight on study contexts most similar to what we would be funding. We have not yet taken that step. On the relationship between the level of vitamin A deficiency and effect of VAS on mortality, Dr. Higgs did a cursory evaluation and tentatively concluded that it would be difficult to estimate the interaction between vitamin A deficiency and VAS on mortality given limitations in data available (quantity and quality) to capture vitamin A deficiency for the included studies.
    • We asked Helen Keller International and several others who work on VAS what would be the most useful new research to fund to improve our external validity adjustment. Our understanding from these conversations is that the two main ways that experts recommend for assessing the value of VAS in current contexts (i.e. determining how much impact delivering VAS will have on child mortality) is with biomarkers (collecting blood samples to test for vitamin A deficiency) and consumption surveys (asking people what their children eat). These two survey types have sometimes given conflicting results, leading to debates among VAS experts about the optimal form of testing. For many countries, the most recent biomarker and consumption surveys are many years old and living standards may have since changed meaningfully.11 We are potentially interested in funding new biomarker and consumption surveys.

Funding landscape

The size of this grant is based on the possibility that funding for VAS from other sources in these nine countries will decrease compared to the last few years. Our understanding is this is driven by two main factors:

  1. Uncertainty about funding from Global Affairs Canada (GAC). Based on conversations with funders and implementers working on VAS, our understanding is that the primary sources of funding for VAS are domestic governments (generally in the form of in-kind contributions), GAC (through Nutrition International and UNICEF), UNICEF's flexible funding, and GiveWell-directed donations. Helen Keller told us that UNICEF's funding from GAC has been decreasing over the last few years. Its current GAC grant is scheduled to end in 2022 and may be renewed at a lower, similar, or higher amount. We're not aware of major recent changes to resources provided by domestic governments or UNICEF through its flexible funding.
  2. Fewer polio vaccination campaigns for VAS to piggyback on. Helen Keller reports that there has been a decrease in the frequency of polio vaccination campaigns in the last few years in the countries in which it works on VAS.12 VAS can be delivered more cheaply when it piggybacks on another campaign and historically polio campaigns were the chosen vehicle.13 With fewer opportunities to piggyback, costs of VAS campaigns have been and may continue to increase.

As part of assessing the funding landscape, we have spoken with:

  • Representatives of ministries of health for five of the nine countries
  • UNICEF representatives working with four of the nine countries
  • Nutrition International (central level)
  • Nutrition team at the Bill and Melinda Gates Foundation
  • Helen Keller International

We got a consistent picture of the funding landscape from these sources. Based on what we learned, we updated the values we use in our cost-effectiveness analysis for each country program for the likelihood that this grant funding will crowd out funding from other sources.14 The programs continued to meet our bar for cost-effectiveness after these adjustments.

We have also told Helen Keller that we intend for the funds to be flexible such that if funding needs in a country in a particular year are less than expected, it can move funds to where it sees a greater need, including the possibility of spending some of the grant in 2025 or later.

Risks and reservations

Our biggest reservations about this grant are:

  • Our biggest question about the cost-effectiveness of VAS is the extent to which results from ~30 years ago translate to today's contexts. The five studies that receive 80% of the weight in the meta-analysis of VAS that our cost-effectiveness analysis is based on were published in 1986-1993.15 We use an external validity adjustment to reduce the expected effect size in the nine countries supported by this grant. This adjustment is based on vitamin A deficiency rates and the proportion of deaths that are due to infectious diseases, measles, and diarrhea.16 We are continuing to work on the structure and inputs for the external validity adjustment and may substantially revise this in the future. In particular, the adjustment is highly sensitive to estimates of vitamin A deficiency that we have sourced from the Global Burden of Disease,17 and recent work to learn more about changes in these estimates has led us to reconsider whether to continue using the estimates. That work is ongoing.
  • This grant may be larger than it needs to be given uncertainty about what funding will be available from other sources. That may have one of two effects (or a combination of these). Ideally, Helen Keller will be able to use less of the funding from this grant in 2022-2024 in these nine countries and use the funding to extend its work beyond 2024 and/or use it to support VAS in other countries (in consultation with GiveWell). However, as discussed above, the grant may have the effect of causing other funders to give less to VAS and use that funding for programs that may be less cost-effective.
  • It is possible that by making this grant, we are setting the expectation for other VAS funders and for domestic governments that GiveWell funding will be available to fill future gaps. This belief may, in turn, lead them to direct funding that they would have directed toward VAS to other programs and services.

Plans for follow up

  • We will continue to have monthly calls with Helen Keller International. In particular, we will ask regularly about how it is navigating the funding landscape and working to avoid crowding out other funders.
  • We will continue to get annual updates on spending and number of children reached, which we will use to update the estimates in our cost-effectiveness analysis of the cost per child supplemented.
  • We will get at least annual updates on Helen Keller International's room for more funding, which we'll use, in combination with conversations with Helen Keller, to understand more about how the funding landscape is changing.

Internal forecasts

Confidence Prediction By time
40% Helen Keller International spends more than 20% of this grant after 2024. End of 2024
40% Conditional on updating our cost-effectiveness analysis to incorporate actual spending, numbers of children reached, and any other updates to the model, we conclude that more than 15% of this grant supported programs that were less than 8x cash. After grant funds are spent (conservatively by end of 2025, though could be longer)
15% Conditional on updating our cost-effectiveness analysis to incorporate actual spending, numbers of children reached, and any other updates to the model, we conclude that more than 15% of this grant supported programs that were less than 5x cash. After grant funds are spent (conservatively by end of 2025, though could be longer)

Our process

Our process for this grant relied heavily on (a) our prior work on modeling the cost-effectiveness of VAS, and (b) our monthly discussions with Helen Keller. For this particular grant, we:

  • Revisited the external validity adjustment in our cost-effectiveness analysis (see above)
  • Spoke to representatives from:
    • Institute of Health Metrics and Evaluation (IMHE)
    • Nutrition departments of the ministries of health in Burkina Faso, Côte d'Ivoire, Guinea, Mali and Nigeria
    • Global Affairs Canada's nutrition team
    • West and Central Africa UNICEF regional office
    • USAID
    • Nutrition team at Bill and Melinda Gates Foundation

For internal review, a Senior Researcher and three Program Officers who were not otherwise involved in the grant investigation gave feedback on the plan for investigating the grant and two of these individuals reviewed the final case for the grant.

Sources

Document Source
Daulaire et al. 1992 Source (archive)
GiveWell, "All Content on Helen Keller International's Vitamin A Supplementation Program," 2021 Source
GiveWell, "GiveDirectly," 2020 Source
GiveWell, "Helen Keller International's Vitamin A Supplementation Program," 2021 Source
GiveWell, "Recommendation to Open Philanthropy for Grants to Top Charities," 2019 Source
GiveWell, "Vitamin A Supplementation," 2018 Source
GiveWell, “Allocation of Funds Donated to the Top Charities Fund in Q1 2021,” 2021 Source
GiveWell, 2022 cost-effectiveness analysis version 3 Source
GiveWell, Helen Keller's spending on VAS campaigns, 2018-2020 Source
GiveWell, HKI VAS - information for country-level cost-effectiveness analyses [2021 - GBD 2019 mortality edits] Source
GiveWell, Room for more funding analysis for HKI (mass campaigns only) (public), 2021 Source
Helen Keller International, Room for more funding report, 2021 Source
Herrera et al. 1992 Source (archive)
Imdad et al. 2010 Source
Ross et al. 1993 Source
Sommer et al. 1986 Source
West et al. 1991 Source
Source URL: https://www.givewell.org/research/grants/Helen-Keller-International-vitamin-A-supplementation-January-2022