Helen Keller International - Interim Review of Vitamin A Supplementation Program - October 2017 Version

We have published a more recent review of this organization. See our most recent report on Helen Keller International's vitamin A supplementation program.

Published: November 2017

Summary

What do they do? Helen Keller International (HKI) supports programs focused on reducing malnutrition and averting blindness and poor vision; this review only focuses on HKI's work on vitamin A supplementation (VAS) in sub-Saharan Africa. HKI provides technical assistance, engages in advocacy, and contributes funding to government-run vitamin A supplementation programs. (More)

Does it work? The evidence base for VAS programs is somewhat mixed, but our conclusion from the evidence is that VAS is likely effective at averting deaths among young children when implemented in areas with high baseline rates of child mortality. Most countries where HKI has recently supported VAS programs have child mortality rates that are as high or nearly as high as locations where significant effects of VAS have been found in randomized controlled trials. HKI has conducted studies to determine whether its mass distribution programs reach a large proportion of targeted children; these studies have generally found moderately positive results. We have a variety of major remaining questions on the effectiveness of HKI's VAS programs. (More)

What do you get for your dollar? Based on a preliminary cost-effectiveness analysis, it appears that HKI's vitamin A supplementation work may be in the range of cost-effectiveness of our top charities, but there are several key factors about which we have limited information. (More)

Is there room for more funding? HKI told us that it has substantial room for more funding for its VAS programs. We have not yet undertaken a careful review of the global funding landscape for this work, but our impression is that VAS has been relatively well-funded historically. The likelihood of other funders filling the gaps HKI has identified is a major remaining question for us. (More)

What are GiveWell’s next steps? HKI has successfully completed the first phase of our investigation process, and we have made a $100,000 grant to HKI (as part of our "top charity participation grants"). We now plan to continue our review process of HKI to try to answer our remaining questions.

Our investigation process

In April 2017, we invited Helen Keller International (HKI) to apply to be considered for a top charity recommendation for its vitamin A supplementation program. To date, our investigation of HKI has consisted of:

• Conversations with HKI staff.1
• Reviewing documents HKI shared with us.

What do they do?

Helen Keller International (HKI) supports programs focused on reducing malnutrition and averting blindness and poor vision in countries in Africa and Asia; it also provides vision screenings and distributes eyeglasses at schools in the United States.2

In this review, we focus only on HKI's vitamin A supplementation (VAS) programs, which operate in countries in sub-Saharan Africa.3 VAS programs supported by HKI aim to provide all preschool-aged children (aged 6 to 59 months) in areas where vitamin A deficiency (VAD) is a public health problem with vitamin A supplements two to three times per year.4 HKI supports these programs by providing technical assistance, engaging in advocacy, and contributing funding to governments implementing the programs.

What is vitamin A supplementation?

Vitamin A is an essential nutrient that serves many purposes in the body; in particular, the immune and visual systems require it to function properly.5 Essential nutrients must be obtained through diet since the body cannot produce them on its own.6

Vitamin A deficiency (VAD) can cause stunting, anemia, xerophthalmia (dry eyes, which can lead to blindness), increased severity of infections, and death.7 The World Health Organization (WHO) notes that VAD results from chronic low vitamin A intake from diets and that people who have diets containing few animal sources and little vitamin A-fortified food may be particularly susceptible to VAD.8 WHO estimates that VAD is most common in its Africa and South-East Asia Regions.9 Infants, children, and pregnant or lactating women with low vitamin A intake appear to have a particularly high risk of the negative health impacts caused by VAD.10

WHO notes that vitamin A from high-dose supplements can be stored in the liver and used as needed in the body for several months.11 To prevent childhood morbidity and mortality, WHO recommends vitamin A supplementation (VAS) every four to six months for all children aged 6 to 59 months in areas where VAD is a public health problem.12

More information about VAS is available in our vitamin A supplementation intervention report.

How are vitamin A supplements distributed and administered to preschool-aged children?

HKI-supported VAS programs in sub-Saharan Africa generally use one or more of the following methods to distribute vitamin A supplements to preschool-aged children:13

• Mass distribution campaigns: HKI supports delivery of vitamin A supplements through two different types of mass distribution campaigns:
  • National Immunization Days: VAS has been added to door-to-door mass campaigns for polio vaccination (often called "National Immunization Days") in many countries, which occur one or more times per year.14 Due to recent progress in polio elimination, many National Immunization Day programs have been eliminated or scaled down.15
  • Child Health Days: Countries that have eliminated or scaled down National Immunization Days have often transferred VAS programs to national "Child Health Days," biannual events providing a package of health interventions for preschool-aged children.16 Some Child Health Day programs implement outreach activities to encourage caregivers to bring their children to fixed sites to receive health services; others send healthcare workers door to door.17
• Routine delivery: HKI supports creating a "contact point" at six months of age for VAS in infants' immunization schedules.18 In countries implementing a six-month contact point for routine delivery of VAS, caregivers can take their infants to health facilities to receive VAS when the infants turn six months old (rather than waiting for the next mass campaign to occur).19 HKI also supports routine delivery for older infants and children under 5 (in which caregivers take their children to facilities to receive VAS every six months).20

We would guess that the implementation of the distribution methods discussed above varies somewhat from country to country in HKI-supported programs, but we have not yet investigated these differences in depth.

Health workers implementing VAS programs are expected to cut vitamin A capsules open with scissors and squeeze the contents of the capsules directly into children's mouths.21 We have not yet investigated how often these instructions are followed in practice in different HKI-supported programs.

What does HKI do to support VAS programs?

HKI provides the following types of support to government-run VAS programs:22

• Technical assistance: HKI assists governments with monitoring and evaluation,23 training health workers and managers,24 policy design,25 program supervision,26 planning and budgeting,27 and demand generation28 for VAS programs (details in footnotes). HKI mainly focuses on providing technical assistance at the sub-national level; UNICEF and Nutrition International (formerly Micronutrient Initiative) often provide additional technical assistance for VAS programs at the national level in countries where HKI works.29
• Advocacy: HKI encourages national governments to prioritize budgeting for and implementing VAS mass campaigns,30 and advocates for routine distribution of vitamin A supplements through health facilities.31
• Funding: HKI also provides grants to governments to cover a portion of the implementation costs of VAS programs.32

Our understanding is that HKI's specific activities vary considerably by country; we have not yet investigated these differences in depth.33

HKI's spending on VAS programs

Global Affairs Canada has provided around $80 million CAD to HKI for its VAS programs in sub-Saharan Africa since 2006.34 Most recently, Global Affairs Canada granted $29 million CAD to HKI to support VAS programs in thirteen countries in sub-Saharan Africa between 2013 and 2016.35

We have seen a full breakdown of how much funding from the most recent grant HKI spent in each country, and a breakdown of what general types of activities within countries the funding was used or budgeted for (e.g., monitoring and evaluation) for the second and third years of the grant. See this spreadsheet for details.

Our understanding is that HKI has not used substantial amounts of funding from sources other than Global Affairs Canada for its recent VAS programs.36

Does it work?

Trials of VAS conducted in the 1980s and 1990s found that VAS greatly reduces child mortality, but a more recent trial with more participants than all previous studies combined (the Deworming and Enhanced Vitamin A, or DEVTA, trial) did not find a statistically significant effect. We remain uncertain about what could explain this difference in results. We believe the results of the recent trial raise questions about what impact we should expect from VAS programs today.

One plausible explanation for the difference between the results of DEVTA and previous trials is that the population studied in DEVTA had lower baseline child mortality rates than many previously studied populations, increasing the probability that mortality that may have been averted by VAS in worse-off populations would already have been averted through other means (e.g., increased immunization rates) in the DEVTA population. Some researchers have also suggested that DEVTA did not find a statistically significant mortality effect because it actually reached fewer children than reported with vitamin A supplements, or because of methodological flaws in the study.

To estimate what effect we should expect from VAS in locations where HKI supports VAS programs, and to evaluate HKI's track record at expanding access to VAS, we have considered the following questions (see sections below for our answers):

• Is HKI supporting VAS programs in areas where child mortality is high?
• How prevalent is vitamin A deficiency in areas where HKI works? Has vitamin A fortification (adding vitamin A to common foods) reduced the impact of supplementation?
• Is there evidence that a large proportion of targeted children receive vitamin A supplements?
• How does HKI's support affect program outcomes?

Is there independent evidence that the program is effective?

In brief:

• A meta-analysis of trials conducted in the 1980s and 1990s found that VAS reduces child mortality by 24% with a 95% confidence interval ranging from 17% to 31%.37 The Deworming and Enhanced Vitamin A (DEVTA) study, a more recent trial in India with around one million participants, estimates that VAS reduced child mortality by 4% and cannot rule out the possibility that VAS did not affect child mortality at all (the 95% confidence interval ranged from a 3% increase in child mortality to an 11% decrease).38 If the underlying effect of VAS on child mortality were the same in populations studied in DEVTA and in the previous VAS trials, random chance alone would be unlikely to account for such disparate results.39 A meta-analysis combining DEVTA with previous trials estimates that VAS causes a 12% reduction in child mortality with a 95% confidence interval ranging from 7% to 17%.40
• We are uncertain about what could explain why the earlier trials and DEVTA found such different results. Some potential explanations include:
  • The population treated by DEVTA had lower baseline child mortality rates and may have had better overall health than many previously studied populations.41 Deaths averted by VAS in worse-off populations may have already been averted through other means (e.g., increased vaccination rates) in the DEVTA population.42 This hypothesis is undermined somewhat by the apparent lack of a correlation between how much mortality risk was reduced and baseline mortality rate in non-DEVTA trials.43
  • Some researchers not involved in the study have pointed to evidence suggesting that DEVTA may have failed to achieve as high a coverage rate as it reported.44
  • DEVTA may have had methodological weaknesses that caused it to fail to detect a statistically significant mortality effect, even if VAS had a real effect on mortality rates in the population studied (details in footnote).45

For more details, see our vitamin A supplementation intervention report. A shorter summary of our views is available in our blog post on vitamin A supplementation programs.

Are programs targeted at areas where they are likely to be effective?

Is HKI supporting VAS programs in areas where child mortality is high?

VAS programs may have a limited impact on child mortality rates if baseline rates are relatively low (see discussion of VAS evidence base above).

The terms of Global Affairs Canada's 2013-2016 grant to HKI state that only countries that had relatively high under-five mortality rates would receive VAS program support under the grant.46 We estimate that in 2015, eight out of the thirteen countries supported by HKI under this grant had mortality rates for children aged 6 to 59 months higher than 10.6 per 1,000 child-years—the lowest baseline rate among major trials of VAS that found a significant impact on mortality. The average mortality rate in HKI-supported countries (weighted by spending in 2013-2016) was 11.0 deaths per 1,000 child-years.47 See this spreadsheet for details.

There are major limitations to our analysis so far of baseline child mortality rates in areas where HKI works. In particular:

• We have only reviewed data on national average child mortality rates. To evaluate the impact of HKI's sub-national technical assistance work, it would be more appropriate to use regional or local mortality data. We have not yet looked into whether reasonably high-quality data exists at the sub-national level; our impression is that even national-level data may be fairly flawed.
• It seems unlikely to us that there is a real baseline mortality rate "threshold" for the effectiveness of VAS (i.e., that VAS has an impact on child mortality when baseline rates are above 10.6 per 1,000 child-years, but no effect when baseline rates are below 10.6 per 1,000 child-years). We compare baseline mortality rates in areas where HKI works to 10.6 per 1,000 child-years because it may be useful as a general indication of where VAS programs are more or less effective at reducing mortality.
• We are comparing mortality rates from 2015 with control-group mortality rates from VAS trials. This is a flawed comparison because VAS programs may have contributed to a reduction in mortality rates to their 2015 levels.
• To date, we have only investigated all-cause child mortality rates in areas where HKI works. It may also be useful to compare the proportion of recent all-cause mortalities for which VAS could plausibly make a difference (e.g., deaths from diarrhea and pneumonia but not from accidents) in these countries and in populations studied in trials of VAS. If we find that, for example, infectious disease mortality now makes up a much smaller proportion of all-cause child mortality than it did in populations studied in trials of VAS, we may conclude that we should expect the impacts of current VAS programs to be smaller than the impacts observed in VAS trials.

How prevalent is vitamin A deficiency in areas where HKI works?

It appears unlikely that low rates of vitamin A deficiency (VAD) explain the DEVTA results, but it is still plausible that low rates of VAD in an area may indicate that VAS programs will have a limited effect on mortality there.48 (See footnote for arguments on ways in which VAD rates may not be indicative of the impact of VAS on child mortality; we have not yet evaluated these arguments carefully.)49 Trials of VAS included in the Cochrane review Imdad et al. 2017 were conducted in areas where VAD was a recognized concern, but the amount of specific information we have on baseline VAD rates in populations studied in the trials varies (details in footnote).50

We remain highly uncertain about the prevalence of VAD among preschool-aged children in areas where HKI works. To date, we have considered the following sources of information to learn more about VAD rates in areas where HKI works:

• Vitamin A deficiency surveys: The prevalence of VAD in a population can be estimated using representative surveys of serum retinol concentrations (measured by blood tests) or clinically assessed eye signs of VAD (e.g., Bitot's spots).51 So far, we have only searched for serum retinol surveys in areas where HKI works, since our understanding is that these surveys are more common than surveys assessing eye signs of VAD, especially for young children.52 We have listed the most recent serum retinol surveys of preschool-aged children in countries where HKI supports VAS programs in this spreadsheet ("VAD surveys" sheet). In most countries, surveys found high rates of VAD (above the 20% prevalence threshold for a problem of "severe public health importance" set by WHO), but these surveys occurred 15 to 20 years ago.53
• Stevens et al. 2015 incorporates the most recent available VAD surveys and other relevant information (e.g., availability of animal-source foods) into a mathematical model to estimate rates of VAD as of 2013.54 We have not carefully reviewed the methodology used in this paper. Stevens et al. 2015 concludes that VAD was likely to be high (above 40%) in 2013 throughout sub-Saharan Africa.55 Two more recent vitamin A deficiency surveys from Sierra Leone and Malawi found considerably lower rates of VAD among preschool-aged children than the lower bound estimate for sub-Saharan African countries in Stevens et al. 2015.56
• Food fortification and other vitamin A programs: We note that all countries with HKI-supported VAS programs also have policies encouraging the production and consumption of biofortified crops and mandating fortification of vegetable oil with vitamin A.57 We have not yet investigated whether these policies have resulted, in practice, in higher vitamin A intakes.
• Conversations with HKI: We have discussed our concerns about the lack of recent data on vitamin A deficiency with HKI. HKI told us that it believes it would be very surprising if vitamin A deficiency were no longer a problem throughout sub-Saharan Africa, especially in countries with high child mortality and malnutrition rates.58

Are vitamin A supplements delivered to and ingested by recipients?

We have seen evidence suggesting that a large proportion of targeted children are usually reached through HKI-supported mass distribution campaigns for VAS. The evidence we have seen on coverage rates achieved through routine distribution systems is much more limited.

Mass distribution campaigns

HKI collaborates with governments to implement post-event coverage surveys (PECS) following VAS mass distribution events.59 In each of the surveys, surveyors visit a sample of households and ask parents (or other caregivers) whether preschool-aged children in their household took vitamin A supplements during the recent campaign.60 HKI sent us documents on the general guidelines used to design these surveys, as well as some example reports and academic papers on specific coverage surveys.61 We have summarized the details of the methodologies of the recent coverage surveys we have seen in this spreadsheet ("Methods" sheet).62 The methodologies of these surveys generally appear to be strong, but we note that we have reviewed detailed reports from only two of HKI's recent surveys so far.63

Our main concern about coverage surveys of this type (similar to surveys for other mass distributions) is whether what caregivers say is likely to be accurate—their ability to recall the information may be flawed or their responses may be biased toward what they believe surveyors want to hear. Time between distribution and survey may be important for accurate recall; HKI's guidelines for coverage surveys state that surveys should take place within six weeks of the distribution but we have limited information on what has been done in practice.64 There may be ways to check the accuracy of caregivers' recall (e.g. vaccination coverage surveys are often able to check child health cards), but we are not aware of a method for doing this for VAS mass distributions.65

HKI's post-event coverage surveys have often found lower coverage rates than administrative data (which is based on tally sheets submitted by distributors).66 We have summarized the results of recent coverage surveys we have seen in this spreadsheet ("Results" sheet). Overall, coverage was above 80% (HKI's target) in around 60% of HKI's recent coverage surveys.67

We estimate that we have seen results from coverage surveys that cover around 13% of vitamin A supplements delivered with HKI support between 2013 and 2016 (details in footnote).68 We do not have complete information on how HKI or its partners decided which distributions to conduct surveys for; this information would be helpful for evaluating whether the selection criteria are likely to result in survey results that are representative of HKI's work overall or may, deliberately or unintentionally, target areas with high or low coverage rates.69 For example, hard-to-reach areas could be both less likely to be surveyed than other areas and have disproportionately low coverage rates, or coverage surveys may be focused on areas where program staff suspect that the program failed to achieve high coverage rates.70 We use coverage survey results in our cost-effectiveness analysis to estimate the number of children reached. We also look at past coverage surveys to determine whether HKI has a track record of monitoring its work such that, if we recommend additional funding to HKI for VAS, we could expect to learn whether the additional work was effective.

Routine delivery

HKI supports creating a "contact point" at six months of age for VAS in infants' immunization schedules (see above for details).

We currently have a limited understanding of how HKI measures coverage rates for these programs. We have seen some results from one set of surveys before and after the six-month contact points were implemented, but we are unsure whether HKI and governments continue to implement these surveys beyond the program's pilot phase.71

How does HKI's support affect program outcomes?

HKI supports government-run VAS programs by providing technical assistance, engaging in advocacy, and contributing funding (see above for details). We have not yet done an in-depth investigation into how HKI's support affects programs; we outline our initial thoughts here and list our remaining questions below.

HKI's support may be impacting the outcome of VAS programs in the following ways:

• Increasing coverage rates in VAS mass campaigns: HKI identifies districts or regions participating in mass campaigns that have low VAS coverage rates and provides sub-national governments in those regions with technical support.72 Specifically, HKI has pointed to two examples of countries in which it worked with governments to raise coverage rates: Cameroon, where VAS coverage increased from around 50% to around 80% over a few years, and Kenya, where coverage increased from 10% to 70% after HKI piloted the use of Early Child Education Centers to distribute VAS.73 We have not yet prioritized carefully reviewing the examples HKI sent, or assessed the extent to which the successful outcomes in these programs can be attributed to HKI. We expect that we would not find these example cases convincing on their own, but expect to consider this question in more depth in the next phase of our review. Ideally, we would like to see more comprehensive information about HKI's successes, and more counterfactual examples of VAS coverage in similar areas that did not receive support from HKI (see remaining questions on HKI's track record below).
• Preventing coverage losses when transitioning to routine distribution: HKI has told us that countries are at risk of large drops in VAS coverage rates as polio vaccination campaigns scale down (since VAS is often delivered through National Immunization Day campaigns).74 HKI has sent us examples of cases in which it provided support for VAS programs transitioning to routine distribution; we have not yet reviewed these cases in depth.75
• Causing campaigns or other VAS programs to occur: In the absence of external technical assistance and funding contributions, HKI told us it is possible that VAS campaigns and programs would not occur at all in some countries.76
• Decreasing the age at which infants receive their first vitamin A supplement: Successful implementation of routine distribution at six-month contact points would allow many infants to receive their first dose of VAS at a younger age than they would if they had to wait until the next mass campaign.77 A study by HKI staff found that routine distribution of VAS at six months could reduce child mortality by an additional 1.95 percentage points compared with VAS delivered only through mass campaigns; we have not yet carefully reviewed this study.78

Are there any negative or offsetting impacts?

Some preschool-aged children experience side effects after taking vitamin A supplements, including loose stools, headache, irritability, fever, nausea, and vomiting.79 WHO cites an estimate of the prevalence of these types of side effects of 1.5% to 7%; we have not vetted this estimate.80

Chronic excessive vitamin A intake can cause a serious condition called vitamin A toxicity (also known as hypervitaminosis A), but this condition is very rare globally.81 Our understanding is that VAS programs are not thought to cause vitamin A toxicity (except perhaps in case of mistaken overdose), but we have not investigated this issue in depth.82

We have not yet investigated whether there are other potential negative impacts of VAS programs.

What do you get for your dollar?

Cost per vitamin A supplement delivered

For our cost-effectiveness analyses, we prefer to include all costs incurred to carry out a project, not just those that the charity in question pays for itself. We believe that this method gives the best view of what it costs to achieve a particular impact (such as saving a life) and also avoids the lack of clarity and complications of leverage in charity.

HKI sent us two estimates of the cost to deliver a vitamin A supplement in programs it supports (see footnote for details).83 We adjusted these two cost per supplement estimates to account for additional costs and for likely overestimates of coverage rates in administrative data. Our adjusted estimates are:

• $0.31 per supplement delivered in Kasai-Oriental, Democratic Republic of the Congo (DRC) in Winter 2015; and
• $0.85 per supplement delivered in Littoral Region, Cameroon in Fall 2013.

For full calculations, see this spreadsheet. The cost per supplement estimates above are from HKI-supported mass distribution campaigns; we do not yet have enough information to estimate the cost to deliver a supplement through routine systems.

HKI also sent us information on its total spending from its recent Global Affairs Canada grant between 2013 and 2016, total numbers of treatments delivered, and some information on costs covered by other actors in the second year of the program, but we were not able to use this information to calculate a cost per treatment estimate accounting for contributions from all actors (details in footnote).84

Our adjusted cost per supplement estimates may not be fully comparable to cost analyses for our current top charities. Since these estimates only cover a single supplementation round, they may not be comparable to cost analyses over longer time periods or estimates that we have vetted more thoroughly.85

Cost-effectiveness

Based on a preliminary cost-effectiveness analysis, it appears that HKI's vitamin A supplementation programs may be in the range of cost-effectiveness of our priority programs, but there are several key factors about which we have limited information.

Our preliminary cost-effectiveness model is here (make an editable copy here). An overview of some of the key assumptions and limitations of our cost-effectiveness analysis (CEA) is in the following footnote.86

There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

Is there room for more funding?

HKI told us that it has substantial room for more funding for its VAS programs. We have not yet undertaken a careful review of the global funding landscape for this work, but our impression is that VAS has been relatively well-funded historically. The likelihood of other funders filling the gaps HKI has identified is a major remaining question for us.

HKI's funding gap

HKI roughly estimates that it has a funding gap of around $6 to $7 million per year for its VAS programs in sub-Saharan Africa.87

Since 2006, HKI's VAS programs have primarily been funded by Global Affairs Canada.88 Most recently, Global Affairs Canada granted $29 million CAD to HKI to support VAS programs in 13 countries in sub-Saharan Africa between 2013 and 2016.89 HKI submitted a concept note to Global Affairs Canada for the continuation of these programs between 2016 and 2021, but HKI did not receive a new grant.90

Global Affairs Canada has also made grants to UNICEF and Nutrition International to support VAS programs in sub-Saharan Africa.91 HKI told us that, going forward, Global Affairs Canada planned to grant funds only to UNICEF for VAS programs in order to ease administrative burdens;92 additionally, it appears that Global Affairs Canada is now granting considerably less funding per year for VAS programs in total than it did between 2013 and 2016.93

HKI expects to receive some funding through UNICEF for seven out of the thirteen countries in which it supported VAS programs between 2013 and 2016. HKI told us that it does not have any funding available for VAS programs in the other six countries.94

HKI told us that it has received limited interest from other potential funders for closing its funding gap for VAS programs.95 If HKI received additional funding, it would prioritize providing more support to VAS programs transitioning from door-to-door distribution to routine delivery.96

Major remaining questions

We have spent significantly less time investigating HKI and have substantially less insight into HKI's activities and track record than we do for our current top charities. As such, we have a number of remaining, high-level questions about its work supporting VAS programs.

On cost-effectiveness:

• What are the child mortality rates in the specific, sub-national regions where HKI has focused its technical assistance? If HKI had additional funding, would it support regions with similar child mortality rates?
• What proportion of child deaths in areas where HKI works are due to causes that might be affected by VAS (e.g. infectious diseases)? How does this proportion compare to the control groups of the VAS trials?
• Have vitamin A food fortification and/or crop biofortification programs successfully reduced VAD in countries where HKI works?
• To what degree is prevalence of VAD a useful predictor of the impact of VAS on mortality?
• Are there any additional costs of VAS programs (e.g., in-kind government contributions) that we have not yet accounted for in our cost analyses?
• Is there enough information available for us to be able to estimate total costs per supplement delivered over time periods longer than a single supplementation round?
• Do any HKI-supported programs aim to provide three rounds of VAS per year?
• To what extent would HKI reallocate funding from other sources away from VAS if it received additional funding restricted to VAS?

On track record:

• To what extent should we attribute success in HKI-supported VAS programs to HKI?
  • Have districts or regions that have received technical support from HKI for mass campaigns or routine distribution achieved higher coverage rates than similar districts or regions without HKI support?
  • Have districts or regions that have received technical support from HKI for implementing six-month contact points achieved higher coverage rates among six-month-old infants than other similar districts or regions without HKI support?

On monitoring:

• To what degree are the coverage survey results we have seen representative of all of HKI's work on mass distribution programs?
• What proportion of recent HKI-supported treatments have been delivered through routine distribution systems? What proportion of treatments would be delivered through routine systems in areas where HKI would like to use additional funding?
• How does HKI plan to monitor coverage rates for routine distribution systems over the long term?

On room for more funding:

• What specific types of work and locations would HKI prioritize if it had additional funding?
• If HKI does not receive additional funding for VAS programs, to what extent will other organizations (e.g., UNICEF and Nutrition International) use funding to fill the same roles in VAS programs that HKI has filled in the past?

We also list our additional remaining questions on the evidence base for VAS in our intervention report.

Sources

Document	Source
Affiche déparasitage SASNIM 1 2013 - Cameroon	Source
Affiche integree SASNIM1 2013 - Cameroon	Source
Aguayo and Baker 2005	Source (archive)
Aguayo et al. 2005	Source
Awasthi et al. 2013	Source (archive)
Beaton et al. 1993	Source (archive)
Benn 2017	Source (archive)
Benn, Fisker, and Aaby 2013	Source (archive)
Canada Privy Council Office Machinery of Government Changes 2015	Source (archive)
CBC News 2013	Source (archive)
Child health card - DRC	Source
Child health card - Senegal	Source
Child health card - Sierra Leone (back)	Source
Child health card - Sierra Leone (front)	Source
Clohossey et al. 2014	Source (archive)
Dhillon et al. 2013	Source (archive)
Engle-Stone et al. 2017	Source (archive)
Fiedler et al. 2008	Source (archive)
GAVA regional workshop Dakar 2016	Source
GAVA VAS regional symposium report 2016	Source
GAVA website homepage	Source (archive)
GBD 2015	Source (archive)
GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013	Source
GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017	Source
Grieg and Neufeld 2012	Source
Growth monitoring card - Sierra Leone	Source
Herrera et al. 1992	Source (archive)
HKI 6-month contact point presentation Côte d'Ivoire (French) 2013	Source
HKI 6-month contact point presentation Senegal (French) 2013	Source
HKI 6-month contact point presentation Senegal 2013	Source
HKI 6-month contact point presentation Sierra Leone 2012	Source
HKI 6-month contact point protocol - Côte d'Ivoire	Source
HKI 6-month contact point protocol - Senegal	Source
HKI 6-month Contact Point Protocol - Tanzania	Source
HKI 6-month Contact Point Research Guide - Tanzania 2012	Source
HKI 6-month contact point sample monthly report to health posts - Senegal (French)	Source
HKI 6-month contact point SMS lessons learned detailed memo Senegal 2013	Source
HKI 6-month contact point SMS lessons learned summary memo Senegal 2013	Source
HKI 6-month contact point standard methodology	Source
HKI Action Points Based on VAS Evaluation Report 2015	Unpublished
HKI cost-effectiveness analysis of VAS in DRC (French) 2016	Source
HKI country situation matrix for VAS	Unpublished
HKI country-level technical support related to vitamin A supplementation	Unpublished
HKI DRC cost-effectiveness presentation 2016	Source
HKI Drops of Life (French) 2007	Source (archive)
HKI Drops of Life 2007	Source (archive)
HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015	Unpublished
HKI grant agreement VAS Project 2013 - 2016	Unpublished
HKI integrated 6-month contact point pilot project report Mozambique 2015	Source
HKI lessons learned on VAS in six urban health districts in Cameroon 2014	Source
HKI mobile data collection presentation 2014	Source
HKI monitoring tools: multiple use of LQAS for VAS	Source
HKI monitoring tools: quality of care checklist for supportive supervision of VAS services	Source
HKI monitoring tools: supporting the supply chain for the provision of VAS services	Source
HKI monitoring tools: VAS community assessment guide	Source
HKI monitoring tools: VAS sustainability assessment checklist	Source
HKI performance monitoring framework for VAS	Source
HKI post-event coverage survey data analysis manual 2014	Source
HKI post-event coverage survey manual 2014	Source
HKI post-event coverage survey report Côte d'Ivoire (French) 2012	Source
HKI post-event coverage survey report Nigeria - Ekiti and Katsina states 2014	Source
HKI post-event coverage survey report Sierra Leone 2013	Source
HKI post-event coverage survey report Tanzania 2015	Source
HKI poster - nutrition - Nigeria	Source
HKI poster - SIAN vitamin A supplementation - Mali	Source
HKI poster - vaccination and supplementation calendar - Cameroon	Source
HKI poster - VAS administration - Mali	Source
HKI poster - vitamin A - DRC	Source
HKI poster - vitamin A supplementation - Nigeria	Source
HKI poster - vitamin A supplementation - Tanzania	Source
HKI poster: VAS and vaccination calendar - Senegal	Source
HKI poster: VAS at 6 months - Niger	Source
HKI poster: VAS at 6 Months (poster 1) - Senegal	Source
HKI poster: VAS at 6 months (poster 2) - Senegal	Source
HKI poster: Vitamin A Rich Foods - Senegal	Source
HKI presentation abstract: Reaching the hard to reach with vitamin A supplementation in low-performing health zones of DR Congo	Source
HKI presentation abstract: Routine delivery of Vitamin A Supplementation at six months in Senegal using SMS reminder messages	Source
HKI presentation abstract: Strengthening district support to the micronutrient program in a low income setting-rural Mali 2014	Source
HKI presentation abstract: Validation of administrative coverage for vitamin A supplementation and deworming through Integrated National Immunization Days in Guinea	Source
HKI presentation: Impact of ebola on mass vitamin A supplementation and deworming coverage in Sierra Leone	Source
HKI presentation: Kenya VAS integrated into Early Childhood Development Centers 2016	Unpublished
HKI presentation: Reaching the hard to reach with vitamin A supplementation in low-performing health zones of DR Congo	Unpublished
HKI presentation: Routine Delivery of Vitamin A Supplementation at Six Months in Senegal using SMS Reminder Messages	Source
HKI presentation: Senegal Successful M-Health strategy: The routine delivery of Vitamin A Supplementation at six months using SMS appointment reminders	Source
HKI presentation: SMS reminders and vocal messages increase adherence to immunization and 6-month vitamin A supplementation 2016	Source
HKI presentation: Strengthening District Support to the Micronutrient Program in a Low Income Setting: Rural Mali	Source
HKI presentation: Using Mobile Phones for Data Collection to Improve Program Operation	Source
HKI presentation: Using results from coverage assessment surveys to improve program operation 2014	Source
HKI responses to GiveWell's questions May 2017	Unpublished
HKI routine VAS pilot project report Kenya 2016	Source
HKI SMS reminder pilot study report Côte d'Ivoire 2015	Source
HKI Tanzania social mobilization toolkit: deworming dosage card (Swahili)	Source
HKI Tanzania social mobilization toolkit: fact sheet on lives saved	Source
HKI Tanzania social mobilization toolkit: factsheet for community leaders	Source
HKI Tanzania social mobilization toolkit: mobilization script	Source
HKI Tanzania social mobilization toolkit: mobilization script (Swahili)	Source
HKI Tanzania social mobilization toolkit: tally sheet VASD	Source
HKI Tanzania social mobilization toolkit: VAS administration guide	Source
HKI Tanzania social mobilization toolkit: VAS administration guide (Swahili)	Source
HKI Tanzania social mobilization toolkit: VASD job aids for district health management team	Source
HKI Tanzania social mobilization toolkit: VASD Logos	Source
HKI Tanzania social mobilization toolkit: VASD posters	Source
HKI Tanzania social mobilization toolkit: vitamin A dosage card (Swahili)	Source
HKI Tanzania social mobilization toolkit: vitamin A logo	Source
HKI Tanzania social mobilization toolkit: vitamin A rich foods (photo)	Source
HKI VAS administration guide: DRC - side 1 (French)	Source
HKI VAS administration guide: DRC - side 2 (French)	Source
HKI VAS administration guide: Guinea (French)	Source
HKI VAS administration guide: Tanzania (Swahili)	Source
HKI VAS brochure Cameroon	Source
HKI VAS brochure Côte D'Ivore	Source
HKI VAS brochure Guinea	Source
HKI VAS brochure Sierra Leone	Source
HKI VAS brochure Tanzania	Source
HKI VAS concept note	Unpublished
HKI VAS costs presentation 2016	Source
HKI VAS documents guide for GiveWell 2017	Unpublished
HKI VAS overview brochure	Source
HKI VAS project year 1 report 2014	Unpublished
HKI VAS project year 2 report 2015	Unpublished
HKI VAS project year 3 report 2016	Unpublished
HKI VAS summary table	Unpublished
HKI VAS supervision checklist: DRC (French)	Source
HKI VAS supervision checklist: Mali (French)	Source
HKI VAS supervision checklist: Tanzania	Source
HKI VAS supervision checklist: universal	Source
HKI VAS television commercial: DRC (French)	Source
HKI website About Us	Source (archive)
HKI website Where we work	Source (archive)
Hodges et al. 2013	Source (archive)
Hodges et al. 2014	Source (archive)
Hodges et al. 2015	Source (archive)
Imdad et al. 2010	Source (archive)
Imdad et al. 2017	Source (archive)
Janmohamed and Doledec 2017	Source (archive)
Janmohamed, Klemm, and Doledec 2017	Source (archive)
Kagin et al. 2015	Source (archive)
Katcher et al. 2014	Unpublished
Klemm et al. 2016	Source (archive)
Kupka et al. 2016	Source (archive)
Lyatuu et al. 2016	Source (archive)
Malawi micronutrient survey 2017	Source (archive)
Masanja et al. 2006	Source (archive)
Nankap et al. 2013	Source
Neidecker-Gonzales et al. 2007	Source (archive)
Palmer et al. 2012	Source (archive)
Rolf Klemm, Vice President of Nutrition, HKI, email to GiveWell on July 14, 2017	Unpublished
Saitowitz et al. 2001	Source (archive)
Schemann et al. 2003	Source (archive)
Sesay et al. 2015	Source (archive)
Sommer and West 1996	Source (archive)
Sommer, West, and Martorell 2013	Source (archive)
Stevens et al. 2015	Source (archive)
Stevens et al. 2015 appendix	Source (archive)
UN Inter-agency Group for Mortality Estimation website 2015 estimates	Source (archive)
UNICEF 2007	Source (archive)
UNICEF vitamin A supplementation interactive dashboard 2016	Source (archive)
WHO Adverse events following administration of vitamin A supplements	Source (archive)
WHO Global Database on Vitamin A Deficiency: Cote d'Ivoire 2006	Source (archive)
WHO Global Database on Vitamin A Deficiency: DRC 2007	Source (archive)
WHO Global Database on Vitamin A Deficiency: Kenya 2006	Source (archive)
WHO Global Database on Vitamin A Deficiency: Mozambique 2006	Source (archive)
WHO Global Database on Vitamin A Deficiency: Nigeria 2007	Source (archive)
WHO Global Database on Vitamin A Deficiency: Tanzania 2007	Source (archive)
WHO Global prevalence of vitamin A deficiency in populations at risk 2009	Source (archive)
WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011	Source (archive)
WHO regional offices	Source (archive)
WHO vitamin A supplements adverse events	Source (archive)
WHO vitamin A supplements usage guide 1997	Source (archive)
Wirth et al. 2016	Source (archive)
Wirth et al. 2017	Source (archive)

• 1

  We have published notes from one of our conversations with HKI staff: GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017.

• 2
  • "Founded in 1915, Helen Keller International is dedicated to saving and improving the sight and lives of the world's vulnerable by combatting the causes and consequences of blindness, poor health and malnutrition.

    "We currently have more than 120 programs in 20 African and Asian countries.

    "Part of this work is focused on preventing blindness and vision loss for millions of vulnerable people through cataract surgery, vision correction, vitamin A supplementation, screening and treatment for diabetic retinopathy, and distribution of treatments and cures for neglected tropical diseases.

    "We also work to reduce malnutrition by promoting solutions aimed at improving nutrition practices for millions of families. These include vitamin A supplementation, maternal and child nutrition education, fortification of staple foods with essential nutrients, globally recognized family-led agricultural programs and community-based management of acute malnutrition." HKI website About Us

  • "United States: We provide the gift of clear vision to tens of thousands of children every year by providing free school-based vision screenings, prescription eyeglasses, and referral for further care through our innovative ChildSight program." HKI website Where we work

• 3
  • "Helen Keller International (HKI) has been on the forefront of the development and scaling up of strategies to effectively deliver micronutrients, starting with our programs in vitamin A supplementation (VAS) in the early 1970s. Historically most of HKI’s work in combating vitamin A deficiency (VAD) and other micronutrient deficiencies began in Asia. Since 1997, HKI has aggressively built the program base in sub-Saharan Africa and currently operates 13 country programs in Sub-Saharan Africa." HKI VAS project year 3 report 2016, Pg 1.
  • "The current project provides technical assistance to 13 countries identified as suffering from chronic vitamin A deficiency. These countries are Burkina Faso, Mali, Senegal, Tanzania, Cameroon, Mozambique, Kenya, Niger, Nigeria, Sierra Leone, Côte d’Ivoire, the Democratic Republic of the Congo, and Guinea." HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, Pg vi.

• 4
  • WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011:
    • "In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation). The quality of the available evidence for all-cause mortality was high, whereas for all other critical outcomes it was moderate to very low. The quality of the available evidence for outcomes in human immunodeficiency virus (HIV)- positive children was moderate for all-cause mortality." Pg 1.
    • One dose of 100,000 IU of vitamin A is recommended for infants aged 6 to 11 months of age, and a 200,000 IU dose of vitamin A is recommended for children 12 to 59 months of age every four to six months. Table 1, Pg 5.
  • WHO defines vitamin A deficiency to be of mild public health importance when rates of vitamin A deficiency (defined as a measure of serum or plasma retinol <0.70 µmol/l) among preschool-aged children or pregnant women are between 2% and 10%, moderate public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are between 10% and 20%, and severe public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are greater than or equal to 20%. WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 8, Table 5.
  • "HKI provides a package of interventions and services that include training, policy development, advocacy, monitoring & evaluation, service delivery, behavior change communication, social mobilization and supervision in all 13 countries where HKI assists host-country governments to implement universal preschool vitamin A supplementation programs." HKI country-level technical support related to vitamin A supplementation, Pg 1.
  • "Helen Keller International (HKI) has been at the forefront of VAS since the 1990’s, supporting the research that revealed the major impact of VAS on mortality and then supporting the sharp increase of coverage of VAS in up to 16 countries (13 currently) of Sub-Saharan Africa and helping most of them reach up to 95% coverage among children under five years in the last years." HKI VAS overview brochure, Pg 1.

• 5
  • "Vitamin A is an essential nutrient needed in small amounts for the normal functioning of the visual system, and maintenance of cell function for growth, epithelial integrity, red blood cell production, immunity and reproduction. Essential nutrients cannot be synthesized by the body and therefore must be provided through diet." WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 1.
  • "Vitamin A is required for normal functioning of the visual system, maintenance of cell function for growth, epithelial integrity, red blood cell production, immunity, and reproduction (Sommer 1996)." Imdad et al. 2017, Pg 8.
    • See Sommer and West 1996 (cited in Imdad et al. 2017 in the bullet point above) chapters 8 and 9 for a detailed description of how vitamin A is understood to function in visual, immune, and other bodily systems.

• 6

  "Vitamin A is an essential nutrient needed in small amounts for the normal functioning of the visual system, and maintenance of cell function for growth, epithelial integrity, red blood cell production, immunity and reproduction. Essential nutrients cannot be synthesized by the body and therefore must be provided through diet." WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 1.

• 7
  • WHO Global prevalence of vitamin A deficiency in populations at risk 2009:
    • "Deficiency of sufficient duration or severity can lead to disorders that are common in vitamin A deficient populations such as xerophthalmia (xeros = dryness; -ophthalmia = pertaining to the eye), the leading cause of preventable childhood blindness, anaemia, and weakened host resistance to infection, which can increase the severity of infectious diseases and risk of death." Pg 1.
    • "The term xerophthalmia encompasses the clinical spectrum of ocular manifestations of VAD, from milder stages of night blindness and Bitot’s spots, to potentially blinding stages of corneal xerosis, ulceration and necrosis (keratomalacia). . . . The stages of xerophthalmia are regarded both as disorders and clinical indicators of VAD, and thus can be used to estimate an important aspect of morbidity and blinding disability as well as the prevalence of deficiency. As corneal disease is rare, the most commonly assessed stages are night blindness, obtainable by history, and Bitot’s spots, observable by handlight examination of the conjunctival surface. Standard procedures exist for assessing xerophthalmia (17). Although night blindness and Bitot’s spots are considered mild stages of eye disease, both represent moderate-to-severe systemic VAD, as evidenced by low serum retinol concentrations (19), and increased severity of infectious morbidity (i.e. diarrhoea and respiratory infections) and mortality in children (5) and pregnant women (6, 20).” Pgs 2-3.
  • "Vitamin A deficiency (VAD) impairs body functions and may cause death. Adverse health consequences may also include xerophthalmia (dry eyes), susceptibility to infection, stunting, and anaemia (Sommer 1996; Rice 2004)." Imdad et al. 2017, Pg 8.

• 8

  WHO Global prevalence of vitamin A deficiency in populations at risk 2009:

  • "The main underlying cause of VAD as a public health problem is a diet that is chronically insufficient in vitamin A that can lead to lower body stores and fail to meet physiologic needs (e.g. support tissue growth, normal metabolism, resistance to infection)." Pg 1.
  • "Dietary deficiency can begin early in life, with colostrum being discarded or breastfeeding being inadequate, thereby denying infants of their first, critical source of vitamin A (1). Thereafter, into adulthood, a diet deficient in vitamin A lacks foods containing either preformed vitamin A esters, such as liver, milk, cheese, eggs or food products fortified with vitamin A or lacking its carotenoid precursors (mainly beta-carotene), such as green leaves, carrots, ripe mangos, eggs, and other orange-yellow vegetables and fruits. Where animal source or fortified foods are minimally consumed, dietary adequacy must rely heavily on foods providing beta-carotene. However, while nutritious in many ways, a diet with modest amounts of vegetables and fruits as the sole source of vitamin A may not deliver adequate amounts, based on an intestinal carotenoid-to-retinol conversion ratio of 12:1 (2). This ratio reflects a conversion efficiency that is about half that previously thought, leading to greater appreciation for why VAD may coexist in cultures that heavily depend on vegetables and fruits as their sole or main dietary source of vitamin A.

    "Usually, VAD develops in an environment of ecological, social and economical deprivation, in which a chronically deficient dietary intake of vitamin A coexists with severe infections, such as measles, and frequent infections causing diarrhoea and respiratory diseases that can lower intake through depressed appetite and absorption, and deplete body stores of vitamin A through excessive metabolism and excretion (3, 4). The consequent 'synergism' can result in the body’s liver stores becoming depleted and peripheral tissue and serum retinol concentrations decreasing to deficient levels, raising the risks of xerophthalmia, further infection, other VADD and mortality." Pg 1.

• 9
  • “WHO regional estimates indicate that the highest proportion of preschool-age children affected by night blindness, 2.0%, is in Africa, a value that is four times higher than estimated in South-East Asia (0.5%). This also means that Africa has the greatest number of preschool-age children affected with night blindness (2.55 million), and corresponds to almost half of the children affected globally (Table 10). A comparable and high proportion of pregnant women affected by night blindness are in Africa (9.8%) and South-East Asia (9.9%), each of which is estimated to have over 3 million pregnant women affected, or one third of the pregnant women affected globally. The estimates show that the Africa and South-East Asia regions also contain the highest proportions of preschool-age children with biochemical VAD, as indicated by a serum retinol concentration <0.70 µmol/l, with South-East Asia having the greatest number of children and pregnant women affected." WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pgs 10-11.
  • See WHO regional offices for countries included in the Africa and South-East Asia Regions.

• 10

  “Low vitamin A intake during nutritionally demanding periods in life, such as infancy, childhood, pregnancy and lactation, greatly raises the risk of health consequences, or vitamin A deficiency disorders (VADD).” WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 1.

• 11
  • "Provision of high doses of vitamin A every 6 months until the age of 5 years was based on the principle that a single, large dose of vitamin A is well absorbed and stored in the liver, and then mobilized, as needed, over an extended period of time (11). A dose of 100 000 International Units (IU) in infants 6–11 months of age and 200 000 IU in children 12–59 months of age is considered to provide adequate protection for 4–6 months, with the exact interval depending on the vitamin A content of the diet and the rate of utilization by the body (8, 12)." WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011, Pg 3.
  • "Vitamin A is a term used for a subclass of retinoic acids, a family of lipid-soluble compounds (Bates 1995). Vitamin A is found in two main forms: provitamin A carotenoids and preformed vitamin A. Provitamin A carotenoids are found in plants; beta-carotene is the only one that is metabolised by mammals into vitamin A. Though fruits and vegetables are nutritious in other ways, normal dietary intake of plants may not deliver adequate amounts of vitamin A because the intestinal carotenoid-to-retinol conversion ratio varies with type of food, ranging from 6:1 to 26:1 (US Institute of Medicine, Food and Nutrition Board; Van Lieshout 2005). Consequently, VAD can exist in places with high vegetable and fruit consumption (West 2002). Preformed vitamin A (retinol, retinal, retinoic acid, and retinyl esters), is the most active form of vitamin A and is found in animal sources. Supplements usually use preformed vitamin A (Shenai 1993; Bates 1995)." Imdad et al. 2017, Pg 8.

• 12
  • WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011:
    • "In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation). The quality of the available evidence for all-cause mortality was high, whereas for all other critical outcomes it was moderate to very low. The quality of the available evidence for outcomes in human immunodeficiency virus (HIV)- positive children was moderate for all-cause mortality." Pg 1.
    • One dose of 100,000 IU of vitamin A is recommended for infants aged 6 to 11 months of age, and a 200,000 IU dose of vitamin A is recommended for children 12 to 59 months of age every four to six months. Table 1, Pg 5.
  • WHO defines vitamin A deficiency to be of mild public health importance when rates of vitamin A deficiency (defined as a measure of serum or plasma retinol <0.70 µmol/l) among preschool-aged children or pregnant women are between 2% and 10%, moderate public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are between 10% and 20%, and severe public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are greater than or equal to 20%. WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 8, Table 5.

• 13

  HKI country situation matrix for VAS, "Country Summary" sheet describes the distribution methods for each HKI-supported VAS program in sub-Saharan Africa in 2015:

  • Burkina Faso: Child Health Days (door-to-door) coupled with National Immunization Days
  • Cameroon: Child Health Days (door-to-door) coupled with National Immunization Days
  • Cote d'Ivoire: Child Health Days (door-to-door) coupled with National Immunization Days
  • Democratic Republic of the Congo: Child Health Day pilots and door-to-door National Immunization Days
  • Guinea: Polio National Immunization Days
  • Kenya: Routine delivery and Child Health Days (fixed sites and outreach)
  • Mali: Polio National Immunization Days
  • Mozambique: Child Health Days (fixed sites and outreach)
  • Niger: Polio National Immunization Days and Child Health Day pilots
  • Nigeria: Child Health Days (fixed sites and outreach)
  • Senegal: Child Health Days (door-to-door) and routine delivery (8 districts)
  • Sierra Leone: Child Health Days (door-to-door)
  • Tanzania: Child Health Days (fixed sites and outreach)

• 14
  • "Mass distribution campaigns are the main delivery mechanism for VAS. These campaigns are organized at least every 6 months (sometimes much more often) and have been instrumental in reaching more than 95% of the children targeted. National Immunization Days are the most common strategy, organized as nationwide door-to-door events. Health workers leave their facilities and go in communities to administer vitamin A in people’s homes. The events require intensive planning and coordination by national and district level authorities." HKI VAS overview brochure, Pg 2.
  • "HKI establishes systems to assess the quality of the technical assistance it provides using the tools and methods described under question four including pre-post-tests as part of training programs, supervision checklists, and the post-event coverage surveys. In countries where door-to-door VAS distribution 'piggy-backs' on National Immunization Days, vitamin A capsule coverage has achieved sustained high-coverage, and therefore could be judged as a 'success'. However, as polio is eradicated from Africa (only 4 cases were reported in the past 2 years, and they were from the conflict-ridden area in Borno in Northeastern Nigeria), the externally supported infrastructure (i.e. vehicles, per diems, etc.) will disappear and countries must find alternative platforms for delivering vitamin A to preschool-age children. In many countries, the weak link, distance and infrequent contact between health facility-based services and the community will present a major challenge for VAS delivery." HKI VAS documents guide for GiveWell 2017, Pg 7.
  • "In 2000, yearly polio campaigns were estimated to cost US$1 per child, and estimates of the incremental cost of adding VAS ranged from 2% to 10%, with the higher incremental costs incurred in smaller countries. Despite these benefits, the integrated campaigns have also long been recognized as introducing additional challenges, including the need for increased logistical coordination of the additional supplies, training of additional workers/volunteers/supervisors and the streamlining of the actual event processes (e.g., the setup of the vaccine/VAS delivery location). By 1998, 22 countries in Africa had added vitamin A distribution to their NIDs, and in the same year 60% of children under 5 received a vitamin A supplement through this or another means. Early on, it was expected that polio NIDs and sub- national immunization days would provide the opportunity to deliver VAS until 2002 or 2003, at which point measles immunization campaigns or others could be used instead. However, it is also recognized that the provision of vitamin A with routine immunization services is a more sustainable, longer-term solution." HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, Pg 3.
  • "Our analysis shows that the implementation of effective and sustained policies and programs for the control of vitamin A deficiency can bring about a reduction of up to 25% in child mortality rates in sub-Saharan Africa, compared with the mortality rates in 1995, before the onset of large-scale vitamin A supplementation with National Immunization Days for polio eradication. In many countries in sub-Saharan Africa, one high-dose vitamin A capsule is given annually on National Immunization Days for polio eradication, thus ensuring a vitamin A reserve of four to six months to more than 80% of children 6 to 59 months old." Aguayo and Baker 2005, Pg 353.
  • We note that some of the sources above discuss National Immunization Days as annual events, but the HKI VAS overview brochure states that they occur every six months or more. We have not yet investigated how often National Immunization Days occur in each country in sub-Saharan Africa with HKI-supported VAS programs.

• 15
  • "Approximately 80 countries currently implement VA programs—the vast majority of which are centered on supplementation. Early VAS programs generally achieved high once-annual coverage of children by being 'piggy backed' onto National Immunization Day campaigns for polio eradication. In some African countries, this remains a mode of VA delivery. But in most other countries, as polio eradication activities ceased or became subnational, VAS delivery has transitioned to semiannual Child Health Weeks." Klemm et al. 2016, Pg 3.
  • "In countries where door-to-door VAS distribution 'piggy-back' on National Immunization Days, vitamin A capsule coverage has achieved sustained high-coverage, and therefore could be judged as a “success”. However, as polio is eradicated from Africa (only 4 cases were reported in the past 2 years, and they were from the conflict-ridden area in Borno in Northeastern Nigeria), the externally supported infrastructure (i.e. vehicles, per diems, etc.) will disappear and countries must find alternative platforms for delivering vitamin A to preschool-age children." HKI VAS documents guide for GiveWell 2017, Pg 7.

• 16
  • "The transition towards a Child Health Day (CHD) approach was a point of discussion early on in the implementation of NIDs, given that this approach had been successfully implemented in Latin America around the same time. CHDs are a type of selective primary health care, the structure of which varies by country but generally consists of biannual events that deliver a package of public health interventions to children under the age of five. These are combined with extensive awareness-raising and social mobilization efforts, particularly in areas considered 'hard to reach' or with otherwise reduced access to the health system.

    "CHDs go by different names in different countries, and are sometimes referred to as Regular Events to Advance Child Health (REACH). Depending on the context of the country, the package of preventive services offered can include VAS, deworming, immunization, malaria prophylaxis, antenatal care, growth monitoring, promotion of family practices/behaviors and nutrition education. Importantly, these events build on the existing primary health care infrastructure and staff through the use of fixed sites and outreach. The degree of centralization of the health system in the country implementing the CHD has implications for how it is carried out, with decentralized systems having greater flexibility in the timing and duration of CHDs than those with centralized systems (UNICEF, 2008). CHDs can also vary in duration, ranging from a few days to an entire month. In a 2006 evaluation of CHDs in six sub-Saharan African countries, CHDs contributed to improvements in VAS coverage ranging from 15 to 90 percentage points and were the basis for the inclusion of deworming programs in many of these countries, which had previously achieved low coverage through government and private sector projects aimed at preschool aged children (UNICEF, 2008)." HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, Pgs 3-4.

  • "Several countries in Sub Saharan Africa have been implementing Child Health Days (or weeks) delivering a package of health and nutrition services targeting mothers and children, either through a door-to-door approach or a mix of facility-based and outreach activities. As a cost effective way to reach mothers and children with essential services when access to routine health system services is limited, Child Health Days constitute an intermediary model between NiD’s and routine delivery." HKI VAS overview brochure, Pg 2.

• 17
  • "Several countries in Sub Saharan Africa have been implementing Child Health Days (or weeks) delivering a package of health and nutrition services targeting mothers and children, either through a door-to-door approach or a mix of facility-based and outreach activities. As a cost effective way to reach mothers and children with essential services when access to routine health system services is limited, Child Health Days constitute an intermediary model between NiD’s and routine delivery." HKI VAS overview brochure, Pg 2.
  • HKI country situation matrix for VAS, "Country Summary" sheet describes the distribution methods for each HKI-supported VAS program in sub-Saharan Africa in 2015:
    • Burkina Faso: Child Health Days (door-to-door) coupled with National Immunization Days
    • Cameroon: Child Health Days (door-to-door) coupled with National Immunization Days
    • Cote d'Ivoire: Child Health Days (door-to-door) coupled with National Immunization Days
    • Democratic Republic of the Congo: Child Health Day pilots and door-to-door National Immunization Days
    • Guinea: Polio National Immunization Days
    • Kenya: Routine delivery and Child Health Days (fixed sites and outreach)
    • Mali: Polio National Immunization Days
    • Mozambique: Child Health Days (fixed sites and outreach)
    • Niger: Polio National Immunization Days and Child Health Day pilots
    • Nigeria: Child Health Days (fixed sites and outreach)
    • Senegal: Child Health Days (door-to-door) and routine delivery (8 districts)
    • Sierra Leone: Child Health Days (door-to-door)
    • Tanzania: Child Health Days (fixed sites and outreach)

• 18

  "Because mass campaigns take place only every 4 to 6 months, children who reach the age of 6 months between two campaigns, may have to wait several months before they get their first dose of Vitamin A despite being the most vulnerable age group.

  "To remedy this, HKI is working closely with country-level health sector experts to add a contact point in national immunization calendars – at 6 months, when no other vaccination is scheduled.

  "Additionally, HKI supports routine facility-based and outreach delivery of vitamin A for all children under 5 in countries where stronger health systems offer sufficient access to quality services. Few countries are ready for this approach and these still need to develop social mobilization actions to create demand to match the capacity to offer services." HKI VAS overview brochure, Pg 2.

• 19
  • "Because mass campaigns take place only every 4 to 6 months, children who reach the age of 6 months between two campaigns, may have to wait several months before they get their first dose of Vitamin A despite being the most vulnerable age group.

    "To remedy this, HKI is working closely with country-level health sector experts to add a contact point in national immunization calendars – at 6 months, when no other vaccination is scheduled.

    "Additionally, HKI supports routine facility-based and outreach delivery of vitamin A for all children under 5 in countries where stronger health systems offer sufficient access to quality services. Few countries are ready for this approach and these still need to develop social mobilization actions to create demand to match the capacity to offer services." HKI VAS overview brochure, Pg 2.

  • "In the past, most VAS programs in Sub-Saharan Africa have delivered supplements to children in door-to-door mass campaigns tied to polio immunization campaigns, but countries need to find alternative methods of delivering VAS, since many polio campaigns are ending due to progress in polio elimination. Transitioning to 'routine delivery' (in which caregivers bring children to facilities combined with periodic outreach/delivery posts within communities to receive VAS at appropriate ages) appears to be the most sustainable long-term option." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 5-6.

• 20

  "Because mass campaigns take place only every 4 to 6 months, children who reach the age of 6 months between two campaigns, may have to wait several months before they get their first dose of Vitamin A despite being the most vulnerable age group.

  "To remedy this, HKI is working closely with country-level health sector experts to add a contact point in national immunization calendars – at 6 months, when no other vaccination is scheduled.

  "Additionally, HKI supports routine facility-based and outreach delivery of vitamin A for all children under 5 in countries where stronger health systems offer sufficient access to quality services. Few countries are ready for this approach and these still need to develop social mobilization actions to create demand to match the capacity to offer services." HKI VAS overview brochure, Pg 2.

• 21
  • "For distribution sites visited by an independent HKI supervisor, 86% met the criteria for minimum quality threshold for service delivery defined as 1) health worker used scissors to cut the capsule 2) health worker asked the age of the child 3) health worker squeezed the drops into the child’s mouth 4) health worker used the correct dose of VAS and 5) were there no stock-outs of VAC." HKI VAS project year 1 report 2014
  • See the guidelines for health workers in HKI VAS supervision checklist: universal and HKI Tanzania social mobilization toolkit: VAS administration guide.

• 22
  • "To date, HKI’s VAS project has undertaken three main types of activities, which can roughly be categorized as disbursing sub-grants to the government, providing technical assistance, and engaging in advocacy efforts." HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, Pg 35.
  • "HKI provides a package of interventions and services that include training, policy development, advocacy, monitoring & evaluation, service delivery, behavior change communication, social mobilization and supervision in all 13 countries where HKI assists host-country governments to implement universal preschool vitamin A supplementation programs." HKI country-level technical support related to vitamin A supplementation, Pg 1.
  • "In concrete terms, HKI, in consultation with national government counterparts, directs its support to low performing areas to help local program managers identify and solve VAS coverage barriers. This involves organizing workshops with state and district health authorities to analyze what worked and what did not. HKI teams then spend time with health managers to help them identify feasible and cost-effective solutions to improve performance of the targeted services and accompany them through the whole programming cycle (i.e. planning, budgeting, implementation, real time supervision and monitoring, and finally evaluation of the progress made). One cycle sometimes proves insufficient so the HKI teams continue working with each targeted health district until minimum thresholds of performance are met. Funds are used to support deployment of HKI teams in remote areas, to support financing workshops and joint field supervisions, to provide training for field actors, or to organize coverage surveys and review meetings at the end of the exercise. In some cases, HKI provides funds directly to the local authorities to fill financial gaps they may experience ensuring rigorous financial accountability. When conditions for a change of approach are met, HKI provides technical assistance to local authorities to design, implement and monitor with them innovative approaches such as the 6-month contact point or SMS messaging." HKI VAS documents guide for GiveWell 2017, Pg 4.

• 23
  • Coverage surveys:
    • HKI assists governments with implementing surveys to assess coverage (i.e., the percentage of targeted children who actually received vitamin A supplements) following VAS mass distribution campaigns:
      • "Since 2010, HKI has developed a methodology to assess the true coverage of vitamin A supplementation and identify barriers and determinants of high coverage. Based on a cross sectional survey methodology, the Post-Event Coverage Surveys (PECS) are conducted by health system personnel using mobile phones and allow multiple indicators to be collected. More than 50 surveys have been conducted in 15 countries and provide data to improve performance of VAS programs and ensure that all children have equitable access to essential child survival services." HKI VAS overview brochure, Pg 2.
    • See this spreadsheet for a summary of the results and methodology of HKI's recent coverage surveys.
  • Administrative data:
    • "HKI helps to track national VAS coverage through the governments tally-sheet system (also referred to as 'Administrative Data')." HKI VAS documents guide for GiveWell 2017, Pg 6.
    • We have not yet seen specific descriptions of how HKI assists governments with tracking administrative data.

• 24
  • Developing training materials:
    • "In all countries, HKI developed comprehensive training packages to allow governments conduct training and organized training of trainers." HKI country-level technical support related to vitamin A supplementation, Pgs 1-2.
    • "All of HKI’s work in VAS/CHDs programs focuses on alignment with and use of existing national systems. This requires significant engagement of human resources at national and decentralized levels and a sound understanding of national systems, and how they can be leveraged to achieve program scale. Critical entry points for HKI engagement include supporting development of national policy, catalyzing national and district level planning, training front-line delivery agents and program managers, supporting formative supervision and strengthening monitoring systems and encouraging use of data for program quality improvement and management for results." HKI VAS project year 3 report 2016, Pg 2.
    • Specific examples we have seen of this type of activity:
      • HKI Tanzania social mobilization toolkit: VAS administration guide
      • HKI Tanzania social mobilization toolkit: VASD job aids for district health management team
      • HKI VAS administration guide: Guinea (French)
  • Direct training of health workers:
    • "Facilitate refresher training of 62786 Community workers and health worker (24747 in 1st Round and 38039 in second Round)" HKI VAS project year 1 report 2014, Pg 42 (in list of HKI's activities in Democratic Republic of the Congo during the year).

• 25
  • HKI assists governments with designing "6-month contact point" policies:
    • "Because mass campaigns take place only every 4 to 6 months, children who reach the age of 6 months between two campaigns, may have to wait several months before they get their first dose of Vitamin A despite being the most vulnerable age group.

      "To remedy this, HKI is working closely with country-level health sector experts to add a contact point in national immunization calendars – at 6 months, when no other vaccination is scheduled." HKI VAS overview brochure, Pg 2.

    • See HKI 6-month contact point standard methodology for a detailed description of the policy.

• 26
  • "Capacity development was also commonly conducted through supportive supervision, organized jointly with local health managers to help field actors identify their strengths and weaknesses and find solutions by themselves." HKI country-level technical support related to vitamin A supplementation, Pg 2.
  • HKI sent us its "supervision checklists" used during VAS campaigns and routine distribution:
    • HKI monitoring tools: quality of care checklist for supportive supervision of VAS services
    • HKI VAS supervision checklist: DRC (French)
    • HKI VAS supervision checklist: Mali (French)
    • HKI VAS supervision checklist: Tanzania
    • HKI VAS supervision checklist: universal
  • We have not yet investigated how often health workers implementing VAS programs are supervised using these checklists.

• 27
  • "In concrete terms, HKI, in consultation with national government counterparts, directs its support to low performing areas to help local program managers identify and solve VAS coverage barriers. This involves organizing workshops with state and district health authorities to analyze what worked and what did not. HKI teams then spend time with health managers to help them identify feasible and cost-effective solutions to improve performance of the targeted services and accompany them through the whole programming cycle (i.e. planning, budgeting, implementation, real time supervision and monitoring, and finally evaluation of the progress made). One cycle sometimes proves insufficient so the HKI teams continue working with each targeted health district until minimum thresholds of performance are met. Funds are used to support deployment of HKI teams in remote areas, to support financing workshops and joint field supervisions, to provide training for field actors, or to organize coverage surveys and review meetings at the end of the exercise. In some cases, HKI provides funds directly to the local authorities to fill financial gaps they may experience ensuring rigorous financial accountability. When conditions for a change of approach are met, HKI provides technical assistance to local authorities to design, implement and monitor with them innovative approaches such as the 6-month contact point or SMS messaging." HKI VAS documents guide for GiveWell 2017, Pg 4.
  • Specific example of this type of work:
    • "Meeting with State Director of Public Health to initiate the VAS workplan development in 8 States" and "Obtain annual costed workplan from HKI-supported states" are listed as activities completed by HKI in 2013 in Nigeria. HKI VAS project year 1 report 2014, Pg 96, Table 72.

• 28
  • "Creating demand by promoting healthy behaviors:
    "HKI promotes social mobilization campaigns that inform caregivers of children aged 6 to 59 months of the importance of VAS and that a supplementation campaign is taking place in their neighborhood. Some of HKI’s achievements include:
    • Using of SMS reminders sent to every caregiver;
    • Cooperating with religious and traditional leaders;
    • Involving community health workers and volunteers to organize regular population census that ensure that each child is reached at home;
    • Developing with health authorities and rolling-out comprehensive social mobilization toolkits that include multi-media communication and sensitization tools and messages; and
    • Associating social mobilization with comprehensive social & behavior change strategies."

    HKI VAS overview brochure, Pg 3.

  • Examples of social mobilization and marketing materials:
    • HKI Tanzania social mobilization toolkit: VASD posters
    • HKI Tanzania social mobilization toolkit: mobilization script
    • HKI Tanzania social mobilization toolkit: mobilization script (Swahili)
    • HKI VAS television commercial: DRC (French)

• 29
  • "HKI teams work closely with national governments to support the policy, strategy and tool development mentioned above, but HKI’s major added value is its capacity to rapidly deploy technical support to the sub national level to assist local health authorities with implementing national VAS strategies. HKI focus its efforts sub-nationally because local level (at state and/or district level) health system performance is key to ensuring high VAS coverage. It also allows HKI to support other health system functions that also improve the delivery of other maternal and child health services.

    "In concrete terms, HKI, in consultation with national government counterparts, directs its support to low performing areas to help local program managers identify and solve VAS coverage barriers. This involves organizing workshops with state and district health authorities to analyze what worked and what did not. HKI teams then spend time with health managers to help them identify feasible and cost-effective solutions to improve performance of the targeted services and accompany them through the whole programming cycle (i.e. planning, budgeting, implementation, real time supervision and monitoring, and finally evaluation of the progress made). One cycle sometimes proves insufficient so the HKI teams continue working with each targeted health district until minimum thresholds of performance are met. Funds are used to support deployment of HKI teams in remote areas, to support financing workshops and joint field supervisions, to provide training for field actors, or to organize coverage surveys and review meetings at the end of the exercise. In some cases, HKI provides funds directly to the local authorities to fill financial gaps they may experience ensuring rigorous financial accountability. When conditions for a change of approach are met, HKI provides technical assistance to local authorities to design, implement and monitor with them innovative approaches such as the 6-month contact point or SMS messaging." HKI VAS documents guide for GiveWell 2017, Pgs 3-4.

  • "Micronutrient Initiative (MI) (the name of the organization changed ~1 month ago to Nutrition International or NI) is only active in 4 of the 13 countries were HKI is operational, however MI provides the needed number of vitamin A capsules to all countries where HKI works. MI’s role essentially takes place at the national level, providing technical and policy guidance to governments. In most cases, MI delivers the vitamin A capsules to UNICEF, who organizes their management with the national government and ensures that they reach the field. UNICEF’s role is mainly at national level to support all aspects of maternal and child health. This large portfolio gives them the capacity to weigh strongly on decisions at national level but also prevents them from providing specific technical assistance, especially at the sub-national and district levels, where needed. HKI, being more flexible and specialized, takes on this technical support role, and builds evidence, and adjusts its activities to the evolving needs of the program." HKI country-level technical support related to vitamin A supplementation, Pg 2.

• 30

  "In most countries, HKI teams spent around 10% of their time working with the national government to advocate for VAS. HKI advocated for domestic budgets to take a greater proportion of the costs of VAS, to integrate VAS in national health and nutrition policy documents and in pluriannual strategies or action plans, supporting coordination between actors and sectors and promoting monitoring of VAS at national level to provide the government with a comprehensive vision of the services for the whole country." HKI country-level technical support related to vitamin A supplementation, Pg 1.

• 31

  "Because mass campaigns take place only every 4 to 6 months, children who reach the age of 6 months between two campaigns, may have to wait several months before they get their first dose of Vitamin A despite being the most vulnerable age group.

  "To remedy this, HKI is working closely with country-level health sector experts to add a contact point in national immunization calendars – at 6 months, when no other vaccination is scheduled." HKI VAS overview brochure, Pg 2.

• 32
  • "In concrete terms, HKI, in consultation with national government counterparts, directs its support to low performing areas to help local program managers identify and solve VAS coverage barriers. This involves organizing workshops with state and district health authorities to analyze what worked and what did not. HKI teams then spend time with health managers to help them identify feasible and cost-effective solutions to improve performance of the targeted services and accompany them through the whole programming cycle (i.e. planning, budgeting, implementation, real time supervision and monitoring, and finally evaluation of the progress made). One cycle sometimes proves insufficient so the HKI teams continue working with each targeted health district until minimum thresholds of performance are met. Funds are used to support deployment of HKI teams in remote areas, to support financing workshops and joint field supervisions, to provide training for field actors, or to organize coverage surveys and review meetings at the end of the exercise. In some cases, HKI provides funds directly to the local authorities to fill financial gaps they may experience ensuring rigorous financial accountability." HKI VAS documents guide for GiveWell 2017
  • "To date, HKI’s VAS project has undertaken three main types of activities, which can roughly be categorized as disbursing sub-grants to the government, providing technical assistance, and engaging in advocacy efforts." HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, Pg 35.
  • Spending categorized as "Service Delivery" is funding HKI has granted to governments for program implementation. See this spreadsheet, "By category" sheet, for details.
  • Our understanding is that the proportion of funding provided by different actors (HKI, other non-profit organizations, national and sub-national governments) varies for different VAS programs. We have not yet seen complete information on funding contributions from HKI and other actors for most of HKI's recent VAS programs (see here for more details).

• 33

  "In general, HKI’s technical assistance for VAS includes training, policy development, advocacy, monitoring & evaluation, service delivery, behaviour change communication, social mobilization and supervision. However, in each country, the level of effort for each component varies based on context and capacity. HKI has grouped counties into these categories:

  • Category 1. Weak health systems capacity and door-to-door VAS delivery: These are countries where VAS is delivered through door-to-door National Immunization Days (NiD’s) and where capacity of the health system is limited: Mali, Niger, Burkina Faso, Guinea, Cameroon, DRC. In these countries, training and service delivery represented the largest components in terms of funding and level of efforts. Such countries offered little capacity for innovation as most efforts aimed at ensuring that coverage would reach minimum standards. Advocacy takes longer in these countries than in others to promote sustainable approaches to governments showing little interest for them.
  • Category 2. Moderate health systems capacity and mixed VAS approaches: These are countries with slightly better health systems capacity: Cote d’Ivoire, Sierra Leone, Senegal. In these three countries, campaigns were still using door-to-door delivery piggy-backed onto National Immunization Days, but HKI and its partners managed to create a strong momentum towards more sustainable approaches. The 6-month contact point was piloted as an integrated approach to VAS delivery along with immunizations in health facilities (in Senegal, Sierra Leone and Cote d’Ivoire), and then scaled up in Senegal and Sierra Leone. These activities represented most of HKI’s level of effort.
  • Category 3. Countries where VAS delivery has changed to fixed site and facility-based VAS distribution: Mozambique, Kenya, Tanzania, Nigeria. In such countries, a larger diversity of activities is implemented. Innovative approaches were tested in all countries."

  HKI VAS documents guide for GiveWell 2017, Pg 8.

• 34
  • In this review, we refer to the agency with its current name, Global Affairs Canada. Documents we cite may refer to former name of the agency, the Department of Foreign Affairs, Trade and Development of Canada (DFATD), or the Canadian International Development Agency (CIDA), which was absorbed into DFATD in 2013.
    • "The names of several departments are being changed as follows:
      […]
      Foreign Affairs, Trade and Development Canada to Global Affairs Canada" Canada Privy Council Office Machinery of Government Changes 2015
    • "The agency that handles Canada's international aid is going to be brought into the Department of Foreign Affairs, the government announced Thursday in the federal budget.

      "It's not yet clear how the move will affect the work of the Canadian International Development Agency, which is currently the responsibility of International Co-operation Minister Julian Fantino, but the fact the minister's powers are about to be enshrined in law is seen as a positive sign for its future.

      "In the past, ministers in charge of CIDA haven't had the same enshrinement in law as other federal cabinet ministers.

      "The new department will be known as the Department of Foreign Affairs, Trade and Development." CBC News 2013

  • HKI plays a range of technical assistance roles to national vitamin A supplementation programs in Africa. These have been almost exclusively supported from grants from the Canadian Government, the most recent being a grant entitled, “Scaling Up Nutrition through Integrated Life-saving Interventions Project-2013-2016.” HKI VAS documents guide for GiveWell 2017, Pg 1.
  • "Annex 1 - History of grants devoted to VAS implemented by HKI with support from GAC [Global Affairs Canada]" lists five grants from GAC to HKI, beginning in January 2006, totaling around $80 million CAD in total funding. HKI VAS concept note, Pg 22.

• 35
  • "Since 2002, Helen Keller International (HKI) has partnered with UNICEF and the Department of Foreign Affairs, Trade and Development of Canada (DFATD), formerly the Canadian International Development Agency (CIDA), to work towards increasing and sustaining vitamin A supplementation (VAS) coverage in countries with a high burden of child mortality and nutrition-related disease. The partnership started in five sub-Saharan countries, and expanded during the 2005–2008 period to nine countries, where it focused primarily on shifting to twice-yearly distribution, sustainability and building national capacity. The subsequent project (Scaling up Child Health and Nutrition in Sub-Saharan Africa) ran from 2009–2013 and operated in 13 countries.

    "The current grant amounts to CDN$29,000,000 over a three-year period (2013–2016) and is entitled “Scaling Up Nutrition through Integrated Life-Saving Interventions.” The two primary anticipated outcomes of the project are:

    1. enhanced healthy nutritional practices for children 6–59 months through maintained high coverage of child health and nutrition services; and
    2. increased national ownership of child survival activities (in transitioning countries)."

    HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, Pg 1.

  • "Annex 1 - History of grants devoted to VAS implemented by HKI with support from GAC [Global Affairs Canada]" lists a grant of $29 million CAD from Global Affairs Canada to HKI for 13 countries in sub-Saharan Africa implemented between February 2013 and May 2016. HKI VAS concept note, Pg 22.

• 36
  • "HKI plays a range of technical assistance roles to national vitamin A supplementation programs in Africa. These have been almost exclusively supported from grants from the Canadian Government, the most recent being a grant entitled, 'Scaling Up Nutrition through Integrated Life-saving Interventions Project-2013-2016 HKI VAS documents guide for GiveWell 2017, Pg 1.
  • "Annex 1 - History of grants devoted to VAS implemented by HKI with support from GAC [Global Affairs Canada]" lists five grants from GAC to HKI, beginning in January 2006, totaling around $80 million CAD in total funding. HKI VAS concept note, Pg 22.
  • HKI also notes that its funding "is largely project-based with minimal availability of unrestricted funds." Comment provided in response to a draft of this page in August 2017.

• 37
  • “Vitamin A was associated with a 24% reduction in all-cause mortality (RR = 0.76 (95% CI 0.69 to 0.83)), though there was moderate heterogeneity.” Imdad et al. 2010, Pg 18.
  • One of the seventeen included trials had zero weight in the analysis. “One [additional randomized trial] reported no events [i.e. deaths] (Lin 2008)” and therefore had zero weight in the analysis. Imdad et al. 2010, Pg 18.
  • The authors included two quasi-randomized trials in the review. "Post hoc, we included two studies in which participants were assigned using a quasi-random method (Herrera 1992; Stansfield 1993)." Imdad et al. 2010, Pg 10. Only one of these, Herrera et al. 1992, was included in the analysis of all-cause mortality. Imdad et al. 2010, Pg 18. In this trial, “[a]ssignment to treatment group was achieved by the two interviewers visiting alternate households throughout the village.” Herrera et al. 1992, Pg 267. We agree with the Cochrane authors’ conclusion that the study “(i) had the desirable characteristics of randomization and (ii) [was] of no greater risk of bias than other included studies” because allocating alternate households to the treatment and control group “was not likely to result in systematically different groups.” Imdad et al. 2010, Pgs 10, 17. Baseline statistics suggest that allocation was as good as random. “There were no important differences between vitamin A and placebo groups in rate of xerophthalmia (2.8% vs 2.9%), vitamin A intake, age distribution or nutritional status.” Herrera et al. 1992, Pg 269. Furthermore, Herrera et al. 1992 “reported no effect (RR = 1.06 (95% CI 0.92 to 1.37)), indicating that these trials were not likely to influence [Cochrane’s] results in a positive direction.” Imdad et al. 2010, Pg 19.

• 38
  • “Deaths per child-care centre at ages 1.0–6.0 years during the 5-year study (the primary trial endpoint) were 3.01 retinol versus 3.15 control (absolute reduction 0.14 [SE 0.11], mortality rate ratio [RR] 0.96, 95% CI 0.89–1.03, p=0.22), suggesting absolute risks of death between ages 1.0 and 6.0 years of approximately 2.5% retinol versus 2.6% control. Although this finding suggests that overall child mortality was 4% lower in vitamin A than in control blocks, this 4% reduction includes the possibility of no benefit and the possibility of appreciable benefit (95% confidence limit for reduction 11%).” Awasthi et al. 2013, Pg 1473.
  • "DEVTA trial 2013...
    Eligibility: children aged 1-6 years were eligible for inclusion in the review
    Sample: total clusters were 72, of which 36 clusters received vitamin A supplementation while 36 acted as control. Authors claimed to include 1 million children in the trial." Imdad et al. 2017, Pg 55.

• 39

  The DEVTA researchers conducted a meta-analysis of DEVTA and eight previous large trials where pre-school children were provided with multiple doses of VAS per year. They found “heterogeneity between DEVTA and subtotal of eight previous trials p = 0.0010.” Awasthi et al. 2013, Pg 1475.

• 40

  "At longest follow-up, there was a 12% observed reduction in the risk of all-cause mortality for vitamin A compared with control using a fixed-effect model (risk ratio (RR) 0.88, 95% confidence interval (CI) 0.83 to 0.93; high-quality evidence)." Imdad et al. 2017, Pg 2.

• 41

  See our vitamin A intervention report for sources and details.

• 42
  • “The biggest specific cause of death that VAS reduces is diarrhea. Deaths from diarrhea are falling but still a leading cause of childhood mortality globally.” GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013, Pg 2.
  • “In a reanalysis of one of the original eight trials, the beneficial effect was limited to unvaccinated children and there were strong sex-differential effects of vitamin A supplementation in vaccinated children. Hence, the roll-out of the vaccination programme might be one environmental factor that has modified the effect of vitamin A.” Benn, Fisker, and Aaby 2013, Pg 593.

• 43
  • Beaton et al. 1993:
    • "The second consideration might be overall mortality rates. Figure 5.3 portrays the relative effectiveness of vitamin A supplementation in relation to control group mortality rates (a poor proxy for baseline mortality rate). No particular relationship is apparent and none could be detected in statistical analyses involving a variety of models in which individual projects were weighted (see Technical Annex)." Pg 67.
    • See Figure 5.3, Pg 68.
  • Beaton et al. 1993's analysis includes eight VAS trials. We have not completed an up-to-date analysis of this type for all the VAS trials that measured all-cause mortality included in Imdad et al. 2017.

• 44
• "Coverage was ascertained from logbooks of overworked government community workers (anganwadi workers), and verified by a small number of supervisors who periodically visited randomly selected anganwadi workers to question and examine children who these workers gathered for them. Both anganwadi worker self-reports, and the validation procedures, are fraught with potential bias that would inflate the actual coverage . . . Although 76% of children aged 0–71 months in 2005–06 lived in areas covered by an anganwadi worker, only 22% of children received any service from the anganwadi worker. Thus, it is hard to understand how DEVTA ramped up coverage to extremely high levels (and if it did, why so little of this effort was sustained). DEVTA provided the anganwadi workers with less than half a day’s training and minimal if any incentive." Sommer, West, and Martorell 2013, Pg 591.

• 45
  • In our vitamin A supplementation intervention report, we note that DEVTA did not target children in remote areas, who may have been more likely than other children to suffer from VAD (see here and here). We also note that some control group members may have received some doses of vitamin A (see here).
  • Additionally, HKI told us the following:
    • "DEVTA represented an earnest attempt to evaluate the impact of Anganwadi delivery of vitamin A capsules on preschool child mortality and vitamin A deficiency. The DEVTA trial included about a million children and found a small mortality benefit (~4%) for vitamin A supplementation, although not statistically significant. The DEVTA findings generated controversy because many experts believe that the methods for the delivery of the intervention and the assessment of the primary outcome (i.e. all-cause mortality) were not rigorous (Habicht 2013; Mannar 2013; Mayo-Wilson 2013; Sloan 2013; Sommer 2013). For example, investigators did not count children at baseline or obtain informed consent, and methods of follow up and data collection were not [r]igorous (Mannar 2013; Sommer 2013). In this cluster-randomized trial, vitamin A capsules were distributed by Anganwadi workers who had contact with only 26% of the children living in the study area (Sommer 2013). In reply to his extensive criticism, authors of DEVTA emphasized that results of this trial need to be interpreted alongside previously published studies (Peto 2013)." HKI responses to GiveWell's questions May 2017, Pg 6.
  • We have not yet vetted the sources cited in the bullet point above.

• 46
  • In this review, we refer to the agency with its current name, Global Affairs Canada. Documents we cite may refer to former name of the agency, the Department of Foreign Affairs, Trade and Development of Canada (DFATD), or the Canadian International Development Agency (CIDA), which was absorbed into DFATD in 2013.
    • "The names of several departments are being changed as follows: […] Foreign Affairs, Trade and Development Canada to Global Affairs Canada" Canada Privy Council Office Machinery of Government Changes 2015
    • "The agency that handles Canada's international aid is going to be brought into the Department of Foreign Affairs, the government announced Thursday in the federal budget.

      "It's not yet clear how the move will affect the work of the Canadian International Development Agency, which is currently the responsibility of International Co-operation Minister Julian Fantino, but the fact the minister's powers are about to be enshrined in law is seen as a positive sign for its future.

      "In the past, ministers in charge of CIDA haven't had the same enshrinement in law as other federal cabinet ministers.

      "The new department will be known as the Department of Foreign Affairs, Trade and Development." CBC News 2013

  • "Helen Keller International and CIDA shall jointly agree on country selection.
    1. Country selection shall be based on agreed upon criteria including:
       1. A focus on sub-Saharan Africa;
       2. High prevalence of undernutrition-related disease;
       3. High prevalence of wasting (low weight for height) and stunting (low height for age) in children under five;
       4. Under-five mortality rates are 70 per 1,000 live births or above;
       5. a SUN country, with the exception of fragile states; and
       6. Complementarity and alignment with CIDA MNCH countries, where possible"

    HKI grant agreement VAS Project 2013 - 2016, Pg 7.

• 47
  • See the "Improved overall health conditions" and "Interpreting the evidence in light of DEVTA" sections of our vitamin A supplementation intervention report for more information on baseline child mortality rates in VAS trials.
  • See this spreadsheet, "Child-years comparison" sheet, for national child mortality rates in countries with HKI-supported VAS programs.

• 48
  • See our vitamin A supplementation intervention report for details about rates of VAD among DEVTA participants.
  • "There is likely a threshold of VAD prevalence below which VAS is unlikely to have much impact on mortality. If there is high-quality data showing low VAD in a region, HKI thinks it is reasonable not to expect VAS to have a mortality impact there.

    "Organizations in the Global Alliance for Vitamin A (GAVA) currently use 10% VAD as the threshold at or above which VAS programs ought to be maintained in a region. The World Health Organization (WHO) classifies VAD rates of 20% or greater among preschool-aged children as a serious public health problem. VAD rates of less than 5% are accepted as not much of a concern." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 2.

  • WHO only recommends VAS programs in areas where VAD is a public health concern. WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011:
    • "In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation). The quality of the available evidence for all-cause mortality was high, whereas for all other critical outcomes it was moderate to very low. The quality of the available evidence for outcomes in human immunodeficiency virus (HIV)- positive children was moderate for all-cause mortality." Pg 1.
    • One dose of 100,000 IU of vitamin A is recommended for infants 6 to 11 months of age, and a 200,000 IU dose of vitamin A is recommended for children 12 to 59 months of age every four to six months. Table 1, Pg 5.
  • WHO defines vitamin A deficiency to be of mild public health importance when rates of vitamin A deficiency (defined as a measure of serum or plasma retinol <0.70 µmol/l) among preschool-aged children or pregnant women are between 2% and 10%, moderate public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are between 10% and 20%, and severe public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are greater than or equal to 20%. WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 8, Table 5.

• 49
  • "A threshold like this might not be observed if vitamin A had a pharmacological effect (i.e., if a large dose of vitamin A directly primed the immune system in some way, regardless of deficiency). However, Dr. Klemm thinks there is not any clear evidence to substantiate this hypothesis." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 2.
  • "Studies from the 1980s and early 1990s showed that vitamin A deficiency (VAD) was associated with increased overall child mortality and high-dose vitamin A supplementation (VAS) reduced overall mortality. This has led to the long-lived and strong assumption that VAS works by preventing VAD. Though intuitive, this assumption is contradicted by several facts.
    "First, high-dose VAS has no sustained effect on VAD, as measured by serum retinol or other biochemical markers. Frequent intakes of vitamin A in physiological doses—e.g., through food-based approaches, including fortification, and through regular low-dose supplementation—are highly effective in increasing serum retinol and reducing VAD. However, when the dose of vitamin A is as high as 200,000 IU (about 100 times the daily allowance), the liver may not be able to store it, and the excess is broken up and excreted. Thus, the rise in serum retinol resulting from 6-monthly VAS is small, transient, and lasts only for 1–3 months.
    "Second, if VAS worked by preventing VAD, then one would expect a clear linear association between the degree of underlying VAD and the effect of VAS: the higher the prevalence of VAD in a community, the larger the effect of VAS. However, this is not the case. Already, the first meta-analysis of the initial eight studies of the mortality effect of VAS noted that there was no association between the effect of VAS on mortality and the degree of underlying VAD at the population level. As presented in a recent review, this conclusion is substantiated when more recent studies are included." Benn 2017, Pg 1.

• 50
  • "There are roughly 40 relevant trials examining the impact of VAS on under-5 mortality, of which about 10 or 12 are large, randomized controlled trials. These trials took place in several different countries, mostly in the 1980s and 1990s.

    "Although the trials were all conducted in areas where VAD was present, the participants in the different trials were likely different from each other in substantial ways (e.g., some groups had higher baseline prevalence rates of infectious diseases or VAD than others)." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 1.

  • See Table 1 in our vitamin A supplementation intervention report for available information on VAD rates in the major trials included in Imdad et al. 2017 (excluding DEVTA).
  • In DEVTA, 64.8% of children in the control group who underwent biomedical visits were VAD (retinol <0.70 μmol/L) and 13.3% were severely VAD (retinol <0.35 μmol/L). Awasthi et al. 2013, Pg 1472.

• 51

  "The main objective of assessing vitamin A status is to determine the magnitude, severity and distribution of VAD in a population. Most surveys assess its prevalence in young children and, with increasing frequency, in pregnant or lactating women, as reported here. Although VAD is likely to be widespread following the preschool years, few data exist to reveal the extent of VAD in school-age and young adolescent children (16). Estimating the national prevalence is to be encouraged as such data aids in targeting regions for interventions, and provides baseline values for monitoring population trends and intervention programme impact over time.

  "Two sets of indicators of VAD are commonly used for population surveys: clinically assessed eye signs and bio-chemically determined concentrations of retinol in plasma or serum. The term xerophthalmia encompasses the clinical spectrum of ocular manifestations of VAD, from milder stages of night blindness and Bitot’s spots, to potentially blinding stages of corneal xerosis, ulceration and necrosis (keratomalacia) (17), as listed in Table 1. The stages of xerophthalmia are regarded both as disorders and clinical indicators of VAD, and thus can be used to estimate an important aspect of morbidity and blinding disability as well as the prevalence of deficiency. As corneal disease is rare, the most commonly assessed stages are night blindness, obtainable by history, and Bitot’s spots, observable by handlight examination of the conjunctival surface. Standard procedures exist for assessing xerophthalmia (17). Although night blindness and Bitot’s spots are considered mild stages of eye disease, both represent moderate-to-severe systemic VAD, as evidenced by low serum retinol concentrations (19), and increased severity of infectious morbidity (i.e. diarrhoea and respiratory infections) and mortality in children (5) and pregnant women (6, 20).

  "Measuring serum retinol concentrations in a population constitutes the second major approach to assessing vitamin A status in a population, with values below a cut-off of 0.70 μmol/l representing VAD (21), and below 0.35 μmol/l representing severe VAD. Although there is not yet international consensus, a serum retinol concentration below a cut-off of 1.05 μmol/l has been proposed to reflect low vitamin A status among pregnant and lactating women (22). While the distribution of serum retinol concentrations below appropriate cut-offs are considered to reflect inadequate states of vitamin A nutriture, a low biochemical concentration of retinol in circulation is not considered a VADD. Also, while an inadequate dietary intake of vitamin A or beta-carotene likely reveals an important and preventable cause of VAD in a population, it is not an indicator of vitamin A status." WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 2.

• 52

  "We used serum retinol concentration as an indicator of vitamin A deficiency because it is the most commonly used biomarker for assessment of subclinical vitamin A deficiency at the population level. Vitamin A deficiency was defined as serum retinol concentrations lower than 0·70 μmol/L. We did not use data for clinical indicators of vitamin A deficiency—namely, xerophthalmia (eg, night blindness, corneal scarring, and Bitot’s spots). We took this approach because night blindness and corneal scarring are not usually measured in children younger than 2 years of age. We attempted to convert prevalence of Bitot’s spots, which is measured more commonly in children, to prevalence of low serum retinol concentration but the association between the two variables was weak and heterogeneous (appendix), and hence did not allow robust conversion. This might be because Bitot’s spots are sometimes associated with previous versus current vitamin A deficiency. Some surveys have used retinol-binding protein as an indicator of vitamin A status instead of serum retinol because its measurement is less costly. However, we identified very few sources that had measured both serum retinol and retinol-binding protein and hence could not develop a statistical model to convert retinol-binding protein to serum retinol." Stevens et al. 2015, Pgs e529-e530.

• 53
  • See this spreadsheet ("VAD surveys" sheet).
  • WHO defines vitamin A deficiency to be of mild public health importance when rates of vitamin A deficiency (defined as a measure of serum or plasma retinol <0.70 µmol/l) among preschool-aged children or pregnant women are between 2% and 10%, moderate public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are between 10% and 20%, and severe public health importance when rates of vitamin A deficiency among preschool-aged children or pregnant women are greater than or equal to 20%. WHO Global prevalence of vitamin A deficiency in populations at risk 2009, Pg 8, Table 5.

• 54

  Stevens et al. 2015:

  • "We collated 134 population-representative data sources from 83 countries with measured serum retinol concentration data. We used a Bayesian hierarchical model to estimate the prevalence of vitamin A deficiency, defined as a serum retinol concentration lower than 0·70 μmol/L. We estimated the relative risks (RRs) for the effects of vitamin A deficiency on mortality from measles and diarrhoea by pooling effect sizes from randomised trials of vitamin A supplementation. We used information about prevalences of deficiency, RRs, and number of cause-specific child deaths to estimate deaths attributable to vitamin A deficiency. All analyses included a systematic quantification of uncertainty." Pg e528.
  • "The hierarchical model shares information to a greater degree where data are non-existent or weakly informative (ie, have large uncertainty), and to a lesser degree in countries or regions and in years with more data. We modelled trends over time as a linear trend. We did not include a non-linear term, as done for stunting, underweight, or anaemia,28–30 because fewer countries had several data sources for vitamin A deficiency than for other nutritional indicators; this data scarcity limits robust estimation of non-linear trends. The estimates were also informed by covariates that might help to predict vitamin A deficiency at the population level, including national income (logarithm of per-person gross domestic product [GDP] in inflation-adjusted international dollars), maternal education, proportion of population that lived in urban areas, mean weight-for-age Z score, and an aggregate metric of availability of calories and animal-source foods.31,32 The model included a variance term that accounted for unobserved design factors (sample design, season, retinol measurement method, etc) that led to variability in the data beyond that expected because of sample size. Finally, the model accounted for the fact that subnational data might have larger variation than national data by including an additional, empirically estimated, random effect for subnational data." Pg e530.

• 55

  Stevens et al. 2015:

  • "Regional prevalences in 1991 ranged from more than 40% in sub-Saharan Africa, south Asia, and east and southeast Asia and Oceania, to less than 25% in Latin America and the Caribbean, and in the region of central Asia, the Middle East, and north Africa. Nationally, the prevalence of vitamin A deficiency was at least 8% in every country; 100 countries had a prevalence of at least 20%, and hence would be classified as having a public health problem by WHO. Trends in the prevalence of deficiency from 1991 to 2013 varied by region, with a slight improvement at the worldwide level to 29% (17–42; PP of being a true decline=0·81). Deficiency significantly decreased in only one region: east and southeast Asia and Oceania, from 42% (19–70) to 6% (1–16; PP=0·99). The prevalence of deficiency might have decreased in Latin America and the Caribbean to 11% (4–23) in 2013 (PP=0·89) and in central Asia, Middle East, and north Africa to 11% (2–27) in 2013 (PP=0·76). In sub-Saharan Africa and south Asia, little change in prevalence occurred during the analysis period; both regions had prevalences of more than 40% for all years during the analysis period." Pg e532.
  • "In 1991, 39% (95% credible interval 27–52) of children aged 6–59 months in low-income and middle-income countries were vitamin A deficient. In 2013, the prevalence of deficiency was 29% (17–42; posterior probability [PP] of being a true decline=0·81). Vitamin A deficiency significantly declined in east and southeast Asia and Oceania from 42% (19–70) to 6% (1–16; PP>0·99); a decline in Latin America and the Caribbean from 21% (11–33) to 11% (4–23; PP=0·89) also occurred. In 2013, the prevalence of deficiency was highest in sub-Saharan Africa (48%; 25–75) and south Asia (44%; 13–79). 94 500 (54 200–146 800) deaths from diarrhoea and 11 200 (4300–20 500) deaths from measles were attributable to vitamin A deficiency in 2013, which accounted for 1·7% (1·0–2·6) of all deaths in children younger than 5 years in low-income and middle-income countries. More than 95% of these deaths occurred in sub-Saharan Africa and south Asia." Pg e528.

• 56

  See footnote above and results for Sierra Leone and Malawi in this spreadsheet ("VAD surveys" sheet).

• 57
  • See this spreadsheet ("Other VA programs" sheet).
  • HKI notes: "We view biofortification (e.g orange-flesh sweet potato and/or biofortified beta-carotene maize) and VA fortification of food vehicles (e.g. oil) as complementary strategies to VAS in deficient populations. If/when these dietary strategies have sufficient reach and quality to ensure adequate and sustained VA intake and status of risk populations (esp young children), VAS will no longer be needed. However, until that time, we view preschool VAS as a “safety net” in VA deficient and high mortality populations because of its life-saving potential. Implementation of industrial- and bio-fortification efforts in low and middle income countries has been challenging. A recent example from Cameroon (Engle-Stone R et al, Nutrients, 2017) showed no change in VA status in women or children one year following the introduction of mandatory fortification of oil with vitamin A. The lack of impact is likely explained by the fact that only 44% of oil was fortified to the target level specified in the regulation—highlighting the challenges of enforcing fortification regulations in countries with weak regulatory infrastructure and capacity." Comment provided in response to a draft of this review in August 2017.
  • We have not yet carefully reviewed Engle-Stone et al. 2017, referenced by HKI in the bullet point above.

• 58

  "There is likely a threshold of VAD prevalence below which VAS is unlikely to have much impact on mortality. If there is high-quality data showing low VAD in a region, HKI thinks it is reasonable not to expect VAS to have a mortality impact there.

  "Organizations in the Global Alliance for Vitamin A (GAVA) currently use 10% VAD as the threshold at or above which VAS programs ought to be maintained in a region. The World Health Organization (WHO) classifies VAD rates of 20% or greater among preschool-aged children as a serious public health problem. VAD rates of less than 5% are accepted as not much of a concern.

  "Despite a lack of recent micronutrient analyses in many African countries, HKI is confident that VAD is prevalent enough in many places for VAS to remain an impactful intervention. For instance, while HKI is not aware of any recent micronutrient deficiency data in Mali, it would be surprising if VAD were not prevalent there, given Mali's child mortality and malnutrition rates." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 2.

• 59
  • "Since 2010, Helen Keller International (HKI) has conducted post-event coverage (PEC) surveys in several African countries to estimate VAS and deworming coverage. These surveys provide a method to validate administrative figures and are important for identifying barriers to achieving high coverage." Janmohamed and Doledec 2017, Pg 822-823.
  • "HKI helps to track national VAS coverage through the governments tally-sheet system (also referred to as 'Administrative Data'. It also assesses VAS coverage in HKI “catchment areas” which for some countries is national scale and in other countries focuses on special high-risk or hard-to-reach regions and/or districts not assisted by other partners. HKI also conducts “Post-Event Coverage Surveys” or PECS, which are population-based representative coverage surveys which provide a more valid estimate of VAS coverage relative to the tally sheet system." HKI VAS documents guide for GiveWell 2017, Pg 2.
  • In all of the reports on post-event coverage surveys HKI shared with us, HKI worked in collaboration with the government to implement the survey:
    • "C’est dans ce contexte que Helen Keller International en collaboration avec le Ministère de la Santé à travers le Programme National de Nutrition a organisé une enquête de couverture de la vitamine A après la campagne de supplémentation en vitamine A intégrée à la vaccination contre la poliomyélite et le déparasitage de Mai 2012." HKI post-event coverage survey report Côte d'Ivoire (French) 2012, Pg 10.
      • Translated into English by Google Translate:
        "It is in this context that Helen Keller International, in collaboration with the Ministry of Health through the National Nutrition Program organized a survey of coverage of Vitamin A after the campaign for vitamin A supplementation in vaccination against polio and deworming in May 2012."
    • "The PEC survey in Ekiti and Katsina state Nigeria for the November/ December 2014 MNCHW campaign for children 6-59 months was conducted by Helen Keller International (HKI) with the support of the National Primary Health Care Development Agency (NPHCDA) and Federal Ministry of Health (FMOH)." HKI post-event coverage survey report Nigeria - Ekiti and Katsina states 2014, Pg 3.
    • "The PEC survey in Ekiti and Katsina state Nigeria for the November/ December 2014 MNCHW campaign for children 6-59 months was conducted by Helen Keller International (HKI) with the support of the National Primary Health Care Development Agency (NPHCDA) and Federal Ministry of Health (FMOH)." HKI post-event coverage survey report Sierra Leone 2013, Pg 3.
    • "In February 2014, Helen Keller International (HKI) and Tanzania Food and Nutrition Centre (TFNC) conducted a Post Event Coverage survey (PECS) following the January 2014 distribution event in Dar es Salaam." HKI post-event coverage survey report Tanzania 2015, Pg 5.
    • Authors of Clohossey et al. 2014, an academic paper on a post-event coverage survey following a child health week in Kenya in 2012, are listed as being affiliated with Helen Keller International and Kenya's Ministry of Public Health and Sanitation. Pg 169.
    • Authors of Dhillon et al. 2013, an academic paper on a post-event coverage survey following a VAS campaign in 2010, are listed as being affiliated with Helen Keller International and Tanzania's Food and Nutrition Centre, as well as other organizations. Pg 1.
    • "To validate VAS coverage and inform strategic planning of the MCHWs, the MoHS [Ministry of Health and Sanitation] and HKI conducted a national PEC survey immediately after the November 2011 MCHW [in Sierra Leone]." Hodges et al. 2013, Pg 173.
    • Authors of Sesay et al. 2015, an academic paper on a post-event coverage survey following a Maternal and Child Health Week in Sierra Leone in 2012, are listed as being affiliated with Helen Keller International and Sierra Leone's Ministry of Health and Sanitation. Pg 26.
    • HKI presentation: Reaching the hard to reach with vitamin A supplementation in low-performing health zones of DR Congo, a presentation on a post-event coverage survey conducted in Democratic Republic of the Congo (DRC) in 2012, lists authors from HKI and DRC's PRONANUT (National Nutrition Department).

• 60

  "Stage 2: Implementation:

  • Travel to survey sites;
  • Meet with local health officials and village leaders;
  • Map the cluster to be surveyed and divide the cluster into four quadrants;
  • Select a starting point at random in each quadrant and identify the first house to be surveyed;
  • Administer questionnaires to the target population (e.g., caretakers of children 6-59 months)."

  HKI post-event coverage survey manual 2014, Pg 4.

• 61
  • Guidelines:
    • HKI post-event coverage survey manual 2014
    • HKI post-event coverage survey data analysis manual 2014
  • Example reports:
    • HKI post-event coverage survey report Tanzania 2015
    • HKI post-event coverage survey report Sierra Leone 2013
    • HKI post-event coverage survey report Nigeria - Ekiti and Katsina states 2014
    • HKI post-event coverage survey report Côte d'Ivoire (French) 2012
    • HKI presentation: Reaching the hard to reach with vitamin A supplementation in low-performing health zones of DR Congo
  • Academic papers on HKI's post-event coverage surveys:
    • Clohossey et al. 2014, on a post-event coverage survey following a child health week in Kenya in 2012
    • Dhillon et al. 2013, on a post-event coverage survey following a VAS campaign in Tanzania in 2010
    • Hodges et al. 2013, on a post-event coverage survey following a child health week in Sierra Leone in 2011
    • Sesay et al. 2015, on a post-event coverage survey following a child health week in Sierra Leone in 2012

• 62

  To date, we have only analyzed documents HKI sent us on coverage surveys that took place during the period of its most recent grant from Global Affairs Canada (April 2013 to September 2016). See here for more information on Global Affairs Canada's most recent grant to HKI.

• 63
  • We have seen detailed reports on two of these coverage surveys: HKI post-event coverage survey report Nigeria - Ekiti and Katsina states 2014 and HKI post-event coverage survey report Tanzania 2015. See this spreadsheet, "Methods" sheet.
  • We note that we have also seen reports and academic papers on earlier HKI coverage surveys (see footnote above), but we have not yet reviewed them in depth.

• 64
  • "PECS should be conducted within six weeks of VAS, deworming and immunization events." HKI post-event coverage survey manual 2014, Pg 5.
  • For both of the coverage surveys that took place during HKI's most recent grant from Global Affairs Canada and for which we have seen detailed reports, surveys were reported to have taken place within six weeks of the distribution, but we have not yet seen information on when other recent surveys have taken place. See this spreadsheet, "Methods" sheet, for more information.

• 65
  • HKI notes: "All PECs conducted by HKI or supervised by HKI are conducted within 6 weeks of the VAS distribution event to minimize recall bias. In addition, when asking about VAS receipt a vitamin A capsule is shown to the respondent so that it is not confused with other products a child may have been given (e.g. oral polio vaccine or a deworming pill). These procedures are part of the training and supervision of PECS data collectors." Comment provided in response to a draft of this review in August 2017.
  • Child Health Cards have been used in some HKI-supported routine distribution systems, but our understanding is that they have not been used in mass distributions.
  • For example, in Sierra Leone, Child Health Cards were revised to include vitamin A supplementation at six months as part of an expanded routine immunization program:
    • "Since 2004, twice-yearly mass vitamin A supplementation (VAS) has equitably reached over 85 % of children 6–59 months old in Sierra Leone. However infants who turn 6 months after the event may wait until they are 11 months old to receive their first dose. The effectiveness of integrating VAS at 6 months into the Expanded Program of Immunization (EPI) in a revised child health card was studied. Health facilities matched according to staff cadre and work load were assigned to provide either a ‘mini package’ of VAS and infant and young child feeding (IYCF), a ‘full package’ of VAS, IYCF and family planning (FP), or ‘child health card’ only." Hodges et al. 2015, Pg 1985.

• 66
  • Janmohamed and Doledec 2017:
    • "Since 2010, Helen Keller International (HKI) has conducted post-event coverage (PEC) surveys in several African countries to estimate VAS and deworming coverage. These surveys provide a method to validate administrative figures and are important for identifying barriers to achieving high coverage. Comparisons of administrative and PEC survey data have revealed sizable discrepancies in VAS coverage in the African context [7,8]. However, this has not been rigorously evaluated and little is known about coverage differences between data sources for specific VAS delivery strategies and child age groups." Pg 2.
    • "Administrative coverage data were compared with PEC survey estimates for 52 VAS and 34 deworming dyads. Health system-reported coverage was higher than PEC estimates in 47 of 52 (90%) VAS campaigns and was 10% higher in 16 dyads, 10-20% higher in 16 dyads, 21-50% higher in 12 dyads, and >50% higher in three comparisons (Table 1). Discrepancies >30% were observed in 8 of 12 (67%) countries." Pg 3.
  • See our comparisons between administrative data and coverage surveys between 2013 and 2016 in this spreadsheet ("Results" sheet).

• 67

  See our summary of the results of recent coverage surveys we have seen in this spreadsheet ("Results" sheet).

• 68
  • See our summary of the results of recent coverage surveys we have seen in this spreadsheet ("Results" sheet). The total number of vitamin A supplements monitored through coverage surveys (according to administrative data) in HKI's recent 2013-2016 Global Affairs Canada grant is 24,555,540.
  • The total number of vitamin A supplements delivered in catchment areas receiving direct HKI support during HKI's recent 2013-2016 Global Affairs Canada grant is 192,281,849. Calculated from HKI VAS summary table, "VAS coverage" sheet.
  • 24,555,540 / 192,281,849 = ~13%.
  • Note that we are only counting "catchment areas receiving direct HKI support" in our denominator. HKI VAS summary table, "VAS coverage" sheet also reports on total vitamin A supplements delivered at a national level, but our understanding is that HKI only supports coverage surveys in regions where it directly supports VAS programs.
  • Our understanding is that the totals included in HKI VAS summary table, "VAS coverage" sheet include vitamin A supplements delivered through routine distributions, which are not monitored through post-event coverage surveys (see here). We don't know what proportion of the totals listed were delivered through routine mechanisms.

• 69
  • We have seen detailed reports on two coverage surveys from distributions conducted under HKI's most recent grant from Global Affairs Canada. See our summary of the methodologies of these surveys in this spreadsheet ("Methods" sheet).
  • Ekiti and Katsina states in Nigeria were chosen to be surveyed in 2014-15 because they had not yet been surveyed by HKI:
    • "Ekiti and Katsina are among the states supported by HKI for VAS implementation during MNCHW in Nigeria. According to tally sheet data the two states have recorded a considerably good VAS coverage rate over a 5 year period. However, there has not been any validation of this data before. Therefore, in order to validate VAS coverage in Ekiti and Katsina, PECS was conducted in January 2015 among caregivers of children aged 6-59 months who attended the November 2014 MNCH Week." HKI post-event coverage survey report Nigeria - Ekiti and Katsina states 2014, Pg 8.
  • The city of Dar es Salaam in Tanzania was chosen to be surveyed in 2015 because it had a history of low VAS coverage:
    • "In 2014, we opted to target the PECS data collection to areas which had historically performed poorly with VASD. Dar es Salaam, the countries business capital and largest city has a history of low VAS coverage, and 2014 proved to be no exception." HKI post-event coverage survey report Tanzania 2015, Pg 6.
  • We have seen results for 28 surveys taking place under HKI's most recent grant from Global Affairs Canada (see this spreadsheet, "Results" sheet), but, with the exceptions of the Ekiti and Katsina states survey and the Dar es Salaam survey mentioned above, we do not know why HKI or its partners chose to conduct surveys in those particular areas.
  • HKI notes: "The reasons can vary from one country to another. In some countries, PECS are undertaken to validate coverage in HKI-assisted regions. In others, the scope of a PECS depends on host country government and national working group interests or requests which might result in a survey implemented to provide a national, sub-national or area-specific (e.g.hard-to-reach) coverage estimate. PECs are NOT implemented for all VAS distribution rounds partly due to funding limitations but also because they are not warranted after each round. For example, if a PECs reveals low VAS coverage, only when demonstrable steps are taken to address the reasons for low coverage would a follow-on PECs be justified." Comment provided in response to a draft of this review in August 2017.

• 70

  We have seen an example of a coverage survey that was conducted in a particular area because HKI believed it was likely to have low levels of coverage (see above footnote). But out of all of the surveys we have seen results from, we are uncertain what proportion of areas were chosen to be surveyed due to suspected low rates of VAS coverage in the area, and what proportion were selected for other reasons.

• 71
  • HKI 6-month contact point standard methodology, Pgs 2-3:
    • "Prior to the introduction of the 6-month contact point activities, a baseline survey must be conducted.
    • "Review the ‘6107 baseline survey’ for both caretakers of children age 9-12 months and for healthcare workers
    • "Modify the 6107 baseline survey to include country specific factors and messaging. Indicators which should not be modified include:
      • Age of first receipt of VAS
      • Age of first receipt of Measles vaccine
    • "For Methodology of the 6107 baseline survey, please refer to the cluster selection handout. The EPI cluster sampling methodology has been selected for the baseline with a minimum of 30 clusters from the targeted region randomly selected, where the probability of a cluster being selected proportional to the population size. A minimum of 10 households with children 9-12 months will be selected to complete the baseline.
      […]
      "After a minimum of 7 months, the 6107 end line survey should be conducted with caretakers of children age 9-12 months and to healthcare workers."
  • HKI notes: "The 6-mo contact point has been a recent innovation piloted in several countries. During the pilot phase in Sierra Leone, Senegal, Cote d’Ivoire and elsewhere, HKI set up robust systems for monitoring receipt of VAS among children at 6-7 m of age. This was done to assess coverage changes as a result of this new contact point. HKI would like to assist governments to scale up the 6 m contact point and, in the process, integrate VAC receipt at 6 m into the Child Health Cards (as a means to record VAC receipt) and into the country’s routine health information system as a means to assess coverage." Comment provided in response to a draft of this review in August 2017.
  • We have seen a presentation on the results of one of these studies (HKI 6-month contact point presentation Sierra Leone 2012), but we have not yet reviewed the results in detail.

• 72

  "HKI teams work closely with national governments to support the policy, strategy and tool development mentioned above, but HKI’s major added value is its capacity to rapidly deploy technical support to the sub national level to assist local health authorities with implementing national VAS strategies. HKI focus its efforts sub-nationally because local level (at state and/or district level) health system performance is key to ensuring high VAS coverage. It also allows HKI to support other health system functions that also improve the delivery of other maternal and child health services.

  "In concrete terms, HKI, in consultation with national government counterparts, directs its support to low performing areas to help local program managers identify and solve VAS coverage barriers. This involves organizing workshops with state and district health authorities to analyze what worked and what did not. HKI teams then spend time with health managers to help them identify feasible and cost-effective solutions to improve performance of the targeted services and accompany them through the whole programming cycle (i.e. planning, budgeting, implementation, real time supervision and monitoring, and finally evaluation of the progress made). One cycle sometimes proves insufficient so the HKI teams continue working with each targeted health district until minimum thresholds of performance are met. Funds are used to support deployment of HKI teams in remote areas, to support financing workshops and joint field supervisions, to provide training for field actors, or to organize coverage surveys and review meetings at the end of the exercise. In some cases, HKI provides funds directly to the local authorities to fill financial gaps they may experience ensuring rigorous financial accountability. When conditions for a change of approach are met, HKI provides technical assistance to local authorities to design, implement and monitor with them innovative approaches such as the 6-month contact point or SMS messaging." HKI VAS documents guide for GiveWell 2017, Pg 2.

• 73
  • "We offer two recent examples, although there are others. The first is from Cameroon where in a three-year time span VAS coverage increased from ~50% to ~80% with HKI technical assistance (see attached report). A second example is from Kenya where, in addition to health-facility based distribution, HKI’s pioneered the use Early Child Education Centers as distribution points for VAS. This resulted in an increase from 10% to 70% in VAS coverage (see attached report)." HKI responses to GiveWell's questions May 2017, Pg 8.
  • Documents on these examples:
    • HKI lessons learned on VAS in six urban health districts in Cameroon 2014
    • HKI presentation: Kenya VAS integrated into Early Childhood Development Centers 2016

• 74

  "In the past, most VAS programs in Sub-Saharan Africa have delivered supplements to children in door-to-door mass campaigns tied to polio immunization campaigns, but countries need to find alternative methods of delivering VAS, since many polio campaigns are ending due to progress in polio elimination. Transitioning to 'routine delivery' (in which caregivers bring children to facilities combined with periodic outreach/delivery posts within communities to receive VAS at appropriate ages) appears to be the most sustainable long-term option.

  "HKI expects its capacity to assist governments to transition to routine service delivery to be particularly critical over the next five years or so. HKI’s flexibility to send teams from one region to another to provide technical assistance and build capacity has been valuable, and HKI has a recognized expertise in this area.

  "Transitioning to routine delivery of VAS can be challenging. When Kenya transitioned from door-to-door to routine delivery without technical assistance, coverage fell from roughly 90% to 20% during one campaign, and it took several years to bring coverage rates back up. HKI hopes to help prevent this in other places, since it is more difficult and requires more resources to bring coverage rates back up than to prevent them from falling. UNICEF has asked HKI to provide transition assistance in Benin and Togo." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 5-6.

• 75
  • HKI presentation: Kenya VAS integrated into Early Childhood Development Centers 2016
  • HKI presentation: Senegal Successful M-Health strategy: The routine delivery of Vitamin A Supplementation at six months using SMS appointment reminders
  • HKI presentation: SMS reminders and vocal messages increase adherence to immunization and 6-month vitamin A supplementation 2016
  • HKI SMS reminder pilot study report Côte d'Ivoire 2015

• 76

  "Many African countries are facing funding shortfalls around VAS, and some planned VAS mass campaigns have had to be cancelled. For instance, in Mali (which HKI does not currently have funds to support, but which received support from HKI for VAS programs in 2013-16), it is not clear whether VAS mass campaigns will occur at all without external technical assistance from HKI. HKI still expects vitamin A capsules to be provided to countries in sufficient numbers, but there is a risk of millions of capsules remaining undistributed if campaigns are underfunded." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 7.

• 77
  • Kupka et al. 2016, a study written by HKI staff and others, concludes that successful six-month contact points would reduce mortality further than the status quo of receiving VAS at a mass distribution event at an average of 9 months of age:
    • "VAS linked to a 6-month visit could reduce infant mortality by an additional 1.95 (95% confidence interval [CI]: 1.38-2.52) and 1.63 (95% CI: 1.15-2.11) percentage points compared to VAS through CHDs and at 9 months, respectively. The HVAC models indicate that VAS at 6 months is safe even in the presence of a second VAS dose 1 month later and other food-based vitamin A control strategies." Pg 1.
  • We have not carefully reviewed the methodology of this study.

• 78
  • Kupka et al. 2016, a study written by HKI staff and others, concludes that successful six-month contact points would reduce mortality further than the status quo of receiving VAS at a mass distribution event at an average of 9 months of age:
    • "VAS linked to a 6-month visit could reduce infant mortality by an additional 1.95 (95% confidence interval [CI]: 1.38-2.52) and 1.63 (95% CI: 1.15-2.11) percentage points compared to VAS through CHDs and at 9 months, respectively. The HVAC models indicate that VAS at 6 months is safe even in the presence of a second VAS dose 1 month later and other food-based vitamin A control strategies." Pg 1.
  • We have not carefully reviewed the methodology of this study.

• 79

  "When the correct age-specific dose of vitamin A is given with immunization, mild side-effects or adverse events may be observed. However, they are rare and transient. Occasionally, some children experience loose stools, headache, irritability, fever, nausea, and vomiting. Depending on age and the dosage given, the excess rate of occurrence of these mild symptoms of intolerance has shown be in the range of 1.5-7% (Florentino et al., 1990; West et al., 1992; Agoestina et al., 1994). These side-effects disappear in practically all children within 24-48 hours (Florentino et al., 1990; West et al., 1992; Agoestina et al., 1994)." WHO vitamin A supplements adverse events, Pgs 1-2.

• 80

  "When the correct age-specific dose of vitamin A is given with immunization, mild side-effects or adverse events may be observed. However, they are rare and transient. Occasionally, some children experience loose stools, headache, irritability, fever, nausea, and vomiting. Depending on age and the dosage given, the excess rate of occurrence of these mild symptoms of intolerance has shown be in the range of 1.5-7% (Florentino et al., 1990; West et al., 1992; Agoestina et al., 1994). These side-effects disappear in practically all children within 24-48 hours (Florentino et al., 1990; West et al., 1992; Agoestina et al., 1994)." WHO vitamin A supplements adverse events, Pgs 1-2.

• 81

  WHO vitamin A supplements adverse events:

  • "The administration of excessive amounts of vitamin A can lead to toxicity, known as hypervitaminosis A. The amount required to cause toxicity will vary among individuals." Pg 1.
  • "Worldwide, the incidence of hypervitaminosis A is a very minor problem compared with the incidence and effects of vitamin A deficiency. An estimated 200 cases of hypervitaminosis A occurs annually…" Pg 1.
  • "Hypervitaminosis does not result from public health intervention programs. Rather toxicity has been associated with the abuse of vitamin A supplements and with diets extremely high in preformed vitamin A (i.e., foods of animal origin). Toxic reactions provoked by large doses of vitamin A are well-known to occur following either intake of liver rich in vitamin A (e.g., polar bear, halibut or whale) or by excessive administration of vitamin A preparations (Miller & Hayes, 1982)." Pg 2.
  • "Acute vitamin A toxicity (single ingestion of 25,000 IU per kg or more): Signs and symptoms may be delayed for 8 to 24 hours and include manifestations such as nausea, vomiting, diarrhea, changes in humour (irritability, drowsiness, dizziness, lethargy), increased intracranial pressure (headache, bulging of fontanelle, diplopia, papilloedema), skin changes (erythema, pruritus, desquamation). Peeling of skin around mouth may be observed from 1 to several days after ingestion and may spread to the rest of the body (Miller & Hayes, 1982; Bendich & Langseth, 1989; Hathcock et al., 1990; CPS, 1999; Parfit, 1999)." Pg 2.

• 82
  • "Hypervitaminosis does not result from public health intervention programs. Rather toxicity has been associated with the abuse of vitamin A supplements and with diets extremely high in preformed vitamin A (i.e., foods of animal origin). Toxic reactions provoked by large doses of vitamin A are well-known to occur following either intake of liver rich in vitamin A (e.g., polar bear, halibut, or whale) or by excessive administration of vitamin A preparations (Miller & Hayes, 1982)." WHO vitamin A supplements adverse events, Pg 2.
  • HKI told us that receiving two doses of vitamin A supplements within a short time period would not meet toxicity thresholds:
    • "[GiveWell:] In countries where six-month contact points have been initiated, is there a risk of a child receiving a 'double dose' of VAS in a short time period (one from a facility visit when the infant is six months old, and another at the next biannual Child Health Day or door-to-door campaign)? Would receiving a double-dose potentially be dangerous? (Even if they aren't dangerous, we're also concerned about double-doses because they wouldn't be an effective use of resources.)

      "[HKI:] This is a legitimate question and one we have had to think about carefully as we started to promote and support the 6 month contact point (6MCP). First, receiving two doses in a short time frame poses some, but minimal, risks for children as the toxicity thresholds go far beyond receiving two doses (see attached document on Adverse events following administration of VAS)."
      HKI responses to GiveWell's questions May 2017

• 83
  • Kagin et al. 2015 estimates that it cost $0.76 to deliver a vitamin A supplement through Child Health Day programs in the HKI-supported Littoral Region of Cameroon in Fall 2013. Pg S178, Table 2. More details from Kagin et al. 2015:
    • Two of the paper's authors are listed as being affiliated with HKI; other authors are affiliated with universities, the Bill and Melinda Gates Foundation, and a consulting group. Pg S172.
    • "The CHD implementation in most of the regions was managed by the Ministry of Health, UNICEF, and other partners. However, the Littoral region (located in Southwest Cameroon and including the city of Douala) was managed by HKI. This made differences in how costs were allocated across categories. To avoid making assumptions about how costs were allocated to similar activities across Cameroon, we instead grouped costs separately for the Littoral region and for the other regions and then aggregated up to the national level. We divided the costs of the Littoral region into 7 categories: (1) distribution, the actual distribution of the VA capsules to the children through door-to-door visits or through fixed points as well as direct communication to households about the program itself; (2) the VA capsule costs including freight and delivery costs; (3) supervision, the labor per diems, transportation, and other costs related to managing the project at different levels. This also includes costs of supervision, for example, management by HKI and their overhead costs; (4) training activities, given to all the teams and schools involved in the CHDs; (5) other communication of the program via TV and radio; (6) evaluation of the program and data collection activities; and (7) central administrative costs divided among the different regions." Pg S176.
    • "To determine the costs of VA programs in Cameroon, we used budgets from existing S174 Food and Nutrition Bulletin 36(Supplement 3) programs, where available. Those obtained were for CHDs that delivered VA capsules and DW tablets and edible oil fortified with VA. In some cases, the budgets were adjusted or adapted to isolate certain interventions or to estimate the costs of a similar intervention, as described subsequently. When actual budgets were not available, we constructed the costs using known unit costs of program components combined with expert knowledge." Pgs S174-175.
  • HKI cost-effectiveness analysis of VAS in DRC (French) 2016 estimates that a vitamin A supplement costs $0.35 to deliver via a Child Health Day, and $0.43 via a door-to-door approach, or a weighted average of $0.40.
    • "Le coût de l’enfant supplémenté pour l’approche JSE et pour l’approche porte à porte se présente respectivement à 0,35 $ et 0,43$. Tandis que pour l’ensemble de deux approches, les coûts par enfant supplémenté est de 0,40$ pour les 4 Zones de Santé enquêtées. Pris isolément, le coût de supplémentation par enfant dans les 4 Zones de Santé varie de 0,35$ à 0,43 $ selon le milieu de résidence avec une moyenne de 0,40 $." Pg 17.
      • Translated into English by Google Translate:
        "The cost of the child supplemented for the JSE [Child Health Day] approach and the door-to-door approach is $ 0.35 and $ 0.43, respectively. While for all two approaches, the costs per child supplemented is $ 0.40 for the 4 Health Zones surveyed. Taken separately, the cost of supplementation per child in the 4 Health Zones varies from $ 0.35 to $ 0.43 depending on the residential environment with an average of $ 0.40."
    • Types of costs included in this total (translated from French to English using Google Translate): "planning meetings, capsule distribution, social mobilization and CCC, supervision, training/briefing, technical assistance coverage survey post-event (PECS), scissors and prints, transport of capsules, coordination, staff of PRONANUT [DRC nutrition program]." Pg 14, Table 1.
    • HKI contributed 65% of the total cost of the program, UNICEF contributed 12%, and DRC's Ministry of Health contributed 22%. Pg 20, Table 7.

• 84
  • HKI spent a total of roughly $23.7 million USD over the course of its recent Global Affairs Canada grant (April 2013 to March 2016). See this spreadsheet. Note that we exclude the "Program Extension" period of April-September 2016 from this total. We exclude the budgeted costs of the program extension period because we have not seen information on treatments delivered in this period.
  • Total VAS doses delivered in HKI's 2013-2015 budget years (April 2013 to March 2016) in countries receiving support from HKI: 506,895,877. Calculated from HKI VAS summary table, "VAS coverage" sheet. Our understanding is that the total doses delivered listed on this sheet are calculated using administrative data.
  • Total VAS doses delivered in HKI's 2013-2015 budget years (April 2013 to March 2016), only in catchment areas receiving direct support from HKI: 192,281,849. Calculated from HKI VAS summary table, "VAS coverage" sheet. Our understanding is that the total doses delivered listed on this sheet are calculated using administrative data.
  • Counting only HKI's spending, the figures above suggest a cost per supplement delivered estimate of $0.05 ($23,700,000/506,895,877) if counting all supplements delivered in countries that received support from HKI, and an estimate of $0.12 ($23,700,000/192,281,849) if only counting supplements delivered in areas that received direct support from HKI.
  • In HKI VAS project year 2 report 2015, HKI provided estimates of the total budget for Child Health Day VAS events in each country it supports, and the percentages of the total budget financed by different actors (HKI, UNICEF, country governments, etc). HKI VAS project year 2 report 2015, Pg 25, Table 16.
  • We were unable to use this information to calculate a cost per supplement estimate accounting for the costs paid for by all actors because we have not seen information on how many vitamin A supplements were delivered through Child Health Days alone in each country. (We have only seen total numbers of treatments delivered through Child Health Days, National Immunization Days, and routine delivery.)

• 85

  For example, see cost analysis in our review of Malaria Consortium's seasonal malaria chemoprevention program.

• 86
  • Our CEA simply treats 2015 child mortality rates as a "baseline." But VAS programs have been ongoing in many countries in sub-Saharan Africa since the 1990s and early 2000s. Our current model may underestimate the impact of VAS since it does not account for the fact that lower mortality rates in 2015 may be in part due to the VAS program itself.
  • We currently assume that monthly mortality rates are constant across the 29- to 364-days of age period; we would guess that mortalities are actually likely to be more concentrated towards the beginning of this period.
  • VAS is only thought to have an impact on the subset of child mortality due to infectious diseases. We have not yet investigated whether "vitamin A-susceptible" mortalities have declined at a faster rate than overall child mortality declines in sub-Saharan Africa. If vitamin A-susceptible child mortalities make up a smaller proportion of overall child mortality rates now than during the time of the VAS RCTs, the all-cause mortality risk ratio calculated in Imdad et al. 2017 may overestimate the mortality impact of VAS programs now.
  • Our CEA uses the fixed effects meta-analysis of the impact of VAS on all-cause mortality calculated in Imdad et al. 2017. We have not yet investigated whether a random effects meta-analysis would be more appropriate, but we note that this choice would make a substantial difference in the outcome of our CEA (a relative risk [RR] of 0.88 for the fixed effects analysis, and an RR of 0.76 for the random effects analysis).
  • Our current CEA uses a simple average of mortality rates in countries in which HKI supported VAS programs in 2013 to 2016. A weighted average, using more specific information on HKI's plans for the use of additional funding, would likely be more accurate.
  • Many of our CEAs discount government and drug costs; our CEA for HKI does not currently use any of these discounts.
  • The only benefit counted in our CEA is child mortalities averted. VAS may also have an impact on child morbidity.
  • We have not yet considered whether donations to HKI for VAS would be partially fungible with donations for other HKI programs, or whether an "alternate funders' adjustment" for donations to HKI would be appropriate.
  • VAS may be delivered alongside other child health interventions, such as deworming. We have not yet determined how or whether to account for benefits from other interventions delivered alongside vitamin A in our cost-effectiveness analysis.

• 87

  "To operate at full capacity in the thirteen countries in sub-Saharan Africa it operated in in 2013 to 2016, HKI would probably need less than the $10 million per year that it previously used. Over the next five years or so, HKI expects that it would primarily be assisting countries in transitioning from door-to-door distribution of vitamin A to the use of routine systems for distribution. HKI estimates that a budget of between $6 million and $7 million per year (i.e. roughly $500,000 per country per year) for the next five to eight years would be sufficient for this work." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 8.

• 88
  • In this review, we refer to the agency with its current name, Global Affairs Canada. Documents we cite may refer to former name of the agency, the Department of Foreign Affairs, Trade and Development of Canada (DFATD), or the Canadian International Development Agency (CIDA), which was absorbed into DFATD in 2013.
    • "The names of several departments are being changed as follows:
      […]
      Foreign Affairs, Trade and Development Canada to Global Affairs Canada" Canada Privy Council Office Machinery of Government Changes 2015
    • "The agency that handles Canada's international aid is going to be brought into the Department of Foreign Affairs, the government announced Thursday in the federal budget.

      "It's not yet clear how the move will affect the work of the Canadian International Development Agency, which is currently the responsibility of International Co-operation Minister Julian Fantino, but the fact the minister's powers are about to be enshrined in law is seen as a positive sign for its future.

      "In the past, ministers in charge of CIDA haven't had the same enshrinement in law as other federal cabinet ministers.

      "The new department will be known as the Department of Foreign Affairs, Trade and Development." CBC News 2013

  • HKI plays a range of technical assistance roles to national vitamin A supplementation programs in Africa. These have been almost exclusively supported from grants from the Canadian Government, the most recent being a grant entitled, “Scaling Up Nutrition through Integrated Life-saving Interventions Project-2013-2016.” HKI VAS documents guide for GiveWell 2017, Pg 1.
  • "Annex 1 - History of grants devoted to VAS implemented by HKI with support from GAC [Global Affairs Canada]" lists five grants from GAC to HKI, beginning in January 2006, totaling around $80 million CAD in total funding. HKI VAS concept note, Pg 22.

• 89
  • "Since 2002, Helen Keller International (HKI) has partnered with UNICEF and the Department of Foreign Affairs, Trade and Development of Canada (DFATD), formerly the Canadian International Development Agency (CIDA), to work towards increasing and sustaining vitamin A supplementation (VAS) coverage in countries with a high burden of child mortality and nutrition-related disease. The partnership started in five sub-Saharan countries, and expanded during the 2005–2008 period to nine countries, where it focused primarily on shifting to twice-yearly distribution, sustainability and building national capacity. The subsequent project (Scaling up Child Health and Nutrition in Sub-Saharan Africa) ran from 2009–2013 and operated in 13 countries.

    "The current grant amounts to CDN$29,000,000 over a three-year period (2013–2016) and is entitled 'Scaling Up Nutrition through Integrated Life-Saving Interventions.' The two primary anticipated outcomes of the project are:

    1. enhanced healthy nutritional practices for children 6–59 months through maintained high coverage of child health and nutrition services; and
    2. increased national ownership of child survival activities (in transitioning countries)."

    HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, Pg 1.

  • "Annex 1 - History of grants devoted to VAS implemented by HKI with support from GAC [Global Affairs Canada]" lists a grant of $29 million CAD from Global Affairs Canada to HKI for 13 countries in sub-Saharan Africa implemented between February 2013 and May 2016. HKI VAS concept note, Pg 22.

• 90
  • "HKI requests support from Global Affairs Canada (GAC) for a five-year program (June 2016 – May 2021) focusing on the following core objectives:
    1. Provide VAS to all children 6 to 59 months in high VAD and high mortality countries in SSA through sustainable, locally managed delivery mechanisms;
    2. Continue institutionalization of VAS services within national health systems;
    3. Integrate VAS services within a comprehensive nutrition and health package for pregnant and lactating mothers and children less than five using a health systems strengthening approach ensure effective management as well as equity, quality and access to interventions that address the unacceptably high prevalence of VAD."

    HKI VAS concept note, Pg 4.

  • "In a three-year period between 2013 and 2016, GAC granted around $30 million (CAD) to HKI as well as provided significant funding to UNICEF and the Canadian-based Nutrition International for VAS programs in SSA. Going forward, GAC will be passing its VAS funding directly to UNICEF to ease the administration burden of managing two separate grants with the expectation that UNICEF will use some of this funding to provide grants to other organizations supporting VAS programs, including HKI." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pgs 6-7.

• 91

  "In a three-year period between 2013 and 2016, GAC granted around $30 million (CAD) to HKI as well as provided significant funding to UNICEF and the Canadian-based Nutrition International for VAS programs in SSA." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 6.

• 92

  "In a three-year period between 2013 and 2016, GAC granted around $30 million (CAD) to HKI as well as provided significant funding to UNICEF and the Canadian-based Nutrition International for VAS programs in SSA. Going forward, GAC will be passing its VAS funding directly to UNICEF to ease the administration burden of managing two separate grants with the expectation that UNICEF will use some of this funding to provide grants to other organizations supporting VAS programs, including HKI." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pgs 6-7.

• 93

  "In a three-year period between 2013 and 2016, Global Affairs Canada (GAC) granted around $30 million (CAD) to HKI and provided significant funding to UNICEF and the Canadian-based Nutrition International for VAS programs in sub-Saharan Africa. Going forward, GAC will be passing its VAS funding directly to UNICEF to ease the administration burden of managing two separate grants with the expectation that UNICEF will use some of this funding to provide grants to other organizations supporting VAS programs, including HKI. Under the new arrangements between GAC and UNICEF, the amount of funding to support VAS appears to be considerably lower than in previous years because it covers four rather than three years of VAS activities, supports immunization activities in addition to VAS, and because the value of the Canadian dollar relative to the US dollar has decreased ~30% since 2013.

  "UNICEF, as a multi-national United Nations program, works in many more countries than HKI does, and has had to prioritize certain countries over others to receive GAC funding. Many UNICEF priority countries for VAS are not countries where HKI has a presence, which has meant fewer resources to continue HKI’s VAS work in those countries not prioritized by UNICEF. There are several other global trends that cause concern for the future of funding VAS programs:

  • HKI is concerned that over recent years, global attention to and funding for VAS has waned, most likely due to competition with other pressing priorities and interventions emerging to address broader maternal-newborn-child-adolescent health and nutrition needs.
  • 'Donor fatigue' may be contributing to declining interest in supporting VAS.
  • Due to the changing political landscapes in the U.S., Europe, and the U.K., HKI anticipates major reductions in development aid for maternal and child health programs, including for nutrition and VAS. This may further threaten the long-term funding situation for child survival interventions such as VAS in countries where the need remains high, including many of those in which HKI has a presence."

  GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pgs 6-7.

• 94

  Rolf Klemm, Vice President of Nutrition, HKI, email to GiveWell on July 14, 2017

• 95

  GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017:

  • "There are several other global trends that cause concern for the future of funding VAS programs:
    • HKI is concerned that over recent years, global attention to and funding for VAS has waned, most likely due to competition with other pressing priorities and interventions emerging to address broader maternal-newborn-child-adolescent health and nutrition needs.
    • 'Donor fatigue' may be contributing to declining interest in supporting VAS.
    • Due to the changing political landscapes in the U.S., Europe, and the U.K., HKI anticipates major reductions in development aid for maternal and child health programs, including for nutrition and VAS. This may further threaten the long-term funding situation for child survival interventions such as VAS in countries where the need remains high, including many of those in which HKI has a presence." Pg 7.
  • "HKI has not received much interest in VAS from other potential funders, even when proposing embedding it in more comprehensive health system support programs. It does have support in some specific cases (e.g. Irish Aid is supporting HKI in Sierra Leone to cover a gap for scale-up to routine service delivery). HKI thinks potential funders may not recognize the importance of maintaining VAS as current platforms for it (e.g. polio programs) disappear." Pg 8.

• 96

  "To operate at full capacity in the thirteen countries in sub-Saharan Africa it operated in in 2013 to 2016, HKI would probably need less than the $10 million per year that it previously used. Over the next five years or so, HKI expects that it would primarily be assisting countries in transitioning from door-to-door distribution of vitamin A to the use of routine systems for distribution. HKI estimates that a budget of between $6 million and $7 million per year (i.e. roughly $500,000 per country per year) for the next five to eight years would be sufficient for this work." GiveWell's non-verbatim summary of a conversation with Helen Keller International, June 1, 2017, Pg 8.