Vitamin A Supplementation - November 2018 version

We have published a more recent version of this page. See our most recent version of this page.

In a nutshell

Vitamin A supplementation involves treating all children aged approximately 6-months to 5-years in areas at high-risk for vitamin A deficiency with high-dose vitamin A supplements two or three times per year. Vitamin A supplementation is generally inexpensive on a per-person basis.

Trials conducted in the 1980s and early 1990s found that vitamin A supplementation greatly reduces child mortality. A more recent trial with more participants than all previous studies combined did not find a statistically significant effect. This raises questions about the impact we should expect from vitamin A supplementation today.

We believe that vitamin A supplementation may be one of the most cost-effective ways to save lives when the program is high quality and delivered in locations with high child mortality rates. See our most recent cost-effectiveness analysis for our cost per life saved estimate for Helen Keller International's vitamin A supplementation programs.

Published: January 2015; Updated: November 2018

Previous version of this page:

  • November 2017 version

    Table of Contents

    Basics of the program

    What is vitamin A deficiency?

    Vitamin A deficiency (VAD) is a common condition in the developing world that can cause stunting, anemia, dry eyes (the leading cause of preventable childhood blindness), susceptibility to infection, and death.1 VAD is most common in the World Health Organization's Africa and South-East Asia Regions.2 Infants, children, and pregnant or lactating mothers with low vitamin A intake are at particularly high risk of the negative health impacts of VAD.3 Deficiency is most likely in diets that include few animal sources and little fortified food.4

    What is vitamin A supplementation?

    Oral vitamin A supplementation is “the most widely practiced approach to controlling VAD in most high risk countries.”5 The version of the intervention we focus on here is periodic prophylactic vitamin A supplementation (VAS) to pre-school children of ages six months to sixty months who are at risk of VAD.6 For the remainder of this report, “VAS” refers to the policy of providing all pre-school aged children in an area with the WHO-recommended dose of vitamin A every four to six months.7

    Vitamin A supplementation is inexpensive—around a dollar per person reached per supplementation round in total costs.8 Children generally receive VAS in combination with other health services.9

    UNICEF has defined three categories of priority countries for vitamin A supplementation:10

    1. countries with an under-five mortality rate exceeding 70 deaths per 1,000 live births.
    2. countries where VAD is a public health problem according to national-level assessments registered in the WHO’s Micronutrient Deficiency System.
    3. countries with a history of VAS programming which may have lowered VAD incidence and the under-five mortality rate.

    Program track record

    Many large, randomized (or quasi-randomized) controlled trials have been conducted to determine the impact of VAS.11 A 2010 Cochrane meta-analysis of studies that primarily took place in the 1980s and early 1990s, estimated that VAS reduces child mortality by 24%, a large and statistically significant reduction.12

    However, these results differ from the results of the Deworming and Enhanced Vitamin A Study (DEVTA), a more recent trial that was “the largest randomized trial ever conducted” and “included four times the combined participants of all included studies in [the Cochrane] review.”13 DEVTA, which took place in Uttar Pradesh, India, estimated that VAS reduced child mortality by a statistically insignificant 4%.14 DEVTA's confidence interval included the possibility of up to an 11% decline in child mortality and also included the possibility of no effect at all.15

    We believe that VAS can have a meaningful impact on childhood mortality, but the contrast between the results of the Cochrane review and the outcome of DEVTA leave us uncertain about the size of this impact and the conditions that are necessary to achieve it.

    Cochrane review

    The Cochrane Collaboration conducted a meta-analysis of randomized (and quasi-randomized) trials of VAS published through October 2010.16 The primary outcome of interest was all-cause mortality, but analyses of cause-specific mortality and other outcomes were also reported.

    All-cause mortality

    Cochrane’s meta-analysis of seventeen randomized controlled trials, which reported data on all-cause mortality, estimated that VAS reduces all-cause mortality by 24% with a 95% confidence interval ranging from 17% to 31%.17 The Cochrane review concludes that the included studies perform well with respect to various potential biases.18

    The main source of uncertainty around the review’s all-cause mortality estimate was the impact of DEVTA. When the Cochrane review was published, initial results from DEVTA had been reported in a publicly available PowerPoint presentation, but the details of the study and its implementation had not been published.19 As a sensitivity test, the Cochrane authors added DEVTA’s results to their meta-analysis and found that incorporating DEVTA’s results alongside the other trials in an inverse-variance weighted average reduced the estimated benefit of VAS by half, to a 12% reduction in child mortality (with a 95% confidence interval ranging from 6% to 16%).20 Unfortunately, because the details of DEVTA’s methods, implementation, and results were not published until 2013, the Cochrane authors could not account for the differences between the results of DEVTA and the results of previous trials.21 DEVTA is discussed in detail below.

    Cause-specific mortality

    The estimates of the effect of VAS on cause-specific mortality in the Cochrane review are imprecise and vulnerable to bias and measurement error.22 These estimates should be interpreted with caution.

    The Cochrane review estimated that VAS reduces diarrhea-related mortality by 28% (95% CI: 9% to 43%), measles-related mortality by 20% (not statistically significant; 95% CI: 24% increase to 49% decrease), and lower respiratory tract infection (LRTI)-related mortality by 22% (not statistically significant; 95% CI: 14% increase to 46% decrease).23

    Other outcomes

    Mortality was the primary outcome of interest in most studies of VAS and there is less evidence of VAS’s impact on morbidity. There is, however, moderate evidence that VAS substantially reduces measles morbidity and weak evidence suggesting that VAS somewhat reduces diarrhea morbidity.24 The Cochrane review found no evidence of an effect on lower respiratory tract infection (LRTI) morbidity.25 VAS also “reduces night blindness and potential precursors to blindness, namely Bitot’s spots and xerophthalmia.”26 Because studies may be more likely to report morbidity data when they find positive results, these analyses are potentially affected by selective reporting bias.27

    Few studies reported data on side effects, so Cochrane’s estimates of side effects are highly uncertain and vulnerable to selective reporting bias. There is some evidence that approximately four percent of children who receive VAS experience short-term vomiting due to the intervention.28 There was not enough data to include other side-effects in the meta-analysis.29

    The Deworming and Enhanced Vitamin A (DEVTA) Study

    The results from DEVTA, a very large VAS trial in north India, differ substantially from the headline results of the Cochrane review.30 The results of DEVTA were published in Awasthi et al 2013, which estimates that VAS reduced child mortality by 4% and cannot rule out the possibility that VAS did not affect child mortality at all (the 95% confidence interval ranged from a 3% increase in child mortality to an 11% decrease).31

    DEVTA was the largest randomized controlled trial ever conducted and included about one million children, roughly four times as many participants as the combined number of participants in all studies included in the Cochrane review, though randomization at the administrative block level limits the statistical power of the study to some extent.32 The study took place in rural Uttar Pradesh (UP), north India from May 1999 through April 2004.33 VAS was administered by workers at village anganwadi child-care centers (AWCs) and children in the treatment blocks received 200,000 IU of vitamin A every six months.34

    By delivering VAS through preexisting government infrastructure, DEVTA tested not only the efficacy of VAS, but also the practicability of implementing large-scale VAS at low cost.35

    DEVTA’s program design may have limited applicability of the study's findings to remote areas and areas without preexisting government infrastructure.36

    Why did DEVTA’s results differ from previous trials?

    DEVTA’s point estimate fell outside of the confidence interval from meta-analyses of previously executed trials.37 The researchers who conducted DEVTA “conclude that the apparent discrepancy between DEVTA and previous trials was probably due mainly to the play of chance,” but we believe this explanation is unlikely to account for the entire discrepancy.38 If the underlying effects were truly the same, the odds of achieving such disparate results due to chance alone would be very slim.39

    There are at least three potential explanations for DEVTA’s deviation from the results of previous trials (each of which we discuss in greater detail below):

    1. DEVTA’s coverage rate may be lower than reported.40
    2. DEVTA may have treated a population with less severe or less prevalent VAD than in previous trials.41
    3. The population treated by DEVTA may have had better overall health than previously studied populations, increasing the probability that mortality that may have been averted by VAS in worse-off populations would already have been averted by other mechanisms in the DEVTA population.42

    It is also possible that DEVTA’s results were significantly affected by the exclusion of areas without functioning AWCs.43 VAS might have had greater efficacy in areas without functioning AWCs, which we suspect are more remote (and therefore may have more severe or prevalent VAD and worse overall health) than the areas covered by DEVTA.44 However, it may also be more costly to implement VAS in these areas, so it is not clear what the net impact on cost-effectiveness would be of extending the program to other areas.

    While we do not arrive at a firm conclusion about why DEVTA did not find an effect on child mortality, we examine evidence with respect to each of these explanations. Before doing so, we review some features of the key mortality studies included in the Cochrane review.

    Additional detail on the key studies included in the Cochrane review

    Five of the seventeen trials included in the Cochrane review account for 80% of the weighted mean in the estimate of VAS’s effect on all-cause mortality.45 Table 1 presents the outcomes of these studies as well as the mortality rates in the control group and some of the reported baseline characteristics.

    Notes:

    • The definition of xerophthalmia may be inconsistent across studies.46
    • The age range of participants varies slightly from study to study, so mortality and disease rates are not directly comparable (since mortality and disease rates are generally different for different ages within the same population).

    Table 1: Characteristics of the five main studies used in the Cochrane review's estimate of the effect of VAS on all-cause mortality
    Study Location Age of children in study Baseline Vitamin A Deficiency Baseline Health and Control Group Mortality47 Mortality Risk Ratio
    (95% Confidence Interval)
    Ross et al 199348 The Kassena-Nankana District, a rural area of Ghana, West Africa.49 6 to 90 months50 No VAS program was in place prior to the study, the local diet was deficient in vitamin A, and VAD was locally recognized as a problem.51

    At baseline, the prevalence of xerophthalmia was 0.7%. According to retinol concentrations, at baseline 14.4% of children were severely VAD (<0.35 μmol/L) and 42.5%were moderately VAD (0.35-0.69 μmol/L).52     		0.1% mean daily prevalence of measles in placebo group.53

    15.9% mean daily prevalence of diarrhea in placebo group.54 The mortality rate in the placebo group was 29.9/1,000 child-years.55     		0.81 (0.68 - 0.98)56
    West et al 199157 The district of Sarlahi, a rural flood plain in Nepal, South Asia.58 6 to 72 months59 At baseline, the prevalence of xerophthalmia was 3.0%.60 6.2% of children in the control group and 5.1% of children in the treatment group had measles during the four months preceding the study. 1% of children had diarrhea during the week preceding the study.61

    The preschool child/infant death rate in the year preceding the study was 3.9% in the control group and 4.1% in the treatment group.62

    The mortality rate in the control group was 16.4/1,000 child-years63 and 14.2/1,000 child-years for children in this group 12 months and older.64     		0.70 (0.56-0.88)65
    Herrera et al 199266 Five rural councils in northern Sudan, North Africa.67 9 to 72 months68 VAD was present and there was local awareness of the problem.69

    At baseline, the prevalence of xerophthalmia was 2.8% in the treatment group and 2.9% in the control group.70     		0.3% of children had measles in the seven days preceding the study.71

    16.9% of children in the treatment group and 17.7% of children in the control group had diarrhea in the seven days preceding the study.72

    The mortality rate in the control group was about 5.3/1,000 child-years.73     		1.06 (0.82-1.37)74
    Daulaire et al 199275 Jumla District, a remote mountainous region of northwestern Nepal, South Asia.76 1 to 59 months77 A survey of 3,651 children under 5 years found active xerophthalmia in 13.2%.78 The mortality rate in the control group was 126/1,000 child-years at risk and 101.6/1,000 child-years among children 12 months or older.79 0.74 (0.55-0.99)80
    Sommer et al 198681 Aceh Province, an area at the northern tip of Sumatra, Indonesia, Southeast Asia.82 12 to 71 months83 At baseline, 1.9% of children in the treatment group and 2.3% of children in the control group had active xerophthalmia. 1.2% of children in the treatment group and 1.4% of children in the control group had Bitot's spots. 1.1% of children in the treatment group and 1.3% of children in the control group had night-blindness.84 ______________________________ At baseline, 22.3% of children in the treatment group and 21.7% of children in the control group had measles at any time in the past.85

    At baseline, 7.1% of children in the treatment group and 8.7% of children in the control group had diarrhea in the past seven days.86

    The mortality rate among preschoolers (12-71 months or age unknown) in the control group was about 7.4/1,000 preschoolers. The total mortality rate in the control group (aged 0-71 months) was 10.6/1,000 children.87 ____________________________    		 0.73 (0.54-0.99) (among 0-71 month olds) 88

    Trials in rural areas of Ghana, Nepal, and Indonesia have found that VAS meaningfully reduced child mortality, while a trial in Sudan found no effect. Though we have not conducted detailed research about the characteristics of the study sites, the trials generally appear to have taken place in areas with high rates of VAD and with high rates of other life-threatening illnesses. A basic comparison suggests that Herrera et al 1992, the study that found no effect for VAS in Sudan, may have taken place in a location with a lower child mortality rate than the other trials.89 There is not evidence, however, that Herrera et al 1992 had a noticeably lower rate of VAD.90

    Coverage

    Unlike many previous trials, DEVTA evaluated a large-scale program, where we would expect treatment delivery to be more challenging.91 If a high proportion of DEVTA participants were not actually receiving treatment, the smaller effect size would be explained.

    DEVTA’s published coverage rate was 86%.92 This rate appears to be only slightly below the rate achieved in previous efficacy trials, though we have not fully investigated the coverage rates of previous trials.93 However, DEVTA’s coverage data has been called into question by researchers who believe the study was not implemented as rigorously as previous trials and that it is implausible to achieve such high coverage at such low cost.94

    DEVTA implemented multiple strategies for achieving high coverage and for monitoring the coverage rate. These strategies seem reasonable overall.95 However, we note that these strategies are not fully documented, may be prone to bias in some cases, and were often not in place until the second half of the study.96 DEVTA's coverage rate would have to be substantially lower than estimated in order to explain the entire gap between the point estimate in the study and the 95% confidence interval from the meta-analysis of previously executed trials.97 This seems very unlikely.

    Rate of vitamin A deficiency

    Reliable, comparable data on VAD is scarce, and we have not fully investigated the data that exists. From what we know, DEVTA’s findings do not appear to be caused by a low rate of VAD among DEVTA participants.

    There are at least four theoretical reasons why we might expect DEVTA participants to have a lower rate of VAD than in previous studies:

    1. Prevalence of VAD might have declined over time.98
    2. DEVTA’s size may have prevented it from narrowly targeting a population with severe VAD.99
    3. DEVTA’s design may have disproportionately failed to reach children in the study area with high VAD.100
    4. Some control group members also received some doses of vitamin A.101

    However, DEVTA reports that 64.8% of children in the control group who underwent biomedical visits were VAD (retinol <0.70 μmol/L) and 13.3% were severely VAD (retinol <0.35 μmol/L).102 3.5% of children in a control group who participated in the biomedical survey had Bitot’s spots, 3.6% had Bitot’s spots or night blindness, and 6.2% had Bitot’s spots, night blindness, or conjunctivitis in the past four weeks.103 When compared with data from Table 1, this data suggests that participants in DEVTA had similar levels of VAD as participants in previous studies.104

    The manner in which VAD data was collected in DEVTA leaves open the possibility that the reported data is not representative of the overall study population. Our understanding of the data collection process is that the reported rates of deficiency are more likely to be underestimates than overestimates.105 That said, our guess at the dynamics of the sampling bias could be mistaken, and we cannot entirely rule out the possibility that DEVTA’s participants had a lower incidence of VAD than participants in other studies.

    Improved overall health conditions

    The absolute risk of death from age 1-6 years in DEVTA's control group was 2.64%, which implies a mortality rate of about 5.3 per 1,000 child-years.106 This is lower than control group mortality in 4 of the 5 trials that account for 80% of the weight in the Cochrane review (see Table 1) — considerably lower than 3 of the 5 — and the same control group mortality as the study that had the lowest rate of the 5 (and also found the smallest impact of VAS).107 Note that data for the same age group as DEVTA was available for only 2 of the 5 studies; in the other 3, the age group studied was somewhat younger than in DEVTA. We don't believe the differences in the age group studied undermine our conclusions (see the footnote for more detail).108

    Table 2: Mortality rate and mortality risk ratio in DEVTA and the five main studies used in the Cochrane review's estimate of the effect of VAS on all-cause mortality109
    Study Age group Control group mortality per 1,000 child-years Mortality risk ratio (95% CI) Deaths/Child-years in treatment vs. control
    Awasthi et al 2013 (DEVTA) 12 to 72 months 5.3 0.96 (0.89 – 1.03)110 12,467/2,464,490 vs. 13,217/2,496,620111
    Ross et al 1993 6 to 90 months 29.9 0.81 (0.68 – 0.98) 397/16,508 vs. 495/16,779112
    West et al 1991 6 to 72 months 16.4 0.70 (0.56 – 0.88) 152/13,175 vs. 210/12,795113
    Herrera et al 1992 9 to 72 months 5.3 1.06 (0.82 – 1.37) 120/21,515 vs. 112/21,224114
    Daulaire et al 1992 1 to 59 months 126 0.74 (0.55 – 0.99) 138/1,480 vs. 167/1,323115
    Sommer et al 1986 0 to 71 months 10.6 0.73 (0.54 – 0.99) 101/12,991 vs. 130/12,209116

    Lower overall child mortality rates may limit the effectiveness of VAS at preventing further mortality if, for instance, the deaths averted in previous trials are already being averted by other improvements in health.

    There is moderate evidence that deaths prevented by VAS are in part due to reduced diarrhea mortality, and possibly due to reduced measles mortality.117 If DEVTA participants were less vulnerable to dying from these diseases than participants in other studies, we would expect VAS to have a smaller effect on mortality. For example, measles vaccination campaigns and access to oral rehydration therapy may have decreased the deadliness of these diseases in some locations in the decade between most of the studies reviewed in the Cochrane report and DEVTA.

    However, we do not have specific evidence that measles or diarrhea were less common in DEVTA than previous studies. Among the nonrandom subsample of DEVTA’s control group that received biomedical visits, 1.4% had measles over a period of four weeks and 44.1% had diarrhea.118 We do not have directly comparable data from the five highly-weighted studies in the Cochrane review, but, when compared to data from Table 1, the prevalence of diarrhea and measles among DEVTA participants does not appear to be very different from the prevalence among participants of other studies.119 Globally, deaths from diarrhea and measles declined between the time of the earlier studies and the time DEVTA took place.120

    It is possible that DEVTA’s finding on the effect of VAS on mortality was caused by lower child mortality—or generally improved health—in the DEVTA study population as compared to previously studied populations.

    Interpreting the evidence in light of DEVTA

    Our overall opinion on VAS depends heavily on our interpretation of the data on VAS’s impact on child mortality. Because of the uncertain and relatively small estimated size of morbidity effects and side effects, other factors are likely to be swamped by the intervention’s effect on mortality (or lack thereof).

    We believe that the best available explanation for the discrepancy between the results of the four major studies reviewed by Cochrane that found a large impact on child mortality and the results of DEVTA is the lower baseline child mortality rate – and possibly better overall health – among DEVTA participants. This suggests that implementing VAS in areas with child mortality rates much lower than 10.6/1,000 child-years – the lowest rate among the four major successful studies – may be considerably less effective (and cost-effective) than implementations in settings with higher baseline mortality.

    The other plausible explanations for heterogeneity between DEVTA and the trials underlying the Cochrane review have different implications for the likelihood that a charity implementing VAS will be a success:

    • If DEVTA’s failure to find an effect was due to lack of coverage, charities would need to have a strong argument for why their own coverage rates would be similar to the high rate found in early efficacy trials for us to believe that they could replicate the earlier trials' effects on mortality.
    • If VAS is only successful in areas with extremely high VAD, extremely high prevalence of diarrhea and measles, or low access to life-saving treatment for these diseases, only charities that could show that they have narrowly targeted such a population would be likely to convince us that their intervention would have higher effects on child mortality than DEVTA.
    • If DEVTA’s smaller impact were due to chance or other contextual factors that we have not thought of, then Cochrane’s meta-analysis (which found that VAS reduces child mortality by 12% (95% CI: 6% to 16%) once DEVTA is included) might be the best guide to future impact.121

    Cost-effectiveness

    We have investigated Helen Keller International's (HKI) VAS program, and found that its cost-effectiveness is in the range of the cost-effectiveness of our priority programs. See our review of HKI and our most recent cost-effectiveness model for details.

    Room for more funding

    In our review of Helen Keller International (HKI), we discuss HKI's room for more funding and global room for more funding for VAS programs.

    Feedback from scholars

    We have discussed this page with Sir Richard Peto and Dr. Simon Read, co-authors of the DEVTA study, as well as with Dr. Evan Mayo-Wilson, co-author of the Cochrane review. See their comments here:

    Questions for further investigation

    We discuss what we might like to learn more about from the existing literature on VAS:

    • Further researching possible causes of heterogeneity between DEVTA and previous studies. This would involve collecting data from various sources on diarrhea prevalence, measles prevalence, measles vaccination rates, access to oral rehydration therapy, and baseline child mortality in the locations and time periods where DEVTA and each of the previous studies took place. We would also try to learn whether earlier trials took place in areas that were more remote or had less preexisting infrastructure than the area where DEVTA took place.
    • Reviewing macro evidence on the success of VAS. This could involve searching for meta-analyses of observational studies on VAS and looking at estimates of whether large-scale VAS programs were followed by changes in mortality rates.
    • Conducting further research on the AWCs in DEVTA. In particular we would like to know how locations for AWCs are chosen and whether villages with functioning AWCs significantly differ from other villages.
    • Attempting to access unpublished records or proceedings from the workshop at Oxford where criticisms of DEVTA were aired.122
    • Investigating whether participants in DEVTA control blocks received any doses of VAS in addition to the known dose distributed by Pulse Polio. This could include VAS doses external to the study or any instances where members of the control group accidentally received treatment through DEVTA.
    • Investigating whether control groups in other studies were contaminated by doses of Vitamin A.
    • Researching alternative methods of combating VAD including fortification.
    • Further investigating VAS's impact on vision problems. In particular, we would consider looking into the severity of Bitot's spots and night blindness and the frequency with which VAD leads to blindness.
    • Investigating dosing schedules for VAS and the rationale for the one recommended by the WHO. We'd also to learn more about what effect the dose distributed by Pulse Polio might have had on the control group.
    • More thorough adjudication of the competing claims in correspondence to the Lancet regarding DEVTA.

    Our process

    2013-2015

    We relied particularly heavily on Imdad et al 2010, the Cochrane review of the effects of universal vitamin A supplementation on preschool-aged children, because it was comprehensive and clearly presented the results of meta-analysis of high-quality trials. We also relied heavily on the primary publications reporting the results of the five highest-weighted studies from that review and on Awasthi et al 2013, Awasthi et al 2013 DEWORMING, and Awasthi et al 2013 APPENDIX, the published results of DEVTA.

    We searched for RCTs or quasi-RCTs published after October 2010 (the last month covered by the Cochrane review) by searching PubMed and Google Scholar for studies that cited the Cochrane review, the DEVTA publications, or any of the five heavily weighted studies from the Cochrane review. We also searched PubMed and Google Scholar for the term “Vitamin A Supplementation” and looked for relevant articles. Lastly, we looked at articles cited by the Cochrane review, the DEVTA publications, and RCTs that we found.

    We spoke with Dr. Evan Mayo-Wilson, co-author of Imdad et al 2010, the Cochrane review of the effects of universal vitamin A supplementation, and with Sir Richard Peto and Dr. Simon Read, co-authors of the DEVTA study.

    2017-2018

    We reviewed a more recently published Cochrane review of the effects of universal vitamin A supplementation on preschool-aged children, Imdad et al. 2017, which combines DEVTA and another smaller recent trial (Fisker et al. 2014) in a meta-analysis with previous trials. Its fixed-effect meta-analysis finds that VAS causes a 12% reduction in child mortality (95% confidence interval 7% to 17% reduction) and its random-effects meta-analysis finds that VAS causes a 24% reduction in child mortality (95% confidence interval 17% to 31% reduction).123 (See the following footnote for a description of the differences between fixed-effect and random-effects meta-analyses.)124 Even though the overall effect found in the updated meta-analysis remains statistically significant, it is unlikely that differences between the results of DEVTA and earlier trials were due to random chance alone.125 Overall, our review of Imdad et al. 2017 did not lead to substantial changes in our views on the evidence base for vitamin A supplementation described above.

    We also reviewed research on a few topics that we believed could substantially affect our cost-effectiveness analysis:

    • Biological plausibility of interactions between vitamin A supplementation and vaccinations: Benn et al. 2009, a re-analysis of data from a vitamin A supplementation RCT in Ghana and Fisker et al. 2014, a vitamin A supplementation RCT from Guinea-Bissau, test the hypotheses that there are interactions between vitamin A supplementation and vaccinations, and that these interactions may increase mortality in some groups.126 We reviewed research on the biological plausibility of interactions between vitamin A supplementation and vaccinations and summarized our findings in this document. We did not find reasons to believe that harmful impacts are highly plausible.
    • Possible developmental effects: Reducing exposure to malaria during childhood may have an effect on long-term productivity and earnings. We include these possible benefits in our cost-effectiveness analyses for seasonal malaria chemoprevention and mass distribution of long-lasting-insecticide-treated nets. In 2018, we reviewed research on whether vitamin A supplementation may also lead to long-term effects on productivity and earnings by reducing incidence of various infectious diseases, and decided to also include a developmental effects parameter for vitamin A supplementation in our cost-effectiveness analyses.
    • Prevention of vision loss and blindness: We reviewed research on the impact of vitamin A supplementation on preventing vision loss (e.g., night blindness) and total blindness. We have not incorporated these benefits into our vitamin A supplementation cost-effectiveness analysis because we expect these benefits to be relatively small and we generally focus our analyses on the core benefits of interventions—see the "Inclusion/exclusion" sheet in our cost-effectiveness analyses for more details.

    Sources

    Document Source
    Awasthi et al 2007 Source (archive)
    Awasthi et al 2013 Source (archive)
    Awasthi et al 2013 APPENDIX Source (archive)
    Awasthi et al 2013 DEWORMING Source (archive)
    Benn et al. 2009 Source (archive)
    Benn, Fisker, and Aaby 2013 Source (archive)
    CDC 2012 Source (archive)
    Chowdhury et al 2002 Source (archive)
    Cochrane Handbook Source
    Cochrane Handbook section 9.5.4: Incorporating heterogeneity into random-effects models Source (archive)
    Daulaire et al 1992 Source (archive)
    Fisker et al 2012 Source (archive)
    GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013 Source
    GiveWell's non-verbatim summary of a conversation with Richard Peto and Simon Read, April 10, 2014 Source
    Fisker et al. 2014 Source (archive)
    Habicht and Victora 2013 Source (archive)
    Herrera et al 1992 Source (archive)
    Imdad et al 2010 Source (archive)
    Imdad et al. 2017 Source (archive)
    Kapil 2004 Source (archive)
    Latham 2010 Source (archive)
    Peto 2013 LECTURE Source (archive)
    Peto et al 2013 Source (archive)
    Ross et al 1993 Source (archive)
    Sommer et al 1986 Source (archive)
    Sommer, West, and Martorell 2013 Source (archive)
    UNICEF 2007 Source (archive)
    UNICEF 2009 Source (archive)
    West et al 1991 Source (archive)
    WHO 1995 Source (archive)
    WHO 1997 Source (archive)
    WHO 2009 Source (archive)
    Wirth et al. 2017 Source (archive)
    Yakymenko et al 2011 Source (archive)
    • 1
      • “About 19.1 million pregnant women and 190 million children under 5 are vitamin A deficient (i.e. serum retinol < 0.70 µmol/l), representing about 33% of children under 5 in populations at risk of VAD (WHO 2009). Based on biochemical VAD in young children, 122 countries have a moderate to severe public health problem (WHO 2009).” Imdad et al 2010, Pg 9.
      • Xerophthalmia (dry eyes), is “the leading cause of preventable childhood blindness” WHO 2009, Pg 1.
      • "The term xerophthalmia encompasses the clinical spectrum of ocular manifestations of VAD, from milder stages of night blindness and Bitot’s spots, to potentially blinding stages of corneal xerosis, ulceration and necrosis (keratomalacia). . . . The stages of xerophthalmia are regarded both as disorders and clinical indicators of VAD, and thus can be used to estimate an important aspect of morbidity and blinding disability as well as the prevalence of deficiency. As corneal disease is rare, the most commonly assessed stages are night blindness, obtainable by history, and Bitot’s spots, observable by handlight examination of the conjunctival surface. Standard procedures exist for assessing xerophthalmia (17). Although night blindness and Bitot’s spots are considered mild stages of eye disease, both represent moderate-to-severe systemic VAD, as evidenced by low serum retinol concentrations (19), and increased severity of infectious morbidity (i.e. diarrhoea and respiratory infections) and mortality in children (5) and pregnant women (6, 20).” WHO 2009, Pgs 2-3.
      • “Vitamin A deficiency (VAD) impairs body functions and may cause death. Adverse health consequences may also include xerophthalmia (dry eyes), susceptibility to infection, stunting and anaemia (Sommer 1996; Rice 2004).” Imdad et al 2010, Pg 9.

    • 2

      “WHO regional estimates indicate that the highest proportion of preschool-age children affected by night blindness, 2.0%, is in Africa, a value that is four times higher than estimated in South-East Asia (0.5%). This also means that Africa has the greatest number of preschool-age children affected with night blindness (2.55 million), and corresponds to almost half of the children affected globally (Table 10). A comparable and high proportion of pregnant women affected by night blindness are in Africa (9.8%) and South-East Asia (9.9%), each of which is estimated to have over 3 million pregnant women affected, or one third of the pregnant women affected globally. The estimates show that the Africa and South-East Asia regions also contain the highest proportions of preschool-age children with biochemical VAD, as indicated by a serum retinol concentration <0.70 µmol/l, with South-East Asia having the greatest number of children and pregnant women affected” WHO 2009, Pgs 10-11.

    • 3

      “Low vitamin A intake during nutritionally demanding periods in life, such as infancy, childhood, pregnancy and lactation, greatly raises the risk of health consequences, or vitamin A deficiency disorders (VADD).” WHO 2009, Pg 1.

    • 4

      “Chronic VAD may develop when animal sources and fortified foods are limited, as in diets that rely heavily on vegetables and fruits (Ramakrishnan 2002).” Imdad et al 2010, Pg 9.

    • 5

      “the most widely practiced approach to controlling VAD in most high risk countries is the periodic delivery of high-potency supplements, containing 200 000 IU of vitamin A, to preschool-age children (<5 years), with half this dose given to infants 6–11 months of age (25).” WHO 2009, Pgs 2-3.

    • 6
      • “Vitamin A supplements are used in two principal situations: to treat those with acute xerophthalmia and to prevent vitamin A deficiency where the periodic administration of supplements is determined to be the most feasible and cost-effective means of improving Vitamin A status.” WHO 1997, Pg 1. "Universal vitamin A distribution involves the periodic administration of supplemental doses to all preschool-age children, with priority given to age groups (usually 6 months–3 years)" WHO 1997, Pg 3. In addition to pre-school children, infants and pregnant mothers are also sometimes recipients of universal VAS. WHO 1997, Pg 4.
      • In particular, the WHO recommends that infants 6-12 months of age receive 100,000 International Units (IU) of Vitamin A every 4-6 months and recommends that children older than 12 months of age receive 200,000 IU every 4-6 months. WHO 1997, Pg 4.
      • “Periodic supplementation takes advantage of the fact that large quantities of vitamin A can be stored in the liver for future use.” WHO 1995, Pg 43.

    • 7

      The trials reviewed in this report generally implement the WHO supplementation policy (described in the previous footnote); however, there is minor variation in dosage, treatment frequency, and age range of children treated.

    • 8

      See our review of Helen Keller International's vitamin A supplementation programs for our most recent estimate of its cost per vitamin A supplement delivered.

    • 9

      “Since its initial linkage with immunization in the 1990s, supplementation has almost always been delivered in combination with other health services, such as vaccines, anti-helminthics and insecticide-treated mosquito nets.” UNICEF 2007, Pg 16.

    • 10

      “This report focuses on vitamin A supplementation in 103 countries (considered ‘priority countries for vitamin A supplementation’) that meet at least one of the following criteria:

      • High under-five mortality rate (U5MR): Supplementation with vitamin A is recommended in all countries where the U5MR exceeds 70 deaths per 1,000 live births, an internationally accepted proxy indicating a high risk of deficiency among children under five. Sixty-one countries in the developing world were included as priority countries for vitamin A supplementation using 2004 U5MR estimates.
      • Vitamin A deficiency as a public health problem: Deficiency may also exist in countries with relatively low under-five mortality. Thirty-five additional priority countries were selected based on an elevated prevalence of vitamin A deficiency, which was determined using data from national-level assessments registered in the Micronutrient Deficiency Information System of the World Health Organization (WHO).
      • History of programming: The Democratic People’s Republic of Korea, Indonesia, Kyrgyzstan, the Occupied Palestinian Territory, Thailand, Uzbekistan, and Viet Nam did not meet the above criteria. Their governments have, however, recognized vitamin A deficiency as a public health problem and are committed to programming for vitamin A supplementation. Significant reductions in under-five mortality and vitamin A deficiency have been achieved by some of these governments, partly as a result of ongoing supplementation efforts – some dating back more than 25 years.” UNICEF 2007, Pg 5.

    • 11

      For example, Imdad et al 2010 reviewed forty-three such studies. "This review, including 43 randomised trials representing 215,633 children, shows that giving vitamin A capsules to children aged 6 months to 5 years can reduce death and some diseases." Imdad et al 2010, Pg 2.

    • 12
      • “Vitamin A was associated with a 24% reduction in all-cause mortality (RR = 0.76 (95% CI 0.69 to 0.83)), though there was moderate heterogeneity”. Imdad et al 2010, Pg 18. “Seventeen trials . . . contributed 194,795 children (90% of the children included in the review)” to Cochrane’s meta-analysis of VAS’s effect on all-cause mortality. Imdad et al 2010, Pg 18.
      • Out of the seventeen studies included in Cochrane's meta-analysis of all-cause mortality, five accounted for eighty percent of the weight. “[T]he overall effect [on all-cause mortality] was strongly influenced by five studies that accounted for over 80% of the weighted mean.” Imdad et al 2010, Pg 19. Those five studies took place during the following years:
        • 1989-1991. "The Survival Study trial was carried out between September, 1989, and December, 1991, and the Health Study trial between June, 1990, and August, 1991." Ross et al 1993, Pg 8.
        • 1989-1990. "A randomised, double-masked, placebo-controlled vitamin A supplementation trial was carried out from September, 1989, to December, 1990, in the rural, plains (Terai) district of Sarlahi." West et al 1991, Pg 67.
        • 1988-1990. "The study was conducted between June, 1988, and December, 1990 . . . " Herrera et al 1992, Pg 267.
        • 1989. "During three weeks in May-June 1989 a single high dose capsule of retinol palmitate (with vitamin E) was given to all children in the selected districts." Daulaire et al 1992, Pg 207.
        • 1982-1984. "Teams first visited villages between September, 1982, and August, 1983, and follow-up visits were made by the same team in the same sequence 9-13 months later." Sommer et al 1986, Pg 1169.

    • 13

      Imdad et al 2010, Pg 15. DEVTA was “a factorial trial of albendazole, vitamin A (retinol), both, or neither,” but this write up focuses on DEVTA’s test of VAS. Awasthi et al 2013 DEWORMING, Pg 1478.

    • 14

      “Deaths per child-care centre at ages 1.0–6.0 years during the 5-year study (the primary trial endpoint) were 3.01 retinol versus 3.15 control (absolute reduction 0.14 [SE 0.11], mortality rate ratio [RR] 0.96, 95% CI 0.89–1.03, p=0.22), suggesting absolute risks of death between ages 1.0 and 6.0 years of approximately 2.5% retinol versus 2.6% control.” Awasthi et al 2013, Pg 1473.

    • 15

      “Deaths per child-care centre at ages 1.0–6.0 years during the 5-year study (the primary trial endpoint) were 3.01 retinol versus 3.15 control (absolute reduction 0.14 [SE 0.11], mortality rate ratio [RR] 0.96, 95% CI 0.89–1.03, p=0.22), suggesting absolute risks of death between ages 1.0 and 6.0 years of approximately 2.5% retinol versus 2.6% control.” Awasthi et al 2013, Pg 1473.

    • 16
      • “We included randomized controlled trials (RCTs) and cluster RCTs evaluating the effect of synthetic VAS in children aged 6 months to 5 years. . . . Post hoc, we included two studies in which participants were assigned using a quasi-random method (Herrera et al 1992; Stansfield 1993). In both cases, the authors and the editorial team agreed that the methods of assignment (i) had the desirable characteristics of randomization and (ii) were of no greater risk of bias than other included studies.” Imdad et al 2010, Pg 10.
      • “Review content assessed as up-to-date: 26 October 2010.” Imdad et al 2010, Pg 1.

    • 17
      • “Vitamin A was associated with a 24% reduction in all-cause mortality (RR = 0.76 (95% CI 0.69 to 0.83)), though there was moderate heterogeneity”. Imdad et al 2010, Pg 18.
      • One of the seventeen included trials had zero weight in the analysis. “One [additional randomized trial] reported no events [i.e. deaths] (Lin 2008)” and therefore had zero weight in the analysis.Imdad et al 2010, Pg 18.
      • The authors included two quasi-randomized trials in the review. "Post hoc, we included two studies in which participants were assigned using a quasi-random method (Herrera 1992; Stansfield 1993)." Imdad et al 2010, Pg 10. Only one of these, Herrera et al 1992 was included in the analysis of all-cause mortality. Imdad et al 2010, Pg 18. In this trial, “[a]ssignment to treatment group was achieved by the two interviewers visiting alternate households throughout the village.” Herrera et al 1992, Pg 267. We agree with the Cochrane authors’ conclusion that the study “(i) had the desirable characteristics of randomization and (ii) [was] of no greater risk of bias than other included studies” because allocating alternate households to the treatment and control group “was not likely to result in systematically different groups.” Imdad et al 2010, Pgs 10, 17. Baseline statistics suggest that allocation was as good as random. “There were no important differences between vitamin A and placebo groups in rate of xerophthalmia (2.8% vs 2.9%), vitamin A intake, age distribution or nutritional status.” Herrera et al 1992, Pg 269. Furthermore, Herrera et al 1992 “reported no effect (RR = 1.06 (95% CI 0.92 to 1.37)), indicating that these trials were not likely to influence [Cochrane’s] results in a positive direction.” Imdad et al 2010, Pg 19.

    • 18
      • Inadequate blinding: Ten out of the sixteen trials used in the all-cause mortality outcome described efforts to blind participants and providers, but one study (Daulaire et al 1992) was unblinded and five studies did not report enough detail to determine whether they were blinded. See Imdad et al 2010, Pg 17 for a list of which studies reported efforts to blind participants and providers and Imdad et al 2010, Pg 18 for a list of the studies included in the all-cause mortality meta-analysis.
        Despite this, “the primary outcome, mortality, is very unlikely to have been influenced by lack of blinding,” because mortality is concrete, well-measured and unlikely to be affected by the placebo effect, the Hawthorne effect, or any desire to please an investigator. Imdad et al 2010, Pg 17. More generally, bias due to inadequate blinding is “if anything, likely to underestimate effects; for example, a teacher would be more likely to give extra food to a child receiving placebo rather than the reverse.” Imdad et al 2010, Pg 17.
      • Missing data: “Missing data are much more likely to influence secondary analyses than the primary outcome. Results for all-cause mortality are known for 91% of randomised participants. Of the 17 studies (40%) that reported [all-cause mortality], 7 were Unclear, but 4 of these had minimal attrition (Vijayaraghavan 1990; Ross 1993 HEALTH; Ross 1993 SURVIVAL; Venkatarao 1996) and the others failed to report reasons for dropout. In 2 studies, missing data were not adequately handled (Pant 1996; Chowdhury 2002), but together these studies contributed only 5% to the pooled estimate.” Imdad et al 2010, Pg 17.
      • Selective reporting: “For the primary outcome, there is no meaningful risk of bias; the outcome was reported for large trials. Data were missing in small studies of short duration, which likely observed few deaths.” Imdad et al 2010, Pg 18.
      • Small study bias: “There was some evidence of visual asymmetry on the funnel plot we produced, but the overall effect was strongly influenced by five studies that accounted for over 80% of the weighted mean; even if the result was influenced by small study bias, the magnitude of the effect was small. To impact the results, missing studies would need to be very large and show no difference or harmful effects, as demonstrated in the third sensitivity analysis.” Imdad et al 2010, Pg 19.
      • Incorrect estimations of intracluster correlation coefficients (ICCs): “The ICC is an estimate of the relative variability within and between clusters (Donner 1980). It describes the ‘similarity’ of individuals within the same cluster.” Cochrane Handbook, chapter 16.3.4. If an ICC is known, it can be used by reviewers to correct the standard errors in cluster-randomized trials that inappropriately calculated standard errors as though they were individually randomized, a common mistake which could cause certain studies to be erroneously overweighted. For three studies included in Cochrane’s all-cause mortality analysis, the ICC was unknown and had to be estimated. Imdad et al 2010, Pg 19. The Cochrane authors determined that their analysis was not sensitive to this estimate by “repeat[ing] the primary analysis using a much larger and much smaller ICC estimate.” Imdad et al 2010, Pg 19. They found that “over-weighting these three studies in the analysis would not impact the conclusions of this review; further inflating their SEs [standard errors] would increase the size of the effect estimate.” Imdad et al 2010, Pg 19. In other words, the authors’ adjustment for this effect was conservative.

    • 19

      Awasthi et al 2007

    • 20

      “In our analysis of 18 trials, DEVTA 2007 accounted for 65.2% of the combined effect, which remained significant (RR= 0.88 (95% CI 0.84 to 0.94)) with substantial and significant heterogeneity (Chi² = 44.31, df = 16 (P = 0.0002); I² = 64%). (We assumed the study authors adjusted for clustering. As the results were not significant, inflating the SE would not change our interpretation of the trial’s results, although it would decrease its weight in the analysis.)” Imdad et al 2010, Pg 19.

    • 21

      “As we were unable to assess the trial, we cannot explain this substantially different result.” Imdad et al 2010, Pg 19.

    • 22
      • The estimates were imprecise in that confidence intervals included a wide range of possible effects:
        • “Seven trials . . . reported a combined 28% reduction in diarrhea mortality (RR = 0.72 (95% CI 0.57 to 0.91)) with no important heterogeneity (Chi² = 6.12, df = 6 (P = 0.41); I² = 2%) . . .” Imdad et al 2010, Pg 20.
        • “Five trials . . . reported a lower risk of measles mortality, but the effect was not statistically significant (RR = 0.80 (95% CI 0.51 to 1.24)). There was no important heterogeneity (Chi² = 0.40, df = 4 (P = 0.98); I² = 0%) . . .” Imdad et al 2010, Pg 20.
        • “Seven trials . . . reported a lower risk of LRTI mortality, but the effect was not statistically significant (RR= 0.78 (95% CI 0.54 to 1.14)). There was no important heterogeneity (Chi² = 7, df = 6 (P = 0.32); I² = 14%).” Imdad et al 2010, Pg 21.
        • Imdad et al 2010 also reported results on meningitis mortality, but those results were based on three studies and had an extremely high confidence interval. “Three trials . . . reported a lower risk of meningitis mortality, but the effect was not statistically significant (RR= 0.57 (95% CI 0.17 to 1.88)). There was no important heterogeneity (Chi² = 0.75, df = 2 (P = 0.69); I² = 0%).” Imdad et al 2010, Pg 21.
      • The effects are reported in relatively few studies with few participants. The results for diarrhea-related mortality are based on seven studies with 90,951 participants. Imdad et al 2010, Pg 3. The results for measles-related mortality are based on five studies with 88,261 participants. Imdad et al 2010, Pg 4. The results for lower respiratory tract infection (LRTI) related mortality are based on seven studies with 90,951 participants. Imdad et al 2010, Pg 4. The effective sample size of each of these studies is smaller, though, because many of them are randomized at the cluster level.
      • “Missing data are much more likely to influence secondary analyses than the primary outcome.” Imdad et al 2010, Pg 17. These studies are particularly likely to be affected by selective reporting bias because studies may only report data on cause-specific mortality if they find positive results. “Most of the trials in the review included multiple outcome measures, and positive results are more likely to be included in reports than negative results. Only 5 (12%) trials appeared to be free of selective outcome reporting . . . Twenty four (56%) were Unclear, while 14 (33%) were at High risk of bias . . . For many of the secondary analyses, which included only a few trials representing a small proportion of the overall sample, adding unreported data might influence the observed effects.” Imdad et al 2010, Pgs 17-18.
      • One opponent of VAS has strongly criticized the literature’s methods for establishing cause of death, arguing that family members’ diagnoses are often inaccurate. “The ‘causes’ of death in such studies are established by ‘verbal autopsies’ (a wonderful oxymoron) from family members, often many weeks after each child’s death. It appeared entirely feasible, based on many years experience in the field in Africa, that many deaths recorded as due to respiratory infections, diarrhoea or fever (recorded as malaria) might in fact be measles deaths. Measles can cause all of these symptoms. Malaria can only be diagnosed by identifying plasmodia in blood.” Latham 2010.

    • 23
      • “Seven trials . . . reported a combined 28% reduction in diarrhea mortality (RR = 0.72 (95% CI 0.57 to 0.91)) with no important heterogeneity (Chi² = 6.12, df = 6 (P = 0.41); I² = 2%) . . .” Imdad et al 2010, Pg 20.
      • “Five trials . . . reported a lower risk of measles mortality, but the effect was not statistically significant (RR = 0.80 (95% CI 0.51 to 1.24)). There was no important heterogeneity (Chi² = 0.40, df = 4 (P = 0.98); I² = 0%) . . .” Imdad et al 2010, Pg 20.
      • “Seven trials . . . reported a lower risk of LRTI mortality, but the effect was not statistically significant (RR= 0.78 (95% CI 0.54 to 1.14)). There was no important heterogeneity (Chi² = 7, df = 6 (P = 0.32); I² = 14%).” Imdad et al 2010, Pg 21.
      • Imdad et al 2010 also reported results on meningitis mortality, but those results were based on three studies and had an extremely high confidence interval. “Three trials . . . reported a lower risk of meningitis mortality, but the effect was not statistically significant (RR= 0.57 (95% CI 0.17 to 1.88)). There was no important heterogeneity (Chi² = 0.75, df = 2 (P = 0.69); I² = 0%).” Imdad et al 2010, Pg 21.

    • 24
      • The Cochrane review estimated that VAS reduces diarrhea incidence by 15% (95% CI 0.82 to 0.87). “Thirteen trials . . . reported an 18% [sic] decrease in diarrhoea incidence (RR = 0.85 (95% CI 0.82 to 0.87)), though statistical heterogeneity was substantial and highly significant (Chi² = 218.62, df = 11 (P 0.00001); I² = 95%), Figure 6.” Imdad et al 2010, Pg 21. However, there are significant questions about the quality of this evidence. The trials were “highly heterogenous and two studies account for the majority of the effect (Cheng 1993; Chowdhury 2002).” Imdad et al 2010, Pg 21. “Four studies contributing just over 25% of the weight of the estimated effect were at risk of selection or attrition bias.” Imdad et al 2010, Pg 7. Chowdhury et al 2002, a trial that accounted for 27.6% of the weight of the meta-analysis and found a strongly positive effect, did not adequately describe the procedure for blinding providers or participants. Imdad et al 2010, Pgs 21, 46. Inadequate blinding could substantially bias this outcome because participants’ likelihood of reporting diarrhea may be affected by their knowledge of whether they are in the treatment or control group.
      • “Six trials . . . reported a 50% decrease in measles incidence (RR = 0.50 (95% CI 0.37 to 0.67)) with no important heterogeneity (Chi² = 0.55, df = 5 (P = 0.99); I² = 0%).” Imdad et al 2010, Pg 22.

    • 25

      “Nine trials . . . reported no combined effect on LRTI incidence (RR = 1.14 (0.95 to 1.37)) with no important heterogeneity (Chi² = 7.66, df = 6 (P = 0.26); I² = 22%).” Imdad et al 2010, Pg 22.

    • 26
      • Imdad et al 2010, Pg 24. “One trial . . . reported no effect on Bitot’s spots incidence (RR = 0.93 (95% CI 0.76 to 1.14)). Four trials . . . reported a 53% reduction in Bitot’s spots prevalence (RR = 0.45 (95% CI 0.33 to 0.61)) with substantial and significant heterogeneity (Chi² = 8.25, df = 3 (P = 0.04); I² = 64%).” Imdad et al 2010, Pg 22. However, “there was a risk of attrition bias in Pant 1996, which was assigned 47% weight” in the analysis of Bitot’s spot. Imdad et al 2010, Pg 8.
      • “One trial (Herrera 1992) reported a 47% reduction in night blindness incidence (RR = 0.53 (95% CI 0.28 to 0.99)). Two trials . . . reported a 68% reduction night blindness prevalence (RR = 0.32 (95% CI 0.21 to 0.50)) with no heterogeneity (Chi² = 0.19, df = 1 (P = 0.66); I² = 0%).” Imdad et al 2010, Pgs 22-23.
      • “Three trials . . . reported no combined effect on xerophthalmia incidence (RR = 0.85 (95% CI 0.70 to 1.03)), though statistical heterogeneity was substantial and significant (Chi² = 2.69, df = 1 (P = 0.10); I² = 63%). Two trials . . . reported a 69% reduction in xerophthalmia prevalence (RR = 0.31 (95% CI 0.22 to 0.45)) with no statistical heterogeneity (Chi² = 0.22, df = 1 (P = 0.64); I² = 0%).” Imdad et al 2010, Pg 23.

    • 27

      “Most of the trials in the review included multiple outcome measures, and positive results are more likely to be included in reports than negative results. Only 5 (12%) trials appeared to be free of selective outcome reporting . . . Twenty four (56%) were Unclear, while 14 (33%) were at High risk of bias . . . For many of the secondary analyses, which included only a few trials representing a small proportion of the overall sample, adding unreported data might influence the observed effects.” Imdad et al 2010, Pgs 17-18.

    • 28

      “Three trials . . . reported a significant increase in risk of vomiting (RR= 2.75 (95%CI 1.81 to 4.19)) with statistical heterogeneity that was not important (Chi² = 2.53, df = 2 (P = 0.28); I² = 21%). Immediately following the intervention, the rate of vomiting increased from 2% to 6%.” Imdad et al 2010, Pg 23.

    • 29

      “Three trials . . . reported fontanelle side effects, but only one could be analysed because the others reported insufficient data, which reported no effect (RR = 5.00 (95% CI 0.24 to 103.72)). Most studies included children over 1 year old and would not have assessed this side effect.” Imdad et al 2010, Pg 23.

    • 30

      Other than DEVTA (discussed below), we know of only one randomized trial of VAS whose results were published after the Cochrane review was completed. Yakymenko et al 2011 was a yearlong trial testing whether “girls benefit more from a lower dose of VAS than the one currently recommended by WHO, the effect being strongest if diphtheria-tetanus-pertussis vaccine was the most recent vaccination.” Yakymenko et al 2011, Pg 1. This study found “[t]here was no significant difference in mortality at 6 months and 12 months of follow up between the low dose VAS group and the recommended dose VAS group; the MRR being 1.23 (0.60-2.54) after 6 months and 1.17 (0.73-1.87) after 12 months.” Yakymenko et al 2011, Pg 4. The study’s “sample size [of 8,626 children] does not permit firm conclusions since mortality was lower than expected.” Yakymenko et al 2011, Pg 1. The study probably suffered from selection bias. “As expected the non-participants differed significantly from participants with regard to many baseline characteristics, indicating strong selection bias. We have attempted to control for all known confounders, but the comparisons between participants and non-participants should nonetheless be interpreted with great caution. Selection bias, however, is unlikely to explain why only boys benefitted significantly from participation in the study.” Yakymenko et al 2011, Pg 8. Overall, this study does not affect our interpretation of the evidence on VAS.

    • 31

      “Deaths per child-care centre at ages 1.0–6.0 years during the 5-year study (the primary trial endpoint) were 3.01 retinol versus 3.15 control (absolute reduction 0.14 [SE 0.11], mortality rate ratio [RR] 0.96, 95% CI 0.89–1.03, p=0.22), suggesting absolute risks of death between ages 1.0 and 6.0 years of approximately 2.5% retinol versus 2.6% control. Although this finding suggests that overall child mortality was 4% lower in vitamin A than in control blocks, this 4% reduction includes the possibility of no benefit and the possibility of appreciable benefit (95% confidence limit for reduction 11%).” Awasthi et al 2013, Pg 1473.

    • 32
      • “DEVTA is the largest randomised controlled trial ever conducted, including approximately one million children. That is, the trial included four times the combined participants of all included studies in this review.” Imdad et al 2010, Pg 15. “As the study began it had a million participants, and during it another million 1-year-olds joined it and a million 6-year-olds left (figure 2), so at one time or another it included about 2 million children.” Awasthi et al 2013, Pg 1470. In a meta-analysis including DEVTA and all seventeen trials from the Cochrane all-cause mortality meta-analysis, DEVTA “accounted for 65.2% of the combined effect.” Imdad et al 2010, Pg 19.
      • “Within the participating blocks, the study population was all young children living in the defined catchment areas of 8338 village anganwadi child-care centres (AWCs; anganwadi means courtyard) that the ICDS regarded as functional (except in one block 17 )” Awasthi et al 2013, Pg 1471.
      • “Of 118 administrative blocks in the seven DEVTA districts in north India, 46 with many AWCs not functioning in 1998 were not included.” Awasthi et al 2013, Pg 1471.
      • “The 72 participating blocks were those within a few hours drive that the ICDS director in Lucknow judged in 1998 to have a reasonably well functioning ICDS system with willing district and block ICDS directors and paid workers in most village anganwadi child-care centres.” Awasthi et al 2013 DEWORMING, Pg 1479.

    • 33

      Awasthi et al 2013, Pgs 1470-1471.

    • 34
      • “[I]t is intended that all children of age 3-6 years should be cared for each weekday for a few hours by a local anganwady worker. The coverage achieved by this system is, however, very incomplete; in practice, only about one-third of all the villages in UP have an anganwady centre, and even in villages where there is an anganwady centre most children of age 3-6 years attend it rarely or never, and few attend it regularly. The younger children (less than 3 years old), although not supposed to be seen daily, are supposed to be enumerated and monitored by the system, partly to ensure that they are developing appropriately.” Awasthi et al 2013 APPENDIX, Pg 3. “A typical rural AWC serves a village population of 1000 (10–15% aged 1–6.0 years); large villages can have more AWCs. The AWC workers, usually local women (plus assistants), give pre-school education, give nutritional supplements to malnourished children, and record births and pre-school deaths. AWCs register about two-thirds of under-5s (and one-third of 5-year-olds) for possible nutritional supplementation, although levels of AWC attendance and supplementary nutrition in Uttar Pradesh are often low. AWC workers can help health workers to organise immunisation and pre-school health services.” Awasthi et al 2013 DEWORMING, Pg 1479.
      • Some non-trial VAS occurred in some villages, so “the comparison should in principle be described as being of routine vitamin A versus occasional vitamin A supplementation.” Awasthi et al 2013 APPENDIX, Pgs 3. About halfway through the trial, many of the children in the control group received one dose of vitamin A from an organization called Pulse Polio. Peto 2013 LECTURE, 18:40-18:57. “No placebos were used” Awasthi et al 2013, Pg 1470.

    • 35

      “UP is an ideal setting for this study, because government programmes have already been instituted that should be able to supply food supplements and micronutrients to preschool children. The study will use this infrastructure (the anganwady system) to deliver deworming medicine and vitamin A to children in those blocks randomly allocated to one and/or other active treatment. . . . The cost-effectiveness of mass treatments . . . depends not only on the costs and effectiveness of the drugs themselves, but also on the practicability of delivering them to children reliably and sustainably at low additional cost. Both of these important questions will be addressed.” Awasthi et al 2013 APPENDIX, Pg 2.

    • 36

      All villages without functioning AWCs and administrative blocks with many villages with nonfunctioning AWCs were excluded from the study (Figure 1, Awasthi et al 2013, Pg. 1470). We would guess these areas tend to be more remote than the rest of UP. Children in these areas may have higher mortality rates, greater incidence of VAD, or less access to healthcare services. Each of these factors suggests that the clinical effect of VAS could be higher in these locations. On the other hand, the cost of implementing VAS would likely be higher in these areas and the coverage rate would likely be lower.

    • 37

      The Cochrane review found that: “Vitamin A was associated with a 24% reduction in all-cause mortality (RR = 0.76 (95% CI 0.69 to 0.83).” Imdad et al 2010, Pg 18. The 95% confidence interval therefore ranged from a 17% reduction to a 31% reduction. DEVTA’s point estimate was just a 4% reduction in mortality. Awasthi et al 2013, Pg 1473.

    • 38

      Peto et al 2013, Pg 595.

    • 39

      The DEVTA researchers conducted a meta-analysis of DEVTA and eight previous large trials where pre-school children were provided with multiple doses of VAS per year. They found “heterogeneity between DEVTA and subtotal of eight previous trials p = 0.0010.” Awasthi et al 2013, Pg 1475.

    • 40

      “Coverage was ascertained from logbooks of overworked government community workers (anganwadi workers), and verified by a small number of supervisors who periodically visited randomly selected anganwadi workers to question and examine children who these workers gathered for them. Both anganwadi worker self-reports, and the validation procedures, are fraught with potential bias that would inflate the actual coverage . . . Although 76% of children aged 0–71 months in 2005–06 lived in areas covered by an anganwadi worker, only 22% of children received any service from the anganwadi worker. Thus, it is hard to understand how DEVTA ramped up coverage to extremely high levels (and if it did, why so little of this effort was sustained). DEVTA provided the anganwadi workers with less than half a day’s training and minimal if any incentive.” Sommer, West, and Martorell 2013, Pg 591.

    • 41

      “There is thus no plausible evidence that the intervention was or was not given to those who could most benefit.” Habicht and Victora 2013, Pg 592. “At such a large scale, it is harder to target deficient populations. At lower deficiency levels, you would expect to find a smaller effect (such as the DEVTA point estimate). DEVTA occurred significantly later (mid - 2000s) than many of the smaller trials (late '80s - early '90s). During this period, there was a worldwide reduction in vitamin A deficiency as a result of population - level interventions to reduce it.” GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013. “Xerophthalmia, leading to blinding keratomalacia, remains a public health nutrition problem and even an emergency in some parts of some lower-income countries. However, most nutritionists, physicians and others with direct field experience have, over the last several years, almost universally expressed their certainty that serious xerophthalmia and resulting blindness is now very rare now compared with the estimates made in the 1970s.” Latham 2010, Pg 33.

    • 42
      • “The biggest specific cause of death that VAS reduces is diarrhea. Deaths from diarrhea are falling but still a leading cause of childhood mortality globally.” GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013.
      • “In a reanalysis of one of the original eight trials, the beneficial effect was limited to unvaccinated children and there were strong sex-differential effects of vitamin A supplementation in vaccinated children. Hence, the roll-out of the vaccination programme might be one environmental factor that has modified the effect of vitamin A.” Benn, Fisker, and Aaby 2013, Pg 593.

    • 43
      • “Within the participating blocks, the study population was all young children living in the defined catchment areas of 8338 village anganwadi child-care centres (AWCs; anganwadi means courtyard) that the ICDS regarded as functional (except in one block 17 ) (figure 1).” Awasthi et al 2013, Pg 1471.
      • “Of 118 administrative blocks in the seven DEVTA districts in north India, 46 with many AWCs not functioning in 1998 were not included.” Awasthi et al 2013, Pg 1471.
      • “The 72 participating blocks were those within a few hours drive that the ICDS director in Lucknow judged in 1998 to have a reasonably well functioning ICDS system with willing district and block ICDS directors and paid workers in most village anganwadi child-care centres.” Awasthi et al 2013 DEWORMING, Pg 1479.

    • 44

      Note that AWCs generally serve populations with slightly higher than average child mortality rates: "The DEVTA study sites had slightly higher child mortality than the average in rural UP. This is because the DEVTA trial used AWCs to deliver VAS, and AWCs generally serve populations that are somewhat needier than average. However, AWCs are common in rural India, so the difference was not great." GiveWell's non-verbatim summary of a conversation with Richard Peto and Simon Read, April 10, 2014, Pgs. 5-6

    • 45
      • “[T]he overall effect [on all-cause mortality] was strongly influenced by five studies that accounted for over 80% of the weighted mean.” Imdad et al 2010, Pg 19.
      • Studies were weighted by the inverse of their variance. "We performed meta-analysis using Review Manager Software Version 5 (RevMan 2008). When data were extracted in several formats that could not be combined directly in RevMan, we used the generic inverse variance option; data were entered into Comprehensive Meta-Analysis Version 2 and the log RR and SE were entered into RevMan." Imdad et al 2010, Pg 12.

    • 46
      • This "Baseline Vitamin A Deficiency" column includes the baseline rate of xerophthalmia reported in each study. Not all studies report their definition of xerophthalmia, which is a term that can encompass a wide range of conditions, so comparisons among studies must be made cautiously.
      • "The term xerophthalmia encompasses the clinical spectrum of ocular manifestations of VAD, from milder stages of night blindness and Bitot’s spots, to potentially blinding stages of corneal xerosis, ulceration and necrosis (keratomalacia) (17), as listed in Table 1. The stages of xerophthalmia are regarded both as disorders and clinical indicators of VAD, and thus can be used to estimate an important aspect of morbidity and blinding disability as well as the prevalence of deficiency. As corneal disease is rare, the most commonly assessed stages are night blindness, obtainable by history, and Bitot’s spots, observable by handlight examination of the conjunctival surface. Standard procedures exist for assessing xerophthalmia" WHO 2009, Pg 2.

    • 47This column includes baseline infant/child mortality rates as one measure of baseline health. We report participants' ages next to infant/child mortality rates because mortality rates among infants are generally much higher than mortality rates among other pre-schoolers.
    • 48
      • Ross et al 1993 was a "double-blind, randomised, placebo-controlled trial[].. Ross et al 1993, Pg 7. The study "included 21,906 children aged 6-90 months in 185 geographical clusters." Ross et al 1993, Pg 7.
      • "In the Survival Study, children were visited and dosed by trained fieldworkers every 4 months for 2 years in seven survey rounds." Ross et al 1993, Pg 8. "[C]hildren aged less than 6 months were not eligible, children 6-11 months old received 100,000 IU retinol equivalent or placebo, and children 12 months or older received 200,000 IU retinol equivalent or placebo." Ross et al 1993, Pg 8.
      • "All children with confirmed active xerophthalmia or its sequelae (corneal scars) were withdrawn from the trial at diagnosis." Ross et al 1993, Pg 9.

    • 49"The trials took place in the guinea savannah area of Ghana in the Kassena-Nankana District, on the border with Burkina Faso (Figure 1). . . . The study populations were exclusively rural." Ross et al 1993, Pg 7.
    • 50Ross et al 1993, Pg 7. The study "included 21,906 children aged 6-90 months in 185 geographical clusters." Ross et al 1993, Pg 7.
    • 51"The main staple foods are millet, sorghum, and groundnuts, and the diet is deficient in carotenoids and vitamin A. Red palm oil, a major source of carotenoids in coastal West Africa, was found in only 2% of compounds during baseline surveys. However, about a third of compounds owned at least one mango tree, and about 10% one or more pawpaw (papaya) trees. Vitamin A deficiency and xerophthalmia are recognised as problems locally; words exist for night blindness in all the local languages and cases of xerophthalmia are reported by local ophthalmological services. There was no vitamin A supplementation programme within the study area." Ross et al 1993, Pg 7.
    • 52"The prevalence of xerophthalmia at baseline was 0.7% in the Survival Study children. . . . The proportions of children with low baseline serum retinol concentrations were substantial in both trial populations. In the Survival Study, 14.4%were severely deficient (< 0.35 mol/L) and 42.5% moderately deficient (0.35-0.69 ) μmol/L)." Ross et al 1993, Pg 10.
    • 53Ross et al 1993, Pg 10, Table 1.
    • 54Ross et al 1993, Pg 10, Table 1.
    • 55
      • "There were 892 deaths among the children in the Survival Study, which gave an overall mean mortality rate for all clusters of 27.11 per 1,000 child-years of follow-up. 397 of the deaths were in vitamin A clusters (mean mortality rate 24.4 per 1000 child-years) and 495 in placebo clusters (29.9 per 1000 child-years). " Ross et al 1993, Pg 10.

    • 56Ross et al 1993, Pg 10, Table 2
    • 57
      • West et al 1991 was "a randomized, double-masked, placebo-controlled vitamin A supplementation trial." West et al 1991, Pg 67.
      • "After a random start, each of the 261 wards was systematically allocated to one of four coded batches of supplements. Two codes contained a standard 60,000 ug retinol equivalent (200,000 IU) dose of vitamin A and two served as control, containing 300 retinol equivalents (1000 IU) of vitamin A (retinyl palmitate in arachis oil; Roche, Basel, Switzerland). . . . All households in a ward were visited by trained field staff every 4 months for a year. At baseline, children of 60 months and younger were enrolled; at follow-up visits only infants born during the study were later enrolled. . . . At each visit infants aged 6-11 months received half the contents of a capsule (6 drops of oil), equivalent to either 30 000 or 150 retinol equivalents, respectively, and children of 12 months and older received the whole contents of the capsule." West et al 1991, Pg 68. "The initial 24,805 children plus 3825 infants who entered the 6-11 month age range at the first or second 4-monthly visits (1879 controls, 1946 vitamin A) provided a total of 28,630 children who contributed 25,970 child-years of observation for this analysis (table III)." West et al 1991, Pg 69.
      • "At baseline an ocular survey was carried out in a 15% random sample of wards (n = 40). . . . All xerophthalmic and severely ill children were treated with vitamin A and referred to local health posts for follow-up, as indicated. These children are included in the analysis." West et al 1991, Pg 68.

    • 58"A randomised, double-masked, placebo-controlled vitamin A supplementation trial was carried out from September, 1989, to December, 1990, in the rural, plains (Terai) district of Sarlahi. This area was selected because it has endemic vitamin A deficiency, no previous vitamin A supplementation programme, and ecological, cultural, and demographic similarities to the Gangetic floodplain communities of South Asia, with which it is continuous; the factor enhances the general applicability of the findings." West et al 1991, Pg 67.
    • 59"A randomised, double-masked, placebo-controlled community trial of 28 630 children aged 6-72 months was carried out in rural Nepal." West et al 1991, Pg 67.
    • 60"The pooled prevalence of xerophthalmia in the random subsample of wards was 3.0%." West et al 1991, Pg 69. Baseline prevalence of xerophthalmia was higher in the control group (3.5%) than in the treatment group (2.6%). West et al 1991, Pg 68, Table I.
    • 61West et al 1991, Pg 68, Table I.
    • 62West et al 1991, Pg 68, Table II.
    • 63
    • 64West et al 1991, Pg 68, Table IV.
    • 65West et al 1991, Pg 68, Table III.
    • 66
      • Herrera et al 1992 was "a double-blind, placebo-controlled trial of vitamin A supplementation among children aged 9-72 months in the Sudan." Herrera et al 1992, Pg 267.
      • "Assignment to treatment group was achieved by the two interviewers visiting alternate households throughout the village." Herrera et al 1992, Pg 267.
      • "Colour-coded capsules containing either 200 000 IU vitamin A and 40 IU vitamin E (vitamin A group) or 40 IU vitamin E (placebo group) were provided by Hoffman-La Roche, Basel, Switzerland." Herrera et al 1992, Pg 267. "Our objectives were: (a) to test the efficacy of large doses of vitamin A given every 6 months . . ." Herrera et al 1992, Pg 267.
      • "There were 17,301 eligible households enrolled at baseline. 29,615 children were screened, of whom 862 (3%) were diagnosed as xerophthalmic, treated, and not followed up further (figure). Thus, 28,753 children were enrolled." Herrera et al 1992, Pg 268.
      • "Xerophthalmic children were removed from the remainder of the study." Herrera et al 1992, Pg 268.

    • 67"The study was conducted between June, 1988, and December, 1990, among children between 9 and 72 months of age in five rural councils in northern Sudan where vitamin A deficiency was present." Herrera et al 1992, Pg 267.
    • 68"The study was conducted between June, 1988, and December, 1990, among children between 9 and 72 months of age." Herrera et al 1992, Pg 267.
    • 69"The study was conducted between June, 1988, and December, 1990, among children between 9 and 72 months of age in five rural councils in northern Sudan where vitamin A deficiency was present. There are local expressions for night blindness such as "jahar", suggesting awareness of the problem among the target population." Herrera et al 1992, Pg 267.
    • 70"There were no important differences between vitamin A and placebo groups in rate of xerophthalmia (2.8 vs 2.9%), vitamin A intake, age distribution, or nutritional status." Herrera et al 1992, Pg 269.
    • 71Herrera et al 1992, Pg 269, Table I.
    • 72Herrera et al 1992, Pg 269, Table I.
    • 73
      • “During the 18 months of follow-up, there were 120 deaths (8.4/1000) in the vitamin A group and 112 deaths (7.9/1000) [in the control group].” Herrera et al 1992, Pg 269.
      • Therefore, in the control group there were 7.9 deaths/1,000 children over a course of 18 months. To calculate the deaths per 1,000 child years, we multiply that number by 2/3. 7.9*2/3 = 5.3.

    • 74Herrera et al 1992, Pg 269, Table III.
    • 75
      • Daulaire et al 1992 was a randomized, controlled, unblinded trial with no placebo. 7,197 children aged 0-59 months in 16 subdistricts were included in the study. 8 subdistricts were randomly assigned to the treatment group. Daulaire et al 1992, Pg 207.
      • "We randomly selected by card eight of the 16 subdistricts for vitamin A supplementation. The other eight sub-districts served as concurrent observation areas; there was no placebo or blinding." Daulaire et al 1992, Pg 207.
      • "During three weeks in May-June 189 a single high dose capsule of retinol palmitate (with vitamin E) was given to all children in the selected districts. The standard dose was 200,000 IU vitamin A for children aged 12-59 months, 100,000 IU for infants aged 6-11 months, and 50,000 IU for infants under 6 months." Daulaire et al 1992, Pg 207.
      • There were 3,786 children in the treatment group and 3,411 children in the control group. Daulaire et al 1992, Pg 208, Table I. There were therefore 7,197 children in the study. "Only neonates with jaundice were excluded from coverage." Daulaire et al 1992, Pg 207.

    • 76"Jumla district lies in a remote mountainous region of northwestern Nepal." Daulaire et al 1992, Pg 207.
    • 77Daulaire et al 1992, Pg 208, Table III.
    • 78"In a survey of 3,651 children under 5 years, active xerophthalmia was detected in 13.2%." Daulaire et al 1992, Pg 207. For this statistic, the authors cite a study that GiveWell was unable to access: Acharia et al (1990). Vitamin A Deficiency in a Mountain Area of Nepal: the 1989 Jumla Xerophthalmia Survey. In: Proceedings of the Twelfth IVACG Meeting, Kathmandu, Nepal. Washington, DC: International Life Sciences Institute-Nutrition Foundation.
    • 79
      • Daulaire et al 1992, Pg 208, Table III.
      • Total child years at risk among children aged 12-59 months = 320.9 child years at risk among 12-23 month olds + 270.8 child years at risk among 24-35 year olds + 237.7 child years at risk among 36-47 year olds + 223.5 child years at risk among 48-59 month olds = 1,052.9. Total deaths among children aged 12-59 months = 71 deaths among 12-23 month olds, 22 deaths among 24-35 month olds, 11 deaths among 36-47 month olds, and 3 deaths among 48-59 month old= 107 deaths among 12-59 month olds. Death rate among 12-59 month olds = 1,000*107 deaths/1052.9 child years at risk=101.6.

    • 80Daulaire et al 1992, Pg 208, Table III.
    • 81
      • Sommer et al 1986 was a randomized, controlled, unblinded study with no placebo. 25,939 children aged 12-71 months at baseline were included in the study. The trial was randomized by village. Villages already receiving VAS were excluded from the study. Sommer et al 1986, Pg 1169.
      • "From a random start, 450 villages were systematically selected for the study; these were then randomized for capsule distribution after the baseline examination (programme villages, n=229) or after the follow-up examination (control villages, n =221). 18 villages from among those still in the sampling frame were substituted for adjacent villages found to have started vitamin A supplementation before the baseline survey." Sommer et al 1986, Pg 1169.
      • "Of the 25,939 children with baseline and follow-up information, details of the initial ocular examination are available for 91.9% of programme children and 90.5% of controls." Sommer et al 1986, Pg 1170.
      • "The Government of Indonesia would not condone the use of placebos but field-workers collecting demographic data were unaware that mortality was a research issue." Sommer et al 1986, Pg 1172.
      • "The present study was thus undertaken, partly to determine whether supplements of vitamin A given to preschool children (12-71 months of age) would reduce their mortality by at least 15%." Sommer et al 1986, Pg 1172.
      • The capsule nipple was snipped off and the contents (200,000 IU vitamin A and 40 IU vitamin E) were expressed into the child's mouth. . . The first dose was given 1-3 months after the baseline examination and the second 6-8 months later." Sommer et al 1986, Pg 1169.
      • "All children with active xerophthalmia at baseline examination received at least one large dose of vitamin A and were referred to the local health unit. They were excluded from the analyses of subsequent morbidity and mortality." Sommer et al 1986, Pg 1169.

    • 82"The study was carried out in Aceh Province, which is at the northern tip of Sumatra and where xerophthalmia is prevalent." Sommer et al 1986, Pg 1169.
    • 83"Capsules containing 200000 IU vitamin A were distributed to preschool children aged over 1 year ... Among children aged 12-71 months at baseline, mortality in control villages... was 49% greater than in those where supplements were given." Sommer et al 1986, Pg 1169.
    • 84Sommer et al 1986, Pg 1170, Table I.
    • 85Sommer et al 1986, Pg 1170, Table III.
    • 86Sommer et al 1986, Pg 1170, Table III.
    • 87Sommer et al 1986, Pg 1171, Table VI.
    • 88The study reports the inverse of the mortality rate ratio as 1.36 (1.01-1.85). Sommer et al 1986, Pg 1171, table VI. We invert this ratio to get 0.73 (0.54-0.99). The mortality rate ratio is slightly lower among 12-71 month olds.
    • 89

      The mortality rate in Herrera et al 1992’s control group was 5.3/1,000 child-years. The mortality rate in the control groups from the other studies were:

    • 90
      • “Comparison of baseline rates of xerophthalmia across studies does not suggest that the severity of vitamin A deficiency influenced the efficacy of supplementation. Tamil Nadu had the highest rate followed by, in descending order, Hyderabad, Nepal, Sudan, and Indonesia. However, eye signs of vitamin A deficiency are an imprecise endpoint: they were used because other practical and more valid methods were not available.” Herrera et al 1992, p. 270.
      • The baseline prevalence of xerophthalmia in Herrera et al 1992 was about 2.9% (see Table 2), which is comparable to rates from most of the other trials. The baseline prevalence of xerophthalmia in Ross et al 1993 was 0.7% (see Table 2). Baseline prevalence in West et al 1991 was 3.0% (see Table 2). A survey of children in the area covered by Daulaire et al 1992 found active xerophthalmia in 13.2% of children (see Table 2). In Sommer et al 1986, 2.3% of children in the control group and 1.9% of children in the treatment group had active xerophthalmia at baseline (see Table 2). These comparisons should be interpreted modestly, however, because definitions of xerophthalmia may vary across studies.

    • 91
      • “As the study began it had a million participants, and during it another million 1-year-olds joined it and a million 6-year-olds left (figure 2), so at one time or another it included about 2 million children.” Awasthi et al 2013, Pg 1470. By contrast, Ross et al 1993, the trial that received the biggest weight in the Cochrane meta-analysis, included just 21,906 participants and West et al 1991 included 28,630 participants.
      • "To achieve 96% coverage in Uttar Pradesh in children found in the anganwadi workers’ registries would have been an astonishing feat; covering 72% of children not found in the anganwadi workers’ registries seems even more improbable. In 2005–06, shortly after DEVTA ended, only 6·1% of children aged 6–59 months in Uttar Pradesh were reported to have received a vitamin A supplement in the previous 6 months according to results from the National Family Health Survey,2 a national household survey representative at national and state levels. The level of contact between anganwadi workers and children has historically been very low. Although 76% of children aged 0–71 months in 2005–06 lived in areas covered by an anganwadi worker, only 22% of children received any service from the anganwadi worker. Thus, it is hard to understand how DEVTA ramped up coverage to extremely high levels (and if it did, why so little of this effort was sustained). DEVTA provided the anganwadi workers with less than half a day’s training and minimal if any incentive. Each of their 18 study monitors was responsible for overseeing the work of 463 anganwadi workers and the status of 55 000 children." Sommer, West, and Martorell 2013, Pg 1.
      • "The reference implies that the DEVTA program did not achieve the compliance it had reported. However, DEVTA was completed in autumn 2004, a significant amount of time before the National Family Health Survey. Additionally, DEVTA operated in only 7 out the 70 districts in UP and provided VAS in only 3% of the blocks (sub-districts) in UP. Among those blocks, VAS was distributed only from ones that had functioning AWCs." GiveWell's non-verbatim summary of a conversation with Richard Peto and Simon Read, April 10, 2014, Pg. 8

    • 92

      “Independently, enquiries a week after mass-treatment days about lists of named children from the mid-study census confirmed supplementation of 96% (133 602/138 966) of those registered with the AWC and 72% (80 048/111 227) of those not, largely independent of age (data not shown). Because two-thirds of under-5s and one-third of 5-year-olds were AWC-registered, overall compliance was about 86% (not 91%, as previously estimated).” Awasthi et al 2013, Pg 1472.

    • 93

      Some coverage rates from other trials are given below. We have not ensured that they are directly comparable to DEVTA's published coverage rate.

      • West et al 1991 reported that “[c]apsule coverage remained high and equal in each group throughout the trial-about 88% of children in each group received the capsule contents from the staff at each visit. Capsules were left in the home for about 10% of children at each visit, of whom about 65% were estimated (by history on follow-up of a subsample) to have received the capsule contents from the parents. Thus, about 93% of all eligible children in each group received a supplement at each visit.” West et al 1991, Pg 69. The true compliance rate is slightly smaller because more than 3% of children did not enter the study at all. West et al 1991, Pg 69.
      • "Dosing compliance was similar in the two groups; an average of 89.5% of eligible children were successfully treated in each round." Ross et al 1993, Pg 8.
      • "Thus, 28,753 children were enrolled. 3,320 children did not receive one or two of the three vitamin A or placebo capsules." Herrera et al 1992, Pg 268. There were therefore 28,753*3 = 86,259 total doses intended to be distributed. The compliance rate was somewhere between 92%=[1 – (3,320*2/86,259)]*100 if all of the noncompliant children missed two doses and 96%=[1 – (3,320/86,259)]*100 if all of the noncompliant children missed just one dose.
      • "During the supplementation campaign 3,345 (88%) of the 3,786 eligible children received vitamin A." Daulaire et al 1992, Pg 208.
      • "Over 93% of preschool children (12-71 months at baseline) living in programme villages received at least one large dose of vitamin A between baseline and follow-up examinations; 78% received two doses." Sommer et al 1986, Pgs 1170-1171.

    • 94
      • “The program was delivered very inexpensively compared with previous programs and it used existing centers and care providers. Limited investment in the implementation of the intervention (e.g. training and monitoring) may help explain the relatively poor result.” GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013.
      • “Excluding donated drug, the total cost was US $100 000 per year (US$0.10 per child), but this expense was mostly for evaluating the intervention.” Awasthi et al 2013 DEWORMING, Pg 1484.
      • According to an account of an unpublished workshop on DEVTA, “DEVTA provided the anganwadi workers with less than half a day’s training.” Sommer, West, and Martorell 2013, Pg 591, citing a workshop on the DEVTA Study at Worcester College, Oxford, UK from Nov. 3-4, 2008; the workshop proceedings have not been published.
      • "But this was neither a rigorously conducted nor acceptably executed efficacy trial: children were not enumerated, consented, formally enrolled, or carefully followed up for vital events, which is the reason there is no CONSORT diagram." Sommer, West, and Martorell 2013, Pg. 1

    • 95
      • “When treatment became due (in April or October), in each treatment-allocated block a mass-treatment day was selected on which all children of apparent age 6-72 months in all study AWCs in that block would be given the trial treatment by their AWC worker. Those treated were checked off on the AWC lists, and the few children missed were often found by the AWC worker and treated soon afterwards.” Awasthi et al 2013, Pg 1470.
      • “Implementation of the mass-treatment days was encouraged and monitored independently by project staff and state government staff . At the previous monthly block meeting, each supervisor in the AWCs to be treated collected 150 tablets (plus extras, for large AWCs) and, on the selected day, administered treatment. Villagers were alerted beforehand. To ensure AWC workers undertook mass-treatment on the chosen day, they were trained before the study (with refresher training in 45 blocks), discussed matters at their monthly block meetings before and afterwards, and knew that on all mass-treatment days unannounced visits were made to every fourth AWC on a numbered list for each block to check that distribution had started, followed by visits 1 week later to check numbers dispensed and interview focus groups about receiving treatment.” Awasthi et al 2013 DEWORMING, Pgs 1479-1480.
      • “Compliance with treatment allocation was good. Un-announced visits to a quarter of the AWCs in vitamin A blocks on or just after each mass-treatment day found 99% (10 925/11 090 visits) were distributing treatment. Independently, enquiries a week after mass-treatment days about lists of named children from the mid-study census confirmed supplementation of 96% (133 602/138 966) of those registered with the AWC and 72% (80 048/111 227) of those not, largely independent of age (data not shown). Because two-thirds of under-5s and one-third of 5-year-olds were AWC-registered, overall compliance was about 86% (not 91%, as previously estimated 18 ). Awasthi et al 2013, Pg 1472. The compliance rates appear to have been calculated from a subsample of 25 AWCs per block. “[T]his mid-study census was used before each subsequent treatment day to list all in 25 AWCs per block with predicted age 12–70 months (ie, well within the range to be treated). Monitors established soon after whether the children listed had actually been treated (and whether they were currently registered with the AWC).” Awasthi et al 2013 DEWORMING, Pg 1480.
      • “Consistent with this finding [of high compliance], the biomedical surveys of randomly chosen AWCs, which tended to over-sample AWC-registered children, found caregivers reporting 91% (2337/2581) had received retinol on the previous mass-treatment day. . . .Allocation to retinol halved the prevalence of severe deficiency 1–5 months after mass-treatment days. Retinol assays were available (only during the second half of the study) for 5165 children with complete information on all assays and questionnaire replies, 2581 in retinol-allocated versus 2584 in control blocks (table 1). Among them, mean retinol was 0.72 (SE 0.01) μmol/L versus 0.62 (0.01) μmol/L, a 16% increase (p<0.00001). These retinol values are both 1.2 times higher (see appendix) than the uncalibrated values reported previously. 18 Since compliance with the previous mass supplementation was 91% in children who gave blood, but only 86% overall, the proportional increase in plasma retinol in the whole study population in retinol-allocated blocks was about 15%. Among those whose blood was tested, the prevalence of severe vitamin A deficiency (retinol <0.35 μmol/L) was 6% versus 13%. Haemo globin, weight, and height were not significantly affected. The prevalence of Bitot’s spots (1.4% vs 3.5%) was also halved, as was that of conjunctivitis within the past 4 weeks. Night blindness is difficult to assess in young children, and was uncommon in the absence of Bitot’s spots.” Awasthi et al 2013, Pg 1472-1473.
      • The study’s protocol described additional checks on the AWC workers. “The anganwady workers will record the fact of giving the doses to each child in their regular anganwady registers. Drug supplies that are left over will be collected at the next monthly meeting.” Awasthi et al 2013 APPENDIX, Pg 4.

    • 96
      • The description of the methods for the mid-study census contains inadequate detail to determine the data’s accuracy. “During the third study year, study monitors abstracted from local records name, sex, age, and father’s name for all (just over 1 million) pre-school children in the study areas. There should have been little undercount of children, but numbers by month of age showed, as expected, substantial undercount of young infants, and ages were often approximate.” Awasthi et al 2013 DEWORMING, Pg 1480. “[A]pparently duplicated records (about 1% at ages 1.0-6.0 years)” were removed. Awasthi et al 2013, Pg 1480. No information is given about the reliability of these local records or the process for abstracting them. The census was used, however, as the sample for measuring compliance and to create a list of children who were absent for mass-treatment days and with whom AWC workers treated individually. Therefore, a census undercount could have led to both overestimates of compliance and reduced actual compliance.
      • Biomedical tests of a convenience sample of participants were consistent with high VAS compliance, but members of the convenience sample were probably disproportionately easy to reach. “Annually, Oxford randomly selected one AWC per block for teams (one phlebotomist, two fieldworkers, one driver) to survey 1–5 months after a mass-treatment day 30 pre-school children (six per year of age; if fewer were available, a neighbouring AWC was included), seeking from them blood and faecal samples. In practice, full information with complete assay results was obtained for about 24 children per block. Children surveyed were not randomly chosen, and tended to be selected by AWC workers from those registered with the AWC. Villagers decided after public discussion whether the village would cooperate; if it did, no caregiver then chose to refuse consent. Only one selected village refused; an adjacent village replaced it.” Awasthi et al 2013 DEWORMING, Pg 1480.
      • “Fieldworker teams were trained to seek Bitot’s spots and measure finger-prick haemoglobin. . . . Retinol assay results were, however, available only after mid-study because earlier samples were biologically contaminated in storage and were unmeasurable.” Awasthi et al 2013, Pg 1471. The study’s census was also not available until mid-study. “During the third study year, study monitors abstracted from local records name, sex, age, and father’s name for all (just over 1 million) pre-school children in the study areas.” Awasthi et al 2013 DEWORMING, Pg 1480. Because the compliance data relied on the study to identify a sample, there is no compliance data from the first two years of DEVTA. It is plausible that compliance could be lower during those years. In general, we would guess that distribution improves over time as workers and monitors become familiar with their roles. More importantly, however, during the early years of the study, for which we have no compliance data, workers were unable to use the Census data to identify, locate, and treat children who were absent from mass treatment days. “Those treated were checked off on the AWC lists, and the few children missed were often found by the AWC worker and treated soon afterwards.” Awasthi et al 2013, Pg 1470. This factor is unlikely to be very important because the mortality rate ratio during the first two years of the study (0.96) is the same as the mortality rate ratio from the last three years of the study. (0.96). Awasthi et al 2013, Pg 1474. If DEVTA’s results were caused by lax procedures at the study’s start, we would expect to see a smaller effect in the first half of the study.

    • 97

      DEVTA reported a mortality risk reduction of 4%. Assuming that the meta-analysis of the previously executed trials estimates the mortality risk reduction among those treated (these trials seem to have achieved very high coverage rates) and assuming that the mortality risk reduction scales linearly with the coverage rate, DEVTA would have had to have a coverage rate of about 24% in order for the mortality risk reduction among those treated in DEVTA to equal the upper bound of the 95% confidence interval (17% RR) for the mortality risk reduction from the meta-analysis of previously executed trials (0.04/0.24 = 0.17). It seems very unlikely that the coverage rate was this low.

    • 98
      • Evan Mayo Wilson, one of the authors of the Cochrane review, believes this may have been a factor. “DEVTA occurred significantly later (mid - 2000s) than many of the smaller trials (late '80s - early '90s). During this period, there was a worldwide reduction in vitamin A deficiency as a result of population - level interventions to reduce it.” GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013.
      • A 2009 World Health Organization report could not firmly conclude whether VAD is on the increase or on the decline. “Although these numbers are very difficult to compare due to differences in the methodology used to produce them, considering the growth of the world’s population, there appears to be some indication that the number of preschool-age children affected by xerophthalmia may be decreasing, but that the number of preschool-age children and pregnant women with biochemical VAD, based on deficient serum concentrations of retinol, is increasing, possibly due to better methods of assessment and a wider population being assessed.” WHO 2009, Pg 17.
      • UNICEF, however, found a large increase in the percentage of children in priority countries who are fully protected with two doses of VAS. “In 1999, just 16 per cent of children in the 103 priority countries were fully protected with the recommended two doses. By 2004, this proportion had climbed to 58 per cent, representing more than a threefold increase.” UNICEF 2007, Pg 9.

      We have not fully vetted either organization’s data.

    • 99

      “At such a large scale, it is harder to target deficient populations. At lower deficiency levels, you would expect to find a smaller effect (such as the DEVTA point estimate).” GiveWell's non-verbatim summary of a conversation with Evan Mayo-Wilson, June 10, 2013.

    • 100
      • Our intuition is that VAD might be more severe in more remote parts of the study area. We would also guess that coverage would be lower in more remote areas.
      • If coverage was lower in more remote areas, DEVTA's methods for monitoring coverage might not have picked up this lower coverage. We would guess that DEVTA's biomedical survey, which is some of the strongest evidence for DEVTA's high coverage rate, disproportionately surveyed easy-to-reach participants. “Annually, Oxford randomly selected one AWC per block for teams (one phlebotomist, two fieldworkers, one driver) to survey 1–5 months after a mass-treatment day 30 pre-school children (six per year of age; if fewer were available, a neighbouring AWC was included), seeking from them blood and faecal samples. In practice, full information with complete assay results was obtained for about 24 children per block. Children surveyed were not randomly chosen, and tended to be selected by AWC workers from those registered with the AWC. Villagers decided after public discussion whether the village would cooperate; if it did, no caregiver then chose to refuse consent. Only one selected village refused; an adjacent village replaced it.” Awasthi et al 2013 DEWORMING, Pg 1480. We would guess that AWC workers did not choose remote or difficult to find children for this survey.

    • 101
      • About halfway through the trial, many of the children in the control group received one dose of vitamin A from an organization called Pulse Polio. Peto 2013 LECTURE, 18:40-18:57.
      • "The policy in India was to encourage vitamin A supplementation to prevent eye disease, but in many parts of the country this was not widely implemented." Awasthi et al 2013, Pg 1469. Therefore, “the comparison should in principle be described as being of routine vitamin A versus occasional vitamin A supplementation.” Awasthi et al 2013 APPENDIX, Pgs 3-4.
      • Because the WHO recommends that children receive VAS every four to six months, we would not expect the one dose distributed by Pulse Polio to control group members to be responsible for the difference between DEVTA and prior trials. WHO 1997, Pg 4.

    • 102

      Awasthi et al 2013, Pg 1472.

      The World Health Organization has written that “[m]easuring serum retinol concentrations in a population constitutes the second major approach to assessing vitamin A status in a population, with values below a cut-off of 0.70 µmol/l representing VAD, and below 0.35 µmol/l representing severe VAD.” WHO 2009, Pg 2. The WHO characterizes a 20% or higher prevalence of serum retinol levels that are less than 0.70 µmol/l to be a “severe public health problem.” WHO 2009, Pg 11. Overall, the WHO has estimated that 33.3% of pre-school children have VAD in countries with a GDP less than US$15,000. WHO 2009, Pg 10. If the data can be trusted, VAD was easily a severe public health problem among DEVTA’s participants.

    • 103

      Awasthi et al 2013, Pg 1472.

    • 104

      For example, among participants in Ross et al 1993 56.9% of children were VAD and 14.4% were severely VAD according to retinol concentrations at baseline. In Sommer et al 1986, about 1.3% of participants had Bitot’s spots and about 1.2% of participants had night-blindness.

    • 105

      “Annually, Oxford randomly selected one AWC per block for teams (one phlebotomist, two fieldworkers, one driver) to survey 1–5 months after a mass-treatment day 30 pre-school children (six per year of age; if fewer were available, a neighbouring AWC was included), seeking from them blood and faecal samples. In practice, full information with complete assay results was obtained for about 24 children per block. Children surveyed were not randomly chosen, and tended to be selected by AWC workers from those registered with the AWC. Villagers decided after public discussion whether the village would cooperate; if it did, no caregiver then chose to refuse consent. Only one selected village refused; an adjacent village replaced it.” Awasthi et al 2013 DEWORMING, Pg 1480. The children selected were probably the easiest children to reach and therefore may have had different diets, access to healthcare, poverty levels, and overall health from the general population. Our intuition is that the data likely underestimates the prevalence of VAD in the treatment group.

    • 106

      Awasthi et al 2013, Pg 1474.

    • 107

      Herrera et al 1992, Pg 269.

    • 108
      • See Table 1 for the sources for the numbers cited in this footnote.
      • Ross et al 1993: Given that removing the children aged 6 to 12 months didn't change the mortality rate much for West et al 1991, removing the children aged 72 to 90 months would only increase the mortality rate and the mortality rate in Ross et al 1993 far exceeds the mortality rate in DEVTA, we would expect that removing children aged 6 to 12 months and children aged 72 to 90 months from Ross et al 1993 wouldn't make the mortality rate comparable to the mortality rate in DEVTA and similarly for the mortality risk reduction.
      • West et al 1991 reports a mortality rate in the control group for the children aged 12 to 72 months (the same age group examined in DEVTA) of 14.2/1000 child-years, which is a little lower than the mortality rate including all ages in the study, but still higher than the mortality rate in the control group in DEVTA. West et al 1991 reports a mortality risk reduction of 32% for the children aged 12 to 72 months (West et al 1991, Pg. 68, Table IV), which is slightly more than the statistically significant mortality risk reduction of 30% when including all ages in the study.
      • Herrera et al 1992 reports a mortality rate of 5.3/1000 child-years for children aged 9 to 72 months in the control group. Excluding the children aged 9 to 12 months from this calculation would only lower this mortality rate, because younger children tend to have higher mortality rates. We'd also guess that vitamin A doesn't have an unusually large effect on this age group and so excluding these children wouldn't decrease the mortality risk reduction enough to make the result look more like the other trials and less like DEVTA.
      • Daulaire et al 1992 reported a mortality rate of 101.6/1000 child-years for children aged 12 to 59 months in the control group. Adding children aged 59 to 72 month would decrease the mortality rate, but not enough to make it comparable to DEVTA (If we assume that the mortality rate was 0 in the children aged 59 to 72 months, then the mortality rate per 1000 child-years including that age group is 1000*((101.6+0)/(1000 + x)) where x is the number of child-years for the children aged 59 to 72 months. If we assume the overall mortality rate equals that of DEVTA (5.3/1000) and solve for x, we get around 18,000, so the number of child-years for the children aged 59 to 79 would have to about 18 times the number of child-years for the children aged 12 to 59 months for the mortality rate to equal that of DEVTA and even more if the mortality rate in that age group was more than 0). Daulaire et al 1992 reported a mortality risk reduction of 22% for children aged 12 to 59 months (Table III, Daulaire et al 1992, Pg. 208). Given the low number of deaths and child-years at risk for the older age groups, we'd guess that adding children aged 59 to 72 into the sample wouldn't decrease the mortality risk reduction much more.
      • Sommer et al 1986 reported a mortality rate of 7.4/1000 child-years in the control group for children aged 12 to 71 months (nearly the age group examined in DEVTA). The study reported a mortality risk reduction of 34% for children aged 12 to 71 months (Table VI, Sommer et al 1986, Pg. 1171).

    • 109

      Sources for control group mortality and for the mortality risk ratio in Table 2 are in Table 1.

    • 110

      “Deaths per child-care centre at ages 1.0–60 years during the 5-year study (the primary trial endpoint) were 3.01 retinol versus 3.15 control (absolute reduction 0.14 [SE 0.11], mortality rate ratio [RR] 0.96, 95% CI 0.89–1.03, p=0.22), suggesting absolute risks of death between ages 1.0 and 6.0 years of approximately 2.5% retinol versus 2.6% control.” Awasthi et al 2013, Pg 1473

    • 111
      Inputs to mortality rate in DEVTA (xlsx)
    • 112
      • Table 2, Ross et al 1993, Pg. 10
      • "The 21 906 children who entered the Survival Study were followed up for 33 287 child-years (16 508 vitamin A group, 16 779 placebo group)." Ross et al 1993, Pg. 10
      • "There were 892 deaths among the children in the Survival Study, which gave an overall mean mortality rate for all clusters of 27.11 per 1,000 child-years of follow-up. 397 of the deaths were in vitamin A clusters (mean mortality rate 24.4 per 1000 child-years) and 495 in placebo clusters (29.9 per 1000 child-years)." Ross et al 1993, Pg 10

    • 113
      Table III, West et al 1991, Pg. 68
    • 114
      • Table III, Herrera et al 1992, Pg. 269
      • "During the 18 months of follow-up, there were 120 deaths (8.4/1000) in the vitamin A group and 112 deaths (7.9/1000) in the placebo group (RR 1.06, 95% CI 0.82-1.37) (table II)." Herrera et al 1992, Pg. 269

    • 115
      Table III, Daulaire et al 1992, Pg. 208
    • 116
      • Table VI, Sommer et al 1986, Pg. 1171
      • "Teams first visited villages between September, 1982, and August, 1983, and follow-up visits were made by the same team in the same sequence 9-13 months later." Sommer et al 1986, Pg. 1169

    • 117

      See above.

    • 118

      Awasthi et al 2013, Pg 1472.

    • 119

      For example, the mean daily prevalence of measles and diarrhea in the placebo group from Ross et al 1993 were 0.1% and 15.9% respectively. In West et al 1991. About 5.5% of children had measles during the four months preceding the study and 11% of children had diarrhea during the week preceding the study. In Herrera et al 1992, 0.3% of children had measles in the seven days preceding the study and just over 17% had diarrhea in the seven days preceding the study. In Sommer et al 1986, about 22% of children had measles at any time in the past and about 7.9% of children had diarrhea in the seven days before the study. Because of the differing time periods covered, these data are not directly comparable to prevalence rates among DEVTA participants, but they seem to suggest reasonably similar overall levels of illness.

    • 120
      • Decline in diarrhea mortality. "Mortality from diarrhoea has declined over the past two decades from an estimated 5 million deaths among children under five to 1.5 million deaths in 2004,7 which parallels downward trends in overall under-five mortality during this period." UNICEF 2009, Pg 5. We have not vetted this study but do believe the result that deaths from diarrhea have declined.
      • Decline in measles mortality. "In 1980, before widespread global use of measles vaccine, an estimated 2.6 million measles deaths occurred worldwide (1). . . . [T]he estimated number of annual measles deaths worldwide decreased from 733,000 in 2000 to 164,000 in 2008 (3)." CDC 2012, Pg 73. We have not vetted this study but do believe the result that deaths from measles have declined.
      • We would guess that these reduced deaths are due, at least in part, to the increased use of oral rehydration treatment for diarrhea and measles vaccination.
      • One study found that VAS only reduces mortality among children who have not been vaccinated. “[W]ithin the 2-y trial period, VAS only had a beneficial effect for children without vaccinations; the mortality rate ratio (MRR) was 0.64 (95% CI: 0.47, 0.88) in these children compared with 0.95 (95% CI: 0.72, 1.26) in children with vaccinations (test of interaction between VAS and vaccination status, P= 0.057).” Benn et al. 2009, Pg 633. However, another study found that VAS is highly effective among children whose last vaccination was a measles vaccine. “In the 2814 children who had received MV as the most recent vaccine prior to the campaign, the adjusted MRR was 0.34 (0.14; 0.85).” Fisker et al 2012, Pg 6. We have not fully vetted either of these studies.

    • 121

      “In our analysis of 18 trials, DEVTA 2007 accounted for 65.2% of the combined effect, which remained significant (RR= 0.88 (95% CI 0.84 to 0.94)) with substantial and significant heterogeneity (Chi² = 44.31, df = 16 (P = 0.0002); I² = 64%). (We assumed the study authors adjusted for clustering. As the results were not significant, inflating the SE would not change our interpretation of the trial’s results, although it would decrease its weight in the analysis.)” Imdad et al 2010, Pg 19.

    • 122

      Workshop on the DEVTA Study at Worcester College, Oxford, UK from Nov. 3-4, 2008.

    • 123

      Imdad et al. 2017:

      • "At longest follow-up, there was a 12% observed reduction in the risk of all-cause mortality for vitamin A compared with control using a fixed-effect model (risk ratio (RR) 0.88, 95% confidence interval (CI) 0.83 to 0.93; high-quality evidence)." Pg 2.
      • "A sensitivity analysis using a random-effects model found a 24% reduction in mortality, essentially the same as our original estimate (RR 0.76, 95% CI 0.69 to 0.83), published previously (Imdad 2010a)." Pg 25.

    • 124

      Cochrane Handbook section 9.5.4: Incorporating heterogeneity into random-effects models:

      • "A fixed-effect meta-analysis provides a result that may be viewed as a ‘typical intervention effect’ from the studies included in the analysis. In order to calculate a confidence interval for a fixed-effect meta-analysis the assumption is made that the true effect of intervention (in both magnitude and direction) is the same value in every study (that is, fixed across studies). This assumption implies that the observed differences among study results are due solely to the play of chance, i.e. that there is no statistical heterogeneity."
      • "When there is heterogeneity that cannot readily be explained, one analytical approach is to incorporate it into a random-effects model. A random-effects meta-analysis model involves an assumption that the effects being estimated in the different studies are not identical, but follow some distribution. The model represents our lack of knowledge about why real, or apparent, intervention effects differ by considering the differences as if they were random. The centre of this distribution describes the average of the effects, while its width describes the degree of heterogeneity. The conventional choice of distribution is a normal distribution. It is difficult to establish the validity of any distributional assumption, and this is a common criticism of random-effects meta-analyses. The importance of the particular assumed shape for this distribution is not known."
      • "If variation in effects (statistical heterogeneity) is believed to be due to clinical diversity, the random-effects pooled estimate should be interpreted differently from the fixed-effect estimate since it relates to a different question. The random-effects estimate and its confidence interval address the question ‘what is the average intervention effect?’ while the fixed-effect estimate and its confidence interval addresses the question ‘what is the best estimate of the intervention effect?’ The answers to these questions coincide either when no heterogeneity is present, or when the distribution of the intervention effects is roughly symmetrical. When the answers do not coincide, the random-effects estimate may not reflect the actual effect in any particular population being studied."

    • 125

      The DEVTA researchers conducted a meta-analysis of DEVTA and eight previous large trials where pre-school children were provided with multiple doses of VAS per year. They found “heterogeneity between DEVTA and subtotal of eight previous trials p = 0.0010.” Awasthi et al 2013, Pg 1475.

    • 126

      See this section of our review of Helen Keller International's vitamin A supplementation program for more details