Mass Distribution of Long-Lasting Insecticide-Treated Nets (LLINs) - 2012 Report

We have published a more recent review of this intervention. See our most recent report on mass distribution of long-lasting insecticide-treated nets.

Published: December 2012

This page discusses the case for mass distribution of long-lasting insecticide-treated nets (LLINs) for protection against malaria. In general, we focus our discussion on work similar to that of the Against Malaria Foundation.

This intervention involves trying to achieve universal ownership of LLINs within a population, giving free new LLINs to the people who do not already have them. Evidence suggests that when large numbers of people use LLINs to protect themselves while sleeping, the burden of malaria can be reduced, resulting in a reduction in child mortality among other benefits.

LLINs cost under $10 each to purchase and distribute (including all costs), and this intervention is generally considered to be among the most cost-effective ways to save lives. We estimate one life saved for just under $2,300 (this does not include other benefits of ITNs).

Previous versions of this page:

• 2011 report on insecticide treated nets
• 2009 report on insecticide treated nets

What is malaria (the disease targeted by LLINs)?

Malaria is one of the leading causes of child deaths in Africa.1 It is transmitted from person to person by infected mosquitoes.2 It involves flu-like symptoms including fever.3 As discussed below, there is evidence connecting malaria with death (particularly in children under 5), anemia, splenomegaly (enlarged spleen), other nutrition-deficiency-related indicators, and low birthweight.

It is also believed that malaria can cause permanent disability (hearing impairment, visual impairment, epilepsy, etc.).4

What is LLIN distribution and how does it target malaria?

An insecticide-treated net (ITN) is a net (usually a bed net), designed to block mosquitoes physically, that has been treated with safe, residual insecticide for the purpose of killing and repelling mosquitoes, which carry malaria.5 A long-lasting insecticide-treated net (LLIN)6 is an ITN designed to remain effective for multiple years without retreatment.7 The World Health Organization recommends that LLINs be distributed for free to achieve universal coverage (one LLIN for every two persons) of those at risk for malaria.8 An LLIN distribution involves surveying people to determine the need for LLINs; delivering LLINs; and promoting the use of LLINs (to read our notes from visiting an ongoing LLIN distribution, see our page on October 2011 site visits).

What is the evidence regarding the general effectiveness of LLIN distributions?

Evidence from small-scale, high-quality studies

The best evidence for the effectiveness of LLIN distributions comes from randomized controlled trials of insecticide-treated net campaigns, which are reviewed in two Cochrane reviews (Lengeler 2004a; Gamble, Ekwaru, and ter Kuile 2006). We have separately found, examined, and summarized the papers reviewed in Lengeler 2004a.9 The studies are mostly short-term, examining insecticide-treated nets (not necessarily LLINs) over a period of 6 months to 2 years.

Lengeler's (2004a) meta-analysis, examining 22 studies,10 found:

• Mortality (5 studies): a statistically significant impact on all-cause mortality in children under 5, summarized as "5.53 deaths averted per 1000 children protected per year" with no clear dependence on one measure of the regional malaria transmission dynamics.11 Two studies attempted to examine malaria-specific mortality and found smaller or similar-sized effects, which the review author attributes to the difficulty of attributing mortality to malaria.12

  Note that the above figure ("5.53 deaths averted per 1000 children protected per year") is based on people who could be covered by the ITNs distributed, not on people who are confirmed to be using ITNs (i.e., the implication is that .00553 lives are saved for every child under five who could be covered by a distributed ITN, not that .00553 lives are saved for every child under five who is confirmed to be using an ITN).13

• Anemia (9 studies): statistically significant impacts on haemoglobin levels, about +1.3 g/L when insecticide-treated nets were compared to untreated nets and +5.7 g/L when insecticide-treated nets were compared to no nets.14 (More on how to interpret these g/L figures at our writeup on deworming.)
• Splenomegaly (enlarged spleen) (5 studies): about 23% protective efficacy (which we presume means that the nets reduced splenomegaly by 23%) when insecticide-treated nets were compared to untreated nets and 30% protective efficacy when insecticide-treated nets were compared to no nets.15
• Effects that were usually statistically significant in individual trials, but were not combined into summary analysis, for severe malarial disease,16 uncomplicated clinical episodes of malaria,17 prevalence of malaria parasites,18 high parasitemia (prevalence of malaria parasites in the blood),19 and nutrition-related measures (weight for age, weight for height, mean mid-upper arm circumference but not height-for-age or unspecified other measures).20

Gamble et al. (2006) focused on the effects on pregnant women; it examined fewer studies than Lengeler (2004a) (6 vs. 24). It connected ITNs with statistically significant reductions in the risk of low birthweight and fetal loss (only in women with four or fewer previous pregnancies) and in placental malaria (overall), but not in anemia/haemoglobin measures.21

We focus here on Lengeler (2004a) because (a) it reviewed more studies; (b) it had a general-population focus and was thus more in line with the programs we seek to evaluate and the effects we seek to assess (particularly the effects on mortality).

What sorts of programs were carried out in small-scale studies?

We have examined and summarized the papers reviewed in Lengeler (2004a),22 seeking to better understand the basic approaches of the programs that led to the results discussed above. We found:

• The details of the programs are often unclear.
• The programs' approaches varied; most consisted of free ITN distribution, or treatment of existing untreated nets with insecticide (effectively turning untreated nets into ITNs), but some involved promotion/marketing of nets and treatment.
• All of the studies examining child mortality involved distribution of ITNs to entire communities, not just to children under five. Only two studies distributed ITNs specifically to children under five.
• In most studies, coverage of ITNs was very low prior to the program (though there was often high coverage of untreated nets), and substantially higher afterward.
• Many studies report intensive measures to ensure that people used their ITNs consistently and properly - measures well beyond what we would expect to be feasible in a larger-scale distribution (and well beyond the measures taken by the Against Malaria Foundation). Sample quotes:23

  • "Each village was visited daily by a supervisor who checked the dilution of the permethrin and the progress of the installation."
  • "Care was taken to place the nets over all beds in each selected house"
  • "Mothers of the children in the study cohort were reminded weekly how to use the net. Nets which became torn or damaged were repaired or replaced. A survey was conducted every 4 weeks during the rainy season to determine whether the bed nets had been tucked in and the entry flaps placed correctly."
  • "After distribution, study staff went door-to-door to ensure that nets were hung properly."
  • "At mass meetings of the intervention group males, insecticide dipping procedures and net erection methods were demonstrated...Two months after bed net distribution, the teams revisited the trial families to give further encouragement."
• Usage does not appear to have been near-universal. Most studies report usage rates in the range of 60-80%, though some report 90%+ usage.

• Only two studies specifically reported both reported usage and actual usage as determined by surprise visits to homes. In one, actual usage was 70-73% while reported usage was 85%; in the other, actual usage was 85% and reported usage was 97%.24

The author of Lengeler (2004a) has stated to us that few if any randomized controlled trials have been done since this review, and that few are likely to be done, since the efficacy of ITNs is well enough established that such studies could face challenges with ethics boards.25

How have larger-scale distributions compared to the programs addressed in these studies?

Funding for malaria control has increased substantially since 2004, making a large number of national scale-ups possible.26

As noted above, the studies discussed above may have differed substantially from what can be expected of an "average" ITN distribution. The review author notes this, stating, "the bulk of data in this review describe impact under ideal trial conditions (efficacy) rather than impact under large-scale programme conditions (effectiveness). While the difference between efficacy and effectiveness is likely to be small for certain medical interventions (such as vaccination or surgery), it can potentially be large for preventive interventions such as ITNs."27

In order to get a sense for how the large-scale performance of LLINs has compared with the promise of the smaller-scale studies discussed above, we have asked the following key questions:

1. How have officially delivered LLINs matched up with LLINs confirmed to be in use by actual households?
2. What has been the general pattern of LLIN usage, i.e., have people used their LLINs?
3. What has been the connection between LLIN usage and drops in malaria deaths and morbidity?

We feel that malaria scholars have used reasonably credible data to provide helpful answers to #1 and #2. We feel that #3 is substantially harder to answer due to issues with malaria data and the difficulty of isolating the impact of LLINs; it seems likely to us that malaria has generally been on the decline since the increase in LLIN distribution began (and that it was not before the increase began), but we do not have enough information to be highly confident that this is the case, or to confidently attribute the change to LLIN distribution as opposed to other control measures. Details follow.

From LLIN distribution to LLIN ownership

It appears to us that malaria control programs commonly use an "8%-20%-50%" model to estimate the percentage of distributed LLINs that remain in the field years later: they assume that 8% of LLINs distributed 0-12 months ago are no longer in the field (whether because of loss in the process of delivery, falling into disrepair, being given away, etc.) The very limited evidence we have seen on this topic appears consistent to us with the idea that this model is the best available. Further discussion of this topic is available at another page, for the sake of brevity.

From LLIN ownership to LLIN usage

The 2010 World Malaria Report tabulates results from national population surveys and compares (a) the percentage of the population that could theoretically be protected by owned ITNs, based on an assumption that each ITN protects two people;28 (b) the percentage of the population that reports using an ITN.29 It finds 2% ownership/0% use for Swaziland, apparent 100% use for Rwanda, and just over 100% use for Democratic Republic of the Congo and Madagascar (this could be a function of more than 2 people covered by each ITN in some cases, combined with possible over-reporting of usage in surveys). Putting these countries aside, the 8 remaining countries have 40%-85% apparent usage using this model (all but one have 69%+ apparent usage). Using an assumption of 1.8 people covered per ITN, rather than 2 (as we do in our cost-effectiveness analysis below),30 would result in 44-94% estimated usage for these countries (77%+ for 7 of 8). The report displays charts that show strong relationships between the two figures (apparent ITN ownership and apparent ITN use).31

It is possible that survey data overstates usage; in our discussion of small-scale studies above, we cite one study where actual usage (as assessed by spot visits to homes) was 70-73% while reported usage was 85%. Applying this ratio to the reported figures assuming 1.8 people covered per net generates a range (for the seven "mid-range" countries discussed above32 ) of 63%-81% usage,33 similar to the range reported in small-scaled studies.34 These figures imply usage of ITNs that is, very broadly speaking, as good as what was found in the small-scale studies discussed above, though this data and our analysis of it could easily be unreliable.35

From LLIN usage to reduced malaria burden

Funding for malaria control has increased substantially since 2004, making a large number of national scale-ups possible. Accordingly, we have looked into the question of whether the impact of LLINs on the burden of malaria can be directly seen in available data (outside the context of intensive small-scale studies). Our discussion is on a separate page, for brevity.

In brief: data and studies appear to show some cases of apparent malaria control success, and also seem to indicate that the overall burden of malaria in Africa is more likely to be falling than rising. However, in most cases it is difficult to link changes in the burden of malaria to particular malaria control measures, or to malaria control in general; and the data remains quite limited and incomplete, such that we cannot confidently say that the burden of malaria has been falling on average. We can imagine that a malaria scholar, with more context than we have on the strengths and weaknesses of different data sets and the histories of malaria control in different areas, could have a higher degree of confidence in the idea that malaria control (and LLINs in particular) has contributed to major declines in the burden of malaria.

Possible developmental effects

As noted above, ITN distribution appears to have benefits other than reduced mortality, such as reduced anemia (an effect size larger than what we have seen for deworming). In addition, Bleakley 2010 makes a case that reducing the burden of malaria may have a lasting impact on children's development, and thus on their ability to be productive and successful throughout life. This is a similar paper, with similar results, to one of the two major pieces of evidence for the developmental impact of deworming.

Bleakley 2010 analyzes a number of attempts to eliminate malaria in the Americas during the early-to-mid 20th century, focusing on the U.S., Brazil, Columbia, and Mexico. He concludes:

There are good reasons to be cautious in using this study as evidence relevant to ITN distribution:

• The programs studied were very different from LLIN distribution campaigns: they focused on indoor residual spraying and other interventions, and did not involve bednets.37
• The context was very different from that of modern-day ITN distributions: the campaigns took place in the U.S.A. of the 1920s and Latin America of the 1950s, where the impact of malaria could have been very different from the consequences in the modern-day developing world (and the infections themselves may have been quite different as well).
• Bleakley 2010 is not an experimental study, but a retrospective one. That is, rather than setting out to answer a question by collecting new data, the author collected and analyzed a large amount of pre-existing data, raising strong possibilities for publication bias. We believe it is unlikely that this paper would have been published in a major economics journal if it had simply concluded that there was no strong evidence for major benefits of malaria eradication. We are generally very hesitant to use papers of this nature in our work. This worry is enhanced by the use of multiple different outcomes and measures of malaria prevalence across the different countries studied, raising the possibility that the author could have selected those specifications that make the outcomes most interesting (in this case, positive for malaria eradication).

That said, we believe the paper merits some weight on the question of developmental benefits, because:

• It has a plausible strategy for separating the effects of malaria infection from effects of other things (such as poverty) that may correlate with malaria infection. Specifically, it exploits the fact that the eradication campaigns caused a relatively rapid drop in malaria infection rates (which was plausibly exogenous because of technological advancement in the creation of DDT) and that areas with higher initial malaria prevalence saw greater falls in malaria prevalence.38 Thus, it seems possible that a connection between the fall in malaria prevalence and positive life impacts - coinciding with the timing of the campaign - could be attributed specifically to malaria eradication, and not to other factors.
• It uses graphs to illustrate a relationship between malaria prevalence and later-in-life income that, while varying across countries, shows that the correlation between baseline malaria rates and adult income was negative and fairly constant, then turned into "zero effect" (when adjusted using a set of controls), coinciding well with the timing of the eradication campaign. It seems difficult to explain this pattern except by attributing the change to the drop in malaria prevalence, though it is worth noting that the relationship in Colombia becomes positive, which is unexpected.39
• It addresses multiple alternate possible explanations for its observations, including a fairly robust set of controls, though they vary by country (because there is no consistent set of cross-country and cross-time data).40
• It covers a number of very large-scale campaigns. While randomized controlled trials allow for a cleaner connection between a program and its effects, a large-scale study like this seems likely to be less dependent on idiosyncratic aspects of a particular mini-program designed to be studied.

Bleakley 2010 also finds effects on school enrollment that vary by country.41

Possible negative/offsetting impact

• Possible development of insecticide resistance. The use of LLINs may cause insecticide resistance to emerge among the local population. We discuss this issue on another page for brevity. In brief, we feel that resistance is a very serious concern, and would like to see substantially more data available to assess it. That said, it appears to us that the malaria control community has been devoting at least some attention and investigation to this issue for a long time, has developed a reasonable knowledge base (if one that has plenty of room to grow), and still recommends the use of LLINs regardless of the resistance situation. There is strong evidence that LLINs reduce malaria and save lives and only preliminary/suggestive/mixed evidence that insecticide resistance may reduce their impact.
• Possible postponement of human immunity. It has been argued that LLINs, by protecting children from mosquitoes, may not only reduce the short-term burden of malaria, but may also reduce opportunities for humans to acquire immunity, making them more susceptible to malaria over the long term.42 A 2010 paper claims that this question has been "answered, with a clear indication that even seven years after initial exposure of infants to ITN, no increased mortality could be observed"43 and cites three sources: Binka et al. 2002, Diallo et al. 2004, and Lindblade et al. 2004. All of these appear to be follow-ups on the small-scale, high-quality studies that originally established the impact of ITNs on child mortality, and all appear (from their abstracts; we have not vetted the papers further) to find that reductions in child mortality were sustained in the several years after the initial study.44 These follow-ups account for 3 of the 5 studies that originally established an impact of LLINs on child mortality. We have emailed the authors of all five studies to ask whether there have been any follow-ups beyond the latest we're aware of; in 4 of 5 cases, it's been confirmed that no more follow-ups were done, and in the fifth case we are still waiting to hear back.
• Undermining private markets. It is possible that giving away LLINs for free causes people to systematically expect that they will continue to receive LLINs for free, and thus causes people to be unwilling to pay for them. (More at a 2012 blog post.)
• Allergic reactions. We have heard anecdotes of minor allergic reactions (e.g., itching skin) to LLINs. We have located no mention of this issue in the Cochrane reviews discussed above and have located very little information on it in general. We did discuss it with program staff during our visit to an ongoing LLIN distribution (we were told that this issue affects few people and can be avoided by letting an LLIN hang for a few days before use). We believe this to be a very minor concern.
• Equity concerns. It is possible that LLIN distribution may cause conflicts if it is not perceived as fair and equitable. We have seen very little evidence on whether this is an issue. We feel that the Against Malaria Foundation takes appropriate steps to mitigate this concern.

Two key issues around LLIN distributions

Targeted vs. universal coverage

It appears that there has been a shift in emphasis over the last several years from targeted coverage (aiming primarily to cover children under five and pregnant women with ITNs) to universal coverage (aiming to protect everyone in a community with ITNs).

The 2004 Cochrane review discussing the impact of ITNs on under-5 mortality (discussed above) appears to advocate primarily for coverage of children under five,45 and as of 2006 this seemed to be the focus of the Roll Back Malaria Partnership, the United Nations Millennium Development Goals, and the US President's Malaria Initiative as well.46 However, in 2007 the World Health Organization issued a recommendation for universal coverage that appears to have been an explicit change in position.47

We did a brief investigation of the arguments for targeted vs. universal coverage, because we wondered whether the Against Malaria Foundation's focus on universal coverage was justified. It seemed to us that if the main benefit of ITNs is in reducing mortality for children under five, then targeting children under five could achieve most of the benefits of universal coverage, for a fraction of the cost. On the other hand, there is a potential argument for universal coverage based on the possibility that universal coverage will be more likely to have community-level effects.48

Our investigation consisted of:

• Speaking with Christian Lengeler, the author of the Cochrane review of the evidence regarding ITN distribution discussed above. (This Cochrane review appears to us to be the most widely cited evidence of the efficacy of ITNs.49 ) We have published our notes from this conversation.50
• Speaking with two scholars that Professor Lengeler referred us to on this question, Dave Smith and Thomas Smith. We have published notes from each of these conversations.51
• Reviewing literature that these scholars referred us to.

Our conclusions are:

• The evidence for the efficacy of ITNs in the Cochrane review is based on studies of universal coverage programs, not targeted programs. In particular, all five studies relevant to the impact of ITNs on mortality involved distribution of ITNs to the community at large, not targeted coverage (see above). (We confirmed this in our examination of the individual studies cited in the Cochrane review.52 ) Thus, there is little basis from the Cochrane review for determining how the impact of ITNs divides between individual-level effects (protection of the person sleeping under the net, due to blockage of mosquitoes) and community-level effects (protection of everyone in communities where ITN coverage is high, due to reduction in the number of infected mosquitoes, caused either by mosquitoes' being killed by insecticide or by mosquitoes' becoming exhausted when they have trouble finding a host).
• The people we spoke to all believe that the community-level effect of ITNs is likely to be a significant component of their effect, though none believe that this effect has been conclusively demonstrated or well quantified.53
• There is some empirical evidence suggesting that the community-level impact of ITNs is significant.
  • We reviewed one study, Hawley et al. 2003, which examined data from a randomized study of ITNs and compared (a) households not selected to receive ITNs but living near other households that had been selected to receive ITNs to (b) households not selected to receive ITNs living far from other households that had been selected to receive ITNs.54 Non-ITN households were divided into quartiles based on proximity to ITN households, and statistically significant trends were observed for child mortality, hemoglobin level and percentage anemic (e.g., non-ITN households showed more favorable performance on these measures as they got closer to ITN households), though the trends were not significant for two other measures of malaria burden.55 For the set of non-ITN households closest to ITN households, malaria outcomes were similar to those for ITN households themselves.56 We find this study credible, although we note that this was a separate analysis of data from a trial of ITN efficacy, so it may be susceptible to publication bias.
  • This study cites eight other studies arguing for significant community-level effects on malaria transmission and one study arguing against this idea. Of these nine, only two (both arguing for community-level effects) examine measures of malaria morbidity/mortality as opposed to simply transmission.57 We have not examined these studies.
  • Counter-evidence comes from the Cochrane review focused on pregnant women (which is not the main one we have discussed):
    
    The most recent trial from western Kenya by Njagi et al. is informative in this respect, as it is the only trial that compared the effects of ITNs versus no nets using simple randomisation by individual in an area with low ITN coverage (little or no mass effect) [24,29]. This trial and the community-randomised trial by ter Kuile et al. [27] were conducted simultaneously in contiguous areas with similar malaria transmission at baseline, and similar socioeconomic and educational status and ethnicity of the trial population. The effect estimates were similar between the two trials (in women not randomised to IPTp-SP), suggesting that ITNs may work equally well when provided to individuals as part of antenatal care in the second trimester or when provided to entire communities.58

The scholars we spoke with also pointed us to papers attempting to model malaria transmission mathematically.

• One such paper, Killeen et al. (2007), that we were pointed to by two of the scholars we spoke to and has been described as a particularly influential paper,59 uses a relatively detailed model of mosquito behavior, which we have not independently vetted and assessed, in order to conclude that "high (80% use) but exclusively targeted coverage of young children and pregnant women (representing <20% of the population) will deliver limited protection and equity for these vulnerable groups. In contrast, relatively modest coverage (35%–65% use, with this threshold depending on ecological scenario and net quality) of all adults and children, rather than just vulnerable groups, can achieve equitable community-wide benefits equivalent to or greater than personal protection."60
• Smith et al. (2009) proposes a model for the relationship between ITN coverage and effective protection for the community. This model implies that each incremental increase in ITN coverage is more valuable to the community than the previous increase (e.g., going from 60% to 70% coverage would be more valuable than going from 50% to 60% coverage, which in turn would be more valuable than going from 40% to 50% coverage).61

We asked one scholar whether there were any prominent papers in this category making the opposite argument (that community-level effects are likely to be insignificant) and he replied that he did not know of any.62

We do not find the evidence for community-level impact to be conclusive, but we believe that the best interpretation of the available evidence suggests at least some community-level impacts, consistent with the consensus of the scholars we spoke to and the World Health Organization.

Free vs. cost-recovering distributions

There has been some debate about whether ITNs should be sold or given freely, with some arguing that selling them (even for highly subsidized prices) may improve the likelihood that they get to people who will use them. We believe that the weight of the (limited) available evidence supports giving out ITNs rather than selling them. Evidence implies that charging a fee has significantly reduced demand for the product, without leading to corresponding increases in utilization rates (and has not significantly impacted the costs of the program).

• One high-quality study evaluated the impact of user fees on net purchase and on utilization of the net. A program distributing nets at prenatal clinics in Kenya found that increased prices (from $0 to $0.75, the price at which they were sold) reduced demand by approximately 75%,63 but were not associated with higher rates of utilization.64
• A review of high-quality studies on this general issue cites two other studies. One found that charging for deworming drugs significantly reduced demand while raising little revenue and failing to improve the targeting of recipients65 ; another found that charging for water disinfectant "led to a rapid drop-off in take-up, with no evidence of increased targeting to the most vulnerable" while slightly increasing utilization rates.66
• The study on The Gambia (discussed above) reports that individuals given free nets in year one, who were then asked to pay for insecticide to retreat them in year two, saw a significant reduction in coverage and a rise in child mortality.67
• The study on Kenya's program (discussed above) reports that Kenya began selling nets at subsidized prices starting in 2002.68 Net coverage rates remained extremely low, at 25%,69 so in 2004, Kenya sold more heavily subsidized nets.70 In 2006, the program began distributing nets for free,71 raising net coverage to 79% between 2006 and 2007.72
• As discussed above, there is a case that ITNs provide community-level protective benefits, so increasing the level of coverage in a community could be beneficial to everyone, not just ITN users. This gives some additional reason for giving out ITNs for free.

How cost-effective is LLIN distribution?

We provide a spreadsheet73 that estimates the (a) cost per life saved, (b) cost per person protected per year, and (c) cost per child under 14 protected per year. Cells highlighted in green represent particularly debatable and/or variable parameters; readers can change these (or any other figures) to see how the outcomes are affected.

Our analysis is relatively simplified and unlikely to capture all of the key issues. Some key assumptions and choices made in our model:

• We build the model on the Cochrane review's, conclusion that each effective year of protection for children under five results in 0.00553 lives saved. Note that this figure is based on people who could be covered by the ITNs distributed, not on people who are confirmed to be using ITNs (i.e., the review discussed above implies that .00553 lives are saved for every child under five who could be covered by a distributed ITN, not that .00553 lives are saved for every child under five who is confirmed to be using an ITN).74 Our use of this figure entails a couple of implicit simplifications:
  • We do not model variations in the transmission setting (prevalence and behavior of mosquitoes, presence of cattle, etc.). Note that the review we are using states that its ".00553 lives saved per child protected" figure does not appear sensitive to one measure of transmission dynamics.75
    
  • Our use of this figure also implies that each person-year of protection provides an equivalent protective effect - e.g., going from 80% to 81% coverage has the same effect as going from 0% to 1% coverage. The information we have (discussion above) suggests that ITNs are actually more effective at higher levels of coverage (because part of their effect is on community-level transmission of malaria); this in turn implies that distributing ITNs in an area where coverage is already reasonably high should be more cost-effective than distributing ITNs in an area of low prior coverage, as in the small-scale studies. However, we are not confident in this dynamic and do not model it.
• We include a simple adjustment to account for the fact that child mortality rates are lower today than they were at the time of the studies on ITNs.76 This adjustment assumes that ITNs avert the same proportion of under-5 deaths that they averted at the time of the studies. This could be incorrect.
  • The deaths averted by ITNs may be the same deaths that could be averted by, for example, vitamin A supplementation (as noted above, many of the deaths averted by ITNs are not specifically attributable to malaria). So perhaps other improvements in general health are independently averting all the deaths that ITNs could avert in their absence; under this model it’s possible that ITNs don’t avert any deaths at all.
  • On the flipside, there may be increasing returns to improved general health: perhaps there are children who previously (at the time of the studies) would have been in such poor health that they would have died even with ITNs, but now can have their deaths averted by ITNs.
• LLINs are assumed to last 2.22 years on average, consistent with the decay model discussed above. We assume that this decay model accounts for a substantial amount of the extent to which LLINs are lost, not delivered, or discarded. We provide a less optimistic estimate where LLINs are assumed to last 1 year on average (note that the evidence on ITN effectiveness is generally limited to a time frame of one year); we provide a more optimistic estimate where LLINs are assumed to last 3 years on average (this could be warranted if Against Malaria Foundation distributions result in fewer lost/undelivered/discarded LLINs).

• We assume that LLIN usage is in line with what was observed in the small-scale studies discussed above. We believe this is an appropriate assumption based on the available data, as discussed above. Those who believe that effective usage is likely to be lower can model this assumption using the "Wastage" cell in our sheet (which allows one to specify a percentage of ITNs that are lost, misused, or otherwise paid for but unimpactful, beyond the already-implicit assumptions in the model). Our model may not sufficiently adjust for this sort of additional "wastage," if (as discussed above) ITNs are more effective at higher levels of prior coverage.
• Rather than explicitly model usage, we base our calculation only on coverage, using a "lives saved per person-year of protection" figure that is itself based on coverage, not usage. This is why, when calculating pre-existing coverage of the population, we take reported usage figures and divide them by 70% (this assumes that coverage is greater than usage such that usage = 70% * coverage).
• We model community-level effects crudely. We use a single figure (referred to in this paragraph as X) for the "percentage of the effects of ITNs that comes from community-level effects." If coverage of children under 5 is C1, and coverage of the population as a whole is C2, then we model the "effective coverage" of children under 5 as (X*C2+(1-X)*C1). In other words, when 10% of the ITN effect is presumed to be from community-level effects, this means that the "effective coverage" of children under 5 comes from taking a weighted average of the under-5 ownership and the community ownership, with 90% weight on the former and 10% weight on the latter. Using this model, and 50% for X, would have an implication similar to that of the more sophisticated model we discuss above.
• Our estimate does not include the possible concerns discussed above around negative/offsetting impact (delaying the development of immunity, causing allergic reactions, etc.) and does not attempt to account for the possibility that mosquitoes have become (or will become) resistant to the insecticides used. We believe that most of the concerns noted above are minor, but the issue of insecticide resistance may be major.
• Our estimate also does not include the many possible benefits of malaria control aside from saving the lives of children under 5. These may include developmental effects, saving the lives of people over the age of 5,77 benefits to the health system of a reduced malaria burden, etc.

Two more general notes on the limitations to cost-effectiveness analysis such as this:

• Such analysis will usually draw its main estimates of program effects from small-scale, high-quality studies (our analysis here is no exception). These studies may be unrepresentative of real-world conditions, on account of being carried out in optimal locations and with unusual amounts of funding and intensity.
• We believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

Our best-guess estimate comes out to about $2,500 per life saved using the total cost per net and about $2,300 per life saved using the marginal cost per net (see details on total versus marginal costs per net in our our updated AMF review). This does not include other benefits of ITNs. We also show the number of children under 14 who are protected for each life saved, since protection may yield non-mortality-related benefits comparable to those of deworming, and we provide some rows that account for this additional benefit in order to allow more intuitive comparisons between the two interventions.

We also provide reference points for conversions to the cost per disability-adjusted life-year (DALY), a common metric in public health. We have found this metric to add more confusion than clarification,78 but we provide this information for those who find it helpful. Note that our cost per DALY figures are based only on lives saved and do not account for other benefits.

See our spreadsheet analysis

Is there room for more funding in LLIN distribution?

As discussed above, we have what we believe to be fairly comprehensive data on the number of LLINs distributed to each country over the last several years. By combining this with different versions of the decay function discussed above and different estimates of the population at risk for malaria, we have estimated how many LLINs would still be needed to achieve 100% (theoretical) coverage, for each country as of the end of 2011 and 2012.79

Regardless of which version of the calculation we use - even when assuming that LLINs remain 100% in use for 3 years after distribution and when using the lowest "population at risk" figures we have - we see substantial gaps in LLIN coverage. Six of the seven countries emphasized by AMF in its discussion of room for more funding - have a gap of ~1.1 million LLINs or more, thus each of these gaps would take $5.5+ million (using Against Malaria Foundation costs per LLIN distributed) to close. The exception is Malawi, where the data used in the model indicates that the gap has been more than filled (AMF believes gaps remain in at least some areas of the country80 ). When examining the full list of countries listed by AMF, adding up the smallest modeled LLIN gap for each country (excluding the "negative gap" in Malawi) results in a total LLIN gap of over 63 million, or over 12 million excluding the estimated 51 million LLINs needed in Nigeria (thus over $60 million would be needed to close the non-Nigeria gap).

Our coverage analysis81 extends back to 2006 (using delivery data since 2004), showing a picture of fairly persistent LLIN gaps.

View our coverage analysis (XLS)

Sources

Uncited supplementary materials

• 1

  "In Africa, malaria accounts for an estimated 25% of all childhood mortality below age five, excluding neonatal mortality (WHO 2003)." Lengeler 2004a, Pg 2.

• 2

  "Human infection begins when the malaria vector, a female anopheline mosquito, inoculates plasmodial sporozoites from its salivary gland into humans during a blood meal. The sporozoites mature in the liver and are released into the bloodstream as merozoites. These invade red blood cells, causing malaria fevers. Some forms of the parasites (gametocytes) are ingested by anopheline mosquitoes during feeding and develop into sporozoites, restarting the cycle." Jamison et al. 2006, Pg 413.

• 3

  "Most malaria infections cause symptoms like the flu, such as a high fever, chills, and muscle pain. Symptoms tend to come and go in cycles. One type of malaria may cause more serious problems, such as damage to the heart, lungs, kidneys, or brain. It can even be deadly." WebMD. Malaria: Topic overview

• 4

  See Jamison et al. 2006, Pg 416, Table 21.3 for estimates of cases of hearing impairment, visual impairment, epilepsy, etc. caused by malaria.

• 5

  "Using mosquito nets as a protection against nuisance insects was practiced in historical times (Lindsay 1988). During World War II, Russian, German, and US armies treated bed nets and com- bat fatigues with residual insecticide to protect soldiers against vector-borne diseases (mainly malaria and leishmaniasis) (Curtis 1991). In the late 1970s, entomologists started using synthetic pyrethroids: their high insecticidal activity and low mammalian toxicity made them ideal for this purpose.

  In the 1980s, studies of ITNs showed that pyrethroids were safe and that ITNs had an impact on various measures of mosquito biting (such as the proportion of mosquitoes successfully feed- ing on humans and the number of times a mosquito bit humans in one night). These studies showed that pyrethroids worked by both repelling and killing mosquitoes. In addition, researchers determined optimal doses of various insecticides with different materials (Curtis 1991; Curtis 1992a; Curtis 1996; Lines 1996; Rozendaal 1989a). The cost-effectiveness of ITNs has also been demonstrated (Goodman 1999; Hanson 2003)." Lengeler 2004a, Pg 2.

• 6

  See "Abbreviations" - @World Health Organization. World malaria report (2010)@ Pg ix.

• 7
  • "Re-treating ITNs semiannually
    or just before the annual peak in transmission is essential for
    effective vector control and is proving a major logistical and
    financial challenge. Fortunately, new types of nets with a long-
    lasting insecticidal property are now available, and re-
    treatment will soon cease to be an issue." Jamison et al. 2006, Pg 421.
  • "Sumitomo Chemical's Olyset net uses the latest research and technology to achieve a breakthrough in the global fight against malaria. Permethrin is incorporated inside the Olyset fibres to create a bed net guaranteed to last at least five years.* It is tear-proof, wash-proof, and never requires treatment.

    The pyrethroid insecticide permethrin is a synthetic molecule similar to natural pyrethrin, which comes from a species of chrysanthemum. Because it poses minimal toxic risk to humans, Olyset is particularly valuable where babies and small children are concerned." Sumitomo Chemical. Olyset net

• 8

  "WHO recommendations for vector control are the following:

  1. Because high coverage rates are needed to realize the full potential of insecticide-treated nets (ITNs) and IRS, WHO recommends that all people at risk in areas targeted for malaria prevention should be covered with LLINs, i.e “universal coverage” (9,10). It is currently proposed that one LLIN should be distributed for every two persons. This approach may require refinement for implementation at household level: for example, one option is to distribute to each household one LLIN for every two members of the household, rounding up in households with an odd number of members.
  2. LLINs should be either provided free of charge or highly subsidized. Cost should not be a barrier to making them available to all people at risk of malaria, especially those at greatest risk such as young children and pregnant women.
  3. Universal coverage with LLINs is best achieved and maintained by a combination of delivery systems: mass distribution campaigns can achieve rapid initial coverage, but need to be supplemented by routine delivery to pregnant women through antenatal services and to infants at immunization clinics.
  4. In order to be protected, households must not only own LLINs but also use them. Behaviour change interventions including information, education, communication (IEC) campaigns and post-distribution “hang-up campaigns” are strongly recommended.
  5. Only LLINs recommended by the WHO Pesticide Evaluation Scheme (WHOPES) should be procured by national malaria control programmes and partners for malaria control. These nets are designed to maintain their biological efficacy against vector mosquitoes for at least three years in the field under recommended conditions of use, obviating the need for regular insecticide treatment)." World Health Organization. World Malaria Report (2010) Pg 4.

• 9

  GiveWell. Summary of ITN RCTs

• 10

  "The remaining 22 trials, including 1 trial that is currently unpublished, met the inclusion criteria for this review. These trials are described below." Lengeler 2004a, Pg 6.

• 11

  "The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to 7.67; Analysis 1.2). I performed a regression analysis of the natural logarithm of the rate difference on the entomological inoculation rate and could not find a trend (r^2 = 0.52, F = 3.2 on 1,3 degrees of freedom, P = 0.2). In contrast to protective efficacies, the risk differences seemed to have a tendency towards a higher effect with a higher entomological inoculation rate. This apparent paradox is because the baseline mortality rates are higher in areas with high entomological inoculation rates." Lengeler 2004a, Pg 9. Note on the entomological inoculation rate: “The intensity of malaria transmission measured by the frequency with which people living in an area are bitten by anopheline mosquitoes carrying sporozoites. This is often expressed as the annual entomological inoculation rate (EIR), which is the number of inoculations of malaria parasites received by one person in one year.” World Health Organization. Guidelines for the Treatment of Malaria, 2nd ed. (2010) pg vii.

• 12

  "The impact of ITNs on malaria-specific death rates was looked at only briefly because of the problems using verbal autopsies in determining malaria deaths. In the two trials for which the data were available, the percentage reduction in malaria-specific mortality was similar or smaller than the percentage reduction in all-cause mortality: 14% (versus 23%) for Gambia (D'Alessandro et al. 1995), and 22% (versus 18%) for Ghana (Binka et al. 2002). One interpretation is that malaria-specific death rates were not reflecting the true impact of ITNs on mortality (since a much higher specific impact would have been expected)." Lengeler 2004a, Pg 9.
  

• 13

  "The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to 7.67; Analysis 1.2)." Lengeler 2004a, Pg 8. We have confirmed with the author that this figure is based on an "intention to treat" analysis, e.g., "protected" refers to children in the treatment group, not to children who were confirmed to own or use ITNs.

• 14
  • "Anaemia: expressed in mean packed cell volume (PCV); it is equivalent to the percentage haematocrit. Results given in g/decilitre were converted with a standard factor of 3:1, that is, 1 g/decilitre equals 3%PCV." Lengeler 2004a, Pg 4.
  • " The nine trials that measured anaemia were conducted in areas of stable malaria; six trials compared treated to untreated nets (Appendix 13), and three trials compared treated nets to untreated nets (Appendix 14). Overall, the packed cell volume of children in the ITN group was higher by 1.7 absolute packed cell volume per cent compared to children not using nets. When the control group used untreated nets, the difference was 0.4 absolute packed cell volume per cent." Lengeler 2004a, Pg 9.
  • 0.4 packed cell volume converts to (0.4/3 * 10) = 1.3 g/L using the conversion factor provided (and multiplying by 10 to convert from g/dL to g/L). Similarly, 1.7 packed cell volume converts to (1.7 / 3 * 10) = 5.7 g/L.

• 15

  "Four out of the five trials that measured splenomegaly were carried out in areas with stable malaria (Appendix 15 and Appendix 16). Because the exception was one trial carried out in Thailand whose weight is very small (only 2.6% in the relevant comparison) (Thailand (Luxemberger)), I did not carry out a subgroup analysis. Splenomegaly was significantly reduced for both types of controls: there is a 30% protective efficacy when controls were not using nets, and a 23% protective efficacy when the control group used untreated nets." Lengeler 2004a, Pg 9.

• 16

  "Only one trial examined severe malarial disease as an outcome Kenya (Nevill). The trial used passive and hospital-based case as- certainment, and observed a 45% (cluster-adjusted 95% CI 20 to 63) reduction in the frequency of severe malaria episodes follow- ing the introduction of ITNs (Appendix 4)." Lengeler 2004a, Pg 8.

• 17

  "Uncomplicated clinical episodes:

  The trial results are available in Appendix 5 for no nets controls and in Appendix 6 for untreated nets controls. A summary of the main findings for protective efficacies is available in Appendix 7; confidence intervals were not calculated as this analysis includes both cluster and individually randomized controlled trials. No risk or rate differences were calculated because the denominators were not uniform and the sensitivity of the reporting systems of the different trials is likely to have varied considerably. Three findings can be highlighted.

  • The effect of ITNs on uncomplicated clinical episodes of malaria is shown by large effect estimates in all trials. Overall, the reduction in clinical episodes was around 50% for all subgroups (stable and unstable malaria; no nets and untreated nets) and for both P. falciparum and P. vivax.
  • The protective efficacy is higher (at least 11% for P. falciparum) when the control group had no nets. This was expected and it was the reason to create two separate comparisons. In areas with stable malaria (entomo- logical inoculation rate > 1) the differences in protective efficacies against uncomplicated malaria was 11% (50% no nets versus 39% untreated nets). In areas with unstable malaria (entomological inoculation rate < 1), the differences were bigger: 23% (62% no nets versus 39% untreated nets) for P. falciparum, and 41% (52% no nets versus 11% untreated nets) for P. vivax.
  • In areas of unstable malaria (entomological inoculation rate < 1), the impact against P. falciparum episodes seemed to be higher than the impact against P. vivax episodes."

  Lengeler 2004a, Pg 9.

• 18

  "Parasite prevalence:

  The results are available in Appendix 8 for no nets and in Appendix 9 for untreated nets controls. The results for both groups are summarized in Appendix 10; confidence intervals were not calculated as this analysis includes both cluster and individually randomized controlled trials. Two points can be highlighted from these results.

  • In areas of stable malaria, impact on prevalence of infection (measured through cross-sectional surveys) was small: 13% reduction when the control group did not have any nets and 10% reduction when the control group had untreated nets.
  • In areas with unstable malaria, the results are of limited value because there was only a single trial in each subgroup (treated versus no nets; and treated versus un- treated nets).

  Lengeler 2004a, Pg 9.

• 19

  "High parasitemia:

  The results are shown in Appendix 11 for no nets and Appendix 12 for untreated nets controls. This outcome was only assessed for trials in areas of stable malaria, where parasitaemia does not necessarily lead to a clinical episode, and where parasitaemia cut- offs are useful to define disease episodes. Five trials measured this outcome: four used 5000 trophozoites/ml as the cut-off, while the fifth trial used an age-specific cut-off (Kenya (Phillips-Howard)). The protective efficacy was 29% for the two trials in which the control group did not have nets, and was 20% for the three trials in which controls had untreated nets." Lengeler 2004a, Pg 9.

• 20

  "Three trials carried out with ITNs have demonstrated a positive impact on anthropological measurements in children sleeping under treated nets.

  In Gambia (D'Alessandro et al. 1995), mean z-scores of weight-for-age and weight-for-height were higher in children from treated villages (-1.36 and -0.98, respectively) than in those from untreated villages (-1.46 and -1.13, respectively). The differences were statistically significant after adjustment for area, age, differential bed net use, and gender (P = 0.008 and P = 0.001, respectively). There was no statistically significant difference in me significant an z-scores for height-for-age.

  In the trial carried out in Kenya (Kenya (Nevill)), infants sleeping under ITNs in the intervention areas had statistically significantly higher z-scores for weight-for-age than control infants not under treated nets (analysis of variance allowing for season, gender, and age: F = 21.63, P = 0.03). Mean mid-upper arm circumference z-scores were also statistically significantly higher among infants in the intervention communities (analysis of variance allowing for survey, gender, and age: F = 19.0, P = 0.005) (Snow 1997).

  In Kenya (Kenya (Phillips-Howard)), protected children under two years of age had a statistically significantly better weight-for- age z-score than unprotected children (P < 0.04). No other statistically significant differences were measured for other parameters or other age groups, although all z-score differences between intervention and control groups were in favour of the protected group." Lengeler 2004a, Pgs 9-10.

• 21

  “Six randomized controlled trials were identified, five of which met the inclusion criteria: four trials from sub-Saharan Africa compared ITNs with no nets, and one trial from Asia compared ITNs with untreated nets. Two trials randomized individual women and three trials randomized communities. In Africa, ITNs, compared with no nets, reduced placental malaria in all pregnancies (risk ratio (RR) 0.79, 95% confidence interval (CI) 0.63 to 0.98). They also reduced low birthweight (RR 0.77, 95% CI 0.61 to 0.98) and fetal loss in the first to fourth pregnancy (RR 0.67, 95% CI 0.47 to 0.97), but not in women with more than four previous pregnancies. For anaemia and clinical malaria, results tended to favour ITNs, but the effects were not significant. In Thailand, one trial randomizing individuals to ITNs or untreated nets showed a significant reduction in anaemia and fetal loss in all pregnancies but not for clinical malaria or low birthweight.” Gamble, Ekwaru, and ter Kuile 2006, Pg 1.

• 22

  GiveWell. Summary of ITN RCTs

• 23

  GiveWell. Summary of ITN RCTs

• 24

  GiveWell. Summary of ITN RCTs

• 25

  "To the best of my knowledge there have been no more RCTs with treated nets. There is a very strong consensus that it would not be ethical to do any more. I don't think any committee in the world would grant permission to do such a trial." Christian Lengeler, author of Cochrane Review of insecticide-treated bed nets, phone conversation with GiveWell, November 2, 2011

• 26

  See World Health Organization. World Malaria Report (2010) charts on Pg 13.

• 27

  "The results presented in this review are from randomized controlled trials where the intervention was deployed under highly controlled conditions, leading to high coverage and use rates. The one exception is Gambia (D'Alessandro et al. 1995), which was a randomized evaluation of a national ITN programme in which the intervention deployment was not as good as in the other trials. Therefore, the bulk of data in this review describe impact under ideal trial conditions (efficacy) rather than impact under large-scale programme conditions (effectiveness). While the difference between efficacy and effectiveness is likely to be small for certain medical interventions (such as vaccination or surgery), it can potentially be large for preventive interventions such as ITNs.

  Some of the consequences of moving from a scientific trial towards a large-scale programme is illustrated by the results of the two mortality trials carried out in The Gambia. The first trial was carried out under well-controlled implementation conditions, with a high coverage rate in the target population (Gambia (Alonso)). Unfortunately it was not randomized and hence not included in the present analysis. The second one was the evaluation of a national impregnation programme carried out by primary health care personnel and which faced some operational problems (leading, for example, to a lower than expected insecticide dosage) and a lower coverage rate (around 60%) of the target population (Gambia (D’Alessandro)). The difference of impact between the two studies is important: the first trial achieved a total reduction in mortality of 42%, while the protective efficacy in the second trial was 23%. It is not clear whether the difference in the baseline mortality rate (42.1 versus 24.3 deaths per 1000 in the control group) played a role in this difference of impact." Lengeler 2004a, Pg 10.

• 28

  "In reviewing household surveys that provide the most recent results available on ITN coverage for 27 malaria-endemic countries between 2003 and 2009, it was evident that relatively low propor- tions of households own an ITN (median 16%, lower quartile 5%, upper quartile 45%); only 7 surveys were conducted during the massive expansion of ITN programmes from 2008 to 2010. However, within all surveys, a high proportion of available nets appear to be used (approximately 80%) assuming that one net can cover two people (Fig. 4.6a). Some countries such as Madagascar (2008) and Rwanda (2008) have higher rates of use than others. These results are consistent with previous analyses which suggest that the main constraint to enabling persons at risk of malaria to sleep under an ITN is lack of availability of nets (3). " World Health Organization. World Malaria Report (2010) Pg 23.

• 29

  World Health Organization. World Malaria Report (2010) Table 4.2, Pg 19.

• 30

  The assumption of 1.8 people covered per ITN is based in data from the Against Malaria Foundation’s distribution in Malawi: “In the last pre-distribution survey, there were 26,385 sleeping spaces (47,824 people) in the area and 6,326 useable nets. There were fewer nets in Dedza than in Ntcheu. In total they found that on average 1 net is required to cover 1.8 people (some people sleep alone; others share sleeping spaces).” GiveWell. Notes from site visit with Concern Universal in Malawi (October 2011), Pg 2.

• 31

  World Health Organization. World Malaria Report (2010), Pg 22.

• 32

  Excluding Rwanda, Democratic Republic of the Congo and Madagascar (>100% implied usage) as well as Swaziland (2% coverage, 0% usage) and Namibia (15% coverage, 6% usage).

• 33
  • As stated above, the 7 'mid-range' countries discussed range from 77-94% reported usage.
  • The study discussed has 85% reported usage and 70-73% directly measured usage, for a 82%-86% ratio of directly measured usage to reported usage.
  • Applying the 82%-86% ratio to the 77%-94% reported usage results in a range of 63% (77%*82%) to 81% (94%*86%) implied actual usage.

• 34

  GiveWell. Summary of ITN RCTs

• 35

  It does seem possible that the surveys used for the above analysis are less prone to over-reporting than the surveys done in the small-scale studies, because the latter were likely done in the context of ITN promotion programs whereas the analysis above relies on large-scale surveys covering a large number of topics (full surveys are available via Measure DHS. Malawi DHS 2010). On the other hand, the intensiveness of the interventions in the small-scale studies may have meant more accurate data collection and/or a higher ratio of actual (correct) usage to reported usage.

• 36

  Bleakley 2010.

• 37
  • "The federal government’s large-scale efforts against malaria in the south began with World War I (WWI). In previous wars, a significant portion of the troops were made unfit for service because of disease contracted in or around encampments. The PHS, working now with a strong knowledge base on malaria control and greatly increased funding, undertook drainage and larviciding operations in southern military camps, as well as in surrounding areas. After WWI, the IHB and PHS expanded the demonstration work further. By the mid-1920s, the boards of health of each state, following the IHB/PHS model, had taken up the mantle of the malaria control in all but the most peripheral areas of the region (Williams 1951). The south experienced a drop in malaria mortality of more than 60 percent in the decade of the 1920s." Bleakley 2010, Pgs 5-6.
  • "In the early 1950s, the World Health Organization (WHO) proposed a worldwide campaign to eradicate malaria. While the WHO mostly provided technical assis- tance and moral suasion, substantial funding came from USAID and UNICEF. The nations of Latin America took up this task in the 1950s. While individual nations had formal control of the design and implementation of the programs, their activities were comparatively homogeneous as per the dictates of their international funders. The central component of these programs was the spraying of DDT, principally in the walls of houses. Its purpose was not to kill every mosquito in the land, but rather to interrupt the transmission of malaria for long enough that the existing stock of parasites would die out. After that, the campaigns would go into a maintenance phase in which imported cases of malaria were to be managed medically.
    The Latin American countries analyzed in the present study (Mexico, Colombia, and Brazil) all mounted malaria eradication campaigns, and all saw large declines in malaria prevalence. Panel A of Figure 1 shows malaria cases per capita in Colombia. (Comparable time-series data were not available for the US, Brazil, or Mexico. Nevertheless, there is little doubt that malaria declined in all four countries immediately following these campaigns.) A decline of approximately 80 percent is seen in the graph. Throughout Latin America, the campaign ultimately proved inadequate to the task, and, in many areas, malaria partially resurged two decades later. But in almost all parts of the hemisphere, malaria never returned to its levels from before the application of DDT." Bleakley 2010, Pg 6

• 38

  "How realistic is the assumption that areas with high infection rates benefited more from the eradication campaign? Mortality and morbidity data indicate drops of 50 to 80 percent in the decade after the advent of the eradication efforts. Such a dramatic drop in the region’s average infection rate, barring a drastic reversal in the pattern of malaria incidence across the region, would have had the hypothesized effect of reducing infection rates more in highly infected areas than in areas with moderate infection rates. Data on malaria cases by Colombian departments allow us to examine this directly. The decline in malaria incidence as a function of intensity prior to the eradication campaign is found in panel B of Figure 1. The basic assumption of the present study, that areas where malaria was highly endemic saw a greater drop in infection than areas with low infection rates, is borne out. (Similar results are seen for US and Mexican states. Data for Brazilian states were not available.)
  Finally, the timing of the eradication campaign should induce variation in childhood malaria infection that has a marked pattern across year-of-birth cohorts. The present study considers the effects of childhood malaria infection on later- life outcomes, so it is useful to characterize childhood exposure to an eradication campaign. This is shown in Figure 2. Consider a campaign that starts in year zero and takes effect instantaneously. Cohorts born after this date will be exposed to the campaign for their entire childhood. On the other hand, those cohorts who were already adults in year zero will have no childhood exposure to the campaign, while the “in-between” cohorts will be partially exposed during childhood, as shown in Figure 2. I exploit this timing in two ways. First, in Section III, I compare the “born after” cohorts to the “already adult” cohorts by taking differ- ences across these cohort groups. Second, in Section IV, I use the functional form of childhood exposure in estimation using data for all cohorts. (I discuss some alternative functional forms in Section IV.B.)
  These four factors (the external origin of the campaigns, the quick reduction of malaria that followed, the use of nonmalarious areas for comparison, and the differential incidence of eradication benefits across cohorts) combine to form the research design of the present study." Bleakley 2010, Pgs 8-9.

• 39

  See Bleakley 2010, Figure 4, Pg 26.

• 40

  See Bleakley 2010, Appendix III, Pg 40, for the full list of controls by country.

• 41

  "Mixed results are found for years of education, in contrast with consistently positive effects of malaria eradication on income and literacy. Furthermore, in no country can the change in income be accounted for by the change in years of schooling. These facts are in no way discordant with the economic theory of schooling, which compares returns with opportunity costs. Childhood health plausibly raises both, leaving an ambiguous effect on the optimal time to spend in school. This combination of results, interpreted with simple price-theoretic reasoning regarding the education decision, show that we should be cautious in using changes in time in school as a sufficient statistic by which development and health policies are evaluated." Bleakley 2010, Pg 35.

• 42
  “From 1995 onwards, coinciding with the results from the main African ITN trials, a vigorous debate arose about the possibility that the short-term mortality improvements observed in trials of 1–2 years' duration could be offset by increased mortality at older ages — a "delayed mortality" effect (4, 5). The underlying hypothesis was that immunity to malaria would develop more slowly under reduced transmission, leading to a longer period of susceptibility. No direct evidence was available at the time either to support or to refute this hypothesis.” Lengeler 2004b

• 43

  Kilian, Wijayanandana, and Ssekitoleeko 2010.

• 44
  • " A 17% efficacy in preventing all-cause mortality in children aged 6–59 months was previously reported from a cluster-randomized controlled trial of insecticide-treated mosquito nets (ITNs) carried out in the Kassena-Nankana District of northern Ghana from July 1993–June 1995. A follow-up until the end of 2000 found no indication in any age group of increased mortality in the ITN group after the end of the randomized intervention. These results should further encourage the use of ITNs as a malaria control tool in areas of high endemicity of Plasmodium falciparum." Binka et al. 2002. This appears to be a followup on Binka et al. 1996 (cited in our GiveWell. Summary of ITN RCTs)
  • "OBJECTIVES: To determine the impact of insecticide-treated curtains (ITC) on all-cause child mortality (6-59 months) over a period of six years. To determine whether initial reductions in child mortality following the implementation of ITC are sustained over the longer term or whether 'delayed' mortality occurs … A rural population of ca 100 000 living in an area with high, seasonal Plasmodium falciparum transmission was studied in Burkina Faso. Annual censuses were conducted from 1993 to 2000 to measure child mortality. ITC to cover doors, windows, and eaves were provided to half the population in 1994 with the remainder receiving ITC in 1996. Curtains were re-treated or, if necessary, replaced annually. FINDINGS: Over six years of implementation of ITC, no evidence of the shift in child mortality from younger to older children was observed. Estimates of the reduction in child mortality associated with ITC ranged from 19% to 24%." Diallo et al. 2004, abstract. This appears to be a followup on Habluetzel et al. 1997 (cited in our GiveWell. Summary of ITN RCTs).
  • "A total of 130,000 residents of 221 villages in Asembo and Gem were randomized to receive insecticide-treated bednets at the start of phase 1 (111 villages) or phase 2 (110 villages) … Mortality rates did not differ during 2002 (after up to 6 years of bednet use) between children from former intervention and former control households born during phase 1 (HR, 1.01; 95% CI, 0.86-1.19)." Lindblade et al. 2004, abstract. This appear to be a follow-up on Phillips-Howard et al. 2003 (cited in our GiveWell. Summary of ITN RCTs

• 45

  "An approximate extrapolation to the current population of children under five years of age at risk for malaria in sub-Saharan Africa (14% of approximately 480 million population at risk, or 67 million children) indicates that approximately 370,000 child deaths could be avoided if every child could be protected by an ITN." Lengeler 2004a, Pg 10.

• 46

  "The Roll Back Malaria Partnership, the United Nations Millennium Development Goals, and the US President's Malaria Initiative have set a target of at least 80% use of ITNs by young children and pregnant women (the people most vulnerable to malaria) by 2010." Killeen et al. 2007, Pg 1258. Note that this paper was "received December 22, 2006."

• 47
  • "This Position Statement from the WHO Global Malaria Programme (WHO/GMP) describes a shift in guidance on malaria prevention through the use of insecticide- treated nets (ITNs).

    The WHO/GMP calls upon national malaria control programmes and their partners involved in insecticide-treated net interventions to purchase only long-lasting insecticidal nets (LLINs). LLINs are designed to maintain their biological efficacy against vector mosquitoes for at least three years in the field under recommended conditions of use, obviating the need for regular insecticide treatment.

    In order for their full potential to be realized, LLINs should be deployed as a vector control intervention. WHO/GMP, therefore, recommends full coverage of all people at risk of malaria in areas targeted for malaria prevention with LLINs." World Health Organization. Insecticide-treated mosquito nets: A WHO position statement (2007) Pg 1.

  • "GMP now recommends universal coverage with Long Lasting Insecticidal Mosquito Nets. Wherever any of the earlier documents refers to targeted coverage of risk groups only, please refer to superseding guidance on this matter now provided in the 2007 GMP position statement Insecticide-treated mosquito nets: a position statement [the source quoted directly above]." World Health Organization. Publications on insecticide-treated materials: Guidelines

• 48

  There is a distinction between individual-level effects (protection of the person sleeping under the net, due to blockage of mosquitoes) and community-level effects (protection of everyone in communities where ITN coverage is high, due to reduction in the number of infected mosquitoes, caused either by mosquitoes' being killed by insecticide or by mosquitoes' becoming exhausted when they have trouble finding a host).

• 49
  

  Examples:

  • The World Health Organization position statement discussed immediately previously cites this review as its main source: "On the basis of five community-randomized trials, a Cochrane review concluded that, when full cover- age is achieved, ITNs reduce all-cause child mortality by an average 18% (range 14–29%) in sub- Saharan Africa (5). The general implication of this is that 5.5 lives could be saved per year for every 1000 children under 5 years of age protected. It was also concluded that ITNs reduce clinical episodes of malaria caused by Plasmodium falciparum and P. vivax infections by 50% on average (range 39– 62%), as well as reducing the prevalence of high-density parasitaemia." World Health Organization. Insecticide-treated mosquito nets: A WHO position statement (2007) Pg 3.
  • The Disease Control Priorities Report also cites this review as its main source for ITN efficacy: "We drew estimates of the effectiveness of ITNS from a recent meta-analysis of WHO/TDR-sponsored trials conducted in Sub-Saharan Africa [Lengeler 2004a]." Jamison et al. 2006, Pg 423.

• 50

  Christian Lengeler, author of Cochrane Review of insecticide-treated bed nets, phone conversation with GiveWell, November 2, 2011

• 51
  • Dave Smith, University of Florida Associate Professor of Zoology, phone conversation with GiveWell, November 4, 2011.
  • Thomas Smith, Swiss Tropical Institute Head of the Biostatistics and Computational Sciences Unit, phone conversation with GiveWell, November 8, 2011

• 52

  GiveWell. Summary of ITN RCTs

• 53
  • "When you're moving from protecting only children to protecting everybody, you multiply your costs by a factor of five, but you also get at least twice as much effect and we know that from both modeling and empirical impact assessments … I don't think we will ever really have the reply to that question [about the relative magnitude of community-level effects]. Though one thing you can do is mathematical modeling. There are different groups in the world who are doing modeling for the effect of increasing coverage of ITNs, and they will be able to give you some sort of answer … The two people who occur to me are Thomas Smith at the Swiss Tropical Institute and Dave Smith at the University of Florida's Emerging Pathogens Institute." Christian Lengeler, author of Cochrane Review of insecticide-treated bed nets, phone conversation with GiveWell, November 2, 2011
  • "There are empirical studies that show that there seems to be a community-level impact of ITNs. There are theoretical studies that show that there ought to be a community-level effect. What I don't think there is is any attempt to estimate the magnitude of the community-level effect from empirical data. We've done this with mathematical models, but haven't completed the linkage with empirical data that could help with this estimate." Thomas Smith, Swiss Tropical Institute Head of the Biostatistics and Computational Sciences Unit, phone conversation with GiveWell, November 8, 2011.
  • "The argument for ITNs has always been that ITN coverage is a public good. Some of the good accomplished is not internalized to the person using the net. Some people argue that there's a negative externality as well - unprotected people are getting bitten by the mosquitoes that would have bitten the protected people - but I don't think you can make that argument with ITNs … What I do is analysis of data using mathematical models. In general, the leading models say that the more people use the nets, the bigger the positive externalities are … If you're talking about the marginal dollar for more nets, there's a lot more uncertainty on the cost-effectiveness of that than we'd like. The only data that we have is bad data. Improved surveillance would be helpful." Dave Smith, University of Florida Associate Professor of Zoology, phone conversation with GiveWell, November 4, 2011

• 54

  "In late 1996, the villages in Asembo were randomized into ITN (intervention) and control villages … for two years, 1997 and 1998, people in half of the villages (intervention villages) in Asembo had access to properly treated bed nets and half (control villages) did not. Periodic spot checks during this period showed that few (<3%) nets were sold or moved outside the study area, although adherence to net use, measured as the proportion of study participants directly seen to be sleeping under nets during early morning observational surveys, was usually approximately 70% … Although the study area is divided into villages with approximate boundaries as shown in Figure 1A, these are social constructs that do not reflect variation in population density … The 300-meter interval for the main exposure variables was based upon the observation that this distance range divides the number of compounds of each type into approximately equal quartiles (Table 1). All distance calculations were done on the basis of intent to treat; all compounds in ITN villages were assumed to have ITNs and all compounds in control villages were assumed to lack ITNs." Hawley et al. 2003, Pg 121.

• 55

  "… Compounds in control villages that are ≥900 meters from the nearest ITN village served as the comparison group for these analyses. A total of seven dichotomous variables were created that contained all other compounds. For control villages, we created three variables for compounds that were 0–299, 300–599, or 600–899 meters from the nearest ITN compound. For ITN villages, we created four variables for compounds 0–299, 300–599, 600–899, and ≥900 meters from the nearest control compound. The intent of this analysis was to allow us to illustrate spatial patterns of effect in a straightforward way; however, the cost of this approach was some loss in statistical power. These distance categories approximate compound distribution by quartiles as shown in Table 1 … The effect of the primary geographic exposure variable, distance to nearest compound of different type, is illustrated in Figure 2, which shows the effect of each of seven different exposure categories on five malaria-related outcomes, as well as geohelminth infection. For geohelminth infection, no effect of ITN use or distance is apparent. In contrast, a protective effect of living in ITN villages (stippled area of Figure 2) is apparent at all distance categories for all-cause mortality of children 1–59 months of age and for high-density parasitemia, anemia, and hemoglobin level … The overall pattern of effects is similar for all five malaria-related outcomes: protective effects are observed within the ITN compounds, with a similar level of protection seen in control compounds located within 300 meters of intervention compounds; these effects are statistically significant for the morbidity measures most sensitive to changes in transmission levels: anemia and hemoglobin level … Results of tests of trend reflect the patterns visible in Figure 2 for clinical malaria (odds ratio [OR] = 0.92, 95% confidence interval [CI] = 0.75, 1.12, P = 0.38), high-density parasitemia (OR = 0.89, 95% CI = 0.78, 1.01, P = 0.08), moderate anemia (OR = 0.78, 95% CI = 0.69, 0.89, P = 0.0001), hemoglobin level regression coefficient (OR = 0.18, 95% CI = 0.06, 0.31, P = 0.0048), and child mortality (hazard ratio = 0.94, 95% CI = 0.90, 0.98, P = 0.002). For all of these malaria-related outcomes, trends are in the direction indicating greater protection in compounds closer to intervention villages, with strongly significant trends seen for mortality, anemia, and hemoglobin level. For geohelminth infection, no trend was observed (OR = 1.09, P = 0.47)."

  

  Hawley et al. 2003, Pgs 123-125.

• 56

  "This community effect on nearby compounds without nets is approximately as strong as the effect observed within villages with ITNs." Hawley et al. 2003, Pg 121.

• 57

  "studies from various parts of Africa and Papua New Guinea indicate the presence of a beneficial community effect (we propose use of this term rather than the less descriptive mass effect) on malaria transmission [references 1-6], severe malaria [reference 7] and mortality [reference 8] in children. One study, in contrast, yielded no evidence for either beneficial or harmful community-wide effects. [reference 9]" Hawley et al. 2003, Pg 121.

  We looked up reference 9 and confirmed that it discusses only transmission: "In The Gambia, the use of permethrin-treated bed nets has led to a reduction in morbidity and mortality from malaria in children. However, no clear evidence has been found for a 'mass-killing effect' on the mosquito vectors as a result of this intervention. Two further entomological studies to investigate this phenomenon have been carried out. In one study, 20 villages were paired so that bed nets in one member of each pair were treated with permethrin. In the other, a cross-over design was used in which treated and untreated bed nets were exchanged between 2 villages. Longevity, biting rate and resting density of the malaria vector population and sporozoite rates were assessed in both studies. Malaria vectors were equally abundant and long-lived, and as likely to be infective, in villages with treated bed nets as in those with untreated nets. However, a clear reduction in the density of the indoor-resting population of mosquitoes in rooms with treated bed nets was found, probably reflecting the excito-repellency of the insecticide. This study confirmed that, in The Gambia, the protection against death and morbidity from malaria seen in children using treated bed nets must be due primarily to personal protection rather than to a 'mass-killing effect' on the mosquito vector population at a village level." Quinones et al. 1998, abstract.

• 58

  Gamble, Ekwaru, and ter Kuile 2006, Pg 14.

• 59

  "An important paper that had some influence in the political debate about mass distribution (as opposed to distribution to infants and pregnant women only) was: Killeen et al. 2007." Thomas Smith, Swiss Tropical Institute Head of the Biostatistics and Computational Sciences Unit, email to GiveWell, November 8 2011.

• 60

  Killeen et al. 2007, Pg 1246. The paper includes a spreadsheet with the full specification of its model.

• 61

  See Smith et al. 2009, Table 1, Pg 514. Note that the higher the existing level of ITN coverage (column headings), the lower the additional amount of ITN coverage needed (figure in the cell minus figure in the column heading) to reach the specified target. For example, at a PfPR of 50% and a goal of reducing it to 1% (10th row, 4th-6th columns), one would need to go from 0% to 71% effective coverage (71% absolute change), or from 10% to 77% coverage (67% effective change), or from 20% to 81% effective coverage (61% effective change).

• 62

  "Holden Karnofsky: Do you know of any papers or scholars arguing the opposite, that community-level effects are negligible and/or that universal coverage has little value-added above and beyond covering pregnant women and children under five?

  Thomas Smith: I don't." Thomas Smith, Swiss Tropical Institute Head of the Biostatistics and Computational Sciences Unit, phone conversation with GiveWell, November 8, 2011.

• 63

  "This is very consistent with the results based on net sales; they suggest that demand for ITNs is 75 percent lower at the current cost-sharing price in Kenya (50Ksh or $0.75) than it would be under a free distribution scheme." Cohen and Dupas 2007, Pg 11. See also Figure 1, Pg 11.

• 64

  Cohen and Dupas 2007, Pg 15, Figure 2.

• 65

  "When a school-based NGO deworming program in western Kenya introduced cost sharing, the take-up of deworming medication fell sharply and revenue failed to outpace administrative costs, despite the health and education benefits of the program. User fees did not help target treatment to the sickest children. As deworming pills are delivered directly into children's mouths, there was no gap between take-up and usage, so pricing had no potential psychological impact on use through sunk costs." Kremer and Holla 2009, Pg 95.

• 66

  "Two-stage pricing randomization has also been used to test the two potential routes by which pricing could affect use in a door-to-door marketing campaign for water disinfectant on the outskirts of Lusaka, Zambia. As with deworming and nets, pricing led to a rapid drop-off in take-up, with no evidence of increased targeting to the most vulnerable. Pricing had no statistically significant psychological effect on use. The authors of the study argue that there was some screening effect since those who were more willing to pay for the disinfectant were more likely to have chlorine in their water at later random checks than those who received it for free." Kremer and Holla 2009, Pg 99.

• 67

  "In 1993 a cost-recovery programme was introduced and villages given free insecticide during the first year of the study were asked to pay 5.00 Dalasi (US$0.50) per bednet treated. Unfortunately this led to a dramatic drop in coverage and a return of child mortality rates in these villages to their pre-intervention values." D'Alessandro et al. 1995, Pg 483.

• 68

  "In January 2002 the UK Department for International Development (DFID) awarded PSI-Kenya US$33 million over 5 y to socially market partially subsidised ITN within the existing retail sector... A two-tier pricing system of 350 Kenya Shillings (KES) (equivalent to US$4.7) in urban settings versus KES100 (US$1.3) in rural settings was implemented." Noor et al. 2007, Pg 1342.

• 69

  Noor et al. 2007, Pg 1344, Table 1.

• 70

  "The programme began in October 2004, and during the first 6 mo Supanet ITNs were bundled with separate Powertab net treatment tablets (for every 6 mo) and distributed to MCH attendees. In May 2005 an additional US$37 million was committed by DFID to PSI to procure and distribute Supanet-branded long-lasting insecticidal nets (LLINs), Olyset and Permanet. These public sector nets were heavily subsidized pretreated nets (KES50; US$0.7) and branded with the MoH logo." Noor et al. 2007, Pg 1342.

• 71

  "During round four of the GFATM awards in April 2004, Kenya's application was successful and US$17 million was approved to procure and distribute 3.4 million LLINs (Olyset and Permanet brands) free of charge to children under the age of 5 y. This represented, at the time, the largest successful award for free distribution of LLINs in Africa." Noor et al. 2007, Pg 1342.

• 72

  Noor et al. 2007, Pg 1344, Table 1.

• 73

  GiveWell. Cost-effectiveness analysis for LLIN distributions updated

• 74

  "The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to7.67; Analysis 1.2)." Lengeler 2004a, Pg 8. We have confirmed with the author that this figure is based on an "intention to treat" analysis, e.g., "protected" refers to children in the treatment group, not to children who were confirmed to own or use ITNs.

• 75

  “The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to 7.67; Analysis 1.2). I performed a regression analysis of the natural logarithm of the rate difference on the entomological inoculation rate and could not find a trend (r 2 = 0.52, F = 3.2 on 1,3 degrees of freedom, P = 0.2). In contrast to protective efficacies, the risk differences seemed to have a tendency towards a higher effect with a higher entomological inoculation rate. This apparent paradox is because the baseline mortality rates are higher in areas with high entomological inoculation rates." Lengeler 2004a, Pg 8.

• 76

  See penultimate section of our 2012 blog post on the subject

• 77
  “Global malaria deaths increased from 995 000 (95% uncertainty interval 711 000—1 412 000) in 1980 to a peak of 1 817 000 (1 430 000—2 366 000) in 2004, decreasing to 1 238 000 (929 000—1 685 000) in 2010. In Africa, malaria deaths increased from 493 000 (290 000—747 000) in 1980 to 1 613 000 (1 243 000—2 145 000) in 2004, decreasing by about 30% to 1 133 000 (848 000—1 591 000) in 2010. Outside of Africa, malaria deaths have steadily decreased from 502 000 (322 000—833 000) in 1980 to 104 000 (45 000—191 000) in 2010. We estimated more deaths in individuals aged 5 years or older than has been estimated in previous studies: 435 000 (307 000—658 000) deaths in Africa and 89 000 (33 000—177 000) deaths outside of Africa in 2010.” Murray et al. 2012, Abstract.
  
• 78

  See our blog posts on the topic:

  • General issues with current DALY-based analyses
  • Specific problems with published DALY figures on deworming

• 79

  GiveWell. LLIN gap analysis (2012) Note about the data on which our analysis is based: “This map is a theoretical model based on available long-term climate data. It has a resolution of about 5x5 km. Although it is reasonably accurate, it is not based on actual malaria data and may not reflect the real malaria status. It shows the theoretical suitability of local climatic, and therefore the potential distribution of stable malaria transmission in the average year. Please note that climatic conditions, and therefore malaria transmission, vary substantially from one year to the next. Malaria control activities can also dramatically alter the malaria transmission situation.” Mapping Malaria Risk in Africa. Information on maps

• 80

  "We have approved a distribution of 250,000 nets in the districts of Balaka and Dedza in Malawi. The distribution would take sleeping space coverage levels from 30-60% (the level is currently unknown) to 90% and above." Against Malaria Foundation. Recently approved distributions and others being assessed

• 81

  GiveWell. LLIN gap analysis (2012), sheet "Basic gap analysis."