We have published a more recent version of this page. See our most recent version of this page.

Published: June 2016; Last updated September 2018

Previous versions of this page:

• March 2018 version
• 2016 version
• 2012 version

Background on insecticide resistance

What is resistance?

"Resistance" can be used to refer to genetic properties of mosquitoes, to refer to behavioral properties of mosquitoes, or to refer directly to failures of mosquito control.1 In the context of malaria control, we've generally seen it used to refer to the first two; as discussed below, there is relatively little evidence of control failure due to these factors, so "resistance" is usually used to discuss specific risk factors for control failure.

The four types of resistance we've seen discussed in this context are:2

• Target-site resistance (of which "knock-down resistance" is the main type we've seen discussed): mosquitoes may develop mutations that make insecticides less effective in disabling them after reaching their "target site" (the part of the mosquito that the insecticide seeks to directly affect, often within the nervous system).
• Metabolic resistance: mosquitoes possess internal systems to detoxify foreign materials; these systems may evolve to more effectively detoxify insecticides.
• Cuticular resistance: mosquitoes' exteriors may become less prone to absorbing insecticide.
• Behavioral resistance: mosquitoes may change their behavior to avoid control efforts - for example, being more active at the times of day when humans are less likely to be indoors and protected by insecticides. The World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors mentions that "[a]ll behavioural traits, however, may not be negative, as they could lead mosquitoes to feed on non-human animals. It is also possible to initially mistake the decline of a vector species as behavioural resistance."3

We've seen fairly little discussion of the last two types. The World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors comments:

What is the relationship between “resistance” and ITN control failure?

The presence of “resistance,” as defined above, does not necessarily imply that ITNs will be ineffective. There are multiple reasons that ITNs may retain effectiveness even against mosquitoes that are "resistant" (in the sense of demonstrating low mortality rates in laboratory settings). In addition to the fact that ITNs provide a physical barrier, the insecticide may also repel mosquitoes (and cause them to seek out other targets) even when it does not kill or fully disable them.4 In addition, it's possible that mosquitoes are still killed by the insecticide (despite reduced susceptibility) when they have enough contact with it; that insecticide may inhibit them in other ways that stop them from transmitting malaria; that resistant mosquitoes are less fit overall or less prone to transmitting malaria; or that mosquitoes that are resistant to insecticides at young ages may become less resistant as they age (and that older mosquitoes are more relevant to malaria transmission).5 We have seen one study that argues for the last of these phenomena.6

The World Health Organization writes that "[i]t is broadly accepted that different resistance mechanisms have differing capacity to cause control failure, kdr [knock-down resistance] tending to be less likely than metabolic resistance (or a combination of mechanisms) to cause control failure."7

Which insecticides are used on ITNs?

Pyrethroids are the primary class of insecticide recommended by the WHO for use on ITNs.8 We do not have a strong sense of when new insecticides may be developed, but some experts estimate that no replacement insecticide suitable for use on ITNs will be available for a decade.9

How common is insecticide resistance? Is it increasing?

When evaluating current levels of insecticide resistance, it is relevant to consider both the prevalence of resistance (i.e., the existence of at least some amount of resistance, as measured by a standardized test discussed in more detail below) and the strength of resistance (i.e., the intensity of insecticide resistance in a given population of mosquitoes, which can be measured in a variety of ways and may give a better indication of whether resistance is severe enough to affect malaria control). Strong resistance in a small subset of a vector population may be a greater threat to malaria control than more prevalent weak resistance (for further elaboration, see example in footnote).10

It appears that at least weak insecticide resistance is common across sub-Saharan Africa and that insecticide resistance is becoming more prevalent. We have seen very little information about the strength of resistance in sub-Saharan Africa.

In forming our views about insecticide resistance, we rely mainly on recent commentaries by Ranson and Lissenden 2016 and Hemingway et al 2016, and on the sources cited therein. We rely on these sources because they are the most comprehensive and recent literature reviews that we were able to find (more on our process below).

Details follow.

How is insecticide resistance typically measured?

The most common test used to measure insecticide resistance is a World Health Organization (WHO)-standardized diagnostic dose assay (referred to as a “WHO bioassay” for the rest of this document). WHO bioassays measure resistance by exposing wild-caught mosquitoes to twice the minimum concentration of insecticide necessary to kill 100% of susceptible mosquitoes, and define a population of mosquitoes as resistant if the observed mosquito mortality rate is <90% (details of the testing process in following footnote).11 Our understanding is that this test is often used to determine whether insecticide resistance exists in an area (i.e., the prevalence of insecticide resistance).12 It is unclear to us whether this test is useful for determining the strength of resistance; some experts argue that it is inadequate for this purpose.13

There are a variety of other possible tests for measuring the strength of insecticide resistance. One example of such a test that we have seen was from a study in Burkina Faso which exposed mosquitoes to insecticide for varying amounts of time until 50% of the population was killed.14 Professor Hilary Ranson of the Liverpool School of Tropical Medicine told us that tests measuring the strength of resistance have not been a common part of insecticide resistance monitoring in sub-Saharan Africa to date and we have not seen much information on strength of resistance in our review of the literature.15

There are many other tests of insecticide resistance, such as experimental hut trials and cone bioassays, which may also offer useful information.16 We have not yet tried to determine which tests of insecticide resistance provide the most useful information for predicting the effect of insecticide resistance on malaria control.

Where is there insecticide resistance?

We focus our discussion on sub-Saharan Africa since the Against Malaria Foundation (AMF) is most likely to distribute nets in this region.

According to WHO bioassay data collated by IR Mapper in 2016, it appears that resistance is prevalent in all areas across sub-Saharan Africa where researchers have reported measuring resistance levels,17 though there seems to be some local variation in the prevalence of resistance (i.e., some tests found mosquito populations to be susceptible in areas nearby locations where tests found resistance):

IR Mapper is a visualization tool which collates resistance data from peer-reviewed scientific papers and the reports of insecticide resistance monitoring institutions.18 We have not vetted IR Mapper's data.

Information about the strength of insecticide resistance in sub-Saharan Africa appears to be too limited to draw broad conclusions. We are aware of only three studies that explicitly attempt to measure the strength of resistance, and these studies cover small regions.19 The studies aimed to measure strength of resistance either by exposing mosquitoes to increasing amounts of insecticide until 50% of the population died or by exposing them to a fixed amount of insecticide for a longer period of time until 50% of the population died; headline results from all three studies are described in the following footnote.20 In one of the studies (Toé et al 2014 in Burkina Faso) the authors found no significant difference in mosquito mortality according to WHO bioassays over a three year period, but found that strength of resistance increased 1,000-fold over the same period.21 We are unsure how meaningful these measures of strength of resistance are, and we do not know how to translate these results into likely effects on malaria control.

Because we have limited information about the best ways to predict insecticide resistance’s impact on malaria control, we do not have a strong understanding of which parts of sub-Saharan Africa are likely to be most threatened by insecticide resistance.

Is insecticide resistance increasing?

We have seen a few data points that suggest that insecticide resistance is increasing in sub-Saharan Africa. These data points, combined with the theoretical argument that insecticide resistance should be expected to increase over time as insecticides exert selective pressure on mosquitoes, lead us to believe that insecticide resistance is likely increasing. Therefore, we would guess that insecticide resistance is likely to be a greater threat to ITNs’ effectiveness in the future.

First, the following figure from Ranson and Lissenden 2016 collates WHO bioassay results for two types of pyrethroids for two species of mosquito (the dominant species that carry malaria) in sub-Saharan Africa,22 and appears to show that the average mosquito mortality rate as measured in WHO bioassays has fallen over time:

Reprinted from Trends in Parasitology, Vol 32(3), Ranson, H., and Lissenden, N., Insecticide Resistance in African Anopheles Mosquitoes: A Worsening Situation that Needs Urgent Action to Maintain Malaria Control, pp. 187-196, Copyright (2016), with permission from Elsevier.

The above figure may not be representative of the average mosquito mortality rate in sub-Saharan Africa since researchers may be more likely to measure insecticide resistance in areas where they expect it to exist, but it nonetheless seems to suggest that insecticide resistance is increasing.

Second, Ranson and Lissenden 2016 also provides a figure showing that mosquito mortality rates appear to be falling over time in a few locations (though we do not know how representative these locations are):23

Reprinted from Trends in Parasitology, Vol 32(3), Ranson, H., and Lissenden, N., Insecticide Resistance in African Anopheles Mosquitoes: A Worsening Situation that Needs Urgent Action to Maintain Malaria Control, pp. 187-196, Copyright (2016), with permission from Elsevier.

The data presented above is from IR Mapper.24 We have not vetted IR Mapper's data.

Third, some academic commentaries that we have seen argue that resistance is increasing over time, and we have not seen any papers that argue that resistance is not increasing.25

We have limited information about whether the strength of resistance is increasing over time in sub-Saharan Africa. The limited evidence that we have seen on the strength of resistance is discussed above.

Note that this discussion covers only target-site and metabolic resistance, which make insecticides less effective in killing mosquitoes; by the nature of WHO bioassays, they do not address behavioral resistance. As discussed above, there is currently relatively little evidence regarding behavioral resistance and no clear consensus about its importance.

Is resistance making ITNs less effective?

The implications of current levels of insecticide resistance for the effectiveness of ITNs are unclear. There are no high-quality studies that estimate the impact of insecticide resistance on ITNs’ effectiveness, partly because of ethical restrictions on testing ITNs via new randomized controlled trials (RCTs). Experts generally seem to argue that ITNs remain highly effective in most locations, but this conclusion seems to be based on low-quality information.

Why is there a lack of RCTs connecting insecticide resistance and clinical malaria outcomes?

Ethical restrictions prevent running RCTs comparing ITNs to a control group without bed nets or to a control group with untreated nets (UTNs, i.e. nets without insecticide) in areas with insecticide resistance.26 Comparing ITNs to no net controls or UTNs in resistant areas is considered unethical because ITNs are a proven intervention for decreasing malaria risk and participants cannot ethically be randomized to a known lower standard of care.27 (Note we discuss below a forthcoming study which nevertheless seems to have gained ethical approval to compare ITNs and no net controls.)

What other literature aims to estimate the effect of insecticide resistance on malaria control?

We are aware of several studies that attempt to estimate the effect of insecticide resistance on malaria control. However, they all use non-randomized study methods that we believe have major limitations for reaching accurate conclusions on the effectiveness of ITNs. We discuss how we arrived at our best guess of the impact of resistance on ITNs' effectiveness in the cost-effectiveness section.

Below, we provide examples of the kinds of non-randomized studies that we have seen. We provide relatively detailed descriptions of Ochomo et al 2017 and Bradley et al 2017 because they are part of a recent, multi-country series of studies coordinated by the WHO to address the question of the epidemiological impact of resistance.28

The kinds of studies that we have seen are:

• Cohort or cross-sectional studies, such as Ochomo et al 2017, Bradley et al 2017, Lindblade et al 2015, and Asidi et al 2012, which attempt to estimate the protective effect of ITN use by comparing ITN users with non-users, or by comparing malaria indicators in areas with higher versus lower levels of estimated insecticide resistance.29 Details on each study are:
  • Ochomo et al 2017 was conducted in about 50 villages in Kenya with a sample of about 2,000 young children.30 The study followed its participants for about 3 months, testing them for malaria roughly every 2 weeks.31 It compares (a) overall malaria incidence rates in villages with higher versus lower levels of resistance, and (b) incidence rates among net users versus non-users, both overall and interacted by resistance level.32 It finds that malaria incidence rates were not higher in higher resistance areas, that net users had significantly lower rates of malaria than non-users, and that there was not a significant difference in the protective efficacy of nets in high versus low resistance areas.33 However, it does not rule out the possibility that confounding factors significantly affected the results (e.g., there could be selection bias associated with comparing net users to non-users).34 Two malaria researchers we spoke to cautioned against interpreting this as evidence that resistance has little effect on public health; they noted that these studies may have taken place in areas with low resistance overall and that the study designs are not able to account for effects of resistance on community-level protection.35 We have not vetted these arguments. The authors of Ochomo et al 2017 seem to interpret their results as showing that nets remain useful, but caution that nets may be less effective than they were in the past (in both lower and higher resistance areas).36
  • Bradley et al 2017 conducted a cross sectional survey in 21 villages (1,621 children) at a single point in time, comparing malaria prevalence rates in villages with higher versus lower levels of resistance and malaria prevalence among net users versus non-users.37 It found that children who used nets had lower malaria prevalence compared to non-users. It did not find evidence that differences in malaria prevalence between net users and non-users were different in higher versus lower resistance areas, and it did not find an association between resistance and malaria prevalence at the village level.38 This study's methodology seems to have the same limitations as Ochomo et al 2017 (see discussion above), and the authors of Bradley et al 2017 also conclude that nets seem to remain effective, but note that this study is unable to determine whether nets are less effective than they have been in the past.39
  • Lindblade et al 2015 compares malaria incidence among people who used bed nets against incidence among non-users, and attempts to adjust for potential confounding factors.40 It finds that ITNs reduced the incidence of malaria infection by about 30% compared to no bed nets in an area with moderate resistance.41 The sample size of non-users was very small and it seems challenging to adequately adjust for all confounders, so we do not feel comfortable relying on this study.42
  • Asidi et al 2012 compared mosquito biting and mosquito survival rates before and after treating ITNs with pyrethroids in areas with insecticide resistance and areas without resistance.43 It found that ITNs were no better than untreated nets in areas with insecticide resistance.44 This was a non-randomized study with a very small sample size (16 households), so we do not put much weight on its results.45
• Laboratory and field studies, such as those analyzed by Strode et al 2014. Strode et al 2014 does not attempt to directly measure the effect of insecticide resistance on malaria incidence. It is a meta-analysis of cone tests, tunnel tests, and experimental hut trials (methodologies of these tests are described in the following footnote) which compares mosquito mortality and blood feeding following exposure to ITNs against mortality and feeding following exposure to UTNs at different levels of insecticide resistance.46 It finds that ITNs are more protective than UTNs at any level of resistance.47 However, the authors suggest interpreting these results with caution because of "substantive heterogeneity" in the pooled studies.48 Furthermore, low data quality, inconsistent experimental design, and variance in methods of resistance measurement made it impossible for the authors to assess resistance's effect on malaria transmission or clinical outcomes.49 Accordingly, we did not closely investigate Strode et al 2014.
• Modeling studies, such as Churcher et al 2016, that use laboratory and field studies combined with malaria transmission models to predict the impact of resistance.50 Though these studies are non-randomized, they seem to be the simplest path available for roughly estimating the impact of resistance on malaria control. We discuss some of these studies in more detail below.

Have there been cases of malaria control failure due to insecticide resistance?

There are reports of ITNs failing to control malaria in some locations due to insecticide resistance, but these have not been rigorously confirmed via high-quality studies. We have not closely examined the studies mentioned below.

Pre-2010, two longitudinal studies in Senegal (relating to ITNs and malaria treatment) and South Africa (relating to indoor residual spraying) observed an initial drop in malaria cases when malaria control interventions were implemented, followed by a rebound in malaria cases as resistance emerged.51

Recent observational evidence on ITN effectiveness in areas with resistant malaria vectors is mixed. We have seen some post-2010 observational evidence suggestive of control failure: namely, evidence that malaria case rates have stayed stable or even increased following ITN distributions in areas of Burkina Faso, Uganda and Zambia.52 Professor Hilary Ranson of the Liverpool School of Tropical Medicine told us that ITNs continue to be effective in most areas, but that there are credible reports of potential control failure in some areas.53 We have also seen other kinds of evidence of resistance causing possible ITN failure: a study observing resistant mosquitoes resting inside holed ITNs,54 and a study documenting early evening feeding by resistant vectors in areas with moderate coverage of malaria control interventions.55 Yet other observational evidence suggests that ITNs are more protective in high resistance areas; this seems theoretically implausible, but shows that outcomes from observational studies vary substantially.56

What do experts say about the impact of insecticide resistance on malaria control?

Our general impression is that experts in the malaria community believe that ITNs remain effective in most areas. The non-randomized studies discussed above generally seem to find that ITNs are still effective. Professor Hilary Ranson told us that ITNs continue to be effective in most areas but that there are credible reports of potential control failure.57 We have seen some experts argue that it would probably be difficult to directly associate ITN failure with insecticide resistance because of confounding factors.58

Do new types of LLIN help to mitigate resistance?

PBO nets are a newer type of net incorporating piperonyl butoxide (PBO) alongside the standard pyrethroid insecticide used in other nets. PBO is a synergist which inhibits the metabolic enzymes that grant immunity in mosquitoes.59

In June 2017, the World Health Organization gave PBO nets an interim endorsement to be used in vector control,60 and recommended they be considered for deployment in areas of intermediate insecticide resistance where the mechanism of resistance is susceptible to PBO.61 The interim endorsement is based on evidence from one randomized controlled trial, Protopopoff et al. 2018.62

We believe Protopopoff et al. 2018 provides some evidence that PBO nets give additional protection over standard LLINs in areas with insecticide resistance, so we believe PBO nets may be the most cost-effective net distribution option in some cases. We have used the results of this trial to estimate the magnitude of the effect of insecticide resistance on ITNs’ effectiveness (more here). We note some reasons to be cautious in generalizing from the study below and may revisit our view when an RCT of PBO nets in Uganda is completed.63

Details follow.

Study Design

Protopopoff et al. 2018 was a cluster-randomized trial of PBO nets and indoor residual spraying (IRS) in Tanzania between March 2014 and December 2016.64 10,560 households (48 clusters) were randomized into four groups to receive:

1. Standard LLINs only.
2. PBO LLINs only.
3. Standard LLINs plus IRS.
4. PBO LLINs plus IRS.65

The primary outcome was the prevalence of malaria infection in children aged 6 months to 14 years measured by rapid diagnostic test 4, 9, 16 and 21 months after the intervention.66 The study had 80% power to detect a relative reduction in prevalence of infection of 28% between PBO nets and standard nets.67

Results

Groups targeted to receive PBO nets had significantly lower malaria prevalence than groups targeted to receive standard nets at 9 months, 16 months, and 21 months after the intervention. There was no significant difference at 4 months.68 Averaging over the four surveys, the relative risk of malaria with PBO nets compared to standard nets in areas not also receiving indoor residual spraying was 0.68 (i.e. a 32% reduction in malaria prevalence for groups targeted to receive PBO nets).69

Limitations

We note a number of reasons to be cautious in interpreting and generalizing from this study:

• The study was conducted in Muleba, Tanzania, in a population where the vectors had developed insecticide resistance.70 As the primary mechanism through which PBO nets are hypothesized to provide additional protection is through killing pyrethroid-resistant mosquitoes,71 the additional protection provided by PBO nets over standard nets is likely to be small in areas with low resistance to pyrethroids, or where resistance mechanisms are not susceptible to PBO.72
• PBO nets were not significantly better than standard nets when used in conjunction with indoor residual spraying (IRS) of the pesticide pirimiphos-methyl.73 The authors state it is unclear whether IRS with insecticides other than pirimiphos-methyl would partner well with PBO nets.74 We do not have a strong understanding of what proportion of areas covered by future distributions of PBO nets will also be covered by IRS, but believe it is unlikely to be a significant proportion based on communication with AMF.75
• The distance between clusters was a minimum of 300 metres, which the authors note is smaller than other trials (although we have not investigated this claim in detail). This distance may be insufficient to prevent contamination between different intervention groups (i.e. distributing PBO nets in one area might cause reductions in malaria prevalence in a neighboring area which did not receive PBO nets).76 Contamination would have led the study to underestimate the relative effectiveness of PBO nets.
• The study was not powered to detect an effect on all-cause mortality. We assume the reduction in mortality from malaria is equal to the reduction in malaria prevalence.77
• The trial compared two kinds of nets, Olyset Plus (PBO), and Olyset Net (Standard). Olyset Plus nets are made with a smaller mesh size than Olyset Nets, making it possible that their increased effectiveness was partially due to characteristics other than the addition of PBO.78 We believe it is likely that the addition of PBO was the main reason for the greater effectiveness of Olyset Plus nets since this is the simplest explanation, but have not investigated this question in detail. Against Malaria Foundation is currently trialling two types of PBO net in Uganda; Olyset Plus (the same as Protopopoff et al. 2018), and a net where only the roof is treated with PBO.79

We may revisit our estimate of the relative effectiveness of PBO nets to standard nets when an RCT of PBO nets in Uganda is completed.80

How does insecticide resistance affect the expected cost-effectiveness of donations to AMF?

Though we have not found any experimental studies that directly estimate the magnitude of the impact of insecticide resistance on ITNs’ effectiveness, we believe that it is necessary to include some adjustment for insecticide resistance when projecting the cost-effectiveness of AMF.

In 2016, our very rough best guess was that ITNs are about one-third less effective in the areas where AMF is working than they would be in the absence of insecticide resistance. We revisited this conclusion in 2018 based on new information, and slightly negatively updated our estimate. We discuss our reasoning below.

2016 reasoning:

• An explicit model of the public health impact of insecticide resistance suggests that resistance may reduce ITNs' effectiveness by about 40%.
  • Lacking any experimental studies connecting insecticide resistance with clinical outcomes, we must rely on the best non-experimental studies on this topic. We rely on Churcher et al 2016, the most recent and comprehensive study we are aware of that explicitly models the effect of insecticide resistance on malaria outcomes.81 Two of the authors of Churcher et al 2016 told us that they are not aware of other studies that estimate the effect of resistance on public health, with the exception of non-experimental studies that compare malaria rates in net users versus non-users, a methodology that we believe has substantial potential confounding issues (see discussion above).82
  • Churcher et al 2016 uses studies of insecticide resistant mosquito behavior (e.g., how often susceptible versus resistant mosquitos are deterred from entering houses by bed nets, how frequently resistant mosquitos feed on people with nets, etc.) as inputs into a malaria transmission model to estimate the public health effects of different levels of resistance.83 Churcher et al 2016 aims to take into account the community-level effects of insecticide resistance.84 We have not carefully vetted Churcher et al 2016 or understood its methodology in depth.
  • Assuming that AMF’s marginal distributions will occur in areas with moderate insecticide resistance on average (i.e., mosquito mortality rate of about 50%),85 we estimate that Churcher et al 2016 would predict a reduction in the effectiveness of bed nets of about 40%.86 We have not vetted the model underlying this estimate and are highly uncertain about the best estimate of mosquito mortality in AMF-supported regions.
• We considered a variety of sanity checks and concluded that an estimated one-third reduction in the effectiveness of bed nets due to insecticide resistance seems in the range of a reasonable estimate (see footnote for detail).87
• After completing the above research, we changed our final insecticide resistance adjustment in our cost-effectiveness model for AMF from about 33% to about 26% because we expected AMF to spend a substantial portion of its funding on PBO nets in Uganda, which we expected to be less susceptible to insecticide resistance.88

2018 reasoning:

In 2018, we revisited this adjustment based on (i) new summary data on mosquito mortality to permethrin (a pyrethroid insecticide used in LLINs) collected by the WHO,89 and (ii) Protopopoff et al. 2018, a randomized controlled trial comparing the effectiveness of PBO nets to standard nets.

Overall, this work led to a slight change in our insecticide resistance adjustment (from a roughly 26% negative adjustment previously to a roughly 35% negative adjustment in 2018).90 This negative update is largely due to the fact that insecticide resistance appears to be getting worse over time, so we are accounting for the continued expected decline of the effectiveness of standard nets as time goes on. However, the declining effectiveness of standard nets is slightly offset by our expectation that AMF will allocate a portion of its funding toward distributing PBO nets, which we believe are less susceptible to resistance (see above). Further details include:

• Protopopoff et al. 2018 found that people targeted with PBO nets were about 32% less likely to be infected with malaria than people targeted with standard LLINs in areas not covered by indoor residual spraying. For more discussion of Protopopoff et al. 2018, see the section above.
• The results in Protopopoff et al. 2018 are consistent with a simplified model for predicting the effect of insecticide resistance which assumes that 20% of the protective effect of nets is due to the physical barrier, and 80% of the protective effect scales linearly with mosquito mortality as measured by relevant WHO bioassays.91
• To update our insecticide resistance adjustment, we estimated mosquito mortality rates to permethrin in countries in which AMF works, based on summary data from a WHO report released in 2017 (see WHO Global Report on insecticide resistance 2010-2016), and linearly extrapolated these estimates into the future (when AMF nets would be distributed).92 On average, we expect mosquito mortality to permethrin alone to be ~40% in AMF-targeted regions. Using the simple model described above and accounting for a proportion of delivered nets being PBO nets implies a 35% downward adjustment to cost-effectiveness to account for the effects of insecticide resistance.93
• We have not vetted the quality or representativeness of the mosquito mortality data from the WHO report. We were concerned that this data may be more likely to be collected in areas with severe resistance, which would underestimate the effectiveness of standard LLINs. We contacted an insecticide resistance researcher, and they told us that use of "sentinel sites" for measuring resistance is now more common, so they did not expect this to be a major issue.94 We have not prioritized independently vetting their view. We may revisit this in the future.

We may update this estimate when results from RCTs testing new types of ITNs are available (more).
Our full cost-effectiveness analysis for AMF is available here, and the calculations for our 2018 estimate of the effect of insecticide resistance are here.

Forthcoming literature on insecticide resistance and malaria control

The following studies measuring clinical malaria outcomes in areas with confirmed resistance and ITN distributions are currently being conducted:

• Two randomized controlled trials of new types of ITNs, which have been shown in experimental hut trials to induce higher mortality and to reduce blood feeding in resistant mosquitoes when compared to standard ITNs:95
  • A cluster-RCT in rural Burkina Faso assessing whether Olyset Duo, a PPF-ITN which contains 2% permethrin (a pyrethroid) and 1% pyriproxyfen (an insect juvenile hormone mimic which reduces mosquito fecundity), provides better protection against clinical malaria in children than the pyrethroid-only Olyset ITN, which contains just 2% permethrin.96 The protocol provides that there will be ~40 clusters (20 in each arm) and at least 45 children being actively monitored for malaria in each cluster.97
  • An RCT of PBO nets distributed by AMF in Uganda; for more information, see here.
• A cluster-RCT comparing (among other things) the efficacy of conventional ITNs to no net controls in south-central Ethiopia. According to the published protocol, this study will measure malaria cases based on diagnoses at health posts and will conduct contemporaneous entomological studies. It will have ~44 clusters of 35 households, and data collection was estimated to be completed in December 2016, but we have not yet seen results.98
• Kleinschmidt et al 2015, a four-year study in Sudan, Kenya, India, Cameroon and Benin designed to quantify the potential loss of epidemiological effectiveness of ITNs due to decreased susceptibility of malaria vectors to insecticides.99 The study includes ~300 clusters, where insecticide resistance levels are measured annually using WHO standardized methods and malaria case rates are actively detected in cohorts of children.100 With the exception of India, data collection was finalized in 2015. Data collection in India was expected to be finalized in 2016.101 We have seen results from two of the countries included in this series of studies (see Ochomo et al 2017 and Bradley et al 2017) and have not yet seen results from the other countries.

We expect all the RCT results to estimate the decrease in efficacy of ITNs in resistant areas. The RCT comparing ITNs to no net controls will directly measure the protective effect of ITNs in certain resistance contexts. The RCTs comparing conventional ITNs to combination ITNs where the relevant vector is 100% susceptible to the combination insecticide will estimate the loss in protective efficacy attributable to insecticide resistance. For PPF nets, we expect few confounding factors because the PPF nets being trialled in Burkina Faso are exactly the same as the conventional nets they are being compared to, except for the addition of pyriproxyfen.102 We have found that it is difficult to predict when studies will be released and there are often long delays, so we do not know when results will be available but we plan to follow up on the above studies so that we see their results as soon as possible.

Our process for finding information on insecticide resistance

2016

In 2016:

• We searched Google Scholar and PubMed for papers looking at trends in insecticide resistance and, in particular, papers connecting insecticide resistance with potential malaria control failure. We relied on Ranson and Lissenden 2016 and Hemingway et al 2016 as reviews of relevant literature.
• We searched the Malaria Journal for "resistant" and "resistance," scanned for publications relevant to insecticide resistance as it relates to LLINs, and read the abstracts of those papers.
• We spoke with Prof. Hilary Ranson, a member of the Against Malaria Foundation's Malaria Advisory Group (conversation notes are available here). We also spoke with Prof. Immo Kleinschmidt, principal investigator of a multi-country study of ITN resistance’s effect on clinical malaria outcomes.

2017

In 2017-2018:

• We updated this page to include results from two studies (Ochomo et al 2017 and Bradley et al 2017) that were part of a recent, multi-country series of studies coordinated by the WHO to address the epidemiological impact of resistance (see Kleinschmidt et al 2015).
• We updated this page to provide more details on Churcher et al 2016, a modeling study that we referenced in previous versions of this page but that was previously unpublished.
• We spoke with two malaria researchers (see GiveWell's non-verbatim summary of a conversation with Dr. Thomas Churcher and Professor Hilary Ranson, May 31, 2017) about recent studies on insecticide resistance and asked them whether there were any new studies particularly relevant to our key questions.103
• We continued to check for results of the new studies highlighted in the forthcoming literature section.

2018

In 2018:

• We reviewed the results of Protopopoff et al. 2018, a randomized controlled trial which tested the effectiveness of “PBO” nets relative to standard nets (more here).
• We updated our insecticide resistance adjustment based on Protopopoff et al. 2018, and a report by the World Health Organization (WHO Global Report on insecticide resistance 2010-2016), which estimated country-level rates of insecticide resistance. This slightly negatively updated our estimate (more here).
  

Sources

Document	Source
Against Malaria Foundation - AMF funds 10.7 million nets for distribution in Uganda	Source (archive)
AMF - Uganda Distributions Spreadsheet	Source
Asidi et al 2012	Source (archive)
Bagi et al 2015	Source (archive)
Bradley et al 2017	Source (archive)
Briet et al 2013	Source (archive)
Choi et al 2014	Source (archive)
Churcher et al 2016	Source (archive)
ClinicalTrials.gov registration of PBO net study	Source (archive)
Corbel et al 2010	Source (archive)
Deressa et al 2016	Source (archive)
GiveWell - Insecticide resistance adjustment incorporating PBO nets	Source
GiveWell's non-verbatim summary of a conversation with Dr. Thomas Churcher and Professor Hilary Ranson, May 31, 2017	Source
GiveWell's non-verbatim summary of a conversation with Hilary Ranson, Professor, Medical Entomology, Liverpool School of Tropical Medicine, March 18, 2016	Source
Hemingway et al 2016	Source (archive)
International Vector Control Coalition - Active Ingredient Portfolio - 2016	Source (archive)
IR Mapper	Source (archive)
IR Mapper – Data sources – 2016	Source (archive)
IR Mapper – How it works – 2016	Source (archive)
IRAC – Monitoring Methods – CDC Bottle Assay	Source (archive)
Jagannathan et al 2012	Source (archive)
Jones et al 2012	Source (archive)
Kleinschmidt et al 2015	Source (archive)
Kristan et al 2016	Source (archive)
Lindblade et al 2015	Source (archive)
Louis et al 2015	Source (archive)
Maharaj, Mthembu and Sharp 2005	Source (archive)
Malaria Journal	Source (archive)
Matowo et al. 2015	Source (archive)
Mawejje et al 2013	Source (archive)
Mukonka et al 2014	Source (archive)
Nardini 2014	Source (archive)
National Malaria Control Programme Uganda - An epidemiological profile of malaria and its control in Uganda	Source (archive)
Ngufor et al 2014	Source (archive)
Ochomo et al 2013	Source (archive)
Ochomo et al 2017	Source (archive)
Ojuka et al 2015	Source (archive)
Professor Hilary Ranson, email to GiveWell, July 29, 2018	Unpublished
Protopopoff et al. 2018	Source (archive)
Ranson and Lissenden 2016	Source (archive)
Reuters, article on Interceptor G2, July 2017	Source (archive)
Rob Mather, AMF Founder, conversation with GiveWell, June 18, 2018	Unpublished
Rob Mather, AMF Founder, email to GiveWell, April 17, 2018	Unpublished
Sinka et al 2012	Source (archive)
Strode et al 2014	Source (archive)
Test procedures for insecticide resistance monitoring in malaria vector mosquitoes - World Health Organization - 2013	Source (archive)
Tiono et al 2015	Source (archive)
Toé et al 2014	Source (archive)
Tokponnon et al 2014	Source (archive)
Trape et al 2011	Source (archive)
WHO Global Report on insecticide resistance 2010-2016	Source (archive)
World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017	Source (archive)
World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors	Source (archive)

• 1

  "Molecular genotyping of resistance is the identification of the underlying genes that confer the inherited trait of resistance (15). Identification of a resistance gene provides evidence of the underlying evolutionary process. Depending on the type of resistance mechanism, this provides understanding of both the degree of resistance expressed in individual insects with the resistance gene, and the frequency of such insects in the population.

  "Phenotypic resistance is the basic expression of the genetic cause of resistance, shown by a vector’s ability to resist and survive the effects of the insecticide. Phenotypic resistance is measured in a susceptibility test of vector mortality when subjected to a standard dose of the insecticide. WHO has defined phenotypic resistance as “development of an ability, in a strain of insects, to tolerate doses of toxicants, which would prove lethal to the majority of individuals in a normal population of the same species” (16). Phenotypic resistance is the phenomenon most commonly referred to in public health.

  "Resistance leading to control failure - while phenotypic resistance provides an indication of the effects of resistance on the vector, the most informative way of looking at resistance is as an epidemiological phenomenon, in which resistance is identified as the cause of increasing malaria transmission. In the notion of resistance leading to control failure, evidence of resistant vectors is linked directly to the failure of vector control programmes in the field. Resistance leading to control failure can be defined as the 'selection of heritable characteristics in insect population that results in repeated failure of an insecticide product to provide intended level of control when used as recommended.' Resistance leading to control failure is the phenomenon most commonly referred to in agriculture. National malaria control programmes should not, however, wait for control failure to occur before implementing strategies to manage insecticide resistance. There is no acceptable level of control failure in public health, and waiting could result in delaying action until it is too late." World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors, Pg 27.

• 2

  "Target-site resistance occurs when the site of action of an insecticide (typically within the nervous system) is modified in resistant strains, such that the insecticide no longer binds effectively and the insect is therefore unaffected, or less affected, by the insecticide. Resistance mutations, known as knock-down resistance (kdr) mutations, can affect acetylcholinesterase, which is the molecular target of organophosphates and carbamates, or voltage-gated sodium channels (for pyrethroids and DDT) (15, 17).

  "Metabolic resistance is related to the enzyme systems that all insects possess to detoxify foreign materials. It occurs when increased or modified activities of an enzyme system prevent the insecticide from reaching its intended site of action. The three main enzyme systems are: esterases, mono-oxygenases and glutathione S-transferases. While metabolic resistance is important for all four insecticide classes, different enzymes affect different classes (15, 17).

  "Although most resistance mechanisms (especially kdr resistance) have been studied for decades in previous cases of resistance, the detailed study of mono-oxygenase metabolic resistance is relatively new, and our understanding of it is fairly limited. Indeed, cases of mono-oxygenase resistance in mosquitoes were unknown before its identification in South Africa in 2000–2001 (see section 1.2.3 for details).

  "As described below, metabolic and target site resistance can both occur in the same vector population and sometimes within the same individual mosquito. The two types of resistance appear to have different capacities to reduce the effectiveness of insecticide-based vector control interventions, with metabolic resistance being the stronger and more worrying mechanism (see section 1.2.3 for details).

  "Behavioural resistance is any modification in insect behaviour that helps it to avoid the lethal effects of insecticides. Several publications have suggested the existence of behavioural resistance and described changes in vectors’ feeding or resting behaviour to minimize contact with insecticides. Studies in New Guinea and the Solomon Islands showed that Anopheles farauti vectors stopped biting later in the night (23:00–03:00) after the introduction of indoor DDT spraying and instead bit only in the earlier part of the evening, before humans were protected by sleeping in a sprayed room (18). In most cases, however, there are insufficient data to assess whether behavioural avoidance traits are genetic or adaptive; genetic traits could have major implications for the types of vector control interventions needed. All behavioural traits, however, may not be negative, as they could lead mosquitoes to feed on non-human animals. It is also possible to initially mistake the decline of a vector species as behavioural resistance.

  "Cuticular resistance is reduced uptake of insecticide due to modifications in the insect cuticle that prevent or slow the absorption or penetration of insecticides. Examples of reduced penetration mechanisms are extremely limited and only one study has suggested correlation between cuticle thickness and pyrethroid resistance in An. funestus (19). Microarray experiments have identified two genes that encode cuticular proteins that are up-regulated in pyrethroid-resistant strains of Anopheles mosquitoes. Experience with other insects suggests that if cuticular resistance emerges in mosquitoes it could have a significant impact when combined with other resistance mechanisms." World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors, Pg 27-28.

• 3
  World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors, Pg. 28.
  
• 4

  "Although there are limited options for [resistance management] with LLINs, they may retain an effect despite increased resistance to pyrethroids. Firstly, nets provide a physical barrier against biting by mosquitoes as long as they are intact (2). Secondly, in most vector species, resistance to pyrethroids does not completely reduce the effect of the insecticide. It has also been observed that the irritancy of pyrethroids (‘hyperexcitatory response’) may reduce mosquito blood-feeding or encourage diversion to other hosts by certain vector species that do not feed exclusively on human hosts. This effect can vary, however, by species and geographical location." World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors, Pg 45.

• 5
  • "Possible reasons for widespread insecticide resistance with no obvious impact on the effectiveness of vector control:…
    • "Vectors are still killed because of multiple exposures to an insecticide in the field. Vectors may still be killed by an insecticide if they come into contact with it on many occasions within a short time (6). While a single standard dose of insecticide might not have the desired effect, multiple exposures in the field (for example landing on a sprayed surface three to four times) can lead to a higher accumulated dose of the insecticide, which may be sufficient to kill the vector. Similarly, vectors that are resistant may have a lower excito-repellent response when exposed to pyrethroids. Therefore, they may remain exposed to the insecticide for longer and receive a larger dose, and thus be killed anyway (3).
    • "Older vectors are more susceptible. Metabolic resistance is in some cases expressed more strongly in young mosquitoes, but, when they are older, susceptibility returns. Therefore, insecticides still kill older mosquitoes (7). Since it is these old mosquitoes that transmit malaria, the resistance may not cause an increase in transmission.
    • "Resistant vectors are less capable of transmitting malaria. It has been suggested that the ability of resistant vectors to transmit malaria might be reduced as, even if they are not killed by an insecticide within 24 hours, the insecticide might still inhibit their ability to live for the 12 days necessary to develop the malaria parasite. Alternatively, a vector's resistance to an insecticide might be offset by a lower level of infection, which is not as great a threat to the human population." World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors, Pg 111.
  • We are aware of some entomological evidence from Kristan et al 2016 that suggests that resistant mosquitoes may be less fit to transmit the malaria parasite (see below). However, Ranson and Lissenden 2016, Pg. 8 states that "[r]elatively little is known about the fitness costs of resistance." We have not looked further into the entomological evidence about whether resistance inhibits malaria transmission.
    • "We collected An. gambiae s.s. homozygous for kdr-L1014S mutation, fed them on blood samples from 42 P. falciparum-infected local patients, then exposed them either to nets treated with sub-lethal doses of deltamethrin or to untreated nets. ... Both prevalence and intensity of infection were significantly reduced in deltamethrin-exposed mosquitoes, compared to those exposed to untreated nets. With low doses (2.5–5.0 mg/m2), prevalence was reduced by 59 % (95 % CI = 22 %-78 %) and intensity by 41 % (95 % CI = 25 %-54 %). With high doses (10–16.7 mg/m2), prevalence was reduced by 80 % (95 % CI=67 %-88 %) and intensity by 34 % (95 % CI=20 %-46 %). ... Conclusions: We showed that, with locally-sampled parasites and mosquitoes, doses of pyrethroids that are sub-lethal for resistant mosquitoes can interfere with parasite development inside mosquitoes. This mechanism could enable pyrethroid-treated nets to prevent malaria transmission despite increasing vector resistance." Kristan et al 2016, Pg 1.

• 6

  "The development of malaria parasites into the infectious sporozoite stage takes 10 or more days following a blood-meal [35]. The older cohort of An. gambiae s.s. used in this study (17-19 days old) could potentially harbour infectious parasites but would be more susceptible to insecticide-based control. Targeting older, and arguably more epidemiologically significant mosquitoes, has been proposed as an alternative strategy for wider malaria control in which the selection on resistance to the control agent is reduced [36]." Jones et al 2012, Pg. 8-10.

• 7

  World Health Organization's 2012 Global Plan for Insecticide Resistance Management in Malaria Vectors, Pg 36.

• 8
  • “Pyrethroids are the only class of insecticide recommended by WHO for use on longlasting insecticide-treated bednets.” Hemingway et al 2016, Pg. 1.
  • We are aware of a new net (Interceptor G2) which uses the insecticide chlorfenapyr and has received an "interim recommendation" from the WHO. We have not fully investigated this new net but one news article reports that "a WHO spokesman said the Geneva-based organization’s interim recommendation meant it still had to evaluate the net’s public health impact and it was requesting more data from the chemicals company." See Reuters, article on Interceptor G2, July 2017. We are not yet aware of organizations that use this net on a large scale.

• 9
  • “The cornerstone to IVCC’s mission is the development of three new active ingredients, each with a novel mode of action, in effect, a different way of killing mosquitoes. These will form the basis of a new generation of insecticides suitable for public health purposes.” International Vector Control Coalition - Active Ingredient Portfolio - 2016.
  • “With no new insecticide class to replace the pyrethroids expected for a decade, the threat of resistance derailing malaria control has become an issue of urgency that can no longer be ignored without risking a global public health catastrophe.” Hemingway et al 2016, Pg. 1.
  • We are aware of a new net (Interceptor G2) which uses the insecticide chlorfenapyr and has received an "interim recommendation" from the WHO. We have not fully investigated this new net but one news article reports that "a WHO spokesman said the Geneva-based organization’s interim recommendation meant it still had to evaluate the net’s public health impact and it was requesting more data from the chemicals company." See Reuters, article on Interceptor G2, July 2017. We are not yet aware of organizations that use this net on a large scale.

• 10

  "Furthermore, by using prevalence of resistance as the metric, it is not possible to identify regions where resistance is likely to be posing the greatest threat to malaria control. Mosquitoes collected from site A may yield 50 % mortality in a discriminating dose assay whereas mosquitoes from site B may have been just under the threshold with 85 % mortality. This does not however mean that resistance is less of a threat in site B. The 15 % that did survive may have an extremely high level of resistance enabling them to readily survive long periods of time on a treated surface and thus potentially transmit malaria despite high coverage with vector control. In contrast, if the 50 % that survived the discriminating dose in site A have a relatively weak phenotype they will be killed when exposed to field concentrations of insecticide and thus pose less of an immediate threat." Bagi et al 2015, Pg. 2.

• 11

  WHO bioassays place age-standardized mosquitoes in tubes and expose them to paper impregnated with twice the minimum concentration of insecticide known to kill 100% of susceptible mosquitoes. Exposure lasts 60 minutes, and the number of mosquito fatalities is recorded after 24 hours. The population is classified as susceptible if >98% of mosquitoes are killed, and resistant if <90% of mosquitoes are killed. Values between 90% and 98% are considered “suspected resistance,” and further tests are recommended. We do not know how the amount of insecticide that a mosquito is exposed to in a bioassay compares to the amount of insecticide that a mosquito would receive from its typical interaction with an ITN. Sources:

  • “The WHO insecticide susceptibility bioassay that is the subject of these revised guidelines is a simple direct response-to-exposure test. Mosquitoes are exposed to known concentrations of an insecticide for a fixed period of time at the end of which the number of fatalities is recorded. In its present form, the test is designed to distinguish between baseline susceptibility and resistance to insecticides in adult mosquitoes. As such, the test is intended to be used as a field and laboratory surveillance tool with the limitation that it gives little information on the underlying mode(s) or mechanism(s) conferring resistance where detected.” Test procedures for insecticide resistance monitoring in malaria vector mosquitoes - World Health Organization - 2013, Pg. 4.
  • "In order to be certain that all susceptible mosquitoes are killed, WHO has traditionally defined its discriminating concentrations in one of two ways, that is, as either:
    • twice the lowest concentration that gave systematically 100% mortality after 60 minutes exposure and a holding period of 24 hours on a susceptible strain or a susceptible population; or
    • twice the LC99.9 value as determined by baseline susceptibility testing of a susceptible strain or a susceptible population.

    Table 3.1 lists the WHO recommended discriminating concentrations for insecticides used in malaria control and/or for research purposes (e.g. dieldrin) for adult malaria vectors." Test procedures for insecticide resistance monitoring in malaria vector mosquitoes - World Health Organization - 2013, Pg. 6.

  • “A mortality in the range 98–100% indicates susceptibility”; “A mortality of less than 98% is suggestive of the existence of resistance and further investigation is needed”; “If mortality is less than 90%, confirmation of the existence of resistant genes in the test population with additional bioassays may not be necessary, as long as a minimum of 100 mosquitoes of EACH species was tested. However, further investigation of the mechanisms and distribution of resistance should be undertaken.” Test procedures for insecticide resistance monitoring in malaria vector mosquitoes - World Health Organization - 2013, Pg. 18.

  Note that information about the three main kinds of tests for insecticide resistance – cone or tube tests (such as these), tunnel tests and experimental hut trials is available at: "Box 1. Types of Studies Included," Strode et al 2014, Pg 2.

• 12

  See the data collected in IR Mapper, much of which relies on WHO bioassays.

• 13
  • "World Health Organization (WHO)-standard bioassays ... don’t measure the degree of resistance. ... Recently researchers have pushed for bioassay data that is useful in predicting the impact of resistance rather than just monitoring its spread. Some (including those working on the President’s Malaria Initiative (PMI) five-country investigation of the impact of insecticide resistance on malaria vector control) have begun to use more quantitative bioassays that expose mosquitoes to increasing doses of insecticide to determine how much is needed to exterminate a given population." GiveWell's non-verbatim summary of a conversation with Hilary Ranson, Professor, Medical Entomology, Liverpool School of Tropical Medicine, March 18, 2016.
  • "[WHO] standardized methodology is useful for tracking the spread of resistance but does not provide information on the strength of this resistance or its impact. The concentration of insecticide used has no relationship to the quantity of insecticide used in field applications but is instead set as twice the concentration required to kill a susceptible strain of the same species." Bagi et al 2015, Pg. 2.

• 14

  "In 2011 and 2012, the 50% lethality time (LT50) for the VK7 strain of An. gambiae mosquitoes was determined by varying the length of exposure time (60–600 min). The mean mortality rate was recorded per time point, and the LT50 was estimated by fitting a logistic regression model by using logit-transformed probabilities (10) in R statistical software (http://www.r-project.org)." Toé et al 2014, Pg. 1692.

• 15

  "In Prof. Ranson’s opinion, researchers have relied for too long on World Health Organization (WHO)-standard bioassays. These bioassays are designed so that exposure to a given level of insecticide kills 100 percent of a susceptible mosquito population (any survivors indicate resistance), which makes them useful tools for detecting the appearance of resistance in a population. However, they don’t measure the degree of resistance.
  "Recently researchers have pushed for bioassay data that is useful in predicting the impact of resistance rather than just monitoring its spread. Some (including those working on the President’s Malaria Initiative (PMI) five-country investigation of the impact of insecticide resistance on malaria vector control) have begun to use more quantitative bioassays that expose mosquitoes to increasing doses of insecticide to determine how much is needed to exterminate a given population." GiveWell's non-verbatim summary of a conversation with Hilary Ranson, Professor, Medical Entomology, Liverpool School of Tropical Medicine, March 18, 2016.

• 16
  • Experimental hut trials are conducted as follows: "Methods: Studies in areas where mosquitoes breed. Volunteers sleep in experimental huts for a specific period under an ITN or an UTN, with one hut per person. The huts are identical in construction, and incorporate exit traps to catch wild mosquitoes entering and exiting the hut prematurely. Each morning of the trial, both live and dead mosquitoes, blood fed and non-blood fed, are collected and counted from both inside the hut and the exit traps. Live mosquitoes are monitored for a further 24 h to assess delayed mortality. Volunteers and nets are randomly allocated to huts at the start of the trial and are usually rotated to avoid bias. Often huts are cleaned between rotations to avoid cross-contamination of huts from the different treatment arms [25]." Strode et al 2014, Pg. 2.
  • "Perhaps of more direct relevance in decision making are assays that measure the response of local vectors to locally implemented vector control tools. The simplest example of this is the cone bioassay in which mosquitoes are exposed to a bed net or a sprayed wall for a fixed exposure time and then mortality recorded 1 h and 24 h after exposure [15]. Results from cone bioassays paint an alarming picture, with very low kill rates being observed even after exposure to new nets or freshly sprayed surfaces in several settings [8,11,13,16–18]." Ranson and Lissenden 2016, Pg. 3.

• 17

  “The distribution of pyrethroid resistance in African malaria vectors was described in a 2011 article in this journal [3]. At this time, pyrethroid-resistant populations of Anopheles gambiae were prevalent in western and central Africa but were rarer in southern and eastern countries of the continent. Pyrethroid resistance is now widely spread across the continent with An. gambiae in Kenya, much of Tanzania, Zambia, and Zimbabwe resistant to this insecticide class. Data remain scarce for much of central Africa, although reports are emerging of pyrethroid resistance across the Democratic Republic of Congo [4]. In summary, although An. gambiae populations fully susceptible to pyrethroids are still present in 2015 [e.g., in parts of Angola, Madagascar, and Mozambique...] they are becoming increasingly outnumbered by resistant populations (Figure 1A).

  "Data on resistance in Anopheles funestus remain limited (Figure 1B) but, whereas previously pyrethroid resistance in this species was thought to be restricted to southern Africa, it has now been detected in Uganda [5], Kenya [5], Benin [6], and Cameroon [7]. As with An. gambiae, fully susceptible populations of An. funestus have been reported in some areas of Mozambique (Figure 2D) but resistance is very prevalent in other areas of the country [8]. Furthermore, the trend is similar to that observed in An. gambiae, with susceptible populations becoming the exception rather than the norm.” Ranson and Lissenden 2016, Pgs. 1-2.

• 18
  • ”Data are extracted monthly from peer reviewed scientific publications and other published reports including the President’s Malaria Initiative Country Insecticide Susceptibility Summaries and IRBase (a section of VectorBase). Note: some countries have no published records on insecticide resistance and hence data are not currently reported on IR Mapper." IR Mapper – Data sources – 2016.
  • "IR Mapper is a joint initiative. Data collection and collation is performed by Vestergaard, data proofreading is conducted by KEMRI/CDC and the interactive map platform was developed by ESRI Eastern Africa (collectively “IR Mapper”)." IR Mapper – Data sources – 2016.

• 19
  • "Alternative measures of resistance in which the intensity or strength of the resistance is measured have been described [11] and are being adopted in a small number of field studies [12–14]. Again, longitudinal monitoring of resistance intensity can provide an important insight on the rapid changes occurring in malaria vectors. For example, the exposure time required to kill 50% of the An. gambiae population in an area in southwestern Burkina Faso was found to increase tenfold over a single year [13]." Ranson and Lissenden 2016, Pg. 3.
  • "The resistance levels we report in the current study are alarming. Because few studies have attempted to quantify resistance strength in field populations, it is difficult to know if this extreme resistance phenotype is exceptional or symptomatic of the status of pyrethroid resistance in malaria vectors in Africa. Two other studies have used the LT50 method to assess the strength of resistance to pyrethroids in field populations compared with susceptible (laboratory raised) strains. In 2011, deltamethrin resistance ratios of 138-fold were recorded in Tiassalé, Côte d’Ivoire (13) and 292-fold in Jinja, Uganda (16). Thus, to our knowledge, deltamethrin resistance levels of 730-fold in 2012 (estimated by LT50) and >1,000 fold in 2013 (estimated by LD50) reported in the current study are the highest in the published literature." Toé et al 2014, Pg. 1694.
  • The two other studies referenced in Toé et al 2014 are Mawejje et al 2013 and Choi et al 2014.

• 20
  • “In 2011 and 2012, the 50% lethality time (LT50) for the VK7 strain of An. gambiae mosquitoes was determined by varying the length of exposure time (60–600 min). The mean mortality rate was recorded per time point, and the LT50 was estimated by fitting a logistic regression model by using logit-transformed probabilities (10) in R statistical software (http://www.r-project.org).
    "In 2013, CDC bottle bioassays were used to quantify the level of resistance to deltamethrin. Glass 250-mL bottles were coated with different concentration of deltamethrin ranging from 3.125 µg/mL to 125 µg/mL at CNRFP. Bottles were prepared according to CDC guidelines (11). Female mosquitoes (3–5 days) were aspirated into bottles for 1 h and subsequently transferred to insecticide-free paper cups for 24 h of observation. Four to six replicates were performed for each concentration and for the control bottles (impregnated with acetone). Equivalent age mosquitoes of the Kisumu strain were exposed to various insecticide concentrations (range 0.001 µg/mL–0.5 µg/mL). The 50% lethal dose (LD50) was determined by using R statistical software.” Toé et al 2014, Pg. 1692.
  • "The resistance levels we report in the current study are alarming. Because few studies have attempted to quantify resistance strength in field populations, it is difficult to know if this extreme resistance phenotype is exceptional or symptomatic of the status of pyrethroid resistance in malaria vectors in Africa. Two other studies have used the LT50 method to assess the strength of resistance to pyrethroids in field populations compared with susceptible (laboratory raised) strains. In 2011, deltamethrin resistance ratios of 138-fold were recorded in Tiassalé, Côte d’Ivoire (13) and 292-fold in Jinja, Uganda (16). Thus, to our knowledge, deltamethrin resistance levels of 730-fold in 2012 (estimated by LT50) and >1,000 fold in 2013 (estimated by LD50) reported in the current study are the highest in the published literature." Toé et al 2014, Pg. 1694.

• 21
  • "The resistance levels we report in the current study are alarming. Because few studies have attempted to quantify resistance strength in field populations, it is difficult to know if this extreme resistance phenotype is exceptional or symptomatic of the status of pyrethroid resistance in malaria vectors in Africa. Two other studies have used the LT50 method to assess the strength of resistance to pyrethroids in field populations compared with susceptible (laboratory raised) strains. In 2011, deltamethrin resistance ratios of 138-fold were recorded in Tiassalé, Côte d’Ivoire (13) and 292-fold in Jinja, Uganda (16). Thus, to our knowledge, deltamethrin resistance levels of 730-fold in 2012 (estimated by LT50) and >1,000 fold in 2013 (estimated by LD50) reported in the current study are the highest in the published literature." Toé et al 2014, Pg. 1694.
  • "[N]o major difference was seen in pyrethroid mortality rates over a 3-year period with a fixed exposure of insecticide. However, when exposure time or concentration was varied, increases in the strength of the resistance were observed." Toé et al 2014, Pg. 1694.

• 22
  • "In Africa, An. gambiae, An. arabiensis and An. funestus are co-dominant across much of the continent, whereas in the Asian-Pacific region there is a highly complex situation with multi-species coexistence and variable species dominance." Sinka et al 2012, Pg. 1.
  • On diagnostic test outcomes, see Figure 1, Ranson and Lissenden 2016, Pg. 2.

• 23

  See Figure 2, Ranson and Lissenden 2016, Pg. 3.

• 24
  • "Data are extracted monthly from peer reviewed scientific publications and other published reports including the President’s Malaria Initiative Country Insecticide Susceptibility Summaries and IRBase (a section of VectorBase). Note: some countries have no published records on insecticide resistance and hence data are not currently reported on IR Mapper." IR Mapper – Data sources – 2016.
  • "IR Mapper is a joint initiative. Data collection and collation is performed by Vestergaard, data proofreading is conducted by KEMRI/CDC and the interactive map platform was developed by ESRI Eastern Africa (collectively “IR Mapper”)." IR Mapper – Data sources – 2016.

• 25

  See, e.g.:

  • "Malaria control is reliant on insecticides to control the mosquito vector. As efforts to control the disease have intensified, so has the selection pressure on mosquitoes to develop resistance to these insecticides. The distribution and strength of this resistance has increased dramatically in recent years and now threatens the success of control programs.” Ranson and Lissenden 2016, Pg. 1.
  • "Pyrethroid resistance was first detected in the two major malaria vectors Anopheles gambiae s.l. and Anopheles funestus in a small number of sites at the end of the last century [2–4]. The 21st century has witnessed a rapid spread of this resistance phenotype across the continent and it is now difficult to find sites in Africa where both vectors remain fully susceptible to pyrethroids [5]," Bagi et al 2015, Pg. 2.

• 26

  "Researchers lack a clear understanding of both the extent and the clinical impact of insecticide resistance. In particular, there are not any high-quality studies that enable researchers to confidently estimate the expected impact of current levels of insecticide resistance on clinical outcomes (e.g., mortality due to malaria)." GiveWell's non-verbatim summary of a conversation with Hilary Ranson, Professor, Medical Entomology, Liverpool School of Tropical Medicine, March 18, 2016. This quote does not provide direct evidence for the claim that RCTs have not been done because of ethical restrictions, but it supports the claim that high-quality studies have not been done on this topic.

• 27

  "[I]t is a recognised ethical principle that patients should receive the best proven standard of care whenever feasible." Nardini 2014, Pg. 5.

• 28

  “These examples do not suggest that insecticide resistance does not pose a substantial and real threat to malaria vector control, but rather they demonstrate how difficult it is to assess the evidence of epidemiological impact of insecticide resistance when relying on historical data. To address this, a multi-country prospective study to assess the impact of insecticide resistance on the effectiveness of long lasting insecticidal nets (LLINs) and IRS was initiated in five countries, namely Benin, Cameroon, India, Kenya and Sudan, co-ordinated by the World Health Organization (WHO) and with primary funding from the Bill and Melinda Gates Foundation. This paper describes the design that was adopted for conducting this study.” Kleinschmidt et al 2015

• 29

  See:

  • Lindblade et al 2015 (in Malawi):
    • On sample size: "A fixed cohort of 1,199 children aged six to 59 months was seen monthly for one year and at sick visits to measure malaria infection and use of ITNs." Pg. 1
      "Table 3, Predictors of malaria incidence in a fixed cohort of 1,199 children age six to 59 months at baseline, Liwonde, Malawi 2012- 2013 ... Characteristic Bed net use ITN 1710 UTN 92 No bed net 86," Pg. 11.
    • On methods and results: "Methods: One deltamethrin ITN was distributed in the study area for every two individuals in each household plus one extra ITN for households with an odd number of residents. A fixed cohort of 1,199 children aged six to 59 months was seen monthly for one year and at sick visits to measure malaria infection and use of ITNs. Insecticide resistance among malaria vectors was measured. The effect of ITN use on malaria incidence was assessed, adjusting for potential confounders using generalized estimating equations accounting for repeated measures.
      "Results: There were 1,909 infections with Plasmodium falciparum over 905 person-years at risk (PYAR), resulting in an observed incidence of 2.1 infections per person-year (iPPY). ITNs were used during 97% of the PYAR. The main vector was Anopheles funestus: mortality in WHO tube assays after exposure to 0.05% deltamethrin was 38% (95% confidence interval (CI) 29–47), and resistance was due to elevated oxidase enzymes. After adjusting for potential confounders, the incidence of malaria infection among ITN users was 1.7 iPPY (95% CI 1.5-2.1) and among non-bed net users was 2.6 iPPY (95% CI 2.0-3.3). Use of ITNs reduced the incidence of malaria infection by 30% (rate ratio 0.7; 95% CI, 0.5-0.8) compared to no bed nets." Pg. 1.
  • Asidi et al 2012 (in Benin):
    • On sample size: "Eleven households at the southern sites (where mosquitoes are resistant) and 5 households at the northern site (where mosquitoes are susceptible) participated in the study." Pg. 1102.
    • On methods: "In this comparative study in areas of contrasting pyrethroid resistance and susceptibility, we used vector blood feeding as a surrogate for malaria risk and demonstrated that ITNs lose their capacity to provide protection once An. gambiae M form develops pyrethroid resistance of the type found in southern Benin. These findings clearly show that ITNs in local use fail to protect against An. gambiae populations that contain kdr resistance at high frequency. The mechanisms of resistance in southern Benin are complex, and metabolic resistance appears to contribute. The demonstration of contrasting blood-feeding and survival rates between resistant and susceptible An. gambiae mosquitoes in the household trial corroborates findings and predictions from earlier experimental hut studies in southern and northern Benin and confirms the veracity of experimental huts as a tool for predicting protection or control in the home." Pg. 1104.
  • Ochomo et al 2017 (in Kenya): "Insecticide resistance might reduce the efficacy of malaria vector control. In 2013 and 2014, malaria vectors from 50 villages, of varying pyrethroid resistance, in western Kenya were assayed for resistance to deltamethrin. Long-lasting insecticide-treated nets (LLIN) were distributed to households at universal coverage. Children were recruited into 2 cohorts, cleared of malaria-causing parasites, and tested every 2 weeks for reinfection. Infection incidence rates for the 2 cohorts were 2.2 (95% CI 1.9–2.5) infections/person-year and 2.8 (95% CI 2.5–3.0) infections/person-year. LLIN users had lower infection rates than non-LLIN users in both low-resistance (rate ratio 0.61, 95% CI 0.42–0.88) and high-resistance (rate ratio 0.55, 95% CI 0.35–0.87) villages (p = 0.63). The association between insecticide resistance and infection incidence was not significant (p = 0.99). Although the incidence of infection was high among net users, LLINs provided significant protection (p = 0.01) against infection with malaria parasite regardless of vector insecticide resistance." Abstract.
  • Bradley et al 2017 (in Benin): "Cross sectional surveys were carried out in each cluster in July 2015. In each cluster forty households were randomly selected from a list (which came from census carried out in 2013). Each household was visited, and if anyone was present, written informed consent was sought to participate in the study. An adult was asked questions about socio economic indicators and bed net use. Up to 3 children per household aged between 6 months and 10 years were chosen at random for malaria microscopy. Blood slides were read by 2 microscopists and if they disagreed on the presence of malaria parasites, a third reader adjudicated. Those testing positive were traced and treated according to national guidelines or referred to a local health facility." Pg. 2.

• 30

  Ochomo et al 2017:

  • "Insecticide resistance might reduce the efficacy of malaria vector control. In 2013 and 2014, malaria vectors from 50 villages, of varying pyrethroid resistance, in western Kenya were assayed for resistance to deltamethrin. Long-lasting insecticide-treated nets (LLIN) were distributed to households at universal coverage. Children were recruited into 2 cohorts, cleared of malaria-causing parasites, and tested every 2 weeks for reinfection. Infection incidence rates for the 2 cohorts were 2.2 (95% CI 1.9–2.5) infections/person-year and 2.8 (95% CI 2.5–3.0) infections/person-year. LLIN users had lower infection rates than non-LLIN users in both low-resistance (rate ratio 0.61, 95% CI 0.42–0.88) and high-resistance (rate ratio 0.55, 95% CI 0.35–0.87) villages (p = 0.63). The association between insecticide resistance and infection incidence was not significant (p = 0.99). Although the incidence of infection was high among net users, LLINs provided significant protection (p = 0.01) against infection with malaria parasite regardless of vector insecticide resistance." Abstract
  • The study included 2 cohorts of about 1,000 children each (see Table 1, Pg. 760). "Approximately 1,000 children were recruited into each active infection cohort. The median age of children at recruitment was 2.5 years for cohort 1 and 2.2 years for cohort 2. For cohort 1, each child was followed for 80 days, and a total of 279 infections were detected; for cohort 2, each child was followed for 95 days, and a total of 483 infections were detected (Table 1). LLIN use was 81.3% for cohort 1 and 85.7% for cohort 2. The overall incidence rate of infection with the malaria parasite was 2.2 (95% CI 1.9–2.5) infections/person-year for cohort 1 and 2.8 (95% CI 2.5–3.0) infections/person-year for cohort 2. The subcounty-specific infection incidences were 1.2–3.0 infections/person-year in cohort 1 and 1.8–4.1 infections/person-year in cohort 2 (Table 2)." Pg. 760
  • "We conducted this study in 4 malaria-endemic subcounties in western Kenya described previously (22). In brief, in 2014, the National Malaria Control Programme conducted a massive campaign to distribute nets; a mix of PermaNet 2.0 (treated with deltamethrin) and Olyset nets (treated with permethrin) were distributed in the 4 subcounties Bondo, Teso, Rachuonyo, and Nyando to meet the universal coverage threshold of 1 net per 2 persons. Subsequent routine distribution was conducted through health facilities to pregnant women and children <5 years of age. Twenty sublocations (hereafter referred to as clusters) were randomly selected from each of the 4 subcounties where the initial insecticide resistance assessment was conducted (21,22). After the assessment, the clusters were categorized into 3 groups: those with >80% mosquito mortality to deltamethrin or permethrin (categorized as low-resistance clusters), those with mosquito mortality <80% but >60% (categorized as medium-resistance clusters), and those with mosquito mortality <60% (categorized as high-resistance clusters). Finally, 13 low- and high-resistance clusters were selected in Rachuonyo, 11 in Teso, 16 in Bondo, and 10 in Nyando, giving a total of 50 clusters for subsequent studies. Each cluster had 10–30 villages, each with ~100 households. We recruited children 6–59 months of age from households immediately around larval habitats that were sampled by the entomology teams for assessing insecticide resistance; moving out in concentric circles from the larva habitats, we recruited study participants until 20 eligible and consenting households were enrolled." Pg. 759

• 31

  Ochomo et al 2017:

  • "Insecticide resistance might reduce the efficacy of malaria vector control. In 2013 and 2014, malaria vectors from 50 villages, of varying pyrethroid resistance, in western Kenya were assayed for resistance to deltamethrin. Long-lasting insecticide-treated nets (LLIN) were distributed to households at universal coverage. Children were recruited into 2 cohorts, cleared of malaria-causing parasites, and tested every 2 weeks for reinfection. Infection incidence rates for the 2 cohorts were 2.2 (95% CI 1.9–2.5) infections/person-year and 2.8 (95% CI 2.5–3.0) infections/person-year. LLIN users had lower infection rates than non-LLIN users in both low-resistance (rate ratio 0.61, 95% CI 0.42–0.88) and high-resistance (rate ratio 0.55, 95% CI 0.35–0.87) villages (p = 0.63). The association between insecticide resistance and infection incidence was not significant (p = 0.99). Although the incidence of infection was high among net users, LLINs provided significant protection (p = 0.01) against infection with malaria parasite regardless of vector insecticide resistance." Abstract
  • "Population-based active surveillance can complement routine passive sentinel surveillance systems by providing public health data and insights into the complex epidemiology of disease. Active infection-detection cohorts are studies that involve clearing participants of infections, following them up, and testing them at regular intervals, regardless of whether they are symptomatic, until the first infection appears, at which point the follow-up is discontinued. These types of studies provide estimates of time to infection in participants and are useful because they enable estimations of various parameters associated with disease. The main advantage of using population-based malaria parasite surveillance is that it provides the data needed to determine the infection rate and the populations at risk for infection.

    This study was implemented as part of a large, multicountry program set up to quantify the impact of insecticide resistance on the effectiveness of insecticide-based vector controls. To determine if insecticide resistance altered the effectiveness of LLINs in malaria endemic subcounties of western Kenya, we conducted population-based malaria parasite active infection-detection cohort studies." Pg. 759

• 32

  Ochomo et al 2017:

  • "Insecticide resistance might reduce the efficacy of malaria vector control. In 2013 and 2014, malaria vectors from 50 villages, of varying pyrethroid resistance, in western Kenya were assayed for resistance to deltamethrin. Long-lasting insecticide-treated nets (LLIN) were distributed to households at universal coverage. Children were recruited into 2 cohorts, cleared of malaria-causing parasites, and tested every 2 weeks for reinfection. Infection incidence rates for the 2 cohorts were 2.2 (95% CI 1.9–2.5) infections/person-year and 2.8 (95% CI 2.5–3.0) infections/person-year. LLIN users had lower infection rates than non-LLIN users in both low-resistance (rate ratio 0.61, 95% CI 0.42–0.88) and high-resistance (rate ratio 0.55, 95% CI 0.35–0.87) villages (p = 0.63). The association between insecticide resistance and infection incidence was not significant (p = 0.99). Although the incidence of infection was high among net users, LLINs provided significant protection (p = 0.01) against infection with malaria parasite regardless of vector insecticide resistance." Abstract
  • "We found no association between malaria parasite infection incidence and insecticide resistance when comparing high- and low-resistance clusters. For cohort 1, incidence was 2.2 (95% CI 1.8–2.7) infections/person-year among children living in low-resistance clusters and 2.0 (95% CI 1.6–2.4) infections/person-year among children living in high-resistance clusters (adjusted RR 0.9, 95% CI 0.5–1.6; p = 0.68) (Table 4). For cohort 2, infection incidence was 2.8 (95% CI 2.4–3.2) infections/person-years among children residing in low-resistance clusters and 2.7 (95% CI 2.4–3.1) infections/person-years among children residing in high-resistance clusters (adjusted RR 0.8, 95% CI 0.5–1.2; p = 0.33). After plotting data from 93/100 clusters (data from all subcounties and both years), we found no association between deltamethrin insecticide resistance and malaria parasite infection incidence (Figure 1)." Pg. 761

• 33

  Ochomo et al 2017:

  • "Insecticide resistance might reduce the efficacy of malaria vector control. In 2013 and 2014, malaria vectors from 50 villages, of varying pyrethroid resistance, in western Kenya were assayed for resistance to deltamethrin. Long-lasting insecticide-treated nets (LLIN) were distributed to households at universal coverage. Children were recruited into 2 cohorts, cleared of malaria-causing parasites, and tested every 2 weeks for reinfection. Infection incidence rates for the 2 cohorts were 2.2 (95% CI 1.9–2.5) infections/person-year and 2.8 (95% CI 2.5–3.0) infections/person-year. LLIN users had lower infection rates than non-LLIN users in both low-resistance (rate ratio 0.61, 95% CI 0.42–0.88) and high-resistance (rate ratio 0.55, 95% CI 0.35–0.87) villages (p = 0.63). The association between insecticide resistance and infection incidence was not significant (p = 0.99). Although the incidence of infection was high among net users, LLINs provided significant protection (p = 0.01) against infection with malaria parasite regardless of vector insecticide resistance." Abstract
  • "We found no association between malaria parasite infection incidence and insecticide resistance when comparing high- and low-resistance clusters. For cohort 1, incidence was 2.2 (95% CI 1.8–2.7) infections/person-year among children living in low-resistance clusters and 2.0 (95% CI 1.6–2.4) infections/person-year among children living in high-resistance clusters (adjusted RR 0.9, 95% CI 0.5–1.6; p = 0.68) (Table 4). For cohort 2, infection incidence was 2.8 (95% CI 2.4–3.2) infections/person-years among children residing in low-resistance clusters and 2.7 (95% CI 2.4–3.1) infections/person-years among children residing in high-resistance clusters (adjusted RR 0.8, 95% CI 0.5–1.2; p = 0.33). After plotting data from 93/100 clusters (data from all subcounties and both years), we found no association between deltamethrin insecticide resistance and malaria parasite infection incidence (Figure 1)." Pg. 761

• 34

  Ochomo et al 2017:

  • "Our study had weaknesses that might have affected results, the first being the highly variable nature of the susceptibility data from 1 year to the next and from 1 cluster to the next. As mentioned previously, the WHO tube bioassay is not very informative of the intensity of insecticide resistance. The categorization of net users and non–net users might have substantially confounded results given that net use was not randomly assigned and non–net users were a relatively small number of children who did not prefer to use nets. Last, our study did not consider insecticide resistance in the population of An. funestus mosquitoes, a reemerging vector in the region, mostly because of the difficulty of rearing them in the lab and finding them in larval habitats." Pgs. 762-763
  • "Several factors might explain why we did not observe a correlation between insecticide resistance and malaria parasite infection incidence. First, as previously mentioned, LLINs serve as a barrier to prevent human–vector contact. If the nets are in good condition and are used consistently and properly, they reduce the chances of mosquito bites and hence malaria parasite transmission. Second, the WHO tube bioassay does not indicate what level of insecticide resistance is expected to lead to vector control failure, which is a major weakness of the assay. Therefore, even though we observe insecticide resistance, the mosquito populations might still be susceptible to the toxic effects of the chemical doses used on the nets. This highlights the need for more quantitative methods for monitoring insecticide resistance. In addition, a recent study in deltamethrin-resistant mosquitoes showed that sublethal doses of pyrethroids can interfere with parasite development. Even though these mosquitoes do not succumb to exposure with insecticides, their ability to transmit the malaria parasite is reduced, and therefore, increasing insecticide resistance does not necessarily directly and immediately lead to a major increase in incidence of malaria parasite infection." Pg. 762

• 35
  • "GiveWell: Are there any other studies that provide explicit quantitative estimates of the impact of resistance that should inform our cost-effectiveness analysis?

    Professor Ranson: Results from a five-country WHO study on this topic should be published within a year or so. However, we, and others, have some issues with the methodology used. Aside from that, I do not know of any other relevant papers.

    Dr. Churcher: Not that I know of.

    GiveWell: What exactly are your concerns with the methodology in the WHO study?

    Professor Ranson: Firstly, I should say that it is incredibly hard to get good estimates because there are so many confounders. No single study was ever going to be definitive. One problem, as mentioned, is that the study looked only at personal protection, ignoring non-net users. Another is the sites where the trials were conducted. Data presented at the 2016 meeting of the American Society of Tropical Medicine and Hygiene suggested there was a low prevalence of resistance in the mosquito population in the study areas, and one may not expect to see much impact of resistance on net effectiveness in such locations. There may be a greater impact in very high-resistance areas, such as Burkina Faso, Côte d'Ivoire, and eastern Uganda, and resistance is increasing rapidly across Africa. It is therefore important not to extrapolate too much from the WHO’s findings.

    GiveWell: The WHO study looked at high, medium and low resistance areas, classifying them according to mosquito mortality rates. Would you prefer to use mortality rates where mosquitoes are exposed to much larger quantities of insecticide, and other indicators of resistance intensity?

    Professor Ranson: Ideally, yes, but there is not strong agreement on the best way to measure resistance. We employed the same methodology as the WHO, using the prevalence of resistance in the mosquito population as an indicator of the intensity of resistance. Dr. Churcher’s work found a reasonably good correlation between tube bioassays and the more natural situation of experimental huts, so these estimates are probably not too far off. They can probably be improved upon, but there is no better metric currently available. This is a major area that warrants a lot more research, not just on the intensity of resistance, but also on how good the resistant vectors are at transmitting malaria and whether they are compromised in any way by contact with insecticide. This is one reason why it is so difficult to make clear inferences from the level of resistance in tube bioassays to the level of malaria in a population.

    Dr. Churcher: The main problem may not be the metric but the thresholds used. The WHO based its classification on the median resistance in the observed populations: everything above was “high resistance” and everything below was “low resistance.” Since all its sites probably had relatively low resistance, it called levels “high” that we would call “low.” Obviously it is not possible to randomize by resistance level, which makes it very hard to conduct good studies. The WHO might be comparing areas with high resistance and low transmission to areas with low resistance and high transmission. Different people use different definitions for when a mosquito is resistant, though we don’t know which is best. I recommend GiveWell be careful when using these definitions as, though a mosquito might be resistant to some actions of the insecticide, it might not be sufficient to have a public health impact.” GiveWell's non-verbatim summary of a conversation with Dr. Thomas Churcher and Professor Hilary Ranson, May 31, 2017

  • Note that Ochomo et al 2017 was part of the WHO multi-country study to which Dr. Churcher and Professor Ranson are referring: "This study was implemented as part of a large, multicountry program set up to quantify the impact of insecticide resistance on the effectiveness of insecticide-based vector controls (21)" Pg. 759, Ochomo et al 2017.
  • Regarding insecticide resistance rates, Ochomo et al 2017 includes the following passages:
    • "We used insecticide resistance data (percentage mosquito mortality upon exposure to deltamethrin) to dichotomize clusters into high- and low-resistance clusters by using the median mortality for that year, namely, 88% for 2013 (clusters with mortality rates >88% were categorized as low resistance and those with mortality rates <88% as high resistance) and 67% for 2014 (clusters with mortality rates >67% were categorized as low resistance and those with mortality rates <67% as high resistance). In combined analysis of both years, we used the overall median mortality (82%) to dichotomize clusters into high or low resistance for net users and non–net users." Pg. 760
    • "Mosquito mortality ranged 55%–100% in 2013 and 30%–98.5% in 2014. The median (25%–75% interquartile range) mortality rates were 88% (81%–97%) for 2013 and 67% (51%–80%) for 2014 (Figure 2)." Pg. 761

• 36

  "In conclusion, insecticide resistance, especially to pyrethroids, continues to increase in countries in sub-Saharan Africa where LLINs and indoor residual spraying are the mainstays of vector control. The results of this study indicate a utility for continuing LLIN use despite the increasing levels of insecticide resistance in the malaria vector population. However, in our study, even among users of nets, malaria parasite incidence remained alarmingly high. Taken together with other reports suggesting an increase in malaria prevalence in parts of western Kenya with high LLIN coverage, the malaria parasite transmission taking place in this region urgently needs to be addressed. Because of their reduced susceptibility, LLINs might not be killing mosquitoes as effectively as they used to. More emphasis needs to be placed on maximizing the coverage and use of LLINs, fully implementing the guidelines on resistance monitoring, and developing more vector control tools to complement existing ones." Pg. 763, Ochomo et al 2017.

• 37

  Bradley et al 2017:

  • "Cross sectional surveys were carried out in each cluster in July 2015. In each cluster forty households were randomly selected from a list (which came from census carried out in 2013). Each household was visited, and if anyone was present, written informed consent was sought to participate in the study. An adult was asked questions about socio economic indicators and bed net use. Up to 3 children per household aged between 6 months and 10 years were chosen at random for malaria microscopy. Blood slides were read by 2 microscopists and if they disagreed on the presence of malaria parasites, a third reader adjudicated. Those testing positive were traced and treated according to national guidelines or referred to a local health facility." Pg. 2
  • "Mortality of mosquitoes exposed to deltamethrin was measured in all 21 clusters. The median number of mosquitoes exposed per cluster was 81 [interquartile range (IQR) 53–101]. Median mosquito mortality was 55.2% (IQR 47.4–68.5%). Mortality to permethrin was measured in 20 of the 21 clusters. The median number of mosquitoes exposed was 25 (IQR 22–35). Median mortality was 18.2% (IQR 8.1–32.2%). In all assays there was 0% mortality in the control group of mosquitoes. There was poor correlation between cluster specific deltamethrin and permethrin mortality (correlation coefficient = 0.20, p = 0.407) and therefore analyses were conducted separately for each active ingredient. In the 21 study clusters, 1621 children from 813 households had blood taken for microscopy. Of these 836 (51.6%) had a malaria infection. Nets were used by 1231 (75.9%) of the children the night before the survey. Net use was associated with lower risk of malaria infection, OR 0.76 (95% CI 0.59, 0.98, p = 0.033) (Table 1). Prevalence was not significantly different in children who used a deltamethrin compared to a permethrin net: OR 1.13 (95% CI 0.76, 1.69, p = 0.543)." Pgs. 2-3

• 38

  Bradley et al 2017:

  • "Children who used LLINs had lower levels of malaria infection [odds ratio = 0.76 (95% CI 0.59, 0.98, p = 0.033)]. There was no evidence that the effectiveness of nets was different in high and low resistance locations (p = 0.513). There was no association between village level resistance and village level malaria prevalence (p = 0.999)." Abstract, Bradley et al 2017.
  • "Malaria prevalence was similar in clusters with lower resistance to deltamethrin compared to those with higher resistance OR 1.11 (95% CI 0.65, 1.90, p = 0.698) (Table 2); this was true even when restricted just to those children who slept under deltamethrin treated nets OR 0.82 (95% CI 0.44, 1.53, p = 0.532). The estimated effect of a 10% increase in mosquito mortality was negligible: OR 1.00 (95% CI 0.85, 1.17, p = 0.999). There was also little association between malaria prevalence in clusters with lower resistance to permethrin compared to clusters with higher resistance OR 0.88 (95% CI 0.50, 1.53, p = 0.640) (Table 2), even when restricted to children who slept under permethrin treated nets OR 0.66 (95% CI 0.28, 1.54, p = 0.336). The estimated effect of a 10% increase in mosquito mortality was negligible: OR 0.99 (95% CI 0.90, 1.09, p = 0.893)." Pg. 3
  • "There was no evidence that the effect of nets was different in clusters with lower [OR 0.82 (95% CI 0.59, 1.17), p = 0.282] or higher [OR 0.69 (95% CI 0.48, 1.00), p = 0.052] resistance to deltamethrin (interaction p = 0.513) (Table 1) or for lower [OR 0.73 (95% CI 0.51, 1.04), p = 0.082] or higher [OR 0.84 (95% CI 0.58, 1.20), p = 0.335] resistance to permethrin (test for interaction: p = 0.595)." Pg. 3

• 39

  Bradley et al 2017:

  • "Since it is not possible to randomize villages to different levels of insecticide resistance and not ethical to randomize children to not using nets, all studies of this issue must be observational—and, therefore, subject to confounding. In this study, two important confounders (age and SES) were adjusted for which made little impact on the analysis, but some residual confounding may remain." Pgs. 3-4
  • "While insecticide resistance in malaria vectors is widespread, there is little evidence of its impact on the effectiveness of nets and how in turn this influences epidemiological outcomes. This study found that use of insecticide treated nets was associated with lower risk of malaria infection in an area of high pyrethroid resistance. Furthermore, there was no evidence that the effect of nets differed in villages stratified by vector resistance. The study also found no association between pyrethroid resistance and malaria prevalence. The fact that no impact of pyrethroid resistance on net effectiveness and malaria prevalence was detected in this study is both an interesting and comforting result. It suggests that whilst insecticide resistance is a grave threat to the long-term sustainability of malaria control it has not apparently reached a level, at least in Benin, where it would render LLINs ineffective. Control programmes should continue to strive to attain high LLIN coverage at the same time researchers and industry seek to develop alternative control options...It would, however, be reckless to conclude from this study that insecticide resistance has no impact on malaria transmission. There are a number of reasons why an association between resistance and malaria prevalence might not have been observed in this study, even if there is an impact of resistance on malaria control. Resistance to pyrethroids was observed in all study clusters and, therefore, there may be an impact of resistance on malaria prevalence across all study clusters but the absence of truly susceptible mosquito populations and a relatively insensitive resistance definition prevents us detecting an effect." Pg. 3

• 40

  “Methods: One deltamethrin ITN was distributed in the study area for every two individuals in each household plus one extra ITN for households with an odd number of residents. A fixed cohort of 1,199 children aged six to 59 months was seen monthly for one year and at sick visits to measure malaria infection and use of ITNs. Insecticide resistance among malaria vectors was measured. The effect of ITN use on malaria incidence was assessed, adjusting for potential confounders using generalized estimating equations accounting for repeated measures.” Lindblade et al 2015, Pg. 1.

• 41

  “The objective of this investigation was to assess the effectiveness of ITNs to prevent malaria in an area of Malawi with moderate pyrethroid resistance...Use of ITNs reduced the incidence of malaria infection by 30% (rate ratio 0.7; 95% CI, 0.5-0.8) compared to no bed nets." Lindblade et al 2015, Pg. 1.

• 42

  “Results: There were 1,909 infections with Plasmodium falciparum over 905 person-years at risk (PYAR), resulting in an observed incidence of 2.1 infections per person-year (iPPY). ITNs were used during 97% of the PYAR. The main vector was Anopheles funestus: mortality in WHO tube assays after exposure to 0.05% deltamethrin was 38% (95% confidence interval (CI) 29–47), and resistance was due to elevated oxidase enzymes. After adjusting for potential confounders, the incidence of malaria infection among ITN users was 1.7 iPPY (95% CI 1.5-2.1) and among non-bed net users was 2.6 iPPY (95% CI 2.0-3.3). Use of ITNs reduced the incidence of malaria infection by 30% (rate ratio 0.7; 95% CI, 0.5-0.8) compared to no bed nets." Lindblade et al 2015, Pg. 1.

• 43

  “Pyrethroid resistance is becoming widespread in Anopheles gambiae mosquitoes, coinciding with expanded use of insecticide-treated nets (ITNs) throughout Africa. To investigate whether nets in use are still protective, we conducted household trials in northern and southern Benin, where An. gambiae mosquitoes are susceptible and resistant, respectively, to pyrethroids. Rooms were fitted with window traps and monitored for mosquito biting and survival rates before and after the nets were treated with pyrethroid. Sleeping under an ITN in the location with resistant mosquitoes was no more protective than sleeping under an untreated net, regardless of its physical condition. By contrast, sleeping under an ITN in the location with susceptible mosquitoes decreased the odds of biting by 66%. ITNs provide little or no protection once the mosquitoes become resistant and the netting acquires holes. Resistance seriously threatens malaria control strategies based on ITN.” Asidi et al 2012, Abstract.

• 44

  “Sleeping under an ITN in the location with resistant mosquitoes was no more protective than sleeping under an untreated net, regardless of its physical condition. By contrast, sleeping under an ITN in the location with susceptible mosquitoes decreased the odds of biting by 66%. ITNs provide little or no protection once the mosquitoes become resistant and the netting acquires holes. Resistance seriously threatens malaria control strategies based on ITN.” Asidi et al 2012, Abstract.

• 45

  “Eleven households at the southern sites (where mosquitoes are resistant) and 5 households at the northern site (where mosquitoes are susceptible) participated in the study." Asidi et al 2012, Pg. 1102.

• 46

  “The researchers identified 25 reports of laboratory and field studies of the impact of ITNs on African malaria vectors that measured the mosquitoes’ resistance to pyrethroid insecticides at the time of the study. The laboratory studies used two assays to measure entomological outcomes. The cone test measured mosquito mortality (death), percent of mosquitoes knocked down (immobilized) after 60 minutes, and the time to knock down 50% or 95% of the mosquitoes after brief exposure to an ITN or untreated bed net (UTN). In the tunnel test, mosquitoes had to pass through a holed ITN or UTN to reach animal baits; counts of live and dead mosquitoes, and fed and unfed mosquitoes on both sides of the net measured deterrence, blood feeding, and mosquito mortality. In the field studies, volunteers slept under an ITN or UTN in an experimental hut. Subsequent counts of live and dead mosquitoes and fed and unfed mosquitoes inside the huts and in exit traps measured deterrence, blood feeding, mosquito mortality, and induced exophily.” Strode et al 2014, Pg. 32.

• 47
  • Description of experimental hut trials: "Field Trials

    "Methods: Studies in areas where mosquitoes breed. Volunteers sleep in experimental huts for a specific period under an ITN or an UTN, with one hut per person. The huts are identical in construction, and incorporate exit traps to catch wild mosquitoes entering and exiting the hut prematurely. Each morning of the trial, both live and dead mosquitoes, blood fed and non-blood fed, are collected and counted from both inside the hut and the exit traps. Live mosquitoes are monitored for a further 24 h to assess delayed mortality. Volunteers and nets are randomly allocated to huts at the start of the trial and are usually rotated to avoid bias. Often huts are cleaned between rotations to avoid cross-contamination of huts from the different treatment arms.

    "Outcomes: Deterrence, blood feeding, mosquito mortality, and induced exophily.

    "Advantages: Given that this method assesses the response of wild mosquitoes to human volunteers, it is a more realistic representation of how effective ITNs are in terms of entomological outcomes, compared with laboratory methods." Strode et al 2014, Pg. 2.

  • Description of the meta-analysis: "We included laboratory and field studies of African malaria vectors that measured resistance at the time of the study and used World Health Organization–recommended impregnation regimens. We reported mosquito mortality, blood feeding, induced exophily (premature exit of mosquitoes from the hut), deterrence, time to 50% or 95% knock-down, and percentage knock-down at 60 min. Publications were searched from 1 January 1980 to 31 December 2013 using MEDLINE, Cochrane Central Register of Controlled Trials, Science Citation Index Expanded, Social Sciences Citation Index, African Index Medicus, and CAB Abstracts. We stratified studies into three levels of insecticide resistance, and ITNs were compared with untreated bed nets (UTNs) using the risk difference (RD). ... Included were 36 laboratory and 24 field studies, reported in 25 records. Studies tested and reported resistance inconsistently. Based on the meta-analytic results, the difference in mosquito mortality risk for ITNs compared to UTNs was lower in higher resistance categories. However, mortality risk was significantly higher for ITNs compared to UTNs regardless of resistance. For cone tests: low resistance, risk difference (RD) 0.86 (95% CI 0.72 to 1.01); moderate resistance, RD 0.71 (95% CI 0.53 to 0.88); high resistance, RD 0.56 (95% CI 0.17 to 0.95). For tunnel tests: low resistance, RD 0.74 (95% CI 0.61 to 0.87); moderate resistance, RD 0.50 (95% CI 0.40 to 0.60); high resistance, RD 0.39 (95% CI 0.24 to 0.54). For hut studies: low resistance, RD 0.56 (95% CI 0.43 to 0.68); moderate resistance, RD 0.39 (95% CI 0.16 to 0.61); high resistance, RD 0.35 (95% CI 0.27 to 0.43). However, with the exception of the moderate resistance category for tunnel tests, there was extremely high heterogeneity across studies in each resistance category (chi-squared test, p,0.00001, I2 varied from 95% to 100%)." Strode et al 2014, Pg. 1.

• 48

  "This meta-analysis found that ITNs are more effective than UTNs regardless of resistance. There appears to be a relationship between resistance and the RD for mosquito mortality in laboratory and field studies. However, the substantive heterogeneity in the studies’ results and design may mask the true relationship between resistance and the RD, and the results need to be interpreted with caution." Strode et al 2014, Pg. 1.
  

• 49

  "Studies overall are very poor in characterising the resistance pattern of the mosquitoes, and the classification systems are unclear and lack uniformity. ... We observed a large amount of heterogeneity and bias across studies, which was particularly acute in the field studies. ... In addition, the execution of the field trials was not uniform across the studies, e.g., washing of nets, rotation of nets/sleepers, season in which the trial took place, length of the trial, decontamination of huts, and exclusion of male mosquitoes from the analysis." Strode et al 2014, Pg. 28.

• 50

  "Here we propose that information on the current malaria endemicity, mosquito species and population prevalence of pyrethroid resistance (as measured by bioassay mortality) can be used to predict the public health impact of pyrethroid resistance and choosing the most appropriate LLIN for the epidemiological setting. Firstly (1) a meta-analysis and statistical model are used to determine whether mosquito mortality in a bioassay can be used to predict the proportion of mosquitoes, which die in experimental hut trials and to define the shape of this relationship. Secondly (2), another meta-analysis of experimental hut trial data is analysed to characterise the full impact of pyrethroid resistance on LLIN effectiveness. Thirdly, information from (1) and (2) is used to parameterise a widely used malaria transmission dynamics mathematical model to estimate the public health impact of pyrethroid resistance in different settings taking into account the community impact of LLINs." Pg. 3, Churcher et al 2016.

• 51
  • Trape et al 2011 reports results as follows: "Methods We did a longitudinal study of inhabitants of Dielmo village, Senegal, between January, 2007, and December, 2010. We monitored the inhabitants for fever during this period and we treated malaria attacks with artesunate plus amodiaquine. In July, 2008, we offered longlasting insecticide (deltamethrin)-treated nets (LLINs) to all villagers. We did monthly night collections of mosquitoes during the whole study period, and we assessed asymptomatic carriage from cross-sectional surveys. Our statistical analyses were by negative binomial regression, logistic regression, and binomial or Fisher exact test.
    "Findings There were 464 clinical malaria attacks attributable to Plasmodium falciparum during 17 858 person-months of follow-up. The incidence density of malaria attacks averaged 5·45 (95% CI 4·90–6·05) per 100 person-months between January, 2007, and July, 2008, before the distribution of LLINs. Incidence density decreased to 0·41 (0·29–0·55) between August, 2008, and August, 2010, but increased back to 4·57 (3·54–5·82) between September and December, 2010—ie, 27–30 months after the distribution of LLINs. The rebound of malaria attacks were highest in adults and children aged 10 years or older: 45 (63%) of 71 malaria attacks recorded in 2010 compared with 126 (33%) of 384 in 2007 and 2008 (p<0·0001). 37% of Anopheles gambiae mosquitoes were resistant to deltamethrin in 2010, and the prevalence of the Leu1014Phe kdr resistance mutation increased from 8% in 2007 to 48% in 2010 (p=0·0009).
    Interpretation Increasing pyrethroid resistance of A gambiae and increasing susceptibility of older children and adults, probably due to decreasing immunity, caused the rebound and age shift of malaria morbidity. Strategies to address the problem of insecticide resistance and to mitigate its effects must be urgently defined and implemented." Pg. 1.
  • Maharaj, Mthembu and Sharp 2005 interpreted the spike in malaria cases in KwaZulu-Natal as follows: "Archival malaria case data show that low levels of malaria transmission have occurred in South Africa since the early 1970s. A significant increase in the number of malaria cases was detected in KwaZulu-Natal following the discontinuation of DDT use in 1996 and the discovery of a pyrethroid-resistant vector mosquito species during the 1999/2000 epidemic. With the increasing burden of malaria morbidity and on the advice of experts, the national health department took the decision to return to DDT indoor residual spraying in March 2002. The effect of spraying was immediately made evident by the rapid decline in the number of cases reported from high-risk areas treated with DDT. A further decrease in malaria incidence occurred after the mop-up spraying in May 2000. It was evident that DDT contributed significantly to this decrease in morbidity since it was the only variable to have changed at the peak of the epidemic. Further gains in malaria control were made following the introduction of artemisinin-based combination therapy in February 2001." Pg. 873.

• 52

  We rely on the following three observational studies:

  • Louis et al 2015 found that under-5 malaria cases increased following a 2010 ITN distribution in Burkina Faso. See: "Objective To investigate if the first national insecticide-treated bed-net campaign in Burkina Faso, done in 2010, was followed by a decrease in childhood malaria in a district with high baseline transmission of the disease. ... Findings The bed-net campaign was associated with an increase in the reported use of insecticide-treated nets. In 2009, 73% (630/869) of children reportedly slept under nets. In 2011, 92% (449/487) did. The campaign had no effect on the proportion of young children with P. falciparum parasitaemia after the rainy season; 52% (442/858) in 2009 and 53% (263/499) in 2011. Cases of malaria increased markedly after the campaign, as did the number of children presenting with other diseases. The campaign was not associated with any changes in child mortality." Pg. 750.
  • Jagannathan et al 2012 followed a cohort of 100 children in Tororo, Uganda from 2008 to 2011, finding that risk of malaria increased 52% even despite provision of ITNs and prompt medical treatment. See: "Methods: A cohort of 100 children, aged six weeks to 10 months of age, were enrolled in an area of high malaria transmission intensity and followed through 48 months of age. Children were given a long-lasting insecticide-treated bed net (LLIN) at enrolment and received all care, including monthly blood smears and treatment with artemisinin-based combination therapy (ACT) for uncomplicated malaria, at a dedicated clinic. The incidence of malaria was estimated by passive surveillance and associations between malaria incidence and age, calendar time and season were measured using generalized estimating equations.
    "Results: Reported compliance with LLINs was 98% based on monthly routine evaluations. A total of 1,633 episodes of malaria were observed, with a median incidence of 5.3 per person-year (PPY). There were only six cases of complicated malaria, all single convulsions. Malaria incidence peaked at 6.5 PPY at 23 months of age before declining to 3.5 PPY at 48 months. After adjusting for age and season, the risk of malaria increased by 52% from 2008 to 2011 (RR 1.52, 95% CI 1.10-2.09). Asymptomatic parasitaemia was uncommon (monthly prevalence <10%) and rarely observed prior to 24 months of age.
    Conclusions: In Tororo, despite provision of LLINs and prompt treatment with ACT, the incidence of malaria is very high and appears to be rising. Additional malaria control interventions in high transmission settings are likely needed." Pg. 1.
  • Mukonka et al 2014 show that malaria prevalence in Nchelenge District, Luapula Province, Zambia increased from 2006 to 2012 even as intervention coverage with ITNs and indoor residual spraying increased. See: "Methods: Yearly aggregated information on cases of malaria, malaria deaths, use of malaria diagnostics, and malaria control interventions from 2006 to 2012 were obtained from the Nchelenge District Health Office. Trends in the number of malaria cases, methods of diagnosis, malaria positivity rate among pregnant women, and intervention coverage were analysed using descriptive statistics.
    "Results: Malaria prevalence remained high, increasing from 38% in 2006 to 53% in 2012. Increasing numbers of cases of severe malaria were reported until 2010. Intense seasonal malaria transmission was observed with seasonal declines in the number of cases between April and August, although malaria transmission continued throughout the year. Clinical diagnosis without accompanying confirmation declined from 95% in 2006 to 35% in 2012. Intervention coverage with long-lasting insecticide-treated nets and indoor residual spraying increased from 2006 to 2012." Pg. 1.

• 53

  "Prof. Ranson believes bed nets are still working well in most places, but researchers have found evidence of bed net failure in some areas with high resistance." GiveWell's non-verbatim summary of a conversation with Hilary Ranson, Professor, Medical Entomology, Liverpool School of Tropical Medicine, March 18, 2016.
  

• 54

  "Methods: Cross-sectional surveys of LLINs were conducted in houses of seven and four villages in Gem and Bungoma Districts in western Kenya, respectively. Condition (number and area of holes in the nets), number and species of mosquitoes resting inside them, and insecticidal activity of nets were quantified. Mosquitoes collected inside nets were allowed to lay eggs and progeny tested for susceptibility to deltamethrin and permethrin, pyrethoids commonly deployed in LLINs in western Kenya.
  "Results: In Gem, 83.3% of nets were less than three years old and 32.4% had at least one hole of any size; while in Bungoma, 92% were less than three years old and 48% had at least one hole. No anopheline and five Culex spp. mosquitoes were found resting inside nets in Gem regardless of the number and size of holes, while 552 Anopheles gambiae s.l., five Anopheles funestus s.l. and 137 Culex spp. were in nets in Bungoma. The number of mosquitoes resting inside nets increased with hole areas >50 cm in Bungoma. In WHO resistance assays, f1 offspring of samples collected in nets in Bungoma were 94 and 65% resistant to deltamethrin and permethrin, respectively. Nets from Bungoma retained strong activity against a susceptible laboratory strain, but not against f1 offspring of field-collected An. gambiae s.s. All An. gambiae s.s. samples collected in nets were homozygous for the kdr genotype L1014S." Ochomo et al 2013, Pg. 1.

• 55

  Ojuka et al 2015 reports: "Background: Southwestern Uganda has high malaria heterogeneity despite moderate vector control and other interventions. Moreover, the early biting transmission and increased resistance to insecticides might compromise strategies relying on vector control. ...
  "Results: ... In both areas and seasons, an unusually early evening biting peak was observed between 6 - 8 p.m. In Engari, insecticide bioassays showed 85%, 34% and 12% resistance to DDT during the rainy season, dry season and to deltamethrin during the dry season, respectively. In Kigorogoro, 13% resistance to DDT and to deltamethrin was recorded. There was no resistance observed to bendiocarb and pirimiphos methyl." Pg. 1.

• 56

  See, e.g., Tokponnon et al 2014, which found that ITNs were more effective at reducing malaria risk in high resistance areas in South-West Benin than in low resistance areas: See: "Methods: The study was conducted in four rural districts and included 32 villages categorized as low or high resistance areas in Plateau Department, south-western Benin. Larvae collection was conducted to measure vector susceptibility to deltamethrin and knockdown resistance (kdr) frequency. In each resistance area, around 500 children were selected to measure the prevalence of malaria infection as well as the prevalence of anaemia associated with the use of LLINs.
  "Results: Observed mortalities of Anopheles gambiae s.s population exposed to deltamethrin ranged from 19 to 96%. Knockdown resistance frequency was between 38 and 84%. The prevalence of malaria infection in children under five years was 22.4% (19.9-25.1). This prevalence was 17.3% (14.2-20.9) in areas of high resistance and 27.1% (23.5-31.1) in areas of low resistance (p = 0.04). Eight on ten children that were aged six - 30 months against seven on ten of those aged 31–59 months were anaemic. The anaemia observed in the six to 30-month old children was significantly higher than in the 31–59 month old children (p = 0.00) but no difference associated with resistance areas was observed (p = 0.35). The net use rate was 71%. The risk of having malaria was significantly reduced (p < 0.05) with LLIN use in both low and high resistance areas. The preventive effect of LLINs in high resistance areas was 60% (95% CI: 40–70), and was significantly higher than that observed in low resistance areas (p < 0.05)." Pg. 1.

• 57
  • "Prof. Ranson believes bed nets are still working well in most places, but researchers have found evidence of bed net failure in some areas with high resistance." GiveWell's non-verbatim summary of a conversation with Hilary Ranson, Professor, Medical Entomology, Liverpool School of Tropical Medicine, March 18, 2016.
  • ”LLINs in good physical condition undoubtedly still provide protection against malaria and the spread of resistance should not derail plans to increase access to the most effective tool to reduce malaria transmission. However, the malaria community cannot afford to be complacent about insecticide resistance. Critical knowledge gaps on the causes and consequences of insecticide resistance need to be filled (see Outstanding Questions), the development, evaluation, and implementation of new products must be accelerated, and an evidence base for how best to deploy insecticide to minimize the spread of resistance must be generated to ensure the success of future malaria control efforts." Ranson and Lissenden 2016, Pg. 9.

• 58

  "Whilst there is extensive evidence of resistance in Anopheles mosquitoes, there is little evidence of control programme failure associated directly with insecticide resistance largely because of many confounding factors." Kleinschmidt et al 2015, Pg. 2.

• 59
  “Mosquito nets that include both a pyrethroid insecticide and the synergist piperonyl butoxide (PBO) have become available. PBO is a synergist that acts by inhibiting certain metabolic enzymes (e.g., mixed-function oxidases) within the mosquito that detoxify or sequester insecticides before they can have toxic effect on the mosquito. Therefore, compared to a pyrethroid-only net, a pyrethroid-PBO net should, in theory, have an increased killing effect on malaria vectors that express such resistance mechanisms. However, the entomological and epidemiological impact of pyrethroid-PBO nets may vary depending on the bioavailability and retention of PBO in the net, and on the design of the net (i.e., whether only some or all panels are treated with PBO).” World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017, Pg 1.
  
• 60

  “Based on the epidemiological findings and the need to deploy products that are effective against pyrethroid-resistant mosquitoes, pyrethroid-PBO nets are being given an interim endorsement as a new WHO class of vector control products.” World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017, Pg 2.

• 61

  “National malaria control programmes and their partners should consider the deployment of pyrethroid-PBO nets in areas where the main malaria vector(s) have pyrethroid resistance that is: a) confirmed, b) of intermediate level (as defined above), and c) conferred (at least in part) by a monooxygenase-based resistance mechanism, as determined by standard procedures.10 Deployment of pyrethroid-PBO nets must only be considered in situations where coverage with effective vector control (primarily LLINs or indoor residual spraying [IRS]) will not be reduced; the primary goal must remain the achievement and maintenance of universal coverage for all people at risk of malaria.” World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017, Pg 2.

• 62

  “Epidemiological data from one cluster randomized controlled trial indicated that a pyrethroid-PBO net product had additional public health value compared to a pyrethroid-only LLIN product in an area where the main malaria vector had confirmed pyrethroid resistance of moderate intensity conferred (at least in part) by monooxygenase-based resistance mechanism as determined by standard procedures.9, 10 This conclusion is based on a comparison of malaria infection rates in children in village clusters allocated pyrethroid-PBO nets (Olyset® Plus) and rates in village clusters allocated pyrethroid-only LLINs (Olyset® Net) over a period of 2 years in Muleba, United Republic of Tanzania.” World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017, Pg 2.

• 63

  For more information on this trial, see here.

• 64
  • “We did a four-group cluster randomised controlled trial using a two-by-two factorial design of 48 clusters derived from 40 villages in Muleba (Kagera, Tanzania). We randomly assigned these clusters using restricted randomisation to four groups: standard long-lasting insecticidal nets, PBO long-lasting insecticidal nets, standard long-lasting insecticidal nets plus indoor residual spraying, or PBO long-lasting insecticidal nets plus indoor residual spraying. Both standard and PBO nets were distributed in 2015. Indoor residual spraying was applied only once in 2015. We masked the inhabitants of each cluster to the type of nets received, as well as field staff who took blood samples. Neither the investigators nor the participants were masked to indoor residual spraying. The primary outcome was the prevalence of malaria infection in children aged 6 months to 14 years assessed by cross-sectional surveys at 4, 9, 16, and 21 months after intervention. The endpoint for assessment of indoor residual spraying was 9 months and PBO long-lasting insecticidal nets was 21 months. This trial is registered with ClinicalTrials.gov, number NCT02288637.” Protopopoff et al. 2018
  • “The RCT started on March 1, 2014. The post-intervention assessment period was initially planned for 18 months (from Jan 1, 2015, to June 30, 2016) and was subsequently extended on our request to the funding agency to 24 months (from Jan 1, 2014, to Dec 31, 2016) to enable further assessment of the PBO long-lasting insecticidal net (figure 1).” Protopopoff et al. 2018

• 65
  • “We used restricted randomisation to allocate the 48 clusters to the four study groups: standard long-lasting insecticidal nets, PBO long-lasting insecticidal nets, standard long-lasting insecticidal nets plus indoor residual spraying, and PBO long-lasting insecticidal nets plus indoor residual spraying. We limited potential imbalance using three restriction variables: malaria infection prevalence in children aged 6 months to 14 years, usage of long-lasting insecticidal nets, and socioeconomic status, as recorded in the baseline survey between September and October, 2014. Of the 200 000 random allocations, 29 478 met the restriction criteria of no more than 7% difference in mean malaria prevalence, 10% in mean usage of long-lasting insecticidal nets, and 10% of households in the lowest socioeconomic status tertile between study groups. After verifying that clusters were independently allocated to study groups, we randomly chose one of the eligible allocations.” Protopopoff et al. 2018
  • “We used the following vector control products: Olyset Net (Sumitomo Chemicals, Japan) containing 2% permethrin (standard long-lasting insecticidal net), Olyset Plus (Sumitomo Chemicals, Japan) containing 2% permethrin and 1% PBO (PBO long-lasting insecticidal net), and Actellic 300CS containing microencapsulated pirimiphos-methyl (indoor residual spraying).” Protopopoff et al. 2018
  • “7184 (68·0%) of 10 560 households were selected for post-intervention survey, and 15 469 (89·0%) of 17 377 eligible children from the four surveys were included in the intention-to-treat analysis.” Protopopoff et al. 2018

• 66
  • “The primary outcome was the prevalence of Plasmodium spp infection measured by the rapid diagnostic test” Protopopoff et al. 2018
  • “The primary outcome was the prevalence of malaria infection in children aged 6 months to 14 years assessed by cross-sectional surveys at 4, 9, 16, and 21 months after intervention.” Protopopoff et al. 2018

• 67

  “This study had 80% power to detect a relative reduction in prevalence of infection of at least 28% (prevalence ratio 0·72) for each of the two main effects (ie, indoor residual spraying vs no indoor residual spraying, and PBO long-lasting insecticidal nets vs no PBO long-lasting insecticidal nets) and a 40% difference between any of the individual groups, with 24 clusters of 80 individuals per cluster being tested in each of these comparisons, and assuming a mean prevalence of 20% in the reference groups and a coefficient of variation of 0·3 (based on data from the earlier study).18” Protopopoff et al. 2018
  

• 68

  “In the intention-to-treat factorial analysis for the prevalence of malaria infection, the effect of indoor residual spraying versus no indoor residual spraying was evident at 4 months (OR 0·50, 95% CI 0·31–0·82; p=0·0071) whereas there was no evidence of a difference between PBO long-lasting insecticidal nets and standard long-lasting insecticidal nets (0·68, 0·39–1·18; p=0·1630; table 2). A clear effect was observed 9 months after intervention for indoor residual spraying versus no indoor spraying (OR 0·33, 95% CI 0·19–0·55; p<0·0001) and PBO long-lasting insecticidal nets versus standard nets (OR 0·37, 0·21–0·65; p=0·0011). During the second year, the prevalence of malaria infection in the PBO long-lasting insecticidal net groups remained less than in the standard long-lasting insecticidal net groups (OR 0·47, 95% CI 0·26–0·87; p=0·0173 after 16 months of intervention; and 0·40, 0·20–0·81; p=0·0122 after 21 months of intervention), whereas the effect of the single round of indoor residual spraying applied previously waned over time (OR 0·59, 95% CI 0·34–1·04 after 16 months; and 0·58, 0·29–1·14 after 21 months; table 2).” Protopopoff et al. 2018
  

• 69
  • Protopopoff et al. 2018 reports results in odds ratios (see Table 2). We have converted these results to relative risks for ease of comprehension.
  • See GiveWell - Insecticide resistance adjustment incorporating PBO nets, Sheet "Propopotoff et al. 2018" for calculations of relative risk.

• 70
  • “We did a four-group cluster randomised controlled trial using a two-by-two factorial design of 48 clusters derived from 40 villages in Muleba (Kagera, Tanzania)” Protopopoff et al. 2018
  • “High levels of resistance to pyrethroids have been reported in A gambiae in the study area, and synergy bioassay tests done with PBO and pyrethroid together partially restored the toxicity of pyrethroids.” Protopopoff et al. 2018
  • ”Anopheles gambiae s.l. (Diptera: Culicidae) in Muleba, Tanzania has developed high levels of resistance to most insecticides currently advocated for malaria control. The kdr mutation has almost reached fixation in An. gambiae s.s. in Muleba. This change has the potential to jeopardize malaria control interventions carried out in the region. Trends in insecticide resistance were monitored in two intervention villages using World Health Organization (WHO) susceptibility test kits. Additional mechanisms contributing to observed phenotypic resistance were investigated using Centers for Disease Control (CDC) bottle bioassays with piperonylbutoxide (PBO) and S,S,S-tributyl phosphorotrithioate (DEF) synergists. Resistance genotyping for kdr and Ace-1 alleles was conducted using quantitative polymerase chain reaction (qPCR). In both study villages, high phenotypic resistance to several pyrethroids and DDT was observed, with mortality in the range of 12-23%. There was a sharp decrease in mortality in An. gambiae s.l. exposed to bendiocarb (carbamate) from 84% in November 2011 to 31% in December 2012 after two rounds of bendiocarb-based indoor residual spraying (IRS). Anopheles gambiae s.l. remained susceptible to pirimiphos-methyl (organophosphate). Bendiocarb-based IRS did not lead to the reversion of pyrethroid resistance. There was no evidence for selection for Ace-1 resistance alleles. The need to investigate the operational impact of the observed resistance selection on the effectiveness of longlasting insecticidal nets and IRS for malaria control is urgent.” Matowo et al. 2015

• 71

  “PBO is a chemical synergist that acts by inhibiting enzymes involved in the natural defense mechanisms of insects, which results in pyrethroid not being detoxified in the insect and the pyrethroid on the long-lasting insecticidal net remaining potent against mosquitoes despite resistance. Such PBO-pyrethroid-treated long-lasting insecticidal nets appear to have similar or better efficacy against resistant mosquitoes under controlled household conditions than standard long-lasting insecticidal nets that do not have PBO.” Protopopoff et al. 2018

• 72

  “Mathematical modelling work drawing on relevant entomological data indicates that the added benefit of pyrethroid PBO nets compared to pyrethroid-only LLINs is expected to be the greatest where pyrethroid resistance is at “intermediate levels”, meaning where mosquito mortality after exposure to a pyrethroid insecticide in WHO test kits or CDC bottle assays ranges from 10% to 80%.11 The benefit of pyrethroid PBO nets is expected to diminish where bioassay mortality is outside of this range. Pyrethroid-PBO nets are not expected to have any added benefit in areas where the main malaria vectors are susceptible to pyrethroids and/or do not harbor resistance mechanism(s) that are affected by PBO, i.e., monooxygenase-based resistance mechanism.10” World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017

• 73

  “Combining indoor residual spraying with pirimiphos-methyl and PBO long-lasting insecticidal nets provided no additional benefit compared with PBO long-lasting insecticidal nets alone or standard long-lasting insecticidal nets plus indoor residual spraying.” Protopopoff et al. 2018

• 74

  “Whether another indoor residual spraying insecticide, which does not require activation by cytochrome P450s, would prove an effective partner to PBO long-lasting insecticidal nets is not known.” Protopopoff et al. 2018

• 75
  • AMF told us they believe it is unlikely that a significant proportion of PBO nets will be distributed in areas covered by indoor residual spraying due to resource constraints. We have seen written confirmation from partners in Malawi that IRS is not planned for districts covered by PBO nets supplied by the Global Fund.
    • Rob Mather, AMF Founder, email to GiveWell, April 17, 2018
    • Rob Mather, AMF Founder, conversation with GiveWell, June 18, 2018
  • We have not prioritized additional work to vet this conclusion.

• 76
  “Buffer areas of 300 m were small compared with what has been used in other trials,17, 18 which might not have totally prevented contamination. However, any spill-over would have lessened rather than increased the effect size between intervention groups.” Protopopoff et al. 2018

• 77

  For more information on this assumption, see this section of our report on seasonal malaria chemoprevention.

• 78
  • “Olyset Plus nets (with PBO) are made of yarn with a smaller mesh size compared to Olyset nets.” GiveWell's non-verbatim summary of a conversation with Hilary Ranson, Professor, Medical Entomology, Liverpool School of Tropical Medicine, March 18, 2016
  • “The effectiveness of other pyrethroid-PBO nets in comparison to the product for which data were generated in Tanzania needs to be determined. Evaluation procedures to determine whether other 4 products in a class perform at least as well as the product(s) for which epidemiological data were generated, and for which a product class has been established, are under development. Comparing the effectiveness of different pyrethroid-PBO nets will be aided by:
    c.i. Identifying appropriate entomological indicators to assess the effectiveness of subsequent products entering an existing product class, given that these products will not be required to generate epidemiological data;
    c.ii. Conducting comparative experimental hut trials on different pyrethroid-PBO nets to determine the relative effectiveness of different compositions of net (e.g., PBO applied to the roof panel of the net only versus all panels of the net), as well as different formulations including initial PBO treatment dosages and release properties;
    c.iii. Conducting bioassays using characterized reference strains of insecticide-resistant Anopheles mosquito(es) on pyrethroid-PBOnets following a minimum of 2 to 3 years of routine use to determine the bioavailability and chemical retention of PBO over time. Current information suggests that PBO retention rates and wash resistance indices are much lower than for the pyrethroid component of the formulations. Studies should be conducted on the PBO-LLIN product assessed in Tanzania, with comparative studies performed on other products of the same class” World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017, Pgs 3,4.

• 79

  “VCAG will review further epidemiological trial data as soon as they become available, such as from a randomized controlled trial planned in Uganda using two pyrethroid-PBO nets (the same product as is being tested in Tanzania, treated with PBO on all panels, and another pyrethroid-PBO net with only the net roof treated with PBO). These data will provide additional evidence on how pyrethroid-PBO nets perform in another geographical setting and whether there are notable differences in effectiveness between products in this class. If VCAG is able to confirm additional public health value, it will allow the interim endorsement of pyrethroid-PBO nets to be converted into the full establishment of the class.” World Health Organization - Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017
  

• 80

  For more information on this trial, see here.

• 81

  Briet et al 2013 appears to be an older modeling study and we have not investigated it closely, in part because it draws on older and less comprehensive data. Briet et al 2013 incorporates 9 data sources from 2007 - 2010 (see "Table 1 Data sources," Pg. 2), whereas Churcher et al 2016 incorporates 44 data sources from 2008 - 2015 (see Table 1 and Tables 3-5, Churcher et al 2016).

• 82
  • "GiveWell: Are there any other studies that provide explicit quantitative estimates of the impact of resistance that should inform our cost-effectiveness analysis?

    Professor Ranson: Results from a five-country WHO study on this topic should be published within a year or so. However, we, and others, have some issues with the methodology used. Aside from that, I do not know of any other relevant papers.

    Dr. Churcher: Not that I know of.

    GiveWell: What exactly are your concerns with the methodology in the WHO study?

    Professor Ranson: Firstly, I should say that it is incredibly hard to get good estimates because there are so many confounders. No single study was every going to be definitive. One problem, as mentioned, is that the study looked only at personal protection, ignoring non-net users. Another is the sites where the trials were conducted. Data presented at the 2016 meeting of the American Society of Tropical Medicine and Hygiene suggested there was a low prevalence of resistance in the mosquito population in the study areas, and one may not expect to see much impact of resistance on net effectiveness in such locations. There may be a greater impact in very high-resistance areas, such as Burkina Faso, Côte d'Ivoire, and eastern Uganda, and resistance is increasing rapidly across Africa. It is therefore important not to extrapolate too much from the WHO’s findings.

    GiveWell: The WHO study looked at high, medium and low resistance areas, classifying them according to mosquito mortality rates. Would you prefer to use mortality rates where mosquitoes are exposed to much larger quantities of insecticide, and other indicators of resistance intensity?

    Professor Ranson: Ideally, yes, but there is not strong agreement on the best way to measure resistance. We employed the same methodology as the WHO, using the prevalence of resistance in the mosquito population as an indicator of the intensity of resistance. Dr. Churcher’s work found a reasonably good correlation between tube bioassays and the more natural situation of experimental huts, so these estimates are probably not too far off. They can probably be improved upon, but there is no better metric currently available. This is a major area that warrants a lot more research, not just on the intensity of resistance, but also on how good the resistant vectors are at transmitting malaria and whether they are compromised in any way by contact with insecticide. This is one reason why it is so difficult to make clear inferences from the level of resistance in tube bioassays to the level of malaria in a population.

    Dr. Churcher: The main problem may not be the metric but the thresholds used. The WHO based its classification on the median resistance in the observed populations: everything above was “high resistance” and everything below was “low resistance.” Since all its sites probably had relatively low resistance, it called levels “high” that we would call “low.” Obviously it is not possible to randomize by resistance level, which makes it very hard to conduct good studies. The WHO might be comparing areas with high resistance and low transmission to areas with low resistance and high transmission. Different people use different definitions for when a mosquito is resistant, though we don’t know which is best. I recommend GiveWell be careful when using these definitions as, though a mosquito might be resistant to some actions of the insecticide, it might not be sufficient to have a public health impact.” GiveWell's non-verbatim summary of a conversation with Dr. Thomas Churcher and Professor Hilary Ranson, May 31, 2017

  • Note that Ochomo et al 2017 was part of the WHO multi-country study to which Dr. Churcher and Professor Ranson are referring: "This study was implemented as part of a large, multicountry program set up to quantify the impact of insecticide resistance on the effectiveness of insecticide-based vector controls" Pg. 759, Ochomo et al 2017.

• 83

  Churcher et al 2016:

  • "Following a widely used transmission dynamics model of malaria (Griffin et al., 2010; Walker et al., 2015) it is assumed that an LLIN can alter a host-seeking mosquito behaviour in one of three ways: firstly it can deter a mosquito from entering a hut (an exito-repellency effect); secondly the mosquito can exit the hut without taking a bloodmeal; and thirdly it could kill a mosquito (with the mosquito either being fed or unfed). A mosquito that isn’t deterred, exited or killed will successfully blood-feed and survive. The public health benefit of LLINs depends not only on their initial effectiveness but also on how the properties of the net changes over its life-time. The ability of a net to kill a mosquito will decrease over time as the quantity of insecticide active ingredient declines. The non-lethal protection provided by the LLIN may also decrease with the decay of the active ingredient and the physical degradation of the net (i.e. the acquisition of holes). It is assumed that the underlying difference in hut trial mortality between sites for standard LLINs is caused by the mosquito population having a different population prevalence of pyrethroid resistance." Pg. 17
  • "Transmission dynamics mathematical models are a useful tool for disentangling the different impacts of LLINs. Though a person under an LLIN requires high pyrethroid resistance before LLINs start to fail (Figure 3C), the models predict that at a population level even low pyrethroid resistance can increase the number of malaria cases over the life-time of the net (Figure 4A). Hut trials measure feeding when the volunteer is underneath a bednet whilst in reality (and in the mathematical model) a percentage of mosquito bites are taken when people are not in bed. The loss of LLIN induced mosquito mortality is likely to decrease the community impact of LLINs, increasing average mosquito age and the likelihood that people are infected whilst unprotected by a bednet. This is primarily due to the shorter duration of insecticide potency of LLINs in mosquito populations with a higher prevalence of resistance (Wanjala et al., 2015). Without this change in the duration of pyrethroid activity, the epidemiological impact of pyrethroid resistance will only become evident once it reaches a high level (Figure 4A). The change in the community impact of LLINs can be seen in the increase in the number of cases in people who do not use nets. This change is substantial, reinforcing the need to consider community effects in any policy decision." Pg. 11
  • "Here we propose that information on the current malaria endemicity, mosquito species and population prevalence of pyrethroid resistance (as measured by bioassay mortality) can be used to predict the public health impact of pyrethroid resistance and choosing the most appropriate LLIN for the epidemiological setting. Firstly (1) a meta-analysis and statistical model are used to determine whether mosquito mortality in a bioassay can be used to predict the proportion of mosquitoes, which die in experimental hut trials and to define the shape of this relationship. Secondly (2), another meta-analysis of experimental hut trial data is analysed to characterise the full impact of pyrethroid resistance on LLIN effectiveness. Thirdly, information from (1) and (2) is used to parameterise a widely used malaria transmission dynamics mathematical model to estimate the public health impact of pyrethroid resistance in different settings taking into account the community impact of LLINs." Pg. 3

• 84

  "Here we propose that information on the current malaria endemicity, mosquito species and population prevalence of pyrethroid resistance (as measured by bioassay mortality) can be used to predict the public health impact of pyrethroid resistance and choosing the most appropriate LLIN for the epidemiological setting. Firstly (1) a meta-analysis and statistical model are used to determine whether mosquito mortality in a bioassay can be used to predict the proportion of mosquitoes, which die in experimental hut trials and to define the shape of this relationship. Secondly (2), another meta-analysis of experimental hut trial data is analysed to characterise the full impact of pyrethroid resistance on LLIN effectiveness. Thirdly, information from (1) and (2) is used to parameterise a widely used malaria transmission dynamics mathematical model to estimate the public health impact of pyrethroid resistance in different settings taking into account the community impact of LLINs." Pg. 3, Churcher et al 2016.

• 85
  • Following Strode et al 2014, we define “moderate resistance” as a mosquito mortality rate of about 25-80% in a WHO bioassay, or an average mortality rate of roughly 55%. See Strode et al 2014, Table 4, Pg. 5.
  • We are highly uncertain about the most appropriate estimate to use for the mosquito mortality rate (according to WHO bioassays) in regions where AMF would conduct marginal distributions; we used a mosquito mortality rate of about 50% as a very rough guess. A few data points that we considered are:
    • Very roughly, Figure 1 from Ranson and Lissenden 2016 (Pgs. 1-2) shows an average of a ~65% mosquito mortality rate in 2014 for WHO bioassays collected across sub-Saharan Africa, with the rate falling about 10% every 5 years. However, this estimate may be substantially biased toward a lower mortality rate than the true median mosquito mortality rate if researchers are more likely to collect WHO bioassays in areas where they expect to find high levels of insecticide resistance.
    • AMF’s largest distribution to date is in Uganda, where it is presently working to support the distribution of about 10.7 million ITNs. The following graph (provided by AMF) collates the percentage mortality in all published WHO bioassays testing for resistance to pyrethroids in Uganda:

      

      Based on this graph, we estimate that the average prevalence of insecticide resistance (as measured in WHO bioassays) in areas where AMF will distribute nets was ~55% in 2013. We expect this rate to fall over time and believe it will likely be worse during 2016-2019 (when the bulk of AMF nets are expected to be distributed and to be covering the people targeted by the distribution); on the other hand, these measures of insecticide resistance may have been taken in areas where researchers thought they would be more likely to find resistance, biasing the estimate toward lower mosquito mortality rates. We note that we have not vetted the studies underlying the graph, or whether it is a comprehensive account of all bioassay studies in Uganda. We also note that AMF's distribution in Uganda will include distribution of new nets (called PBO nets) that are expected to be more effective against mosquitos that have developed resistance to standard nets.

      • "AMF has signed an agreement with Uganda’s Ministry of Health to fund 10.7 million long-lasting insecticidal nets (LLINs) for distribution in Uganda’s upcoming universal coverage campaign (UCC). ... Key elements of our agreement include: AMF is funding 10.7m LLINs (US$26,438,253.60)," Against Malaria Foundation - AMF funds 10.7 million nets for distribution in Uganda.
      • "Tab 2: Insecticide resistance (Considered in assessing appropriate districts and sub-districts for distribution of PBO nets) - A list and summary of all Insecticide resistance (IR) studies in Uganda," AMF - Uganda Distributions Spreadsheet.
    • Ochomo et al 2017, which took place in Kenya, had an overall median mortality of about 82%, though the median mortality was about 88% in 2013 and 67% in 2014 and so could have fallen substantially in the intervening years. However, Ochomo et al 2017 may have selected a region that the authors believed was especially likely to have high levels of resistance: "We used insecticide resistance data (percentage mosquito mortality upon exposure to deltamethrin) to dichotomize clusters into high- and low-resistance clusters by using the median mortality for that year, namely, 88% for 2013 (clusters with mortality rates >88% were categorized as low resistance and those with mortality rates <88% as high resistance) and 67% for 2014 (clusters with mortality rates >67% were categorized as low resistance and those with mortality rates <67% as high resistance). In combined analysis of both years, we used the overall median mortality (82%) to dichotomize clusters into high or low resistance for net users and non–net users." Pg. 760, Ochomo et al 2017.
    • Bradley et al 2017, which took place in Benin, had median mosquito mortality rates of about 55% (for deltamethrin) and about 18% (for permethrin), though it may have been conducted in a region that the authors believed was especially likely to have high levels of resistance: "Mortality of mosquitoes exposed to deltamethrin was measured in all 21 clusters. The median number of mosquitoes exposed per cluster was 81 [interquartile range (IQR) 53–101]. Median mosquito mortality was 55.2% (IQR 47.4–68.5%). Mortality to permethrin was measured in 20 of the 21 clusters. The median number of mosquitoes exposed was 25 (IQR 22–35). Median mortality was 18.2% (IQR 8.1–32.2%). In all assays there was 0% mortality in the control group of mosquitoes." Pgs. 2-3, Bradley et al 2017.

• 86
  • Given a mosquito mortality rate of 50%, the model estimates that, where the dominant vector is Anopheles gambiae and there is 80% LLIN coverage, we should expect an absolute increase in clinical malaria cases of ~250 per 1,000 people per year. For a fully susceptible population, the model predicts ~175 clinical cases of malaria per 1,000 people per year. For a completely resistant population, it predicts ~800 clinical cases of malaria per 1,000 people per year. Thus, the model predicts that instead of ITNs averting 625 malaria cases per 1,000 people, they avert only 375, a 40% drop in efficacy (1 - (375/625) = 0.4).
    • To arrive at these estimates, we used Figure 5A, Pg. 9, Churcher et al 2016. Note that there are a variety of figures available that have different dominant malaria vectors and different nets coverage levels. It also appears that Figure 4A, Pg. 7, Churcher et al 2016 would provide a slightly different estimate (using the same technique, it suggests a 33% reduction in efficacy (1 - 500/750 = 0.333)). We are uncertain of the reason for the differences between these figures. Because we are using Churcher et al 2016 to get a rough sense of the magnitude of an appropriate discount we have not deeply analyzed differences between models.
    • Note that Anopheles gambiae is commonly a dominant vector for malaria in sub-Saharan Africa. For example, it is the dominant vector in Uganda. "Anopheles gambiae s.s is the dominant vector species and shows a sympatric distribution with An. funestus and less abundant An. arabiensis." National Malaria Control Programme Uganda - An epidemiological profile of malaria and its control in Uganda, Pg. 13.
    • Our impression is that it is reasonable to expect at least an 80% coverage rate of nets in most regions supported by AMF. See here for more information.
  • We would guess that estimating the reduction in the efficacy of nets in this way does not account for the fact that untreated nets still provide a physical barrier against mosquitos. Therefore, this method may overestimate the reduction in effectiveness of bed nets due to resistance.

• 87
  
  Some sanity checks that we considered include:
  • At a high level, it seems that we should expect insecticide resistance to reduce the effectiveness of bed nets by somewhere between 5% and 80%. The existence of any detectable insecticide resistance suggests that we should expect nets to at least have a small but meaningful reduction in effectiveness (i.e., at least about 5%). On the other hand, we estimate that about 20% of the impact of bed nets is attributable to the physical barrier of the net alone (i.e., without insecticide), implying a worst case scenario of an 80% reduction in effectiveness due to insecticide resistance. See sheet “Bednets-IR” in the 2016 version of our cost-effectiveness analysis for details. [Update March 8, 2019: We noticed that an error in the calculations for sheet “Bednets-IR” implies that actually about 25% of the impact of bed nets is attributable to the physical barrier of the net. We are in the process of updating this and aim to fix the references on this page to that calculation later this year.] We do not know whether the implied adjustment we drew from Churcher et al 2016 would take into account the protective effect of untreated nets.
  • Informally, it seems that insecticide resistance is a major concern but that most malaria experts believe that nets remain highly effective. Roughly translating this into a quantitative estimate might imply a discount in the 15-50% range. We have asked malaria researchers to give us their subjective quantitative estimates of the impact of resistance but all of them have declined.
  • Though there are limitations to the methodologies, we interpret recent observational studies as suggesting that nets continue to be effective. We have not quantified the implied effectiveness of nets from these studies relative to their historical effectiveness since the studies do not offer a straightforward method to do this, but the fact that the studies do not detect major limitations to nets’ effectiveness suggests that a discount of 50% or more would be too extreme.
  • An extremely simplistic model of insecticide resistance could assume a linear relationship between mosquito mortality and the effectiveness of bed nets. If 80% of bed nets’ effectiveness is due to insecticide rather than the physical barrier, and the insecticide is now 50% less effective than it was (i.e., assuming a mosquito mortality rate of about 50%), this would imply a reduction in the effectiveness of bed nets of about 40% (80%*50%=40%).
  • Previously, we had arbitrarily assumed that insecticide resistance reduced nets’ effectiveness by about 20% (see the 2015 version of our cost-effectiveness analysis on this page). Since recent evidence seems to imply that insecticide resistance is worsening over time, it seems that we should adjust our cost-effectiveness analysis so that our discount for nets' effectiveness increases over time.

• 88
  • For discussion of AMF's work in Uganda, see here
  • For the final adjustment in our cost-effectiveness model for 2017, see Row 14, Sheet "Nets," 2017 GiveWell Cost-Effectiveness Analysis — Version 5

• 89

  WHO Global Report on insecticide resistance 2010-2016

• 90

  For further details, see our cost-effectiveness analysis changelog page and search "insecticide resistance".

• 91
  • We noted last year that the results of other methods of estimating the insecticide resistance adjustment were roughly consistent with this simplified model. See this footnote.
  • See GiveWell - Insecticide resistance adjustment incorporating PBO nets for calculations showing the consistency of the model with the results of Protopopoff et al. 2018.

• 92
  • For calculations, see GiveWell - Insecticide resistance adjustment incorporating PBO nets
  • We have not confirmed that 100% of AMF-funded standard nets use permethrin, but our informal understanding is that this is the case.

• 93
  • WHO Global Report on insecticide resistance 2010-2016, Annex 1
  • For calculations, see GiveWell - Insecticide resistance adjustment incorporating PBO nets

• 94

  Professor Hilary Ranson, email to GiveWell, July 29, 2018

• 95
  • On pyrethroid and pyriproxyfen (PPF) nets see, e.g., "Conclusion: Olyset Duo was superior to Olyset Net in terms of personal protection and killing of pyrethroid resistant An. gambiae, and sterilized surviving blood-fed mosquitoes. Mixing pyrethroid and pyriproxyfen on a LN shows potential for malaria control and management of pyrethroid resistant vectors by preventing further selection of pyrethroid resistant phenotypes." Ngufor et al 2014, Pg. 1.
  • On pyrethroid and piperonyl butoxide (PBO) nets see, e.g., "Methods: A multi centre experimental hut trial was conducted in Malanville (Benin), Vallée du Kou (Burkina Faso) and Pitoa (Cameroon) to investigate the exophily, blood feeding inhibition and mortality induced by PermaNet® 3.0 (i.e. a mosaic net containing piperonyl butoxide and deltamethrin on the roof) comparatively to the WHO recommended PermaNet® 2.0 (unwashed and washed 20-times) and a conventionally deltamethrin-treated net (CTN).

    Results: The personal protection and insecticidal activity of PermaNet 3.0 and PermaNet® 2.0 were excellent (>80%) in the “pyrethroid-tolerant” area of Malanville. In the pyrethroid-resistance areas of Pitoa (metabolic resistance) and Vallée du Kou (presence of the L1014F kdr mutation), PermaNet® 3.0 showed equal or better performances than PermaNet® 2.0. It should be noted however that the deltamethrin content on PermaNet® 3.0 was up to twice higher than that of PermaNet® 2.0. Significant reduction of efficacy of both LLIN was noted after 20 washes although PermaNet® 3.0 still fulfilled the WHO requirement for LLIN." Corbel et al 2010, Pg. 1.

• 96

  "Methods: A 2 armed cluster-randomised controlled trial will be conducted in Burkina Faso to assess whether PPF-LLIN (containing 2% permethrin and 1% pyriproxyfen w/w) provide better protection against clinical malaria in children than 2% permethrin-treated LLINs. Study subjects will be recruited and provided with LLINs at the start of the study. The LLINs will be exchanged for PPF-LLIN by cluster in a step-wedge fashion so 3 months before the end of the 2 year trial all participants will have a PPF-LLIN. The primary endpoint will be clinical malaria incidence measured by passive case detection in a cohort of children, aged 6 months to 5 years. Anaemia and parasite prevalence will also be measured in children during cross-sectional surveys. Exposure to malaria parasites will be assessed using light traps followed by identification of common vector species and their sporozoite infection rates. Safety evaluation will include recording of adverse events and pregnancy outcomes. The main endpoint analysis will include adjusting for distance between village clusters with different types of nets, as the impact of PPF-LLIN is likely to increase as larger areas are dominated by PPF-LLIN, reducing the spill over of mosquitoes from villages with LLINs." Tiono et al 2015, Pg. 1.

• 97

  "Assuming prevalence of P. falciparum around 30% [37], 20 clusters in each arm and CV = 0.3, the study could detect a 30% reduction of prevalence, with 5% significance and 80% power, between the study arms if longitudinal cohorts for each cluster contain at least 45 children. Assuming mean Hb around 10.5g/dL, 20 clusters in each arm and CV = 0.1, the study could detect a difference of 1.0g/dL in mean Hb, with 5% significance and 80% power, between the study arms if at least 35 children were followed in each cluster." Tiono et al 2015, Pg. 9.

• 98

  "Background: Long-lasting insecticidal nets (LLINs) and indoor residual spraying (IRS) are the main malaria prevention interventions in Ethiopia. ... Methods/Design: This 2 x 2 factorial cluster randomized controlled trial is being carried out in the Adami Tullu district in south-central Ethiopia for about 116 weeks from September 2014 to December 2016. The trial is based on four arms: LLINs + IRS, LLINs alone, IRS alone and control. Villages (or clusters) will be the unit of randomization. The sample size includes 44 clusters per arm, with each cluster comprised of approximately 35 households (about 175 people). Prior to intervention, all households in the LLINs + IRS and LLINs alone arms will be provided with LLINs free of charge. Households in the LLINs + IRS and IRS alone arms will be sprayed with carbamate propoxur once a year just before the main malaria transmission season throughout the investigation. The primary outcome of this trial will be a malaria incidence based on the results of the rapid diagnostic tests in patients with a fever or history of fever attending health posts by passive case detection. Community-based surveys will be conducted each year to assess anemia among children 5–59 months old. In addition, community-based malaria prevalence surveys will be conducted each year on a representative sample of households during the main transmission season. The cost-effectiveness of the interventions and entomological studies will be simultaneously conducted. Analysis will be based on an intention-to-treat principle." Deressa et al 2016, Pg. 1.

• 99
  • "A multi-country study was undertaken in Sudan, Kenya, India, Cameroon and Benin to quantify the potential loss of epidemiological effectiveness of ITNs and IRS due to decreased susceptibility of malaria vectors to insecticides." Kleinschmidt et al 2015, Pg. 1
  • "Table 2: Study implementation details by country" Kleinschmidt et al 2015, Pg. 7.

• 100
  • "Methods: Malaria disease incidence rates by active case detection in cohorts of children, and indicators of insecticide resistance in local vectors were monitored in each of approximately 300 separate locations (clusters) with high coverage of malaria vector control over multiple malaria seasons. Phenotypic and genotypic resistance was assessed annually. In two countries, Sudan and India, clusters were randomly assigned to receive universal coverage of ITNs only, or universal coverage of ITNs combined with high coverage of IRS." Kleinschmidt et al 2015, Pg. 1.
  • "Measuring IR in many locations is also challenging and requires standardization of procedures and testing conditions such as temperature and humidity. There are more sensitive methods of resistance determination such as generation of population specific concentration or lethal exposure time curves [24, 38]. However these better metrics of the intensity of IR require multiple exposures per cluster which was not possible in logistically constrained study settings. The WHO susceptibility test measuring mosquito mortality within 24 h after 60 min of exposure to a standard concentration of insecticide on impregnated papers is commonly used and was considered operationally the most feasible." Kleinschmidt et al 2015, Pg. 12.

• 101

  "In Sudan, Kenya, Cameroon and Benin data collection is due to be completed in 2015. In India data collection will be completed in 2016." Kleinschmidt et al 2015, Pg. 1.

• 102

  “Comparison was made to a WHOPES-recommended LN treated with permethrin alone (Olyset Net; Sumitomo Chemical Company Ltd) and a LN treated with pyriproxyfen alone, which was formulated to the same technical specifications as Olyset Duo.” Ngufor et al 2014, Pg. 3.

• 103

  "GiveWell: Are there any other studies that provide explicit quantitative estimates of the impact of resistance that should inform our cost-effectiveness analysis?

  Professor Ranson: Results from a five-country WHO study on this topic should be published within a year or so. However, we, and others, have some issues with the methodology used. Aside from that, I do not know of any other relevant papers.

  Dr. Churcher: Not that I know of" GiveWell's non-verbatim summary of a conversation with Dr. Thomas Churcher and Professor Hilary Ranson, May 31, 2017