Estimating Equivalent Coverage Years for Long-Lasting Insecticide-Treated Nets (LLINs) – November 2020 version

We have published more recent information on our approach to estimating coverage years. See this section of our report on insecticide-treated nets.

Published: November 2020

Measuring Net Durability

Post-distribution net loss occurs for multiple reasons: attrition (i.e., nets were discarded due to damage, appropriated for other uses, given away, moved, or stolen), extensive holes in the net that make it permeable to mosquitoes, and decay of the net's insecticide component to the point that it is no longer effective. This section summarizes the quality control and field testing procedures used to evaluate net durability outcomes, which inform our estimate of equivalent coverage years.

LLIN Product Qualification

The World Health Organization's Pesticide Evaluation Scheme (WHOPES) maintains product testing standards for qualifying bednets for purchasing and field use. The full list of LLINs that meet WHO's prequalification standards and are eligible for procurement is available here. To date, all of the brands and types of bednets that AMF has procured appear on WHO's prequalified product lists and meet WHO's quality requirements.2

However, many of the quality control tests for WHO bednet prequalification are lab-based rather than long-term and field-based, and it is unclear how this qualification translates into estimated net lifespan in a 'typical use' environment. Testing requirements do not necessarily demonstrate that all brands of approved LLINs are identical in quality. The most vetted brands of LLINs pass three phases of testing, including long-term testing that includes three years of field monitoring for insecticide performance, and passing this test results in receiving a full recommendation from WHO and being named a 'reference class' product. Products that aspire to become reference class nets may receive an interim recommendation from WHO based on passing the first two phases of the full qualification process. Other 'generic' brands of nets receive a WHO recommendation based on equivalent performance to a WHO-recommended product with the same chemical and physical specifications in the first phase of qualification (lab tests for textile strength and chemical analysis of the insecticide component).3

Field Durability Monitoring

In 2011, WHO established field durability monitoring guidelines for LLINs.4 The President's Malaria Initiative (PMI) also maintains LLIN field durability monitoring advice for practitioners5 and a database of field durability sites. WHO released further instructions for collecting field durability data and using it to calculate net survival in 2013.6

Under current guidelines, a full battery of LLIN durability monitoring tracks four outcomes: attrition (survivorship), physical integrity, insecticidal activity, and residual insecticide content.7 Attrition and physical integrity are the easiest outcomes to measure, requiring no special equipment, and our impression is that they are the most commonly measured outcomes. Not all monitoring studies have the resources to track all potential outcomes.8 For optimal monitoring, WHO recommends a prospective monitoring design that tags a group of nets for follow-up at several predetermined points in time.9

Attrition

The attrition rate is the proportion of cohort nets that is no longer present or being used as intended at each point of follow-up.10 This rate grows cumulatively over time. There are several reasons why nets may be absent at follow-up. Households may have discarded nets due to damage; appropriated them for other uses; or reported them given away, moved, or stolen.11

Attrition is likely to be the most important factor determining net lifespan. The least effective net is one that isn't available for use. In addition, attrition heavily influences assessment of the condition of surviving nets, since it tends to remove the most damaged nets from the study cohort over time, while only remaining nets can be evaluated for physical condition. Attrition depends on subjective household decisions about when and why to dispose of nets, and these decisions don't necessarily align with the end of a net's useful lifespan.12 The visible, physical condition of a net influences decisions about whether to discard it, but households may discard nets that still have protective efficacy, including nets with only a few holes.13 Other households may hang onto nets that are in poor condition well beyond their expiration date. The household's net supply also influences its decisions to give away nets.14

Net attrition is somewhat difficult to measure, and monitoring data may be biased. The expectation that households' net usage will be monitored may cause them to change their behavior and keep used nets for longer than unobserved households.15 Prospective monitoring studies that follow the same group of households over time are able to track a consistent cohort of nets and reduce difficulties with recall, but these studies likely affect behavior, and we expect that, by reducing net disposal and reallocation, they may underestimate the role of attrition relative to damage in assessing net survivorship. Retrospective studies that follow up with households once at a designated point after distribution don't suffer the same behavioral effects, but they may also suffer from bias if researchers don't know how many nets surveyed households originally received.16 In either case, we can't verify the true reason nets are absent, and recall and reporting biases may affect self-reports of the causes of missing nets.17

Physical integrity

Physical integrity of LLINs refers to the durability of the net's textile. Nets eventually develop holes that may allow mosquitoes to pass through and feed. WHO recommends monitoring physical integrity by counting holes of different sizes in the net's fabric at each follow-up point and calculating a Proportionate Hole Index (pHI) value that relates to the total surface area of holes in the net. WHO divides nets into three condition categories based on pHI, which facilitates comparison of net condition across sites:18

1. Good (pHI=0-64): no reduction in efficacy.
2. Serviceable (pHI=65-642): damaged but still significantly more effective than no net.
3. Too torn (pHI=643+): doubtful protective efficacy and in need of replacement.

Our impression is that these net condition cutoffs are backed by some evidence of the extent of mosquito feeding inhibition at different levels of surface damage, but we have not vetted WHO's sources for its recommendation in detail. The WHO guidelines indicate that the pHI cutoff for a 'too torn' net was set conservatively based on the evidence.19

Physical integrity measurements relate to attrition because nets may be discarded over time due to visible wear. Physical condition can only be measured for remaining nets, so pHI measurements may decrease early in a distribution cycle as nets become damaged and rebound later in the cycle as households discard nets in poor condition.

Insecticide content and activity

LLINs contain a durable pyrethroid insecticide reservoir that is either incorporated within or coated onto the net fibers.20 This insecticide treatment makes up a large part of a net's protective efficacy. In our insecticide resistance adjustment, we estimate that in net RCTs, the insecticide component was responsible for 73% of the protective effect, though we'd expect that the role of insecticide relative to the physical barrier has since decreased for standard LLINs as insecticide resistance has increased.21 The goal of LLIN insecticide monitoring is to determine whether the net's insecticide continues to inhibit mosquitoes effectively over time (i.e., to determine bioefficacy). Insecticidal activity is typically measured by performing WHO cone bioassay or tunnel tests on a small sample of nets collected from surviving nets. The cone bioassay tests expose susceptible (non-resistant to insecticide) mosquitoes to sections of the net for a few minutes and subsequently measure the degree of mosquito knock-down and mortality to test insecticide bioavailability.22 WHO grades "optimal" insecticide performance as causing at least 80% mortality or at least 95% knock-down in cone or tunnel tests after three years of use.23 Testing for residual insecticide content in nets is a supplementary measure that helps with interpretation of the bioefficacy results, but bioefficacy results take precedence, since bioefficacy isn't directly correlated with insecticide content.24 Despite the importance of insecticidal activity for protective efficacy, WHO doesn't currently recommend factoring insecticide performance into estimates of median net lifespan. There are several reasons for this decision:

• Qualification trial methodology for 'reference class' nets requires that 80% of nets exhibit optimal performance on bioassay tests after three years in the field. This means that 'good' insecticide performance is baked into the LLIN qualification process, so we may expect that insecticide isn't the limiting performance factor.25
• The current methodology for measuring insecticide performance isn't ideal and can't test all nets enrolled in a trial. Bioassay tests are expensive and require removing a subset of nets from the field to a lab, so the sample size is small (usually 30-50 nets per time point).26 Therefore, the results have higher variance.
• There is not yet a simple test for insecticide residue in nets that is ready for field use.27 There also isn't a simple connection between insecticide performance and the amount of insecticide remaining in the net. While insecticide content does appear to decay rapidly in LLINs over time, nets can still perform well in bioassays with low insecticide residue.28
• It is unclear where to place the threshold for minimally acceptable insecticide performance. Studies sometimes use minimum criteria of 50% mortality or 75% knockdown, but these were not ratified by WHO as of the publication of this report in 2013.29

WHO's 2013 decision not to factor insecticide performance into median lifespan estimates was intended to be provisional, with the option of later incorporating information on insecticide.30

Note that the above describes quality control and testing procedures for standard LLINs that contain a pyrethroid insecticide. This report doesn't include analysis of the next generation of LLINs that include the chemical piperonyl butoxide (PBO) in addition to pyrethroid insecticide or other next generation nets with dual active ingredients. PBO durability is an active research question, and it's possible that PBO is less durable than standard insecticide treatments.31 We plan to look at PBO net insecticide durability separately in the future.

Differences between today's LLINs and nets used in RCTs

GiveWell's cost-effectiveness analysis of interventions that distribute bednets for the prevention of malaria relies on evidence from randomized controlled trials (RCTs) using conventionally treated nets (CTNs) to calculate the effectiveness of nets at averting child mortality. However, the CTNs used in these trials differ in key ways from the long-lasting insecticide-treated nets (LLINs) used today, namely in how they are impregnated with insecticide, when they are replaced, and what information and encouragement is given to recipients about the care and use of the nets. In order to accurately estimate the effectiveness of current net distributions at averting child mortality, we need to understand how net survival compares between LLINs and nets used in the RCTs that inform our estimate of net effectiveness at averting child mortality.

Unfortunately, net trials provide limited information on net durability, and trial conditions probably affected net coverage and maintenance behaviors in ways that would not apply to other contexts. In order to evaluate the nets used in trials, we have given the most weight to information about net distribution protocols from Phillips-Howard et al. 2003, a trial included in a Cochrane meta-analysis of the impact of insecticide-treated nets on mortality and malaria morbidity.32 Phillips-Howard et al. 2003 is the bednet mortality trial that makes up the majority of the child mortality effect estimate in the Cochrane meta-analysis.33 Additionally, we have relied on information about technical differences between the types of nets used and reviewed the additional underlying RCTs of the Cochrane child mortality effect estimate for information on net survival and the studies’ distribution protocols.34

Based on the somewhat limited information that we have, our understanding is that the nets used in trials likely differed from today's nets in the following ways:

• Insecticide technology. The CTNs (conventionally treated nets) that trials provided were a more primitive type of net with a less durable insecticide treatment than today's LLINs. CTNs are manually treated with insecticide and require frequent retreatment to maintain insecticide content, while LLINs are factory-treated with insecticide that releases over time.35 The studies included in our mortality estimate maintained a protocol of insecticide retreatment of the CTNs every 6 months to combat insecticide loss, while LLINs don't require retreatment.
• Shorter target use duration. All of the child mortality RCTs included in the 2018 Cochrane meta-analysis lasted for 24 months, while AMF generally funds distributions that take place every 36 months. This difference means that nets in trials were younger on average and thus probably less physically damaged on average than nets distributed today.
• Higher population coverage. Our research indicates that trials likely started with a higher baseline target coverage for the number of nets per person distributed, compared to AMF's target initial distribution coverage rate.36 In addition, some of the RCTs used in the Cochrane child mortality effect estimate, including Phillips-Howard et al. 2003 and Binka et al. 1996, distributed additional nets during the experiment.37 Phillips-Howard et al. 2003 reported minimal net attrition rates over 24 months,38 and another trial reported no reduction in coverage between the first and second year of the trial.39
• More intensive training and monitoring. Phillips-Howard et al. 2003 provided subjects with intensive training on net care and use. Researchers provided subjects with extensive training on net use and care, quarterly monitoring visits, and materials for net repair at each monitoring point.40 Another trial, Nevill 1996, also reported extensive training and support for net recipients.41 We expect that these experimental conditions may have resulted in less net attrition and better upkeep in trials compared to field distributions, although it's possible that net upkeep is better today due to households having more experience using and caring for nets.

Available literature on LLIN lifespan and field durability

We would ideally like to find a systematic review that aggregates many field studies of net durability across multiple brands of LLINs together into an empirical estimate of an average LLIN lifespan, highlighting possible quality differences between different brands of LLINs. To the best of our knowledge, this type of work is not available to date. We have found two sources that provided some particularly helpful synthesis of the available information. While the methodology is somewhat limited, they suggest that median lifespan for an LLIN is around two years. Both sources find that net lifespan is variable across contexts.

Synthesis source 1: Bhatt et al. 2015

Bhatt et al. 2015 is a mathematical modeling paper that compares data on net access, coverage, and usage from national household surveys to programmatic reports on the number of nets distributed across 40 countries in Sub-Saharan Africa.42 This study is limited in that it only tracks net attrition over time and not the condition of surviving nets, so it may overestimate functional net survival. This study is also retrospective and depends heavily on the accuracy of administrative data sources to assess net attrition relative to nets distributed. Note that we have not examined the modeling calculations behind the results or vetted the quality of the study in detail.

The main net survival result from Bhatt et al. 2015 finds a median net lifespan of 23 months (95% confidence interval: 20-28 months) across 40 countries in Sub-Saharan Africa between 2011 and 2013, with substantial variation between different countries.43

Synthesis source 2: Kilian et al. 2018

Kilian et al. 2018 is a poster presentation summarizing and comparing field monitoring data from the President’s Malaria Initiative (PMI) spanning 24 months post-distribution for five different brands of LLINs distributed in five different countries.44 Unlike Bhatt et al. 2015, this presentation provides evidence on net condition over time and across different net brands, rather than on attrition only,45 though it is more limited in terms of breadth of countries and time periods studied.

Kilian et al. 2018 does not present an average lifespan result, but it reports median lifespan estimates for each brand by location. These estimates vary widely—from 1.4 years for the Dawa Plus 2.0 in the Democratic Republic of the Congo to 5.6 years for the same brand in Nigeria.46 It concludes that differences in results between locations were greater than differences across brands.47 The study also examined the effectiveness of the LLIN insecticide and found that insecticidal effectiveness was sufficient at most sites 24 months post-distribution.48

GiveWell's model of LLIN equivalent coverage years

Using the three key components of net durability (attrition, physical integrity, and insecticide activity), we have estimated the expected survival and protective coverage of LLINs distributed by AMF, relative to the CTNs used in the RCTs that GiveWell relies on to estimate bednet effectiveness for averting child mortality. Our model is available here. Our current best guess is that an LLIN distributed by AMF confers 2.11 equivalent coverage years over 36 months of potential use (before the next distribution round occurs), where one "equivalent coverage year" represents a similar quality of protection to a CTN distributed in one of the bednet RCTs over one year of the trial. This can be interpreted to mean that today's LLINs provide about 30% less coverage and protection per year (when averaged across the three years of a typical distribution) compared to the coverage and protection provided by trial nets per year (averaged across two years).49 Note that this is a simplification, since our calculations indicate that LLIN coverage declines substantially over the course of 36 months.

Note that this analysis is a simplified model that relies on limited data and does not take into account a number of factors. There is significant uncertainty surrounding our assumptions. Additionally, we frame this input as a performance outcome for a reference class net in an 'average' net distribution, but we understand that durability outcomes may in fact vary substantially across time and space. We provide these simplified estimates (a) for transparency about important assumptions used in our cost-effectiveness analysis and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

Below, we explain the key choices and assumptions that we made in our model.

Data Sources

Our current input starts with a durability estimate for Vestergaard's PermaNet 2.0 LLIN, which is based on all field monitoring studies of PermaNet 2.0 that we are aware of that present results compliant with WHOPES monitoring guidelines. We then make rough adjustments for other types of nets that AMF purchases that may have more limited performance data.

Why we chose the PermaNet 2.0 as a reference net

In order to balance having a consistent set of inclusion principles with tractability, we decided to limit the scope of this project and focus primarily on the PermaNet 2.0 (the net which makes up the largest proportion of AMF’s purchases). While we'd ideally have a comprehensive set of durability monitoring data across all available brands of LLINs that might allow us to establish which types of LLINs are the most cost-effective overall, our preliminary research indicated that this would be difficult. Many monitoring studies of LLINs are available, and it would be time-consuming to search out, review, and extract data from all of them. Instead of doing a comprehensive search, we explore some rough comparisons of the PermaNet 2.0 with other brands of nets that AMF purchases in large volumes, particularly the Yorkool LN, below.

We chose the PermaNet 2.0 to serve as a reference net for the following reasons:

• Relevance to AMF's distributions. AMF has purchased significant quantities of both PermaNet 2.0s and Yorkool LNs.50
• Extensive data availability. There appears to be a particularly large amount of field monitoring data available for the PermaNet 2.0. Studies are available from several different countries, and multiple studies have long follow-up duration up to 36 months.
• Tractability. Focusing on one key brand of net as a starting point for our model was easier and clearer to execute than other possible sub-divisions of the literature. We expected searching for "PermaNet 2.0" would reliably yield the relevant research published to date with few studies that we needed to filter out. Identifying another filter to use, such as study methodology or duration, would be challenging without first finding and reviewing the majority of published field monitoring studies and likely would also lead to discarding a large amount of information (for example, there are many monitoring studies with fewer than 3 years of follow-up).

The criteria we used for inclusion in our dataset are:

• The study must name the PermaNet 2.0 as one of the nets studied.
• The study must present results from post-distribution field monitoring that captured net performance in real-life conditions. We excluded lab, hut, and wash trials from the evidence base.
• The study must report information on attrition and/or pHI at a follow-up point up to 36 months post-distribution, and it must report results in a way that conforms to WHOPES monitoring guidelines. For example, results that reported only average pHI rather than the proportion of “too torn” nets were excluded. The study does not necessarily have to include a full set of usable outcomes to be included as long as at least one variable is relevant.

In total, our criteria identified 11 monitoring studies.51 These studies span distributions that occurred between 2005 and 2017 in 10 different countries.52

Aggregating data from multiple monitoring surveys

We have averaged the data from the 11 PermaNet 2.0 monitoring surveys that met our inclusion criteria in this spreadsheet. We aggregated the PermaNet 2.0 data by taking a simple average of the proportion of surviving nets and the proportion of surviving nets classified as 'too torn' across all studies at each follow-up interval of 6, 12, 18, 24, 30 and 36 months post-distribution.53 Results are not available for every time point for every study, so each point-in-time estimate is generally based on fewer than 11 results. This gives a rough average of net degradation for each year post-distribution and allows us to better compare net durability between CTNs and LLINs.

Note that while some nets still survive in serviceable condition 36 months post-distribution, we don't track these nets or include them in our coverage years estimate, since we expect them to be superseded by the next round of net distribution. This is a simplifying assumption; in some cases, surviving nets will affect how long households wait to start using new nets. We adjust for pre-existing nets in a separate parameter in our model.

Calculating equivalent coverage years for the PermaNet 2.0, relative to CTNs used in trials

In order to account for the difference in coverage levels between LLINs in today's mass campaigns and CTNs in trials, we estimate LLIN coverage in terms of "equivalent coverage years" (i.e., years of coverage at a level equivalent to the coverage provided by CTNs in the trials we rely on to estimate benefits). See above for more information on the differences between LLINs and trial nets, which inform these assumptions.

In order to estimate an input for equivalent LLIN coverage years, we need to understand how each facet of net durability compares between LLINs and nets used in the RCTs that inform our estimate of net effectiveness at averting child mortality. Estimated annual mortality benefits from trials should scale with the protective efficacy of the nets provided during the experiments. If we were to use an estimate of how quickly LLINs wear out in absolute terms to calculate coverage duration in our cost-effectiveness analysis of bednets, we would be implying that the nets used in trials remained in good condition and coverage was maintained for the full target population throughout the trial period. This is unlikely to be the case.

We make the following assumptions to compare PermaNet 2.0 LLINs to CTNs used in trials:

• Attrition. We count all measured attrition of LLINs in PermaNet 2.0 monitoring studies, less nets that we estimate were given away and used elsewhere, as lost relative to net coverage years in RCTs. As explained above, it is likely that attrition was minimal or even negative in net trials. As a result, we expect net attrition to be an important point of difference in the level of protection offered between today's field distributions and earlier RCTs.
• Physical integrity. None of the relevant RCTs reported data on holes in nets, so this input is highly uncertain. We expect that LLINs in AMF distributions likely have somewhat more holes on average than CTNs used in trials, because trial nets may have deteriorated more slowly due to increased training, monitoring, repair, and redundancy (due to high target coverage rates under trial conditions). We use a rough guess that nets used in trials took one-third less damage than the average for PermaNet 2.0 nets for each relevant monitoring point up to 24 months post-distribution (when trials ended).54 This assumption could be wrong if the behavioral effects of monitoring led households to avoid throwing away worn-out nets in RCTs, artificially reducing attrition.55
• Insecticide. We do not make adjustments for the effectiveness of the insecticide component of LLINs relative to trial CTNs, based on evidence that average insecticide content for LLINs over a 3-year horizon is similar to CTNs that are fully re-treated every 6 months as specified in trials, as well as evidence that CTNs maintained adequate insecticidal activity over the study period in the one underlying RCT for which data is available. This is an uncertain assumption because it's based on limited evidence that's somewhat challenging to interpret. One study, from Uganda, found that both LLINs and CTNs lose a considerable amount of insecticide over time, with different time profiles—insecticide content in CTNs decays more quickly, and with regular retreatment it would experience frequent peaks and valleys as insecticide degrades and is replenished, compared to a gradual decline of roughly 20% per year in insecticide content for LLINs.56 Based on data from this paper, we estimate that the differential timing of peaks and troughs in the insecticide content of CTNs roughly balances out over the long run to be equivalent to the average insecticide content of LLINs.57 We interpret these results cautiously because comparing average residual insecticide content may not translate linearly into comparing insecticidal bioavailability to kill or knock down mosquitoes. However, one RCT collected data on insecticidal activity and found that CTNs performed well across the study period.58 The available data on insecticide content from RCTs suggest that regular retreatment was reasonably successful at restoring insecticide content, though insecticide performance may have suffered in trials if CTNs were not retreated on time.59

We estimate that the PermaNet 2.0 provides 2.27 equivalent coverage years across a 3-year distribution. See our calculations here. In the following sections, we explain how we adjust this figure to reflect how we expect other brands of nets to perform relative to the PermaNet 2.0.

Accounting for differences between net brands

Evidence on quality differences

Many of the other LLINs that AMF has purchased differ in their product specifications (e.g., fabrication, density of the weave, type of insecticide) from the PermaNet 2.0.60 The evidence on whether we should expect major quality differences between net brands appears to be limited and mixed. As described above, preliminary evidence comparing field performance across several countries suggests that differences between settings may be more important than differences between brands. However, not all net products have faced the same rigor and duration of quality control testing as the PermaNet 2.0 (see here). We are aware of at least two brands of LLIN that received product recommendations from WHO but later had their recommendations revoked over failure to pass field testing and manufacturing defects.61 (AMF has never purchased nets from either of these brands.)

AMF-specific adjustment for Yorkool LN net quality

AMF has purchased several different brands and types of LLINs over its history. The PermaNet 2.0 and the Yorkool LN have both been purchased frequently over time.62 The Yorkool LN is of particular interest to us given the relatively high number that AMF has purchased and its status as a 'generic' net product that did not have to undergo field durability testing for WHO approval. If there are quality differences between the Yorkool LN and PermaNet 2.0, these seem the most likely to drive changes in our equivalent coverage years estimate.

We did a literature review of durability data available for the Yorkool LN and found four relevant studies.63 One study, Rahaivondrafahitra et al. 2017, provided evidence on the durability of both the Yorkool LN and the PermaNet 2.0 after 24 months of use in Madagascar,64 allowing us to directly compare the two brands. In our head-to-head comparison based on this study, we found that Yorkool nets performed similarly to PermaNet 2.0 on physical survival and performed worse on insecticidal activity.65

The additional three studies provide data on Yorkool LN nets, but not PermaNet 2.0. Thus, we compare the data on Yorkool LN nets in these studies to our expectations for how well PermaNet 2.0 would perform on physical integrity and to WHO performance requirements for insecticide. On physical survival, one of these three studies found more nets in serviceable condition (according to pHI) relative to our expectations for PermaNet 2.0, another found somewhat fewer Yorkool nets surviving in serviceable condition, and the third did not report on physical survival.66 On insecticidal activity, two of the three remaining studies found that the Yorkool LN net passed WHO guidelines for mosquito mortality in cone tests, while one found that the Yorkool LN net failed to meet requirements.67

Altogether, the four studies suggest that Yorkool LN performs similarly to PermaNet 2.0 on physical integrity and performs worse on insecticide. However, the evidence on insecticide is difficult to interpret. In the two trials from Madagascar where Yorkool LN performed poorly on insecticide, other brands of nets (including PermaNet 2.0 in the one head-to-head trial) also failed to meet WHO requirements, though Yorkool performed worse than PermaNet 2.0. A possible interpretation of this evidence is that Madagascar is a particularly harsh context for net survival.68

Based on this evidence, we calculated a rough effectiveness adjustment for the Yorkool LN relative to the PermaNet 2.0. We currently estimate that the Yorkool LN provides about 15% worse protection than our estimate for the PermaNet 2.0,69 which equates to 1.93 equivalent coverage years over a 3-year distribution.70
Here is a summary of our calculations for the Yorkool net:

• We assume that 40% of the net's protective effect comes from the physical barrier and 60% comes from the insecticide component. Details in footnote.71
• In a head-to-head comparison of Yorkool and PermaNet 2.0 in the 2015 Madagascar distribution, we estimate that Yorkool nets performed 24% worse at 24 months post-distribution (calculation here). The physical survival component of the Yorkool nets performed about the same, while the insecticide component performed about 37% worse.
• After assigning 50% weight to the head-to-head comparison results and 50% weight to results from the other relevant literature, the performance adjustment falls to approximately 15% worse than the PermaNet 2.0.

Bottom Line

As a result of this investigation, in our cost-effectiveness analysis, we are now using an input of 2.11 for the equivalent coverage years of an LLIN provided through AMF’s distributions, relative to nets used in the RCTs that the cost-effectiveness analysis relies on to estimate the impact on child mortality. Nets distributed in DRC receive a 17% durability penalty on this input in our cost-effectiveness analysis based on evidence that nets may decay more quickly in this setting.72

This does not represent a median net lifespan estimate (the outcome reported by some literature). Rather, it is a relative input intended to scale along with the expected coverage and protective efficacy of nets used in RCTs.

Our input for equivalent coverage years for a PermaNet 2.0, our reference point net, is 2.27.73 Although AMF purchases a significant number of PermaNet 2.0s, it also purchases many other brands of nets, some of which we believe may provide somewhat worse protection than PermaNet 2.0. In order to arrive at our estimate of 2.11 equivalent coverage years for AMF nets,74 we used our estimate for PermaNet 2.0 as a starting point, then adjusted the estimate based on:

• our rough guess of the effectiveness of each net type compared to PermaNet 2.075
• the proportion of each net type as a total of AMF’s net purchases from 2018 through 2020.

Our Process

We searched Google Scholar, VectorWorks' resources page, and PMI's monitoring database for synthesis studies on LLIN lifespan and durability, and later repeated these searches for literature testing the durability of the Vestergaard PermaNet 2.0 and Yorkool LN nets specifically. We also relied heavily on WHO publications regarding LLIN product qualification, field testing, and durability measurement.

We spoke with three outside experts regarding this project, two experts who have led projects to synthesize research on net quality and durability and one funder who has supported research in this area. Finally, we reviewed documentation on net purchases from AMF. AMF reviewed this page prior to publication.

Sources

Document	Source
Ahogni et al. 2020(a)	Source (archive)
Ahogni et al. 2020(b)	Source (archive)
Alaii et al. 2003	Source (archive)
Bhatt et al. 2015	Source (archive)
Binka et al. 1996	Source (archive)
Fettene, Balkew, and Gimblet, 2009	Source (archive)
GiveWell, DRC net durability adjustment, 2021	Source
Gleave et al. 2018	Source (archive)
Habluetzel 1997	Source (archive)
Hakizimana et al. 2014	Source (archive)
Ketoh et al. 2018	Source (archive)
Kilian et al. 2011	Source (archive)
Kilian et al. 2015	Source (archive)
Kilian et al. 2018	Source
Koenker et al. 2014	Source (archive)
List of AMF distributions	Unpublished
List of AMF distributions, public version	Source
Lorenz et al. 2019	Source (archive)
Morgan et al. 2015	Source (archive)
Nevill et al. 1996	Source (archive)
Phillips-Howard et al. 2003	Source (archive)
Pryce, Richardson, and Lengeler 2018	Source (archive)
Rahaivondrafahitra et al. 2017	Source (archive)
Randriamaherijaona, Raharinjatovo, and Boyer 2017	Source (archive)
Sahu et al. 2020	Source (archive)
Strode et al. 2014	Source (archive)
Tan et al. 2016	Source (archive)
The Global Fund, QA Information Notice, DawaPlus 2.0, 2019	Source (archive)
USAID, Durability Monitoring of LLINs in Myanmar, 2019	Source (archive)
USAID, Durability Monitoring of LLINs in Zanzibar, Tanzania, 2019	Source (archive)
USAID, LLIN durability monitoring guidelines, 2019	Source (archive)
Van Roey et al. 2014	Source (archive)
WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016	Source (archive)
WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011	Source (archive)
WHO, Prequalified Lists: Vector control products	Source (archive)
WHO, Vector Control Technical Expert Group Report to MPAC, 2013	Source (archive)
WHO, World Malaria Report, 2012	Source (archive)

• 1

  See our 2012 investigation of net lifespan.

• 2

  List of AMF distributions

• 3
  • There are 3 phases for full product qualification for a 'reference' (innovator) LLIN with respect to insecticidal protection:
    • Phase I: laboratory tests of regeneration time, wash resistance index, and active ingredient content..
      • “[T]he following parameters are currently used for the laboratory (Phase I) evaluation of an innovator (“reference”) pyrethroid-LLIN: regeneration time, wash resistance index following a minimum of 20 standard laboratory washes and active ingredient content (with ± 25% tolerance limit of the specification).” WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016, Pg 23.
    • Phase II: hut trials that serve as small-scale field tests to validate Phase I results.
      • “Phase II is a small scale field trial for an artificially aged net to understand its efficacy and validate Phase I defined characteristics through field testing in experimental huts. Phase II studies do not measure absolute values of mortality and blood-feeding inhibition but measure relative efficacy of a candidate LLIN compared with a positive control LLIN.” WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016, Pg 23.
      • “Phase II (experimental hut trials)” WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016, Pg 28.
    • Phase III: a full 3-year field evaluation.
      • “[T]he Phase I and II studies are not designed to determine the operational durability of LLINs in the field, which is the purpose of Phase III testing.” WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016, Pg 23.
      • See WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016, Pg 28, Annex 3, “Phase III (large-scale trials)”, “Evaluation parameters for reference LLIN”: “Long-lasting insecticidal efficacy (up to 3 years)”
  • WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016 implies that the physical properties and material durability of the nets are also assessed.
    • "The chemical and physical properties are defined within WHO specifications. These include description of the LLIN, active ingredient (identity, content), wash resistance index, physical properties (mesh, dimensional stability, bursting strength, weight of netting), flammability and storage stability." Pg 23.
  • "Time-limited interim recommendations are issued for innovator LLINs after phase II testing and evaluation by WHOPES." WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016 Pg 53.
  • However, not all LLIN products approved for use by WHO have the same level of evidence backing. Products are broadly divided into reference class LLINs (which have undergone at least one long-term field trial of protective effectiveness) and generic LLINs (which perform similarly to reference class nets in short-term lab tests) (see Pg 28, Annex 3). To obtain 'Equivalence' product certification for a generic LLIN, the product must show identical chemical and physical properties/performance to a reference class LLIN, based only on results of Phase I laboratory testing.
    • “Currently, equivalent LLINs must demonstrate they have identical chemical and physical properties and release characteristics as the reference (“comparator”) nets. The chemical and physical properties are defined within WHO specifications. These include description of the LLIN, active ingredient (identity, content), wash resistance index, physical properties (mesh, dimensional stability, bursting strength, weight of netting), flammability and storage stability. Release characteristics are defined in Phase I laboratory testing and include regeneration time and wash resistance index. These must be identical to the reference LLIN for equivalency.” Pg 23.
    • Also see Pg 28, Annex 3, “Phase II” and “Phase III”, “No test required at present”.
  • For this method of determining equivalency to be valid, we have to assume that Phase I results correlate strongly with Phase II and Phase III performance in the field. WHO cautions that "Phase I and II studies are not designed to determine the operational durability of LLINs in the field, which is the purpose of Phase III testing." Pg 23.
  • It is possible that the actual performance of generic nets varies more widely than reference class LLINs, which may make us more skeptical of generic nets. However, Phase III qualification is a lengthy process, and high barriers to entry in the nets market may be problematic for innovation and pricing.
  • See WHO, Determination of Equivalence for public health pesticides and pesticide products, 2016 Pg 28, Annex 3, for additional details on the evaluation parameters for reference and generic LLINs.

• 4

  WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011

• 5

  USAID, LLIN durability monitoring guidelines, 2019

• 6

  WHO, Vector Control Technical Expert Group Report to MPAC, 2013
  

• 7

  “LLIN monitoring measures the effect of normal daily use on four outcomes: 1. Attrition (survivorship), as measured by the loss of nets from households, 2. Physical durability, as measured by the number and size of holes in the net, 3. Insecticide effectiveness, as measured directly but imprecisely by bioassay, and 4. Insecticide content analysis, as measured accurately by chromatography.” USAID, LLIN durability monitoring guidelines, 2019, Pg 1.

• 8
  • "At a minimum, all countries should have the capacity to measure attrition and physical integrity. These outcomes do not require any special equipment or expertise." USAID, LLIN durability monitoring guidelines, 2019, Pg 2.
  • "Second, bio-assays are difficult to do in the field. When done in an entomological laboratory they need - like chemical residue analysis - destructive sampling, meaning that the nets have to be removed and can no longer be followed. In addition, both tests are quite expensive and not available everywhere." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 9.

• 9

  "Prospective studies are the first choice, as they have the key advantage of allowing full control of distribution of LLINs and close monitoring of all outcomes, especially attrition and integrity." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 4.

• 10

  “Attrition (opposite of survivorship) is the proportion of nets no longer in use as intended after a defined period after their distribution to the households. Attrition can be categorized by the main reasons why a net is no longer used, namely decay (e.g. destroyed, so torn and worn out that it is considered useless for protection against mosquitoes), absence (e.g. stolen, given away, moved) or used for other purposes.” WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011, Pgs 3-4.

• 11

  Koenker et al. 2014, Pg 4, Table 2.

• 12

  "Household decisions around end of net life are highly subjective. Decision-making about when a net is no longer useful has been discussed in one study in Senegal [11], where respondents were asked hypothetical questions about when they would discard nets in varying degrees of disrepair, and what they would do with it. Most respondents stated that they would prefer to get a new net when possible rather than attempt to repair their nets when damaged. Batisso et al. [7] found in Ethiopia that the primary reason for non-use was that nets were considered too old or torn, although the condition of these ‘unusable nets’ was similar to other nets in use in the community: nets were considered old when they only had a few holes. One third of nets were discarded when they were just under a year old, but the physical condition of these nets was not reported. Reports from qualitative research in Madagascar on decisions to give up nets for recycling also shed light on the reasons why households might prefer to discard or keep old nets. These depended on whether the family felt they had sufficient nets to protect all family members, whether they had paid for the net or received it free, whether they were currently using it for an alternative purpose, among other reasons. [12]” Koenker et al. 2014 Pg 2.

• 13

  A survey from Ethiopia (methodology unknown) found that nets discarded weren't in worse average condition than nets still present, and nets with only a few holes may have been discarded. "Batisso et al. [7] found in Ethiopia that the primary reason for non-use was that nets were considered too old or torn, although the condition of these 'unusable nets' was similar to other nets in use in the community: nets were considered old when they only had a few holes." Koenker et al. 2014, Pg 2.

• 14

  "The rate of giving away nets very much depends on the supply situation (i.e. whether or not the household has enough, or more than enough, nets to cover all family members) as well as the general attitude towards sharing with others in the family, extended family or community. Neither of these considerations is directly linked to the qualities of the LLIN product, although the acceptability or preference for a type of net could influence the decision of which net to give away provided more than one brand was distributed to the same household (which is uncommon in campaigns and not recommended by the guideline for prospective studies)." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 6.

• 15

  "In a 'real life situation', net users will discard a net when they feel its condition is so poor that it no longer serves its purpose. In a prospective study, however, where an ID-number is attached to the net and users are aware that the net will be followed up over time, they tend to keep a damaged net much longer than they normally would." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 4.

• 16

  "[R]etrospective methods have a major disadvantage, in that attrition (loss of nets) is difficult to estimate unless there is accurate information on number of nets originally distributed to each household. Without this information, we can observe how many nets are present in the household at a given time point, but we cannot know what proportion they represent of the original number that were distributed; thus the proportion lost since distribution remains unknown." WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011, Pg 6.

• 17

  "When a net is found to be no longer present in a household, care must be taken to determine the reason for its loss or absence. It is difficult to rule out the possibility of bias when a high proportion of nets are reported as having been given away or moved to other locations while still intact." WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011 Pgs 5-6.

• 18
  • ”The solution to the problem is to use pHI cut-offs that distinguish the following categories:
    1. LLIN in “good” condition where there is no reduction of efficacy compared to an undamaged net
    2. LLIN in “acceptable” condition in the sense that their effectiveness is somewhat reduced but still provide significantly more protection than no net at all.
    3. LLIN “torn” where its protective efficacy for the user is in serious doubt and the net should be replaced as soon as possible.

    WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 7

  • Also see page 8, Table 1.

• 19
  • "The 100 cm2 hole surface area for a "good" ITN is based on strong evidence from hut trials that there is no reduction of feeding inhibition compared to an intact ITN at that level. The evidence on the upper limit of 1000cm^2 is less strong as not studied well, but there is reasonably good evidence that there still is protection/feeding inhibition between 900 and 2000cm^2 hole surface and therefore the approximate lower limit of this range has been chosen. These hole surface area values were then back-translated into pHI values based on the surface ratio from the table." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 8.
  • Also see Pg 8, Table 1.

• 20

  "The CTNs (which require dipping into insecticide and which also require retreatment at least once a year) must have been impregnated with a WHO-recommended pyrethroid with the recommended formulation and dose (see Table 1 for recommended impregnation regimens). The LLINs (which are factory-treated nets where the insecticide is incorporated within or bound around the net fibres) must have had either interim or full recommendation from the WHO." Strode et al. 2014

• 21

  This may not be true for next-generation nets with additional additives to combat insecticide resistance, such as PBO nets.

• 22

  "WHO cone bioassays: WHO cone assays should be conducted on 25 cm x 25 cm pieces cut from positions 2, 3, 4 and 5 of each sampled net, which should be adjacent to the places
  from which the netting for chemical assay was collected. Position 1 should be excluded, as it may be exposed to excessive abrasion in routine use, as this portion of the net is frequently handled when it is being tucked under the bed or mattress. Two standard WHO cones are fixed with a plastic manifold onto each of the four netting pieces (Figure 3). Five susceptible, non-blood-fed, 2–5-day-old female Anopheles (species to be stated in the test report) are exposed for 3 min in each cone and then held for 24 h with access to sugar solution. Two replicates should be placed at each position. Knock-down is measured 60 min after exposure, and mortality is measured after 24 h." WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011, Pg 12-13.

• 23
  • "Insecticidal activity (bioefficacy) is the degree of knock-down, mortality or inhibition of blood-feeding induced in susceptible mosquitoes, as determined by standard WHO test procedures and criteria (i.e. cone bioassay, tunnel test) (1). Insecticidal activity is associated with the type and content or availability of insecticide. The insecticide content is expressed as g/kg or mg/m2 of the LN and is determined by the method outlined in WHO specifications for LNs. This information is of value in interpreting data on bioefficacy. Insecticidal activity can be assessed as a function of length of use." WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011, Pg 4.
  • ”The bioassay results for the netting pieces from each sampled LN should be pooled to
    determine if the net meets the WHO efficacy requirement, i.e. ≥ 80% mortality or ≥ 95%
    knock-down. If the net fails these criteria, a tunnel test should be conducted on one of the
    four net samples that caused mortality closest to the average mortality in the cone bioassay.” WHO, Guidelines for monitoring the durability of long-lasting insecticidal mosquito nets under operational conditions, 2011, Pg 13.
  • “The WHOPES cut-off of 80% of nets effective in WHO cone bioassays (with ≥95% Knock-Down or ≥80% mortality) or tunnel tests (≥80% mortality or ≥ 90% blood-feeding inhibition) is designated as the optimal level required after three years of use to decide on recommendation for public health use, not as an end-of life determination or a minimally acceptable performance." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 9.

• 24

  "The measurement of insecticidal content is a supplementary tool for the monitoring of insecticidal activity that may be done on the same cohort of nets sampled for bioassays. Content testing should not be done independently of bioassays. Determination of insecticidal content can be used to confirm the bioassays and estimate insecticide retention rates across different settings and in different LLIN products. However, measurement of insecticidal content requires highly specialized capacity that is likely limited or absent in nearly all PMI countries. Therefore, this must be done either at CDC or at a WHO collaborating center where the cost of analysis is approximately $150 - $350 per sample. Furthermore, in some cases there is a poor correlation between insecticidal content and insecticidal activity, particularly for some LLINs made of polyethylene with insecticide directly incorporated into the fiber. We do not generally recommend carrying out content testing for nets types which incorporate insecticide in solution in the net fiber." USAID, LLIN durability monitoring guidelines, 2019, P. 3.

• 25

  "Current WHOPES evaluations for LLIN include in phase II wash resistance and regeneration criteria in the laboratory. In addition, the phase III field testing assesses the performance of each LLIN brand after three years of field use requiring that at least 80% of surviving nets exhibit optimal bio-efficacy [6]. This usually includes the performance of the LLINs in several locations and settings as part of the evaluation. It can, therefore, be assumed that if a product has received WHOPES recommendation, there is sufficient evidence that insecticidal protection will last for at least three years. On this basis, it is recommended to exclude insecticidal effectiveness for the time being from the estimation of LLIN survival and base estimations on measures of attrition and integrity only." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 10.

• 26

  "[B]io-assays are difficult to do in the field. When done in an entomological laboratory they need – like chemical residue analysis – destructive sampling, meaning that the nets have to be removed and can no longer be followed. In addition, both tests are quite expensive and not available everywhere. As a result, usually only a small sub-sample of 30-50 nets is tested [6]. This excludes a determination of insecticidal outcome for each net in a cohort, which would be needed for a net survival estimate." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 9.

• 27

  "[F]ield applicable tests for pyrethroid residue that allow a quantitative or at least a semi-quantitative assessment of a minimal dose would solve the problem if they can be shown to correlate well with the bio-available insecticide. A variety of such tests have been developed [14-19], but none of them is ready for general application in “routine” surveys." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 10.

• 28

  "Existing data suggests that high Knock-Down rates and mortality can be achieved at very low levels of insecticide [7-11], much lower than what has been seen after three years in WHOPES Phase III evaluations (see WHOPES reports)." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 9.

• 29
  • "[T]here is as yet no clear definition of the minimal effective dose still acceptable and its equivalent in bio-assay tests. The WHOPES cut-off of 80% of nets effective in WHO cone bioassays (with ≥95% Knock-Down or ≥80% mortality) or tunnel tests (≥80% mortality or ≥ 90% blood-feeding inhibition) is designated as the optimal level required after three years of use to decide on recommendation for public health use, not as an end-of life determination or a minimally acceptable performance." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 9.
  • "In the early days of WHOPES Phase III testing WHO (Pierre Guillet, personal communication) had recommended a “minimal effectiveness” based on cone bio-assay tests of ≥75% KD or ≥50% mortality. This criterion had then been used in the field [13] establishing corresponding levels of chemical residue for a susceptible malaria vector strain. However, these minimal effectiveness criteria never appeared in any WHO guidelines." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pg 9.

• 30

  "The question of how to deal with insecticidal protection in the context of estimation of
  functional net survival remains challenging … Once methods are available that allow the determination of the “insecticidal effectiveness” in the field (thus enabling all LLINs in the survey sample to be tested), incorporation of the insecticidal component can be re-considered." WHO, Vector Control Technical Expert Group Report to MPAC, 2013, Pgs 9-10.

• 31

  "Questions remain about the durability of PBO on nets, as the impact of pyrethroid‐PBO LLINs on mosquito mortality was not sustained over 20 washes in experimental hut trials." Gleave et al. 2018, abstract.

• 32

  Pryce, Richardson, and Lengeler 2018

• 33

  Phillips-Howard et al. 2003 is given 66% weight in Pryce, Richardson, and Lengeler 2018's under-5 mortality calculation. See Pg 55, Analysis 1.1.

• 34
  • We reviewed Binka et al. 1996, Habluetzel 1997, and Nevill 1996. One additional trial, Smithuis 2013, is included in the Cochrane child mortality estimate, but only receives a negligible 0.1% weight.
  • See Pryce, Richardson, and Lengeler 2018, Pg 55, Analysis 1.1.

• 35

  "The CTNs (which require dipping into insecticide and which also require retreatment at least once a year) must have been impregnated with a WHO-recommended pyrethroid with the recommended formulation and dose (see Table 1 for recommended impregnation regimens). The LLINs (which are factory-treated nets where the insecticide is incorporated within or bound around the net fibres) must have had either interim or full recommendation from the WHO." Strode et al. 2014

• 36
  • AMF's net allocation method can vary by country and/or distribution, but coverage usually falls in the range of 1 LLIN for every 1.8 to 2 people. "The allocation we use of 1.8 people per net is widely accepted, given its recommended by WHO and AMP." Rob Mather, CEO, Against Malaria Foundation, comments on a draft of this page, September 4, 2020.
  • "At the household level, the distribution of 1 LLIN for every 2 members of the household will entail rounding up in households with an odd number of members (e.g. 3 LLINs for a household with 5 members, etc.) Because of this rounding up, the achievement of 1 LLIN for every 2 people at household level requires an overall ratio, for procurement purposes, of 1 LLIN for every 1.8 people in the target population." WHO, World Malaria Report, 2012, Pgs. 4-5.

• 37
  • On average, Phillips-Howard et al. 2003 provided net coverage of 1.46 persons per ITN during the course of the study, and the maximum attrition rate of the study-provided nets over 24 mos of follow-up in Phillips-Howard et al. 2003 was 8%, which is quite low compared to the LLIN monitoring data we've collected. "The initial ratio of people to project-issued ITNs based upon records of ITNs issued was 1.46 (2,176 people to 1,488 ITNs). Coverage ratios in houses with children were higher than in houses without children (1.82 versus 1.19), reflecting the fact that children are more likely to share sleeping places, and thus ITNs, than adults. Of the 1,488 ITNs issued to the households we sampled, we found 1,372, or 92%, still in place. The percentage of ITNs present in houses that were in use was 73.5% in 1997 and 67.5% in 1998, but this difference was not statistically significant (P = 0.10)." Alaii et al. 2003 (companion to Phillips-Howard et al. 2003 RCT).
  • Phillips-Howard et al. 2003 also provided additional nets in the second year to cover immigrants and newborns, amounting to a ~25% addition over the nets originally distributed (+4,600 nets in addition to about 17,000 initially distributed). The source is somewhat unclear if the 8% attrition figure takes into account the additional nets or not, but we believe that it does not (it cites the initial number of nets distributed as the denominator on the survival calculation), which means it's likely that there were actually more total nets present in the community in year 2 than in year 1, although these these nets did not directly replace lost nets. "Upon ITN distribution in November and December 1996, participants were shown how to hang ITNs over either beds or floor mats, and were provided with twine and nails if needed. By the beginning of 1997, more than 17,000 permethrin-treated dark green bed nets (Siamdutch Mosquito Netting Co., Bangkok, Thailand) had been distributed in the 40 of 79 villages randomized to receive ITNs in Asembo. Our aim was to cover all previously counted and measured sleeping places (beds or mats) with an ITN of appropriate size. During 1998, an additional 4,600 ITNs were distributed for new immigrants and newborns." Alaii et al. 2003 (companion to Phillips-Howard et al. 2003 RCT)
  • "In June 1993 all the 6053 compounds in the 48 areas randomly selected to receive impregnated nets (Figure I) were visited and 21,000 nets were provided for all women, children and, in most compounds, men. Impregnation of the nets with per-methrin was done under supervision. An additional 7000 nets were provided to men and other in-migrants during the re-impregnation in January 1994. In July 1994, a further 2600 nets were provided to 745 compounds that due to a shortfall in insecticide, were not visited during the re-impregnation in January 1994, and to new compounds that had been built since the start of the trial. Successful re-impregnation was achieved in 81% (23350/28 938), 83% (25,060/30,292) and 79% (24 418/31 071) of the nets in January and July 1994 and January 1995 respectively." Binka et al. 1996, Pgs 150-151.

• 38

  "The initial ratio of people to project-issued ITNs based upon records of ITNs issued was 1.46 (2,176 people to 1,488 ITNs)... Of the 1,488 ITNs issued to the households we sampled, we found 1,372, or 92%, still in place." Alaii et al. 2003 (companion to Phillips-Howard et al. 2003 RCT)

• 39

  "In the first intervention year, the proportion of houses with correctly installed, treated curtains (93%) was determined from forms filled in by the fieldworkers for each compound during the re-treatment in November/December 1994. A survey conducted in August 1995 revealed equally high coverage (94%) in the second year of intervention." Habluetzel 1997, Pg 857

• 40
  • "The ITN project, described in detail elsewhere, incorporated extensive educational activities, including community wide meetings in every village, informal discussions with traditional birth attendants associated with the project, participatory educational theater, an art competition for a calendar design involving schoolchildren that attracted hundreds of entries, and distribution of information sheets in Luo. Messages stressed through all media were 1) take ill children to health clinics promptly, 2) sleep under ITNs all year round every night and tuck in ITNs to keep mosquitoes from entering, 3) correct and consistent ITN use can reduce illness and death in young children, 4) wash ITNs only just before insecticide retreatment, and 5) keep ITNs in good condition by sewing up holes." Alaii et al. 2003 (companion to Phillips-Howard et al. 2003 RCT).
  • "Bed nets were pre-impregnated with 0.5 g of permethrin/m2 of netting and were re-treated to that target dose every 6–9 months. At each retreatment exercise, net and sewing materials were provided to residents whose ITNs were in need of repair. The ITNs were distributed to residents of the 39 control villages in the first quarter of 1999." Alaii et al. 2003 (companion to Phillips-Howard et al. 2003 RCT).

• 41

  "Bednets were distributed between June and July 1993 to each intervention household and issued according to the 'bed' registers of each house. Distribution was accompanied by pre-tested demonstrations of correct hanging and care of nets by trained field staff. Education in bednet use was continued the following day by government public health technicians who reviewed net hanging and held discussion meetings with small groups of mothers. Posters and school-based bednet plays and interactive learning sessions continued to be used throughout the trial (Marsh et al. in press). Local bednet committees were formed through the existing primary health care system to identify, resolve or report subsequent bednet issues, losses and difficulties within their communities." Nevill et al. 1996, Pg 142.

• 42
  • “We developed a Bayesian model using data from 102 national surveys, triangulated against delivery data and distribution reports, to generate year-by-year estimates of four ITN coverage indicators.” Bhatt et al. 2015, Abstract
  • Bhatt et al. 2015, Pg 12, Figure 8 shows the national household surveys that were used: Demographic and Health Survey (DHS), Multiple Indicator Cluster Survey (MICS), and Malaria Indicator Survey (MIS).
  • “National malaria control program reports … NMCP reports provided information about the number of LLINs and cITNs distributed in a county within a given year. These data were provided to WHO by NMCPs and were available for 365 of the 560 country-years addressed in the study.” Bhatt et al. 2015, Pg 19.
  • "Some 718 million LLINs have been delivered across the 40 endemic countries since their introduction in 2004." Bhatt et al. 2015, Pgs 3-4.

• 43
  • "Averaged over all years and all countries, we found the median retention time for LLINs in households was 23 (20–28) months. We found no statistically significant evidence of continent-wide temporal trends in retention times, but substantial between-country variation." Bhatt et al. 2015, Pg 6.
  • See also Bhatt et al. 2015 Pg 7, Figure 5, which shows “[e]stimated long-lasting insecticidal net retention curves for each country individually (blue lines) and combined (red line), in both cases relating to the average of the most recent 3 years, 2011–2013.”

• 44

  See Kilian et al. 2018, Figure 2. Note that not all brands of LLINs were distributed in all countries.

• 45

  “Primary outcomes are the proportion of cohort nets surviving in serviceable condition at each time point (see Figure 1) and the estimated median survival in serviceable condition in years.” Kilian et al. 2018, “Methods”

• 46

  See Kilian et al. 2018, Figures 3 and 4.

• 47

  "Results for up to 24 months show that differences between locations (behaviors and environment) exceed differences between brands. However, in harsh environments such as Northwest DRC a significant difference in physical survival between a 100 denier polyester and 145 denier polyethylene LLIN was seen which can be attributed to both the textile qualities and behavioral differences." Kilian et al. 2018, “Conclusion”.

• 48
  • “Assessment of insecticidal effectiveness was undertaken in entomological laboratories in-country using the standard WHO cone assay and where needed additional tunnel tests.” Kilian et al. 2018, “Methods”.
  • "Insecticidal effectiveness was found to be sufficient after two years either by bio-assay or chemical residue." Kilian et al. 2018, “Conclusion”.
  • See also the pie charts in Kilian et al. 2018, Figures 3 and 4.

• 49
  • We have based our calculation on the assumption that a CTN distributed in trials provides 2.0 equivalent coverage years across its 2 year distribution (i.e., 1.0 equivalent coverage years per year).
  • Our calculation of 2.11 equivalent coverage years across a 3-year distribution suggests an annual average of 0.7 equivalent coverage years.
  • However, we do not assume that LLINs provide the same level of coverage in their third year post-distribution as in their first.

• 50

  List of AMF distributions

• 51

  We searched Google Scholar, VectorWorks' resources page, and PMI's monitoring database for relevant studies. While we attempted to include all field studies of the PermaNet 2.0, it's possible that we missed some relevant evidence.

• 52

  See this spreadsheet.

• 53

  We smooth results for some monitoring points with sparse data.

• 54

  We're using a simplistic model of physical integrity that only takes into account nets that are rated as too torn and in need of replacement based on pHI. We also expect the fraction of nets that are damaged but not fully worn out to grow over time.

• 55

  We also expect prospective field monitoring studies of net condition to be affected by repeated monitoring, so the behavioral effects would have to be even stronger in RCTs to create a difference.

• 56

  "[T]he loss rate was...approximately 20% per year irrespective of the differing initial doses and deltamethrin being used in the other LLIN brands...In contrast, the median alpha-cypermethrin concentration for the conventional ITN [without retreatment] decreased by 69% after 12 months to a median of 7.5 mg/m² and by 93% after two years with 1.6 mg/m²." Kilian et al. 2011. We assume that retreatment of conventional nets per protocol in trials would instead restore these nets to baseline insecticide concentrations at 6-month intervals.

• 57

  See our comparison of insecticide loss in CTNs and LLINs in Uganda, based on data from Kilian et al. 2011.

• 58
  • Habluetzel 1997 is the only relevant RCT that collected information on insecticidal activity. Measured mosquito mortality rates in bioassays were close to or exceeded the WHO optimal performance threshold at all monitoring points. Note that this trial differs from others in that it distributed conventionally treated curtains rather than bednets. "The efficacy of the netting in killing mosquitoes was assessed every 3 months by bioassay (WHO 1989). On each occasion 2500 field collected, freshly fed Anopheles gambiae females were exposed to 15 curtains in 5 villages and to 10 positive and negative control curtains. I and 3 months after the first treatment, mortality rates of 70% and 72%, respectively, were achieved. These did not differ from those obtained on freshly treated reference curtains (63% and 67%). 4 months after treatment the mortality rate increased to 96% and similarly high values (99% and 88%) were obtained I and 3 months after the first retreatment. Mortality rates of 85%, 96% and 97% were observed 2, 4 and 9 months after the second retreatment, confirming the high efficacy of the treatment." Habluetzel 1997, Pg 857.
  • WHO grades "optimal" insecticide performance as causing at least 80% mortality or at least 95% knock-down in cone or tunnel tests after three years of use. See above.

• 59
  • Net retreatment was successful at restoring insecticide content in Habluetzel 1997. "During the first and second years of the intervention, the concentration of permethrin on curtain samples was measured by high performance liquid chromatography (HPLC). After treatment in June/July 1994, the mean permethrin concentration was 0.58 g/m^2. After the first and second retreatments mean concentrations of 1.38 g/m^2 and 1.23 g/m^2 were recorded. [The target dose was 1.0 g/m^2.]" Habluetzel 1997, Pg 857.
  • Another RCT, Nevill 1996, tested only insecticide concentration but not activity at an unspecified point in time (these measurements may have preceded insecticide retreatment). "Concentrations of permethrin on nets were assessed by high performance liquid chromatography (HPLC) on 72 randomly selected netting swatches: 65% of samples had active, cis-permethrin concentrations in excess of 0.5 g/m^2 [the target dose]." Nevill et al. 1996, Pg 142.

• 60

  Strode et al. 2014, Table 2, while somewhat outdated, summarizes some of the similarities and differences between different types of LLINs.

• 61

  WHO has revoked its recommendation for the DawaPlus 2.0 and NetProtect nets.

  • NetProtect failed its Phase III trial after receiving interim recommendation, and it no longer appears on WHO’s prequalified lists.
    • "Netprotect did not meet the minimum WHO criteria for bio-efficacy after 12, 24 and 36 months." Van Roey et al. 2014, Pg 1
    • WHO, Prequalified Lists: Vector control products
  • DawaPlus 2.0 is listed in Strode et al. 2014 as having an interim WHO recommendation, but it no longer appears on WHO's prequalified lists.
    • Strode et al. 2014, Pg 4, Table 2
    • WHO, Prequalified Lists: Vector control products
    • The investigation into DawaPlus 2.0 revealed manufacturing defects that had led to a run of out-of-specification nets: "Initially, 21 batches were found to have a content of deltamethrin of less than 1.5 g/Kg which is the formal minimum threshold. Based on review of QC data, the manufacturer concluded that a majority of nets manufactured between January 2017 and April 2018 were not manufactured according to formulation. The products were therefore not fully in conformity with approved specifications." The Global Fund, QA Information Notice, DawaPlus 2.0, 2019, Pg 2.

• 62
  • List of AMF distributions.
  • We have reviewed information on the quantities of different brands of nets that AMF has purchased over time to inform this view. This complete list of purchases is not publicly shareable at this time.
  • About 80% of nets are standard LLINs, and most of the remaining 20% are PBO nets, which combine permethrin insecticide with piperonyl butoxide. @List of AMF Distributions, public version@

• 63
  • Randriamaherijaona, Raharinjatovo, and Boyer 2017
  • Ahogni et al. 2020(a)
  • Rahaivondrafahitra et al. 2017
  • Ketoh et al. 2018

• 64

  "During a mass Long Lasting Insecticide Treated Net (LLIN) distribution campaign in Madagascar from October 2015 to November 2015, which took place in 92 of Madagascar's 112 districts, three types of LLIN were distributed: PermaNet 2.0, Yorkool, DawaPlus. A cohort study on PermaNet 2.0 was conducted in four malaria endemic districts: Ankazobe, Mananjary, Antshihy and Toliary II. This cohort study was conducted over three rounds, at 3-6 months, 12 months and 24 months after the mass distribution." Rahaivondrafahitra et al. 2017, Pg 2.

• 65

  See our head-to-head comparison in this spreadsheet.

• 66
  • Yorkool LN physical survival results from a 2017 distribution in Benin are better than our expectations for the PermaNet 2.0. See Ahogni et al. 2020(a), Pg 82, Table 2.
  • Yorkool LN physical survival results from a 2013 distribution in Madagascar are somewhat worse compared to our expectations for the PermaNet 2.0. See Randriamaherijaona, Raharinjatovo, and Boyer 2017, Pg 7, Table 6.
  • Ketoh et al. 2018 did not report on physical survival.
  • See our expectations for PermaNet 2.0 in this spreadsheet.
  • Additional details can be found in this spreadsheet.

• 67
  • Yorkool LN passed WHO requirements for effective performance on insecticidal activity in a 2017 distribution in Benin (see Ahogni et al. 2020(a), Figure 8) and in hut trials in Togo (see Ketoh et al. 2018, Pg 8, Table 3).
  • Yorkool LN failed WHO requirements for effective performance on insecticidal activity in a 2013 distribution in Madagascar: “Yorkool, in contrast, presented the lowest induced mortality. The average mortality was 48.6%, with only 20.8% (10/38) being above the minimum threshold (Fig. 4).” Randriamaherijaona, Raharinjatovo, and Boyer 2017, Pg 6.
  • Additional details can be found in this spreadsheet.
  • Note that these studies measured bioefficacy via cone bioassays, and most results are based on mosquito mortality rates in these tests. This reflects one out of four possible WHO requirements to confirm insecticide bioefficacy (additional tests thresholds include a 95% knockdown rate in cone bioassays and 80% mortality or 95% knockdown in tunnel tests).

• 68
  Compare the average mosquito mortality rates of Yorkool LN and PermaNet 2.0 nets in Rahaivondrafahitra et al. 2017 Pg 29, Table 15. Average mosquito mortality for both nets is below WHO requirements, but is lower for Yorkool LN nets. (Note that Yorkool LN nets were only measured 24 months post-distribution in this study.)
  ”In cone bio-assays, mortality of new Yorkool nets was surprisingly low (48.6%), mortality was 90.2% and 91.3% for Netprotect and Royalsentry (F(2, 131) = 81.59, P < 0.0001), respectively. At 12 month use, all tested nets were below the WHO threshold for replacement.” Randriamaherijaona, Raharinjatovo, and Boyer 2017

• 69
  • See our calculation here.
  • Note that we rounded our calculated value (17.1%) to the nearest multiple of 5 to reflect our uncertainty about this input.

• 70

  See our calculation here.

• 71
  • See here for the calculation of this input.
  • We make an additional adjustment (separate from our adjustment for equivalent coverage years) in our cost-effectiveness analysis of bednets to account for the reduction in effectiveness of bednets due to insecticide resistance.
  • We calculate that, during trials that we rely on to estimate the effectiveness of nets on child mortality, the physical barrier was responsible for slightly less than 30% of the protective effect. See our calculation here.
  • Insecticide resistance has since increased, making the insecticide component about 40% less effective than in trials (estimates vary by country; this estimate is based on the average reduction in insecticidal effectiveness for LLINs across countries tracked in row 29 here). The insecticide resistance adjustments in our main cost-effectiveness analysis account for PBO net purchases, which we expect to mitigate some of the impacts of resistance, but we use unadjusted values here because the Yorkool LN is not a PBO net.
  • Taking into account the expectation that insecticide is now less effective than in trials, we calculate that the physical barrier now contributes about 40% of the protective effect.

• 72

  See our calculation here.

• 73

  See our calculation here.

• 74

  See our calculation here.

• 75
  • For PBO nets, we used the same estimate of equivalent coverage years as for the PermaNet 2.0. This is because, to date, the majority of the PBO nets that AMF has purchased have been those that have the same specifications and manufacturers as reference class LLINs, with the addition of PBO. List of AMF distributions
  • We used the Yorkool LN as the exemplar for the generic category and applied a 15% deduction to the PermaNet 2.0 estimate in order to estimate the equivalent coverage years for AMF's purchases of generic LLINs. The 15% deduction is derived from our calculations here. We have not investigated other brands of generic LLINs in detail.
  • We have not yet investigated PBO net durability or made any additional adjustments for PBO longevity. We take into account evidence of increased protective efficacy for PBO nets separately in GiveWell's insecticide resistance adjustment.