2022 Cost-Effectiveness Analysis Changelog

This page provides details about changes that were made to our cost-effectiveness analysis (CEA) in 2022. Each changelog entry represents our understanding at the time the change was made. For past versions of our CEA, see this page.

Version 5 — Published August 4, 2022

Link to the cost-effectiveness analysis (CEA) file: 2022 CEA — version 5

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Incorporated subnational data into our Malaria Consortium CEA for FCT and Oyo, Nigeria

In our CEA of Malaria Consortium's seasonal malaria chemoprevention (SMC) program, we use malaria mortality and prevalence estimates from the Institute of Health Metrics and Evaluation's (IHME's) Global Burden of Disease (GBD) project to estimate the number of deaths we expect would be averted by the program in each of the geographic contexts we model. For our CEAs of Malaria Consortium's SMC program in FCT and Oyo states, Nigeria, we previously adjusted the national GBD malaria mortality and prevalence estimates for Nigeria using state-level malaria mortality and prevalence estimates from the Malaria Atlas Project (MAP). MAP provides malaria mortality estimates for each Nigerian state but doesn't break down mortality by age group. We applied the proportion of malaria deaths occurring among different age groups in GBD's estimates for Nigeria overall to the state-level malaria mortality estimates from MAP to come up with mortality estimates by age group for FCT and Oyo states.1

Through an agreement with the IHME Client Services Team, we have now seen state-specific GBD malaria mortality and prevalence estimates for FCT and Oyo, Nigeria. We have updated the CEA to use the state-level IHME estimates in place of our previous method.2

Change 2: Updated the leverage and funging adjustment in the Against Malaria Foundation (AMF) CEA for Global Fund-supported states in Nigeria

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur in the absence of philanthropic support for a particular program (e.g., "government costs would replace philanthropic costs" or "distributions would go unfunded"). In our AMF CEA, one of the scenarios we include is that the Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund) or the President's Malaria Initiative (PMI) would pay for the program in the absence of AMF's support. For a previous grant we made to AMF to fill a funding gap for long-lasting insecticide-treated net (LLIN) distributions scheduled to occur in Global Fund-supported states in Nigeria in 2022, we estimated that there was a 20% probability that the Global Fund would replace philanthropic costs.3 We increased our probability estimate to 35% to reflect our estimate that the Global Fund would replace philanthropic costs for a similar funding gap for distributions scheduled to occur in Global Fund-supported states in Nigeria in 2023.4 The key reasons for this increase were (1) we thought it was more likely that the Global Fund would be able to reallocate funding in time to fill the later gap and (2) AMF having established a presence as a funder in this new location for the 2022 gap increases the risk that the Nigerian national malaria program and the Global Fund might assume that additional funding would be available from AMF for future gaps.

Change 3: Updated the insecticide resistance adjustments for Chad, Democratic Republic of Congo (DRC), and Guinea in the Against Malaria Foundation (AMF) CEA

In a previous changelog entry, we described changes we made to our methodology of adjusting for insecticide resistance in our AMF CEA for Nigeria. We subsequently applied those methodological changes to our insecticide resistance adjustments for Togo and Uganda. We have now applied those changes to our insecticide resistance adjustments for Chad, DRC, and Guinea as well.5 See our insecticide resistance adjustment calculations in this sheet.

Key uncertainties

• Resistance mechanism data in Guinea: In areas where a significant proportion of mosquitoes are resistant to standard insecticide, AMF sometimes funds distributions of nets treated with a piperonyl butoxide (PBO) synergist in addition to standard insecticide. Evidence suggests that PBO nets are effective in combating a certain type of insecticide resistance called mono-oxygenase-based resistance. We use data on insecticide resistance mechanisms from the World Health Organization's (WHO's) Malaria Threat Map to estimate the proportion of a country where insecticide resistance is likely to be responsive to PBO nets. WHO's Malaria Threat Map doesn’t include resistance mechanism data for Guinea, so we instead use data from Nigeria, the geographically nearest country in the CEA for which we have data.
• Bioassay data in Chad: We calculate insecticide resistance adjustments using bioassay data from IR Mapper. The IR Mapper datasets we use include the results of tests (called "bioassays") that measure the proportion of mosquitoes that die when exposed to insecticide. The IR Mapper datasets include bioassays testing standard insecticide alone and standard insecticide plus PBO. Because we believe insecticide resistance rates are likely to be changing meaningfully over time, we typically use data points from 2015 or later. However, the IR Mapper datasets contain very few data points from Chad since 2015, and these data points are all from bioassays conducted in or around the same two cities and published as part of the same study.6 These data points suggest very low mortality rates for mosquitoes exposed to both standard insecticide alone (1-5%) and standard insecticide plus PBO (3-14%). We are hesitant to assume that the limited recent bioassay data we have represents insecticide resistance rates in Chad more broadly, especially given how extreme the results of these bioassays are compared to those conducted in other countries. For now, we have chosen to incorporate some older bioassay results into our insecticide resistance adjustment for Chad as well; these bioassays were conducted across a wider variety of locations and found more varied levels of resistance.7 This brings our insecticide resistance adjustment for Chad more in line with what we would expect, but we remain highly uncertain about the appropriate value for this adjustment, and we may do more work to refine this adjustment prior to making funding decisions about future long-lasting insecticide-treated net (LLIN) distributions in Chad.

Change 4: Updated the internal and external validity adjustments and added Nigerian states to the Helen Keller International CEA

The previous version of our Helen Keller International CEA calculated a cost-effectiveness estimate for Nigeria at the national level. We have now updated the CEA to calculate the cost-effectiveness of a number of individual Nigerian states instead.

We have also updated the internal and external validity adjustments in our Helen Keller International CEA. We base our estimate of the impact of vitamin A supplementation on child mortality on the effect size reported in Imdad et al. 2017, a meta-analysis of vitamin A supplementation (VAS) trials. We then use internal and external validity adjustments to account for our uncertainties about the effect size found in the meta-analysis (internal validity) and differences between the contexts of the trials included in the meta-analysis and the contexts of programs we're considering funding (external validity).8

Internal validity

In the previous version of our CEA, we used an internal validity adjustment of 85%, based on our rough subjective guesses about the below factors:

• Cause of death: We are uncertain whether VAS has an impact on all mortalities caused by infectious diseases or only a subset of specific infectious diseases. Our uncertainty about the mechanism of the intervention decreases our confidence in the mortality effect found in the Imdad et al. 2017 meta-analysis. This leads us to believe that the effect size reported in Imdad et al. 2017 may be an overestimate of the true impact of vitamin A supplementation on mortality.
• Trial methodology: Most of the trials included in Imdad et al. 2017 were conducted many years ago, in the 1980s and 1990s. While we don't have specific criticisms of the methodologies of these trials, we believe that trials conducted a long time ago are likely to have been methodologically weaker than trials conducted more recently.

We have now updated and formalized our thinking on the above factors and have arrived at a new internal validity adjustment of 74%. This change was primarily driven by our analysis of the plausibility of the effect size found in Imdad et al. 2017, given a variety of assumptions about the mechanisms of the intervention. The conclusion of our analysis is that the results of the Imdad et al. 2017 meta-analysis imply that VAS reduces child mortality by a larger amount than we would expect based on the intervention's effect on infectious disease.9 See our calculations here.

External validity

We apply context-specific external validity adjustments for each location in our CEA, which are intended to account for differences between the contexts of the trials included in Imdad et al. 2017 and the contexts of programs we're considering funding. Specifically, we base our external validity adjustments on estimates of the prevalence of vitamin A deficiency and the proportion of child mortalities caused by infectious diseases in a given location.

We have made two updates to our methodology for calculating these external validity adjustments.

• Disease weighting: When comparing the proportion of under-five mortalities caused by infectious diseases between trial contexts and the contexts in our CEA, we previously put 85% weight on the proportion of deaths caused by measles or diarrhea and 15% weight on the proportion of deaths caused by infectious disease overall.10 We now put 80% weight on the former and 20% on the latter.11
• Non-independence adjustment: We apply a non-independence adjustment to our external validity calculations to account for the fact that differences in vitamin A deficiency prevalence and infectious disease burden between two contexts are likely to be correlated. We noticed that the way we were previously calculating this adjustment was leading to estimates that didn't make sense for some countries.12 We have updated our calculations to allow the non-independence adjustment to vary with the level of the vitamin A deficiency and infectious disease burden.13

Change 5: Added South Sudan to the Against Malaria Foundation (AMF) CEA

As part of our investigation into a grant we made to AMF to potentially fund a long-lasting insecticide-treated net (LLIN) distribution in South Sudan, we added South Sudan to our AMF CEA.14

Insecticide resistance in South Sudan

We typically use insecticide resistance data from IR Mapper to inform the insecticide resistance adjustments in our AMF CEA.15 The IR Mapper datasets we use include the results of tests (called "bioassays") that measure the proportion of mosquitoes that die when exposed to insecticide. The IR Mapper datasets include bioassays testing standard insecticide alone and standard insecticide plus piperonyl butoxide (PBO), a synergist that has been shown to increase the effectiveness of insecticide-treated nets in areas with high insecticide resistance.16 However, IR Mapper's datasets don't include any bioassay results from South Sudan for standard insecticide or standard insecticide plus PBO.

To estimate resistance to standard nets in South Sudan, we instead use bioassay results from a report published by the South Sudanese Ministry of Health that we received from AMF. The report includes results from ten bioassays of standard insecticide conducted in 2020 across various sites in South Sudan.17 However, the Ministry of Health report doesn't include any bioassays testing standard insecticide plus PBO. In the absence of direct bioassay data from South Sudan, we instead average together bioassay results from two neighboring countries (DRC and Uganda) to estimate resistance to PBO nets in South Sudan.

Because we have seen more limited data on insecticide resistance in South Sudan than we have for other countries, we are more uncertain about our insecticide resistance adjustment for this country. Increasing our confidence in the adjustment is the fact that the data we have seen was collected recently and from several points around the country (increasing the likelihood that we are seeing a representative view of insecticide resistance in South Sudan). See our calculations in our insecticide resistance adjustment analysis.

Version 4 — Published April 12, 2022

Link to the cost-effectiveness analysis (CEA) file: 2022 CEA — version 4

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Fixed an error in our calculation of the unadjusted reduction in malaria prevalence for the treatment population due to seasonal malaria chemoprevention (SMC) in our CEA of Malaria Consortium

Some health programs for children may positively impact development, leading to lasting increases in their productivity and earnings in adulthood ("development effects").18 In our CEA of Malaria Consortium, we model development effects as resulting from reductions in malaria prevalence among treated (children under five) and untreated (children aged 5-14) populations. We noticed an error in the formula we use to calculate the percentage point reduction in malaria prevalence in the treated population. In the formula, we were not multiplying by our estimate of the proportion of annual malaria mortality that occurs within the high-transmission season. This is important because we expect SMC to reduce malaria prevalence only within the 4- to 5-month period during which the intervention is implemented.19 We have corrected this error.20

Change 2: Updated our estimate of the cost per infant enrolled in our CEA of New Incentives

We received updated program information from New Incentives that led us to increase our estimate of the cost per infant enrolled in New Incentives' program from $28.93 to $29.52.21 We have updated our CEA of New Incentives with the new cost per infant enrolled estimate.22

Change 3: Updated our estimates of the cost per child dewormed in our CEA of Sightsavers.

Our 2020 analysis of the cost per child dewormed for Sightsavers' deworming program relies on information about costs incurred and number of treatments delivered as part of the program for the 2017-2019 period. In early 2022, we updated that analysis (see our 2022 cost per child dewormed per year analysis) to incorporate program cost and treatment information from 2020. We have replaced estimates of the cost per child dewormed and the proportions of costs covered by different actors from the 2020 analysis with estimates from our updated 2022 analysis into our Sightsavers CEA.23

Version 3 — Published March 31, 2022

Link to the cost-effectiveness analysis (CEA) file: 2022 CEA — version 3

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updated the cost per net estimates in our Against Malaria Foundation (AMF) CEA for the Democratic Republic of the Congo (DRC)

We received information from AMF that led us to change our estimates of AMF's costs for each long-lasting insecticide-treated net (LLIN) it purchases for distributions in DRC. This change decreased our estimate of the total cost per LLIN (including costs covered by actors other than AMF) from $5.54 to $5.51. We have updated this estimate in our AMF CEA for DRC.24

Change 2: Updated our methodology for incorporating a fifth cycle of seasonal malaria chemoprevention (SMC) in our CEA of Malaria Consortium

Malaria Consortium's SMC programs typically deliver four "cycles" of seasonal malaria chemoprevention (SMC) per year. Each cycle includes a four-day distribution period and lasts 28 days, after which a new cycle starts.25 In late 2020, we learned that Malaria Consortium was planning to support a fifth SMC cycle per year in some countries going forward.26 We previously integrated a fifth cycle of SMC into our CEA by adding a percentage adjustment to our cost-effectiveness estimates for countries in which we expected Malaria Consortium to support five cycles. See our previous changelog entry describing this approach in more detail here.

We are now using a different approach for incorporating a fifth cycle of SMC into our CEA, which we believe is simpler and less likely to lead to errors. Previously, we calculated our cost estimates for Malaria Consortium's SMC program in terms of "cost per equivalent child treated with all cycles of SMC." We now calculate costs in terms of "cost per SMC cycle administered," and multiply by the average number of cycles we expect to be implemented in each country (or state) to arrive at an estimate of the cost per child fully treated.27

Change 3: Updated our analysis of the cost per child covered with seasonal malaria chemoprevention (SMC) for Malaria Consortium

Our 2020 cost per child covered with SMC analysis relied on information from Malaria Consortium's programs between 2015 and 2019. Our updated January 2022 analysis incorporates information on costs and SMC coverage for Malaria Consortium's program in 2020. We have updated our Malaria Consortium CEA with the new cost estimates.28

Change 4: Added Mozambique to our CEA of Malaria Consortium

During the 2020-2021 and 2021-2022 seasonal malaria chemoprevention (SMC) seasons, Malaria Consortium used GiveWell-directed funding to pilot an SMC program in several districts of Nampula province, Mozambique (unlike in the Sahel, where SMC programs are typically delivered, the malaria season in Mozambique runs across calendar years).29 During our investigation of a potential grant to Malaria Consortium to fund the expansion of its program in Mozambique to cover all of Nampula province, we added Mozambique to our CEA of Malaria Consortium.30

Change 5: Updated some of the supplemental charity-level adjustments in our CEA of Helen Keller International (Helen Keller)

In our CEAs, we apply percentage adjustments to programs' cost-effectiveness to account for the risk of "wastage" (e.g., double treatment with vitamin A supplements or deworming tablets), quality of monitoring and evaluation, and confidence in funds being used for their intended purpose. We refer to these adjustments as "supplemental charity-level adjustments." Based on information we have received from Helen Keller about its vitamin A supplementation (VAS) programs in 2020, we have decided to update the values of some of the supplemental charity-level adjustments in our Helen Keller CEA.

In our Helen Keller CEA, the adjustment for "double treatment" is intended to account for the possibility that VAS may be delivered through the program to children who have already received VAS from another source, such as routine health care. We have not completed a systematic review of existing information on the coverage of VAS outside of campaigns, but we believe that we were underestimating this adjustment before. Our impression from expert opinion is that some children under the age of one receive VAS at health facilities when they visit for routine vaccinations (e.g., Helen Keller's 2021 room for more funding report notes that roughly 20% of children between 6 and 59 months receive VAS in primary health care facilities each year31 ). To account for the fact that double treatment rates may be higher than we had previously thought and the fact that double treatment may disproportionately affect children under one,32 who have higher mortality rates than older children targeted by VAS programs,33 we have increased our adjustment for double treatment from 5% to 15%.34

In our Helen Keller CEA, the adjustment for "misappropriation without monitoring results" is intended to account for the possibility that seeing monitoring results from only some of Helen Keller's VAS distributions may lead us to believe that coverage rates are higher across all distributions than they really are, for example if coverage rates tended to be lower for distributions for which we haven't seen monitoring results. In 2020, we received monitoring results from six out of the seven countries Helen Keller supports with GiveWell-directed funding, which increased our confidence that we are seeing a representative picture of Helen Keller's work.35 We have decreased our adjustment for misappropriation without monitoring results from 8% to 6%.36

Change 6: Added the Littoral and South regions of Cameroon to our CEA of Sightsavers

We learned that Sightsavers is seeking funding to implement its deworming program in the Littoral and South regions of Cameroon. We have added the Littoral and South regions of Cameroon to our Sightsavers CEA.37

Change 7: Added a supplemental intervention-level adjustment for long-lasting insecticide-treated net (LLIN) coverage to our Malaria Consortium CEA in Nigeria

We use data from the Institute of Health Metrics and Evaluation's (IHME's) Global Burden of Disease (GBD) project to inform our estimates of malaria mortality and prevalence rates in our cost-effectiveness analysis of Malaria Consortium's seasonal malaria chemoprevention (SMC) program in Nigeria. Our understanding is that IHME's malaria modeling factors the expected impact of past LLIN distributions into its malaria mortality and prevalence estimates. The most recent GBD estimates available are for the year 2019.

Our understanding is that LLIN coverage in Nigeria has been increasing year-to-year as a result of increased funding for LLINs compared to previous years. As a result, we think it's likely that the LLIN coverage estimates informing GBD's malaria modeling for Nigeria have become somewhat outdated since 2019 and that true malaria mortality and prevalence rates in Nigeria may be lower than GBD estimates would suggest. To account for this, we added a rough downward adjustment of 5%—representing our expectation that SMC is less impactful if malaria rates are lower—to the "Supplemental intervention-level adjustments" section of our cost-effectiveness estimate of SMC in Nigeria.38

Change 8: Removed arbitrary estimates of the percentage of funding allocated to each country from the CEA

We calculate cost-effectiveness in our CEA by calculating how much good we expect will result from hypothetical donations to our top charities in terms of lives saved or improved. Because we calculate cost-effectiveness on a per-dollar basis, the size of the hypothetical donation we use in the CEA is arbitrary, meaning that changing the size of the donation doesn't affect how much good we expect to result from each dollar donated. Previously, we calculated the size of this arbitrary donation by dividing up a $100,000 donation among the countries supported by a top charity, weighted according to our estimates of the percentage of marginal funding that would fund programs in each country. We use the same estimates to calculate the percentage of people covered by the intervention who live in each country.39 In earlier iterations of our CEA, we used these estimates to assign a numerical weight to each country a top charity works in, in order to calculate each top charity's "overall" cost-effectiveness. In May 2021, we removed "overall" cost-effectiveness estimates from most of our top charity CEAs.40

Therefore, for most of our CEAs, our estimates of the percentage of marginal funding that would go toward each country no longer serve a purpose beyond calculating the size of the hypothetical donation, and our estimates of the percentage of people covered by the intervention who live in each country no longer serve a purpose at all.41 As such, we decided to remove both parameters from all of the CEAs for which they do not affect our cost-effectiveness estimates, and we have instead set hypothetical donations equal to $100,000 in each country in each CEA.42

Change 9: Updated our leverage and funging adjustment for our CEA of Helen Keller International (Helen Keller)

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur in the absence of philanthropic support for a particular program (e.g., "government costs would replace philanthropic costs" or "distributions would go unfunded"). We have updated our estimates of these probabilities for Helen Keller's vitamin A supplementation (VAS) program for each country in our CEA, based on additional information we've received from Helen Keller about the recent and future funding landscape for VAS in the countries it supports43 and conversations we've had with Helen Keller and other VAS stakeholders.44

Version 2 — Published March 29, 2022

Link to the cost-effectiveness analysis (CEA) file: 2022 CEA — version 2

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updated the New Incentives CEA with subnational mortality and vaccination data

In our CEA of New Incentives, we previously used national-level estimates of the probability of death before age five from vaccine-preventable diseases in Nigeria from the Institute for Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) project.45 Because New Incentives' program operates in only three states in Nigeria, all located in the northwestern part of the country, we then applied an adjustment to these estimates to account for our expectation that under-five mortality rates are higher in these states than in the country overall.46 We then used national vaccination coverage data from the World Health Organization (WHO) and UNICEF to estimate the probability of death before age five for vaccinated and unvaccinated children.47

Through an agreement with the IHME Client Services Team, we received state-level estimates of the probability of death before age five from vaccine-preventable diseases in Nigeria.48 We updated our CEA of New Incentives to use these state-level estimates, rather than the adjusted national-level estimates described above.49 We also began using some state-level estimates of baseline vaccination coverage in our calculations using data from IHME's Viz Hub tool.50

Change 2: Updated cost per infant immunized analysis for our New Incentives CEA

In our previous cost per infant immunized analysis for our New Incentives CEA, we used cost data from New Incentives from June 2019 through May 2020. We now use New Incentives' cost data through August 2021 to estimate its costs per infant enrolled in its program.51

Our cost per infant immunized analysis also includes estimates of costs incurred by the Nigerian government and Gavi. Updates to our methods for estimating these costs:

• Estimating total costs per infant immunized: The World Health Organization (WHO) reports estimates of immunization coverage, total spending on routine infant immunization, and government spending on routine infant immunization by country and year.52 We previously used data for 2018 from these WHO sources to estimate total costs per infant immunized in Nigeria.53 After noticing that there was substantial variation in WHO's reported estimates of total and government spending on routine infant immunization by year, we decided to instead use an average across 2014-2018 of spending and immunization coverage rates in our calculations—we chose to use data from 2014-2018 because those were the only recent years with spending estimates for Nigeria in WHO's database.54 This method resulted in an estimate of around $90 in total spending per infant fully immunized.55
• Estimating proportions of total costs covered by the Nigerian government and Gavi: We assume that the costs of $90 per infant fully immunized are covered by a combination of the Nigerian government and Gavi.56 The Nigeria Joint Immunization Strategy Document 2018-2028 describes a plan to phase out Gavi support and increase the proportion of costs covered by the Nigerian government over time.57 We are uncertain about the extent to which the plan developed in 2018 to phase out Gavi's contributions has been implemented, so as a rough estimate of the proportion of costs covered by Gavi and the Nigerian government in upcoming years, we average the historical data we have seen from 2014 to 2018 with the estimates of proportions of costs covered by each actor from 2022-2024 from the Nigeria Joint Immunization Strategy Document 2018-2028.58 In the previous version of our cost per infant immunized analysis, we relied only on the projections from the Nigeria Joint Immunization Strategy Document 2018-2028.59

We have updated our New Incentives CEA to incorporate the cost estimates from our updated cost per infant immunized analysis.60

Change 3: Updated our CEA of Malaria Consortium's seasonal malaria chemoprevention (SMC) program in Nigeria with subnational data

Our CEA of Malaria Consortium's SMC program in Nigeria encompasses its program in six Nigerian states.61 In our CEA, we previously used national malaria mortality and prevalence data for Nigeria from the Institute for Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) project to estimate the benefits of SMC. In order to make our cost-effectiveness estimate more representative of the specific states in which the program will be implemented, we have updated our CEA to use state-level malaria mortality and prevalence data we received through an agreement with the IHME Client Services Team.62

Change 4: Corrected error in our Malaria Consortium CEA for FCT and Oyo, Nigeria

We realized we had made an error when entering population data into our CEA of Malaria Consortium's SMC program in FCT and Oyo, Nigeria. We had copied population estimates from the Chad column of the CEA into the FCT and Oyo columns, when we should have copied population estimates from the Nigeria column of the CEA. We have corrected this error.63

Change 5: Updated our CEA of the Against Malaria Foundation (AMF) in Nigeria with subnational data

In Nigeria, most states are assigned to receive funding for malaria programs either from the Global Fund to Fight AIDS, Tuberculosis and Malaria ("the Global Fund"), from the President's Malaria Initiative (PMI), or through loan financing from the World Bank and the Islamic Development Bank. 64 In our CEA, we estimate the cost-effectiveness of AMF's long-lasting insecticide-treated net (LLIN) distributions in Global Fund-supported states and PMI-supported states separately.

Previously, we used national malaria mortality and prevalence data for Nigeria from the Institute for Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) project to estimate the benefits of LLIN distributions in both Global Fund-supported states and PMI-supported states. As with changes 1 and 3 above, we updated our CEA using state-level data we received from IHME through an agreement with the Client Services Team in order to make our cost-effectiveness estimates more representative of the specific states in which the program will be implemented.65

Version 1 — Published March 24, 2022

Link to the cost-effectiveness analysis (CEA) file: 2022 CEA — version 1

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updated estimates from the Institute for Health Metrics and Evaluation (IHME) in our moral weights

GiveWell uses moral weights to make cost-effectiveness comparisons between interventions achieving different types of outcomes (e.g., averting the death of a child from malaria vs. doubling a very poor individual's consumption for a year). See this page for more discussion on how GiveWell uses moral weights.

The moral weights we use in our CEA rely on estimates of the distribution of deaths across age groups from the Global Burden of Disease (GBD) project of the Institute of Health Metrics and Evaluation (IHME). We use these estimates to assign value to averting deaths according to age.

Previously, our CEA relied on data from the 2017 version of GBD.66 However, in October 2020 IHME released an updated version of GBD for 2019, which included updated estimates of the distribution of deaths by age.67 We have incorporated these updated estimates into the moral weights calculations in our CEA.68

Change 2: Added Nigerian states supported by the Global Fund to our CEA of the Against Malaria Foundation (AMF)

In Nigeria, most states are assigned to receive funding for malaria programs either from the Global Fund to Fight AIDS, Tuberculosis and Malaria ("the Global Fund"), from the President's Malaria Initiative (PMI), or through loan financing from the World Bank and the Islamic Development Bank.69 We learned that AMF is considering supporting long-lasting insecticide-treated net (LLIN) distributions in a group of Nigerian states that receive support for malaria programs from the Global Fund. We have added a column to our CEA of AMF with cost-effectiveness estimates for these states.70

Change 3: Updated the adjustment for insecticide resistance in our CEA of the Against Malaria Foundation (AMF) for Nigeria

Our CEA of AMF includes an adjustment to account for the impact of insecticide resistance on the effectiveness of long-lasting insecticide-treated nets (LLIN) distributions. Our adjustment attempts to account for the impact of insecticide resistance on the effectiveness of both standard LLINs and LLINs that have been treated with piperonyl butoxide (PBO) in addition to standard insecticide. Previously, our methodology for incorporating PBO nets into the adjustment was based on a very limited amount of data on the effectiveness of PBO nets. We have now updated this methodology for Nigeria as part of an investigation into whether to direct funding to LLIN distributions in Ondo and Anambra states.71

Background

Populations of mosquitoes can adapt to become resistant to the type of insecticide used in LLINs, which may lead to a reduction in the effectiveness of LLINs at preventing malaria.72 This issue seems to be fairly common across sub-Saharan Africa, and may be increasing in severity over time.73 We have written a report on this topic here. Our report also includes a discussion of the evidence that PBO nets, a newer type of LLIN incorporating a synergist, piperonyl butoxide, provide additional protection in areas where mosquitoes have developed insecticide resistance.74

Adjustment in our CEA of AMF for insecticide resistance

Our CEA of AMF includes an adjustment for the expected reduction in effectiveness of LLINs due to insecticide resistance, based on data from insecticide resistance research by country and on the proportion of PBO nets and standard LLINs we expect AMF to purchase for upcoming distributions in each country. As of February 2021, we estimated that the impact of insecticide resistance on the effectiveness of AMF's upcoming LLIN distributions would vary widely by country, ranging from a 1% decrease (in Nigeria) to a 32% decrease (in Guinea).75

Impact of insecticide resistance on the effectiveness of standard LLINs

Insecticide resistance is typically measured using tests called "bioassays," which involve capturing wild mosquitoes from a particular site, exposing them to a dose of insecticide that would be expected to kill 100% of the mosquitoes if no insecticide resistance were present, and measuring the percentage of mosquitoes that actually die from the exposure.76 The World Health Organization's (WHO's) Global report on insecticide resistance reports average mosquito mortality rates found in bioassays conducted between 2010 and 2016 by country. The report also provides an estimate of the overall percentage point increase in resistance to permethrin (the insecticide used to treat LLINs) observed over this six-year period. We use this information to project how high we expect mosquito mortality rates to be when upcoming distributions occur in each country in our CEA.77

Mosquito mortality caused by exposure to insecticide is unlikely to be the only mechanism through which LLINs reduce malaria transmission. In our calculations, we assume that some proportion of the protective effect of LLINs is due to the physical barrier created by the net, such that we would expect LLINs to provide some degree of protection against malaria transmission, even in areas where 100% of mosquitoes are resistant to insecticide. We use information from a meta-analysis on the effectiveness of LLINs in reducing malaria, Lengeler 2004, to calculate an estimate of the proportion of LLINs' effectiveness that is due to insecticide and the proportion that is due to the physical barrier of the net.78

To arrive at our insecticide resistance adjustments for standard LLINs in each country, we multiply our estimate of the proportion of LLINs' effectiveness that is due to insecticide by our estimates of expected mosquito mortality rates at the time of upcoming distributions in each country, then add this value to the proportion of LLINs' effectiveness that is due to the physical barrier of the net (and thus unaffected by insecticide resistance).79

Impact of insecticide resistance on the effectiveness of PBO nets

Evidence suggests that the addition of PBO to LLINs restores some, but not all, of their effectiveness against insecticide-resistant mosquitoes.80 We use an estimate of the proportion of permethrin-resistant mosquitoes that are also resistant to PBO and multiply it by our insecticide resistance adjustments for standard LLINs in each country to arrive at insecticide resistance adjustments for PBO nets in each country.81

Calculating the final insecticide resistance adjustment

Finally, we average together our insecticide resistance adjustment for standard LLINs and our insecticide resistance adjustment for PBO nets to arrive at an overall insecticide resistance adjustment for each country, weighted according to the proportion of standard LLINs and PBO nets we expect to be appropriate for each country. Our final insecticide resistance adjustments vary significantly by country, from as low as a 1% decrease in cost-effectiveness for Nigeria to a 32% decrease in cost-effectiveness for Guinea.82

Limitations of this methodology

There are a number of limitations to this methodology. For example:

• It relies on non-representative bioassay results. We base our estimates of how high we expect mosquito mortality rates to be at the time of upcoming LLIN distributions in each country on the average mosquito mortality rates found in bioassays conducted between 2010 and 2016 in each country, as reported in WHO's global report on insecticide resistance for that time period. Our understanding is that the bioassay sites included in the WHO report come from multiple studies where the authors had various reasons for selecting a particular location. In other words, there's no reason to think they were selected on the basis of making a nationally representative sample. We also understand that subnational variance in insecticide resistance prevalence can be quite high. As such, we think the average mosquito mortality rates reported for each country may not be representative of mosquito mortality rates in the countries overall or for the specific areas targeted by upcoming LLIN distributions.
• It assumes linear increases in insecticide resistance over time. In our calculations, we assume that the mosquito mortality rates found in bioassays are decreasing at a rate that is constant over time and consistent across countries. If the decrease in mosquito mortality rates has, instead, been accelerating over time, or if the rate of change varies significantly across locations, our estimates of how high mosquito mortality rates will be at the time of upcoming distributions could be significantly off.

Updates to the insecticide resistance adjustment for PBO nets in Nigeria

We have updated our methodology for incorporating the impact of PBO nets into our insecticide resistance adjustment for Nigeria. We decided to update our insecticide resistance adjustment for Nigeria as part of an investigation into whether to direct funding to LLIN distributions in Ondo and Anambra states.83 We now use country-specific synergist bioassay data from IR Mapper and country-specific insecticide resistance mechanism data from WHO's Malaria Threat Map to inform our adjustment.

Proportion of insecticide resistance that remains when using PBO

Previously, we used reported mosquito mortality rates from two studies conducted in Muleba, Tanzania, to estimate the proportion of insecticide-resistant mosquitoes that are also resistant to PBO. For this update, we downloaded a dataset from IR Mapper containing information on bioassays of pyrethroid insecticides conducted in Africa in the 2005-2018 period. This dataset included results from sets of bioassays that had tested pyrethroids alone and pyrethroids plus PBO at the same site.84 Seven of the bioassays included in the dataset were conducted in Nigeria, four of which tested pyrethroids alone and three of which tested pyrethroids plus PBO. We used the average mosquito mortality found in the pyrethroid-only bioassays and the average mosquito mortality found in the pyrethroids plus PBO bioassays to calculate an estimate of the proportion of insecticide resistance that remains when using PBO.85

Adjusting for country representativeness

Previously, our insecticide resistance adjustment calculations implicitly assumed that PBO nets would be more effective than standard LLINs everywhere insecticide resistance is present. However, our understanding is that PBO only improves the effectiveness of LLINs in locations where insecticide resistance is driven by a particular resistance mechanism. Specifically, WHO recommends the use of PBO nets only in locations where a "monooxygenase-based" resistance mechanism has been detected.86

To account for this, we have added an external validity adjustment to our calculations based on information from WHO's Malaria Threat Map on the proportion of test sites in Nigeria with confirmed pyrethroid insecticide resistance and the proportion of sites where a monooxygenase-based resistance mechanism was detected.87 We use this information to calculate the estimated proportion of Nigeria where PBO nets are more effective than standard LLINs.88 We multiply this estimate by our estimate of the proportion of resistance that remains when using PBO to calculate our revised insecticide resistance adjustment for PBO nets for Nigeria.89

Change 4: Corrected an error in our CEA of New Incentives

In our CEA of New Incentives, we use estimates of vaccination coverage in Nigeria from the World Health Organization (WHO) and UNICEF90 to calculate an estimate of the probability of death from vaccine-preventable disease among unvaccinated children under five in North West Nigeria for the year 2019 (the most recent year of data available as of this change). We noticed that we had made an error in calculating average vaccination coverage rates for children under five. We had previously taken averages of yearly vaccination coverage rate estimates from 2013 to 2019. However, because we are interested in estimating vaccination coverage for children who were under the age of five as of 2019, we should only have averaged vaccination coverage rates across the previous five years, from 2015 to 2019. We have corrected this error.91

Change 5: Added additional states in Nigeria to our Sightsavers and Deworm the World CEAs

We learned that Sightsavers is considering seeking funding for its programs in six Nigerian states for which it had previously received funding from the U.K.'s Foreign, Commonwealth & Development Office (FCDO).92 We also learned that Deworm the World is considering seeking funding to expand its program to an additional state in Nigeria. We have added columns to our CEAs of Sightsavers and Deworm the World with cost-effectiveness estimates for these states.93

Change 6: Updated the cost per LLIN in our CEA of the Against Malaria Foundation (AMF) for Uganda

For long-lasting insecticide-treated nets (LLIN) distributions supported by AMF, our estimate of the total cost per LLIN distributed relies on our best guess of the proportion of nets purchased by AMF that have been treated with piperonyl butoxide (PBO) in addition to standard insecticide for a given distribution, as PBO nets are more expensive than standard LLINs. There is some evidence that LLINs treated with PBO are more effective than standard LLINs in areas where mosquitoes have demonstrated high levels of resistance to the class of insecticides most commonly used to treat LLINs.94

We received information from AMF that led us to change the proportion of PBO nets we expect to be appropriate for Uganda. Our current best guess is that all of the nets for distributions in Uganda will be PBO nets. This change increased our estimate of the cost per net distributed in Uganda from $4.29 to $4.65. We have updated our CEA of AMF to incorporate this change.95

Change 7: Corrected an error in the malaria prevalence reduction estimate in our CEA of the Against Malaria Foundation (AMF)

We noticed an error in our calculation of the expected reduction in malaria prevalence rates caused by long-lasting insecticide-treated nets (LLIN) distributions in our AMF CEA. The previous version of our calculation incorrectly multiplied the estimated malaria prevalence reduction by our estimate of the "proportion of mortality attributed to malaria in areas AMF works vs. the contexts of trials in the Cochrane Pryce et al. 2018 meta-analysis." This was incorrect because the proportion of mortality attributed to malaria isn't clearly related to expected reductions in malaria prevalence. We have corrected this error by removing our estimate of the "proportion of mortality attributed to malaria in areas AMF works vs. the contexts of trials in the Cochrane Pryce et al. 2018 meta-analysis" from the calculation.96

Change 8: Added Chad to our CEA of the Against Malaria Foundation (AMF)

We learned that AMF is planning to support the 2023 long-lasting insecticide-treated nets (LLIN) campaign in Chad.97 We have added a column to our CEA of AMF with cost-effectiveness estimates for Chad.98

Change 9: Updated terminology in our CEA

Our CEAs include a core model for each top charity and three additional types of adjustments:

• Downside adjustments: We make adjustments to account for the risk of "wastage" (e.g., double treatment with vitamin A supplements or deworming tablets), quality of monitoring and evaluation, and confidence in funds being used for their intended purpose.99
• Adjustments for excluded effects: We make adjustments to account for our best guesses about the impact of intervention-level factors excluded from our core CEA model.100
• Leverage and funging adjustments: Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. Our leverage and funging adjustments are informed by our estimates of the value of spending by other actors. We calculate these estimates on the "Counterfactual non-philanthropic" sheet of our CEA. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

To improve clarity, we made the following changes to the terminology used in our CEA:

• We changed the name of the "Adjustments for excluded effects" sheet to "Supplemental intervention-level adjustments."
• We changed the name of the "Downside adjustments" used in each top charity CEA to "Supplemental charity-level adjustments."
• We changed the name of the "Counterfactual non-philanthropic" sheet to "Value of counterfactual spending by other actors."101

Change 10: Updated our calculations for the value of counterfactual spending by other actors

GiveWell uses moral weights to make cost-effectiveness comparisons between interventions achieving different types of outcomes (e.g., averting the death of a child from malaria vs. doubling a very poor individual's consumption for a year). In order to make these comparisons, we calculate the moral value of different outcomes in terms of the same arbitrary units, which we call "units of value." See this page for more discussion on how GiveWell uses moral weights.

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. Our leverage and funging adjustments are informed by our estimates of the value of spending by other actors.102 We calculate these estimates on the "Value of counterfactual spending by other actors" sheet of our CEA.

Previously, we calculated the value of counterfactual spending by other actors in terms of "under-5 life saved equivalents." For consistency, we now calculate these in terms of the same "units of value" that we use for our moral weights.103

Change 11: Updated the reference value for cost-effectiveness comparisons in multiples of cash transfers

GiveWell uses moral weights to make cost-effectiveness comparisons between interventions achieving different types of outcomes (e.g., averting the death of a child from malaria vs. doubling a very poor individual's consumption for a year). In order to make these comparisons, we calculate the moral value of different outcomes in terms of the same arbitrary units, which we call "units of value." See this page for more discussion on how GiveWell uses moral weights.

Our CEA reports the cost-effectiveness of funding opportunities in terms of multiples of the cost-effectiveness of GiveDirectly's unconditional cash transfer program. We do this by calculating an estimate of the expected units of value generated per dollar donated to a funding opportunity and dividing by our estimate of the expected units of value generated per dollar donated to GiveDirectly. Previously, we performed this calculation using our estimate of GiveDirectly's cost-effectiveness before applying supplemental intervention-level and charity-level adjustments. However, we realized that our best guess of the cost-effectiveness of GiveDirectly's program is better represented by our estimate of GiveDirectly's cost-effectiveness after applying these adjustments. Therefore, we think it makes more sense to use our estimate of GiveDirectly's cost-effectiveness after applying all adjustments when comparing the cost-effectiveness of other funding opportunities to the cost-effectiveness of GiveDirectly. We have updated the calculations in our CEA accordingly.104

Change 12: Changed some hard-coded inputs to dynamic references

For some parameters in our CEA, we use estimates from the CEA of one charity to inform calculations in the CEA of another charity or to calculate the cost-effectiveness of funds spent by other actors. Previously, we used hard-coded values for these calculations, meaning that when we wanted to use an estimate from one charity's CEA to inform calculations elsewhere, we entered the numerical value for that estimate from a specific previously published version of our CEA, rather than pulling in the current value of the estimate using cell references.

We used this process for several parameters within our CEA:

• Our estimate of the counterfactual value of donated deworming drug costs was set at 10% of our estimate of the overall "units of value generated per dollar spent" in our CEA of the Against Malaria Foundation (AMF) from Version 2 of the 2020 CEA.105
• Our estimate of the counterfactual value of the Global Fund's spending was set at 38% of our estimate of the overall "units of value generated per dollar spent" in our CEA of AMF from Version 1 of the 2020 CEA. We used the same value for the counterfactual value of Gavi's spending.106
• Our estimate of the "units of value from development effects" for our CEAs of Helen Keller International and New Incentives were calculated using our estimate of the "units of value from development effects per treatment-year of SMC" in our CEA of Malaria Consortium from Version 2 of the 2020 CEA.107

We have updated these parameters to instead pull from estimates within the current version of the CEA using cell references, meaning that the values for these parameters will now change automatically if the values of the reference parameters are changed.108

We made this update so that the values for these parameters would dynamically change in response to users modifying CEA inputs and remain internally consistent.

Change 13: Updated the cost per child dewormed in our Sightsavers CEA for Nigeria

We updated our estimates of the cost per child dewormed per year and the percentage of costs covered by different actors for the Nigerian states we added to our CEA of Sightsavers in Change 5 above. Because we have not supported Sightsavers' deworming programs in these states before, we don't have cost data for these states from previous years to inform our estimates of the cost per child dewormed per year and the percentage of costs covered by different actors. Previously, we used weighted averages of these estimates across all other locations where Sightsavers supports deworming programs with funding directed by GiveWell. Using these weighted averages, our cost-effectiveness estimates for most of these states were high enough for us to consider directing funding to them.109

However, before making a decision to direct funding to Sightsavers' programs in these states, we wanted to account for the possibility that the cost per child dewormed per year in these states may be higher than in other areas we've directed funding to and, therefore, our cost-effectiveness analysis may have been underestimating the true costs of deworming programs in these states. One alternative would have been to use information from Sightsavers' budget for these programs to estimate the cost per child dewormed in these states, but we were concerned that these may also be underestimates. In order to ensure that we were confident in directing funding to these opportunities even under conservative assumptions, we decided to calculate the cost per child dewormed per year in two ways and use the greater of the two calculations, as described below.

For our estimate of the cost per child dewormed per year, we now use either the weighted average of our estimates for other states in Nigeria (excluding Benue State)110 or Sightsavers' budgeted cost per child dewormed per year for these states (including additional costs budgeted for COVID-19 mitigation), whichever is greater. For our estimates of the percentage of costs covered by different actors, we now use weighted averages of our estimates for other states in Nigeria (excluding Benue State).111

We believe these updated estimation methods better approximate expected costs in these states. 112

Change 14: Removed the representativeness adjustment from our CEA of the SCI Foundation

Our recommendation of mass deworming programs primarily relies on a series of follow-up studies to the experiment described in Miguel and Kremer 2004.113 We apply a "worm burden" adjustment to our cost-effectiveness analyses of deworming programs to account for differences in the prevalence and intensity of worm infections between the population studied in Miguel and Kremer 2004 and populations reached by our deworming top charities.114

We use survey data on the prevalence and intensity of soil-transmitted helminth and schistosomiasis infections to inform our worm burden adjustments for the countries and states supported by our deworming top charities. Previously, we believed that many of the surveys conducted in areas the SCI Foundation (SCI) supports used a sampling methodology that likely oversampled locations with above-average infection prevalence and intensity, which may have biased worm burden results upward. To account for this, we applied downward "survey representativeness" adjustments to our worm burden adjustment calculations for most of the countries and states supported by SCI.115

We have now had discussions with SCI that lead us to believe that surveys in many of the areas SCI supports used sampling methods that would be less likely to cause bias (such as random sampling) than the methods we previously thought had been used.116 As a result, we no longer believe it is appropriate to apply a survey representativeness adjustment for many of the countries and states SCI supports and have removed it from our worm burden adjustment calculations for all SCI-supported programs.117 There may be some programs for which a survey representativeness adjustment would still be appropropriate, and we may revisit this adjustment for those programs in the future.

Change 15: Added an adjustment to account for deaths averted by seasonal malaria chemoprevention (SMC) delivery in our CEA of the Against Malaria Foundation (AMF) for Nigeria, Chad, and Togo

In our CEA of AMF, we rely on under-five mortality rate estimates from Institute of Health Metrics and Evaluation's (IHME's) Global Burden of Disease (GBD) project to calculate counterfactual under-five mortality rates for each country in our CEA, which represent our best guesses of what under-five mortality rates in these countries would be if long-lasting insecticide-treated nets (LLIN) distributions ceased. These estimates inform our calculations of the mortality reduction caused by LLINs; all else equal, the higher the counterfactual mortality rate in a given area, the more deaths we expect an LLIN distribution will avert in that area.

We learned from IHME that while its 2019 model for estimating mortality rates takes into account the expected impact of LLIN distributions on mortality, it doesn't take into account the expected impact of SMC delivery on mortality.118 This means that our CEA was overestimating counterfactual mortality rates in areas where both SMC and LLIN distributions are implemented, because we would expect SMC to avert some number of under-five deaths and, therefore, lower the baseline under-five mortality rates in these areas. This, in turn, means our CEA was overestimating the mortality reduction caused by LLIN distributions in these areas.

To account for this, we have added a parameter to our CEA of AMF that estimates the number of under-five malaria deaths per 1,000 children per year that we expect would be averted by SMC in countries where both LLIN distributions and SMC delivery are implemented. We then subtract those deaths from our estimates of counterfactual mortality rates in those countries.119

Change 16: Updated the worm burden adjustment in our Sightsavers CEA for Taraba State, Nigeria

We received additional worm burden survey data (see above) from Sightsavers for Taraba State, Nigeria, that we had not previously seen. This data suggests that the prevalence of schistosomiasis infections in Taraba State may be higher than we had previously thought.

We have incorporated this new survey data into our worm burden adjustment for Sightsavers' program in Taraba. Interpreting the data we received involved making analytical choices that we have some uncertainty about (see details in footnote).120 We have updated our Sightsavers CEA for Taraba with the new worm burden adjustment.121

Change 17: Added the Federal Capital Territory (FCT) and Oyo State, Nigeria, to our CEA of Malaria Consortium

In September 2021, we learned that Malaria Consortium was seeking funding to implement its SMC program in FCT and Oyo State, Nigeria.122 We have added columns to our CEA of Malaria Consortium's SMC program with cost-effectiveness estimates for FCT and Oyo State.123

Change 18: Updated the insecticide resistance adjustments in our CEA of the Against Malaria Foundation (AMF) for Nigeria, Togo, and Uganda

In Change 3 above, we describe updates we made to our methodology for calculating our insecticide resistance adjustment for our AMF CEA for Nigeria. We updated our methodology for Nigeria first as part of an investigation into whether to direct funding to long-lasting insecticide-treated nets (LLIN) distributions in Ondo and Anambra states.124 We have now made additional updates to this methodology for Nigeria, as described below. We have also applied both sets of methodological changes to our insecticide resistance adjustments for Togo and Uganda.

Incorporating additional bioassay data

We learned that the bioassay dataset we had downloaded from IR Mapper for Change 3 above had been compiled for a specific publication, Moyes et al. 2019,125 and that more comprehensive datasets are available. We received two additional datasets from IR Mapper, one containing results from bioassays testing pyrethroids alone and one containing results from bioassays testing pyrethroids plus piperonyl butoxide (PBO). For Nigeria, Togo, and Uganda, these datasets contained results from bioassays conducted since 2010. We also received a third dataset from IR Mapper containing results from bioassays of pyrethroids alone and pyrethroids plus PBO conducted in Nigeria in 2019 by the U.S. President's Malaria Initiative (PMI).126

Insecticide resistance adjustment for standard LLINs

Previously, we used average mosquito mortality rates from bioassays conducted between 2010 and 2016, as reported in the World Health Organization's (WHO's) Global report on insecticide resistance, to calculate expected mosquito mortality rates at the time of upcoming distributions in each country. We now use bioassay results from the datasets we received from IR Mapper to calculate the average mosquito mortality rates found in bioassays of pyrethroids alone conducted in each country since 2015,127 which we then use to project mosquito mortality rates at the time of upcoming distributions.128

Insecticide resistance adjustment for PBO nets

We use mosquito mortality rates found in bioassays of pyrethroids alone and bioassays of pyrethroids plus PBO to estimate the proportion of insecticide resistance that remains when using PBO nets. In order to directly compare mosquito mortality rates for pyrethroids and pyrethroids plus PBO, we paired up bioassays from the three datasets we received from IR Mapper that were conducted at the same sites as part of the same studies and listed them on this spreadsheet.129 We then calculated the average mosquito mortality rates for pyrethroids alone and pyrethroids plus PBO among the paired bioassays, excluding those that had been conducted before 2015.130 We use the average mosquito mortality rate for pyrethroids plus PBO and the average mosquito mortality rate for pyrethroids alone to calculate our estimate of the proportion of insecticide resistance that remains when using PBO for each country.131

Our understanding is that PBO only improves the effectiveness of LLINs in locations where insecticide resistance is driven by a particular resistance mechanism. Specifically, WHO recommends the use of PBO nets only in locations where a "monooxygenase-based" resistance mechanism has been detected.132 To account for this, we apply an external validity adjustment to our calculations that reflects our estimate of the proportion of a country where PBO nets are more effective than standard LLINs. Previously, we based this estimate on summary information from WHO's Malaria Threat Map on the proportion of test sites within a country with confirmed pyrethroid insecticide resistance and the proportion of sites where a monooxygenase-based resistance mechanism was detected.

However, upon further investigation, we were unable to replicate the summary estimates from the Malaria Threat Map using the publicly available WHO data we found. In addition, the proportion of test sites where a relevant resistance mechanism was detected reported in the summary data (40-45% for each of Nigeria, Togo, and Uganda) seemed lower than what we would expect, based on a conversation we had with an insecticide resistance expert133 and our expectations of the proportions of PBO nets that are appropriate for these countries. As such, we were concerned that using these figures could lead us to underestimate the benefits of PBO nets in our CEA. To address this concern, we used data downloaded from WHO's Malaria Threat Map to calculate our own estimates of the proportion of insecticide resistance mechanism tests where a relevant resistance mechanism was detected for Nigeria, Togo, and Uganda.134

Calculating the final insecticide resistance adjustment

We combine our calculated insecticide resistance adjustments for standard LLINs and insecticide resistance adjustments for PBO nets to arrive at our final insecticide resistance adjustments for each country. Previously, we had assumed that the PBO nets purchased for a distribution would be distributed randomly within a country, rather than targeted at the specific areas where we would expect PBO nets to be more effective than standard LLINs. Based on conversations we've had with AMF,135 we have updated our calculations to instead assume that PBO nets will be targeted to the areas where we expect them to be more effective than standard LLINs.136

Change 19: Corrected an error in our CEA of the Against Malaria Foundation (AMF) for Chad

We noticed that we had incorrectly copied data from Institute of Health Metrics and Evaluation's (IHME's) Global Burden of Disease (GBD) project on malaria mortalities into our AMF CEA for Chad, which we added in Change 8 above. We have corrected this error.137

Change 20: Updated the worm burden adjustments for all of our deworming CEAs

When calculating worm burden adjustments (see above) for our deworming CEAs, we adjust unusually high worm burden results downward by setting results above the 90th percentile (across all programs) equal to the value of the 90th percentile.138 We make this adjustment because the sources of worm prevalence data we use are subject to error, imprecision, and variations in methodology, and we believe it's likely that true worm prevalence is less than what the data suggests for programs with unusually high worm burden results.

Because our worm burden adjustment calculations rely on the value of the 90th percentile across all programs, adding programs to the list (or updating our calculations for any one of them) can shift the worm burden adjustments for other programs. This means that changes 5, 14, and 16 above may have had an impact on the worm burden adjustment values for locations other than those directly addressed by these changes. To account for this, we have updated the worm burden adjustments for all locations in our deworming CEAs.139

Change 21: Updated the leverage and funging adjustment in our CEA of Malaria Consortium's seasonal malaria chemoprevention (SMC) program in Nigeria

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur in the absence of philanthropic support for a particular program (e.g., "government costs would replace philanthropic costs" or "distributions would go unfunded"). Since publishing the previous version of the CEA, we have learned more about the malaria funding landscape in Nigeria and have revised our views about the likelihood that the Global Fund to Fight AIDS, Tuberculosis and Malaria ("the Global Fund") or the President's Malaria Initiative (PMI) would fund SMC there in the absence of philanthropic support.140
Based on this new information, we reduced our estimate of the probability that Malaria Consortium's support for SMC delivery in Nigeria would be replaced by funding from PMI or the Global Fund in Malaria Consortium's absence from 55% to 25%.141

• 1

  See the adjusted malaria mortality and prevalence adjustments in the version of the CEA preceding this change here. See our calculations for these adjustments here.

• 2

  See the version of the CEA following this change here. See the effect this change had on our overall cost-effectiveness estimates here.

• 3

  See the version of the CEA preceding this change here.

• 4

  See the version of the CEA following this change here. See the effect this change had on our overall cost-effectiveness estimates here. For more detail on why we chose this probability estimate, see this spreadsheet.

• 5

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 6

  IR Mapper contains five bioassay results for standard insecticide alone and four bioassay results for standard insecticide plus PBO since 2015, all of which were conducted in 2018 in or around the cities of N'Djamena or Massakory. The results of these bioassays are published in Ibrahim et al. 2019.

• 7

  For bioassays of standard insecticide alone, we use data from 2013 and 2014 in addition to the 2018 data points. For bioassays of standard insecticide plus PBO, we use data from 2010 in addition to the 2018 data points. In both cases, these were the next most recent sets of data points available in IR Mapper's dataset.

• 8

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 9
  • Vitamin A deficiency is associated with increased susceptibility to infectious disease and death. Vitamin A supplementation is thought to reduce child mortality by reducing the prevalence and severity of infectious disease, such as measles and diarrheal disease. For more, see our full report on vitamin A supplementation.
  • The effect size found in Imdad et al. 2017 is higher than we would expect based on the rates of infectious disease and the reductions in infectious disease mortality found in the trials. For more, see the "Mortality mechanisms analysis" sheet of our internal validity adjustment calculations.

• 10

  See the previous version of our calculations here.

• 11

  See our calculations here.

• 12

  Specifically, our previous methodology accounted for non-independence by applying a 30% increase to our external validity adjustments across the board. This meant that for countries where the proportion of deaths caused by infectious diseases were already similar to those in trial contexts, like Cameroon, our external validity adjustment was being artificially inflated by the non-independence adjustment. See the previous version of our calculations here.

• 13

  See our calculations here.

• 14

  See the version of the CEA following this change here.

• 15

  See our insecticide resistance adjustment calculations here.

• 16

  See our report on insecticide resistance and malaria control here.

• 17

  See the bioassay results from the report here.

• 18

  For example, we discuss potential development impacts of mass deworming programs in this section of our intervention report.

• 19

  Malaria Consortium's SMC programs typically deliver four "cycles" of SMC per year. Each cycle includes a four-day distribution period and lasts 28 days, after which a new cycle starts. In late 2020, we learned that Malaria Consortium was planning to support a fifth SMC cycle per year in some countries going forward.

• 20

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 21

  See the updated version of our cost per infant immunized analysis for New Incentives here. See the previous version here.

• 22

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 23

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 24
  • We have not made our cost per net analysis spreadsheet publicly available because we have not received permission to publish country-specific cost estimates we received from the Global Fund, which our calculations rely on. See this section of our review of AMF for additional information.
  • See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 25

  See this section of our review of Malaria Consortium for more details.

• 26

  See our 2021 room for more funding analysis for Malaria Consortium's SMC program, sheets "Source: Workings" and "[From 2020] Source: 2020-23 Projections" for more details.

• 27

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 28

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 29
  • The implementation of SMC during the rainy season in Mozambique across calendar years is described in this cell of the "[From 2020] Source: 2020-23 Projections" sheet of our 2021 room for more funding analysis for Malaria Consortium's SMC program.
  • For more details about the pilot, see the "Mozambique" row of the "Source: 2021-24 projections" sheet of our 2021 room for more funding analysis for Malaria Consortium's SMC program.

• 30

  See the version of the CEA following this change here.

• 31

  "Vitamin A supplements in Kenya are administered to children aged 6-59 months through various delivery approaches. Throughout the year, children can access VAS in primary health care facilities, but this routine coverage only accounts for around 20 percent of children, essentially children below 12 months, as many caregivers do not bring their children to the health facilities after the end of the immunization contact points at one year of age." Helen Keller International, Room for more funding report, 2021, Pg. 21.

• 32

  "Throughout the year, children can access VAS in primary health care facilities, but this routine coverage only accounts for around 20 percent of children, essentially children below 12 months, as many caregivers do not bring their children to the health facilities after the end of the immunization contact points at one year of age." Helen Keller International, Room for more funding report, 2021, Pg. 21.

• 33

  Compare mortality rate estimates for the post-neonatal (28 to 364 days of age) and 1 to 4 year-old age groups for countries where Helen Keller supports VAS programs in the Institute for Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) project here. We expect that if double treatment were concentrated among the sub-group of the target population with the highest mortality rates, it would have a greater impact on the cost-effectiveness of the program than if rates of double treatment were roughly equal across subgroups of the target population.

• 34

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 35

  We had previously seen monitoring results from all four countries Helen Keller supported with GiveWell-directed funding in 2018 and from five out of the six countries it supported with GiveWell-directed funding in 2019. For information on 2018 monitoring, see here. For information on 2019 monitoring, see here. For information on 2020 monitoring, see here.

• 36

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 37

  See the version of the CEA following this change here.

• 38

  See the version of the CEA following this change here.

• 39

  Our CEAs for GiveDirectly and New Incentives do not include these estimates because we don't calculate cost-effectiveness for multiple countries in these CEAs.

• 40

  The exceptions are for SCI Foundation, where we continue to calculate overall cost-effectiveness across countries, and for GiveDirectly and New Incentives, for which we don't calculate cost-effectiveness for multiple countries.

• 41

  The exception is our CEA for SCI Foundation, where we use these estimates to calculate the program's overall cost-effectiveness across countries.

• 42

  See the version of the CEA preceding this change here and the version of the CEA following this change here. For an example of the effect of this change, see the "Donation to Deworm the World (arbitrary size)" row before the change and after the change.

• 43

  See our 2021 room for more funding analysis of Helen Keller's VAS program for more details.

• 44
  • Our views on the funding landscape for VAS are informed by conversations we've had with Helen Keller International, Global Affairs Canada's nutrition team, UNICEF's West and Central Africa regional office, USAID, the nutrition team at the Bill and Melinda Gates Foundation, and the nutrition departments of the Ministries of Health in Burkina Faso, Côte d'Ivoire, Guinea, Mali, and Nigeria.
  • See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 45

  For more details on the data we used in our calculations, see our New Incentives CEA supplemental information sheet from June 2021, sheets "Probability of death" and "Baseline vaccination coverage." To see how we used these calculations to inform our cost-effectiveness estimates, see this section of the New Incentives CEA from the version of the CEA preceding this change.

• 46

  See our calculations for this adjustment in this spreadsheet.

• 47

  National-level vaccination coverage data from WHO and UNICEF is available on this sheet, where we estimate an overall effective vaccination coverage rate of 50%. We used that 50% effective coverage estimate in our calculations of the probability of death from vaccine-preventable diseases for both vaccinated and unvaccinated children in Nigeria in the "Adjustment for all-cause mortality effect" section in the version of the New Incentives CEA preceding this change here.

• 48

  The data we received is available in this spreadsheet.

• 49

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 50

  As of the date of this update, the most recent state-level vaccine coverage estimates in Nigeria available in IHME's VizHub tool were for the first and third doses of the DPT (diphtheria, pertussis, and tetanus) vaccine in 2016. We compare state-level and national-level coverage estimates for those vaccines on this spreadsheet. We then use the ratio of vaccinate coverage estimates for the first and third DPT doses between the states where New Incentives works and Nigeria overall to adjust national-level estimates of coverage for all routine vaccinations on this sheet, which leads us to estimate an effective baseline vaccination rate in the states where New Incentives works of 16%. The effective vaccination coverage rate of 16% is used in the CEA following this change here.

• 51

  See the previous version of our cost per infant immunized analysis here, and the updated version of our cost per infant immunized analysis here.

• 52

  See the following sheets in our 2021 cost per infant immunized analysis for New Incentives: "Source: WHO total expenditure routine immunization," "Source: WHO gov expenditure routine immunization," "Baseline vaccine coverage and population data"

• 53

  See the previous version of our cost per infant immunized analysis here.

• 54

  See the updated version of our cost per infant immunized analysis here. For WHO's data on spending on routine infant immunization by year, see "Source: WHO total expenditure routine immunization" and " Source: WHO gov expenditure routine immunization"

• 55

  See our estimate here. Note that this estimate does not include New Incentives' spending.

• 56

  Our assumption is based on information in the Nigeria Joint Immunization Strategy Document 2018-2018:

  • Stakeholders listed as being involved in the preparation of the document include the Federal Ministry of Health and Gavi. Pg 8.
  • Total costs are listed as being projected to be covered by a combination of the Nigerian government and Gavi between 2018 and 2028. Pg 24, Figure 8.

• 57

  Total costs are listed as being projected to be covered by a combination of the Nigerian government and Gavi between 2018 and 2028. Nigeria Joint Immunization Strategy Document 2018-2028, Pg 24, Figure 8.

• 58

  See the "Costs covered by Gavi and government" section of our spreadsheet here. The joint immunization strategy document is listed as being published in 2018. Nigeria Joint Immunization Strategy Document 2018-2028, Pg 1.

• 59

  See our previous cost per infant immunized calculations here, "Gavi vs. government costs" section.

• 60

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 61

  These states are Bauchi, Kebbi, Kogi, Nasarawa, Plateau, and Sokoto. Malaria Consortium also supports SMC in FCT and Oyo states in Nigeria, but we estimate the cost-effectiveness of SMC in these states in separate columns of our Malaria Consortium CEA.

• 62

  See this malaria mortality and prevalence data here. See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 63

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 64For more details, see this page. The two states not covered by one of these funding sources are Ondo and Anambra.
• 65

  See this malaria mortality and prevalence data here. See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 66See the version of the CEA preceding this change here.
• 67"Published in The Lancet in October 2020, GBD 2019 provides for the first time an independent estimation of population, for each of 204 countries and territories and the globe, using a standardized, replicable approach, as well as a comprehensive update on fertility and migration. GBD 2019 incorporates major data additions and improvements, and methodological refinements." GBD 2019 Resources page
• 68See the version of the CEA following this change here.
• 69For more details, see this page. The two states not covered by one of these funding sources are Ondo and Anambra.
• 70See the version of the CEA following this change here.
• 71We have since recommended that Open Philanthropy grant $27.7 million to Malaria Consortium to support LLIN distributions in these states.
• 72

  See the "What is resistance" and "What is the relationship between 'resistance' and ITN control failure?" sections of our report on insecticide resistance and malaria control.

• 73

  See the "Where is there insecticide resistance?" and "Is insecticide resistance increasing?" sections of our report on insecticide resistance and malaria control.

• 74

  See this section of our report on insecticide resistance and malaria control for our review of the evidence.

• 75
  • See our insecticide resistance adjustment calculations in this spreadsheet.
  • We have not made our estimates of the proportion of PBO nets and standard LLINs AMF plans to purchase publicly available because we have not received permission to publish this information from AMF.

• 76

  For more on how insecticide resistance is measured, see this section of our insecticide resistance report.

• 77

  See our calculations here.

• 78
  • We use an update to the Lengeler 2004 meta-analysis, Pryce et al. 2018, to inform our estimate of the malaria mortality reduction caused by LLIN distributions in our AMF CEA.
  • See our calculations here.

• 79

  See our calculations here.

• 80

  For more details, so this section of our report on insecticide resistance and malaria control.

• 81

  See our calculations here.

• 82
  • See our calculations here.
  • We have not made our estimates of the proportion of PBO nets and standard LLINs AMF plans to purchase publicly available because we have not received permission to publish this information from AMF.

• 83We have since recommended that Open Philanthropy grant $27.7 million to Malaria Consortium to support LLIN distributions in these states.
• 84See this dataset here.
• 85See our calculations here.
• 86

  "On the basis of the current evidence, WHO concludes and recommends the following: 1. Epidemiological data from one cluster randomized controlled trial indicated that a pyrethroid-PBO net product had additional public health value compared to a pyrethroid-only LLIN product in an area where the main malaria vector had confirmed pyrethroid resistance of moderate intensity conferred (at least in part) by monooxygenase-based resistance mechanism as determined by standard procedures." WHO, Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017, Pg. 2.

• 87See the information we used from WHO's Malaria Threat Map here.
• 88See our calculations here.
• 89
  • See our calculations here.
  • See the version of the CEA following this change here and our calculations on this spreadsheet.

• 90See here.
• 91See the version of the CEA preceding this change here and the version of the CEA following this change here.
• 92We have since recommended that Open Philanthropy grant $4.4 million to Sightsavers to support its work in some of these states.
• 93See the version of the CEA following this change here. The change can be seen here and here.
• 94For more on the evidence behind PBO nets and their role in AMF-supported distributions, see this blog post.
• 95
  • We have not made our cost per net analysis spreadsheet publicly available because we have not received permission to publish country-specific cost estimates we received from the Global Fund, which our calculations rely on. See this section of our review of AMF for additional information.
  • See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 96See the version of the CEA preceding this change here and the version of the CEA following this change here.
• 97See further discussion of AMF's plans to support the 2023 LLIN campaign in Chad here.
• 98See the version of the CEA following this change here.
• 99

  See here for an example of how we apply these adjustments in our CEA for Malaria Consortium.

• 100
  • See the "Adjustments for excluded effects" sheet here for details on how we calculate these adjustments.
  • See here for an example of how we apply these adjustments in our CEA for Malaria Consortium.

• 101See the version of the CEA following this change here.
• 102For a full introduction to our approach to leverage and funging adjustments, see this blog post.
• 103See the version of the CEA following this change here.
• 104See the version of the CEA preceding this change here and the version of the CEA following this change here. The new calculation can be seen in the "Initial cost-effectiveness estimate in multiples of cash transfers" row for each CEA; for example, here in the Malaria Consortium sheet.
• 105See the parameter here, based on the version of the CEA here.
• 106See the parameter here, based on the version of the CEA here.
• 107See the parameters here and here, based on the version of the CEA here.
• 108
  • Our estimate of the counterfactual value of donated deworming drug costs is now set at 10% of our current estimate of the overall "units of value generated per dollar spent" in our AMF CEA for DRC. See the calculation here.
  • Our estimate of the counterfactual value of the Global Fund's spending is now set at 38% of our current estimate of the overall "units of value generated per dollar spent" in our AMF CEA for DRC. We use the same value for the counterfactual value of Gavi's spending. See the calculation here.
  • Our estimate of the "units of value from development effects" for our CEAs of Helen Keller International and New Incentives were calculated using our current estimate of the "units of value from development effects per treatment-year of SMC" in our Malaria Consortium CEA for Nigeria. See the calculations in this row and this row.
  • See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 109At the time of this CEA change, we were primarily looking to direct funding to opportunities that we estimated to be at least eight times as cost-effective as GiveDirectly's unconditional cash transfer program ("8x cash"). Our cost-effectiveness estimates for five out of the six new Nigerian states were above this bar: Kaduna (30x), Kano (13x), Niger (42x), Adamawa (22x), and Zamfara (39x). Our cost-effectiveness estimate for Katsina state was slightly below this bar at 7x. See the version of the CEA preceding this change here.
• 110We chose to take a weighted average of estimates across other Nigerian states supported by Sightsavers because we believed this would better approximate costs in the new Nigerian states than taking a weighted average of costs across all countries supported by Sightsavers. However, we would note that there is a less than $0.01 difference between these two averages, so we think this decision did not make much difference in practice. We exclude Benue State from our weighted average because the program supported by Sightsavers in Benue is an integrated neglected tropical disease program involving the distribution of drugs for diseases other than soil-transmitted helminthiasis and schistosomiasis, which are the two diseases modeled in our CEA.
• 111We chose to take a weighted average of estimates across other Nigerian states supported by Sightsavers because we believed this would better approximate the proportion of costs covered by different actors in the new Nigerian states than taking a weighted average across all countries supported by Sightsavers. We exclude Benue State from our weighted averages because the program supported by Sightsavers in Benue is an integrated neglected tropical disease program involving the distribution of drugs for diseases other than soil-transmitted helminthiasis and schistosomiasis, which are the two diseases modeled in our CEA.
• 112See the version of the CEA following this change here. The cells that have been changed are highlighted.
• 113

  For further discussion, see:

  • this section of our intervention report on mass deworming programs
  • the following posts on the GiveWell blog: Why I mostly believe in Worms and How thin the reed? Generalizing from "Worms at Work"

• 114See a more detailed explanation of our worm burden adjustment here and our calculations here.
• 115We did not apply survey representativeness adjustments for SCI's programs in Uganda and Niger. The information we had seen about surveys in these two countries led us to believe they were less likely to be biased toward high-prevalence locations than surveys in other areas supported by SCI.
• 116Fiona Fleming, Director, Monitoring, Evaluation & Research, SCI Foundation, conversation with GiveWell, April 28, 2021 (unpublished).
• 117See our worm burden calculations here. See the version of the CEA preceding this change here and the version of the CEA following this change here.
• 118Alexandra Walker, Engagement Officer, IHME, email to GiveWell, July 21, 2021 (unpublished).
• 119See the version of the CEA following this change here. The previous version of the CEA, linked here, did not include this parameter.
• 120The survey data we received included surveys labeled as "mapping" surveys, as well as several surveys labeled as "impact" surveys. We decided to exclude the surveys labeled as "impact" surveys because we think they are less likely to be representative than the mapping surveys. The survey data did not include information on infection intensity, nor did it distinguish between infections caused by S. haematobium and S. mansoni, the two most common schistosomiasis-causing worm species in sub-Saharan Africa. We imputed the prevalence of light, moderate, and heavy infections, as well as the prevalence of infections caused by S. haematobium and S. mansoni, using national survey data for Nigeria. For more details on the different types of schistosomiasis infection, see our deworming report.
• 121See the version of the CEA preceding this change here and the version of the CEA following this change here.
• 122We have since recommended that Open Philanthropy grant $15.9 million to Malaria Consortium to support this work.
• 123See the version of the CEA following this change here.
• 124We have since recommended that Open Philanthropy grant $27.7 million to Malaria Consortium to support LLIN distributions in these states.
• 125See this dataset here.
• 126See these datasets in this spreadsheet, sheets "[IR Mapper] Phenotypic resistance," "[IR Mapper] Synergist assays," and "[IR Mapper] PMI 2019 Nigeria data."
• 127We use bioassay results from 2015 onward because our understanding is that the prevalence of insecticide resistance can change rapidly over time, such that restricting our analysis to more recent bioassay results may improve the accuracy of our calculations.
• 128See our calculations here.
• 129For some of the bioassay results, we were unable to identify matches within the two datasets. In a few cases, we were able to find matches within the original IR Mapper dataset that had been compiled for Moyes et al. 2019 or within the President's Malaria Initiative's Vectorlink Nigeria final entomology report from 2018 instead. In other cases, we were unable to find matches at all. We excluded bioassay results without matches from the calculations that follow.
• 130We use bioassay results from 2015 onward because our understanding is that the prevalence of insecticide resistance can change rapidly over time, such that restricting our analysis to more recent bioassay results may improve the accuracy of our calculations.
• 131See our calculations here.
• 132 "On the basis of the current evidence, WHO concludes and recommends the following: 1. Epidemiological data from one cluster randomized controlled trial indicated that a pyrethroid-PBO net product had additional public health value compared to a pyrethroid-only LLIN product in an area where the main malaria vector had confirmed pyrethroid resistance of moderate intensity conferred (at least in part) by monooxygenase-based resistance mechanism as determined by standard procedures." WHO, Conditions for deployment of mosquito nets treated with a pyrethroid and piperonyl butoxide, 2017, Pg. 2.
• 133GiveWell's non-verbatim summary of a conversation with Dr. Kevin Ochieng Opondo, August 19, 2021.
• 134See our calculations for these estimates here and our integration of these estimates into our insecticide resistance adjustment calculations here.
• 135AMF, conversation with GiveWell, August 8, 2021 (unpublished).
• 136See our updated insecticide resistance adjustment calculations here. See the version of the CEA preceding this change here and the version of the CEA following this change here.
• 137See the version of the CEA preceding this change here and the version of the CEA following this change here.
• 138See our worm burden adjustment calculations here, sheet "Worm burden synthesis: program adjustments," columns L-N.
• 139See the versions of the CEAs preceding this change here (Deworm the World), here (END Fund), here (SCI Foundation), and here (Sightsavers). See the versions of the CEAs following this change here (Deworm the World), here (END Fund), here (SCI Foundation), and here (Sightsavers).
• 140We plan to discuss this updated view in depth in a forthcoming grant page.
• 141See the version of the CEA preceding this change here and the version of the CEA following this change here.