2021 Cost-Effectiveness Analysis Changelog

This page provides details about changes that were made to our cost-effectiveness analysis (CEA) in 2021. For past versions of our CEA, see this page.

Table of Contents

Version 3 — Published September 28, 2021

Link to the cost-effectiveness analysis (CEA) file: 2021 CEA — version 3

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updated downside adjustments for New Incentives

For each of our top charities, we use a core cost-effectiveness model along with a set of additional adjustments. One of the types of adjustments we use are "downside adjustments," which account for the risk of "wastage" (e.g. double treatment with vitamin A supplements or deworming tablets), quality of monitoring and evaluation, and our confidence in funds being used for their intended purpose.1

Our CEA for New Incentives previously used an initial rough best guess of the total downside adjustment factor of 5%.2 To make our thinking explicit and improve the accuracy of our cost-effectiveness estimate, we updated the CEA to include specific estimates and explanations for each input we consider when making downside adjustments, which led to a total downside adjustment of 7%.3

Change 2: Updated estimates from the Institute of Health Metrics and Evaluation (IHME) in our New Incentives CEA

Our CEA of New Incentives relies on data from the Institute of Health Metrics and Evaluation (IHME), primarily its estimates of child mortality from vaccine-preventable diseases in Nigeria. We use two IHME data sources, its Global Burden of Disease (GBD) project for national estimates, and its Local Burden of Disease (LBD) project for subnational estimates.4

Previously, the New Incentives CEA relied on data from the 2017 versions of GBD and LBD.5 In October 2020, however, IHME released an updated version of GBD for 2019, and a portion of data from LBD has also been updated to include estimates for 2019.6 We have incorporated these more recent estimates into our CEA of New Incentives, which we previously did for AMF, Malaria Consortium, and Helen Keller International.7

Change 3: Added Guinea-Bissau and Yobe State, Nigeria to Sightsavers CEA

We have added columns to our CEA of Sightsavers with cost-effectiveness estimates for Yobe State and Guinea-Bissau.8

Change 4: Updated adjustment for seasonal malaria chemoprevention (SMC) distributions with five cycles

In late 2020, Malaria Consortium told us it planned to support five cycles of SMC per year in some countries, instead of the four cycles it typically supports.9 Each cycle includes a four-day distribution period and lasts 28 days, after which a new cycle starts.10

We previously added an adjustment to our CEA to account for our best guess of the effect that shifting from four to five cycles has on cost-effectiveness (see the changelog entry below). To calculate this adjustment, we used information from Malaria Consortium on the percentage of districts in Burkina Faso, Chad, Nigeria, and Togo where it expected to support five cycles.11 We have now updated the calculation to include more recent estimates from Malaria Consortium on the proportion of children expected to receive five cycles in each country.12

Change 5: Removed "Cost-effectiveness by funding gap" and room for more funding sheets

The CEA previously included a "Cost-effectiveness by funding gap" sheet, which compared the cost-effectiveness of each spending opportunity to which our top charities planned to allocate additional GiveWell funding, and provided notes on how we prioritize the funding gaps relative to each other.13 The CEA also included "room for more funding" sheets, which presented our projections of charities' yearly funding gaps.14

To simplify our presentation of the CEA, we have removed the "Cost-effectiveness by funding gap" and room for more funding sheets.15 We continue to track our top charities' room for more funding outside of the CEA.16

Version 2 — Published July 6, 2021

Link to the cost-effectiveness analysis (CEA) file: 2021 CEA — version 2

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updated the baseline mortality rate estimate for our CEA of Helen Keller International (Helen Keller) in Nigeria

Helen Keller has proposed supporting vitamin A supplementation (VAS) campaigns in the following four states in Nigeria between 2021 and 2023: Akwa Ibom, Cross River, Adamawa, and Taraba.17 Prior to this change, our CEA for Helen Keller in Nigeria used national-level child mortality rate estimates from the 2019 version of the Institute for Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) project.18

We updated our CEA of Helen Keller in Nigeria using averages of state-level child mortality rates from IHME's 2017 Local Burden of Disease (LBD) project, weighted by Helen Keller's proposed budget in each of the four states between 2021 and 2023.19 Our intention with this change was to increase the accuracy of our Helen Keller Nigeria CEA by relying more on mortality rate estimates specific to the states where Helen Keller expects to work. We understand that Helen Keller may change its plans for which states it supports as it learns more about the needs in each state, however we'd guess that modeling these four states reflects a better estimate of the impact of additional funding for Helen Keller's work in Nigeria than using the national average.

Change 2: Updated our leverage and funging adjustments for the Against Malaria Foundation (AMF) and Helen Keller

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur in absence of philanthropic support for a particular program (e.g. "government costs would replace philanthropic costs" or "distributions would go unfunded"). We reduced our estimate of the probability that AMF's support for long-lasting insecticide-treated net (LLIN) distributions in the Democratic Republic of the Congo (DRC) would be replaced by funding from the Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund) or the President's Malaria Initiative (PMI) in AMF's absence from 52% to 25%.20 This change is based on our expectation that the remaining funding gap in DRC is a low priority for any additional funding from the Global Fund or PMI. Our understanding is that the effect of additional funding for DRC will be to reduce the time elapsed between distributions from 39 months to 30 months (see above for more details). Because it is our understanding that the Global Fund and PMI are funding this reduction in few other locations, we think it is unlikely that they would prioritize this gap over other malaria control gaps.

We also reduced our estimate of the probability that Helen Keller's support for VAS in Cameroon would be replaced by other philanthropic actors in Helen Keler's absence from 40% to 30%.21 Our 30% estimate takes into account our understanding, based on conversations with UNICEF, that it is unlikely that UNICEF will provide funding for VAS campaigns for the specific regions in Cameroon Helen Keller plans to support.22 However, it is also our understanding that there is some chance that other national campaigns (e.g., for polio immunizations) might be implemented in Cameroon in the next few years, to which VAS could be added inexpensively with other philanthropic funding sources.

Change 3: Updated the baseline mortality rate estimate for our CEA of Helen Keller in Cameroon

Helen Keller has proposed supporting four regions in Cameroon between 2021 and 2023: Centre (Central), Littoral, Ouest (West), and Sud (South).23

We have made region-level adjustments for Helen Keller's program in Cameroon using the same methodology as the adjustments described for Helen Keller's program in Nigeria above.24

Change 4: Updated our room for more funding estimates for AMF, Malaria Consortium, and New Incentives

Based on new information provided by AMF, Malaria Consortium, and New Incentives, we updated our estimates of the amount of funding these charities can absorb and the countries where they would allocate additional funding.25

We calculate our updated estimates in our room for more funding analysis spreadsheets.26

Change 5: Corrected an error in our net spacing adjustment for AMF

For each of our top charities, we use a core cost-effectiveness model along with a set of additional adjustments. One type of adjustment, "adjustments for effects excluded from our core model," accounts for our best guesses of the impact of intervention-level factors we have not analyzed in depth.27

In version 1 of our 2021 CEA for AMF, we added a new adjustment for an effect excluded from our core CEA, the "Adjustment for program impact being to move distributions closer together" (see change 11 above). This adjustment accounts for our understanding that in DRC and Nigeria, additional funding for LLIN distributions may lead to reductions in the amount of time between distributions (e.g., moving from a four-year spacing to the recommended three-year spacing). For other countries, our underlying assumption is that AMF's impact is primarily to increase coverage for campaigns that occur every three years.

We noticed that estimates calculated in our net spacing adjustment spreadsheet were copied incorrectly into our CEA for AMF. We have corrected the error.28

Change 6: Q1 2021 Top Charities Fund allocation

We allocated funding from our Top Charities Fund to a set of our top charities' funding gaps. See our funding allocation in our "Cost-effectiveness by funding gap" sheet here. We describe the reasoning behind this allocation on this page.

Change 7: Updates to "Cost-effectiveness by funding gap" sheet

We have updated the room for more funding estimates on the "Cost-effectiveness by funding gap" sheet to display our top charities' funding gaps after taking the April 2021 funding allocation into account. See this sheet.

Version 1 — Published May 5, 2021

Link to the cost-effectiveness analysis (CEA) file: 2021 CEA — version 1

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updated Global Burden of Disease (GBD) estimates in our CEA of the Against Malaria Foundation (AMF)

Our CEA of AMF relies on country-level estimates of malaria prevalence, child mortality rates, and population distributions by age from the Institute of Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) project.29

We previously relied on the 2017 version of the project (GBD 2017), but in October 2020, IHME released an updated version, GBD 2019.30 We have incorporated updated GBD 2019 estimates into our CEA of AMF.31

Change 2: Updated GBD estimates in our CEA of Malaria Consortium

We also incorporated updated GBD 2019 estimates for malaria prevalence, child mortality rates, and population distributions by age into our CEA of Malaria Consortium (see change 1 above).32

Change 3: Corrected an error in our leverage and funging adjustment for AMF

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

Our leverage and funging adjustments for AMF are calculated on a non-public spreadsheet. We have not made this spreadsheet publicly available because we have not received permission to publish country-specific cost estimates we received from the Global Fund to Fight AIDS, Tuberculosis and Malaria, which our calculations rely on.33

We noticed that the discount rate on the separate AMF leverage and funging spreadsheet had not been updated along with the discount rate in the main CEA in 2020 (see our 2020 CEA changelog here). We have corrected the error.34

Change 4: Updated percentage of program costs covered by different organizations for our CEA of Helen Keller International in Cameroon

In December 2020, Helen Keller International provided us with updated information on the proportions of costs covered by different organizations for a proposed vitamin A supplementation (VAS) program in Cameroon.35 We have incorporated this updated information into our CEA of Helen Keller International.36

Change 5: Updated Helen Keller International CEA with GBD 2019 mortality rate estimates

Our CEA of Helen Keller International also relies on GBD data (see change 1 and change 2 above). We have updated the mortality rate estimate inputs in our Helen Keller International CEA from GBD 2017 to GBD 2019.37

Our CEA of Helen Keller International also relies on estimates of the prevalence of vitamin A deficiency (VAD) from GBD, but we have not yet updated these estimates from GBD 2017 to GBD 2019.38 We have not yet updated the VAD prevalence estimates because we do not have a clear understanding of the changes in IHME's modeling assumptions and data sources for VAD between GBD 2017 and GBD 2019—we plan to seek more information from IHME this year about our remaining questions.39

Change 6: Added our CEA of New Incentives

In November 2020, we added New Incentives to our list of top charities.

Our original CEA of New Incentives is here. We have adapted the format of the New Incentives CEA and added it to the workbook with our other top charity CEAs (see the version of the CEA following this change here).40

Change 7: Updated AMF cost per net and insecticide resistance adjustment for the Democratic Republic of the Congo

Insecticide resistance is a threat to the effectiveness of long-lasting insecticide-treated nets (LLINs) in preventing malaria.41 We believe that there is some evidence that a newer type of LLIN incorporating piperonyl butoxide ("PBO nets") provide additional protection in areas where mosquitoes have developed insecticide resistance.42

In January 2021, AMF shared updated information with us on the proportion of PBO nets it planned to purchase for upcoming LLIN distributions in the Democratic Republic of the Congo (DRC).43 We have updated our CEA of AMF to account for the greater cost of PBO nets and their expected effectiveness in DRC.44

Change 8: Updated additional external validity adjustment for Helen Keller International in Kenya

For our 2020 CEA — version 2, we added an additional external validity adjustment for Helen Keller International in Kenya to account for relatively low baseline mortality rates and a micronutrient survey finding low prevalence of vitamin A deficiency (see our 2020 changelog here).

After further investigation in 2021, we increased the size of our downward adjustment to Helen Keller International's expected cost-effectiveness in Kenya. We describe our reasoning for our updated adjustment in this document.45

Change 9: Updated equivalent coverage years estimate for LLINs in DRC

We expect that LLINs have limited lifespans—over time, the insecticide on the nets may wear off or the nets may acquire holes.46 In countries other than DRC, we estimate that AMF's LLINs provide 2.11 years of effective protection during each three-year distribution cycle (relative to the effectiveness of the bed nets used in the randomized controlled trials we rely on for our estimate of the impact of bed nets on child mortality).47

We have previously applied a 10% downwards adjustment (to 1.90 equivalent coverage years) for AMF's bed net distributions in DRC, relative to its distributions in other countries.48 That adjustment was based on our rough impression that post-distribution monitoring (PDM) results we had seen from AMF's distributions in DRC indicated shorter LLIN lifespans than data we had seen from other countries.49

After further investigation, we have updated our estimate of the durability of LLINs in DRC. Based on PDM results for AMF distribution in Equateur province, we now expect that nets in DRC have around 17% lower durability (1.74 equivalent coverage years) than nets AMF distributes in other countries—see our full updated analysis in this spreadsheet.50

Change 10: Split large funding gaps in DRC for AMF and in Nigeria for Malaria Consortium into separate cost-effectiveness estimates

Our CEAs generally provide country-level cost-effectiveness estimates.51 With this change, we have split up our cost-effectiveness modeling for AMF in DRC into two groups (the first ~$20 million of AMF's funding gap and the last ~$20 million) and we have split our model of Malaria Consortium in Nigeria into two groups (the first ~$15 million of the funding gap and the last ~$10 million).52 We have split our modeling of these large funding gaps in order to vary some of the inputs of our CEA between filling the first and last parts of each gap. These inputs are varied in change 11 and change 16 below.

Change 11: Edited adjustments for effects excluded from our core model for AMF

For each of our top charities, we use a core cost-effectiveness model along with a set of additional adjustments.53 We use one of these types of adjustments, "adjustments for effects excluded from our core model," to account for our best guesses of the impact of intervention-level factors we have not analyzed in depth.54

For AMF, we have added a new adjustment for an effect excluded from our core CEA ("Adjustment for program impact being to move distributions closer together") and updated our estimates for a previous adjustment ("Marginal funding goes to lower priority areas").

Adjustment for program impact being to move distributions closer together

Our CEA for AMF is structured to model the impact of LLIN distributions occurring at three-year intervals.55 Our underlying assumption for most countries where AMF works is that, without funding from AMF, distributions would still happen on three-year intervals but fewer LLINs would be distributed, leading to lower coverage levels.

Our understanding is that in DRC and Nigeria, additional funding for LLIN distributions may lead to reductions in the amount of time between distributions (e.g., moving from a four-year spacing to the recommended three-year spacing), rather than increasing coverage for campaigns that occur every three years.56 Without an additional adjustment, we expect that our CEA would overestimate cost-effectiveness of funding used to reduce the amount of time between distributions.57

To address this issue, we have added an "adjustment for program impact being to move distributions closer together" for AMF in DRC and Nigeria to our list of adjustments for effects excluded from our core model.58 The values for this adjustment are calculated in this spreadsheet.

For Nigeria, we estimate that it is 35% less cost-effective to shift bed net distributions from occurring every 3.75 years to every three years, compared to the cost-effectiveness of moving from no bed net distributions to distributions every three years.59 This calculation is based on estimating how many incremental bed net coverage years are gained by shifting from distributions every 3.75 years to distributions every three years, relying on assumptions on how quickly bed nets in Nigeria wear out after distribution.60

Using similar calculation methods for DRC, we estimate that it is 41% less cost-effective to decrease the amount of time between distributions than it would be to move from no bed net distributions to distributions every three years.61

Adjustment for marginal funding going to lower priority areas

One of the adjustments for effects excluded from our core AMF CEA is an adjustment for marginal funding for LLIN distributions being allocated to lower priority areas.62 If there is not enough funding available to fully reach entire populations with LLIN distributions, our guess is that some countries will make prioritization decisions to exclude some lower priority areas (e.g., areas with lower malaria prevalence), and that providing additional funding for LLIN distributions in effect allows these areas to be covered.63

As discussed above, our understanding is that in Nigeria and DRC governments respond to limited funding availability by increasing the spacing between LLIN distributions. Because the governments of Nigeria and DRC are using spacing between LLIN distributions to triage resources for malaria programs, our guess is that these countries implement less prioritization by geographic areas. We have accordingly reduced our adjustment for the impact of marginal funding going to lower priority areas in Nigeria and DRC.64

Change 12: Edited adjustments for effects excluded from our core model for Malaria Consortium

We also use an adjustment for marginal funding going to lower priority areas for Malaria Consortium (see our discussion of this adjustment for AMF above).65 We previously used a single estimate for this adjustment across all of the countries in which Malaria Consortium operates SMC programs, but we have now created country-specific adjustments.66 We describe our reasoning for each of our country-specific adjustments in our Malaria Consortium CEA spreadsheet.67

Change 13: Removed "Results" sheet

Our past CEAs have included a "Results" sheet, displaying country-specific and overall cost-effectiveness estimates for each charity.68 To simplify our presentation of cost-effectiveness estimates, we have removed the "Results" sheet from this version of the CEA—cost-effectiveness results remain available in each top charity's sheet and on the "Cost-effectiveness by funding gap" sheet.69

Change 14: Added Nigeria to our CEA of AMF

In early 2021, AMF told us that it could use additional funding to support LLIN distributions in a set of states in Nigeria where the President's Malaria Initiative (PMI) works.70 AMF expected that it would use additional funding to support LLIN distributions in Akwa Ibom and Bauchi State in 2021, and that there would be additional funding gaps for LLIN distributions in Nigeria in 2023.71

We have added a column to our CEA of AMF with cost-effectiveness estimates for Nigeria.72

Change 15: Updated pre-existing nets adjustment for AMF

We include an adjustment in our CEA of AMF to account for our expectation that some proportion of individuals receiving LLINs from AMF distributions already have functional LLINs.73 Our previous adjustment was based on rough guesses on the percentage of individuals with pre-existing nets and the proportion of "extra" nets that are eventually used.74

We have completed additional analysis in this spreadsheet to update our pre-existing nets adjustment.75 Our updated analysis includes separate adjustments for distributions on a three-year cycle, distributions moved from a four-year cycle to a three-year cycle with additional funding, and DRC.76

Change 16: Updated leverage and funging adjustments for AMF and Malaria Consortium

We estimate leverage and funging adjustments for our CEAs, as discussed above.

We updated our estimates of the probability that AMF's support of LLIN distributions in DRC crowds out funding from the Global Fund or PMI (see our calculations in this spreadsheet).77 We also updated our estimates of the probability that Malaria Consortium's support of SMC programs in Nigeria crowds out funding from the Global Fund or PMI (see our estimates in this spreadsheet).78

Change 17: Added adjustment for seasonal malaria chemoprevention (SMC) distributions with five cycles

Malaria Consortium's SMC programs typically deliver four "cycles" of SMC per year. Each cycle includes a four-day distribution period and lasts 28 days, after which a new cycle starts.79 In late 2020, we learned that Malaria Consortium planned to support a fifth SMC cycle per year going forward in some countries.80

Our best guess is that marginal spending on a fifth SMC cycle is less cost-effective than spending on the previous four cycles. We added an adjustment to our list of excluded effects to account for our best guess at the size of this effect.81

Change 18: January and February 2021 funding allocation

In January and February 2021, we granted or recommended grants of $71.8 million to our top charities. Our cost-effectiveness estimates for these grants are available in the "Cost-effectiveness by funding gap" sheet here. We describe our funding allocation in detail on this page.

Change 19: Remove "overall" cost-effectiveness estimates for AMF, Malaria Consortium, Helen Keller International, Sightsavers, END Fund, and Deworm the World

Our cost-effectiveness analyses for our top charities that operate in multiple countries have previously included country-specific cost-effectiveness estimates and overall cost-effectiveness estimates.82 These overall cost-effectiveness estimates are weighted averages calculated based on how much marginal funding we expected each top charity to spend by country.83

We have decided to remove these estimates for AMF, Deworm the World, END Fund, Sightsavers, Malaria Consortium, and Helen Keller International because they add additional complexity to our analyses and because we do not expect to use them in our funding allocation decisions—instead, we expect to focus our allocation decisions for these organizations on specific country-level funding gaps.84 We have not removed our overall cost-effectiveness estimate for SCI Foundation, because we expect to continue to use our overall cost-effectiveness estimate for SCI Foundation for allocation decisions going forward.85

In order to remove the overall cost-effectiveness estimates discussed above, we have also made the following additional changes:

  • Our leverage and funging adjustments (discussed above) rely on estimates of the cost-effectiveness of what governments and other organizations would spend funding on if they were not contributing to programs implemented by our top charities.86 Our estimate of the "counterfactual value of donated deworming drug costs" was previously calculated as being 10% of AMF's overall cost-effectiveness87 and would fluctuate as our estimate of AMF's overall cost-effectiveness increased or decreased. We now anchor our cost-effectiveness estimate for the counterfactual value of donated deworming drug costs at a fixed value equal to 10% of AMF's overall cost-effectiveness, as estimated in the previous published version of our CEA, 2020 GiveWell cost-effectiveness analysis — version 2.88
  • Some health programs for children may positively impact development, leading to lasting increases in their productivity and earnings in adulthood ("development effects").89 Our estimates of the development effects of Helen Keller International's and New Incentives' programs were previously based on our overall development effects estimate for Malaria Consortium's SMC programs90 and would fluctuate as this parameter increased or decreased. We now instead anchor our estimates of the development effects of Helen Keller International's and New Incentives' programs at a fixed value equal to our overall development effects estimate for Malaria Consortium's SMC programs, as estimated in the previous published version of our CEA, 2020 GiveWell cost-effectiveness analysis — version 2.91

Change 20: Update to "Cost-effectiveness by funding gap" sheet

We have updated the "Cost-effectiveness by funding gap" sheet to reflect our understanding of funding gaps remaining after taking our January and February 2021 funding allocation into account.92 We have also updated our notes on our prioritization of each remaining funding gap.93

  • 1

    See this 2020 changelog entry for a description of all three types of additional adjustments.

  • 2

    See the CEA version preceding this change here.

  • 3

    Compare highlighted rows and cell notes in the version of the CEA following this change here to the version preceding this change here.

  • 4

  • 5

    See the previous footnote for examples of how we used 2017 data from GBD and LBD.

  • 6
    • "Published in The Lancet in October 2020, GBD 2019 provides for the first time an independent estimation of population, for each of 204 countries and territories and the globe, using a standardized, replicable approach, as well as a comprehensive update on fertility and migration. GBD 2019 incorporates major data additions and improvements, and methodological refinements." GBD 2019 Resources page
    • Only a portion of data from LBD has been updated to include 2019 estimates. For example, its data on malaria includes 2019 estimates, but its data on tuberculosis does not include estimates past 2017.

  • 7
    • Compare the version of the CEA preceding this change here to the version following this change here.
    • Note that we continue to use 2017 data from LBD for our input on all-cause child mortality by state, as IHME has not yet released updated estimates (see LBD data on child mortality here).

  • 8

    See the version of the CEA following this change here.

  • 9

    See cells on Burkina Faso 2021, Chad 2022, and Nigeria 2022 in the "Assumptions" table in our 2020 room for more funding analysis for Malaria Consortium, "Source: 2020-2023 projections" sheet here.

  • 10

    See this section of our review of Malaria Consortium for more details.

  • 11

    See this spreadsheet for our calculation.

  • 12
    • See this spreadsheet for our updated calculation.
    • Compare the "Adjustments for effects excluded from our core model" section for Malaria Consortium in the version of the CEA preceding this change here to the version following this change here.

  • 13

    See the version of the CEA preceding this change here.

  • 14

    For example, see Malaria Consortium's room for more funding sheet in the version of the CEA preceding this change here.

  • 15

    See the version of our CEA following this change here.

  • 16
    • We create room for more funding analyses for each of our top charities and update them at least yearly, except for the END Fund (see here). Our full 2020 room for more funding analysis for Malaria Consortium, for example, can be found here.
    • We also publish what funding gaps we fill through the Top Charities Fund, which grants funds on a quarterly basis to the charity or charities we believe to have the highest priority funding needs at the time.

  • 17

    See Helen Keller's proposed budget and the number of children it plans to target in each of these states in this spreadsheet, sheet "Helen Keller budget."

  • 18

    See our 2021 GiveWell cost-effectiveness analysis — version 1 here.

  • 19
    • See our calculations in this spreadsheet, which includes Helen Keller's proposed budget for work in Nigeria from 2021 to 2023, as well as LBD data on mortality rates for children under five across Akwa Ibom, Cross River, Adamawa, and Taraba.
      • LBD only provides state-level mortality rates for children under 5, but our Helen Keller CEA relies on estimates of mortality rates for children aged 6-59 months. To reach our estimate of the weighted average mortality rate for children aged 6-59 months in the four states Helen Keller proposes to support, we assume that differences between national and state-level mortality rates for children 6 to 59 months are proportional to differences between national and state-level mortality rate estimates for children under 5. See our calculations in these rows.
    • As of June 2021, the most recent available state-level under-5 mortality rates available through LBD are from 2017. See here.
    • Compare the version of the CEA preceding this change here to the version following this change here.

  • 20
    • Compare the "Probability of scenarios in absence of philanthropic funding" section of the "AMF" sheet in the version of the CEA preceding this change here to the version following this change here.
    • Our full leverage and funging adjustments for AMF are calculated on a non-public spreadsheet. We have not made this spreadsheet publicly available because we have not received permission to publish country-specific cost estimates we received from the Global Fund, which our calculations rely on. See this section of our review of AMF for additional information.

  • 21

    Compare the "Probability of scenarios in absence of HKI's spending" section of the "Helen Keller International" sheet in the version of the CEA preceding this change here to the version following this change here.

  • 22

    Helen Keller has proposed supporting four regions in Cameroon between 2021 and 2023: Centre (Central), Littoral, Ouest (West), and Sud (South). See the number of children Helen Keller plans to target in each of these regions in this spreadsheet.

  • 23

    See the number of children Helen Keller plans to target in each of these regions in this spreadsheet.

  • 24
    • See our calculations in this spreadsheet.
    • Compare the version of the CEA preceding this change here to the version following this change here.

  • 25

    Compare the "RFMF_AMF," "RFMF_MC," "RFMF_NI," and "Cost-effectiveness by funding gap" sheets in the version of the CEA preceding this change here to the version following this change here.

  • 26

  • 27

    See this 2020 changelog entry for a description of all three types of additional adjustments.

  • 28
    • Compare the "Adjustments for excluded effects" section of the "AMF" sheet in the version of the CEA preceding this change here to the version following this change here.
    • Note that correcting this error changed our adjustments by less than 0.1 percentage points.

  • 29

    See highlighted sections of the version of the CEA following this change here.

  • 30
    • "Published in The Lancet in October 2020, GBD 2019 provides for the first time an independent estimation of population, for each of 204 countries and territories and the globe, using a standardized, replicable approach, as well as a comprehensive update on fertility and migration. GBD 2019 incorporates major data additions and improvements, and methodological refinements." GBD 2019 Resources page
    • Compare the version of the CEA preceding this change here to the version following this change here.

  • 31

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 32

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 33

    See this section of our review of AMF for additional information.

  • 34

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 35

    See details in this spreadsheet.

  • 36

    These changes did not have an impact on our cost-effectiveness estimates for Helen Keller International in Cameroon. Compare the version of the CEA preceding this change here to the version following this change here.

  • 37
    • Compare the version of the CEA preceding this change here to the version following this change here.
    • There are changes to two rows in our Helen Keller International CEA: row 33 ("Baseline deaths per 1,000 child-years for children aged 6- to 59-months") and row 37 ("External validity adjustment — VAS"). See links in bullet point above.
    • The "External validity adjustment — VAS" row changed because mortality rate estimates are an input into our external validity calculations for Helen Keller International. Compare the updated version of our external validity spreadsheet preceding this change here to the version following this change here.

  • 38
    • Compare the updated version of our external validity spreadsheet preceding this change here to the version following this change here. In both versions, we use GBD 2017 data on vitamin A deficiency prevalence (see "External validity by country" sheet).
    • Compare the external validity row of the CEA preceding this change here to the version following this change here.

  • 39

  • 40

    There is a small difference between the cost-effectiveness estimate in our original CEA (10.8 times as cost-effective as cash transfers) and the adapted version (10.7 times as cost-effective as cash transfers). This difference is caused by a change in our method for calculating the "Adjustment for all-cause mortality effect" between the two versions. In the original CEA we manually adjusted the all-cause mortality adjustment factor in order to find the adjustment that leads to an estimate of 0.5 deaths indirectly averted for each death directly averted (see this sheet). In the adapted version, we used a goal-seek function rather than making manual adjustments, which resulted in a slightly different adjustment factor (see here).

  • 41

    See our full report on insecticide resistance and malaria control here.

  • 42

    See this section of our report on insecticide resistance and malaria control for our review of the evidence.

  • 43

    Rob Mather, AMF CEO, email to GiveWell January 12, 2021 (unpublished)

  • 44
    • AMF has told us that the PBO nets it purchases are more expensive than standard LLINs. Rob Mather, AMF CEO, email to GiveWell January 12, 2021 (unpublished)
    • Compare the version of the CEA preceding this change here to the version following this change here.

  • 45

  • 46

    For a full discussion, see our Estimating Equivalent Coverage Years for Long-Lasting Insecticide-Treated Nets (LLINs) report.

  • 47

    See this section of our report on estimating effective coverage years for long-lasting insecticide-treated nets.

  • 48

    See the version of the CEA preceding this change here.

  • 49

    See this section of our 2019 review of AMF.

  • 50

  • 51

    See any of the top charity sheets in the version of the CEA preceding this change here.

  • 52

    Compare the AMF and Malaria Consortium sheets in the version of the CEA preceding this change here to the version following this change here.

  • 53

    See this 2020 changelog entry for a description of all three types of additional adjustments.

  • 54
    • See this 2020 changelog entry for a description of all three types of additional adjustments
    • See the "Adjustments for excluded effects" sheet in the version of the CEA following this change here.

  • 55

    See this row in the version of the CEA preceding this change.

  • 56
    • "The benefits of being able to commit now to funding nets that will be distributed in DRC in 2022 and 2023 are significant. First, it allows our co-funding partner, the Global Fund, not to scale back its allocation of funding in the first two years of the three year funding cycle. Without confirmation of our funding, reprogramming would be necessary, including scaling back of interventions and delayed distributions. This would be required to spread funds evenly and avoid a significant lack of funds for the final year of the funding cycle. Any resulting delays to distributions mean that households have to wait longer than three years to receive new nets. In the third year, nets wear out and so delays extend the period during which households are less well protected which can lead to malaria resurging." AMF agrees to fund a further 16.4 million nets for distribution in DRC in 2023
    • AMF told us in February 2021 that it expected that the impact of additional funding for LLIN distributions in Nigeria would be to reduce the amount of time between distributions from four years to three years. See this spreadsheet.

  • 57
    • Our CEA for AMF includes an estimate of the number of effective coverage years an LLIN provides (2.11 years for most countries and 1.74 years in DRC, relative to the coverage years of nets used in randomized controlled trials of bed nets). See here in the version of the CEA preceding this change.
    • This is an estimate of the average LLIN lifespan, but we expect that some LLINs would last for longer periods. In our spreadsheet modeling equivalent coverage years here, we estimate that the relative net coverage years in year following a distribution has decayed due to physical damage and insecticide loss, but it has not been driven down to zero. (We have not explicitly modeled the equivalent coverage years we would expect from remaining nets in the fourth year following a distribution.) See our report on estimating equivalent coverage years for LLINs for full details on our analysis.
    • Due to the remaining effectiveness of some LLINs after three years after a distribution, we expect that funding reducing the amount of time between distributions from four years to three years would have less of an impact on LLIN coverage (and be less cost-effective) in a particular area than funding that increased the number of LLINs distributed during three-year distribution cycles.

  • 58
    • See "Adjustment for effects excluded from our core model" section of our CEA of AMF in the version of the CEA following this change here.
    • We incorporate this adjustment for our CEA of AMF in Nigeria when we add it to our model in change 14.

  • 59

    See our calculation here

  • 60

    See our incremental cost per net coverage year calculations here and our net decay assumptions for Nigeria here.

  • 61

    See our DRC calculations on this sheet and our DRC net decay assumptions here.

  • 62

    See here in the version of the CEA preceding this change.

  • 63
    • See here in the version of the CEA preceding this change.
    • For further discussion of sub-national prioritization of malaria interventions, see this blog post.

  • 64
    • Compare the version of the CEA preceding this change here to the version following this change here
    • We incorporate this adjustment for our CEA of AMF in Nigeria when we add it to our model in change 14.

  • 65

    See here in the version of the CEA following this change.

  • 66

    Our previous estimate was a 3% reduction in cost-effectiveness and our updated country-specific estimates range from 5% to 15%. Compare the version of the CEA preceding this change here to the version following this change here.

  • 67

    See the version of our CEA following this change here.

  • 68

    See the version of the CEA preceding this change here.

  • 69

    See the version of our CEA following this change here.

  • 70

    See this spreadsheet.

  • 71

    See this spreadsheet.

  • 72

    See the version of the CEA following this change here.

  • 73

    See the version of the CEA preceding this change here.

  • 74

    See the version of the CEA preceding this change here.

  • 75

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 76

    See this cell for our three-year cycle adjustment, this cell for our adjustment moving distributions from a four-year cycle to a three-year cycle, and this cell for our DRC adjustment.

  • 77
    • Compare the version of the CEA preceding this change here to the version following this change here.
    • Note that, as mentioned above, we are not able to publish our full leverage and funging adjustments for AMF.

  • 78

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 79

    See this section of our review of Malaria Consortium for more details.

  • 80

    See cells on Burkina Faso 2021, Chad 2022, and Nigeria 2022 in the "Assumptions" table in our 2020 room for more funding analysis for Malaria Consortium, "Source: 2020-2023 projections" sheet here.

  • 81
    • Compare the "Adjustments for effects excluded from our core model" section for Malaria Consortium in the version of the CEA preceding this change here to the version following this change here.
    • See our calculations for this adjustment in this spreadsheet.

  • 82

    See the AMF, Deworm the World, END Fund, SCI Foundation, Sightsavers, Malaria Consortium, and Helen Keller International sheets in the version of the CEA preceding this change here.

  • 83

    See the AMF, Deworm the World, END Fund, SCI Foundation, Sightsavers, Malaria Consortium, and Helen Keller International sheets in the version of the CEA preceding this change here, "Percentage of funding to be allocated to each country with marginal donations" row.

  • 84

    See cost-effectiveness by country-level funding gaps in the version of the CEA following this change here. Also see that overall cost-effectiveness estimates have been removed from the sheets for AMF, Deworm the World, END Fund, Sightsavers, Malaria Consortium, and Helen Keller International.

  • 85

    See our cost-effectiveness estimate for SCI Foundation in the version of the CEA following this change .

  • 86

    See the "Counterfactual non-philanthropic" sheet in the version of the CEA following this change here. The estimates on the "Counterfactual non-philanthropic sheet" are used in the leverage and funging sections on each top charity's CEA sheet.

  • 87

    See here in the version of the CEA preceding this change.

  • 88

    See the "Counterfactual non-philanthropic" sheet in the version of the CEA following this change here.

  • 89

    For example, we discuss potential development impacts of mass deworming programs in this section of our intervention report.

  • 90

    See this section of our Helen Keller International sheet and this section of our New Incentives sheet in the version of the CEA preceding this change.

  • 91

    See this section of our sheet on New Incentives and this section of our sheet on Helen Keller International in the version of the CEA following this change.

  • 92

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 93

    Compare the version of the CEA preceding this change here to the version following this change here.

Source URL: https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/cost-effectiveness-models/changelog-2021