2023 Cost-Effectiveness Analysis Changelog

This page provides details about changes that were made to our cost-effectiveness analysis (CEA) in 2023. Each changelog entry represents our understanding at the time the change was made. For past versions of our CEA, see this page.

Version 4 — Published August 16, 2023

Link to the cost-effectiveness analysis (CEA) file: 2023 CEA — version 4

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Added remaining Nigerian states and updated probability of death estimates in our New Incentives CEA

To understand the potential cost-effectiveness of any further expansion of New Incentives' conditional cash transfers for childhood vaccinations program, we added columns for all remaining Nigerian states to our cost-effectiveness analysis. The CEA previously had cost-effectiveness estimates for 20 states, and now includes all 36 states and the Federal Capital Territory.

We also updated our estimates of the probability of death from vaccine-preventable diseases in each state. We previously calculated the probability of death based on 2019 data from the Institute for Health Metrics and Evaluation's Global Burden of Disease (GBD) project. We have updated our calculations based on preliminary 2021 GBD data.1 Concurrently, we updated our methodology for calculating vaccination rates in each state at the time the 2021 GBD data was collected. We now use state-level estimates of vaccination rates from Nigeria's Multiple Indicator Cluster Surveys (MICS).2 We aggregate vaccination coverage estimates for relevant vaccines into a single coverage estimate by weighting each estimate by that vaccine's contribution to the primary benefit of New Incentives' program (averting deaths of children under the age of five). As a result, coverage of vaccines that avert a greater proportion of under five mortality count for more in the aggregated coverage estimates.

As a result of these changes, our cost-effectiveness estimates decreased by 0.3% to 29% in 14 states and increased by 6% to 22% in 6 states.3

Change 2: Moved the "net spacing adjustment" to the "Number of people covered" section in the Against Malaria Foundation CEA

Our Against Malaria Foundation (AMF) CEA is structured to model the impact of increasing the coverage of long-lasting insecticide-treated nets (LLIN) distributions that occur at three-year intervals.4 Our underlying assumption for most countries where AMF works is that without funding from AMF, distributions would still happen on three-year intervals but fewer LLINs would be distributed, leading to fewer people receiving nets.

However, we think that additional funding for LLIN distributions in certain geographies may lead to reductions in the amount of time between distributions (e.g., moving from a four-year spacing to the recommended three-year spacing) rather than increasing coverage for campaigns that occur every three years. In these geographies, we include an "adjustment for program impact being to move distributions closer together” (or "net spacing adjustment") to account for the reduced cost-effectiveness of funding that decreases the amount of time between LLIN distributions rather than increasing LLIN coverage levels.5

We previously included the net spacing adjustment in the “supplemental intervention-level adjustments” section of the CEA, which is intended to account for additional benefits or downsides of the program beyond the mortality reductions and development effects we model explicitly.6 However, unlike the rest of the adjustments we include in this section, the net spacing adjustment is intended to adjust the mortality reduction benefits and development effects that we model in the CEA downward. As a result, we think its inclusion in this section resulted in an overestimate of the cost-effectiveness of LLIN distributions in geographies where the true effect of AMF funding is to reduce the time interval between LLIN distributions.

To resolve this issue, we moved the net spacing adjustment from the “supplemental intervention-level adjustments” section to the “Number of people covered” section of the CEA, where it is now a factor in our calculation of the total person-years of effective coverage provided by AMF's funding.7 This change reduced cost-effectiveness in our AMF and Malaria Consortium CEAs by 1% to 16% across all geographies.8

Change 3: Updated the counterfactual value of Global Fund spending

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.9

In order to estimate the impact that leverage and funging have on the cost-effectiveness of a program, we need to estimate how cost-effectively other actors would hypothetically spend their money if they didn't spend it on the program (i.e., the "counterfactual value" of their spending). We calculate these estimates on the "Value of counterfactual spending by other actors" sheet of our CEA.

We updated how we model the counterfactual value of spending by the Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund) based on the estimated cost-effectiveness of programs that we expect the Global Fund would likely support through reallocated funding.10 We have replaced our previous estimate of the counterfactual value of Global Fund spending (which we had estimated to be roughly 40% as cost-effective as AMF-funded LLINs in the Democratic Republic of the Congo (DRC)) with the estimate produced by this model.11 This change caused small (<1%) increases to our cost-effectiveness estimates of AMF’s and Malaria Consortium’s programs across all geographies.12

Change 4: Updated our methodology for calculating baseline vaccination coverage and treatment effect in the New Incentives CEA

Our previous CEA for New Incentives assumed that baseline vaccination coverage was the same across all Nigerian states and that the program increased vaccination rates by the same amount in each state. However, both New Incentives' baseline vaccine coverage assessments and national surveys of vaccination coverage in Nigeria show that vaccination rates vary widely by state.13 We expect the treatment effect of New Incentives' program (i.e., the improvement in vaccination coverage) to vary based on baseline coverage, and have updated our methodology to account for this variance.

Our new model estimates baseline vaccination coverage for each state based on data from New Incentives' baseline vaccine coverage assessments and the 2021 Multiple Indicator Cluster Survey for Nigeria. We estimate the treatment effect of New Incentives' program in each state using the reduction in the percentage of unvaccinated children recorded during the randomized control trial of the program. We assume a linear, negative relationship between treatment effect and baseline coverage (i.e., we assume the treatment effect decreases at higher rates of baseline coverage).14

As a result of these changes, our cost-effectiveness estimates decreased in all states except Sokoto (the only state to have a lower estimated baseline vaccine coverage and a higher estimated treatment effect in the new model than in our prior estimates). Our cost-effectiveness estimate increased by 46% in Sokoto, and decreased by 13% to 91% in all other states.15

Change 5: Updated the cost per infant estimate in our New Incentives CEA

We updated our cost per infant analysis for New Incentives to account for lower costs in recent years.16 This decrease in our cost per infant estimate (from $24.85 to $21.27) increased cost-effectiveness by 14% to 17% across all states.17

Version 3 — Published June 13, 2023

Link to the cost-effectiveness analysis (CEA) file: 2023 CEA — version 3

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Corrected errors in the leverage and funging calculations in our Sightsavers CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.18 We account for this possibility in our “leverage and funging adjustment” by calculating the “units of value” that would be created under different counterfactual scenarios without our charities’ philanthropic spending.

We recently noticed and corrected an error in our Sightsavers CEA in which these calculations referenced our initial cost-effectiveness estimates before accounting for supplemental charity-level and intervention-level adjustments.19 After correcting this error, we estimate the cost-effectiveness of Sightsavers’ program to be 1-3% higher across all geographies.20

Change 2: Updated the supplemental adjustment for medical costs averted by life-saving interventions

We include “supplemental intervention-level adjustments” in each of our top charity CEAs to estimate the impact of outcomes we haven't explicitly modeled on the cost-effectiveness of our top charities.21 In our AMF, Malaria Consortium, Helen Keller International, and New Incentives CEAs, we include an adjustment for “treatment costs averted from prevention” because we think it’s likely that reduced disease morbidity and mortality resulting from these interventions leads to reductions in medical costs borne by governments and recipient households.

We recently created a model to estimate medical costs averted that updates our estimate of the “treatment costs averted from prevention” from a rough guess of 6% to 20% across programs.22 We have updated this value in our CEAs of AMF's, Malaria Consortium's, Helen Keller International's, and New Incentives' programs. As a result, our cost-effectiveness estimates of those programs increased by roughly 5-15%.23

Change 3: Updated the leverage and funging adjustment in the Against Malaria Foundation (AMF) CEA for the Democratic Republic of the Congo

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.24 To account for this, we apply a “leverage and funging adjustment” to our cost-effectiveness estimates, based in part on the probability that other counterfactual funding scenarios would occur in the absence of our charities’ philanthropic spending.

In our AMF CEA, we estimate the proportion of Global Fund and/or U.S. President's Malaria Initiative (PMI) funding for long-lasting insecticide-treated net (LLIN) distributions that is crowded out by AMF spending.25 We base this estimate for the DRC on AMF’s stated funding gap for LLIN campaigns in 2024 relative to the amount of PMI funding that we estimate shifted away from net campaigns after AMF became involved in funding those campaigns in 2021 to 2023. This calculation suggests that a 2024 grant would be less likely to funge spending by other philanthropic actors than the previous funging estimate we were using for campaigns in 2021-2023. As a result, we have updated our estimate of the proportion of Global Fund/PMI funding that will be crowded out by AMF spending from 25% to 15%. This change increased our cost-effectiveness estimate of AMF's program in the DRC by 7%.26

Change 4: Updated the net spacing adjustment in the Against Malaria Foundation (AMF) CEA for the Democratic Republic of the Congo

We include “supplemental intervention-level adjustments” in each of our top charity CEAs to estimate the impact of outcomes we haven't explicitly modeled on the cost-effectiveness of our top charities.27

Our AMF CEA estimates the cost-effectiveness of philanthropic funding causing people to receive new LLINs who would otherwise not have received them. However, in situations where we believe the true impact of philanthropic funding is to allow distribution of LLINs to occur sooner than it would have otherwise, we apply a “net spacing adjustment” to account for the difference in cost-effectiveness between these two scenarios.28

Our previous net spacing adjustment for the DRC was based on a projection that the interval between province-level net campaigns in the 2021-23 period would be 30 months if AMF contributed funding and 39 months if AMF did not contribute funding. Using information about actual campaign intervals in the 2021-23 period and planned campaign intervals in the 2024-26 period, we updated our net spacing adjustment. The adjustment is now based on a projection that the interval between province-level net campaigns in the 2024-26 period would be 33 months if AMF contributed funding and 42 months if AMF did not contribute funding. Making this change reduced our previous 41% net spacing adjustment to 30%, which increased our cost-effectiveness estimate of AMF's program in the DRC by about 10%.29

Change 5: Updated cost per net and insecticide resistance adjustment for the Democratic Republic of the Congo in the Against Malaria Foundation (AMF) CEA

Updates to cost per net estimate

Our estimate of the total cost of long-lasting insecticide-treated nets (LLIN) distributed by AMF is based on AMF’s net purchase costs, organizational and logistical costs associated with LLIN campaigns that are borne by AMF and other philanthropic organizations (non-net costs), and in-kind government costs.30
We made several updates to our cost per net estimates:

• We recently received new data from AMF on the Global Fund’s “non-net costs” from 2018-2020 campaigns in AMF-supported regions in Togo and Uganda.31 In addition, a recently completed study of Malaria Consortium’s 2021 LLIN campaign in Ondo State, Nigeria in December 2021 produced an estimate of non-net costs that we think may be roughly representative of the non-net costs for AMF’s campaigns in other Nigerian states.32 We updated our cost per net calculations for all AMF-supported geographies to account for this new data.33
• AMF has also told us that, if it contributes funding to 2024 LLIN campaigns in the DRC, it plans to purchase and distribute a substantial number of nets treated with a second insecticide (chlorfenapyr), as well as a smaller number of other Dual AI nets.34 We updated our DRC cost per net calculation to account for the additional costs of chlorfenapyr nets.
• We updated our cost per net analysis to incorporate AMF's data on the total number of nets distributed in each campaign. The analysis now estimates the total cost per net distributed, rather than the cost per net purchased. Because this estimate accounts for expired or undistributed nets, we removed this supplemental adjustment for product wastage from the CEA.

As a result of these updates, our cost per LLIN estimates declined in all countries except the DRC, where the increased price of chlorfenapyr nets led to a slight increase.35

Update to insecticide resistance adjustment in the DRC

Because we expect that chlorfenapyr nets greatly reduce the risk of net ineffectiveness due to insecticide resistance, we also updated the parameters of our insecticide resistance adjustment to account for the distribution of these and other Dual AI nets in the DRC. After this update, we expect that insecticide resistance will only reduce net effectiveness in the DRC by 4%.36

These changes increased our cost-effectiveness estimates for AMF’s program from 4% to 26%.37

Change 6: Added Madagascar to the Helen Keller International CEA

As part of our investigation into a potential grant to Helen Keller International for vitamin A supplementation in Madagascar, we added Madagascar to our Helen Keller International CEA.38

Change 7: Fixed errors in the cost per life saved calculations for life-saving top charities

We updated our formulas used to calculate the cost per life saved estimates for life-saving top charities (AMF, Malaria Consortium, Helen Keller International, and New Incentives). Our previous formulas excluded some supplemental adjustments that impact our cost per life saved estimates. Fixing these errors lowered our cost per life saved estimates for Malaria Consortium's program in Nigeria and Helen Keller International's program in Madagascar. These changes did not impact our cost-effectiveness estimates (in multiples of cash transfers) for any programs, as the cost per life saved figures are calculated separately.39

Change 8: Updated our malaria mortality calculations and our estimates of the number of people covered by each distributed net in the AMF CEA

We updated our methodology for calculating the mortality reduction from net distributions and for estimating the number of people covered per net distributed. The key elements of this change were:

• Changing the way we model mortality from an intention-to-treat effect (the impact of being targeted for net distributions) to a treatment-on-the-treated effect (the impact of using a net). We now use the reduction in malaria incidence found in a 2018 meta-analysis to calculate the reduction in mortality from net distributions, rather than using the all-cause mortality reduction from that same meta-analysis.40
• Increasing our estimate of deaths indirectly averted by malaria from 0.5 to 0.75 in our AMF and Malaria Consortium CEAs.41
• Changing our definition of ‘people per net’ from the number of people living in recipient households per distributed net to the number of people sleeping under each net that actually gets used. We incorporated data from AMF's post-distribution monitoring surveys to estimate the average number of people sleeping under each AMF-distributed net in each country, rather than assuming this number to be constant across geographies.42

Together these changes increased our cost-effectiveness estimates by 4-27% across AMF geographies, and by 5-18% across Malaria Consortium geographies.43

Change 9: Split the Sightsavers CEA for the DRC into province-level columns with province-specific worm burden adjustments

We split our CEA for Sightsavers' deworming program in the DRC into individual provinces and estimated worm burden adjustments specific to each province. We also updated our leverage and funging adjustments to account for a 10% likelihood that another funder would support deworming in the DRC.44 As a result of these changes, our cost-effectiveness estimate for the program went from 1.0x cash to a range of 0.3-21.4x cash across provinces.45

Change 10: Fixed an error in our cost per life saved calculation in the New Incentives CEA

We corrected an error in our calculation of the cost per life saved in our New Incentives CEA. Our previous calculation mistakenly incorporated some values that did not represent costs of the program, in addition to estimated program costs, resulting in inflated cost per life saved figures.46 Correcting this error did not impact our cost-effectiveness estimates (in multiples of cash transfers) for New Incentives' program, as the cost per life saved figures are calculated separately.47

Change 11: Reconciled the population estimates for 1-59-month-olds in Nigeria in the AMF CEA

We resolved a discrepancy in the way we calculate state-level population estimates of 1-59-month-olds in Nigeria for inputs in our AMF CEA. We previously used two separate population estimates in our calculations of baseline malaria prevalence and malaria-attributable deaths averted in children targeted with seasonal malaria chemoprevention (SMC). To reconcile this discrepancy, we updated the population estimates in the malaria-attributable deaths averted calculation to match those used in the calculation of baseline malaria prevalence.48 This update slightly increased our estimate of the malaria-attributable deaths averted per 1,000 children per year targeted with SMC in PMI states49 from 1.1 to 1.2, and slightly decreased this estimate in Global Fund states from 0.6 to 0.5.

This change increased our cost-effectiveness estimate of AMF's program in Nigeria’s Global Fund states by about 1% and decreased our cost-effectiveness estimate in Nigeria’s PMI states by about 1%.50

Version 2 — Published April 3, 2023

Link to the cost-effectiveness analysis (CEA) file: 2023 CEA — version 2

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updating cost per child estimates in our Deworm the World CEA

Our Deworm the World cost per child analysis relies on program information on the costs incurred and the number of children treated for past campaigns. The previous version of our analysis was based on program information for the 2014-2020 period. We updated our analysis to incorporate program information from 2021.51 More information about our updated cost per child analysis is available here.52

Change 2: Updating cost per supplement estimates in our Helen Keller International CEA

We estimate the cost-effectiveness of Helen Keller International (Helen Keller)'s vitamin A supplementation (VAS) program using estimates of the average cost per child per supplementation round from previous campaigns. We updated our Helen Keller International cost per supplement analysis to incorporate program information from 2020 and 2021.53

Change 3: Updating the leverage/funging probabilities in our Helen Keller International CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.54 As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur without philanthropic support for a particular program as well as how cost-effectively funding by other actors would be spent if they were not contributing to the program.

Based on new information from Helen Keller International's 2022 room for more funding proposal, as well as conversations we've had with UNICEF and Global Affairs Canada, we updated our estimates of the likelihood that domestic governments or other philanthropic actors would fund the vitamin A supplementation (VAS) programs in our Helen Keller International (Helen Keller) CEA in Helen Keller's absence.55 We also increased our estimate of the counterfactual value of other philanthropic actors' spending for Burkina Faso, Cameroon, Côte d'Ivoire, DRC, and Guinea. Our understanding is that much of the funding UNICEF spends on VAS in these countries comes from Global Affairs Canada, which earmarks this funding for VAS. Therefore, we expect that UNICEF funding displaced by Helen Keller would likely be used to support VAS in other, potentially less cost-effective, locations. We very roughly guess that the counterfactual value of this spending would be equal to the counterfactual value of the Global Fund's spending.56 (More recently we updated this to a slightly less rough estimate, see below.) This is higher than the value we use for countries where we don't expect a large portion of UNICEF's spending to be earmarked for VAS.57

Change 4: Updating the external validity adjustments in our Helen Keller International CEA

We base our estimate of Helen Keller International (Helen Keller)'s vitamin A supplementation (VAS) program's effect on mortality among 6-59-month-olds on the findings of Imdad et al. 2017, a meta-analysis evaluating the impact of VAS on child mortality. We use an external validity parameter to capture differences in causes of mortality in populations targeted by Helen Keller's program relative to populations who participated in the trials included in Imdad et al. 2017.58 Vitamin A deficiency (VAD) rates are a key input into this external validity adjustment. Our previous analysis used VAD estimates from the 2017 results of the Institute for Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) as a key input for most programs.59 Because we're uncertain about the accuracy of IHME's model for producing VAD estimates, we decided to calculate new best-guess VAD rates for these programs that incorporate VAD prevalence rates found in national and regional surveys alongside the GBD 2017 estimates. 60 While most VAD surveys are outdated, we have found the survey-based VAD estimates to be more transparent than IHME's methodology. To create these estimates, we:

• Searched for existing vitamin A deficiency surveys from relevant countries. When multiple national surveys were available, we used the most recent survey. When no national VAD results were available, we searched for regional surveys representing portions of relevant countries. We found regional surveys only for Burkina Faso and Mali, which we weigh less highly than national surveys.
• Adjusted available survey results to account for improving circumstances. There are no VAD surveys available from the past few years, with the most recent one we found from 2010.61 We make a rough and uncertain assumption that improving circumstances in relevant countries will tend to lead to better nutritional outcomes and reduced rates of VAD over time and that these outdated VAD surveys likely overstate current rates of deficiency as a result. We assume that VAD rates fall by 1.33% per year based on the annualized rate we found in our more detailed improving circumstances analysis for Nigeria.62
• Created best-guess VAD rates that combine survey results with IHME estimates from GBD 2017. For countries where no survey results were available, we relied exclusively on GBD 2017 estimates. For countries where there were survey results, we averaged the VAD rates from the survey results (adjusted for improving circumstances since the time of the survey) and the VAD rates from GBD 2017. For countries where national survey results were available, we put equal weight on adjusted national survey estimates and GBD estimates.63 For countries where only regional surveys were available, we put a lower 25% weight on the survey results and 75% weight on GBD 2017 because we assume results from small regional surveys are less reliable than national results.

This change increased our estimates of VAD prevalence in some countries and decreased them in others, which doesn't suggest to us that IHME has systematically over- or under-estimated VAD prevalence in these countries.64 We remain uncertain about the VAD estimates we've arrived at with this new methodology and we may revisit these estimates when IHME publishes a new GBD model. We conducted a sensitivity analysis to investigate how sensitive our final cost-effectiveness estimates would be to changes in our VAD prevalence estimates and concluded that further work on these estimates would be unlikely to affect our short-term funding decisions.65

Change 5: Updating the quality of monitoring adjustment in our Deworm the World CEA

In our cost-effectiveness analyses, we make adjustments to account for how different charity-level factors affect our best guess of cost-effectiveness, including two adjustments that account for the quality of a charity's monitoring and evaluation. We revisited these parameters for our Deworm the World CEA after reviewing their recent monitoring information (see our review here).

The first of these adjustments ("misappropriation without monitoring results") is intended to account for the possibility that coverage results from the subset of programs for which monitoring data was collected is not representative of all programs in that geography which are supported by GiveWell. If this is the case, the monitoring data collected may over or understate program coverage, and we make a downward adjustment for the risk that it is overstated. We increased this adjustment from 1% to 5% for all Nigerian states. Deworm the World has generally provided monitoring from all deworming rounds in India, Kenya and Pakistan (with some exceptions because of disruption to their programs during the COVID-19 pandemic).66 The situation is different in Nigeria, where Deworm the World normally conducts a full coverage survey for only 1 of the 2 rounds of deworming it conducts per year (in states where it conducts 2 rounds of deworming).67 We therefore use a higher value for this adjustment in Nigeria than other countries.

The second adjustment ("false monitoring results") is intended to account for the possibility that the monitoring results we have seen are partially or fully fabricated or biased in a way that leads us to overestimate coverage. We increased this adjustment from 0% to 1% for all countries to account for two factors: (1) disruptions to Deworm the World's standard monitoring procedure caused by the COVID-19 pandemic in 2021, and (2) we have not reviewed Deworm the World's process monitoring updates since 2019.68 Our overall adjustment value remains low because we continue to believe that Deworm the World has a strong monitoring procedure.69

Overall, these two changes resulted in decreases to our cost-effectiveness estimates for Deworm the World's program across all countries.70

Change 6: Updating the quality of monitoring adjustment for our Sightsavers CEA

In our cost-effectiveness analyses, we make adjustments to account for how different charity-level factors affect our best guess of cost-effectiveness, including adjustments that account for the quality of a charity's monitoring and evaluation. We revisited these adjustments for our Sightsavers CEA after reviewing their recent monitoring information (see our recent review here). One of these adjustments, ("misappropriation without monitoring results") is intended to account for the possibility that coverage results from the subset of programs for which monitoring data was collected is not representative of all programs in that geography which are supported by GiveWell. If this is the case, the monitoring data collected may over or understate program coverage, and we make a downward adjustment for the risk that it is overstated. We decreased this adjustment from 10%
to 3% across all countries to reflect our updated view of increased comprehensiveness in Sightsavers' monitoring. Our previous higher adjustment accounted for a lack of monitoring in 2019.71 We have now seen monitoring results representing 80% of Sightsavers' spending on relevant programs since 2018.72 Furthermore, we have seen monitoring results representing nearly all of Sightsavers' spending on relevant programs for its Year 4 (2020-2021) and Year 5 (2021-2022) program years.73 Changing this adjustment led to an increase in our estimated cost-effectiveness for Sightsavers across all countries.74

Change 7: Splitting Nigeria into state-level columns in our Malaria Consortium CEA

Our cost-effectiveness analysis previously calculated a single cost-effectiveness estimate for Malaria Consortium's seasonal malaria chemoprevention (SMC) program for several states in Nigeria, as well as separate estimates for two states, the Federal Capital Territory (FCT) and Oyo.75 We have now updated our CEA to calculate separate cost-effectiveness estimates for each Nigerian state in which Malaria Consortium supports SMC using GiveWell-directed funding.76 To do so, we updated malaria mortality and prevalence parameters using state-level estimates rather than national-level estimates.77 Our new state-level cost-effectiveness estimates for Nigeria are here.

Change 8: Fixing the malaria seasonality parameter for FCT and Oyo (Nigeria) in our Malaria Consortium CEA

We corrected an error we had made in estimating malaria seasonality for the Federal Capital Territory (FCT) and Oyo states in Nigeria in our CEA for Malaria Consortium's seasonal malaria chemoprevention (SMC) program. SMC is delivered in monthly cycles during the season of the year when malaria transmission is high. While a full SMC round has typically comprised four monthly cycles, some locations may choose to deliver more or fewer cycles, in line with the length of their high-transmission season. We include a parameter in our CEA that represents the proportion of annual malaria mortality that occurs during the SMC round and therefore can potentially be averted by SMC. Because rainfall data78 suggests that the high-transmission season for malaria is longer in FCT and Oyo than in other SMC-eligible areas, we previously assumed that this meant that a lower proportion of annual malaria mortality occurs during the SMC round and adjusted our value for this parameter to 60%, down from our default value of 70%.79 We now believe this was a mistake because FCT and Oyo deliver five cycles of SMC instead of the usual four, and so these states have a longer SMC round than other locations in our model.80 We updated our CEA to use the default value of 70% for FCT and Oyo, under the assumption that a similar proportion of annual malaria mortality occurs during the SMC round in FCT and Oyo as in other locations in our model, but stretched over a period of five months instead of four.81 We continue to make this same assumption and use the same 70% value for the other locations in our model that deliver five cycles of SMC, including Burkina Faso and several other Nigerian states, which are also areas with longer seasons of high transmission.82

Change 9: Updating the leverage and funging adjustments for our Malaria Consortium CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.83
As part of an investigation into a grant to renew our support for seasonal malaria chemoprevention (SMC) in Burkina Faso, Chad, Nigeria, and Togo, we revisited our estimates for the likelihood that GiveWell-directed funding would crowd out funding from the other major funders of SMC in those countries. See this section of our grant page for a discussion of the information we took into consideration, which modestly lowered our estimates—and therefore increased our cost-effectiveness estimates—for each country.84

Change 10: Updating the cost per SMC cycle estimates for Malaria Consortium

We estimate the cost-effectiveness of Malaria Consortium's seasonal malaria chemoprevention (SMC) programs using estimates of the average cost per cycle of SMC administered from previous campaigns. We updated our Malaria Consortium cost per SMC cycle analysis to incorporate program information on costs and coverage from 2021 for Burkina Faso, Chad and Nigeria (see here). We also made several methodological changes to the analysis, including:

• Updating our calculations to use data from only more recent years (2018 onwards). We had previously used data from 2015 onwards,85 but we decided to make this change because we expect the cost per SMC cycle from more recent program years to be more indicative of the cost per SMC cycle that Malaria Consortium will achieve in future program years.86
• Updating our adherence adjustment (to account for some children not swallowing all three doses of the SMC drugs) with more recently available data.87
• Not putting any weight on projected future budgets for two Nigerian states (FCT and Oyo) where we had previously relied partly on data from previous SMC campaigns in other states in Nigeria and partly on Malaria Consortium’s future budgets. Following this update, all our Nigeria estimates are now based on previous campaign data rather than projections.88 This means that our FCT and Oyo estimates are based on campaign data from other states, since campaigns in FCT and Oyo started in 2022 and data was not yet available at the time of this update.89

Because Malaria Consortium's SMC program in Togo is co-funded by the Global Fund and UNICEF but we do not have information on the costs paid by these actors, we do not calculate a separate cost per SMC cycle estimate for Togo, but rather use a weighted average of our estimates for Burkina Faso, Chad, and Nigeria.90 This weighted average for Togo also changed due to the calculation adjustments described above.

See the impact of these changes on our cost-effectiveness estimates here.

Change 11: Updating the adjustment for marginal funding going to lower priority areas in our Malaria Consortium CEA

In our cost-effectiveness analyses for our top charities, we adjust for the likelihood that when a program is already receiving substantial funds from other sources, additional funds may be more likely used to cover lower-priority areas within the program's target area.91 As part of splitting Nigeria into state-level columns for Malaria Consortium's SMC program (see change 7 above), we set the values we used for this adjustment to zero because GiveWell is the only major funder of SMC in those states.92

Change 12: Updating the supplemental charity-level adjustments for our Malaria Consortium CEA

In our cost-effectiveness analyses, we include an adjustment to account for the possibility that a charity may use funding we direct to them to support research or other work that they see as being related to the program we intended to fund, but that we don't find valuable ("change of priorities"). We are not aware of Malaria Consortium using GiveWell-directed funds in this way in recent years. In general, we believe that we are highly aligned with Malaria Consortium on how it plans to use GiveWell-directed funding, and it has consistently requested our feedback on opportunities it is seeing to redirect funding to a different purpose or on decisions it is making on how to use its discretionary research budget. We have reduced this adjustment from 2% to 1% in our CEA of Malaria Consortium's seasonal malaria chemoprevention program.93

Change 13: Updating cost per child dewormed estimates for our Sightsavers CEA

We estimate the cost-effectiveness of Sightsavers' deworming program using estimates of the average cost per child dewormed per year from previous campaigns and/or cost per child projections based on forward-looking budgets and treatment plans. To generate these estimates, we take into account program information on the costs of the program and the number of children reached. We updated our Sightsavers' cost per child analysis.94 For the Democratic Republic of Congo (DRC), Cameroon, Guinea, Guinea-Bissau, and some Nigerian states,95 we updated our cost per child estimates based on historical program data from 2019-2021. For Chad, Senegal, and some other Nigerian states,96 we calculated cost per child using projections from forward-looking budgets and, in some cases, averages of past program data. For specifics on each country, see the cell notes here.97

Change 14: Updating the worm burden adjustment for our Sightsavers CEA

Our recommendation of mass deworming programs primarily relies on a series of follow-up studies to the experiment described in Miguel and Kremer 2004.98 We apply a "worm burden" adjustment to our cost-effectiveness analyses of deworming programs to account for differences in the prevalence and intensity of worm infections between the population studied in Miguel and Kremer 2004 and populations reached by the deworming programs we consider funding.

See a more detailed explanation of our worm burden adjustment here.

We updated our worm burden adjustments for Sightsavers' deworming program in Chad based on new information about which subnational areas the program would cover. We also noticed that we had been using two different methods for imputing infection intensity for our worm burden adjustments in different Nigerian states (exact intensity figures are not available for most states in Nigeria). We have now updated our methodology to be consistent across states.99 See how these updated worm burden adjustments impacted our cost-effectiveness estimates here.

Change 15: Updating the counterfactual value of philanthropic funding for our Helen Keller International CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.100 Our leverage and funging adjustments are informed by our estimates of how cost-effectively funding by governments and other philanthropic actors would be spent if they were not contributing to these programs. In our Helen Keller International CEA, we updated our estimate of the counterfactual value of other actors' philanthropic spending on vitamin A supplementation (VAS) in Burkina Faso, Cameroon, Côte d'Ivoire, DRC, and Guinea.101 We had previously linked our estimate of the counterfactual value of other philanthropic actors' spending in these countries to our estimate of the counterfactual value of Global Fund spending, which was in turn tied to our cost-effectiveness estimate for an Against Malaria Foundation (AMF)-supported long-lasting insecticide-treated net distribution in DRC.102 We decided to come up with a new method for estimating this value so that our cost-effectiveness estimates for Helen Keller would be more accurate and well-reasoned. Our new estimate is based on the assumption that there is a 50% chance that other philanthropic actors' spending on VAS would be counterfactually used to fund VAS in other countries versus non-VAS programs.103

Change 16: Adding a decay adjustment to long-term income benefits for our deworming CEAs

In response to criticism of our deworming cost-effectiveness analyses by the Happier Lives Institute, we revisited our method for modeling the long-term income benefits of deworming programs. We added an adjustment to these benefits to account for the possibility that the benefits of deworming could decline over time. This is a rough adjustment that incorporates both our prior assumption that the effects of the program remain constant over time and evidence from three long-term follow-ups104 to the RCT Miguel and Kremer 2004 that could be interpreted as suggesting decaying benefits over time.105 See our full write-up on this topic here. This change decreased our cost-effectiveness estimates for deworming programs by 10-12% across the board.106

Change 17: Updating development effects references for our Helen Keller International and New Incentives CEAs

In our cost-effectiveness analyses for Helen Keller International's vitamin A supplementation program and New Incentives' conditional cash transfers for childhood vaccinations program, we include an estimate of development effects, which are the long-term effects of the program on income/consumption.107 For both programs, we don't have direct information on development effects, so we estimate the effects based on the magnitude of the development effects we've modeled for SMC.108 We previously estimated the development effects for these programs by adjusting a benchmark of SMC in Nigeria, because it had roughly average cost-effectiveness among locations where Malaria Consortium implements SMC programs.109 However, we have since updated our model to separate out Nigerian states,110 and Nigeria no longer represents roughly average cost-effectiveness, making it less practical to use as a benchmark.111 We now model development benefits for Helen Keller International's and New Incentives' programs based on the average development benefits of Malaria Consortium's SMC programs overall.112

Change 18: Adding province-level estimates for Pakistan to our Deworm the World CEA

We previously calculated a cost-effectiveness estimate for Deworm the World's program in Pakistan using one overall estimate.

See the version of the CEA prior to this change here.

We split our Deworm the World cost-effectiveness analysis for Pakistan into province-level columns, because we have now investigated worm burden data at the province level and concluded that worm burden differs significantly across provinces.113

Change 19: Updating the supplemental charity-level adjustments in the Helen Keller International CEA

We revisited several of the supplemental charity-level adjustments in our Helen Keller International (Helen Keller) CEA as part of our investigation into a potential renewal grant and made several changes.

We adjust our cost-effectiveness estimates for vitamin A supplementation (VAS) to account for the possibility that children may have already received VAS from another source. We increased this adjustment from 15% to 25%, to account for new information suggesting that VAS coverage through routine delivery systems may be relatively high for 6-12 month-old babies, who may receive vitamin A supplementation when they visit clinics for routine vaccinations.114 We also updated our adjustment that accounts for the possibility that Helen Keller may use funding we direct to them to support research or other work that they see as being related to the program we intended to fund, but that we don't find valuable ("change of priorities"). We reduced this adjustment from 5% to 0% because, over the last couple of years, Helen Keller has consistently used GiveWell grants to support VAS campaigns and associated research aimed at improving VAS delivery.

We also revisited our adjustments that account for the quality of Helen Keller's monitoring and evaluation. The first of these adjustments is intended to account for the possibility that coverage results from the subset of programs for which monitoring data was collected may overstate program coverage overall ("misappropriation without monitoring results"). We updated this adjustment from 6% to 15% because the monitoring results we've seen are not fully comprehensive of Helen Keller's campaigns in recent years, and we have some concerns about the representativeness of these results.115 The second adjustment is intended to account for the possibility that the monitoring results we have seen are partially or fully fabricated or biased in a way that leads us to overestimate coverage ("false monitoring results"). We updated this adjustment from 3% to 2% to account for our increased confidence in coverage survey results after seeing the results of auditing procedures for surveys conducted after three recent campaigns.116

Change 20: Fixing an error in our worm burden adjustment for Sightsavers' program in West, Cameroon

Our recommendation of mass deworming programs primarily relies on a series of follow-up studies to the experiment described in Miguel and Kremer 2004.117 We apply a "worm burden" adjustment to our cost-effectiveness analyses of deworming programs to account for differences in the prevalence and intensity of worm infections between the population studied in Miguel and Kremer 2004 and populations reached by the deworming programs we consider funding.118 We found a copy/paste error in our worm burden adjustment for Sightsavers' program in West, Cameroon. Correcting this error led to a 40% decrease in our cost-effectiveness estimate for the program.119

Version 1 — Published January 13, 2023

Link to the cost-effectiveness analysis (CEA) file: 2023 CEA — version 1

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Added additional Nigerian states to the New Incentives CEA and separated out cost-effectiveness estimates by state.

Our CEA previously calculated a single cost-effectiveness estimate for New Incentives' conditional cash transfer program in Nigeria by averaging together vaccination and disease burden estimates for the three states New Incentives originally operated in: Jigawa, Katsina, and Zamfara. New Incentives has now expanded its program into areas within a couple other Nigerian states120 and is planning to expand to even more states in the future.121 In order to more accurately estimate the cost-effectiveness of New Incentives' program in different areas, we have now updated our CEA to calculate separate cost-effectiveness estimates for each state and incorporated additional vaccination and disease burden data specific to current and potential expansion states.122

Change 2: Updated the counterfactual vaccination coverage rate in the New Incentives CEA

In our CEA, we model the benefits of New Incentives' program in terms of "counterfactually vaccinated" infants, meaning we only count the benefits to vaccinated infants who wouldn't have been vaccinated in the program's absence. In order to make this calculation, we estimate the proportion of infants enrolled in New Incentives' program who would have been vaccinated whether the program existed or not. How we set this estimate can have a significant effect on cost-effectiveness, since the costs of vaccinating these infants are still included in the CEA, even though the benefits aren't.

In the previous version of the CEA, we based this estimate on the endline vaccination rate for the BCG vaccine among the control group of the randomized controlled trial (RCT) of New Incentives' program. After adjusting for self-reporting bias, the results of the RCT imply that nearly half (48%) of enrolled infants would have received BCG vaccinations in the absence of the program.123

New Incentives has begun conducting rapid assessments of baseline vaccination coverage before expanding to new areas, and we have now seen the results of the earliest of these rapid assessments. After adjusting for self-reporting bias, the rapid assessments imply lower average baseline BCG vaccination coverage (34%) than the results of the RCT.124 We expect vaccination coverage in New Incentives' current areas of operation and future expansion areas to be more similar to the areas assessed in the rapid assessments than the areas where the RCT was conducted. However, we also think data from the RCT is likely to be higher quality than data from the rapid assessments.125 We have accordingly decided to put 35% weight on the results of the RCT and 65% weight on the results of the rapid assessments, resulting in a final counterfactual vaccination coverage estimate of 39%.126

Change 3: Updated the cost per infant estimates in the New Incentives CEA

We updated our cost per infant analysis for New Incentives to incorporate program data from September 2021 to July 2022. This resulted in a slight decrease to our estimate of New Incentives' cost per enrolled infant.127

We also added an adjustment to our estimates of spending incurred by the Nigerian government and Gavi, an international organization that primarily supports vaccination programs in low-income countries, to account for some vaccination costs being fixed costs of the immunization platform. We very roughly guess that fixed costs make up about 30% of total costs and have accordingly reduced our estimates of the government's and Gavi's costs per counterfactually vaccinated infant by 30%.128 We are highly uncertain about the appropriate value for this adjustment and may investigate this question further in the future.

Change 4: Updated the proportion of PBO nets purchased for South Sudan in the cost per net and insecticide resistance analyses for the Against Malaria Foundation (AMF) CEA

For some distributions of long-lasting insecticide-treated nets (LLINs), a portion of the nets AMF purchases are treated with a piperonyl butoxide (PBO) synergist in addition to standard insecticide (pyrethroid). These "PBO nets" are more expensive than standard nets, but there is evidence that PBO improves the effectiveness of nets in areas where a significant proportion of mosquitoes are resistant to standard insecticide.

In a previous changelog entry, we described our work creating a CEA for LLIN distributions in South Sudan, including our work estimating an appropriate insecticide resistance adjustment for this country. Based on the data we'd seen on insecticide resistance rates in South Sudan and our initial understanding of AMF's plans, we initially expected that all of the nets purchased by AMF for a distribution in South Sudan would be PBO nets. This assumption increased our cost per net estimate for the program and decreased the size of our insecticide resistance adjustment, which estimates the reduction in LLIN efficacy caused by local insecticide resistance.129

However, we have since learned that about 80% of the LLINs needed for the upcoming 2023 distributions in South Sudan are standard nets that were ordered prior to AMF's involvement in planning the campaign. AMF has told us that it will backfill the funding for those standard nets and that it plans to purchase PBO nets to make up the remaining 20% of nets needed for the campaign. We have updated our cost per net estimate and insecticide resistance adjustment in our South Sudan CEA to incorporate the lower proportion of PBO nets. We also incorporated some additional cost information we received from AMF into our cost per net estimate. These changes decreased our cost per net estimate for South Sudan slightly and increased our insecticide resistance adjustment for South Sudan substantially, leading to an overall decrease in cost-effectiveness.130

Change 5: Updated the leverage and funging and campaign spacing adjustments in the Against Malaria Foundation (AMF) CEA for South Sudan

We learned additional information about the funding landscape for LLIN campaigns in South Sudan and how AMF's contributions will impact those campaigns. This led us to update two adjustments in our AMF CEA for South Sudan: our leverage and funging adjustment and our adjustment for the impact of the program being to reduce the amount of time between LLIN distributions.

Leverage and funging

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur in the absence of philanthropic support for a particular program (e.g., "government costs would replace philanthropic costs" or "distributions would go unfunded"). In our AMF CEA, one of the scenarios we include is that the Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund) or the President's Malaria Initiative (PMI) would fund the program in the absence of AMF's support. We had previously estimated that there was a 30% chance the Global Fund would replace philanthropic costs for a distribution in South Sudan (PMI does not provide funding to South Sudan). We have since gotten feedback from AMF that, given the time-sensitivity of this funding gap, it was very unlikely that the Global Fund would reallocate funding in time to fill it. Based on this feedback, we have decreased our estimate to 15%.131

Campaign spacing

For most countries, our AMF CEA estimates the cost-effectiveness of philanthropic funding causing people to receive new LLINs who otherwise would not have received LLINs. In situations where we believe the true impact of philanthropic funding is to allow a distribution to occur sooner than it would have otherwise, we apply an adjustment to account for the difference in cost-effectiveness between these two scenarios. Based on information we received from AMF and conversations we had with other stakeholders, we expect that the impact of providing funding for LLIN campaigns in South Sudan would be to allow them to take place an average of 10 months sooner than they would have otherwise, moving them from an average interval of 45 months between campaigns to an average of 35 months between campaigns.132 We estimate that this is 37% less cost-effective than the scenario modeled in the unadjusted AMF CEA,133 and have accordingly added a 37% downward adjustment to the CEA.134

Change 6: Updated the counterfactual value of spending by Gavi in the New Incentives CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

In order to estimate the impact that leverage and funging have on the cost-effectiveness of a program, we need to estimate how cost-effectively other actors would hypothetically spend their money if they didn't spend it on the program, which we refer to as the "counterfactual value" of their spending. One of the key benefits we model in our CEA of New Incentives' conditional cash transfers for vaccinations program is leveraging funding from Gavi, an international organization that primarily supports vaccination programs in low-income countries. We previously assumed that the counterfactual value of Gavi's spending is equal to the counterfactual value of the Global Fund's spending, which we estimate to be about 38% as cost-effective as an Against Malaria Foundation (AMF)-funded LLIN distribution in the DRC.135

Conceptually, this estimate was based on the idea that our estimate of the counterfactual value of Gavi's spending should approximate the cost-effectiveness of other programs Gavi supports, assuming that Gavi funds that would be leveraged by New Incentives' program would otherwise be spent on other programs within Gavi's portfolio. However, after investigating this issue in more depth, we concluded that Gavi has consistently been able to raise sufficient funding to cover its entire portfolio of programs, and we expect it will continue to be able to do so in the future. Therefore, it makes more sense to conceptualize the counterfactual value of Gavi's spending as how cost-effectively Gavi's donors, primarily the Bill and Melinda Gates Foundation and high-income country governments, would otherwise spend the funds that would be leveraged by New Incentives' program.

Upon investigating the cost-effectiveness of spending by Gavi's donors, we revised our estimate of the counterfactual value of Gavi's spending downward by more than half, from 0.0167 units of value per dollar to 0.007 units of value per dollar, which increased the overall cost-effectiveness of New Incentives' program.136

Change 7: Added ten additional Nigerian states to the New Incentives CEA

We learned that New Incentives is considering expanding its conditional cash transfer program to ten additional Nigerian states. We have added those states to our New Incentives CEA.137

• 1

  We do not have permission to publish the preliminary 2021 GBD data that our new calculations rely on. We plan to publish our supplemental calculations once the 2021 GBD data is made public. You can see the previous version of our supplemental calculations here.

• 2

  MICS is a nationally representative survey of Nigeria developed by UNICEF that includes a module on childhood vaccinations. In order to estimate vaccination rates for the cohort of children who were under the age of five at the time the 2021 GBD data was collected, we take the average of vaccination rates from the 2017 MICS and the 2021 MICS.

  "The Multiple Indicator Cluster Survey (MICS) was carried out in 2021 by the National Bureau of Statistics (NBS) as part of the Global MICS Programme. . . . The Global MICS Programme was developed by UNICEF in the 1990s as an international multi-purpose household survey programme to support countries in collecting internationally comparable data on a wide range of indicators on the situation of children and women." National Bureau of Statistics (NBS) and United Nations Children’s Fund (UNICEF), MICS 2021, Survey Findings Report, Nigeria, p. ii.

• 3
  • See the impact these changes had on our cost-effectiveness estimates here. The impact on cost-effectiveness is only recorded for the 20 states for which we previously had cost-effectiveness estimates, though the CEA now includes all Nigerian states.
  • See the version of the CEA preceding these changes here and the version of the CEA following these changes here.

• 4

  See the "Equivalent coverage-years for an LLIN over a 36-month distribution" row in our AMF CEA.

• 5
  • Cost-effectiveness is reduced because instead of providing three years’ worth of new LLIN coverage to each person reached, AMF is replacing a shorter period's worth of usage of an LLIN that provides partial protection with that period's worth of usage of a new LLIN. In other words, reducing the time interval between distributions doesn't generate as much additional LLIN coverage as causing a distribution to reach people who otherwise wouldn't have been reached.
  • We currently apply this adjustment to our Democratic Republic of the Congo (DRC), Nigeria, and South Sudan estimates. See our 2021 CEA changelog and 2023 CEA Version 1 changelog for more details.

• 6

  See the "Adjustment for program impact being to move distributions closer together" row's location in the previous version of the CEA here.

• 7

  See the "Adjustment for program impact being to move distributions closer together" row's new location in the CEA here.

• 8
  • See the full impact these changes had on our cost-effectiveness estimates for AMF here and for Malaria Consortium here.
  • These decreases were driven by a reduction in total units of value generated in countries where we apply this adjustment. See the "Total units of value generated" row of our AMF CEA before this change here and after this change here (compare values in the "DRC," "Nigeria (PMI states)," and "South Sudan" columns).
  • At the time we made this change, our estimate of the counterfactual value of Global Fund spending (see Change 3) was linked to our cost-effectiveness estimate of AMF in DRC. As a result, counterfactual units of value created by the program also decreased across all AMF and Malaria Consortium geographies. See the "Overall" row in the "Leverage/Funging Adjustment" section of our AMF CEA before and after this change, and the same row in our Malaria Consortium CEA before and after this change.
  • This change also led to slight increases in our Sightsavers, Deworm the World, and Unlimit Health deworming cost-effectiveness estimates because the counterfactual value of donated drug costs is also linked to the cost-effectiveness of AMF in DRC. See the "Counterfactual value of donated deworming drug costs" row of our CEA preceding this change here and after this change here. See here for the full impact these changes had on our cost-effectiveness estimates for deworming programs.

• 9

  For a full introduction to our approach to leverage and funging adjustments, see this blog post.
  

• 10
  • See our model of the estimated counterfactual value of Global Fund spending here.
  • Our estimate of the cost-effectiveness of reallocated Global Fund spending is based on data the Global Fund shared with us from its unfilled quality demand (UQD) registry, which is a list of programming that it would like to fund but does not have the resources to when it makes its initial allocations. Over the course of the Global Fund grant cycle, a sizable portion of its initially allocated funding is reallocated to support UQD programs. We do not have permission to make the data the Global Fund shared with us public.

• 11
  • We previously estimated the counterfactual value of the Global Fund's spending to be 38% of Against Malaria Foundation (AMF)-funded LLIN distributions in DRC based on these estimates.
  • See the "Counterfactual value of Global Fund spending" row of the "Value of counterfactual spending by other actors" sheet in the CEA preceding this change here and the same row following this change here.

• 12

  See the impacts of this change on our cost-effectiveness estimates for AMF here and for Malaria Consortium here.

• 13

  See the baseline vaccine coverage estimates from New Incentives' surveys here and from the 2021 Multiple Indicator Cluster Survey here.

• 14

  See our New Incentives vaccine coverage and treatment effects write-up here for a discussion of our new methodology.

• 15
  • See the impact these changes had on our cost-effectiveness estimates here.
  • See the version of the New Incentives CEA preceding these changes here and the version of the CEA following these changes here.

• 16

  See our most recent New Incentives cost per infant immunized calculations here. See the previous version of these calculations here.

• 17
  • See the "Cost of the program per enrolled infant, New Incentives" row in our New Incentives CEA preceding these changes here and the following these changes here.
  • See the impact these changes had on our cost-effectiveness estimates here.

• 18

  For a full introduction to our approach to leverage and funging adjustments, see this blog post.

• 19

  See the updated calculations here, which now incorporate supplemental charity-level and intervention-level adjustments.

• 20

  See the version of the CEA preceding this change here and the version of the CEA following this change here. See the impact this change had on our cost-effectiveness estimates here.

• 21

  See this sheet for details on how we calculate these adjustments.

• 22

  For more information about the model, see this write-up.

• 23

  See the version of the CEA preceding this change here and the version of the CEA following this change here. See the impact this change had on our cost-effectiveness estimates here.

• 24

  For a full introduction to our approach to leverage and funging adjustments, see this blog post.
  

• 25

  See this row in our AMF CEA.
  

• 26

  See the version of the CEA preceding this change here and the version of the CEA following this change here. See the impact this change had on our cost-effectiveness estimate here.
  

• 27

  See this sheet for details on how we calculate these adjustments.

• 28

  Our 2021 changelog details the application of this adjustment to our cost-effectiveness calculations for the DRC and Nigeria. We also began applying it to our South Sudan calculations in 2023.

• 29

  See the version of the CEA preceding this change here and the version of the CEA following this change here. See the impact this change had on our cost-effectiveness estimate here.

• 30

  For more information about how we calculate cost per net, see here.
  

• 31

  Data provided by AMF is not publicly available.
  

• 32

  While the results are not yet public, more information about the study can be found here.
  

• 33

  GiveWell's 2023 analysis of AMF's cost per net is only accessible to GiveWell staff because we do not have permission to report costs covered by the Global Fund—see our review of AMF for details.
  

• 34
  • AMF, DRC 2024-2026 Estimated Distribution Schedule, 2023 (unpublished)
  • Dual AI nets are LLINs containing two distinct insecticides. Current evidence suggests that Dual AI nets containing chlorfenapyr may be much more effective at preventing malaria than standard LLINs and LLINs treated with the synergist piperonyl butoxide (PBO) in areas where a large proportion of mosquitoes are resistant to commonly used insecticides.
  • “Malaria infection prevalence at 24 months was 549 (45·8%) of 1199 children in the pyrethroid-only reference group, 472 (37·5%) of 1258 in the pyriproxyfen group (adjusted odds ratio 0·79 [95% CI 0·54-1·17], p=0·2354), 512 (40·7%) of 1259 in the piperonyl butoxide group (0·99 [0·67-1·45], p=0·9607), and 326 [25·6%] of 1272 in the chlorfenapyr group (0·45 [0·30-0·67], p=0·0001)...After 2 years, chlorfenapyr LLINs provided significantly better protection than pyrethroid-only LLINs against malaria in an area with pyrethroid-resistant mosquitoes.” Mosha et al. 2022, pp. 1-2.

• 35

  See our new cost per net estimates here, and our previous estimates here.
  

• 36

  Our updated insecticide resistance adjustment calculations can be found here. For more about insecticide resistance, see this page.
  

• 37

  See the version of the CEA preceding this change here and the version of the CEA following this change here. See the impact this change had on our cost-effectiveness estimate here.
  

• 38

  See the version of the CEA following this change here.
  

• 39

  See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 40

  A 2018 Cochrane review of the effect of insecticide-treated nets on child mortality found that "Insecticide‐treated nets reduce child mortality from all causes by 17% compared to no nets," and that "[i]nsecticide-treated nets also reduce the incidence of uncomplicated episodes of Plasmodium falciparum malaria by almost a half (rate ratio 0.55, 95% CI 0.48 to 0.64; 5 trials, 35,551 participants, high‐certainty evidence)[.]" Pryce et al. 2018, Abstract.

• 41

  We're highly uncertain about the exact value of this input, but we have spoken with malaria experts who told us that it is widely accepted there are roughly 0.5-1 indirect malaria deaths for every direct malaria death. We use a value of 0.75, the midpoint of this range. See this cell note for more information.

• 42

  See our calculations of the average number of people sleeping under each net here. Our previous CEA assumed a constant 1.8 people covered per net distributed.

• 43

  See the impact these changes had on our cost-effectiveness estimates for AMF's programs here, and for Malaria Consortium's programs here.

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 44

  See this cell note for more information.
  

• 45

  See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 46

  For example, the previous CEA version calculated the cost per life saved of New Incentives' program in Adamawa, Nigeria to be $5,335, whereas the new version calculates a cost per life saved of $5,017. This amounts to a ~6% decrease in the cost per life saved.

  ($5,017 - $5,335) / $5,335 = -0.0596
  

• 47

  See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 48

  We believe the estimate used in the baseline malaria prevalence calculation is likely more accurate because it is based on state-level GBD estimates we received from the Institute for Health Metrics and Evaluation (IHME).
  

• 49

  PMI states are states that receive funding from the U.S. President's Malaria Initiative. Global Fund states receive funding from the Global Fund. See the funding landscape for distributions of insecticide-treated nets in Nigeria on page 9 of this report.
  

• 50

  See the version of the CEA preceding this change here and the version of the CEA following this change here. See the impact this change had on our cost-effectiveness estimates here.

• 51

  See the previous version of our cost per child analysis here and the updated version of this analysis here.

• 52

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 53

  See the previous version of our cost per supplement analysis here and the updated version of this analysis here. See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 54For a full introduction to our approach to leverage and funging adjustments, see this blog post.
  
• 55 We decreased our estimate of the likelihood of funging for most countries. We did not change our estimates for Cote d'Ivoire or DRC. For Cameroon, we increased the funging probability from 25% to 33%.
  
• 56See here.
  
• 57We estimate a lower counterfactual value for countries where the funding isn't earmarked, because we think VAS is a more effective use of funds than other programs the funding would likely be spent on.
  
• 58

  See how these external validity adjustments are used in our cost-effectiveness model here.
  

• 59

  See our previous analysis here. We created separate best-guess VAD rates for Nigerian states and Kenya that incorporated information from country surveys.

• 60

  We had been using VAD estimates from GBD 2017 because IHME made a significant methodological change between its GBD 2017 and GBD 2019 models that caused VAD prevalence estimates to significantly decrease. We do not fully understand the reasons for this decrease and have therefore continued to rely on GBD 2017. However, we are also uncertain about the methodology behind the GBD 2017 estimates, and we think it's likely that they are based on outdated survey data.

• 61

  See the years of the most recent VAD survey in each country here.

• 62

  See here.

• 63

  For example, see equal weighting for Cameroon here.

• 64

  See the impact of this change on our cost-effectiveness estimates here.

• 65

  See our sensitivity analysis here.

• 66

  See details of which rounds of monitoring took place in the "Results" sheets of this spreadsheet.

• 67

  "In Nigerian states, coverage validation is only conducted once a year, either in the first or second round." Deworm the World, responses to GiveWell's questions, November 2022 (unpublished).

• 68

  See our review of Deworm the World here for more discussion of their process monitoring.

• 69

  More details in this cell note.

• 70

  See the version of the CEA before this update here, and after this update here.

• 71

  See our previous cell note here.
  

• 72

  See here, Sightsavers coverage surveys [2022], sheet "Comprehensiveness".

• 73

  See here, Sightsavers coverage surveys [2022], sheet "Comprehensiveness".

• 74

  See here, 2023 V1 to 2023 V2 CEA change tracker, sheet "Deworming".

• 75

  See here for our previous overall Nigeria estimate and here for our FCT and Oyo estimates.

• 76

  We created estimates for the following states: Kebbi, Sokoto, Bauchi, Nasarawa, Borno, Kogi, and Plateau. See here.

• 77

  State-level data is in this spreadsheet.

• 78

  See here.

• 79

  See here.

• 80

  See here in our CEA, row "average number of SMC cycles per year".

• 81

  The previous version before this update is here; see this updated parameter here. This change increased our cost-effectiveness estimate of Malaria Consortium's program in FCT and Oyo, Nigeria by about 15%.

• 82

  See here for Burkina Faso, for example, where some areas deliver 5 cycles and some areas deliver 4 cycles. See here for average number of cycles in each location.

• 83

  For a full introduction to our approach to leverage and funging adjustments, see this blog post.
  

• 84

  See the impact of this change on our cost-effectiveness estimates here.

• 85

  See our previous analysis here.

• 86

  See our note here.

• 87

  See here for our updated calculations and assumptions.

• 88

  Our current analysis has one estimate for Nigeria here; our previous analysis had separate estimates for FCT and Oyo here.

• 89

  "In October 2021, GiveWell recommended that Open Philanthropy grant $15.9 million to Malaria Consortium, which we expect will enable them to support seasonal malaria chemoprevention (SMC) in Nigeria in the Federal Capital Territory (FCT) in 2022-2024 and in Oyo state in 2022." GiveWell, Malaria Consortium — Support for SMC in FCT and Oyo States, Nigeria (October 2021).

• 90

  See here.

• 91

  See the "Marginal funding goes to lower priority areas" parameter here.

• 92

  See the version before this update here and after this update here.

• 93

  See the impact of this change on our CEA here.

• 94

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 95

  Kebbi, Kogi, Kwara, Sokoto, Benue, Yobe, and Taraba states. See here.

• 96

  Kaduna, Zamfara, Niger, Kano, Adamawa, Katsina states. See here.

• 97

  See years that we pull source data from here.

• 98

  For further discussion, see:

  • This section of our intervention report on mass deworming programs
  • The following posts on the GiveWell blog: Why I mostly believe in Worms and How thin the reed? Generalizing from "Worms at Work"

• 99

  See the worm burden adjustments before this update here and after this update here.

• 100

  For a full introduction to our approach to leverage and funging adjustments, see this blog post.

• 101

  See the value before this change here and after this change here.

• 102

  In our previous calculations for the value of Global Funds spending here, we pulled from the AMF CEA.

• 103

  See our updated calculations here, which feed into our parameter for the counterfactual value of other philanthropic actors supporting VAS campaigns here.

• 104

  The main piece of evidence we use for the long-term effects of deworming is an RCT in Kenya with follow-ups at ~10 years (KLPS-2), ~15 years (KLPS-3) and ~20 years (KLPS-4) after children received deworming treatment. The RCT is Hamory et al. 2021.

• 105

  See the adjustment for benefits decaying over time here. Our updated analysis also shows the present value of lifetime benefits from a year of deworming both before and after the adjustment.

• 106

  See the impact of this change on our cost-effectiveness estimates here.

• 107

  See our estimate of the development effects of New Incentives here and a write-up about how we estimate VAS development effects here.

• 108

  See our note about estimating VAS development effects and our development effects estimation model for New Incentives.

• 109

  See our cell note here in the version of our CEA before this update.

• 110See Change 7 above.

• 111

  See cost-effectiveness estimates for SMC here.

• 112

  See our new method for estimating development effects for Helen Keller here and for New Incentives here.

• 113

  See our province-level worm burden adjustments here.

• 114

  Vitamin A supplementation may be delivered through both routine (facility-based) systems and mass campaigns. We have heard of cases where coverage may be relatively high for 6-12 month-olds who may receive vitamin A supplementation when they visit clinics for routine vaccinations. This rough value is attempting to account both for (a) the possibility that some children receive vitamin A through campaigns supported by Helen Keller International when they recently received it through visits to clinics, and (b) that the children most likely to receive double treatment are the youngest in the age range (because 6-12 month-olds are more likely to visit clinics as part of receiving routine vaccinations) and this may be a group where vitamin A supplementation has a higher than average impact on mortality (since mortality tends to be highest in the youngest children). Examples:

  • Helen Keller International notes, "Vitamin A supplements in Kenya are administered to children aged 6-59 months through various delivery approaches. Throughout the year, children can access VAS in primary health care facilities, but this routine coverage only accounts for around 20 percent of children, essentially children below 12 months, as many caregivers do not bring their children to the health facilities after the end of the immunization contact points at one year of age. Helen Keller International, Room for More Funding Report, 2021, p. 21.
  • The 2021 Demographic and Health Survey in Madagascar reports (translated by Google), "Among last-born children aged 6–23 months living with their mother, 40% have received vitamin A supplements in the last 6 months [...] Between 2008–09 and 2021, the percentage of children aged 6–59 months who received vitamin A supplements decreased significantly, from 72% to 40%. This is mainly due to the change in mode of distribution of vitamin A, which went from 'campaign' mode to routine mode since 2021." National Institute of Statistics, Antananarivo, Madagascar, Demographic and Health Survey in Madagascar, 2021, p. 224.
  • In 2022, Helen Keller conducted coverage surveys in parts of three countries where there was no external support for VAS campaigns and where Helen Keller may support in the future. These were locations that GiveWell had not yet funded because they were thought to have stronger routine delivery of VAS—they are therefore not representative of all countries supported by Helen Keller. Coverage found in these surveys was 42%, 46%, and 53%. Helen Keller International, email to GiveWell, January 9, 2023 (unpublished).

  Our understanding from Helen Keller is that most of the countries it works in to deliver VAS through campaigns have particularly weak health systems, so may have lower rates of health facility-delivered VAS than the examples above.
  

• 115

  For example, we are concerned that 1) Helen Keller does a survey for only one of the two rounds of VAS each year, and surveys are more frequently occurring during the non-rainy season, when coverage rates may be substantially lower during the rainy season, and 2) some decision-makers for campaigns may know in advance whether or not a coverage survey will be conducted for a campaign and may know which districts will be surveyed. Both of these factors could bias coverage results upwards. See more details in the cell note here.

• 116

  We have reviewed the findings of coverage survey audits for Helen Keller's campaigns in Kasai Oriental, DRC in 2022, Guinea in 2021, and Mali in 2021. These audits found high correspondence between the coverage results reported by initial surveyors and supervisors revisiting surveyed households.

• 117

  For further discussion, see:

  • This section of our intervention report on mass deworming programs
  • The following posts on the GiveWell blog: Why I mostly believe in Worms and How thin the reed? Generalizing from "Worms at Work"

• 118

  See a more detailed explanation of our worm burden adjustment here.

• 119

  See the version before this change here, and after this change here. See the change in cost-effectiveness here.

• 120
  • We recommended grants to fund expansions of New Incentives' program into new areas in August 2021, January 2022, and May 2022. New Incentives has told us that some of the expansion areas funded by the May 2022 grant are located in the states of Bauchi and Sokoto.
  • “This means we can now start operations in Sokoto once the baseline rapid assessments are completed (assuming coverage rates are low). We have submitted a draft MoU for expansion and collaboration to Bauchi State and have requested some members of the Bauchi State Health Research Ethics Committee (BASHREC) to review our draft application for rapid assessments. While we will continue to expand within current states and start expansion in Sokoto, going to Bauchi should help us minimize the proportion of LGAs with a likelihood of network shutdown or security issues, and also help us reduce vaccine supply and stakeholder risks since Bauchi is in the North East Zone of Nigeria. Bauchi was selected after reviewing states for low immunization rates, high Under-5 mortality (IHME), security, stakeholder support, measles incidence (2016), and other factors.” New Incentives, Program update to GiveWell, December 2021 [unpublished].

• 121

  We recommended a grant in November 2022 that New Incentives expects to use to expand its program into six additional states in Nigeria. More information on this grant will be published in a forthcoming grant page.
  

• 122

  See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 123

  See the version of the CEA preceding this change here.
  

• 124

  See the results of the rapid assessments and our calculations here.
  

• 125

  For more detail, see this write-up.
  

• 126

  See our calculations here. See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 127

  See the previous version of our cost per infant analysis here and the updated version of this analysis here.
  

• 128

  See our calculations here. See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 129

  We had previously set the cost per net estimate in South Sudan equal to the cost per net estimate in DRC, partially on the basis of our expectation that AMF would purchase a similarly large proportion of PBO nets for campaigns in South Sudan as in DRC. See our previous cost per net estimate for South Sudan here. See the previous version of our insecticide resistance adjustment calculations here.
  

• 130

  See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 131

  See the version of the CEA preceding this change here and the version of the CEA following this change here.
  

• 132

  See more detail in this spreadsheet

• 133

  See our calculations here.

• 134

  See the version of the CEA preceding this change here and the version of the CEA following this change here.

• 135

  See the version of the CEA preceding this change here.
  

• 136

  See our full write-up on the counterfactual value of Gavi spending here and our calculations here. See the version of the CEA following this change here.
  

• 137

  See the version of the CEA preceding this change here and the version of the CEA following this change here.