How We Produce Impact Estimates

This page shares our general approach to creating impact estimates for the charities and funding opportunities we recommend and walks through the decisions and judgment calls behind those estimates.

Published: November 2021; Last updated: February 2024
(July 2023 version, April 2023 version, July 2022 version, November 2021 version)

Table of Contents

General philosophy

When we’re communicating impact, we strive to adhere to the following principles:

  • Be accurate. Don’t misrepresent or imply that a donation will accomplish more (or less) than we expect it to.
  • Be clear. To the extent possible, make straightforward claims that are easy to understand.
  • Don’t imply a false level of precision. While models may provide estimates down to the dollar, round them so they don’t suggest we believe in that level of specificity.
  • Share the work with those who want it. Make the detail behind impact estimates available for anyone who wants it.

How we generate impact estimates

Impact metrics for grants to GiveWell’s top charities

We use the inputs below on our top charities page and other estimates of impact. You can see more detail in this spreadsheet.

Intervention Top charities implementing this intervention Outcome Cost of outcome (2022 average, rounded up to nearest $500) Output Cost of output (2022 average, rounded to nearest dollar)
Malaria nets Against Malaria Foundation Lives saved $5,500
(2021 average)1 		Number of nets delivered to households $5
(2021 average)2
Malaria medication Malaria Consortium Lives saved $5,000 Number of children treated with a full course of medicine $7
Incentives for immunizations New Incentives Lives saved $5,000 Number of infants vaccinated who would not have been vaccinated without the program $155
Vitamin A supplementation Helen Keller International Lives saved $5,000 Number of children receiving supplements $2

For life-saving charities, we present two cost-effectiveness estimates: one based on the cost of each output and one based on the cost of each outcome. Outcomes refer to the ultimate impact of the donation, such as saving a life, that results from the outputs. Outputs refer to the measurable activities a donation generates, such as treating a child with a full course of malaria medication, which in turn lead to the outcomes.

Our cost-per-output estimates are based on financial and programmatic information from our top charities. When estimating total outputs attributed to a specific amount of funding (e.g., how many nets will $100 deliver?), we divide the total funding by our estimate of how much it costs to deliver that output (e.g., how much it costs to deliver an insecticide-treated net).

For both outcome and output estimates, we use a weighted average of the cost-effectiveness for funds we directed to specific funding opportunities in the most recent calendar year. We use a weighted average because both cost per output and cost per outcome vary among programs we recommend funding for. For example, we made both of the grants below to Malaria Consortium’s seasonal malaria chemoprevention program in March 2020:

  • $2.8M to distribute malaria medication in Chad at a cost of $8 per child treated with a full course of medicine, resulting in an estimated cost-effectiveness of $8,000 per life saved
  • $3.4M to distribute malaria medication in Burkina Faso at a cost of $6 per child treated with a full course of medicine, resulting in an estimated cost-effectiveness of $2,000 per life saved

Cost-per-life-saved estimates have been rounded up to the nearest $500, and cost per output estimates have been rounded to the nearest dollar. Our impact estimates include a weighted average of all grants we made to each program. You can see a list of the specific funding opportunities used in these calculations by looking at grants approved in February 2022 through January 2023 in this spreadsheet. We categorize grants by year according to the date we decided to make or recommend them, not the date funds were disbursed. Our impact numbers don’t include all funds we direct to our top charities. More detail on this below.

Lifetime impact of GiveWell-directed donations to top charities

From 2009 to 2022, GiveWell directed a total of $1.01 billion to our current top charities.3 We estimate that this funding will save a total of about 215,000 lives by:

  • Distributing 67.2 million insecticide-treated nets to households
  • Treating 60.0 million children with a full course of anti-malarial medicine
  • Providing vitamin A supplements to 104.5 million children
  • Vaccinating 690,000 infants who wouldn't have been vaccinated otherwise

To estimate the overall impact of GiveWell-directed donations to top charities, we make a couple of assumptions.

  • We assume funds directed to top charities that are difficult to attribute to specific funding opportunities are about as cost-effective as the average dollar directed to each charity in 2021 (see table here).4 We make this simplifying assumption because we estimate the cost-effectiveness of donations at the funding opportunity level (see below), and it would be difficult to guess how cost-effective these types of donations will be without knowing which specific programs they will be spent on. We make the same assumption for grants to top charities for which we didn't create cost-effectiveness analyses. This includes grants to fund our top charities' activities beyond standard program implementation, such as scoping grants. We can think of reasons to guess that these types of funds may tend to be more or less cost-effective than the average donation.5 In the absence of more information, we expect them to be roughly equally cost-effective overall.
  • We assume funds directed to top charities in 2009-2019 were about as cost-effective as the average dollar directed to each charity in 2021 (see table here).6 We make this simplifying assumption because we have not retroactively calculated the impact of individual grants from before 2020. Because we think it's likely that the cost-effectiveness of our top charities' programs has been decreasing over time, we believe this is a conservative assumption that may underestimate the true impact of GiveWell-influenced donations.

What funding does GiveWell look at to calculate cost-effectiveness?

Funds directed to our top charities by GiveWell can be grouped in five buckets:

  1. Grants we make from our Top Charities Fund and All Grants Fund to specific funding opportunities
  2. Grants we make from GiveWell's pool of unrestricted funding. See our excess assets policy
  3. Grants we recommend to the Open Philanthropy Project
  4. Donations we recommend large individual donors make to specific funding opportunities
  5. Donations that donors make to an organization we recommend, but that aren’t directed to a specific funding opportunity. This includes donations made to charities on our checkout page, donations made directly to the organizations, and donations through other third-party organizations that share GiveWell’s recommendations (e.g., One for the World).

We calculate the cost-effectiveness of specific funding opportunities, not the cost-effectiveness of all the funding an organization uses. Because of this, we are only using buckets 1 through 4 as the basis for calculating the cost-effectiveness numbers we’re using in our impact estimates. This constitutes the majority of the total funding we direct to each top charity.

We exclude the funding in bucket 5 from our impact calculations because we don’t have estimates of the cost-effectiveness of those donations. We only calculate the cost-effectiveness for specific funding opportunities. Funding directed to our top charities could be either more or less cost-effective than the funding directed to specific funding gaps. Rather than guessing at a number and using it for our calculations, we’ve excluded this funding.

We also exclude grants for which we don’t generate a cost-effectiveness estimate. For example, we do not estimate the cost-effectiveness of grants for monitoring surveys to assess the success of programs. These improve our decision-making ability, but don’t have a direct impact.

We do include these types of funding in our lifetime impact estimates (see above).

What are funding opportunities?

GiveWell makes its decisions about where to recommend funding based on estimates of the cost-effectiveness of specific "funding opportunities" rather than the entire portfolio of a charity's work.

A funding opportunity is generally defined as a specific intervention (e.g., seasonal malaria chemoprevention) in a specific country, although funding opportunities may also be defined at the subnational level.

Our cost-effectiveness estimates can differ between funding opportunities. For example, in 2021, we estimated that Malaria Consortium's seasonal malaria chemoprevention program in Burkina Faso was roughly four times as cost-effective as its work in Chad, largely driven by higher rates of malaria mortality in Burkina Faso.

Why does GiveWell calculate cost-effectiveness at the funding opportunity level?

We analyze individual funding opportunities rather than charities because we believe it allows us to better distinguish between different uses of marginal funding. If we directed funding based solely on a single charity-level cost-effectiveness estimate, our analysis would result in recommending funding to only one charity.

We believe that would be misleading and would lead to less cost-effective grant-making. By estimating cost-effectiveness at a more granular level, we are better able to recommend funding to multiple charities and direct funding to the most cost-effective opportunities on the margin. We use these estimates to decide how to allocate funding from the Top Charities Fund.

As well as estimating the cost-effectiveness of specific funding opportunities, we also present aggregate cost-effectiveness estimates for each charity on our top charities page.

How are numbers rounded?

You can see a list of the specific funding opportunities used in these calculations here.

The cost-effectiveness estimates we share for specific funding opportunities don’t exactly match the numbers we’ve shared in the table above. We’ve rounded the numbers to avoid giving a false sense of precision.

  • We’ve rounded cost per life saved estimates up to the nearest $500.
  • We’ve rounded cost per output numbers to the nearest dollar (up or down).

For example, you can see in the linked sheet that the cost-effectiveness of our life-saving charities can vary by hundreds of dollars. We don’t think our models are precise enough to actually differentiate between differences in cost-effectiveness of only a few hundred dollars, so we’ve rounded estimates up to the nearest $500 to better convey this.

How often does GiveWell update impact numbers?

We update our impact estimates annually to incorporate grants from the preceding metrics year (our metrics year runs from February 1 to January 31).

Does GiveWell recalculate past cost-effectiveness estimates regularly?

We finalize our cost-effectiveness estimate for any given funding opportunity shortly before we make the grant to that opportunity (or recommend it to a third party). We don’t go back and update these estimates when our cost-effectiveness model changes. We think it would impose a large administrative burden without substantially improving our decisions for upcoming grants.

This means that we may reach a different estimate of cost-effectiveness for some past grants if we were to re-evaluate them now. You can see more information on changes we've made to our cost-effectiveness analyses here.

Guidance for interpreting impact estimates

Why is the cost to save a life so high?

We often get questions about why we estimate it costs thousands of dollars to save a life if, for example, it only costs approximately $5 to distribute a malaria net, or $7 to distribute antimalarial medicine, or $2 to provide a vitamin A supplement—some of the interventions we recommend to save lives. It's not intuitive!

Here, we'll briefly walk through why our estimate of the cost to save a life is higher than our estimate of the cost per treatment or item. We'll use the example of the Against Malaria Foundation (AMF), a GiveWell top charity.

AMF supports the distribution of insecticide-treated nets to prevent malaria. These nets are hung over sleeping spaces and block and kill malaria-carrying mosquitoes. We estimate that it costs about $5 to purchase and distribute one of these nets.7

Not every person who receives a net from an AMF-supported distribution would have otherwise died of malaria. This is why the cost per life saved is not $5.

In addition, although the nets are highly effective, the nets do not prevent malaria 100% of the time. They wear out over time, developing holes that reduce their effectiveness.8 In addition, mosquitoes have developed some resistance to the insecticide that the nets are treated with, also reducing the nets' effectiveness.9 In addition to factors related to the nets themselves, some people will choose not to use their nets, will not use them consistently, or may use them incorrectly (for example, leaving gaps between the net and their sleeping space).10 People can also be bitten by infected mosquitoes when they are not in bed.11 We adjust our estimate of nets' effectiveness against malaria to account for these factors.

We also incorporate the local malaria mortality rates where AMF works. While nets are distributed to people who are at risk of malaria, even without nets, a fairly small portion of them would die due to the disease. Many would, by chance, avoid infection. Many who do get infected would survive the infection with treatment or without. We estimate the cost per life saved for each location based on the likelihood that a person who receives a net would have died from malaria without the net.

In addition, we include whether we believe that other actors would distribute nets if AMF didn't, so that we can estimate AMF's true impact—and thus our best guess of the true impact of donations to AMF.

See here for a step-by-step walkthrough using a specific AMF distribution in Guinea.

What are marginal funding and marginal impact?

Marginal funding is the last funding that goes to a program. It’s important because the impact of marginal funding is often different from the average impact of funding to a program.

For example, imagine that the Against Malaria Foundation is considering funding the distribution of insecticide-treated nets in Nigeria. They have identified two areas as excellent candidates for distribution:

  • They could distribute $1 million worth of nets in areas with the highest rates of malaria. This would save 200 lives at the cost of $5,000 per life saved.
  • They could distribute an additional $2 million worth of nets in areas with lower rates of malaria. There is less malaria to prevent, so it costs a bit more to save lives. This distribution will save 250 lives at the cost of $8,000 per life saved.

The average cost to save a life with this program if fully funded with $3 million would be about $6,700 per life saved.

But that wouldn’t be the impact of your donation. Your donation would go to one area or the other. If AMF had already raised $1 million before you gave, they would spend it in the areas with the highest malaria prevalence. If you gave after that, your donation would go to the areas with lower malaria prevalence where it cost $8,000 to save a life.

We call this last donation marginal funding. The marginal impact is what your donation actually causes to happen.

Rather than trying to estimate the marginal impact of each dollar, GiveWell typically estimates the marginal impact of giving to a charity’s funding opportunities. These typically refer to a pool of funding directed to a charity's activities in a single country or subnational region. We estimate the cost-effectiveness of each funding opportunity, and regularly reassess what marginal donations will accomplish based on which funding opportunities a charity will spend on next.

What is funging, and how does it affect impact?

When you give, you impact how GiveWell allocates funding from other donors. So even when you give to a specific charity, the impact of your donation may be felt elsewhere based on how GiveWell ultimately allocates its full pool of funding. We call this funging (from fungibility).

For example, imagine that GiveWell has identified two excellent funding opportunities:

  • An opportunity with Helen Keller International is the most cost-effective, and needs $5M to distribute vitamin A supplements.
  • An opportunity with Against Malaria Foundation (AMF) is the next most cost-effective, and needs $20M to distribute malaria nets.

GiveWell has $15 million available to direct wherever it deems best. You decide to donate $50 to Helen Keller International vitamin A supplements.

Because GiveWell will fully fund the most cost-effective opportunity first, your donation effectively moves $50 of GiveWell’s dollars from Helen Keller International to AMF. GiveWell doesn’t have to give that $50 to Helen Keller International — you’ve already provided it.

In 2022, GiveWell granted (or recommended grants) totaling over $400 million to specific funding opportunities that GiveWell chose. With that much funding, many donors who choose to give to a specific charity will effectively have their donation increase the budget of a different organization.

Note that there are different kinds of funging. The above only refers to how donations directed to an individual top charity displace other GiveWell-directed funds. Our estimates also attempt to incorporate the possibility that our top charities displace other funds for an intervention from other sources (e.g., governments).

Why does GiveWell share the impact of grants rather than donations?

Sharing the impact of donations is challenging because individual donations to particular charities can be funged by GiveWell (see above). Because we don’t know the ultimate impact that a given donation will cause through funging, any estimate we shared based on the immediate recipient of the donation would likely be inaccurate.

To avoid sharing potentially misleading estimates, we share the impact of grants from the pool of funding we direct to funding opportunities. Because grants don't displace other GiveWell-directed funding, this allows us to share estimates we’re more confident in. It also gives donors the estimate we think they’re looking for — the impact a certain amount of funding has with that program (without taking funging by GiveWell into account).

Why does GiveWell show past cost-effectiveness rather than projected cost-effectiveness?

Because we finalize the cost-effectiveness estimates of funding opportunities at the time we recommend a grant, we don’t have estimates for the cost-effectiveness of future grants. Our best guess is that they’ll be similar to past grants, but will become less cost-effective over time. This may happen quickly if GiveWell sees a rapid increase in funding we direct, as we will fully fund the most cost-effective opportunities first.

We believe sharing the direct past estimated impact gives donors a useful proxy for what their impact may be without making potentially misleading statements about the future.

An idealized version of the top charities page would present an aggregated (charity-level) cost-effectiveness estimate based on an up-to-date best guess of how incoming funds would be allocated on the margin. For example, if we believed the next $5 million of funding for Malaria Consortium would be spent in Burkina Faso, and the $5 million after that would be spent in Chad, we would present the cost-effectiveness estimate for Burkina Faso until $5 million was donated, and then present the cost-effectiveness estimate for Chad until the next $5 million was donated, and so on.

However, we believe this approach would be impractical:

  • It would be confusing if our impact estimates frequently changed.
  • Keeping an up-to-date estimate of how marginal funds would be spent would create an unnecessary administrative burden. Instead, we make decisions about how to allocate funds on a rolling basis, and prioritize a detailed understanding of how we expect marginal funds to be spent for the most decision-relevant funding opportunities.
  • It can be challenging to communicate clearly about what we mean by "the margin." When making decisions about how to allocate funding, we estimate the marginal dollar based on a projection of how much funding charities will receive from other sources (including donors who give to a specific charity on the basis of GiveWell's recommendation) between now and when the funds will be needed, as well as in the past. For example, based on the example above, if we projected that donors would give $7.5 million to Malaria Consortium, we would expect the marginal additional dollar to be spent in Chad, rather than Burkina Faso. However, that would mean never presenting the cost-effectiveness estimate of Burkina Faso, which we believe would be misleading.

Because of these difficulties, we use a weighted average of programs that GiveWell has directed funds toward in the past calendar year rather than the cost-effectiveness of the specific program we expect would be funded by a marginal donation. We think this is a clearer way to present a rough and intuitive estimate of how much good a donation can accomplish (which is the goal of the top charities page).

Do these cost-effectiveness estimates include all factors that are relevant for choosing which programs to fund?

The purpose of our cost-effectiveness analyses is to help GiveWell make better decisions about how best to allocate funds between different programs.

These estimates abstract from certain features of a program that we think are relevant for choosing which programs to fund. For example, our estimates do not include:

  • Our assessment of an organization's ability to respond effectively to unexpected events
  • The costs and benefits of delaying a decision to preserve the value of retaining future options

Because of these limitations, a cost-effectiveness analysis is not the only tool we use when we decide whether to allocate funding to a program. You can see the principles we follow to allocate funding on this page.

In addition to excluding certain factors that we take into account in our decision-making, our cost-effectiveness estimates are often highly uncertain.

While we make our best effort to produce a reliable estimate, it can be challenging to ensure internal consistency between different estimates. For example, for practical reasons we restrict the scope of our estimates to focus on the most important benefits of a particular program.

You can find more information on what goes into our cost-effectiveness estimates here.

Why does GiveWell compare programs to cash transfers?

When we're making decisions about which programs to fund, we use a single unified metric that compares cost-effectiveness to GiveDirectly's cash transfer program. For example, in 2021, we estimated that some funding opportunities with AMF in the Democratic Republic of the Congo were approximately 10x as cost-effective as cash.

We often model different types of benefits from an intervention. For example, in 2021 we estimated that about 30% of the benefits of AMF's programs are the result of improved long-term economic outcomes for children who are protected from malaria. The "x cash framework" allows us to aggregate a program with different types of modeled benefits into a single metric, which is useful for our decision-making.

An additional benefit of using this metric is that we could spend large amounts of money on direct cash transfers, as the program is relatively straightforward to implement and GiveDirectly has the infrastructure to put significant increases in funding to use. Because of this, cash transfers to GiveDirectly set a clear lower bound on cost-effectiveness that GiveWell should consider (although in practice we aim to fund programs that we believe are substantially more cost-effective than cash transfers; see here for more information).

Despite the benefits of comparing programs to cash transfers, this metric is especially unintuitive. For ease of interpretation, we therefore present our impact estimates in terms of the main benefit from a charity's program.

Is the program with the lowest average cost per life saved the best program GiveWell recommends?

The average cost to save a life differs among our top charities and may change from year to year. However, we don't believe that the organization with the lowest average cost per life saved is a better giving opportunity than our other recommendations, even though these estimates might suggest that.

When we are deciding how to allocate funding to our top charities, our research team decides which of our top charities have the highest priority funding needs at that time. This decision takes into consideration factors such as:

  • Which funding gaps we expect to be filled and unfilled
  • Each organization's plans for additional funding
  • The cost-effectiveness of each funding gap

An organization with a lower average cost per life saved may not, for example, have the capacity to absorb funding for additional cost-effective opportunities. For this reason, we don’t believe that donations to the organization with the lowest average cost per life saved are necessarily more cost-effective than donations to our other top charities.

  • 1

    We are providing 2021 averages for AMF because we made only one grant to AMF in our 2022 metrics year, and our cost-effectiveness estimate for that grant was based on incomplete information about how funding from the grant would be used and is likely unrepresentative of the funding we typically direct to AMF. See the addendum at the top of the grant page for more context.

  • 2

    We are providing 2021 averages for AMF because we made only one grant to AMF in our 2022 metrics year, and our cost-effectiveness estimate for that grant was based on incomplete information about how funding from the grant would be used and is likely unrepresentative of the funding we typically direct to AMF. See the addendum at the top of the grant page for more context.

  • 3

    This only includes funding to our current top charities through the end of our 2022 metrics year (which ended January 31, 2023). See the totals here.

    As of December 2023, we have directed more than $2 billion to all organizations and grants we've recommended, according to the historical metrics reports listed in our 2022 metrics report (in the spreadsheet here), and internal records from 2023.

  • 4

    While we have now updated our analysis of the impact of our grantmaking to include some information about funds directed in 2022, we have not yet assessed whether to update our cost-effectiveness estimates for pre-2020 funds directed to rely on our cost-effectiveness estimates from 2022.

  • 5For example, we may expect that donations given directly to charities on the basis of our recommendations may be used to fund opportunities that we decided weren't cost-effective enough for us to make a grant to. On the other hand, because we account for the direct donations we expect a charity to receive in our room for more funding analyses, we often assume that these donations will be used to fill the highest-priority and most cost-effective funding opportunities first, and therefore that any grants we make to fill remaining funding gaps would be allocated to relatively lower-priority and less cost-effective funding opportunities.
  • 6

    While we have now updated our analysis of the impact of our grantmaking to include some information about funds directed in 2022, we have not yet assessed whether to update our cost-effectiveness estimates for pre-2020 funds directed to rely on our cost-effectiveness estimates from 2022.

  • 7

    We estimate that nets delivered by Against Malaria Foundation cost $5.19 each.

  • 8

  • 9

  • 10
    • "We define LLIN coverage as the proportion of people living in households that were targeted by an AMF distribution who subsequently sleep under LLINs, thereby benefiting from the protective effect against malaria. . . . With one exception, the PDMs [post-distribution monitoring studies] we focus on have found reported coverage rates of between 75% and 90%." GiveWell: Against Malaria Foundation, section on "What proportion of targeted recipients use LLINs over time?"
    • "Interestingly, nets with no holes had an average of just over one mosquito per net. Presumably, this was due to improper usage and residents should be instructed on how to tuck their nets in to prevent mosquitoes from entering them." Ochomo et al. 2013, pg. 7.
    • We adjust our cost-effectiveness estimate for net use here: GiveWell: 2021 GiveWell cost-effectiveness analysis — version 3, "AMF" sheet, cell A51.

  • 11

    For example, one study found that ITNs are less effective in some locations, such as Myanmar, "where the principal malaria vectors bite outdoors early in the evening, before people go in or near their ITN." GiveWell: Mass Distribution of Long-Lasting Insecticide-Treated Nets (LLINs), section on "How might the effectiveness of ITNs vary across settings with different malaria transmission patterns?," footnote 45.

Source URL: https://www.givewell.org/impact-estimates