Process for Identifying Top Charities - 2013 Version

We have published a more recent version of this page. See our most recent version of this page.

This page describes the process we used to identify our top charities

Introduction

GiveWell's mission is to find outstanding charities and to publish the full details of our analysis to help donors decide where to give. Our focus is on finding the most outstanding charities possible rather than completing an in-depth investigation for each organization we consider.

We're currently running two separate research processes with different criteria.

Our "top charities" work, the focus of this page, is the work that we originally focused on, and have carried out since our founding in 2007. It's rooted in our own struggles as donors and our attempt to find charities that were proven, cost-effective, and scalable, such that we could draw a maximally confident, linear, quantified link between donations and outcomes, along the lines of "$X per life saved" or "$Y per person enabled to get a job paying 20% more than they could have gotten otherwise." (See our former criteria.)

We've since come to recognize major limitations to this approach, and we are working to produce recommendations based on an alternate vision of giving — emphasizing higher-risk, higher-upside activities commonly associated with major funders. This alternate approach, the Open Philanthropy Project, hasn't yet produced recommendations. In the meantime we feel that our "top charities" recommendations have substantial value for donors seeking to give now. As such, we continue to provide our recommendations, while also being clear about their limitations.

We don't believe that our top charities offer linear, reliably quantifiable returns along the lines of "$X per life saved," but we do believe that they are distinguished from other charities by their focus on evidence-backed programs aiming to help the global poor and by their transparency and accountability, all of which we believe to be important qualities. (More at Our Criteria.)

This page discusses our process for identifying and researching top charities, as well as some of the context behind what our process has been in the past and how it has evolved to its current state.


Published: November 2013 (2012 version, 2011 version)

Table of Contents

Eligibility

In order to be eligible for "top charity" status, a charity must be explicitly focused on one or more of our priority programs — such as distributing insecticide-treated nets to prevent malaria, treating children for parasites, or direct cash transfers — or conduct compelling monitoring of its own program, such as the randomized controlled trials conducted by Development Media International and Living Goods. These programs represent, according to our research, the most evidence-backed approaches to helping the global poor.

Why these programs?

GiveWell has done multiple in-depth investigations looking to identify charities whose activities can be strongly connected — via empirical evidence — to improved life outcomes.

  • In 2007 (our first year), we conducted an open grant application process and reviewed 59 applications in global health and development.
  • In 2009:
    • We reviewed the websites of over 300 charitable organizations aiming to find ones that either (a) implemented priority programs (as we defined them at the time) or (b) published, on their websites, meaningful evaluations of their programs. Details of this process are available here.
    • We conducted a grant application process for organizations running economic empowerment programs in the developing world (details here).
  • In 2011, we repeated the above process and broadened our search to flag organizations based on additional characteristics (details here).
  • In 2012, we searched for organizations working on immunization, nutrition and other global health programs. Details of this search here.
  • In 2014, we (a) solicited applications from several organizations running priority programs and (b) evaluated two organizations that had conducted randomized controlled trials of their own programs. Details here.
  • In 2015, we (a) solicited applications from several organizations running priority programs and (b) began evaluations on four organizations, two that conduct deworming programs, one that distributes bednets, and one that works on micronutrient fortification programs. Details here.
  • Throughout, we have compiled a set of "intervention reports" which assess the evidence for the programs we have considered; all the programs we have considered are listed here

Based on these investigations, we have concluded that:

  • Highly rigorous evidence connecting aid activities to improved life outcomes (for example, "distributing insecticide-treated nets reduces the burden of malaria and saves lives") is found in academic literature, and not (in any cases that we've seen) in internal self-evaluations by charities. Self-evaluations may provide an important part of the picture, helping to assess whether a charity is carrying out activities as intended, but rigorous evidence that a program has the intended effect on life outcomes is found in academia. For more on how we assess evidence, see our 2012 series on how we assess studies: Our Principles for Assessing Evidence, How We Evaluate a Study, Surveying the Research on a Topic.
  • Even within academic literature, many categories of intervention have insufficient rigorous evidence to determine their effectiveness. See our 2009 write-ups on developing-world education and economic empowerment.
  • Interventions targeting the global poor are likely to accomplish substantially more good (per dollar spent) than interventions targeting the developed-world poor. (More in the next section)

Why focus on the global poor?

Our goal is to help donors accomplish as much good as possible, and we focus our investigations on the opportunities most likely to fit this description. Having done substantial research on both developing-world aid and programs focused in the U.S., we've narrowed our focus to the former, for the following reasons:

  • The U.S. poor are wealthy by developing-world standards. All the empirical analysis we've come across, as well as the qualitative observations we've made on site visits, support the conclusion that "poverty" has very different meanings in the developed world and the U.S., and that even the lowest-income people in the U.S. have (generally speaking) far greater material wealth and living standards than the developing-world poor. More at our 2009 blog post on this topic.
  • We haven't found any U.S. poverty-targeting intervention that compares favorably to our international priority programs, in terms of (a) quality, robustness, and generalizability of evidence; (b) cost-effectiveness, i.e., "bang for the buck" in terms of lives saved or improved per dollar spent. For more, see our write-ups on priority programs and U.S. programs.

We believe the second point is largely explained by the first: the developing-world poor struggle to meet basic needs and to access products and services with well-established benefits, to a greater extent than the U.S. poor.

Finding eligible charities

In past years, we have conducted extensive searches for charities that focus on our priority programs. The details of our processes and findings follow:

Our full list of eligible charities — charities that focus on evidence-backed programs serving the global poor — is available here.

We believe this list to be near-comprehensive, but if you know of organizations we are missing, we encourage you to submit them for our consideration.

Examining charities

We invite eligible charities to participate in our intensive evaluation process, which aims to deeply and critically question the case for the charity's impact, and lay out what we see as the strengths and weaknesses publicly.

Key questions

Our process focuses on answering the following questions:

What do they do?

We aim for a comprehensive understanding of the charity's budget and the nature of its value-added. Large-scale aid projects often require coordination between many actors (including governments), so we aim to understand the charity's specific role thoroughly.

Does it work?

Because eligible charities focus on priority programs, we generally begin our investigation with at least a partial understanding of the evidence base behind what they do. When looking at a potential top charity, we generally greatly deepen our investigation of this evidence base, examining not just the strengths and weaknesses of key studies but also considerations regarding how likely their results are to generalize to larger-scale programs. (See, for example, our write-ups on insecticide-treated nets, deworming and cash transfers.)

In addition, we attempt to examine the specifics of the charity in question, including data from internal reports and interviews with charity representatives, to assess whether its program is being carried out with high quality (and, in some cases, whether the charity is adding substantial value to what the program would look like without its presence).

What do you get for your dollar?

We try to quantify the impact of the program the charity executes, as it is executed (as opposed to in academic studies). We quantify impact using a variety of different measures, including "cost per disability-adjusted life-year (DALY)" as well as "cost per life saved" (when applicable) and "financial benefits to recipients per dollar spent by donors" (when applicable). These estimates involve substantial judgment calls, and we publish the details of our analysis, spell out our major assumptions and allow readers to fill in their own. The most recent iteration of this analysis is our 2012 writeup on the comparative cost-effectiveness of our different top charities' programs.

We recognize that our cost-effectiveness estimates have major limitations, and we do not advise taking them literally. (More.) The purpose of these estimates is to:

  • Identify differences in cost-effectiveness that are large and robust enough to affect our rankings.
  • Thoroughly think through the activities in question, and raise critical questions about them.

Room for more funds

We work to understand the impact of marginal dollars on a charity's activities and impact, because we are looking for charities that can use more funding productively — not just charities that have had strong impact in the past. More than once in the past, we have withdrawn our recommendation of a charity because it had raised as much funding as we felt it could productively absorb.

More on room for more funding.

Our process for answering key questions

Most of our investigation generally consists of conversations with charity representatives and review of their documents, which we make public (as sources for our charity reviews) to the extent we can.

In addition, we often speak to other current and potential funders of the charity in question, and we always conduct at least one site visit to see the charity's work in the field.

Consistent with our commitment to transparency, we share as much as we can from these investigations — notes from conversations, notes and photos from site visits, original documents and details of our analysis — via our charity reviews.

Our process aims to create as complete a picture as possible of the answers to our key questions, and we use this picture to (a) decide whether to recommend a charity; (b) decide how to rank it relative to our other top charities; (c) create a basis for follow-up (discussed below), which allows us to learn more and improve our understanding over time.

Following up

We follow up intensively with our top charities over time, and consider this one of the major arguments in favor of supporting such charities. Because our recommendation directs substantial donations to a charity, top charities are generally willing to engage substantively with us and help us deepen our understanding of their activities and progress over time.

Crucially, we believe — and make clear to our top charities that we believe — in sharing both positive and negative developments, and we have written extensively in the past about unanticipated struggles faced by top charities. See, for example, our series of updates on VillageReach.

Pros and cons of our top charities as giving opportunities

Over the years, we've substantially changed our outlook on what a good giving opportunity should look like, and we are working to produce giving opportunities that could look very different from these ones.

We think the principal advantages of our current top charities are that:

  • They represent the best opportunities we're aware of to help low-income people with relatively high confidence and relatively short time horizons. If you're looking to give this year and you don't know where to start, we'd strongly recommend supporting our top charities.
  • Due to the emphasis on thorough vetting, transparency, and following up, our top charities represent excellent learning opportunities, and we feel that one of the most desirable outcomes of giving is learning more that will inform later giving. Supporting our top charities helps GiveWell demonstrate impact and improves our ability to learn, and we are dedicated to sharing what we learn publicly.

Some counter-considerations:

  • There is an argument for saving money rather than giving, and giving at the point where better information on top giving opportunities is available. We do expect to make substantial progress on the Open Philanthropy Project over the next few years.
  • If you have access to other giving opportunities that you understand well, have a great deal of context on and have high confidence in — whether these consist of supporting an established organization or helping a newer one get off the ground — it may make more sense to take advantage of your unusual position and "fund what others won't," since GiveWell's research is available to (and influences) large numbers of people.