Evaluation of Givewell Blogs

Making Predictions about Our Grants

1 month 1 week ago

From sports announcers to political pundits to friends gossipping about romantic interests, lots of people make probabilistic predictions about the future. But only some actually follow up to see how well their predictions performed. For instance, you may have heard that weather forecasters predicted that the 2024 hurricane season had an 85% chance of being more active than normal and there would be 17 to 25 named storms. But in September, they were surprised that so far, it had been unexpectedly quiet, with climate change likely affecting weather patterns in ways scientists don’t fully understand.

GiveWell researchers often make forecasts about activities, milestones, and outcomes of the programs and research studies we recommend funding for, as well as decisions that GiveWell will make in the future. For example, we might forecast whether we’ll fund more hospital programs to implement Kangaroo Mother Care programs by 2027, or whether data collected about how many people are using chlorinated water will align with our expectations.

As a way to solicit external feedback on some of our predictions, we just launched a page on Metaculus, an online forecasting platform. Periodically, we will post forecasts there about GiveWell’s research and grants for the public to make their own predictions. Metaculus and other contributors will award $2,250 in prizes to people who leave insightful comments on a select group of forecasting questions. The deadline for comments is December 1, 2024.

Read More

The post Making Predictions about Our Grants appeared first on The GiveWell Blog.

Hannah Bell

GiveWell’s 2022 metrics report

11 months 4 weeks ago

In 2022, the most recent year for which data is available and analyzed, GiveWell raised the largest amount of money in our history, over $600 million. We thank our donors for continuing to trust us to find and recommend highly cost-effective giving opportunities. The following table summarizes our funds raised and our funds directed to programs in metrics year 2021 and 2022.

Read More

The post GiveWell’s 2022 metrics report appeared first on The GiveWell Blog.

Isabel Arjmand

The winners of the Change Our Mind Contest—and some reflections

2 years ago

In September, we announced the Change Our Mind Contest for critiques of our cost-effectiveness analyses. Today, we're excited to announce the winners!

We're very grateful that so many people engaged deeply with our work. This contest was GiveWell's most successful effort so far to solicit external criticism from the public, and it wouldn't have been possible without the participation of people who share our goal of allocating funding to cost-effective programs.

Overall, we received 49 entries engaging with our prompts. We were very happy with the quality of entries we received—their authors brought a great deal of thought and expertise to engaging with our cost-effectiveness analyses.

Because we were impressed by the quality of entries, we've decided to award two first-place prizes and eight honorable mentions. (We stated in September that we would give a minimum of one first-place, one runner-up, and one honorable mention prize.) We also awarded $20,000 to the piece of criticism that inspired this contest.

Winners are listed below, followed by our reflections on this contest and responses to the winners.

Read More

The post The winners of the Change Our Mind Contest—and some reflections appeared first on The GiveWell Blog.

Isabel Arjmand

Announcing the Change Our Mind Contest for critiques of our cost-effectiveness analyses

2 years 3 months ago

We're extremely excited to be announcing the Change Our Mind Contest to encourage critiques of our cost-effectiveness analyses that could lead to substantial improvements of our overall allocation of funds. For all the details, see this page.

Cost-effectiveness is the single most important input in our decisions about what programs to recommend, and we believe it's possible that we're missing important considerations or making mistakes that lead us to allocate funding suboptimally. We've been excited to see people engaging with our cost-effectiveness analyses, and we'd like to inspire more of that engagement.

With that in mind, we're inviting you to identify potentially important mistakes or weaknesses in our existing cost-effectiveness analyses and tell us about them!

Read More

The post Announcing the Change Our Mind Contest for critiques of our cost-effectiveness analyses appeared first on The GiveWell Blog.

Isabel Arjmand

GiveWell’s 2021 metrics report

2 years 4 months ago

In 2021, GiveWell directed the largest amount of money in our history, over $500 million, which we believe will be beneficial or life saving to many people in need. We thank our donors for continuing to trust us to find and recommend some of the most highly cost-effective giving opportunities in the world.

Note that this year, we’ve also updated our metrics report to more clearly communicate about our work. Previously, we reported on our "money moved," a metric that tried to provide a composite picture of both the funds we raised and directed in a given year. However, starting with our 2021 report, we'll be reporting on these metrics separately as "funds raised" and "funds directed," which we believe will be simpler and clearer.

Read More

The post GiveWell’s 2021 metrics report appeared first on The GiveWell Blog.

Robin Dey

GiveWell’s money moved in 2020

3 years 1 month ago

2020 was another year of tremendous growth. GiveWell donors contributed over $240 million to our recommended charities (our "2020 money moved"), a 60% increase from the approximately $150 million we directed in 2019. This is part of an exciting, long-term trend. Just a decade ago, in 2010, GiveWell’s total money moved was $1.5 million.

We believe these donations will save tens of thousands of lives and benefit many others. This incredible impact would not be possible without the continued support and generosity of our donors. While our research enables us to identify and recommend highly cost-effective giving opportunities, our donors are responsible for turning those recommendations into real change for some of the poorest individuals in the world.

Read More

The post GiveWell’s money moved in 2020 appeared first on The GiveWell Blog.

Robin Dey

Early signs show that you gave more in 2020 than 2019—thank you!

3 years 7 months ago

Our donor community appears to have given significantly more in 2020 than 2019, according to early data on donations we processed.

Growth was strong relative to previous years—even 2019, which also had strong growth—and across many different dimensions. Overall, donations to GiveWell more than doubled in 2020.

We estimate that these donations will collectively save more than 12,000 lives; provide over 2 million deworming treatments to children, leading to an approximate increase in that group's lifetime earnings of more than $21 million; and deliver almost 3,000 cash transfers to low-income households. For simplicity, the impact estimates in this paragraph exclude some donation types, and so don't represent the full impact of donations to GiveWell in 2020.[1]

Read More

The post Early signs show that you gave more in 2020 than 2019—thank you! appeared first on The GiveWell Blog.

Catherine Hollander

GiveWell’s money moved in 2019

4 years ago

GiveWell donors contributed over $150 million to our recommended charities in 2019. This is the first year in our 13-year history we have reached this remarkable milestone.

While it is easy to celebrate this large headline figure, we should not lose sight of the transformative impact this number represents, nor should we overlook the compassion and generosity of the donors behind these dollars. We estimate that donations made to our top charities in 2019 will:

  • Avert the deaths of between 19,000 and 32,000 people,
  • Provide parasitic worm treatments to over 12 million children, and
  • Deliver unconditional cash transfers to over 14,000 families.[1]

Read More

The post GiveWell’s money moved in 2019 appeared first on The GiveWell Blog.

Robin Dey

GiveWell’s plans for 2020

4 years 6 months ago

Each spring, we share our plans for the year. Here, we highlight the work we plan to do in 2020 that is most likely to help us realize our mission of identifying and directing funding to highly cost-effective giving opportunities.((This post does not include a complete accounting of everything we plan to do in 2020. In particular, it does not include work aimed at primarily internal-facing results, such as improvements to internal staff communications.)) We focus on three projects:

  1. Expanding into new areas of research.
  2. Searching for new, cost-effective funding opportunities in our traditional research areas.
  3. Building our donor community.

Sharing our annual plans and publicly reflecting back on them a year later is our typical practice. This year, of course, is atypical. The plans we laid out internally at the beginning of the year have been disrupted by the COVID-19 pandemic. The plans we share in this post take the pandemic into account, but we are more uncertain than usual about what will happen in 2020. We expect that much of our work will go forward as anticipated, but we will be flexible if there are unforeseen disruptions or changes to our research agenda that result from the pandemic.

Read More

The post GiveWell’s plans for 2020 appeared first on The GiveWell Blog.

Catherine Hollander

Reflecting on our progress in 2019

4 years 7 months ago

GiveWell grew significantly in 2019. We hired 13 full-time staff members, bringing our total size to 37, and expanded our ability to take on new projects across domains. We feel positioned to do more and better work going forward as a result.

We see a strong indication that the amount of funding we directed to our recommended charities increased last year, too. While we haven't reconciled all giving from 2019, the value of donations we processed increased by about 30% in 2019.

We're proud of what we accomplished in 2019. We also fell short of some goals last year. Most notably, we failed to make as much progress as we planned in researching new areas of global health and poverty alleviation.

This blog post provides a brief look at our key successes and failures last year. A more detailed accounting of how our progress in 2019 compared to the goals we set is available on this page.

Read More

The post Reflecting on our progress in 2019 appeared first on The GiveWell Blog.

Catherine Hollander

How did we do in 2019? A preliminary look at our growth.

4 years 10 months ago

We see an early indication that GiveWell continued its trajectory of robust donor growth last year. The total value of donations processed by GiveWell increased 30% in 2019.[1]

We're sharing this data now because we believe it is an informative early update about our growth last year. However, GiveWell-processed donations don't tell the full story of our impact. Many donors who rely on our research give via our partner organizations or directly to our top charities. Their gifts account for the majority of donations due to our work and are not processed by GiveWell. Information about these gifts is time-consuming to gather and has usually led us to release our metrics data many months after the end of the year. We plan to release a complete 2019 metrics report and assessment of our impact, including donations not processed by GiveWell, later this year.

Here's what we know so far, based on the nearly complete information we have about donations we processed:

  • We processed $54.1 million in donations in 2019. Sixty-five percent of this amount was restricted to our recommended charities and 35% was unrestricted, which we may use to support GiveWell's operations.
  • Support from donors giving $10,000 to $100,000 comprised the largest proportion of our growth (35%).[2]
  • Returning donors who gave more than last year made up 75% of our growth in funds donated (excluding anonymous donations).[3]
  • We believe that the majority of our growth was organic and would have occurred without any outreach and marketing efforts from GiveWell, although we can attribute some to specific outreach and marketing initiatives.[4]

We're encouraged by this growth and excited to write about it. We also discuss below some ways that GiveWell-processed donations could be a misleading indicator of our overall impact.

Read More

The post How did we do in 2019? A preliminary look at our growth. appeared first on The GiveWell Blog.

Catherine Hollander

GiveWell’s money moved and web traffic in 2018

5 years 3 months ago

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2018 metrics report, which reviews what we know about how our research impacted donors. Please note:

  • We report on “metrics years” that run from February through January; for example, our 2018 data cover February 1, 2018 through January 31, 2019.
  • In an effort to present a more comprehensive measure of our influence on charitable giving, this year's metrics report includes GiveWell Incubation Grants in our headline "money moved" figure. In previous reports we have excluded Incubation Grants from this figure.

Summary of influence: In 2018, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2018, we tracked $141 million in money moved to our recommended charities and via our Incubation Grants program. Our money moved only includes donations that we are confident were influenced by our recommendations.

Read More

The post GiveWell’s money moved and web traffic in 2018 appeared first on The GiveWell Blog.

Devin Jacob

GiveWell’s plans for 2019

5 years 7 months ago

Our top priorities this year support our goals to (a) increase the impact per dollar of the funds we direct and (b) increase our money moved. In 2019, we are focused on:

  • Building research capacity. (More)
  • Experimenting with approaches to outreach to find ones that we can scalably use to drive additional money moved. (More)
  • Exploring new areas of research. (More)
  • Improving GiveWell's organizational strength. (More)
  • Ongoing research. (More)

Read More

The post GiveWell’s plans for 2019 appeared first on The GiveWell Blog.

Catherine Hollander

Review of GiveWell’s work in 2018

5 years 7 months ago

2018 was a successful year for GiveWell. We achieved most of our goals and our money moved (donations made to our recommended charities due our research) increased significantly.

Each year, we look back at the goals we set the previous year and reflect on how our progress compared to our expectations.

This post will briefly discuss our key achievements and failures in 2018. We describe in detail our progress on the goals we outlined in 2018 here.

In 2018, we:

  • Directed an estimated $65 million in donations to our top charities, not including the contributions of Good Ventures, a large foundation with which we work closely.
  • Added senior hires in operations and outreach: a Director of Operations (Whitney Shinkle) and Head of Growth (Ben Bateman). We expect Whitney and Ben to make major contributions to our work in these domains.
  • Continued to improve and expand our core research product, completing new intervention reports, deepening our analysis for several key inputs into our cost-effectiveness model, and providing more transparent explanations for how we decided to allocate funds between top charities.

Read More

The post Review of GiveWell’s work in 2018 appeared first on The GiveWell Blog.

Catherine Hollander

How GiveWell’s research is evolving

5 years 10 months ago

To date, most of GiveWell's research capacity has focused on finding the most impactful programs among those whose results can be rigorously measured. This work has led us to recommend, and direct several hundred million dollars to, charities improving health, saving lives, and increasing income in low-income countries.

One of the most important reasons we have focused on programs where robust measurement is possible is because this approach largely does not rely on subject-matter expertise. When Holden and I started GiveWell, neither of us had any experience in philanthropy, so we looked for charities that we could evaluate through data and evidence that we could analyze, to make recommendations that we could fully explain. This led us to focus on organizations that had impacts that were relatively easy to measure.

The output of this process is reflected in our current top charities and the programs they run, which are analyzed in our intervention reports.

GiveWell has now been doing research to find the best giving opportunities in global health and development for 11 years, and we plan to increase the scope of giving opportunities we consider. We plan to expand our research team and scope in order to determine whether there are giving opportunities in global health and development that are more cost-effective than those we have identified to date.

We expect this expansion of our work to take us in a number of new directions, some of which we have begun to explore over the past few years. We have considered, in a few cases, the impact our top and standout charities have through providing technical assistance (for example, Deworm the World and Project Healthy Children), supported work to change government policies through our Incubation Grants program (for example, grants to the Centre for Pesticide Suicide Prevention and Innovation in Government Initiative), and begun to explore areas like tobacco policy and lead paint elimination.

Over the next several years, we plan to consider everything that we believe could be among the most cost-effective (broadly defined) giving opportunities in global health and development. This includes more comprehensively reviewing direct interventions in sectors where impacts are more difficult to measure, investigating opportunities to influence government policy, as well as other areas.

Making progress in areas where it is harder to determine causality will be challenging. In my opinion, we are excellent evaluators of empirical research, but we have yet to demonstrate the ability to make good judgments about giving opportunities when less empirical information is available. Our values, intellectual framework, culture, and the quality of our staff make me optimistic about our chances, but all of us at GiveWell recognize the difficulty of the project we are embarking on.

Our staff does not currently have the capacity or the capabilities to make enough progress in this direction, so we are planning to significantly increase the size of our staff. We have a research team of ten people, and we are planning to more than double in size over the next three years. We are planning to add some junior staff but are primarily aiming to hire people with relevant experience who can contribute as researchers and/or managers on our team.

GiveWell's top charities list is not going to change dramatically in the near future, and it may always include the charities we recommend today. Our top charities achieve outstanding, cost-effective results, and we believe they are some of the best giving opportunities in global health and development. We expect to conclude that many of the opportunities we consider in areas that are new for us are less cost-effective than those we currently recommend, but we also think it is possible that we will identify some opportunities that are much more cost-effective. We believe it is worth a major effort to find out.

Read More

The post How GiveWell’s research is evolving appeared first on The GiveWell Blog.

Elie

GiveWell’s money moved and web traffic in 2017

6 years 5 months ago

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2017 metrics report, which reviews what we know about how our research impacted donors. Please note:

  • We report on “metrics years” that run from February through January; for example, our 2017 data cover February 1, 2017 through January 31, 2018.
  • We differentiate between our traditional charity recommendations and the work of the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
  • More context on the relationships between GiveWell, Good Ventures, and the Open Philanthropy Project can be found here.

Summary of influence: In 2017, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2017, we tracked $117.5 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our nine top charities received the majority of our money moved. Our seven standout charities received a total of $1.8 million.

Money moved by size of donor: In 2017, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2017, 90% of our money moved (excluding Good Ventures) came from 20% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 29% year-over-year to 23,049 in 2017. This included 14,653 donors who gave for the first time. Among all donors who gave in the previous year, about 42% gave again in 2017, up from about 35% who gave again in 2016.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2015. Of larger donors (those who gave $10,000 or more in either of the last two years), about 73% who gave in 2016 gave again in 2017.

GiveWell’s expenses: GiveWell’s total operating expenses in 2017 were $4.6 million. Our expenses decreased from about $5.5 million in 2016 due to the Open Philanthropy Project becoming a separate organization in June 2017. We estimate that 67% of our total expenses ($3.1 million) supported our traditional top charity work and about 33% supported the Open Philanthropy Project. In 2016, we estimated that expenses for our traditional top charity work were about $2.0 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.7 million in unrestricted funding (which we use to support our operations) in 2017, compared to $5.6 million in 2016. Our major institutional supporters and the six largest individual donors contributed about 49% of GiveWell’s operational funding in 2017.

Web traffic: The number of unique visitors to our website remained flat in 2017 compared to 2016 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2017 appeared first on The GiveWell Blog.

Maryana Pinchuk

GiveWell’s outreach and operations: 2017 review and 2018 plans

6 years 8 months ago

This is the third of three posts that form our annual review and plan for the following year. The first two posts covered GiveWell’s progress and plans on research. This post reviews and evaluates GiveWell’s progress last year in outreach and operations and sketches out some high-level goals for the current year. A separate post will look at metrics on our influence on donations in 2017. We aim to release our metrics on our influence on donations in 2017 by the end of June 2018.

Summary

Outreach: Before 2017, outreach wasn’t a major organizational priority at GiveWell (more in this 2014 blog post). In our plans for 2017, we wrote that we planned to put more emphasis on outreach, but were at the early stages of thinking through what that might involve. In the second half of 2017, we experimented with a number of different approaches to outreach (more on the results below). In 2018, we plan to increase the resources we devote to outreach primarily by hiring a Head of Growth and adding staff to improve our post-donation follow-up with donors.

Operations: In 2017, we completed the separation of GiveWell and the Open Philanthropy Project and increased our operations capacity with three new hires. In 2018, our top priorities are to hire a new Director of Operations (which we have now done), maintain our critical functions, and prepare our systems for increased growth in outreach.

Outreach 2017 review and 2018 plans

Before 2017, outreach wasn’t a major organizational priority at GiveWell (more in this 2014 blog post). In our plans for 2017, we wrote that we planned to put more emphasis on outreach, but were at the early stages of thinking through what that might involve.

We currently have one staff member, Catherine Hollander, who works on outreach full-time. Two others, Tracy Williams and Isabel Arjmand, each spend significant time on outreach. From August 2017, our Executive Director, Elie Hassenfeld, also started to allocate a significant amount of his time to outreach.

How did we do in 2017?

In 2017, we focused on experimentation. In brief, we found that:

  • Advertising on podcasts has had strong results. Using the methodology described in this blog post, our best guess is that each dollar we spent on podcast advertising returned $5-14 in donations to our top charities.
  • Increasing the consistency of our communication with members of the media had strong results for the time invested.
  • Retaining a digital marketing consultant yielded strong results.
  • Retaining a PR firm to generate media mentions did not have positive results.
  • We’ve had a limited number of conversations with high net worth donors. We don’t yet have enough information to conclude whether this was a good use of time.

You can see our estimates of the five-year net present value of donations generated by each of these activities here. Overall, we spent approximately $200,000 and devoted significant staff time to this work. Our best estimate is that these efforts resulted in $2.5 million to $5.9 million in additional donations to our recommended charities.

We conclude:

  • New work on outreach had a high return on investment in 2017.
  • Some activities, such as podcast advertising and digital marketing improvements, have shown particularly strong results and should be scaled up.

What are our priorities for 2018?

Our marketing funnel has three stages:

  1. Awareness/acquisition: more people hear about GiveWell and visit the website,
  2. Conversion: more people who visit the site donate, and
  3. Retention: over time, donors maintain or increase their donations.

Our current working theory is that we should prioritize (though not exclusively) improving the bottom of this funnel (retention and conversion) before moving more people through it. We also plan to scale up the activities that worked well in 2017 and to continue experimenting with different approaches.

Our primary outreach priorities (which we expect to achieve and devote substantial capacity to) for 2018 are:

  1. Hire a Head of Growth to improve our efforts to acquire and convert new donors via our website. Over the long term, the Head of Growth will be responsible for digital marketing.

    What does success look like? Hire a Head of Growth.

  2. Improve the post-donation experience. We believe we have substantial room to improve our post-donation communication with donors. We have hired a consultant to help us improve our process.

    What does success look like? Significantly improve our process for post-donation follow-up before giving season 2018.

    At this point, we’re still in the earliest stages of figuring out how we’ll do this, so we don’t have concrete goals for the year beyond finalizing our plan in the next few months. Our stretch goal for the year is to succeed in achieving measured improvement in our dollar retention rate/lifetime value of each donor.

Our secondary outreach priorities (which we expect to achieve, but not devote substantial capacity to) for 2018 are:

  1. Continue advertising on podcasts. This advertising was particularly successful in 2017. We want to systematically assess podcast advertising opportunities and increase our podcast advertising. We plan to spend approximately $250,000 to $350,000 on podcast advertising this year.

    What does success look like? Advertise on new podcasts and measure results to decide how much to spend in 2019.

  2. Receive coverage in major news outlets. This has led to increased donations in the past.

    What does success look like? Pitch major news outlets on at least five stories in total and get at least one story covered.

  3. Deepen relationships with the effective altruism community. We want to deepen our relationships with groups in the effective altruism community doing outreach, particularly to high net worth donors.

For a list of other potentially promising projects we’re unlikely to prioritize this year, see this spreadsheet.

Operations 2017 review and 2018 plans

In 2017, we increased our operations staff capacity, made a number of changes to our internal systems, and completed the separation of GiveWell and the Open Philanthropy Project. In addition to maintaining critical functions, our highest priorities for 2018 are to (i) appoint a new Director of Operations and (ii) make improvements to our processes across the board to prepare our systems for major growth in outreach.

How did we do in 2017?

We made a number of improvements to our operations. In brief:

  • We completed the separation of GiveWell and the Open Philanthropy Project.
  • Donations: We hired two new members of our donations team, which allowed us to process donations consistently notwithstanding increased volume. We also added Betterment and Bitpay (for Bitcoin) as donation options.
  • Finance: We hired a Controller. We rolled out a few systems to improve the efficiency of our internal processes (Expensify, Bill.com, and others).
  • Social cohesion: We created a regular schedule for visit days for remote staff and staff events to maintain cohesion.

In January 2018, Sarah Ward, our former Director of Operations, departed. Natalie Crispin (Senior Research Analyst) has been covering her previous responsibilities during our search for a new hire to take them on.

What are our priorities for 2018?

In the first half of 2018, we aim to move from a situation in which we were maintaining critical functions to positioning the organization to grow.

Our two main priorities for the first half of 2018 are to:

  1. Appoint a new Director of Operations (complete). In April 2018, we hired Whitney Shinkle as our new Director of Operations. Between January and April 2018, Natalie Crispin served as our interim Director of Operations.
  2. Prepare our systems for major growth in outreach, which we expect to lead to increases in spending, staff, and donations.
  3. Maintain critical operations across domains: donations, finance, HR, office, website, recruiting, and staff cohesion.

Major operations projects we aim to complete in the first half of 2018 include:

  • A significant improvement in our approach to budgeting making it significantly easier for us to share updated actual spending versus budget.
  • We retained a compensation consultant to help us benchmark GiveWell staff compensation to comparable organizations.
  • We published our 2016 metrics report and plan to publish our 2017 money moved report by the end of June.

The post GiveWell’s outreach and operations: 2017 review and 2018 plans appeared first on The GiveWell Blog.

James Snowden (GiveWell)

Review of our research in 2017

6 years 8 months ago

This is the first of three posts that form our annual review and plan for the following year. This post reviews and evaluates last year’s progress on our work of finding and recommending evidence-based, thoroughly-vetted charities that serve the global poor. The following two posts will cover (i) our plans for GiveWell’s research in 2018 and (ii) GiveWell’s progress and plans as an organization. We aim to release our metrics on our influence on donations in 2017 by the end of June 2018.

Summary

We believe that 2017 was a successful year for GiveWell’s research. We met our five primary goals for the year, as articulated in our plan post from the beginning of the year:

Our primary research goals for 2017 are to:

  1. Speed up our output of new intervention assessments, by hiring a Senior Fellow and by improving our process for reviewing interventions at a shallow level.
  2. Increase the number of promising charities that apply for our recommendation. Alternatively, we may learn why we have relatively few strong applicants and decide whether to change our process as a result. Research Analyst Chelsea Tabart will spend most of her time on this project.
  3. Through GiveWell Incubation Grants, fund projects that may lead to more top charity contenders in the future and consider grantees No Lean Season and Zusha! as potential 2017 top charities.
  4. Further improve the robustness and usability of our cost-effectiveness model.
  5. Improve our process for following the progress of current top charities to reduce staff time, while maintaining quality. We also have some specific goals (discussed below) with respect to answering open questions about current top charities.

We achieved our five primary goals for the year:

  1. Our intervention-related output was greater than in any past year, although we still see room for improvement in the pace with which we complete and publish this work (more). We hired a Senior Fellow and published nine full or interim intervention reports in 2017, compared to four in 2016.
  2. We increased the number of promising charities that applied for our recommendation (more).
  3. We added two new top charities: Evidence Action’s No Lean Season (the first top charity to start as a GiveWell Incubation Grant recipient) and Helen Keller International’s vitamin A supplementation program (which joined our list as a result of our charity outreach work). We continued to follow our current Incubation Grant recipients and made several new Incubation Grants to grow the pipeline of new top charities (more).
  4. We made substantial improvements to our cost-effectiveness analysis (more).
  5. We reduced the amount of staff time spent on following our current top charities. We also completed 17 of the 19 activities outlined in last year’s plan (more).

We discuss progress on each of our primary goals below. For each high-level goal, we include (i) the subgoals we set in our last annual review, (ii) an evaluation of whether we met those subgoals, and (iii) a summary of key activities completed last year.

Goal 1: Speed up intervention assessments

In early 2017, we wrote:

In recent years, we have completed few intervention reports, which has limited our ability to consider new potential top charities. We plan to increase the rate at which we form views on interventions this year by:

  • Hiring a Senior Fellow (or possibly more than one). We expect a Senior Fellow to have a Ph.D. in economics, public health, or statistics or equivalent experience and to focus on in-depth evidence reviews and cost-effectiveness assessments of interventions that appear promising after a shallower investigation. In addition, Open Philanthropy Project Senior Advisor David Roodman may spend some more time on intervention related work.
  • Doing low-intensity research on a large number of promising interventions. We generally start with a two to four hour “quick intervention assessment,” and then prioritize interventions for a 20-30 hour “interim intervention report” (example). We don’t yet have a good sense of how many of these of these we will complete this year, because we’re unsure both about how much capacity we will have for this work and about how many promising interventions there will be at each step in the process.
  • Continuing to improve our systems for ensuring that we become aware of promising interventions and new relevant research as it becomes available. We expect to learn about additional interventions by tracking new research, particularly randomized controlled trials, in global health and development and by talking to select organizations about programs they run that they think we should look into.

How did we do? Achieved our goal.

Due to our uncertainty about the capacity we could devote to intervention assessments, we did not have an explicit target for how many reports we expected to complete. In 2017, we published seven interim intervention reports, two full intervention reports, and completed ~30 quick evidence assessments (defined below). Our research output for 2017 was higher than 2016, when we published one full intervention report, three interim intervention reports, and completed 30 quick evidence assessments.

What did we do?

Goal 2: Increase the pipeline of promising charities applying for our recommendation

In early 2017, we wrote:

We would like to better understand whether we have failed to get the word out about the potential value we offer or communicate well about our process and charities’ likelihood of success, or, alternatively, whether charities are making well-informed decisions about their fit with our criteria. (More on why we think more charities should consider applying for a GiveWell recommendation in this post.)

This year, we have designated GiveWell Research Analyst Chelsea Tabart as charity liaison. Her role is to increase and improve our pipeline of top charity contenders by answering charities’ questions about our process and which program(s) they should apply with, encouraging promising organizations to apply, and, through these conversations, understanding what the barriers are to more charities applying.

We aim by the end of the year to have a stronger pipeline of charities applying, have confidence that we are not missing strong contenders, or understand how we should adjust our process in the future.

How did we do? Achieved our goal.

More charities entered our top charity review process in 2017, although it’s unclear whether this was due to our charity liaison activities. Five charities formally applied in 2017, compared to two in 2016, and four in 2015. One of those charities, Helen Keller International’s vitamin A supplementation program, became a top charity.

While we feel our relationships with well-regarded global health and development implementers and funders have improved, we continue to feel limited in our ability to understand whether there are funding gaps for evidence-backed, highly cost-effective work within large international NGOs and multilateral aid organizations such as the Global Fund to Fight AIDS, Tuberculosis and Malaria.

What did we do?

  • We had at least one conversation with 42 organizations to introduce them to GiveWell’s work in 2017, compared to 16 in 2016.
  • Where organizations running multiple programs expressed interest in applying for our recommendation, we had several calls with them to help determine whether they should apply and which of their programs would be the most promising fit for a top charity evaluation. We had not offered this proactive support to organizations in the past.
  • We hosted two charity-focused events: (i) a conference call for charities with GiveWell senior staff to present an update on our work as it relates to charities and to give them a chance to ask questions directly of our senior team and (ii) a networking event for our recommended organizations in London.
  • We attended seven conferences on global health and development issues to broaden our network and perspective in subject-matter areas that GiveWell has not historically worked on.
Goal 3: Maintain Incubation Grants

In early 2017, we wrote:

We made significant progress on Incubation Grants in 2016 and plan in 2017 to largely continue with ongoing engagements, while being open to new grantmaking opportunities that are brought to our attention.

Among early-to-mid stage grants, we plan to spend the most time on working with IDinsight and New Incentives (where our feedback is needed to move the projects forward), and a smaller amount of time on Results for Development and Charity Science: Health (where we are only following along with ongoing projects).

Another major priority will be following up on two later-stage grantees, No Lean Season and Zusha!, groups that are contenders for a top charity recommendation in 2017. For No Lean Season, a program run by Evidence Action, our main outstanding questions are whether the program will have room for more funding in 2018 and whether monitoring will be high quality as the program scales. We have similar questions about Zusha! and in addition are awaiting randomized controlled trial results that are expected later this year.

How did we do? Exceeded goal.

As expected, our work last year focused on following up on current grantees. No Lean Season, one of our later-stage grantees, graduated to top charity status and we made one grant to a new grantee, the Centre for Pesticide Suicide Prevention. We also made a number of grants to improve our understanding of the evidence base for our priority programs and deepened our partnership with IDinsight.

What did we do?

Goal 4: Improve our cost-effectiveness analysis

In early 2017, we wrote:

We plan to continue making improvements to our cost-effectiveness model and the data it draws on (separate from adding new interventions to the model, which is part of the intervention report work discussed above). Projects we are currently prioritizing include:

  • Making it more straightforward to see how personal values are incorporated into the model and what the implications of those values are.
  • Revisiting the prevalence and intensity adjustment that we use to compare the average per-person impact of deworming in places that our top charities work to the locations where the studies that found long-term impact of deworming were conducted. More in this post.
  • Improving the insecticide-treated nets model by revisiting how it incorporates effects on adult mortality and adjustments for regions with different malaria burdens and changes in malaria burden over time.

How did we do? Achieved goal.

We made substantial progress on improving our cost-effectiveness analysis in 2017.

What did we do?

  • Moved to a system of making more frequent updates to our cost-effectiveness analysis. This has made it easier to identify which specific factors are driving changes in the estimated cost-effectiveness of our top charities.
  • Revisited how we think about leverage and funging (how donating to our top charities influences how other funders spend their money) and updated our cost-effectiveness analysis accordingly.
  • Published a report on how other global actors approach the difficult moral tradeoffs we face.
  • Prior to announcing our 2017 recommendations, we performed a sensitivity check on our cost-effectiveness analysis to identify how sensitive our final outputs were to different uncertain inputs. This has helped us identify which inputs we should prioritize additional research on, and we believe it has made our communication more transparent, particularly around our personal values.
  • Revisited and updated our prevalence and intensity adjustments for deworming.
  • Deprioritized improving how our insecticide-treated net model incorporates effects on adult mortality. A limited number of conversations with malaria experts made us less confident that there was informative research on the question that would improve the accuracy of our models.
  • Deprioritized making adjustments for subnational regions with different malaria burdens because it would take substantial time to deeply understand the assumptions informing the subnational models we have seen. We believe this remains an important weakness of our model and that it limits our ability to make high-quality decisions about prioritization among different regional funding gaps.
Goal 5: Improve our process for following top charities

In early 2017, we wrote:

“In 2017, we plan to have a single staff member do most of this work and expect it to take a half to two-thirds of a full-time job. Three other staff will spend a small portion of their time, totaling approximately the equivalent of one full-time job, on this work.”

How did we do? Achieved goal.

We estimate that it took about 40 percent of the staff member’s time who focused on this work plus a small portion of four other staff members’ time, totaling at most and likely somewhat less than the equivalent of a full-time job (roughly half the time we dedicated to top charity updates in 2016).

We believe we maintained or increased the quality of the top charity updates, as we completed or made major progress on all but two of the activities and questions outlined in last year’s plan.

What did we do?

The table below summarizes our progress on each of the activities and open questions outlined in last year’s plan.

Charity Goals and open questions from 2017 plan Did we meet our goal? What did we do? Evidence Action’s Deworm the World Initiative “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from Nigeria, Vietnam, Kenya, and Ethiopia (see rows 11-20 and an overview of what we learned). GiveDirectly “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from Kenya (1, 2). (Overview of what we learned.) Schistosomiasis Control Initiative “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from 2016 programs in a number of countries. (Monitoring information, overview of what we learned.) Against Malaria Foundation (AMF) “Will AMF’s monitoring processes be high quality?” Yes We commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings. “Going forward, AMF aims to fund larger distributions and commit funding further ahead of when a distribution is scheduled to occur than it has, for the most part, done in the past. Will this increase the extent to which AMF funds displace funds from other sources, or will there continue to be evidence that AMF’s funds are largely adding to the total number of nets distributed?” Partial We learned relatively little about the displacement/fungibility question because AMF signed relatively few new agreements to fund long-lasting insecticide-treated net distributions in 2017. There was an update to how AMF will be tracking displacement, described in the second paragraph here. “In order to estimate AMF’s room for more funding, we will seek out information on the location and size of funding gaps for mass net distribution campaigns from AMF, the African Leaders Malaria Alliance, and possibly other funders of nets. As we have in the past, we will use this information in conjunction with conversations with AMF about non-funding bottlenecks to its ability to fill various gaps.” Yes We got updates on AMF’s room for more funding, as summarized in this post. The END Fund’s deworming program “We have not yet seen monitoring on par with that from our other top charities from the END Fund. We expect results from coverage surveys from END Fund programs this year. Will these surveys be high quality and demonstrate that the END Fund is funding successful programs?” Yes We saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible (more here). “We have not yet tried to compare the cost-effectiveness of the END Fund to our other top charities in our cost-effectiveness model. We will be seeking additional information from the END Fund about cost per treatment and baseline infection rates” Yes We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas where the END Fund works. We completed a cost-effectiveness analysis, though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. “Questions around room for more funding: the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, actual and projected revenue from other sources, and what deworming grantmaking opportunities the END Fund expects to have.” Yes We estimated the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, discussed here. “We visited the END Fund’s programs in Rwanda and Idjwi island, DRC in January 2017 and will publish notes and photos from our visit shortly.” Yes We posted notes and photos from our site visit here. Malaria Consortium’s seasonal malaria chemoprevention program “Further research on the evidence of effectiveness, cost-effectiveness, and potential downsides of seasonal malaria chemoprevention (SMC) (due to time constraints we have not yet completed a full intervention report, though we felt sufficiently confident in the intervention to recommend Malaria Consortium).” Yes We reviewed each of the RCTs included in the Cochrane review for seasonal malaria chemoprevention, and possible negative/offsetting impacts. We updated our interim intervention report to a full intervention report and added new information to our cost-effectiveness analysis. Our key conclusions did not change substantially and SMC remains a priority program. “Getting a better understanding of the methodology Malaria Consortium uses for estimating coverage rates.” Yes We spoke with Malaria Consortium to understand how they measure coverage and updated our cost-effectiveness analysis to account for different levels of coverage in the Malaria Consortium program relative to the headline results of the RCTs in the Cochrane review (conversation notes here). “Completing a more in-depth room for more funding analysis for the program for 2018 than we did for 2017.” Yes We completed a significantly more in-depth room for more funding analysis than we had previously (more here). “We may visit a Malaria Consortium seasonal malaria chemoprevention program in summer 2017.” No We did not conduct a site visit. Sightsavers’ deworming program “We expect to make limited progress this year because the first deworming mass drug administration funded with GiveWell-influenced funds is not expected to take place until September at the earliest and monitoring results aren’t expected until early 2018. Because Sightsavers has done fairly little deworming in the past year, we don’t expect to be able to learn much from its ongoing programs.” Exceeded In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results. “Getting more information from Sightsavers about baseline prevalence and intensity of worm infections in the areas it is working, to inform our cost-effectiveness analysis.” Yes We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis). “Using Sightsavers’ budget for the projects and planned treatment numbers to improve our estimate of the cost per treatment – another input into our cost-effectiveness analysis.” Yes We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis). “Completing a room for more funding analysis for 2018.” Yes We completed a room for more funding analysis (more here). Standout charities “We plan to have at least one phone call with each of these groups to discuss whether anything has changed that might lead us to reopen consideration of the organization as a potential top charity” Yes We spoke with each standout charity. Conversation notes here: Development Media International, Food Fortification Initiative, Global Alliance for Improved Nutrition’s Universal Salt Iodizational program, Iodine Global Network, Living Goods, and Project Healthy Children.

The post Review of our research in 2017 appeared first on The GiveWell Blog.

James Snowden (GiveWell)

GiveWell’s money moved and web traffic in 2016

6 years 8 months ago

In September 2017, we posted an interim update on GiveWell’s 2016 money moved and web traffic. This post summarizes the key takeaways from our full 2016 money moved and web traffic metrics report. Note that some of the numbers, including the total headline money moved, have changed since our interim report. Since then, we decided to exclude some donations from our headline money moved figure (details in the full report), and we corrected some minor errors.

This report was highly delayed (as discussed in the interim update). We expect to publish our report on GiveWell’s 2017 money moved and web traffic much more quickly; our current expectation is that we will publish that report by the end of June.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2016 metrics report, which reviews what we know about how our research impacted donors. Please note:

  • We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
  • We differentiate between our traditional charity recommendations and our work on the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
  • More context on the relationship between Good Ventures and GiveWell can be found here.

Summary of influence: In 2016, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2016, we tracked $88.6 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our seven top charities received the majority of our money moved. Our six standout charities received a total of $2.9 million.

Money moved by size of donor: In 2016, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2016, 93% of our money moved (excluding Good Ventures) came from 19% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 16% year-over-year to 17,834 in 2016. This included 12,461 donors who gave for the first time. Among all donors who gave in the previous year, about 35% gave again in 2016, down from about 40% who gave again in 2015.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2014. Of larger donors (those who gave $10,000 or more in either of the last two years), about 77% who gave in 2015 gave again in 2016.

GiveWell’s expenses: GiveWell’s total operating expenses in 2016 were $5.5 million. Our expenses increased from about $3.4 million in 2015 as the size of our staff grew and average seniority level rose. We estimate that about one-third of our total expenses ($2.0 million) supported our traditional top charity work and about two-thirds supported the Open Philanthropy Project. In 2015, we estimated that expenses for our traditional charity work were about $1.1 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.6 million in unrestricted funding (which we use to support our operations) in 2016, compared to $5.1 million in 2015. Our major institutional supporters and the five largest individual donors contributed about 70% of GiveWell’s operational funding in 2016. This is driven in large part by the fact that Good Ventures funded two-thirds of the costs of the Open Philanthropy project, in addition to funding 20% of GiveWell’s other costs.

Web traffic: The number of unique visitors to our website was down very slightly (by 1%) in 2016 compared to 2015 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

Natalie Crispin
Checked
35 minutes 53 seconds ago
Exploring how to get real change for your dollar.