Charity Science: Health — General Support

Published: November 2016

[Added December 19, 2016: GiveWell's experimental work is now known as GiveWell Incubation Grants.]

Note: This page summarizes the rationale behind a grant to Charity Science: Health made by Good Ventures. Charity Science: Health staff reviewed this page prior to publication.

Summary

As part of GiveWell's experimental work to support the creation of future top charities, in November of 2016 Good Ventures granted $200,000 to Charity Science: Health to support the first year of its work setting up a charity to send SMS immunization reminders in India.

We have had the following conversations about updates on this grant:

Table of Contents

The organization

Charity Science: Health was created by members of the effective altruism community with the goal of creating a potential future GiveWell top charity. We are impressed by the Charity Science: Health team's track record of transparency, and we are confident that they will communicate well with us and that it will be easy for us to learn from their work.

The intervention

Charity Science: Health plans to implement a program sending SMS reminders for immunizations in India. Charity Science: Health pointed us to a relatively large amount of evidence assessing this program1 , and our impression is that there is also some evidence for the effectiveness of SMS reminders for other interventions.2

Grant details

Charity Science: Health's full budget is $250,000. It plans to allocate its budget as follows: 3

  • $120,000 for program costs, including:
    • $90,000 for grants to organizations4 that collect phone numbers and run SMS immunization reminder programs. The first $30,000 of this will go to testing partnership work with three different organizations. The remaining $60,000 will go to the best-performing of these three organizations to support monitoring and evaluation and additional phone number acquisition.
    • $30,000 for Charity Science: Health itself to work on acquiring phone numbers and sending SMS reminders.
  • $72,000 to pay Charity Science: Health staff.
  • $16,000 for travel expenses.
  • $42,000 for contingency and other expenses.

Cost-effectiveness

The cost-effectiveness of Charity Science: Health's program seems potentially promising to us, but is relatively uncertain. At the time we decided to recommend this grant, Charity Science: Health estimated a cost of approximately $2,000 per life saved, whereas our rough cost-effectiveness analysis puts this figure between $5,000 and $10,000.5 At the time of publishing, Charity Science: Health's estimate is around $4,500.6

We plan to improve our estimate of the cost-effectiveness of the program before considering recommending Charity Science: Health as a top charity.

Room for more funding

Before we made this grant, Charity Science: Health had raised $50,000 and was seeking an additional $200,000. Charity Science: Health staff told us they expected that they could relatively easily raise an additional $75,000 from the effective altruism community if GiveWell openly supported the organization (for example, by recommending a grant like this one). We considered making a grant of $125,000, but decided to fully fund Charity Science: Health's room for more funding to enable its staff to spend less time fundraising and avoid the possibility that Charity Science: Health would have to reduce the size and number of experiments it planned to run due to insufficient funding.

Risks and reservations

We have a few reservations about this grant:

  • The Charity Science: Health team has limited experience working in this area.
  • We have not deeply vetted the organization or the evidence for SMS reminders. It is possible that if we were to investigate further, we would find this intervention is significantly less cost-effective than we currently believe it to be.
  • While we did not find any serious issues when we evaluated the studies that Charity Science: Health relies on for evidence, we did find that some of these studies were not as robust as we would hope. For example, some studies have relatively small sample sizes and are published in non-premier journals, and at least one of these studies used self-reported immunization data rather than direct observations of immunization cards. We think it is somewhat likely that we would discover more problems if we investigated these studies further.

Internal forecasts

We are experimenting with recording explicit numerical forecasts of the probability of events related to our decision-making (especially grant-making). The idea behind this is to pull out the implicit predictions that are playing a role in our decisions, and to make it possible for us to look back on how well-calibrated and accurate those predictions were. For this grant, we are recording the following forecasts:

  • Good Ventures gives additional funding to Charity Science: Health in one year: 80%
  • Charity Science: Health becomes (or creates) a GiveWell top charity by giving season 2019: 15%

Sources

Document Source
Charity Science: Health, 2016 Budget Source (archive)
Charity Science: Health, Tentative SMS 2016 Plan Source (archive)
Charity Science: Health's cost-effectiveness analysis of SMS immunization reminders Source (archive)
Charity Science: Health's notes on evidence for SMS reminders Source (archive)
GiveWell's cost-effectiveness analysis of SMS immunization reminders Source (archive)
GiveWell's non-verbatim summary of a conversation with Joey Savoie on July 20, 2016 Source
Horvath et al. 2012 Source (archive)