Published: August 2016
[Added December 19, 2016: GiveWell's experimental work is now known as GiveWell Incubation Grants.]
Note: this page summarizes the rationale behind a grant to IDinsight made by Good Ventures. IDinsight staff reviewed this page prior to publication.
Summary
As part of GiveWell’s general effort to support the development of potential future top charities and improve the quality of our recommendations, in June of 2016 Good Ventures granted $1,985,000 in unrestricted funding over three years to IDinsight. We hope that this grant will allow IDinsight to scale its model of supporting and evaluating development interventions, ideally (in the long term) providing us with research that we will be able to use in our work to recommend top charities.
Table of Contents
The organization
IDinsight supports and conducts rigorous evaluations of development interventions, often involving randomized controlled trials, with an explicit focus on providing useful data to inform funders and policy makers. This "decision-focused evaluation" model appears to us to be both uncommon and particularly aligned with GiveWell's goals.
This blog post by Lant Pritchett provides an overview of IDinsight.1
Grant details
IDinsight plans to use this grant in three main areas:
- Growth: Our understanding is that IDinsight is currently capacity constrained. It expects this grant to allow it to grow over the next three years from its current size of roughly 60 client-facing staff running about 25 projects to roughly 150 client-facing staff running 65 projects.
- Idea dissemination: IDinsight plans to attempt to influence the sector in general to conduct more effective decision-focused evaluations by issuing white papers (such as IDinsight, The Future of Impact Evaluation) and lobbying evaluation funders (e.g. the Gates Foundation, the International Initiative for Impact Evaluation, or the World Bank) to create funding streams to support decision-focused evaluations.
- Innovation: IDinsight plans to test several types of projects that it has not run previously.
Room for more funding
We considered a few different levels of funding for this grant. We ultimately decided to give IDinsight the full amount it requested because we expect it to be worthwhile to enable IDinsight to grow as fast as possible and avoid the need to devote senior staff time to fundraising.
IDinsight expects this grant to allow it to add roughly 90 staff members, which is 30 more than it would have been able to add with $1 million (the next lowest level of funding we considered). Because the majority of IDinsight's operating budget comes from the overhead rates it charges on its projects, our grant will not cover new staff members' entire salaries; instead, our grant will give IDinsight the funding it needs to recruit and train staff before placing them on projects that bring in revenue.
Risks and reservations
The main risks we see in relation to this grant have to do with the probability that IDinsight will produce research in the future that GiveWell can use in our own research to recommend top charities. More detail:
- Our impression is that IDinsight has not yet produced much research of the kind that we expect to be valuable to us. However, we believe this is likely to change as IDinsight grows.
- We think there is some risk that IDinsight could choose to focus on interventions or organizations whose evaluations do not provide enough useful data for us to make this grant worthwhile.
- IDinsight’s evaluations appear to us to often be designed to assess a program’s effectiveness at achieving its proximate goal, without necessarily evaluating the additional assumptions that would need to be true for the intervention to have a positive impact. For example, IDinsight’s evaluation of a program aimed at increasing institutional delivery by pregnant women in Zambia does not also attempt to assess how effective institutional delivery is at reducing maternal or neonatal mortality.2
- We expect the most useful result of this grant for us to be IDinsight growing its staff in order to conduct more evaluations relevant to GiveWell's work. We think IDinsight may not plan to prioritize this growth as highly as we would like relative to its plans for idea dissemination, since we see the latter as less likely to be directly beneficial to our work (though we do expect idea dissemination to have some positive effects).
Goals for the grant
The ideal outcome of this grant from our perspective would be for IDinsight to become one of our main partners on an ongoing basis. This might happen in one of two ways:
- IDinsight creates a GiveWell-focused team that helps evaluate interventions or organizations that we are interested in but currently do not have the necessary data to evaluate. For example, GiveWell is interested in evaluating UNICEF's maternal and neonatal tetanus (MNT) campaign, but we lack data on 1) the baseline MNT mortality rate, 2) the baseline immunization rate, and 3) the campaign's success at vaccinating people. A partner team at IDinsight could potentially obtain this data.
- IDinsight conducts evaluations for our top charities. Our understanding is that Neil Buddy Shah (CEO and Founding Partner of IDinsight) has discussed potential collaborations with Paul Niehaus of GiveDirectly and Jeff Brown of Evidence Action.
Key questions for follow-up
One year from now (i.e. around July 2017), we plan to check in with IDinsight on the following:
- Growth: How many staff has IDinsight added? How many projects is it implementing?
- Idea dissemination: Has IDinsight's work led at least one major evaluation funder to support decision-focused evaluation? How have IDinsight's white papers and blog posts been received, and what influence have they had? What are IDinsight's plans for idea dissemination going forward?
- Innovation: Has IDinsight decided on any new project types to pursue? What are its plans for innovation going forward?
Our process
We have been in contact with Neil Buddy Shah periodically for a couple of years. We previously attempted a project with IDinsight (which ultimately did not go forward) to evaluate the Schistosomiasis Control Initiative, one of our current top charities. Over the last few months, we have had several conversations with Dr. Shah about how IDinsight would plan to use additional unrestricted funds. We have also discussed IDinsight with other people that we respect in the field.
Sources
- 1Archived copy of link: Pritchett 2015
- 2See IDinsight, Mama Kits to Increase Institutional Delivery