Evidence Action — Strengthen Operations (2019)

Published: April 2019

Note: This page summarizes the rationale behind a grant to Evidence Action that GiveWell has recommended to Open Philanthropy. Evidence Action staff reviewed this page prior to publication.

The organization

We consider Evidence Action to be one of the groups with which GiveWell is most aligned; GiveWell and Evidence Action both place a high value on evidence and cost-effectiveness. Evidence Action runs multiple programs including: the Deworm the World Initiative, which is one of our top charities; No Lean Season, which we removed from our list of top charities in November 2018 based on new RCT evidence and in agreement with Evidence Action; and Dispensers for Safe Water, which is one of our standout charities. It also tests programs that have the potential to be highly cost-effective through its Beta Incubator.

Grant details

In March 2019, GiveWell recommended that Open Philanthropy make a grant of $7,751,915 to Evidence Action for organizational strengthening. GiveWell recommended that this multi-year grant should be split into two installments: an initial $3,000,000 installment paid up-front, followed by a $4,751,915 installment.1

The $7.75 million amount is comprised of the following, and is anticipated to benefit Evidence Action through 2024:

• $2.1 million to build a fundraising function:
  • $1.7 million to hire an expected five full-time positions (these are currently expected to be a Director, Associate Director, two proposal writers, and a senior associate for grants management).
  • $0.4 million for consulting services and other fundraising and communications strengthening activities.
• $2.8 million for leadership hires and associated costs:
  • This component includes hiring a Chief Program Officer, another senior leader, an Africa policy position, a senior position in India, and a Chief of Staff-type role in the US, as well as associated costs such as travel, telecommunications, and office rent.
  • While the Chief Program Officer and other senior leader position are not part of the fundraising function, Evidence Action told us that each of these two hires will create approximately 75% of one senior leader's worth of time to devote to institutional fundraising activities. As a result, $1.3 million of this cost can be thought of as contributing to fundraising activities. Fundraising activities therefore comprise $3.4 million of the total grant.2
• $1.5 million for flexible funding.
• $1 million to review compensation:
  • This component includes hiring a consultant to evaluate Evidence Action's compensation scheme and funds to implement the resulting recommendations.
• $0.3 million for Evidence Action's India Foundation.3

Case for the grant

At a high level, the main case for the grant is:

• Evidence Action currently has limited resources to pursue fundraising activities. By funding those activities, GiveWell believes that this grant will enable Evidence Action to raise more money from other donors, predominantly institutional donors. The cost-effectiveness of those donors' funds will likely increase relative to their alternative funding opportunities, as we believe that several of Evidence Action's programs are highly cost-effective.
• In addition, this will reduce Evidence Action's dependence on GiveWell as a donor.
• Evidence Action is facing several other organizational issues as described below, including capacity constraints at the senior leadership level.
• It has limited unrestricted funds with which it can address these organizational issues.
• We have a high opinion of Evidence Action as an organization due to its track record of producing strong programs such as Deworm the World.
• Evidence Action's Beta Incubator is a potential source of future GiveWell-recommended top charities, and so GiveWell has an interest in its long-term health as an organization.

Increasing fundraising activities

• GiveWell funding constitutes a large proportion of Evidence Action's revenue (66% in 2017),4 so for Evidence Action's organizational stability and independence, we believe it is important for it to raise more money from other donors.
• We believe that if Evidence Action is able to raise more money from other donors, predominantly institutional donors, it will increase the cost-effectiveness of those funds—as we believe that several of Evidence Action's programs are highly cost-effective.
• This component of the grant will fund the creation of a fundraising function with five full-time positions (Director, Associate Director, two proposal writers, and a senior associate for grants management), as well as some consulting services.
• In addition, hiring two additional senior leaders (see further details here) will create more senior leadership capacity to spend on fundraising activities. Evidence Action told us that each of these hires will create approximately 75% of one senior leader's worth of time to devote to institutional fundraising activities.5 We therefore count approximately 75% of the costs of those two hires as contributing to fundraising activities.
• We conducted a break-even analysis, which suggests that if Evidence Action raises a cumulative total of an additional $10.0 million by the end of 20246 , the fundraising component of the grant would be equivalently cost-effective to GiveWell's alternative option of providing additional funding to the top charity that we believe is most cost-effective as of March 2019 (Malaria Consortium).7 That conclusion depends on the following key assumptions:
  • Funds spent by Evidence Action are ~6x cash (85% spent on the Beta Incubator at 5.3x cash and 15% spent on its other programs at 9.7x cash on average).8
  • Institutional donors' counterfactual giving is 2x cash.9
• Our best guess is that this fundraising component of the grant leads to $17 million being raised by the end of 2024 that would not otherwise be raised (see this spreadsheet).

Extending leadership capacity

• Evidence Action is currently capacity constrained at the senior leadership level.
• This component of the grant will pay for the hiring of a Chief Program Officer and one other senior leader, in addition to one hire in each of its three main regions of operation: a Policy hire in Africa, a leadership hire in India, and a Chief of Staff-type role in the US.
• Those hires will help to i) make existing senior leaders' workloads manageable, ii) enable Evidence Action to undertake a range of activities that it previously did not have the capacity for, iii) build a second line of leadership in case existing senior leaders were to leave.

Flexible funding

• Evidence Action is a large organization with very little unrestricted funding, and so we believe that it should have some funds for activities that it sees as particularly cost-effective.
• By definition, this component of the grant is for currently unknown future activities, and so we could not analyze the case for this component in much detail.

Compensation review

• Evidence Action believes it is critical to ensure the hiring and retention of high-quality staff as it grows. There are some indications that the current structure of compensation is posing challenges to achieving this objective, though we cannot share details of this as it relates to sensitive information about staff compensation.
• We agree that the ability to hire and retain high-quality staff is important for Evidence Action's long-term health as an organization.
• This component of the grant will enable Evidence Action to hire a consultant to review its organizational compensation approach, and for Evidence Action to implement recommendations as applicable. The significant majority of this component will be for funding compensation increases associated with improving equity within the organization.

Strengthening the India Foundation

• Evidence Action’s current costs of operation for its programs in India (e.g. Deworm the World) are high because it requires the use of a separate, taxable entity.
• This component of the grant will help Evidence Action to strengthen its affiliated entity in India, called the Evidence Action India Foundation (EAIF). We believe this will speed up the rate at which it can move its programs across to operate out of EAIF, thereby making savings on costs of operations.10

Goals for the grant

We see the following as major goals of this grant:

• Evidence Action raises a cumulative total of an extra $15 million in donations as a result of this grant by the end of 2024.11
• Evidence Action has no major programmatic disruptions in GiveWell-recommended programs as a result of leadership departures between the time at which new senior leaders are hired and the end of 2022.
• Leadership at Evidence Action reports being able to move forward on strategic priorities and spend less time on lower value-add items by the end of 2021.
• Evidence Action receives Foreign Contributions Regulation Act (FCRA) approval from the Indian government by the end of 2022.12
• Evidence Action has no major issues in compliance with bilateral funders through the end of 2024.
• Evidence Action makes the proposed full-time hires: five people for the fundraising function (Director, Associate Director, two proposal writers, and a grants senior associate), and the additional five leadership hires (Chief Program Officer, Africa policy position, one leadership hire in India, a Chief of Staff-type position, and one additional senior leader whose position is still to be determined). These positions are what Evidence Action has told us is its current best guess for the positions needed to achieve the other goals of this grant. We recognize that its plans may evolve during the course of this grant.13

Room for more funding

• Evidence Action currently has very little unrestricted funding with which to pay for these organizational strengthening expenditures. In 2017, only 2.9% of Evidence Action's revenues were unrestricted.14
• Our understanding is that very few donors make core operations grants of this size in the field of global development. GiveWell is one of Evidence Action's primary funding sources and therefore is among the most logical funders of its core operations.

Risks and reservations

We believe that there are several main risks with making this grant:

• Evidence Action fails to attract funding from institutional donors. A key assumption underlying the decision to make the fundraising component of the grant is that Evidence Action will be able to attract substantial donations from institutional donors:
  • Our main reservation is that we believe the most likely way that Evidence Action can attract donations from institutional donors is through new programs in its Beta Incubator, and it is possible that its Beta Incubator fails to generate a sufficiently large number of cost-effective new programs to attract substantial donations, particularly in the short term.
• The donations Evidence Action attracts as a result of this grant draw funds away from other GiveWell-recommended top charities. It is our understanding that this risk is fairly small because we believe that the institutional donors that Evidence Action intends to target are unlikely to increase their donations to other GiveWell-recommended top charities in the near future.
• Flexible funding could be spent on activities that are not highly cost-effective. By definition, we have little understanding of what flexible funding will be spent on. It is possible that the activities that Evidence Action funds with this component of the grant are less cost-effective than GiveWell's outside option on the margin.
• Senior leadership hires do not reduce capacity constraints. It is possible that Evidence Action prioritizes expansion with its new senior leadership hires (for example expanding existing programs into new geographical areas), rather than alleviating capacity constraints in its existing activities.

Plans for monitoring the impact of the grant

The main monitoring activities that GiveWell and Evidence Action plan to undertake are:

• GiveWell will measure funds raised as a result of the grant. To do this, GiveWell will count any new donations from institutional donors, where "new" excludes any grant for which Evidence Action had already submitted a concept note or formal grant proposal to the donor prior to receiving this Open Philanthropy grant.15 We will also count any increase in donations made by existing "major gifts" donors (large,16 individual, non-institutional Evidence Action donors) following outreach conducted by Evidence Action's fundraising function (e.g. phone calls).17
• Evidence Action will report on various other outcomes to GiveWell, such as what hires have been made with the grant, how senior leadership capacity has changed and what flexible funds are spent on. For more details, see this spreadsheet.

Internal forecasts

For this grant, we are recording the following forecasts:

• 50% chance that Evidence Action raises a total of $6,120,000 in donations from institutional donors by the end of 2022.
• 50% chance that Evidence Action raises a total of $40,800,000 in donations from institutional donors by the end of 2024.
• 75% chance that the Indian government will allow Evidence Action to accept foreign donations to the Evidence Action India Foundation by the end of 2023.
• 75% chance that Evidence Action will have hired a Chief Program Officer and at least one other leader by the end of 2019.
• 25% chance that Evidence Action will have hired all new full-time positions for the fundraising function and senior leadership by the end of 2020.
• 50% chance that Evidence Action will have finalized a strategy on its approach to compensation by the end of 2020.

Process

• GiveWell had multiple meetings and calls with Evidence Action, predominantly with Kanika Bahl (CEO), but also with John de Wet (Chief Finance and Administrative Officer) and Grace Hollister (Chief Engagement Officer).
• The claims about Evidence Action's plans made throughout this post come from those conversations with Evidence Action, and Evidence Action have reviewed this page to check the accuracy of those claims.
• GiveWell created quantitative models of the two most mathematically tractable components of the grant (see here for the fundraising model), and evaluated all components of the grant qualitatively.

Sources

Document	Source
GiveWell analysis of fundraising component of 2019 Evidence Action organizational strengthening grant	Source
GiveWell rough cost-effectiveness analysis of Evidence Action Beta	Source
GiveWell stress test of the model to change in institutional donor's outside option	Source
Evidence Action, Metrics for grant evaluation	Source
Evidence Action, 2017 financial report	Source

• 1

  In practice, Open Philanthropy will make a second installment of $4,651,915, subtracting $100,000 that was granted in error to Evidence Action's Deworm the World Initiative program in November 2018. For more details, see this page.

• 2

  See this spreadsheet, rows 6-8 for details on this calculation.

• 3

  Evidence Action will undertake activities to strengthen its affiliated entity in India, the Evidence Action India Foundation, in order to operate its programs from that entity rather than through a separate, taxable entity as is its current practice.

• 4

  This figure was calculated based on Evidence Action, 2017 financial report.

• 5

  This 75% is the combination of both direct work on institutional fundraising by these two new senior leadership hires themselves and work on institutional fundraising that is carried out by other senior leaders whose time is opened up as they can pass work over to the new hires.

• 6

  See cell B4 of this spreadsheet.

• 7

  We believe that Malaria Consortium can spend additional funding at a cost-effectiveness of approximately 8.8x cash. See here.

• 8
  • Evidence Action told us that they believe that approximately 85% of additional funds raised by this grant will be spent on the Beta Incubator because it expects that the majority of funds raised as a result of this grant will come from institutional donors, and institutional donors are more likely to fund new programs emerging from the Beta Incubator than its existing programs (such as Deworm the World and Dispensers for Safe Water).
  • We calculate the cost-effectiveness of programs in the Beta Incubator as a weighted average of our best guess for the cost-effectiveness of Beta Incubator programs that are funded by GiveWell and Beta Incubator programs that are funded by institutional donors. We believe that Beta Incubator programs funded by GiveWell will be approximately 8.7x cash (see this spreadsheet, cell B18 for details). We believe that Beta Incubator programs funded by institutional donors could be less cost-effective, as other donors may have less emphasis on cost-effectiveness than GiveWell. We assume that Beta Incubator programs funded by institutional donors are 3x cash. We assume that 40% of Beta Incubator programs are funded by GiveWell and 60% by institutional donors. See the cell note in cell A26 in this spreadsheet.
  • We calculate the cost-effectiveness of spending on Evidence Action's other programs as a weighted average of the cost-effectiveness of Deworm the World, No Lean Season and Dispensers for Safe Water, where we assume that 65% of this expenditure is on Deworm the World, and the rest is split evenly (17.5% each) on No Lean Season and Dispensers for Safe Water. See the cell note in cell A31 in this spreadsheet for further details.

• 9

  We have not done substantial work on this assumption, though our conclusion seems robust to varying it to some degree (see sensitivity analysis here). In general the case for our top charities having outstanding cost-effectiveness is that we've assessed a large number of opportunities and found it difficult to find interventions and charities that approach the cost-effectiveness of our recommendations. So, our prior is that most giving opportunities are meaningfully less cost-effective than, for example, the most cost-effective health interventions we've found.

• 10

  We modelled the cost-effectiveness of this component of the grant. Our model is highly uncertain, but our best guess is that this component of the grant is approximately 24x cash. We cannot publish this model, as it contains sensitive information.

• 11

  We have calculated this $15 million figure as follows:

  • Evidence Action needs to raise an additional $10 million for the fundraising component of the grant to be equivalently cost-effective to GiveWell's outside option of donating to Malaria Consortium at the current margin. For more details, see our fundraising model.
  • In that case, we can think about additional dollars raised above that $10 million threshold as adding impact relative to our outside option. We therefore believe that the target should be significantly more than $10 million.
  • We also believe that there is a reasonable chance that Evidence Action could raise substantially more than $10 million. In our fundraising model, our best guess is that Evidence Action raises an extra $17 million as a result of this grant.
  • We have chosen a target to be between those two figures, at $15 million.
  • We have also set a target of a cumulative total of $2.25 million in additional funds raised as a result of the grant by the end of 2022. This was calculated as 15% of the end of 2024 target. Evidence Action told us that it will take time for institutional donors to increase their donations (as Evidence Action builds the fundraising function, and subsequent trust with donors), and this 15% figure is based on Evidence Action's best guess for the trajectory of institutional donors' donations over the period until the end of 2024. To reiterate, the $15 million and $2.25 million targets are for the sum of additional donations from institutional and individual donors, but we expect donations from institutional donors to make up the majority of the increases in donations.

• 12

  FCRA approval allows the India Foundation to receive donations from non-Indian donors.

• 13

  At the time of writing this grant page, from our communications with Evidence Action we believe that the Chief Program Officer, the Africa policy position, Chief-of-Staff type position, Director/Associate Director for fundraising and proposal writer are positions that Evidence Action is highly likely to make with this grant. The exact role of the other hires described in this page are less certain.
  

• 14

  This figure was calculated based on Evidence Action, 2017 financial report.

• 15

  In addition, GiveWell will exclude any grant renewals made by existing Evidence Action institutional donors if those grants are made for the same program for approximately the same amount as before.

• 16

  Evidence Action counts any individual donor who donates more than $10,000 in a year as a "major gifts" donor.

• 17

  Assessing whether Evidence Action raised the $15 million target in additional funds as a result of this grant is extremely difficult as we cannot know what funds would have been raised in the counterfactual. The approach to monitoring that we are taking here may under or overestimate funds raised by the grant for the following reasons:

  • Some new donations from institutional donors may have occurred anyway in the counterfactual. This will lead us to overestimate the funds raised as a result of this grant.
  • This grant may cause an increase in donations through new major gifts donors, or new or existing retail donors (small, individual, non-institutional donors). Since we have no way of tying these donations to activities taken with the grant as opposed to other factors, we are not counting them towards the $15 million target, and this may lead us to underestimate the funds raised as a result of this grant.

  In addition, the $15 million figure is based on a highly uncertain best guess for the amount of funds that Evidence Action needs to raise for this component of the grant to be a more cost-effective use of funds than GiveWell's outside option of funding Malaria Consortium ($10 million). That $10 million figure depends on a number of assumptions in our fundraising model about which we have little information, such as the cost-effectiveness of institutional donors' counterfactual giving.

  Nonetheless, assuming that the key parameters in our fundraising model are approximately correct, if Evidence Action raises an extra $15 million measured in the way described above, it seems reasonable to believe that the fundraising component of the grant was successful.