New Incentives — General Support (April 2020)

Published: August 2020

Note: This page summarizes the rationale behind a GiveWell Incubation Grant to New Incentives. New Incentives reviewed this page prior to publication.

Summary

In April 2020, New Incentives received the first disbursement ($506,108) of a GiveWell Incubation Grant of up to $1,897,905 to support its program providing conditional cash transfers (CCTs) for infant vaccination in North West Nigeria. This grant supplements a larger GiveWell Incubation Grant that was made in November 2017 to support New Incentives' operations during a randomized controlled trial (RCT) of its program. The additional funding will sustain New Incentives' operations for the duration of the RCT, plus additional time for us to evaluate the results of the RCT and one extra month for New Incentives to plan its wind-down if we decide not to recommend funding for it in the future. From mid-May until our investigation is complete (between mid-July and December 2020), New Incentives will receive monthly disbursements of $253,054, covering its operating expenses for up to 7.5 months in total.

Table of Contents

The intervention

New Incentives offers conditional cash transfers (CCTs) to incentivize infant vaccination in North West Nigeria.1 There is strong evidence that expanding vaccination reduces illness and death among young children.2 CCTs could plausibly be a cost-effective way to increase vaccination rates among children in areas like North West Nigeria where vaccine supply is adequate but uptake is low.3 Our most recent estimates of the cost-effectiveness of New Incentives' program put it in the range of cost-effectiveness of programs we would consider recommending funding.4

(Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.)

More details on New Incentives' program can be found here.

Past grants to support New Incentives

New Incentives has received several GiveWell Incubation Grants to support different iterations of its program, starting in 2014. More information on past grants to New Incentives and related materials are available here.

November 2017 grant

In November 2017, New Incentives received a GiveWell Incubation Grant of $5,944,203. This grant was intended to cover:5

  • Operating expenses for the duration of a randomized controlled trial (RCT), the results of which will be a significant factor in determining whether we recommend funding for New Incentives in the future.
  • Additional operating expenses to support New Incentives' program while we make our decision on further funding for New Incentives.
  • Exit funding in case we decide not to recommend further funding for New Incentives.

Our initial grant recommendation included an additional $745,074 to account for the possibility that a larger-than-expected number of clinics would be chosen for randomization; this funding was not disbursed.6

More information on the November 2017 grant is available here.

Rationale behind this grant

This grant covers all of the funding that we expect New Incentives to need before we finish processing the results from the RCT and decide whether or not to recommend additional funding to New Incentives.

The November 2017 grant was intended to cover New Incentives' operating expenses through September 2020.7 At the time the grant was made, we believed this would be sufficient time to allow for the completion of the RCT and data analysis, reporting of final results,8 and a wind-down period in case we decided not to recommend further funding.9 Since then, however, New Incentives has extended the timeline for the completion of the RCT, and we have revised our projection of the amount of time required to make our decision, hence the need for additional funding to sustain New Incentives' operations.10

What this grant covers

How much additional funding will be required depends on when the final RCT results are delivered and when we make our decision. Currently, we expect that IDinsight, the organization conducting the RCT, will deliver final or near-final RCT results in early July 2020, and that we will make our decision on whether to recommend future funding for New Incentives no later than the end of November 2020.11 At the time of this grant decision, New Incentives had enough funding to operate through mid-May 2020 before it would need to begin using exit funding;12 it therefore requires up to 6.5 additional months of program costs.

The current grant includes enough funding for up to 7.5 months of program costs, which takes into account the above timeline plus an additional month of wind-down time in case we decide not to recommend further funding (see below). New Incentives will receive monthly disbursements of $253,054 from mid-May until our investigation is complete.13

Updates since the previous grant

The factors contributing to the difference between our expectations for timeline/costs in 2017 and our current expectations are:

  • New Incentives requested two months' additional ramp-up time to improve program implementation before starting the RCT. In March 2018, we approved a request by New Incentives to delay the start of the RCT by two months, until July 15, 2018.14 We and New Incentives thought that the additional time would ensure the program was running smoothly before the RCT began, thereby increasing the quality of the data generated.

    Although the RCT was initially delayed by two months, New Incentives now needs funding for only two weeks of this extension period, for two reasons:

    • IDinsight expects to need one month less for the preliminary data analysis than we expected in 2017, reducing the overall delay from two months to one month.
    • New Incentives was able to use budget surplus to cover two weeks of the extension period.15

    The current grant includes approximately $125,000 to cover operating expenses from this delay.16

  • We have decided to wait on final RCT results to inform our decision, rather than use preliminary RCT results. In 2017, we planned to make our decision on recommending future funding for New Incentives based on preliminary RCT results, which IDinsight provided to us in early April 2020.17 We have since decided to wait for the final results to make our decision, as this will allow us to incorporate additional analyses that are not part of the preliminary results. This will also allow more time for IDinsight and an external reviewer to check for errors in the analyses.

    Of the current grant, approximately $500,000 will go toward funding New Incentives for the approximately two-month period expected between preliminary and final RCT results.18

  • We increased our expectation of how long it will take us to make a decision on future funding for New Incentives. In November 2017, we expected that it would take one month for us to decide whether to recommend funding for New Incentives in the future, and we allocated funding accordingly. Although we aim to decide as quickly as possible, we now estimate that we may need up to five months after the final RCT results are in to make this decision. The current grant includes up to approximately $1 million to cover New Incentives' operating expenses for an additional four months, if needed.
  • New Incentives requested an extension of the wind-down period to allow it more time to inform its partners of plans for shutdown, if necessary. New Incentives requested that one more month's worth of operating expenses (approximately $250,000) be added to the wind-down period, allowing time for it to inform its partners and other stakeholders of the RCT results and plan out the program's shutdown in the event we decide not to recommend further funding.19 New Incentives has told us that it did not anticipate the need for this extra month in 2017.20

Next steps and decision timeline

As noted above, we expect to make a decision on whether to recommend further funding for New Incentives no later than the end of November 2020. The primary consideration in whether we recommend additional funding will be our assessment of how cost-effective New Incentives' program is compared with other programs for which we expect to recommend grants this year and in the next few years. As part of assessing cost-effectiveness, we have outlined our plans for assessing the internal and external validity of the RCT results in an addendum to the main pre-analysis plan. If we decide not to recommend further funding, New Incentives will likely make plans to shut down its program, using the portion of the November 2017 grant allocated for this purpose ($634,660)21 as well as the one month of additional funding included in the current grant.

Internal forecasts

For this grant, we are recording the following forecast:

75% chance that GiveWell will make a decision by September 30, 2020, about whether to recommend that Open Philanthropy and other donors continue to fund New Incentives.

Sources

Document Source
Email thread between GiveWell and Svetha Janumpalli, Founder and CEO, New Incentives, June-December 2019 Unpublished
GiveWell's December 2019 analysis of New Incentives' remaining funding needs Source
GiveWell's early 2018 cost-effectiveness analysis of New Incentives (with baseline vaccination rates) Source
GiveWell's January 2020 grant extension questions for New Incentives and answers from New Incentives' founder and CEO, Svetha Janumpalli Source
GiveWell's non-verbatim summary of a conversation with Svetha Janumpalli, Pratyush Agarwal, and Patrick Stadler on August 2, 2017 Source
GiveWell's winter 2017 preliminary cost-effectiveness analysis of New Incentives Source
IDinsight's 2017 report of a site visit to New Incentives Source
IDinsight's pre-analysis plan for the New Incentives RCT Source (archive)
New Incentives webpage: How It Works Source (archive)
New Incentives' November 2017-December 2020 budget Source (archive)