IDinsight – Research on Program Reach Estimates (August 2022)

Note: This page summarizes the rationale behind a GiveWell grant to IDinsight. IDinsight staff reviewed this page prior to publication.

Published: August 2022

In August 2022, GiveWell made a grant of $71,000 to IDinsight to fund desk-based research on approaches to assessing the number of people reached by mass distribution programs. We had discussed this research question with IDinsight, and IDinsight designed this initial phase of work. As described in IDinsight's concept note for the project:

To demonstrate continued cost-effectiveness and room for more funding, top charities have typically conducted coverage surveys. These exercises generate estimates of the percentage of the target population surveyed that the charity has reached. While coverage estimates give an indication of how much of the target population remains unreached (room for more funding), GiveWell ultimately prioritizes knowing the number of people the charity has served for its cost-effectiveness calculations. Converting coverage estimates to the number of people served requires accurate estimates of the target population (“the denominator”), which can be notoriously elusive. Further, GiveWell has discounted program estimates of the number of people reached based on count data because they are not able to gauge how accurate these estimates are.

Emerging technologies may present new opportunities to generate more accurate and more verifiable estimates of program reach. These innovations could span from satellite imagery to estimate populations to digital tools to more accurately and objectively record counts. Several top charities are currently exploring innovative ways to capture count data more accurately. Based on initial conversations between GiveWell and IDinsight, this note outlines an initial engagement to better understand and assess opportunities to improve estimating the denominator or the target population to ultimately lead to better estimates of program reach.1

In the second half of 2022, IDinsight will conduct desk-based research and interviews with GiveWell grantees. It will share a report on its findings and recommendations. More details are in IDinsight's concept note here.

We made this grant because a better process for assessing the number of people reached by mass distribution programs has the potential to change our cost-effectiveness estimates for tens to hundreds of millions of dollars of annual funding. (Relevant programs include vitamin A supplementation, deworming, insecticide-treated nets, and seasonal malaria chemoprevention, which account for the majority of our annual funding—see page 7 here.) This has been a source of uncertainty in our cost-effectiveness estimates for many years, and we've made little progress on it due to competing priorities.

We have worked with IDinsight on several previous projects.2

GiveWell recommended this grant via our policy for small discretionary grantmaking. As a small discretionary grant, this funding opportunity did not receive the same review as larger grants we recommend. Instead, we more minimally evaluated the case for the grant and any potential risks or downsides.

• 1

  IDinsight, Approaches to monitoring program reach concept note, 2022.

• 2

  See all of our published content on IDinsight here.