International Rescue Committee (IRC) — Desk-Based Scoping of Programs to Increase Vaccination Coverage (September 2024)

Note: This page summarizes the rationale behind a GiveWell grant to IRC. IRC staff reviewed this page prior to publication.

Published: December 2024

Background

As discussed in this blog post, we have seen strong results from past work in the space of improving vaccination coverage, and we’re now exploring how to expand our work in this space.

Currently, we are focused on:

• Vaccines delivered to children under-2.
• Layering of high-value interventions that can be delivered alongside vaccines.
• Locations with low vaccine coverage rates and high child mortality rates.
• Interventions that can increase vaccination rates by making takeup much easier, namely by bringing vaccines closer to where parents and infants are, or offsetting the costs through incentives.

We expect to fund scoping, piloting, and evaluation before scale-up for most program types. For some GiveWell-funded interventions, there is a clear-cut intervention (e.g. a commodity given out in a door-to-door campaign) that can be scaled up in very similar ways in different locations. Vaccination programs are more integrated into countries' health systems than campaigns, and our working hypothesis is that whether an intervention is successful and cost-effective likely depends on highly contextual factors.

To that end, we are having discussions with several organizations that work with health systems in countries with high rates of vaccine-preventable disease about generating ideas for and testing approaches to increasing vaccination rates.

Our review of the evidence on the impact of vaccination is here.

What we think this grant will do

The grant will pay for a cost analyst and immunization program manager to work on this project for six months.1

The expected outputs, excerpted directly from the IRC proposal for this project below, are:2

Why we made this grant

Primary reasoning

This is a light-touch initial engagement to generate concrete ideas for programs that GiveWell can consider funding. See Background section above for how this work fits in with GiveWell's work on vaccination more broadly.

We believe IRC is a good fit for this work because:

• It has a broad implementation base, including immunization programs in 16 countries.3
• IRC is very experienced in conflict-affected countries. Conflict-affected countries in sub-Saharan Africa are among the countries with the lowest vaccination rates and highest mortality rates in the world.4 More specifically, IRC implements a $55 million Gavi-funded project to reach unimmunized children in conflict-affected areas.5
• IRC has a Best Use of Resources group that is particularly interested in getting high quality cost per output data and considering evidence on program effectiveness.
• We've found IRC's research and monitoring and evaluation capacity to be strong in projects we've partnered with IRC on.6
• IRC was proactive in proposing this work.

GiveWell has previously recommended grants to IRC for treatment of malnutrition.

Light check on cost-effectiveness

As a light test, we created a rough break-even analysis to show one scenario in which this grant could be a cost-effective use of funding. We used our standard "value of information" cost-effectiveness model and set the inputs so that resulting cost-effectiveness was at our cost-effectiveness bar (10 times as cost-effective as cash transfers—or "x cash" for short).7 In short, in this scenario, we assume that there is a 4% chance that the grant is the reason why GiveWell is able to fund $5 million worth of grants at 12x cash for five years.

This supports our intuition that this is a cost-effective use of resources. In practice, this grant alone is very unlikely to unlock a funding opportunity without further investment in scoping, piloting, and evaluation, and the costs in the model don't take into account that our time costs for this grant will be high relative to the amount of funding. But if GiveWell invests in further development of projects identified by IRC beyond this grant, we believe the likelihood of finding a cost-effective opportunity that unlocks more than $5 million for 5 years is much higher than 4%.

Our main reservations

• It's good to target hard-to-reach children because the benefits to reaching them with vaccines are higher (because they tend to have higher disease burden and lower vaccination coverage), but there are also challenges that may more than offset these benefits:
  • It may be very expensive to reach them.
  • The challenges that a hard-to-reach population faces may be unique to that group, giving us limited ability to apply evaluation results to other locations.
  • Evaluations may be practically impossible, e.g. in conflict-affected areas.
• While we are excited about this partnership and have discussed what we see as IRC’s strengths as a partner above, we also have a few concerns about IRC as a partner:
  • Relative to some of GiveWell's other close partners, we see IRC as having more formal approval processes before moving work forward, which could result in delays.
  • We see them as prioritizing equity and fairness more highly relative to cost-effectiveness, our top priority in evaluating these programs.
  • We've sometimes felt that IRC could increase our trust in their work more by being more transparent with us about both positive and negative updates.

Plans for follow up

• IRC and GiveWell will have monthly meetings to discuss progress and next steps.
• See above for expected deliverables at the end of six months.

Internal forecasts

For this grant, we are recording the following forecasts:

Our process

• IRC and GiveWell had conversations about IRC's experience in vaccination and GiveWell's areas of interest in the space.
• IRC shared a concept note for the project. IRC and GiveWell collaborated on editing the proposal.
• GiveWell created a break-even cost-effectiveness model of the grant.

Sources

• 1

  About $89,000 will fund 80% of a member of the IRC Best Use of Resources team's time, 25% of a Immunization Specialist's time, and some support from other staff members (10-20% of their time). About $38,000 is budgeted for contracted services, but may be substituted for staff time. Indirect costs are 18% of other project costs, totaling about $22,000. From the budget IRC provided to GiveWell on this project and our conversations with them (unpublished)

• 2

  From IRC's proposal to GiveWell for this project

• 3

  Per IRC’s proposal to GiveWell, IRC has active immunization programs in 16 countries.

• 4

  See our best-guess of the cost effectiveness of vaccination programs in different countries here.

• 5

  See this report on the project by IRC. When reviewing a draft of this page, IRC noted that the total is $55 million as of the fourth quarter of 2024.

• 6

  For example, see this previous grant to IRC.

• 7

  See here our Value of Information Back of the Envelope Cost-effectiveness analysis for this project.