Against Malaria Foundation — Support for LLIN Campaign in South Sudan

Note: This page describes our reasoning and the status of AMF's decision-making as of May 2022, when we made this grant. As of August 2022, AMF has now formally committed to supporting the campaign in South Sudan. AMF staff reviewed this page prior to publication.

Addendum added: January 2023

Published: August 2022

Planned activities and budget

This $8,163,835 grant may support a campaign distributing LLINs in South Sudan, scheduled to occur in 2023; AMF has not formally committed to supporting this campaign. AMF has not supported LLIN campaigns in South Sudan before. AMF's model is to provide LLINs in bulk to other non-profit organizations or government agencies, which then distribute the nets. The majority of grant funds will be spent on purchasing nets for this campaign. A small amount (approximately 10%) will be spent on post-distribution monitoring.11 For more on AMF's role in LLIN distributions, see here.

We conduct "room for more funding" analysis to understand what portion of a grantee's ideal future budget it will be unable to support with the funding it has or should expect to have available. We may then choose to either make or recommend grants to support those unfunded activities. To determine the size of this grant, we updated our previous room for more funding analysis for AMF.12

Grant timing and AMF's decision-making process

AMF's typical process for deciding to fill a funding gap begins with discussions with the relevant country's national malaria program and other funding partners in order to determine the precise size of the funding gap, what other sources of funding might be available to fill that gap, and the details of AMF's potential support to the country. Once it has completed this assessment, and if it has decided that it wants to fill the gap, AMF will then sign a formal agreement with the country, which specifies the number of nets it will purchase and the procedures that will be followed in distributing and monitoring these nets.

We have typically made grants to AMF at this latter stage of negotiation, when AMF is ready to sign an agreement with a country and commit funding. In contrast, we made this grant at an earlier stage of AMF's process. AMF is still working out the details of the funding gap in South Sudan and what its support to the country might be. What this means is that the ultimate use of this funding is more uncertain: AMF may decide to fill a gap of a different size in South Sudan, or it may decline to fund South Sudan at all. Our reasoning for proposing this grant now is laid out below.

Because of this increased uncertainty, we have come to a written agreement with AMF regarding the post-grant process our organizations will follow for this grant and other grants that we may recommend at a similar stage. This process will involve the following:

• Prior to signing an agreement with a country, AMF will share with GiveWell new information that becomes available about the relevant funding gap.
• GiveWell will use this information to update our cost-effectiveness model for the relevant funding gap, giving us an updated estimate of its cost-effectiveness.
• AMF will alert GiveWell when it is ready to sign an agreement with a country.
• GiveWell will share input about the ultimate cost-effectiveness of this funding gap and whether or not we are in favor of AMF filling it. (Note that while AMF has agreed to take this input into consideration, we have not requested a formal approval process and have granted AMF ultimate discretion over which funding gaps it fills.)

The case for the grant

• Cost-effectiveness – During this grant investigation, we used our existing cost-effectiveness model for LLIN campaigns and updated various parameters to match the specifics of this funding gap. We estimate that this grant may be 10.2 times as cost-effective as unconditional cash transfers, which we use as our benchmark for comparing the cost-effectiveness of different interventions. More below.
• Funding landscape – We adjust our cost-effectiveness estimates to account for the extent to which we believe our funding may be crowding out funding that would otherwise have come from other sources. We believe that other funders may have partially filled this funding gap if we had not made this grant. This assessment is less well-informed than it would have been if we had waited to make this grant later in AMF's process. More below.
• Benefits of committing funding early – Making this grant when we did may enable AMF to order LLINs in time to avoid delaying this campaign and may improve AMF's capacity to identify new funding opportunities. More below.

Cost-effectiveness

How we use cost-effectiveness estimates in our grantmaking

After assessing a potential grantee's room for more funding, we may then choose to investigate potential grants to support the spending opportunities that we do not expect to be funded with the grantee's available and expected funding, which we refer to as "funding gaps." The principles we follow in deciding whether or not to fill a funding gap are described on this page.

The first of those principles is to put significant weight on our cost-effectiveness estimates. We use unconditional cash transfers as a benchmark for comparing the cost-effectiveness of different funding gaps, which we describe in multiples of "cash." Thus, if we estimate that a funding gap is "10x cash," this means we estimate it to be ten times as cost-effective as unconditional cash transfers. At the time we recommended this grant, we were primarily looking to recommend grants that we estimated were more than 10x as cost-effective as unconditional cash transfers and were more tentatively considering grants that were between 6x and 10x as cost-effective as unconditional cash transfers.

Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes to other grants we have made or considered making, and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible. Our process for estimating cost-effectiveness focuses on determining whether a program is cost-effective enough that it is above our bar to consider funding; it isn't primarily intended to differentiate between values that are above that threshold.

Cost-effectiveness of this grant

We estimate that this grant is 10.2x cash. To generate this estimate, we used our existing cost-effectiveness model for LLIN campaigns and updated various parameters to match the specifics of this funding gap. The key parameter updates we made during this investigation include:

• Efficacy reduction due to insecticide resistance – Our cost-effectiveness model for LLINs includes an adjustment to account for the efficacy lost due to resistance within mosquito populations to the insecticides used in LLINs (see our separate report on this topic for more details).13 While we normally use the IR Mapper datasets and the World Health Organization database for information about mosquitoes' resistance to different chemicals, those sources did not include data from South Sudan. We instead created an insecticide resistance adjustment for South Sudan using results from an unpublished report from the South Sudan National Malaria Control Program, and used averaged resistance data from Uganda and DRC, two bordering countries we had already modeled, for data that wasn't available in the report.14
• Comprehensiveness of monitoring and evaluation – AMF conducts post-distribution monitoring of completed distributions to determine whether LLINs have reached their intended destinations and how long nets remain in good condition.15 We reviewed recent monitoring results from AMF-supported campaigns that had been shared since our last review in late 2020. Though AMF hasn't worked in South Sudan before, reviewing past results from other places is relevant to this grant investigation because we have an adjustment in our CEA that accounts for monitoring comprehensiveness.16 We reviewed monitoring results to update our overall view of AMF's monitoring comprehensiveness, and found no major issues that would negatively impact our views. In short, we have reviewed results from 18 of the 19 large-scale distributions that occurred from 2017-2020, and one distribution from 2021 (more details on these results will be available in a forthcoming update to our AMF review). We therefore decided to retain the same value for this parameter.
• Adjustment for malaria deaths averted by seasonal malaria chemoprevention (SMC) – In our nets CEA, we include an adjustment to account for malaria deaths averted by SMC programs, which distribute antimalarial drugs to children during the high-transmission season. While SMC is not currently delivered in South Sudan, it is being piloted there by Malaria Consortium beginning in 2022 and may be scaled up in the next few years. To account for this possibility, we have updated the "malaria-attributable deaths averted per 1,000 children per year targeted with seasonal malaria chemoprevention" parameter in the South Sudan CEA to reflect a rough best guess that 30% of children who will be covered by this net campaign will also receive SMC.17
• Likelihood that Global Fund and/or the President's Malaria Initiative (PMI) would replace philanthropic costs – See discussion below.

Funding landscape

We adjust our cost-effectiveness estimates to account for the extent to which we believe our funding may be crowding out funding that would otherwise have come from other sources (in the case of LLINs, this is typically the Global Fund and/or PMI). Specifically, these adjustments represent the proportion of a grantee's funding that we believe may crowd out funding from other sources (for example, if we use an adjustment of 25%, we believe that 25 cents of every 1 dollar spent by the grantee would otherwise have come from other sources). See more details in this blog post. We incorporate considerations about the funding landscapes for LLINs in the countries we are considering for grants into location-specific adjustments, which are accounted for in our cost-effectiveness estimates.18

We used a value of 30% for the likelihood that the Global Fund (PMI does not fund work in South Sudan) would replace philanthropic costs in South Sudan if we did not make this grant.19 This spreadsheet contains our reasoning for that value. In short, considerations decreasing our concern that we are crowding out other funding include the fact that PMI does not fund work in South Sudan, that malaria funding to South Sudan from the Global Fund has not increased substantially in recent years,20 and that this is a new location for AMF (so it seems unlikely that the national malaria program or the Global Fund anticipated AMF's support and factored that into their decision-making about how to use malaria funding in the country). On other hand, increasing our concern is the possibility that if we did not make this grant, the Global Fund might reallocate its funding in order to fill the gap, as we have heard that filling gaps for 2023 net campaigns will be a high priority for any reallocated funding.

This value is less well-informed than it would have been if we had waited to make this grant later in AMF's process. For past grants, we have investigated the funding landscape for net campaigns in specific countries by (1) reviewing published data on malaria funding from other major LLIN funders, (2) having conversations with national malaria programs and other major LLIN funders, and (3) reviewing spreadsheets (which AMF asks national malaria programs to complete) that compare each country's malaria funding during the Global Fund's 2018-20 and 2021-23 grant cycles. For this grant, we only took step 1 and did not wait until the point when information from steps 2 and 3 would have been available to us. This is because we decided that the benefit of waiting for this information—increased confidence in the 30% value we use in our cost-effectiveness model—was outweighed by the benefits, described below, of committing funding to this gap early. AMF has agreed to share information related to steps 2 and 3 above when appropriate, as part of the post-grant process described above.

Benefits of committing funding early

As mentioned above, we made this grant at an earlier stage of AMF's process than is typical. We believe that the benefits of this approach are:

1. Making this grant when we did may enable AMF to order LLINs in time to avoid delaying this campaign, if AMF decides to fund LLINs for South Sudan. According to AMF, it can take over a year for LLINs to be ordered, manufactured, shipped to countries, and transported within countries. This means that AMF needs to commit funding to a campaign well in advance of when it is scheduled to occur. Recently, we have heard from multiple sources that LLIN manufacturing and shipping is experiencing longer-than-usual delays, which means that LLINs should ideally be ordered even further in advance of campaigns than they typically would. The LLIN campaign in South Sudan is scheduled to occur from December 2022 through April 2023,21 so we and AMF believe that AMF should commit funding for this campaign as soon as possible. (We note that while we believe that earlier funding will lead, in expectation, to an earlier campaign, it is also possible that some other piece of the planning process for this campaign will cause a delay, such that GiveWell providing earlier funding won't have an effect on the timing of this campaign.)
2. Committing funding early may improve AMF's capacity to identify new funding opportunities. Currently, AMF holds very little available funding, and it has told us that this makes it more hesitant to enter a discussion with a country if it doesn't know that it will be able to fund the country's LLIN needs. According to AMF, it also means that staff time is spent maintaining discussions with countries that AMF is ready to commit to while AMF waits to raise funding. Our intention in making grants at an earlier stage is to improve AMF's confidence in initiating discussions with countries and capacity to identify and consider new funding opportunities.

We describe the risks of this approach below.

Risks and reservations

• Risks of committing funding early – We are recommending the grant earlier in AMF's decision-making process than usual, and we do not have all the information we usually do before making a grant (see below).
• Risk of crowding out other funders – There is a risk that this grant may cause other funders to leave funding gaps for us to fill or to assume their contributions aren't needed when funding gaps do emerge. (see below).
• Lack of track record in South Sudan – AMF has not worked in South Sudan before (see below).
• Uncertain use of future revenue – We don't know how AMF will use the future revenue that is freed up by these grants (see below).

Risks of committing funding early

As mentioned above, we made this grant at an earlier stage of AMF's process than is typical. We believe that the risks of this approach are:

• More uncertainty in our cost-effectiveness analysis – For past grants, we have investigated the funding landscape for net campaigns in specific countries by (1) reviewing published data on malaria funding from other major LLIN funders, (2) having conversations with national malaria programs and other major LLIN funders, and (3) reviewing spreadsheets (which AMF asks national malaria programs to complete) that compare each country's malaria funding during the Global Fund's 2018-20 and 2021-23 grant cycles. For this grant, we only took step 1 and did not wait until the point when information from steps 2 and 3 would have been available to us. Our current CEA for South Sudan also lacks information about what proportions of different net types AMF might purchase to fill this gap (which affects net costs and effectiveness). AMF has agreed to share this information when appropriate, as part of the post-grant process described above.
• Risks inherent in making earlier grants – This grant is an experiment in a new way of working with AMF, in which funding is granted earlier, conditional on AMF agreeing to share information with us and to allow us to weigh in on its decisions. Through the post-grant process we have agreed to, we believe that we will receive the information we need to update our cost-effectiveness model, and that we will have the opportunity to provide input into AMF's decision-making. However, we have not requested a formal approval process and have granted AMF ultimate discretion over which funding gaps it fills. This means that AMF could decide to use this funding to fill a funding gap that we ultimately conclude is below our cost-effectiveness bar.

Risk of crowding out other funders

• Risk of near-term crowding out – We think it is possible that if AMF did not fill this funding gap, the Global Fund might do so by reallocating funding within its broader portfolio. Our understanding is that the first round of such reallocations will occur in early 2022 and that funding gaps for 2023 LLIN campaigns are likely to be a top priority. We could have waited to see whether the Global Fund would reallocate funding to fill this gap before making this grant. However, we decided that the benefits of granting funding early outweighed the benefit of waiting for this information. We have attempted to account for this risk by weighing it heavily in our estimate of the likelihood that Global Fund and/or PMI would replace philanthropic costs (see our discussion of the funding landscape above).22
• Risk of future crowding out – With each of our grants to AMF, we risk communicating to other funders (namely the Global Fund and PMI) and national malaria programs that GiveWell funding will be available to fill future gaps. This may lead them to leave funding gaps for us to fill or to assume their contributions aren't needed when gaps do emerge. This risk only grows as we fund more LLIN campaigns in more countries. In our conversations with national malaria programs and other funders, we have and will continue to communicate that to the extent possible, our goal is for the funding we direct to LLIN campaigns to add to the pool of funding available for those campaigns, rather than to replace funding that would otherwise have been in that pool.

Lack of track record in South Sudan

If AMF decides to fund LLINs for South Sudan, it will be the first time that AMF supports campaigns in the country. Because this is a new location for AMF, AMF will have to establish new connections and partnerships. We see this as riskier than continuing existing partnerships in countries where it already works, and it is possible that issues will arise during AMF's later-stage negotiations with country partners that will prevent AMF from signing an agreement. (If this occurs, we expect to work with AMF to decide where to direct the unused funding.)

In addition, AMF does not yet have a track record from South Sudan that we can assess, which means we must base our expectation of the outcomes of this campaign on past AMF-supported campaigns in other countries. Because AMF has a long track record of purchasing nets for past campaigns that have achieved high coverage, we see this as a lesser concern.

Uncertain use of future revenue

We don't know precisely how AMF will use the future revenue that is freed up by this grant. We project that AMF will raise roughly $34 million over the next year, outside of grants made or recommended by GiveWell. As discussed above, we did not include this revenue in our estimate of the funding available to fill this funding gap. One effect of this grant, therefore, is to free up AMF's future revenue for support to other campaigns, and we don't know which campaigns AMF will choose to support.

Plans for follow up

• We have come to a written agreement with AMF regarding the post-grant process our organizations will follow for this grant. This process will involve the following:
  • Prior to signing an agreement with a country, AMF will share with GiveWell new information that becomes available about the relevant funding gap.
  • GiveWell will use this information to update our cost-effectiveness model for the relevant funding gap, giving us an updated estimate of its cost-effectiveness.
  • AMF will alert GiveWell when AMF is ready to sign an agreement with a country.
  • GiveWell will share input about the ultimate cost-effectiveness of this funding gap and whether or not we are in favor of AMF filling it. (Note that while AMF has agreed to take this input into consideration, we have not requested a formal approval process and have granted AMF ultimate discretion over which funding gaps it fills.)
• We will have monthly calls with AMF to discuss its work.
• We will track whether campaigns occur on schedule or are delayed.
• We will review the registration data and post-distribution monitoring data collected for these campaigns to understand what proportion of LLINs reach their intended recipients, are used by those recipients, and remain effective while they are in use.

Internal forecasts

Confidence	Prediction	By time
60%	AMF will sign an agreement with partners in South Sudan.	The end of 2023
60%	After updating our CEA with the new information we receive from AMF, we will conclude that the overall cost-effectiveness of this grant exceeded 10x cash.	The end of 2024
50%	Campaigns in South Sudan will occur, at the latest, by three months after the estimates AMF provided.23	Quarter 1 of 2025

Our process

Our grant investigation relied heavily on our prior work modeling the cost-effectiveness of net campaigns supported by AMF and our relationship with AMF and knowledge of its work. To generate a cost-effectiveness estimate for South Sudan, we used our existing cost-effectiveness model for LLIN campaigns and updated various parameters (see above).

We had several conversations with AMF regarding the post-grant process our organizations will follow for this grant and came to a written agreement, described above.

For internal review, a Program Officer and a Senior Research Associate who were not otherwise involved in the grant investigation gave feedback on the plan for investigating the grant.

Sources

Document	Source
GiveWell blog, "Revisiting leverage," 2018	Source
GiveWell, 2022 GiveWell cost-effectiveness analysis — version 5	Source
GiveWell, "Against Malaria Foundation — PMI-supported states in Nigeria, 2023 (January 2022)"	Source
GiveWell, "Against Malaria Foundation," 2021	Source
GiveWell, "Insecticide Resistance and Malaria Control," 2020	Source
GiveWell, "Mass Distribution of Long-Lasting Insecticide-Treated Nets (LLINs)," 2021	Source
GiveWell, "Our Top Charities," 2021	Source
GiveWell, "Recommendation to Open Philanthropy for Grants to Top Charities," 2019	Source
GiveWell, GiveWell's room for more funding analysis for AMF [January 2022]	Source
GiveWell, GiveWell's room for more funding analysis for AMF [May 2022]	Source
GiveWell, Global Fund malaria funding, 2018-20 vs. 2021-23	Source
Givewell, Likelihood of crowding out GFATM/PMI (April 2022 countries under consideration), 2022	Source
GiveWell, Likelihood of crowding out GFATM/PMI (October 2020)	Source
GiveWell, Tracking of Global Fund spending in countries receiving funding from AMF [2021] - redacted	Source

• 1

  See here.
  

• 2
  • Six of ten states received net campaigns in 2018-2020: Eastern Equatoria, Western Equatoria, Lakes, Western Bahr el Ghazal, Northern Bahr el Ghazal and Warrap states. See here.
  • Four remaining states were prioritized for this grant cycle: Central Equatoria, Jonglei, Upper Nile and Unity states. See here.

• 3

  See here.

• 4

  "This is a co-funding partnership with non-net costs (shipping, pre-distribution, distribution) funded by the Global Fund" @Against Malaria Foundation, "AMF agrees to fund 5.7 million nets for distribution in South Sudan in Q1 2023," July 2022@

• 5

  See this spreadsheet for AMF's summary of what happened in the last grant cycle, "without AMF funding" section, years 2018-2020.
  

• 6See this adjustment here.

• 7See our estimates of insecticide resistance levels in South Sudan here, and see below for more information on how we calculated it.
  
• 8"PBO nets are more expensive than standard LLINs." @GiveWell, "2022 Cost-Effectiveness Analysis Changelog," section "Change 6: Updated the cost per LLIN in our CEA of the Against Malaria Foundation (AMF) for Uganda."@
• 9See our estimates for insecticide resistance levels in South Sudan here, and our estimated proportion of the country where PBO nets are more effective than standard LLINs (due to insecticide resistance) here.
  
• 10

  We aim to get feedback on our grantmaking from stakeholders other than our grantees, such as government officials, other implementers involved in delivering the program, and other organizations working in the relevant context. The goals of these conversations are to learn more about the context in which a program will be delivered, to confirm the need for additional support of the program, and to seek feedback on the activities that a potential grant to support the program would enable.

• 11

  AMF, Funds status, May 5, 2022 (unpublished).

• 12

  We used the same process to update this room for more funding analysis as we did for our recent grant to AMF for PMI-supported states in Nigeria. See here for more details.

• 13

  See this parameter here

• 14

  Internal process notes (unpublished).

  See more about how we calculated our insecticide resistance adjustment for South Sudan here.

• 15

  See our review of AMF for more information on post-distribution monitoring.

• 16

  See this parameter: 'Misappropriation without monitoring results'.

• 17

  See this parameter here in our cost-effectiveness analysis.
  

• 18

  See this parameter here in our cost-effectiveness analysis.

• 19

  See here in our cost-effectiveness analysis.

• 20

  See here for more information.

• 21

  Campaigns are projected to occur from December 2022 through April 2023. "South Sudan 2023 - Dec 2022 to April 23" AMF, Email to GiveWell, Apr 11, 2022 (unpublished).

• 22

  See this risk here.

• 23

  Campaigns are projected to occur from December 2022 through April 2023. "South Sudan 2023 - Dec 2022 to April 23" AMF, Email to GiveWell, Apr 11, 2022 (unpublished).