Against Malaria Foundation (AMF) - 2014 Review

We have published a more recent review of this organization. See our most recent report on AMF.

Published: November 2014

Summary

What do they do? AMF provides funding for long-lasting insecticide-treated net distributions (for protection against malaria) in developing countries.

Does it work? There is strong evidence that distributing nets reduces child mortality and malaria cases. AMF has relatively strong reporting requirements for its distribution partners and provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity. AMF has begun building a track record of funding and tracking fairly large-scale ($1 million or more) distributions.

What do you get for your dollar? We estimate that the cost to purchase and distribute an AMF-funded net is $5.30 in Malawi and, very roughly, $7.50 in DRC (the two countries that AMF has worked most extensively in). The numbers of malaria cases prevented and lives saved are a function of a number of difficult to estimate factors, which we discuss in detail below.

Room for more funds? AMF currently holds approximately $6.8 million in funds that it could allocate to future distributions; this includes approximately $2 million committed to a distribution scheduled for 2017. We believe AMF could conceivably absorb as much as $25 million more in the next year. Considering AMF’s track record of finalizing distributions and the past scale of its operations, we hope to see AMF receive an additional $10 million.

AMF is recommended because of its:

• Focus on a program with a strong track record and excellent cost-effectiveness.
• Strong processes for ensuring that nets reach their intended recipients and monitoring whether they are used over the long-term.
• Standout transparency and commitment to self-evaluation.
• Room for more funding – we believe AMF will be able to use additional funds to deliver additional nets.

Major unresolved issues include:

• Though we now have increased confidence in AMF's ability to find new partners and countries to work in, there is still significant uncertainty as to its ability to scale up.
• AMF has completed only a small number of large distributions, the type it aims to focus on in the future, and all completed distributions have been with a single partner. Its track record of collecting the data it seeks from these distributions is good so far but limited due to the small number of distributions. Many of its upcoming distributions are in DRC and our impression is that DRC is a particularly difficult place to work.
• The best evidence for nets was collected before they were widely used and there is some evidence that mosquitoes have since adapted to the insecticide used in nets, possibly making them less effective. Further research is needed to determine how important of a problem this may be.

Our review process

We began reviewing AMF in 2009. Our review has consisted of:

• Reviewing AMF's public records for each of its net distributions and other documents AMF has shared with us.
• Extensive communication, including several meetings at AMF's London headquarters with AMF Founder Rob Mather, to discuss AMF's methods and funding needs.
• A visit to AMF's distribution partner organization, Concern Universal, in Malawi in October 2011 (notes and photos from this visit).
• Reviewing materials from AMF’s distributions in Malawi and the Democratic Republic of the Congo (DRC), including pre-distribution registration data, distribution reports, and data from post-distribution follow up surveys.
• Conversations with Peter Sherratt AMF's Executive Chairman; Don de Savigny, a member of AMF's Malaria Advisory Group, and other individuals (who requested to remain anonymous) familiar with AMF's work and its attempts to finalize distributions.

All content on AMF, including past reviews, updates, blog posts and conversation notes, is available here.

A few key terms for this review

• A net refers to any bednet intended to block mosquitoes, whether treated with insecticide or not.
• An ITN is an insecticide-treated net. As discussed at our write-up on ITNs, distribution of ITNs has been rigorously studied and associated with declines in under-5 mortality and overall malaria cases.
• A LLIN is a long-lasting insecticide-treated net, intended to serve as an effective ITN for 4-5 years (details at our write-up on ITNs). The nets distributed by AMF are LLINs,1 and (as noted in our write-up on large-scale ITN distributions) most large-scale donor-financed malaria control today utilizes LLINs.

What do they do?

AMF provides long-lasting insecticide-treated nets (for protection against malaria) in bulk to other organizations, which then distribute them in developing countries.

Prior to November 2011, AMF contributed LLINs to small-scale distributions. Since then, it has de-emphasized small-scale distributions and it expects larger-scale distributions to be the primary way in which it distributes LLINs in the future (more below).

As of November 2014, AMF had completed or was in the process of completing four large-scale distributions with two distribution partners for a total of 1.3 million LLINs (2.7 million total since its inception),2 and had signed agreements to distribute an additional 2.3 million LLINs in 2014-2017.3

Our most thorough review of AMF's processes was done for its first large-scale distribution in Ntcheu, Malawi, and much of the below discussion relates to that distribution. Since then, we have followed AMF progress and asked for updates on how its processes have changed rather than consider each distribution in depth.

AMF's role

AMF's role in LLIN distributions is to:4

1. Identify countries with funding gaps for LLINs.
2. Find distribution partners to carry out LLIN distributions and agree on expectations for the distribution, including who pays for costs other than the purchase price of LLINs and what information will be collected and shared with AMF.
3. Purchase LLINs and have them shipped to the distribution partner.
4. Work with distribution partners to collect reports on the distribution and follow up surveys. AMF posts these reports on its website.

Process for distributions

Selecting locations for distributions

• When selecting locations for future distributions, AMF told us it consults a series of sources as it believes there is no one reliable resource with up-to-date information to determine where nets are needed. Sources it consults include the Alliance for Malaria Prevention's (AMP's) list of countries with significant net gaps, other malaria control funders, in-country technical advisors, the relevant National Malaria Control Program (NMCP), implementing organizations and the African Leaders Malaria Alliance.5

  As an example, we discuss the case of its distribution in Ntcheu, Malawi in 2012. For that distribution, AMF received estimates of country-level gaps from the Alliance for Malaria Prevention (AMP) and from the Malawi National Malaria Control Program (NMCP). Rob Mather, AMF's founder, told us that, although AMF did not have a high level of confidence in the net gap numbers presented by the NMCP, nor those it was sent by the Global Fund, it did have a high level of confidence that the net gap was significantly higher than the 250,000 LLINs AMF was considering providing.6 Note that the materials AMF has sent us from the period prior to its decision to provide funding for nets in Malawi do not include quantified estimates of Malawi's LLIN gap,7 though they do make it clear that there is a gap,8 and later reports (from after AMF had decided on Malawi) show a quantified gap for the country.9

  Note that we did not repeat this analysis for distributions that AMF funded after Ntcheu, Malawi in 2012 or for the distributions it is considering as of November 2014.

• AMF told us that it also consults other malaria control funders, implementing organizations, and the relevant National Malaria Control Program (NMCP) to verify estimates of how many additional LLINs are required to reach full coverage in a country.10 We have seen some correspondence between AMF and the NMCP in Malawi which took place prior to AMF’s decision to fund a distribution in Malawi.11 AMF also shared with us correspondence that it had with representatives of the NMCP in Togo, with regard to potential future distributions.12
• As AMF investigates countries with existing net gaps, it also looks into organizations working within those countries that could serve as distribution partners.13 AMF's distribution partners must have the capacity and willingness to carry out a distribution that meets AMF's requirements (more below).14 Before deciding to work with Concern Universal for the Ntcheu 2012 distribution, AMF requested a distribution proposal, including information on malaria risk in the distribution area (including data on reported malaria cases), other net distributions in the area, coordination with government, and how the distribution would be carried out.15 We have seen similar distribution proposals, which include information on malaria risk and other sources of nets in the area, for distributions in Balaka in 2013, Dedza in 2014, and North Idjwi Island in 2014. We have not yet seen distribution proposals for the in progress (as of November 2014) distribution in Kasaï Occidental or for any future distributions (full list in this spreadsheet).

Requirements for distribution partners

AMF aims to work with distribution partners who will agree to the following requirements. We detail what evidence we have seen that past distributions have complied with these requirements in this spreadsheet.

• Pre-distribution registration: AMF requires that its distribution partners carry out a pre-distribution registration survey (PDRS) in all areas that LLINs will be distributed to establish the number of sleeping spaces and LLINs in a condition such that they have at least one year of remaining use. (In the case that such data already exist, AMF and the partner discuss whether the data is reliable and up-to-date, or whether a new PDRS should be conducted.16 ) We have seen results from both of AMF's completed large-scale distributions: full data from its Ntcheu 2012 distribution and sample results from its Balaka 2013 distribution.17
• Distribution monitoring: An additional requirement is to provide AMF with photos and videos from the distribution and a report on the distribution, including problems encountered.18 AMF's distribution partner, Concern Universal, provided photos and/or videos for all but 2 of the 14 “sublocations” for its Ntcheu 2012 distribution.19 It did not collect photos or video for AMF's other completed large-scale distribution, in Balaka in 2013. AMF expects to receive photos and video from future distributions.20 Concern Universal has provided detailed distribution reports for the three distributions it has completed.
• Post-distribution follow-up: AMF requires partners to conduct follow-up surveys in 5% of households at 6-month intervals for 3-4 years after a distribution.21 We have seen post-distribution survey results from the Ntcheu 2012 distribution (6, 15, and 24 months post-distribution) and the Balaka 2013 distribution (6 months post-distribution). Results below. Results are not yet expected from other distributions.

Previously, AMF expected to collect malaria case rate from the regions in which it funded LLIN distributions. AMF has asked Concern Universal to provide monthly malaria case rate data from all health centers in the three districts in which it has completed distributions: Ntcheu, Balaka and Dedza. AMF has asked for this data for 12 months preceding and 4 years following the distribution.22 AMF has shared data from Ntcheu through through December 2012. We do not believe this data is high quality. In August 2013, it told us that it planned to publish an audit of the data quality by the end of September 2013.23 Later, it told us that it expected to post data in early 2014.24 In November 2014, AMF told us that it expects to share more recent data by the end of January 2015. AMF told us it has been waiting to collect data through to September 2014 to be able to review data for a full year and compare it with the prior year due to changes in the way data is recorded in the health centers.25 AMF has not included in its Agreement with its partner in DRC, IMA World Health, a requirement to collect and share malaria case rate data. Discussions with IMA made it clear the quality of the malaria case rate data was not considered to be reliable in the majority of health centers. IMA told AMF that it believes there will be a some health centres across the distribution area that can provide historic and future malaria case rate data that is reliable. AMF and IMA have agreed to return to this issue in early 2015.26

AMF looks for partners who, in addition to agreeing to the above requirements, are clear and responsive in their communication with AMF.

Prior to the Balaka 2013 distribution, AMF asked all distribution partners to use their own funds or to find another funder for all non-net costs of the distribution.27 More recently, it has paid for some of these costs in certain distributions. AMF told us that it considers funding non-net costs in cases where (a) non-net costs are not covered by other partners, and (b) AMF feels confident that its distribution partners will manage and report on spending well.28 AMF has paid for all non-net costs for the Balaka 2013 and Dedza 2014 distributions and the 24- and 33-month post-distribution check-ups for the Ntcheu 2012 distribution, and agreed to pay for all non-net costs for the Dowa 2015, Balaka 2015, Ntcheu 2015, and Dedza 2017 distributions with Concern Universal, and shipping costs for the Nord Ubangi 2015 distribution.29 It is our understanding that AMF aims to primarily fund distributions where other partners cover non-net costs, which could limit its capacity to distribute nets, though AMF has shown some flexibility on this issue recently.30

Does it work?

On a separate page, we discuss the general evidence behind distribution of LLINs. We conclude that there is strong evidence that these distributions can be expected to reduce child mortality and malaria cases.

When evaluating the effectiveness of an LLIN distribution organization, we seek to answer the following questions:

• Are LLINs targeted at people who do not already have them? AMF requires partners to complete pre-distribution surveys to determine the number of nets required, if reliable and recent survey data are not available. We are not fully satisfied with the pre-distribution surveys' ability to ensure that people aren't under-reporting their existing LLIN ownership in order to get more LLINs, but believe that reasonable measures are being taken to address this issue.
• Do the LLINs reach the intended destinations? Distribution partners are asked to provide pictures and videos of LLINs being handed out (a requirement that has been complied with in one of two completed large-scale distributions), and report on how the distribution went, including problems encountered. Post-distribution surveys provide additional evidence that LLINs reached their destinations.
• Are LLINs targeted at areas with high rates of malaria? AMF seeks out distribution partners in countries that are known to have high rates of malaria.
• Do those who receive the LLINs install them in their homes properly? Do they utilize them consistently over the long term? AMF requires partners to conduct follow-up surveys at 6-month intervals, and for a period of up to 3-4 years. Results are available from 6, 15, and 24 months after the Ntcheu 2012 distribution and 6 months after the Balaka 2013 distribution. We have little data to compare these results to, but they seem consistent with high, proper usage of nets for an extended period after distribution.
• Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF? The limited evidence we have suggests the former.

Details follow.

Are LLINs targeted at people who do not already have them?

AMF requires that distribution partners conduct household-level pre-distribution registration surveys (if recent, reliable household-level data are not available) to determine the number of existing LLINs with at least one year of remaining use and the number of nets required in each household.31 This process includes door-to-door surveys of every household in the targeted district by government health workers. We have seen census results from both of AMF's completed large-scale distributions: full data from its Ntcheu 2012 distribution and sample results from its Balaka 2013 distribution.32

In Malawi, Concern Universal has then conducted village meetings in which its staff read off household names and the number of LLINs that have been allocated to each household; households then indicate if any errors have been made. We observed one such village verification meeting during our visit to Malawi in October 2011.33 In DRC, IMA World Health is not conducting village verification meetings because LLINs are being distributed during the census: health workers go door-to-door checking how many LLINs each household requires and provide the LLINs at the same time.34

Concern Universal conducted pre-distribution registration surveys for the Ntcheu 2012 and Balaka 2013 distributions, and AMF shared the full results from the Ntcheu census and sample data from the Balaka census.35 Based on GiveWell's suggestion, Concern Universal also decided to send its staff to random households prior to the Ntcheu 2012 distribution to spot-check the accuracy of its data.36 After the first two weeks of spot checks, the distribution partner reported that it had not found any discrepancies between the number of nets community members said they needed during the verification meeting, and the number of nets found to be needed during the subsequent household visit.37 AMF has told us that spot checks are now carried out with all distributions.38 We have not seen results from these spot checks.

We believe it is possible that some households may over-report the number of LLINs they need. Cases of households hiding nets in order to receive extra LLINs have been observed in universal campaigns in Senegal and Nigeria.39 That said, we believe that the steps AMF and its partners are taking to minimize the ability of community members to request more nets than they need are reasonable.

In Malawi, household surveys conducted by the National Malaria Control Programme (in districts not receiving AMF-funded LLINs) have found "much larger LLIN gaps than first thought,"40 and it is not clear to us what the cause of this discrepancy is.

Do LLINs reach the intended destination?

AMF requires that partners provide photos and videos of distributions.41 Concern Universal provided photos and/or videos for all but 2 of the 14 “sublocations” in Ntcheu in which it distributed LLINs in 2012.42 It did not provide photos and/or video for the Balaka 2013 distribution, the other large-scale distribution that AMF has completed. AMF told us that it expects photos and video from future distributions.43

During its Ntcheu 2012, Balaka 2013, and Dedza 2014 distributions, Concern Universal provided weekly reports on the number of LLINs distributed, the regions which were covered, and problems encountered in reaching particular villages or individuals.44 As we report in our March 2012 update, the Ntcheu 2012 weekly reports note challenges including attempted thefts, double registrations and logistical problems.45 Concern Universal provided a similar level of detail on challenges encountered in the Balaka 201346 and Dedza 2014 distributions.47 These reports increase our confidence that Concern Universal is aware of potential problems and has a system in place to address them.48

Concern Universal has provided weekly reports of its own accord (AMF does not require such reports). AMF does ask partners to provide summary reports on how distributions have gone after they are completed, and we expect to see such reports from future distributions.49

For the Kasaï Occidental (2014) distribution, AMF's distribution partner, IMA World Health is piloting the use of smartphones to record household data, including GPS coordinates, for registration and LLIN distribution. AMF hopes that the technology can be used to increase accountability, transparency, and data accuracy.50 IMA is funding this work.51 AMF has told us that it will publish the data collected (anonymized for privacy).

Post-distribution surveys, discussed below, provide a further check on whether LLINs have reached their intended destinations. The surveys that have been completed to date have found high coverage rates (90% in the two completed 6-month surveys).

Are LLINs targeted at areas with high rates of malaria?

At the highest level, AMF appears to exclusively target countries with known malaria risk.52 In its distribution proposal for its Ntcheu 2012 and Balaka 2013 distributions, Concern Universal reported the number of malaria cases in the proposed distribution area over a twelve month period preceding the distributions.53 We have not seen similar data for Concern Universal's Dedza 2014 or Dowa 2015 distributions or for the areas in DRC where AMF has or will provide LLINs. AMF has told us that it has received malaria case rate data for Dedza and Dowa districts in Malawi (which it has not yet shared publicly) and that it does not have data from Kasaï Occidental in DRC because much of the data from that area is considered to be unreliable. AMF told us that it may in the future collect and publish data from a subset of health centers in Kasaï Occidental that have reliable data.54

Do those who receive the LLINs install them in their homes properly? Do those who receive the LLINs utilize them consistently over the long term?

AMF requires partners to conduct follow-up surveys at 6-month intervals for a period of 3-4 years to determine whether LLINs are present, are hung, and what condition they are in.55 Results are available from 6, 15, and 24 month surveys following the Dedza 2012 distribution and 6 months following the Balaka 2013 distribution. Details of the methodology used in these surveys are available in our August 2014 update on AMF (see the description of the Ntcheu 15 and 24 month surveys). We discuss the methodology of the Balaka 6 month survey in this footnote56 (details were not yet available when we published the August update). We believe the methodology used was reasonably high quality and that results are fairly, though not fully, representative of all households that received AMF-funded LLINs. We note that have not seen technical details at the level of a high quality academic study on how the surveys were carried out,57 and we therefore have not been able to fully vet the results.

Results from follow-up surveys

Data sources in footnote.58

Definitions:

• Nets hung: We have not seen a precise definition of this term (e.g., must the surveyor observe the net fully covering the sleeping space, hung in a position that it could conceivably cover the sleeping space, etc.). AMF told us that interviewers are asked to observe whether nets are hung by entering interviewees' houses (rather than simply asking interviewees if they are hung).59
• Very good condition: LLIN has fewer than 2 holes of less than 2 cm in size.60
• OK condition: LLIN has fewer than 10 small holes.61
• Poor condition: LLIN has more than 10 small holes or has one or more large holes.62
• Worn out: Survey reports from Concern Universal did not provide a definition of this term. AMF told us that an LLIN is considered worn out if it has multiple large holes and the LLIN is unrepairable, such that it would not provide protection against mosquitos.63
• Used correctly: According to a survey form from 2012, interviewers are asked to "ask the householder to demonstrate how the nets are used at night" and to then select yes or no to the question "are the nets being used correctly."64 AMF told us that the procedure has not changed in more recent surveys.65

79% of individuals in households that received LLINs sleeping under a useable LLIN (at 24-month follow up in Ntcheu) is roughly in line with usage rates of the trials of bednet efficacy documented in our page on Long Lasting Insecticide Treated Nets, in which small-scale studies of bednet efficacy had net usage rates generally in the 60%-80% range.66

The "decay model" we use to anticipate the lifespan of LLINs assumes that 80% of LLINs are effective at 24 months.67 AMF found that 77% LLINs in Ntcheu were in "very good" or "OK" condition at 24 months. LLINs appear to have decayed more quickly in Balaka, where 70% of nets were in "very good" condition at 6-months (and 25% in "OK" condition), compared with 99% in "very good" condition at 6-months in Ntcheu. It is not clear to us whether these rates of decay are comparable to the assumptions in the decay model, in part because it is not clear whether the definitions of useable are similar in the model and AMF's data. We may do further analysis to more directly compare the model and AMF's data in the future.

Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF?

For the Ntcheu 2012 distribution in Malawi, AMF received data from the National Malaria Control Program on population figures and LLINs already distributed by district, as well as LLINs that had been committed to the country by other funders. AMF estimated, in September 2011, that an additional 909,586 nets were needed to achieve universal coverage in the country,68 of which AMF contributed 268,240.69 We have not done a similar analysis for later distributions.

We would guess that AMF is increasing the total number of LLINs because, across Africa, there are substantial funding gaps for LLINs (more below) and because our impression from following AMF's progress over time is that, due to AMF's more limited funding and, perhaps, greater data requirements, governments often seek funding first from larger funders (particularly the Global Fund to fight AIDS, TB, and Malaria) and then may ask AMF to fill gaps. However, we have low confidence in this guess, and note that countries are able to choose how they allocate Global Fund grants among malaria interventions (including LLINs, treatment, and diagnosis), so the availability of funding for LLINs from AMF could cause countries to allocate less Global Fund resources to LLINs.

Possible negative or offsetting impact

• Do donated nets displace government health funding? This could be a concern if government funding that otherwise would have been spent on LLINs is spent on other, less worthwhile budget items. We have little sense of how important a concern this is in AMF's case. In the case of the 2012 distribution in Malawi, all anticipated nets were expected from international donors, and the total supply appeared to be still insufficient for country-wide universal coverage.70
• Will insecticide-treated nets continue to be effective? As discussed at our report on insecticide-treated nets, there is strong evidence for the effectiveness of this intervention, however the best evidence for the intervention was collected before LLINs were widely used and there is some evidence that mosquitoes have since adapted to the insecticide used in LLINs, possibly making them less effective. In 2012 we reviewed the evidence on the state of insecticide resistance. We concluded, "We're very concerned about insecticide resistance. It appears that relatively little is known about the extent, causes, and control implications of resistance; that resistance could substantially affect (or even negate) the effectiveness of malaria control… That said, this issue doesn't change our bottom line that LLIN distribution is a highly cost-effective intervention… There is strong evidence that LLINs reduce malaria and save lives and only preliminary/suggestive/mixed evidence that insecticide resistance may reduce their impact."
• Do free LLIN distributions distort incentives for recipients or distort local markets for nets? As discussed at our report on insecticide-treated nets, we feel that there is a reasonably strong case for distributing LLINs freely rather than selling them at market (or even below-market) prices.
• Could distribution of LLINs be inequitable and unfair, causing problems in the targeted communities? We feel that AMF's process for determining needs for LLINs at the household level is well-suited to ensuring that all who want LLINs receive them.
• Does AMF divert skilled labor from other areas? Net distribution is conducted by low-level government health staff in partnership with the staff of AMF's partner NGO.71 According to AMF's partner in Malawi, government health staff are normally involved in activities such as disseminating health-related information, reporting on levels of stunting and disease, carrying out immunization campaigns, and providing nutrition support.72 We don't know the extent to which net distribution reduces their ability to complete other duties, though we note that net distributions are generally completed within a few days in each local area.

What do you get for your dollar?

Cost per LLIN distributed

Estimate based on data from AMF

Using budgets and estimates provided by AMF, we estimate that the total cost to purchase, distribute and follow up on the distribution of an AMF-funded LLIN is about $5.30 in Malawi and very roughly $7.50 in DRC. We focus on these two countries because they are where AMF has completed recent distributions or is planning distributions.

Note that we prefer to include all costs incurred to carry out a project, not just those that the charity in question pays for itself. We believe that this gives the best view of what it costs to achieve a particular impact (such as saving a life), and also avoids the lack of clarity and complications of leverage in charity. Our estimate of the cost to distribute an AMF-funded LLIN aims to include both costs AMF pays and costs paid by others.

Full details in this spreadsheet.

The estimate for Malawi uses:

• The purchase price per LLIN, calculated from AMF's total LLIN budget and the number of LLINs distributed.
• Actual costs for the distribution and 6-month follow-up survey in Balaka, as reported by Concern Universal, budgeted costs for later follow-up surveys in Balaka, and budgeted costs for all of the non-net costs for the Dedza (2014) and Dowa (2015) distributions.73 AMF told us that the non-net budget includes all monetary costs incurred by Concern Universal, as well as the use of vehicles and managerial staff time;74 it may exclude non-monetary costs such as use of office space.
• An estimate of non-monetary costs incurred by Concern Universal and local governments, based on estimates provided in 2012 by Concern Universal for its distribution in Ntcheu province.75
• An estimate of AMF's organizational costs per net, including an estimate of the value of donated services and volunteer time. Because these costs generally do not vary with the number of LLINs distributed and remain roughly constant on a per year basis, the per LLIN estimate is highly dependent on the time period used and the number of LLINs distributed in that period. We estimate AMF's organizational costs as $0.82 per LLIN for 2013-2015. This estimate assumes that AMF will complete the distributions it currently plans for that period and does not account for any additional distributions.76

We shared our analysis with AMF before publishing and AMF wrote the following in response.

For DRC, the estimate is much rougher. It is based on costs from the Kasaï Occidental (2014) distribution, specifically:

• The purchase price per LLIN, calculated from AMF's total LLIN budget and the number of LLINs distributed.
• An estimate of shipping costs. We do not have an estimate of shipping costs for this distribution, so we have used a rough estimate based on the Nord Ubangi 2015 distribution (also in DRC).
• An estimate of pre-distribution and distribution costs from incomplete data from AMF's distribution partner IMA World Health. This estimate is 2.5 times as high as the Malawi estimate. This may be explained by a considerably more difficult operating environment in DRC,77 or our data for DRC may simply be of lower quality.
• We do not have data for the cost of follow-up surveys in DRC. We have assumed that the cost of these surveys will be about 2.5 times as expensive as in Balaka, consistent with our estimate of the increased costs to distribute LLINs in DRC.
• We have used the same estimate of AMF organizational costs as we used for Malawi.

Global estimate

The Roll Back Malaria Partnership, which estimates funding gaps for LLINs across all African countries,78 uses an estimate of $3.30 to purchase a LLIN and $2.50 for all non-net costs of a distribution, for a total of $5.80 per LLIN.79 This assumption is used in cases where country-specific data is not available. The Co-Chair of the Roll Back Malaria Harmonization Working Group told us that non-net costs can vary significantly by country and that non-net costs in DRC may be more than $5 per LLIN.80 We have little information on how reliable this estimate is likely to be. Broadly, this estimate is consistent with the data we have seen from AMF. We have used estimates based on data from AMF in our cost per life saved calculations, because we have more information on their reliability.

Cost per life saved

Using $5.30 as the total cost per net in Malawi and $7.50 for DRC, we estimate the cost per child life saved through an AMF-funded LLIN distribution at about $3,340.81

This does not include other potential benefits of LLINs (non-fatal cases of malaria prevented, prevention of deaths in age groups other than under-5 year olds, prevention of other mosquito-borne diseases, etc.). Full details at our report on mass distribution of LLINs.

As a general note on the limitations to this kind of cost-effectiveness analysis, we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

Room for more funds?

In November 2013, we wrote, in this blog post, that we believed AMF did not have room for more funding because it had raised over $10.6 million due to GiveWell's recommendation since 2011 and had not spent the bulk of these funds. Since then AMF has completed or signed agreements for several large distributions and we now believe that AMF has room for more funding in the next year. This change is primarily due to:

1. A year ago, AMF had completed two large-scale distributions, both of which were with a single distribution partner, Concern Universal in Malawi. It had not signed agreements with other partners. As of November 2014, AMF has signed agreements for two large distributions with IMA World Health in the DRC (one of which is in progress), helping to demonstrate its ability to finalize distributions with other partners in other locations.
2. AMF has spent or committed a large portion of the funds it raised prior to 2014. As of November 2014, AMF held $4.9 million in uncommitted funds, of which it raised $2.65 million in 2014. Of committed funds, $1.9 million are for expenses that AMF will incur more than a year from now, and AMF might consider reallocating these funds in the near term with the expectation of raising enough to cover these commitments by the time they are due. This brings the total available funds to $6.8 million.82
3. AMF's pipeline for potential distributions includes several distributions with established partners. Given AMF's history with these partners, we would guess that there are fewer barriers to signing agreements for additional distributions. Funding all such distributions would require AMF to raise significantly more funds. AMF may also have opportunities to fund distributions with new partners.

Though we now have increased confidence in AMF's ability to finalize large distributions, there is still significant uncertainty as to its ability to find new partners and countries to work with. To find a sufficiently large distribution often requires negotiating with the national malaria control programs of countries in sub-Saharan Africa, which we perceive to have some discretion in which funders they work with, and which we perceive to be choosing funders based on a variety of factors including size and reporting requirements. Because AMF is able to fund only a relatively small piece of a given country’s distribution, but has substantial reporting requirements, there may be fundamental reasons for governments to prefer other funders. We believe that this is the most important and fundamental explanation for AMF's difficulties in finalizing distributions in the past. We believe that other factors may have also played a role. We discuss this topic in detail in this blog post.

AMF told us that it is not in conversations with any major funders to help fill its funding gap.83

Pipeline of potential distributions

As of November 2014, AMF was considering four main types of potential distributions:

1. Additional distributions with existing distribution partners. Due to ongoing relationships and probable need for additional LLINs, we believe that it is likely that AMF will have opportunities to fund additional distributions with these groups. AMF is considering funding up to $17.2 million worth of LLINs in 2015.
   • Malawi will be carrying out a nationwide LLIN distribution in 2015 and may need additional LLINs.84 AMF is considering funding 600,000 to 1 million LLINs85 for Concern Universal to distribute in this additional district. This would likely cost $2.4-4.0 million,86 including non-net costs (we assume AMF will fund non-net costs as it has for recent Concern Universal distributions).
   • AMF is considering two potential distributions in DRC, where it believes there is a significant funding gap for LLINs in 2015. One distribution would be for 1 to 2 million LLINs and the other would be for 3 million LLINs.87 Purchasing 4 million LLINs and paying for shipping (as AMF has agreed to do for its distribution with IMA in early 2015) would cost about $13.2 million.88
2. Distributions of moderate size (60,000 to 100,000 LLINs) in three other countries. AMF is considering funding 60,000 LLINs in one country and 100,000 LLINs in two other countries,89 for a total cost of about $780,000.90 AMF told us that it would be able to sign agreements for these distributions quickly if it chose to do so and that the reason it has not is because it expects to use its funds for larger distributions instead. If it is unable to sign agreements for larger distributions, it may go ahead with these more moderately sized ones. It will likely agree to fund the distribution for 60,000 LLINs because the distribution would involve a pilot project to test the use of smartphone technology for tracking LLIN distributions, which AMF is interested in learning more about.91
3. Large distributions (1-4 million LLINs) in 2-3 other countries. AMF is in fairly early stages of discussion with governments in two countries to fund 1 to 2 million LLINs ($3-6 million) and up to 4 million LLINs ($12 million), respectively, in 2015.92 AMF does not expect to pay for non-net costs in these countries.93 More speculatively, there may be an opportunity for AMF to fund up to 2 million LLINs in a third country in 2015 and up to 2 million more in 2016-2017.94 AMF has not yet begun conversations with the government of the third country.95
4. Funding research on insecticide resistance. AMF is considering spending approximately $300,000 over three years to fund a UK university to study insecticide resistance in conjunction with AMF's distribution in Nord Ubangi, DRC in 2015. It may also fund a similar project for its Dowa, Malawi distribution in 2015, at a cost of approximately $200,000. This research would (a) establish baseline rates of mosquito susceptibility to the insecticide used on most LLINs and (b) compare the effectiveness of LLINs used in most distributions and a newer type of LLIN that is more expensive but may be more effective at killing mosquitoes.96

If AMF were able to finalize all of the distributions in its pipeline, with the exception of the speculative distribution for 4 million LLINs and the moderately-sized distributions (which it would likely drop in favor of the larger ones), it would need nearly $36 million.97 We believe it is unlikely that all discussions would be successful. AMF currently holds approximately $6.8 million in funds it could allocate to future distributions. We believe AMF could conceivably absorb as much as $25 million more in the next year. Considering AMF’s track record of finalizing distributions and the past scale of its operations, we hope to see AMF receive an additional $10 million. We note that distributions can take over a year to plan98 and funding that AMF receives in the next year may not be used or committed in that year. We expect to continue to follow AMF's progress of building a track record of finalizing and carrying out distributions over the next year.

Effect of additional funds

AMF has told us that the number of opportunities it considers at one time is limited by its funds and that the effect of having additional funds is largely to increase the size of the distributions it considers, rather than increasing the number of distributions it considered. The type of opportunities it will consider depends on how much funding it has available. If it were to raise an additional $10 million in the next year, it would likely continue to primarily consider distributions of 1 to 2 million LLINs. At this size, the failure of one or two negotiations would leave AMF with other possibilities in its pipeline, but the distribution size is still large enough to attract the interest of the type of partner AMF would like to work with. Above $10 million in additional funds, AMF would be able to increase the size of the distributions it considers. Above $20 million additional funds, AMF would primarily consider distributions of about 5 million LLINs, which would give it more leverage in its negotiations and discretion over which partners to work with. AMF told us that it has had several conversations about potential distributions that have not moved forward because AMF was unable to consider a distribution of sufficient size.99 We do not have more details on these conversations, particularly on whether other issues contributed to the discussions not going forward.

Global funding gaps for LLINs

The Roll Back Malaria Partnership's Harmonization Working Group estimates funding gaps for LLINs across all African countries.100 It most recently published a gap analysis in September 2013. At that time, it estimated that about 390 million additional LLINs would be needed in 2013-2016.101 In May and October 2014, we asked the Co-Chair of the Harmonization Working Group for updates on the size of the funding gap. On both occasions she told us that there remained significant gaps in funding for LLINs. She gave the following initial estimates for 2014-2017 (in millions of LLINs):

While we would guess that other funders will contribute some LLINs, the size of the gaps and the persistence of the 2014 gap late into the year, as occurred in 2013 as well,102 suggests that there is, and will continue to be, unmet need for LLINs in African countries.

AMF as an organization

We believe the Against Malaria Foundation to be an exceptionally strong and effective organization:

• Track record: We feel AMF has built up a strong track record of raising money, finding smaller distribution partners, and getting partners to report information publicly at an unusual level, and verifiably getting bednets delivered. More recently, AMF has begun to demonstrate its ability to implement its model at significantly larger-scale. Its work on this is at an early stage.
• Communication: AMF has always communicated extremely clearly and directly with us and given thoughtful answers to our critical questions.
• Self-evaluation: AMF has invested heavily in self-evaluation, going above and beyond what’s usual for data collection on bednet distributions: it has demonstrated a commitment to collecting long-term net usage data.
• Transparency: AMF appears to value transparency as much as any organization we’ve encountered. It hasn’t just shared information with us; it publishes significant amounts of useful information publicly on its own, far more than the norm. We have never seen AMF hesitate to share information publicly (unless it had what we consider a good reason).

However, we see some potential room for improvement:

• AMF does not seem to be as aggressive and ambitious about growth and capacity as it could be. It has operated for 11 years and still has only two full-time staff members; it does not appear to us to have built as robust a fundraising operation as it could.
• AMF has not conducted any field visits to distribution partners (as far as we are aware). We see this as a negative since we find such visits valuable.

More on how we think about evaluating organizations at our 2012 blog post.

Sources

• 1

  "How will the money raised be spent? On bednets (mosquito nets). Specifically, long-lasting insecticidal nets (LLINs)." AMF Frequently Asked Questions.

• 2

  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.

• 3

  GiveWell summary of AMF large-scale distributions, Sheet Costs.

• 4

  This understanding is from many conversations with AMF and following AMF's progress over time.

• 5

  "This is a list of the countries with known gaps and where there are significant contiguous areas without nets, or a significant percentage required, and for which the estimate of need is believed to be reasonably accurate. It does not include countries where there are gaps, typically up to 40% of what the nation needs, but they are spread more uniformly across the country and would therefore require an ‘in-fill campaign’. An in-fill campaign is different from a so-called ‘universal coverage campaign’ because the percentage installed base of nets is higher in the former case and so a pre-distribution registration survey (PDRS) is an absolute requirement to ensure an efficient allocation of nets. Our methodology would lend itself to these campaigns if the relevant National Malaria Control Programme (NMCP) were to embrace a detailed PDRS. The list does not include, in our view, other countries where the need has not yet been quantified. Given there are many countries with needs estimated, we have not chosen to seek out other countries in need of nets. Our assumption is groups like AMP will be a source of reporting on additional countries as quantified needs emerge." Rob Mather, AMF Founder, email to GiveWell, August 8, 2012.

• 6

  Rob Mather, AMF Founder, email to GiveWell, November 20, 2012.

  "Initial (directional) information

  a) Sources of collated numbers (eg AMP, ALMA)
  We liaise with groups that collate statistics on net numbers. AMP’s (Alliance for Malaria Prevention) weekly email is one such source as you correctly identify and ALMA’s (African Leaders Malaria Alliance) monthly update is another. The numbers are not always up to date and do not always agree. However they are useful as a first alert that there is a gap in a national campaign. I have attached two emails that show you the sort of thing we receive. b) Distribution Partners Groups involved in distributions in particular countries may be the first to be aware there is a net gap and they may approach us by email or telephone. c) Individuals/Advisors (to national campaigns). Those involved in national campaigns have also approached us in the past to ask if we are in a position to help close a net gap. This may be a technical malaria advisor who has just come out of a meeting where it has become apparent there is a gap in funding for nets for a near-term program. We refer to this information as ‘directional’ as in all cases detailed information is then required to confirm there is a gap." Rob Mather, AMF Founder, email to GiveWell, June 29, 2011.

• 7

  Examples:

  • Alliance for Malaria Prevention Conference call minutes (June 22, 2011) Pg 13.
  • Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, email to Rob Mather, June 26, 2011.

  "GiveWell: Neither of the AMP emails you sent (May 28, 2011 and June 22, 2011) nor the ALMA email (from June 26, 2011) provide a figure for net gap in Malawi. Do you know why this is? Did you use these groups for directional information in the case of Malawi? Has the net situation changed in Malawi recently?
  AMF: I thought it did – about 500,000 nets. However, a number we needed further proof on as our assessment was these were indicative numbers and not hard and fast numbers. We needed hard and fast numbers to be able to commit funds for nets as we would not do so if there wasn’t a gap. Liaising with the Global Fund, PMI (the two major funders) and the NMCP was, and is, key to this process. We have yet to receive confirmation of net gap which is why we have not as yet committed funds for nets. In part, this is due to the NMCP being required by the Global Fund (rightly in our view) to re-tender a particular element of the net distribution campaign and that has taken priority over work to finalise numbers. As with us, this has halted Global Fund money for nets until all is resolved satisfactorily. I also believe in part it is due to the NMCP having estimates and now they are being quizzed by us and others, they are having to think what to do – admit they are estimates (which we don’t necessarily have a problem with but we have asked to see the assumptions and logic behind those estimates) and/or because they are now coming round to the viewpoint (which we hold) that given the number of perfectly usable nets in place being above 15% of nets needed country-wide, it is economic and has other benefits, to conduct household by household surveys to determine exactly where (on a household basis) nets are needed. I can run through the $9m (cost of nets saved) vs $2-3m (cost of surveys country-wide) economic argument later." Rob Mather, AMF Founder, email to GiveWell, August 6, 2011.

• 8
  • "Issues on quantification: it has been raised that LLINs durability in the country is of 2 years rather than 3 years + the country needs to account for existing nets
  • GF [Global Fund] has confirmed that an AMP TA [Alliance for Malaria Prevention Technical Assistance] mission should take place at the latest in June 2011
  • Strong routine delivery system in place - country will be looking at accounting for existing nets
  • Malawi would be a good case study for how continuous distribution works - one of the countries flagged by VCWG [Vector Control Working Group] for case study.
  • Malawi achieved UC [universal coverage] by the end of 2010 in 3 of 27 districts"

  Alliance for Malaria Prevention Conference call minutes (June 22, 2011) Pg 13.

• 9

  "Malawi...LLIN gap: 2.7 million w/o taking into account any existing LLINs, or 1.5 million if the nets they assume are there are accounted for during HH registration." Alliance for Malaria Prevention Conference call minutes (October 26, 2011) Pg 19.
  As of November 2011, the Alliance for Malaria Prevention had posted minutes from its conference calls since August 3, 2011 at Alliance for Malaria Prevention Conference call minutes

• 10

  "We liaise with a variety of sources including the Global Fund, in-country NGOs such as the Presidents Malaria Initiative, World Vision, UNICEF, Plan International, Concern Universal, and the relevant NMCP to establish a) who has been the source of the AMP estimate; b) how has it been arrived at; and c) what is the level of confidence of these groups it is accurate. We view this information as interesting and necessary but not sufficient for us to allocate nets." Rob Mather, AMF Founder, email to GiveWell, August 8, 2012.

• 11

  "We will typically approach the NMCP (National Malaria Control Program) directly and a) ask if there is a gap; b) ask for data on i) the population, number of nets required, number of nets previously distributed, number of nets considered to still be ‘usable nets’, number of nets required to achieve universal coverage (and all of the above on a regional/district basis) and ii) data on existing funding commitments so we can see who else is funding nets and at what level." Rob Mather, AMF Founder, email to GiveWell, June 29, 2011.

  Data in AMF Malawi universal coverage calculations (September 26, 2011). In addition, AMF provided the email correspondence it had with the Malawi National Malaria Control Program, which involved extensive back-and-forth about the sources of the data and the assumptions used in the estimates of net needs.

• 12

  Tossa Kokou, National Program Manager of the National Malaria Control Program, Togo, email to Rob Mather, May 15, 2012.

• 13

  Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.

• 14

  Rob Mather, AMF Founder, email to GiveWell, August 8, 2012.

• 15

  @AMF LLIN Distribution Proposal Form@. For the Malawi distribution proposal, see Concern Universal Ntcheu 2012 distribution proposal.

• 16

  Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.

• 17

  Concern Universal Ntcheu 2012 pre-distribution census data.
  Concern Universal Balaka 2013 pre-distribution census data.

• 18

  AMF How we work with distribution partners Pg 2.

• 19

  AMF page on Ntcheu 2012 distribution.

• 20
  • "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
    [AMF:] No, CU did not take photos of this one. They are aware of this slip and there will be photos and video footage from Dedza. Photos and videos are also included in the agreement for the DRC-Kasai distribution." Rob Mather, AMF Founder, email to GiveWell, May 22, 2014.
  • AMF told us that future distributions would have the same requirements for photos and video as have past distributions. Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 21

  "[AMF:] Post‐Distribution Surveys (see Note 3). Please confirm you will carry out Post‐Distribution Surveys (PDSs) every 6 months post‐distribution for a period of up to four years to assess the level of net usage (hang‐up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households.
  [Concern Universal:] I can confirm Concern Universal and Dedza District Council will conduct a Post-Distribution Survey every six months following the distribution." Concern Universal Balaka 2013 distribution proposal, Pg 2.
  Concern Universal Dedza 2014 distribution proposal, Pg 2 contains the same language.
  Response to this requirement from Amani Global Works for North Idjwi Island 2014 distribution: "Yes, we will conduct Post-Distribution Surveys every 6 months for up to 4 years, and provide you with the findings." Amani Global Works North Idjwi Island 2014 distribution proposal, Pg 1.
  IMA World Health agreed to conduct surveys every 6 months for 3 year post-distribution. IMA World Health and AMF distribution agreement for Kasaï Occidental 2014, Pgs 3-4 and IMA World Health and AMF distribution agreement for Nord Ubangi 2015, Pgs 3-4.
  

• 22

  We discuss case rate data from Ntcheu in our February 2013 update on AMF.
  "[AMF:] Please confirm you are able to provide monthly malaria case rate data going back at least 12 months for each health centre/clinic in the distribution area and will continue to provide monthly data for a period of four years post‐distribution. This ensures we understand pre‐distribution malaria levels and can monitor them post‐distribution.
  [Concern Universal:] Malaria case rate information for each of the 16 Health Centres is available from the Balaka District Health Office using the Malawi Government’s national Health Management Information System (HMIS). This information will be sent to AMF separately." Concern Universal Balaka 2013 distribution proposal, Pg 2.
  Concern Universal Dedza 2014 distribution proposal, Pg 2 contains the same language.

• 23

  “AMF has requested additional data from local health centers and is in the process of
  reviewing it. It is planning to publish the results of this audit by the end of September.
  At this point, it still believes that the corrected data will show a substantial (~50%+)
  reduction in malaria cases in Ntcheu after its distribution there but is still assessing the
  level of accuracy and therefore reliability of the malaria case rate data from Ntcheu.” Rob Mather, AMF Founder, phone conversation with GiveWell, August 15, 2013.

• 24

  See our August 2014 update on AMF

• 25

  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.

• 26

  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.

• 27

  Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.

• 28

  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.

• 29

  AMF page on non-net costs.

• 30

  "AMF funding non-net costs can remove the final barrier to a distribution proceeding and avoid potential lengthy delays, of up to a year, given the ideal timing of a distribution relative to other events, for example a rainy season." AMF page on non-net costs.

• 31

  Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.

• 32

  Concern Universal Ntcheu 2012 pre-distribution census data.
  Concern Universal Balaka 2013 pre-distribution census data.

• 33

  @GiveWell Notes from Site Visit with Concern Universal in Malawi (October 2011)@.

• 34

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 35

  Concern Universal Ntcheu 2012 pre-distribution census data.
  Concern Universal Balaka 2013 pre-distribution census data.

• 36

  "For the subsequent verification visits (which Givewell witnessed) HSAs don't go into every household- instead they call a village register at a central location to check with the village leaders and individuals as to whether the data from the house to house survey is accurate. Givewell suggested that we should be doing some randomised checks on individual households at this verification stage- we have agreed and have now incorporated these into our verification model." Robin Todd, Concern Universal Malawi Director, email to Rob Mather, November 18, 2011.

• 37

  Robin Todd, Concern Universal Malawi Director, email to Rob Mather, November 18, 2011.

• 38

  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.

• 39

  "In 2010, Senegal and Cross River State in Nigeria worked on mop-up campaigns following earlier integrated campaigns targeting households with children under five years of age. In both countries, trained health workers or volunteers undertook a household registration to determine:

  • the total net need for each household (this was based on one LLIN for two people, rounding up in the case of odd numbers of people in the household, but Senegal also looked at the number of habitual sleeping spaces in each household)
  • how many nets each household already had
  • how many nets in each household were still viable (in Senegal this number was based on net condition, while in Cross River State, it was based on how long the net had been hanging)
  • how many new nets each household would need for full coverage

  Both countries had previously undertaken post-distribution surveys which showed high household coverage with LLINs, but during the mop-up exercise they experienced challenges with finding nets in households. Significantly lower numbers of nets were found (50—60 per cent) than would have been expected based on the surveys...
  In both countries, it seemed that families often hid nets once word spread that ownership of nets meant no new nets would be received. Despite efforts to encourage families to hang pre-existing nets prior to the household registration in Senegal, people hid nets in order to receive more." Alliance for Malaria Prevention Toolkit (version 2.0) - Chapter 3, Pgs 5-6.

• 40

  Alliance for Malaria Prevention Conference call minutes (October 26, 2011) Pg 20.

• 41

  AMF How we work with distribution partners Pg 2.

• 42

  AMF page on Ntcheu 2012 distribution

• 43

  "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
  [AMF:] No, CU did not take photos of this one. They are aware of this slip and there will be photos and video footage from Dedza. Photos and videos are also included in the agreement for the DRC-Kasai distribution." Rob Mather, AMF Founder, email to GiveWell, May 22, 2014.

• 44

  See, for example:

  • Concern Universal Ntcheu 2012 mid-distribution reports, Pg 3.
  • Concern Universal Balaka 2013 week 5 report.
  • Concern Universal Dedza 2014 week 3 report.

• 45

  See our March 2012 update on AMF.

• 46

  All weekly reports at AMF page on Balaka 2013 distribution. Examples of problems reported:

  • "The exercise despite the verification process and data cleaning faced some duplicates that were discovered during the distributions. The duplicate situation had not been dealt with during data cleaning as it was deemed only when both villages were asked to collect nets from the same distribution point that it became clear that there had been duplicate entries. However, different distribution points at the same time though at the same cluster makes it practically impossible for villagers to collect two nets from different sites by double registering." Concern Universal Balaka 2013 week 1 report.
  • "There was a high number of absenteeism during the urban distribution, which resulted in some members receiving the nets on behalf of others. This absenteeism was explained by the unavailability of the beneficiaries due to professional reasons. There were some complaints from beneficiaries whose nets were received by representatives claiming that the nets weren’t handed over. Handing over of nets to representatives was cancelled since it was clear that, unlike in rural areas where all community members know each other and certify the representative’s identification and the nets were handed over to the legitimate beneficiaries, nets were being misappropriated... For the urban distributions we anticipate to conduct them during the weekend to assure that most of the household owners are free from their daily work related activities." Concern Universal Balaka 2013 week 5 report.
  • "One health worker assigned to facilitate the distribution process, in the community under his supervision, was caught by the beneficiaries trying to steal about 25 nets. He was reported reported to the authorities and discharged of his duties. This episode disrupted the distribution process and CU staff had to intervene to keep the population from beating the health worker." Concern Universal Balaka 2013 week 5 report.

• 47

  All weekly reports at AMF page on Dedza 2014 distribution. Examples of problems reported:

  • "The major challenge encountered during this week’s distributions was the misplacing of villages in clusters, which required us to the transfer the nets and distribution registers to the clusters where the villages have presented themselves. This delayed our distribution process but we still managed to reach and carry out the distributions to the affected villages." Concern Universal Dedza 2014 week 1 report.
  • "There were 20 villages under Kaphuka that did not receive the nets because of poor communication as their HSAs were attending performance appraisals and failed to communicate the distribution dates to their respective villagers." Concern Universal Dedza 2014 week 3 report.
  • "The major challenge during the week was duplication of registration in a way that some beneficiaries seem to have been deliberately registered in more than one village. The registration data was corrected and the affected beneficiaries only received the nets they were entitled to according their respective village data." Concern Universal Dedza 2014 week 3 report.

  AMF notes these issue and an additional issue on its blog, "An isolated incident of 300 nets missing from one storage location. This is being investigated and pursued with the police as any nets missing is taken very seriously. 300 nets represents 0.12% of the total nets being distributed. AMF: "Mid-distribution weekly reports for Dedza distribution, Malawi".

• 48

  “AMF has provided regular, public updates on the large, ongoing net distribution in the Ntcheu district of Malawi. Expected data collection has occurred and the distribution has proceeded close to schedule. AMF's distribution partner, Concern Universal, has been transparent about problems it has encountered, and seems to have a robust process to catch problems (such as attempts to steal nets) when they arise.” See our March 2012 update on AMF.

• 49

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 50

  "Our distribution of 676,000 nets in Kasai Occidental in partnership with IMA World Health (IMA) is our first one using smartphone technology for data collection. We see this as an exciting development with significant potential benefits including:

  • Acts against potential theft
  • Improved accountability
  • Greater transparency
  • Greater data accuracy
  • Improved cost effectiveness
  • Additional data can be collected
  • Reduced operational risk

  The use of this technology may become a significant determinant of future net distributions that we fund. We will report publicly on our experience with the Kasai Occidental distribution and the data gathered…
  GPS information can also be gathered helping to locate households and tie the number of nets delivered to each." AMF: "Introduction of smartphone technology to collect distribution data".

• 51

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 52

  AMF lists the countries it has provided nets to at @AMF Countries Involved@. The Malaria Atlas Project has compiled data on malaria risk by location at Malaria Atlas Project Endemic countries.

• 53
  • "Figures from Ntcheu District Hospital show that, in the twelve months to July 2011, 109,972 malaria cases were reported in under five children with a further 133,768 cases reported in the rest of the population." Concern Universal Ntcheu 2012 distribution proposal Pg 1.
  • "All of Malawi (except some isolated highland areas) is designated a high risk malaria area. There were 96,453 reported cases of malaria in Balaka District over the past twelve months." Monthly data broken down into cases among under-5s and others also provided. Concern Universal Balaka 2013 distribution proposal, Pg 2.

• 54

  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.

• 55

  "Continued use of nets is very important. Every six months, a post-distribution survey is carried out to assess net usage and net condition. Approximately 5% of the nets distributed are assessed through visits to randomly selected households. The data collected are used to determine if additional community-level malaria education activities are required. All data are published." AMF information we publish.
  Example from a distribution proposal: "Please confirm you will carry out Post‐Distribution Surveys (PDSs)** every 6 months post‐distribution for a period of up to four years to assess the level of net usage (hang‐up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households." Concern Universal Dedza 2014 distribution proposal, Pg 2.

• 56

  The methodology used in the Balaka 2013 6-month follow up survey seems similar to that used in the Ntcheu surveys discussed in our August 2014 update on AMF. Specifics from Concern Universal Balaka 2013 6-month post-distribution check-up report:

  • Sample selection: "Balaka district has 14 health facilities. It was decided to collect data from 5% of households in each HCCA [health center catchment area], which meant a different number of households in each HCCA as per individual health facility populations. Between 25 and 90 households were randomly selected from each of the selected seven to ten villages, depending on the HCCA, with the villages also selected at random. Villages were randomly selected using the village lists generated from the pre-distribution and distribution work for the October 2013 AMF-funded universal coverage LLIN distribution. A random number table was used to select the villages. Households were randomly selected using the household lists produced during the same campaign. A random number table was used to select the households. Five more households were put on reserve in case no one was at home in the selected households." Pg 5.
  • Who carried out the survey: "10 data collectors and 2 supervisors from the District Health Office were involved in the PDCU. The supervisors were responsible for checking the data collection exercise at the same time monitoring how the data was being collected as per requirement." Pg 5. Previous reports were more specific about who the data collectors were (for example, in the Ntcheu 24-motnh survey they were Health Surveillance Assistants, Malawi's version of community health workers.
  • Data quality audits: "Supervisors were required to visit 5% of the households in their area to check the accuracy of the data collectors’ work and had to check all the completed forms submitted to them before submitting them to the Project Manager. The sampled visited households were also chosen at random so the work of all data collectors was checked." Pg 6.
  • Cost: "The PDCU cost was US$8,060 equal to US$1.78 per household visited (or $0.052 per net originally distributed which numbered 154,230)." Pg 3.
  • Data entry/cleaning: "Improvements in the data entry interface since the last PDCU carried out in Ntcheu (Ntcheu PDCU-24) by AMF meant the data entry proceeded with almost no errors. This reduced the error-checking phase to almost nothing." Pg 6

• 57

  The type of technical information we would ideally like to see includes:

  • Results of audits on data quality.
  • Analysis of the extent to which respondent selection was carried out according to the protocol.
  • Analysis of the extent to which selected households were replaced with pre-selected alternate households (i.e. attrition from the sample).
  • Survey text used in each survey round and detailed protocols for responses that are directly observed by surveyors.

• 58

  Ntcheu 6-month post-distribution check-up data.
  Ntcheu 15-month post-distribution check-up data.
  Ntcheu 24-month post-distribution check-up data.
  Balaka 6-month post-distribution check up data (April 2014).

• 59

  Rob Mather, AMF Founder, email to GiveWell, June 30, 2014.

• 60

  "'very good condition' (fewer than 2 holes of up to 2cms in size)." Concern Universal Ntcheu 2012 24-month post-distribution check-up report, Pg 4.

• 61

  "'OK' (fewer than 10 small holes on them)" Concern Universal Ntcheu 2012 24-month post-distribution check-up report, Pg 4.

• 62

  "'Poor' condition with more than 10 holes or 1 hole larger than 10 cm." Concern Universal Ntcheu 2012 24-month post-distribution check-up report, Pg 4.

• 63

  "Worn out means not usable. Typically, holes will be 10 cms of cms in size and numerous and the net considered effectively unrepairable and/or structural integrity will have gone. If hung the net would not provide protection against mosquitoes given the extent of holes or tears or other loss of structural integrity." Rob Mather, AMF Founder, email to GiveWell, June 30, 2014.

• 64

  AMF How we work with distribution partners, Pg 4.

• 65

  Rob Mather, AMF Founder, email to GiveWell, June 30, 2014.

• 66

  "Usage does not appear to have been near-universal. Most studies report usage rates in the range of 60-80%, though some report 90%+ usage." From our page on Long Lasting Insecticide Treated Nets.

• 67

  "Alliance for Malaria Prevention 2011 lays out a model for estimating the number of LLINs still in use after distributions: 'the number of LLINs already distributed over the last three years and considered to be available in households should be calculated and subtracted from the total need, working with a decay rate of 8 per cent at one year (0-12 months), 20 per cent at two years (13-24 months) and 50 per cent at three years (25-36 months).'…
  Bottom line: We believe that the "8%-20%-50%" model is the most widely used and most reasonable approximation available at the moment for capturing the extent to which LLINs remain in use in the years following distribution, accounting for any factors that might cause LLINs to be discarded or additional LLINs to be purchased. It implies an average of 2.22 years of use for each LLIN distributed. Data and analysis on this topic appears extremely thin; we have little sense for how long LLINs last in practice." GiveWell report on the "decay model" for LLINs.

• 68

  AMF Malawi universal coverage calculations (September 26, 2011).

• 69

  See our March 2012 update on AMF.

  Exact net figure from Rob Mather, AMF Founder, email to GiveWell, November 20, 2012.

• 70

  AMF Malawi universal coverage calculations (September 26, 2011).

• 71

  "Health Surveillance Assistants (HSAs) are Government extension workers- they are the lowest tier of government presence in the decentralized health system." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.

• 72

  "As such they are the first line of response to any public health issues in communities. Their job involves disseminating health related information (such as encouraging people to make use of sanitary facilities, go for immunizations, sleep under mosquito nets etc.), carrying out sanitation and hygiene campaigns and sending data on take-up of facilities to the District Council, conducting basic nutrition support, weighing children and reporting levels of stunting and wasting, detecting common communicable diseases and reporting these to clinicians and other health providers, implementing immunization campaigns etc. As you can see being involved in universal net distribution fits very well with their core public health responsibilities. HSAs need to have a primary school completion certificate as a minimum but the majority of them will have O-Levels (exams sat by pupils aged 16 if they have completed the school system at the recommended pace). Once they have been selected as HSAs they are sent on an initial 9 months intensive training course where they will be trained in many aspects of public health including how to recognize common diseases, how to administer immunizations etc." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.

• 73

  GiveWell estimate of AMF cost per net (November 2014).

  • Balaka budget for pre-distribution and distribution costs: $0.56 per LLIN (Sheet Balaka and Dedza non-net budget, cell L58). Balaka actual costs for pre-distribution and distribution costs: $0.84 per LLIN (Sheet Balaka and Dedza non-net budget, cell V58). AMF told us that non-net costs were higher in Balaka than it expects costs to be in future distributions because of a problem that lead to Concern Universal repeating the pre-dstribution registration survey.
  • Dedza budget for pre-distribution and distribution costs: $0.47 per LLIN (Sheet Balaka and Dedza non-net budget, cell AT58). Column is marked as actual costs, but costs are very similar to budget, so it appears that this column is largely budgeted costs adjusted for a greater number of LLINs needed than originally planned.
  • Dowa budget for pre-distribution ($0.21 from Sheet Dowa non-net budget, cell L127) and distribution costs ($0.23 from Sheet Dowa non-net budget, cell L223): $0.44 per LLIN.

  In estimating the overall cost per LLIN in Malawi, we have used an average of the cost per LLIN for Balaka and budgeted costs per LLIN in future distributions (Dedza and Dowa). In other words, we have taken a weighted average with Balaka weighted at 50%, Dedza at 25% and Dowa at 25%. This is to account for the facts that we would guess that actual costs are often more reliable than budgeted costs, but that the actual costs we have from Balaka may be greater than average due to an unusual circumstances.

• 74

  Rob Mather, AMF Founder, phone conversation with GiveWell, May 23, 2014. Managerial staff time is listed in the budget.

• 75

  See the “CU non-monetary costs Ntcheu” sheet in GiveWell estimate of AMF cost per net (November 2014) for details.

• 76

  See the “AMF org” sheet in in GiveWell estimate of AMF cost per net (November 2014) for details.

• 77
  • AMF told us that in DRC net costs and non-net costs are very roughly the same. Rob Mather, AMF Founder, phone conversation with GiveWell, May 23, 2014.
  • "In other cases, non-net costs are assumed to $2.50 per LLIN, which is the historical average. Non-net costs can vary significantly across countries. For example, in Angola and DRC, non–net costs may be more than $5." Melanie Renshaw, phone conversation with GiveWell, May 23, 2014.

• 78

  See for example Roll Back Malaria Partnership gap analysis (September 2013).

• 79

  Roll Back Malaria gap analysis tool, Sheet LLINs, cells B55:C65.

• 80

  "The gap analysis includes both the cost to purchase LLINs and non-net costs, such as
  shipping and distribution. In cases where national governments provided data on non-net costs, this data was used in the analysis. In other cases, non-net costs are assumed to $2.50
  per LLIN, which is the historical average. Non-net costs can vary significantly across countries. For example, in Angola and DRC, non–net costs may be more than $5." Melanie Renshaw, phone conversation with GiveWell, May 23, 2014.

• 81

  See this spreadsheet.

• 82

  "Funds in hand as at Nov 2013: $12.5 m
  Committed since Nov13: $10.30 m (10.3/12.5 = 82%)

  - DRC, West Kasai 2014 $2.0m (NETS only)
  - DRC, Idjwi 2015 $0.37m (NETS + all non-net costs)
  - Malawi, Dowa, 2015 $1.49m (NETS + all non-net costs)
  - Malawi, Ntcheu 2015 $1.34m (NETS + all non-net costs)
  - Malawi, Balaka 2015 $0.94m (NETS + all non-net costs)
  - Malawi, Dedza 2017 $1.72m (NETS + all non-net costs)
  - DRC, North Ubangi 2015 $2.34m (NETS + shipping)
  - Malawi, Dedza, 2014 $0.10m (additional nets)

  Funds uncommitted $2.2m
  New funds raised $2.65m
  Total funds uncommitted $4.85m"
  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.
  
  
  "If ‘make available’ all Dedza17: $1.720m [i.e. if funds committed to the Dedza 2017 distribution were made available for an earlier distribution]
  Ditto, Yr2+ PDCU Do/Nt/Ba: $0.2 m [refers to future post-distribution check ups in Dowa, Ntcheu, and Balaka]"
  Rob Mather, AMF Founder, email to GiveWell, November 8, 2014.

• 83

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 84

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 85

  Rob Mather, AMF Founder, email to GiveWell, November 6, 2014.

• 86

  Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.

• 87

  Rob Mather, AMF Founder, email to GiveWell, November 6, 2014.

• 88

  For the Kasaï Occidental (DRC) distribution in 2014, AMF has purchased LLINs for $2.88 per net. It expects to pay approximately $0.41 for shipping for its Nord Ubangi (DRC) distribution in 2015. Therefore, we estimate the cost to AMF per LLIN for future distributions with IMA World Health at $3.29.

• 89

  Rob Mather, AMF Founder, email to GiveWell, November 6, 2014.

• 90

  We have assumed that distribution partners will cover non-net costs in these cases and that AMF would purchase LLINs for $3 per net, as AMF has budgeted for many of its future distributions. GiveWell summary of AMF large-scale distributions.

• 91

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 92

  Number of LLINs from Rob Mather, AMF Founder, email to GiveWell, November 6, 2014.
  We have assumed that distribution partners will cover non-net costs in these cases and that AMF would purchase LLINs for $3 per net, as AMF has budgeted for many of its future distributions. GiveWell summary of AMF large-scale distributions.

• 93

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 94

  Number of LLINs from Rob Mather, AMF Founder, email to GiveWell, November 6, 2014.

• 95

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 96

  Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 97

  We have added $4.0 million for Malawi, $13.2 million for DRC, $6 million and $12 million for possible large distributions that are in early stages of discussion, and $0.5 million for insecticide resistance research.

• 98

  "The operational plan for a distribution typically covers 12 months. The planning horizon for a specific distribution is, however, typically 18 to 24 months as discussions to assess and approve a distribution start six months earlier and the funds required (we need to have funds in hand in order to have serious discussions) are aggregated in the preceding months." AMF: "Operational planning (12 months) and planning horizon (18 to 24 months)".

• 99

  All statements preceding this footnote in this paragraph are from Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.

• 100

  See for example Roll Back Malaria Partnership gap analysis (September 2013).

• 101

  Roll Back Malaria Partnership gap analysis (September 2013), sum of cells N45:Q45.

• 102

  Roll Back Malaria Partnership gap analysis (September 2013), sum of cell N45.