We have published a more recent review of this organization. See our most recent report on AMF.
The Against Malaria Foundation (AMF) is one of our top-rated charities, and an organization that we feel offers donors an outstanding opportunity to accomplish good with their donations.
More information:
Published: November 2015
Summary
What do they do? AMF provides funding for long-lasting insecticide-treated net distributions (for protection against malaria) in developing countries.
Does it work? There is strong evidence that distributing nets reduces child mortality and malaria cases. AMF has relatively strong reporting requirements for its distribution partners and provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity. AMF has begun building a track record of funding and tracking fairly large-scale (100,000 nets or more) distributions.
What do you get for your dollar? We estimate that the cost to purchase and distribute an AMF-funded net is $5.15 in Malawi and, very roughly, $6.30 in DRC (the two countries that AMF has worked most extensively in). We also very roughly estimate that the cost per net in distributions AMF is considering funding is $5.31. The numbers of malaria cases prevented and lives saved are a function of a number of difficult to estimate factors, which we discuss in detail below.
Is there room for more funding? AMF currently holds approximately $18.5 million in funds that it could allocate to future distributions. AMF has told us that it has a pipeline of possible future net distributions that add up to roughly $100 million beyond what it currently holds.
AMF is recommended because of its:
- Focus on a program with excellent evidence of effectiveness and cost-effectiveness.
- Strong processes for ensuring that nets reach their intended recipients and monitoring whether they are used over the long-term.
- Standout transparency and commitment to self-evaluation.
- Room for more funding – we believe AMF will be able to use additional funds to deliver additional nets.
Major unresolved issues include:
- Though we now have increased confidence in AMF's ability to find new partners and countries to work in, there is still significant uncertainty as to its ability to scale up.
- AMF has completed only five large distributions, the type it aims to focus on in the future. Additionally, it has only worked with two different distribution partners on its large distributions. Its track record of collecting the data it seeks from these distributions is good so far but limited due to the small number of distributions. Many of the distributions AMF hopes to fund are in countries where it has never worked, which may present unforeseen challenges.
- Recently, AMF has been slower to share documentation from some distributions. We believe this documentation is important for monitoring the quality of AMF's distributions.
- The best evidence for nets was collected before they were widely used and there is some evidence that mosquitoes have since adapted to the insecticide used in nets, possibly making them less effective. Further research is needed to determine how important of a problem this may be. We discuss this issue in more detail in our page on this topic.
Table of Contents
- Summary
- Our review process
- A few key terms for this review
- What do they do?
-
Does it work?
- Are LLINs targeted at people who do not already have them?
- Do LLINs reach the intended destination?
- Are LLINs targeted at areas with high rates of malaria?
- Do those who receive the LLINs install them in their homes properly? Do those who receive the LLINs utilize them consistently over the long term?
- Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF?
- Possible negative or offsetting impact
- What do you get for your dollar?
- Is there room for more funding?
- AMF as an organization
- Sources
Our review process
We began reviewing AMF in 2009. Our review has consisted of:
- Reviewing AMF's public records for each of its net distributions and other documents AMF has shared with us.
- Extensive communication, including several meetings at AMF's London headquarters, with AMF Founder Rob Mather to discuss AMF's methods and funding needs.
- A visit to AMF's distribution partner organization, Concern Universal, in Malawi in October 2011 (notes and photos from this visit).
- Reviewing materials from AMF’s distributions in Malawi and the Democratic Republic of the Congo (DRC), including pre-distribution registration data, distribution reports, and data from post-distribution follow up surveys.
- Conversations with Peter Sherratt, AMF's Executive Chairman; Don de Savigny, a member of AMF's Malaria Advisory Group, and other individuals (who requested to remain anonymous) familiar with AMF's work and its attempts to finalize distributions.
All content on AMF, including past reviews, updates, blog posts and conversation notes, is available here.
A few key terms for this review
- A net refers to any bed net intended to block mosquitoes, whether treated with insecticide or not.
- An ITN is an insecticide-treated net. As discussed at our write-up on ITNs, distribution of ITNs has been rigorously studied and associated with declines in under-5 mortality and overall malaria cases.
- A LLIN is a long-lasting insecticide-treated net, intended to serve as an effective ITN for 4-5 years (details at our write-up on ITNs). The nets distributed by AMF are LLINs,1 and (as noted in our write-up on large-scale ITN distributions) most large-scale donor-financed malaria control today utilizes LLINs.
What do they do?
AMF provides long-lasting insecticide-treated nets (for protection against malaria) in bulk to other organizations, which then distribute the nets in developing countries.
Prior to November 2011, AMF contributed LLINs to small-scale distributions. Since then, it has de-emphasized small-scale distributions and it expects larger-scale distributions to be the primary way in which it distributes LLINs in the future (more below).
As of September 2015, AMF had completed five large-scale distributions with two distribution partners for a total of 1.74 million LLINs.2 AMF has three additional large-scale distributions planned through the end of 2016 for another 1.3 million nets3 and has signed agreements to distribute an additional 1.5 million LLINs in 2018.4 It also has two medium-sized distributions (around 50,000 nets) planned for late 2015 and 2016.5 Below, we provide a summary of AMF's medium and large-scale distributions.6 A more detailed summary of AMF's distributions can be found in our Summary of AMF Distributions spreadsheet.
| Location | Distribution partner | Number of LLINs | Cost to AMF (millions) | Timing |
|---|---|---|---|---|
| Ntcheu, Malawi | Concern Universal | 268,430 | $1.13 | Completed: December 2011-March 2012 |
| Balaka, Malawi | Concern Universal | 154,230 | $0.66 | Completed: October-December 2013 |
| Dedza, Malawi | Concern Universal | 290,770 | $1.11 | Completed: September-October 20147 |
| Kasaï Occidental, DRC | IMA World Health | 630,532 | $1.81 | Completed: August-November 2014 |
| Distribution in flood-affected districts, Malawi | Concern Universal and others8 | 80,000 | $0.30 | Completed: February-March 2015 |
| Dowa, Malawi | Concern Universal | 396,900 | $1.45 | Completed: March-May 2015 |
| Ntcheu, Malawi | Concern Universal | 335,000 | $1.20 | Planned: October-November 2015 |
| Balaka, Malawi | Concern Universal | 235,000 | $0.86 | Planned: November-December 2015 |
| Nord Ubangi, DRC | IMA World Health | 705,468 | $2.14 | Planned: October 2015-March 2016 |
| North Idjwi Island, DRC | Amani Global Works | 62,000 | $0.19 | Planned: November-December 2015 |
| South Idjwi Island, DRC | Amani Global Works | 62,000 | $0.19 | Planned: March-April 2016 |
| Dedza, Malawi | Concern Universal | 430,000 | $1.72 | Planned: July-August 2018 |
| Dowa, Malawi | Concern Universal | 435,000 | $1.74 | Planned: August-September 2018 |
| Ntcheu, Malawi | Concern Universal | 365,000 | $1.46 | Planned: September-October 2018 |
| Balaka, Malawi | Concern Universal | 250,000 | $1.00 | Planned: October-November 2018 |
| Total completed | 1,820,862 | $6.16 | ||
| Total planned | 2,879,468 | $10.50 | ||
| Total | 4,700,330 | $16.66 |
Our most thorough review of AMF's processes was done for its first large-scale distribution in Ntcheu, Malawi, and much of the below discussion relates to that distribution. Since then, we have followed AMF's progress and asked for updates on how its processes have changed rather than consider each distribution in depth.
AMF's role
AMF's role in LLIN distributions is to:9
- Identify countries with funding gaps for LLINs.
- Find distribution partners to carry out LLIN distributions and agree on expectations for the distribution, including who pays for costs other than the purchase price of LLINs and what information will be collected and shared with AMF.10
- Purchase LLINs and have them shipped to the distribution partner.
- Work with distribution partners to collect reports on the distribution and follow up surveys. AMF posts these reports on its website.
Process for distributions
Selecting locations for distributions
When selecting locations for future distributions, AMF told us it consults a series of sources as it believes there is no one reliable resource with up-to-date information to determine where nets are needed. Sources it consults include the Alliance for Malaria Prevention's (AMP's) list of countries with significant net gaps, other malaria control funders, in-country technical advisors, the relevant National Malaria Control Program (NMCP), implementing organizations, and the African Leaders Malaria Alliance.11
As an example, we discuss the case of its distribution in Ntcheu, Malawi in 2012. For that distribution, AMF received estimates of country-level gaps from the Alliance for Malaria Prevention (AMP) and from the Malawi National Malaria Control Program (NMCP). Rob Mather, AMF's founder, told us that, although AMF did not have a high level of confidence in the net gap numbers presented by the NMCP, nor those it was sent by the Global Fund, it did have a high level of confidence that the net gap was significantly higher than the 250,000 LLINs AMF was considering providing.12 Note that the materials AMF has sent us from the period prior to its decision to provide funding for nets in Malawi do not include quantified estimates of Malawi's LLIN gap,13 though they do make it clear that there is a gap,14 and later reports (from after AMF had decided on Malawi) show a quantified gap for the country.15
Note that we have not repeated this analysis for any of the distributions that AMF has funded or considered funding after the Ntcheu, Malawi distribution in 2012.
We have seen some correspondence between AMF and the NMCP in Malawi which took place prior to AMF’s decision to fund a distribution in Malawi.16 AMF also shared with us correspondence that it had with representatives of the NMCP in Togo, with regard to potential future distributions.17
As AMF investigates countries with existing net gaps, it also looks into organizations working within those countries that could serve as distribution partners.18 AMF's distribution partners must have the capacity and willingness to carry out a distribution that meets AMF's requirements (more below).19 Before deciding to work with Concern Universal for the Ntcheu 2012 distribution, AMF requested a distribution proposal, including information on malaria risk in the distribution area (including data on reported malaria cases), other net distributions in the area, coordination with government, and how the distribution would be carried out.20 We have seen similar distribution proposals, which include information on malaria risk and other sources of nets in the area, for distributions in Balaka in 2013, Dedza in 2014, North Idjwi Island in 2014, and Kasai Occidental in 2014. We have not yet seen distribution proposals for the completed distribution in Dowa district. AMF now requires that distribution partners submit a planning document instead of a distribution proposal. We have not seen planning documents or distribution proposals for two of AMF's recent distributions (full list in Summary of AMF Distributions spreadsheet).21
Requirements for distribution partners
AMF aims to work with distribution partners who will agree to the following requirements (see AMF Summary features of an AMF distribution for a brief overview of the requirements). We detail what evidence we have seen that past distributions have complied with these requirements in our Summary of AMF Distributions spreadsheet.
- Pre-distribution registration: AMF requires that its distribution partners carry out a pre-distribution registration survey (PDRS) in all areas that LLINs will be distributed to establish the number of sleeping spaces and LLINs in a condition such that they have at least one year of remaining use. (In the case that such data already exist, AMF and the partner discuss whether the data is reliable and up-to-date, or whether a new PDRS should be conducted.22 ) We have seen results from all of AMF's completed large-scale distributions in Malawi: full data from its Ntcheu 2012 distribution and sample results from its Balaka 2013, Dedza 2014, and Dowa 2015 distributions.23 We have also seen full data from AMF's distribution with IMA in Kasaï Occidental (this data was collected during the distribution rather than prior to it – more).24
- Distribution monitoring: An additional requirement is to provide AMF with photos and videos from the distribution and a report on the distribution, including problems encountered.25 AMF's distribution partner, Concern Universal, provided photos and/or videos for all but 2 of the 14 “sublocations” for its Ntcheu 2012 distribution.26 AMF did not receive photo documentation from Concern Universal for the Balaka 2013 or Dedza 2014 distributions. AMF expects to publish photos it received for the Dowa 2015 distribution by the end of 2015 (they are being recovered from a corrupted disk).27 AMF has told us that it does not expect photos and/or videos from the Balaka 2013 or Dedza 2014 distributions because Concern Universal lost the photo files; however, we have heard conflicting messages about this documentation (see footnote).28 IMA World Health, AMF's distribution partner for the Kasaï Occidental 2014 distribution, sent AMF photos and/or video footage for 6 of the 8 "sublocations" of the distribution.29 AMF expects to receive photos and video from future distributions. AMF notes that it places little weight on the photos when assessing whether or not a distribution was conducted appropriately.30 Concern Universal has provided detailed distribution reports for three of the four distributions it has completed, as well as detailed weekly reports (which AMF does not require) for all four distributions.31 IMA has provided a detailed distribution report and a technology report for the Kasaï Occidental 2014 distribution.32
- Post-distribution check ups (PDCUs): AMF requires partners to conduct follow-up surveys in 5% of households at 6-month intervals for 2.5 years after a distribution.33 We summarize which PDCUs have been completed and whether they were completed on time in this spreadsheet (sheet "PDCUs"). In short, four of five expected PDCUs were completed for the Ntcheu 2012 distribution, all three of the expected PDCUs from the Balaka 2013 have been completed, and the first expected PDCU from the Dedza 2014 distribution was completed. We summarize the results below and in this spreadsheet. PDCUs were also expected to take place for the Kasaï Occidental 2014 distribution in May 2015 and for the distribution in flood-affected districts in Malawi three months after the distribution, which also would have been May 2015 (AMF later told us this was delayed to late 2015); we have not seen results from these PDCUs.34 In early October 2015, AMF told us that results from the Kasaï Occidental PDCU were expected to be published by the end of October.35
- Malaria case rate data: Previously, AMF expected to collect malaria case rates from the regions in which it funded LLIN distributions: in the distribution agreements for the Ntcheu 2011-2012, Balaka 2013, and Dedza 2014 distributions, AMF asked Concern Universal to provide monthly malaria case rate data from before and after the distributions from all health centers in the three districts.36
AMF has told us that Concern Universal has provided this data, but AMF has not yet published it.37
AMF told us that it has not made publication a high priority because it does not believe that the data is of high enough quality to reliably indicate trends in the malaria case rates.38
AMF has not included in its agreement with its partner in DRC, IMA World Health, a requirement to collect and share malaria case rate data. Discussions with IMA made it clear the quality of the malaria case rate data was not considered to be reliable in the majority of health centers. IMA told AMF that it believes there will be a some health centers across the distribution area that can provide historic and future malaria case rate data that is reliable and that it might be able to provide this data to AMF in the future.39 We do not believe that the fact that AMF collects malaria case rate data is a consideration in AMF’s favor, and do not plan to continue to track AMF's progress in collecting malaria case rate data.40
- Non-net costs: Prior to the Balaka 2013 distribution, AMF asked all distribution partners to use their own funds or to find another funder for all non-net costs of the distribution.41 More recently, it has paid for some of these costs in certain distributions. AMF told us that it considers funding non-net costs in cases where (a) non-net costs are not covered by other partners, and (b) AMF feels confident that its distribution partners will manage and report on spending well.42 AMF has paid for all non-net costs for the Balaka 2013, Dedza 2014, and Dowa 2015 distributions and has agreed to pay for all non-net costs for the upcoming Malawi distributions in 2015, 2016, and 2018.43 It has also paid for the costs of the 24- and 33-month post-distribution check-ups for the Ntcheu 2012 distribution, the shipping costs for the Nord Ubangi 2015 and flood-affected districts distributions, and has agreed to pay for all of the non-net costs for the North and South Idjwi Island distributions.44 It is our understanding that AMF aims to primarily fund distributions where other partners cover non-net costs, which could limit its capacity to distribute nets, though AMF has shown flexibility on this issue.45
AMF looks for partners who, in addition to agreeing to the above requirements, are clear and responsive in their communication with AMF.
Other activities
AMF occasionally supports malaria control activities beyond the direct distribution of LLINs. For example:
- AMF is intending to fund a study on insecticide resistance. The study would be run by the Liverpool School of Tropical Medicine during the Nord Ubangi distribution in Equateur, DRC.46 AMF sent us a research proposal for the study which we have not yet reviewed.47
- AMF is in the process of creating and co-funding a "Malaria Unit" in Malawi with Concern Universal.48 The Malaria Unit will consist of several permanent staff members who will work on a variety of malaria control projects: improving malaria case rate data collection practices, monitoring the levels of malaria prevention and treatment supplies at local health centers, developing efficient methods to keep net coverage rates high in between mass distribution campaigns, and more.49 The Malaria Unit is also intended to assist with AMF and Concern Universal's distributions in Malawi.50 AMF has committed approximately $400,000 over the next three years to this project.51
- AMF has encouraged Concern Universal staff to attend national malaria control strategy meetings in Malawi to share AMF's processes and results in other districts. As a result, AMF has told us that some of its monitoring practices are being adopted for a large national distribution (approximately 7 million nets) occurring soon in 18 of Malawi's 28 districts.52 The practices being implemented include a) using sleeping spaces data to calculate the number of nets needed (as opposed to population data), b) spot-checking 5% of the registration data collected before the distribution, and c) putting summary data (at the village or health center level) into electronic form.53 AMF told us in February 2015 that it expects to be able to see the data collected during this distribution.54
Does it work?
On a separate page, we discuss the general evidence behind distributions of LLINs. We conclude that there is strong evidence that these distributions can be expected to reduce child mortality and malaria cases.
When evaluating the effectiveness of an LLIN distribution organization, we seek to answer the following questions:
- Are LLINs targeted at people who do not already have them? AMF typically requires partners to complete pre-distribution surveys to determine the number of nets required, if reliable and recent survey data is not available. We are not fully satisfied with the pre-distribution surveys' ability to ensure that people aren't under-reporting their existing LLIN ownership in order to get more LLINs, but believe that reasonable measures are being taken to address this issue.
- Do the LLINs reach the intended destinations? Distribution partners are asked to provide pictures and videos of LLINs being handed out (a requirement that has been complied with in two of five completed large-scale distributions), and report on how the distribution went, including problems encountered. Post-distribution surveys provide additional evidence that LLINs reached their destinations.
- Are LLINs targeted at areas with high rates of malaria? AMF seeks out distribution partners in countries that are known to have high rates of malaria.
- Do those who receive the LLINs install them in their homes properly? Do they utilize them consistently over the long term? AMF requires partners to conduct follow-up surveys at 6-month intervals, and for a period of 2.5 years. AMF has published documentation showing that its partner Concern Universal has generally complied with this requirement: of the nine surveys that should now have been completed, one was skipped and three were 2-3 months delayed. Data from the first follow-up survey for AMF's first large-scale distribution with another partner was several months overdue as of this writing in October 2015.55 We have little data to compare these results to, but they seem consistent with high, proper usage of nets for an extended period after distribution.
- Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF? The limited evidence we have suggests that donations to AMF increase the total number of LLINs distributed.
Details follow.
Are LLINs targeted at people who do not already have them?
AMF requires that distribution partners conduct household-level pre-distribution registration surveys (if recent, reliable household-level data is not available) to determine the number of existing LLINs with at least one year of remaining use and the number of nets required in each household.56 This process includes door-to-door surveys of every household in the targeted district by government health workers. We have seen registration survey results from all four of AMF's completed large-scale distributions in Malawi: full data from its Ntcheu 2012 distribution and sample results from its Balaka 2013, Dedza 2014, and Dowa 2015 distributions.57
In Malawi, Concern Universal has then conducted village meetings in which its staff read off household names and the number of LLINs that have been allocated to each household; households then indicate if any errors have been made. We observed one such village verification meeting during our visit to Malawi in October 2011.58
In the Kasaï Occidental 2014 distribution, IMA World Health did not conduct village verification meetings because LLINs were distributed during the registration survey: health workers went door-to-door checking how many LLINs each household required and provided the LLINs at the same time (more).59 The lack of a pre-distribution registration survey in Kasaï Occidental led to some issues; for example, many sublocations received either too few or too many nets, because IMA based its estimates of how many nets would be needed on population data that turned out to be outdated.60 IMA and AMF now plan to implement a pre-distribution registration survey for the upcoming distribution in Nord Ubangi, DRC.61
AMF told us in 2011 that once the initial survey is completed, staff are sent to revisit 5% of households (selected randomly) to spot-check the accuracy of the survey data; we have not seen results from these spot checks, and do not know if they have taken place for recent distributions.62
We believe it is possible that some households may over-report the number of LLINs they need. Cases of households hiding nets in order to receive extra LLINs have been observed in universal campaigns in Senegal and Nigeria.63 That said, we believe that the steps AMF and its partners are taking to minimize the ability of community members to request more nets than they need are reasonable.
Do LLINs reach the intended destination?
AMF requires that partners provide three types of documentation from during and after distributions:64
- Photos and videos for each "sublocation" of the distribution
- Distribution reports, detailing the activities completed during the distribution and problems encountered
- Post-distribution check ups 6-months after the distribution, which include the number of AMF nets that were found in surveyed households
Photos and videos of distributions
Concern Universal provided photos and/or videos for all but 2 of the 14 “sublocations” in Ntcheu in which it distributed LLINs in 2012, although there are a fairly small number of photos for each sublocation (around 15 photos per sublocation).65 It has not yet provided photos and/or video for the Balaka 2013, Dedza 2014, or Dowa 2015 distributions, the other large-scale distributions that AMF has completed in Malawi.66 IMA World Health has provided photos and/or video for 6 of the 8 "sublocations" of the Kasaï Occidental 2014 distribution, although there are only a fairly small number of photos for each sublocation (around 15 photos per sublocation).67
Because photos are missing for many recent distributions and there are fairly small numbers of photos available for other distributions, we do not find this alone to be convincing evidence that the nets reached their intended destinations.
Distribution reports
During its Ntcheu 2012, Balaka 2013, Dedza 2014, and Dowa 2015 distributions, Concern Universal provided weekly reports on the number of LLINs distributed, the regions which were covered, and problems encountered in reaching particular villages or individuals.68 As we report in our March 2012 update, the Ntcheu 2012 weekly reports note challenges including attempted thefts, double registrations and logistical problems.69 Concern Universal provided a similar level of detail on challenges encountered in the Balaka 2013,70 Dedza 2014,71 and Dowa 2015 distributions.72 These reports increase our confidence that Concern Universal is aware of potential problems and has a system in place to address them.73
Concern Universal has provided weekly reports of its own accord (AMF does not require such reports). AMF does ask partners to provide summary reports on how distributions have gone after they are completed; we have seen such reports for the Ntcheu 2012, Balaka 2013, Dedza 2014, and Kasaï Occidental 2014 distributions, and we expect to see such reports from future distributions.74 The summary distribution reports vary in their level of detail, but all include an overview of the procedures followed during the distribution, a breakdown of the number of nets distributed into sublocations or villages, and details on challenges encountered.75 We believe that these detailed reports provide reasonable evidence that the distributions occurred and nets were sent to the intended locations.
Post-distribution check-up reports
Post-distribution surveys, discussed below, provide a further check on whether LLINs have reached their intended destinations. The surveys that have been completed to date have found relatively high coverage rates; the two completed 6-month surveys showed a coverage rate of 90%, while a completed 8-month survey showed a coverage rate of 82% (see this summary of results). The surveys also find that most of the nets observed are from AMF (although we do not know if this information is collected by asking respondents or by checking the labels of the nets).76 We believe this provides reasonable evidence that a distribution occurred and nets reached their intended locations, though we note that the surveys have some methodological limitations, including that they are conducted by the same organization that carried out the distribution, which may have an incentive to bias the results.
Other evidence
For the Kasaï Occidental 2014 distribution, AMF's distribution partner, IMA World Health, piloted the use of smartphones to record household data, including GPS coordinates, for registration and LLIN distribution.77 We have seen one image using the GPS data collected that appears to show a pin for each household where nets were distributed.78 AMF has also sent us detailed GPS data that shows the GPS coordinates for each household visited.79
Are LLINs targeted at areas with high rates of malaria?
At the highest level, AMF appears to exclusively target countries with known malaria risk.80 Since 2012, AMF’s large-scale distributions have occurred in the DRC or Malawi.81 The World Health Organization reports that both the DRC and Malawi are in the top 23 countries contributing to the number of global malaria cases and deaths; in 2013, both the DRC and Malawi were estimated to have between 50 and 99 deaths due to malaria per 100,000 people.82 Two of the three additional countries where AMF is, as of October 2015, considering funding distributions (more) appear to have somewhat lower malaria mortality rates.83
In its distribution proposal for the Ntcheu 2012 and Balaka 2013 distributions, Concern Universal reported the number of malaria cases in the proposed distribution area over a twelve month period preceding the distributions.84 We have not seen similar data for Concern Universal's Dedza 2014 or Dowa 2015 distributions or for the areas in the DRC where AMF has provided or will provide LLINs.
Based on our understanding of how nets are allocated within countries, we would guess that looking at the malaria rates in the specific districts AMF works in is somewhat misleading and that AMF's funding should largely be considered to be adding to the total number of LLINs that are available for distribution in a country.
Do those who receive the LLINs install them in their homes properly? Do those who receive the LLINs utilize them consistently over the long term?
AMF requires partners to conduct follow-up surveys at 6-month intervals (plus or minus a month) for a period of 2.5 years, or until the next community-wide net distribution in the same area, to determine whether LLINs are present, are hung, and what condition they are in.85 Results are available from 6-, 15-, 24-, and 33-month surveys following the Ntcheu 2012 distribution, 6 and 14 months following the Balaka 2013 distribution, and 8 months following the Dedza 2014 distribution. Details of the methodology used in these surveys are available in our August 2014 update (see the description of the Ntcheu 15- and 24-month surveys) and August 2015 update (see the descriptions of the Ntcheu 33-, Balaka 14-, and Dedza 8-month surveys) on AMF. We discuss the methodology of the Balaka 6-month survey in this footnote86 (details were not yet available when we published the August 2014 update). We believe the methodology used was reasonably high quality and that results are fairly, though not fully, representative of all households that received AMF-funded LLINs. The surveys were conducted by the same organization that conducted the distribution and which may have an incentive to bias the results. We note that have not seen technical details at the level of a high quality academic study on how the surveys were carried out,87 and we therefore have not been able to fully vet the results.
Results from follow-up surveys
Data sources in footnote.88 We have also summarized the results from the post-distribution check ups (PDCUs) of AMF's large-scale distributions here.
| Ntcheu: 6-month | Ntcheu: 15-month | Ntcheu: 24-month | Ntcheu: 33-month | Balaka: 6-month | Balaka: 14-month | Dedza: 8-month | |
|---|---|---|---|---|---|---|---|
| # of households surveyed | 7,689 | 9,250 | 8,939 | 9,500 | 4,536 | 4,535 | 9,100 |
| # of AMF nets received by surveyed households | 15,814 | 18,580 | 18,448 | 20,688 | 7,977 | 8,540 | 15,676 |
| % of nets surveyed of total distributed | 6% | 7% | 7% | 8% | 5% | 6% | 6% |
| % of nets hung | 90% | 85% | 81% | 52% | 87% | 82% | 93% |
| % of nets missing | 1% | 3% | 4% | 3% | 2% | 2% | 1% |
| % nets in "very good" condition | 99% | 89% | 49% | 13% | 70% | 35% | 57% |
| % of nets in "good" condition | 1% | 6% | 28% | 23% | 25% | 39% | 36% |
| % of nets in "viable" condition | 0% | 2% | 15% | 19% | 4% | 11% | 6% |
| % of nets worn out | 0% | 3% | 8% | 45% | 1% | 14% | 1% |
| % of nets used correctly | 98% | 98% | 90% | 98% | 94% | 100% | 100% |
| % of people covered by nets | 90% | 84% | 79% | 56% | 90% | 73% | 82% |
Definitions:
- Nets hung: We have not seen details of how surveyors are instructed to determine whether a net qualifies as hung (e.g., must the surveyor observe the net fully covering the sleeping space, hung in a position that it could conceivably cover the sleeping space, etc.). AMF told us that interviewers are asked to observe whether nets are hung by entering interviewees' houses (rather than simply asking interviewees if they are hung).89 It is our understanding that none of the nets included in the "nets hung" category are "worn out" (i.e., all are in "viable", "good", or "very good" condition).90
- "Very good" condition: LLIN has fewer than 2 holes of less than 2 cm in size.91
- "Good" condition: LLIN has fewer than 10 small holes.92
- "Viable" condition: LLIN has more than 10 small holes or has one large hole.93
- "Worn out": Survey reports from Concern Universal did not provide a definition of this term. AMF has told us in the past that an LLIN is considered worn out if it has multiple large holes and the LLIN is unrepairable, such that it would not provide protection against mosquitos.94
- Used correctly: According to a survey form from 2012, interviewers are asked to "ask the householder to demonstrate how the nets are used at night" and to then select yes or no to the question "are the nets being used correctly."95 AMF told us last year that the procedure had not changed in more recent surveys.96 Note that some of nets included in this category may not be AMF nets and that it is possible that some "used correctly" nets are also "worn out" nets.97
- People covered by nets: We are unsure of how to interpret this metric. We previously believed that this included all people who sleep under a properly hung and not worn out net. However, the data for some post-distribution check-ups is not consistent with this interpretation.98
The percentage of people covered by nets in AMF's post-distribution check-ups is roughly in line with the net usage rates of the trials of bed net efficacy documented in our page on Long Lasting Insecticide Treated Nets, in which small-scale studies of bed net efficacy had net usage rates generally in the 60%-80% range.99
The "decay model" we use to anticipate the lifespan of LLINs assumes that 92% of LLINs are functional and in use for the first year after a distribution, 80% of nets are functional and in use for the second year, and 50% of nets are functional and in use for the third year.100 AMF's post-distribution check ups for the Ntcheu 2012 distribution generally seem to match the decay model: 6 months after the distribution, 90% of nets were hung and 1% were worn out or missing (the model predicts 92% of nets should be functional and in use); 15 and 24 months after the distribution, 84% and 79% of nets were hung and 6% and 12% were worn out or missing, respectively (the model predicts 80% of nets should be functional and in use); and 33 months after the distribution, 52% of nets were hung and 48% were worn out or missing (the model predicts 50% should be functional and in use).
LLINs appear to have "decayed" more quickly in Balaka and Dedza. In Balaka, 70% of nets were in "very good" condition at 6-months compared with 99% in "very good" condition at 6-months in Ntcheu, and 14% were worn out at 14 months compared with 3% in Ntcheu at 15 months. Dedza has a significantly lower percentage of nets in "very good" condition at 8 months than Balaka or Ntcheu had at 6 months (57% compared to 70% and 99%, respectively).
It is not clear to us how to compare the net quality and net usage rates found in AMF's post-distribution check ups to the assumptions in the decay model, in part because it is not clear whether the definition of a "functional and in use" net in the decay model depends on the type of metrics that AMF's data provides. We may do further analysis to more directly compare the model and AMF's data in the future.
Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF?
For the Ntcheu 2012 distribution in Malawi, AMF received data from the National Malaria Control Program on population figures and LLINs already distributed by district, as well as LLINs that had been committed to the country by other funders. AMF estimated, in September 2011, that an additional 909,586 nets were needed to achieve universal coverage in the country,101 of which AMF contributed 268,240.102 We have not done a similar analysis for later distributions.
We would guess that AMF is increasing the total number of LLINs because, across Africa, there are substantial funding gaps for LLINs (more below) and because our impression from following AMF's progress over time is that, due to AMF's more limited funding and, perhaps, greater data requirements, governments often seek funding first from larger funders (particularly the Global Fund to fight AIDS, TB, and Malaria) and then may ask AMF to fill gaps. However, we have low confidence in this guess, and note that countries are sometimes able to choose how they allocate Global Fund grants among malaria interventions (including LLINs, treatment, and diagnosis), so the availability of funding for LLINs from AMF could cause countries to allocate less Global Fund resources to LLINs.
Possible negative or offsetting impact
- Do donated nets displace government health funding? This could be a concern if government funding that otherwise would have been spent on LLINs is spent on other, less worthwhile budget items. We have little sense of how important a concern this is in AMF's case. In the case of the 2012 distribution in Malawi, all anticipated nets were expected from international donors, and the total supply appeared to be still insufficient for country-wide universal coverage.103
- Will insecticide-treated nets continue to be effective? As discussed at our report on insecticide-treated nets, there is strong evidence for the effectiveness of this intervention, however the best evidence for the intervention was collected before LLINs were widely used and there is some evidence that mosquitoes have since adapted to the insecticide used in LLINs, possibly making them less effective. In 2012 we reviewed the evidence on the state of insecticide resistance. We concluded, "We're very concerned about insecticide resistance. It appears that relatively little is known about the extent, causes, and control implications of resistance; that resistance could substantially affect (or even negate) the effectiveness of malaria control… That said, this issue doesn't change our bottom line that LLIN distribution is a highly cost-effective intervention… There is strong evidence that LLINs reduce malaria and save lives and only preliminary/suggestive/mixed evidence that insecticide resistance may reduce their impact." In 2015, we briefly looked for new studies on this topic, but found no major updates.
- Do free LLIN distributions distort incentives for recipients or distort local markets for nets? As discussed at our report on insecticide-treated nets, we feel that there is a reasonably strong case for distributing LLINs freely rather than selling them at market (or even below-market) prices. We also think that the benefits of distributing LLINs freely to a population likely outweigh the negative consequences of distortion in local net markets.
- Could distribution of LLINs be inequitable and unfair, causing problems in the targeted communities? We feel that AMF's process for determining needs for LLINs at the household level is well-suited to ensuring that LLINs are distributed equitably.
- Does AMF divert skilled labor from other areas? In Malawi, net distributions have been conducted by low-level government health staff in partnership with the staff of AMF's partner NGO.104 AMF's partner in Malawi told us in 2012 that government health staff are normally involved in activities such as disseminating health-related information, reporting on levels of stunting and disease, carrying out immunization campaigns, and providing nutrition support.105 We don't know the extent to which net distribution reduces their ability to complete other duties, though we note that net distributions are generally completed within a few days in each local area.106 Diversion of skilled labor may be more of a concern in DRC where 22 senior district health staff were employed as Field Supervisors for the Kasaï Occidental distribution.107
What do you get for your dollar?
Cost per LLIN distributed
Estimate based on data from AMF
Using budgets and estimates provided by AMF, we estimate that the total cost to purchase, distribute and follow up on the distribution of an AMF-funded LLIN is about $5.15 in Malawi and roughly $6.30 in DRC. The very roughly estimated average cost for future distributions that AMF is considering (more) is $5.31 per net.
Note that we prefer to include all costs incurred to carry out a project, not just those that the charity in question pays for itself. We believe that this gives the best view of what it costs to achieve a particular impact (such as saving a life) and also avoids the lack of clarity and complications of leverage in charity. Our estimate of the cost to distribute an AMF-funded LLIN aims to include both costs AMF pays and costs paid by others.
Full details in this spreadsheet.
The estimate for Malawi uses:
- The purchase price per LLIN, calculated from AMF's total LLIN budget and the number of LLINs distributed.
- Actual costs for most of the Balaka 2013 and Dedza 2014 distributions, as reported by Concern Universal, and budgeted costs for most of the post-distribution check ups. It also includes budgeted costs for the Dowa 2015, Ntcheu 2015, and Balaka 2015 distributions.108 AMF told us that the non-net budgets include all monetary costs incurred by Concern Universal, as well as the use of vehicles and managerial staff time;109 it may exclude non-monetary costs such as the use of office space.
- An estimate of non-monetary costs incurred by Concern Universal and local governments, based on estimates provided in 2012 by Concern Universal for its distribution in Ntcheu province.110
- An estimate of AMF's organizational costs per net, including an estimate of the value of donated services and volunteer time. Because these costs generally do not vary with the number of LLINs distributed and remain roughly constant on a per year basis, the per LLIN estimate is highly dependent on the time period used and the number of LLINs distributed in that period. We estimate AMF's organizational costs as $0.67 per LLIN for 2012-2015.111
For DRC, the estimate is rougher. We constructed the estimate similarly, although the information we have for DRC is likely of lower quality.112 The estimate for the DRC is based on:
- The purchase price per LLIN, calculated from AMF's total LLIN budget and the number of LLINs distributed.
- Actual costs for the Kasaï Occidental 2014 distribution and budgeted costs for the Nord Ubangi 2015 distribution.
- An estimate of post-distribution check up costs for the Nord Ubangi distribution. The estimate we have seen for post-distribution check up costs for Kasaï Occidental is not yet complete, so we have relied on the Nord Ubangi estimate.113
- We have used the same estimate of AMF organizational costs and non-monetary and government costs as we used for Malawi.
The higher cost per net in the DRC may be explained by a considerably more difficult operating environment in DRC,114 or our data for DRC may simply be of lower quality.
Our estimates of the cost per net for future distributions that AMF is considering are also rough; they are based on:
- The purchase price per LLIN, based on what AMF told us it expects the cost to be. In two of the countries, AMF expects the nets to be significantly more expensive (30% and 64% more expensive, respectively) due to the governments requiring particular sizes or shapes of nets.115
- An average of actual and budgeted pre-distribution and distribution costs from the distributions in Malawi and DRC.
- An estimate of AMF's organizational costs per net in the future, assuming similar costs to the past and including the costs of extra activities AMF is considering funding (more above). Because we estimate AMF's organization costs as a constant per year cost and have assumed that AMF will deliver more nets in 2016-2018 than it has in the past few years, we estimate these costs will be about $0.15 per net in the future, compared with $0.67 in our Malawi and DRC estimates. AMF's organizational per net cost will likely be higher than we've estimated if it delivers fewer nets or if it expands its staff.
Note that we have weighted our average cost per net for AMF's future distributions by the number of nets we expect AMF to distribute if it raised enough money to fully fund all the distributions it was considering as of October 2015; these distributions would take place in 2016-2018. We cannot provide more detail for our calculations because the costs for each distribution are still being negotiated.
Global estimate
The Roll Back Malaria Partnership, which estimates funding gaps for LLINs across all African countries,116 uses an estimate of $2.70 to purchase a LLIN and $2.50 for all non-net costs of a distribution, for a total of $5.20 per LLIN.117 This assumption is used in cases where country-specific data is not available. Note that we have not seen details about where this estimate comes from, so we put little weight on it. The Co-Chair of the Roll Back Malaria Harmonization Working Group told us that non-net costs can vary significantly by country and that non-net costs in DRC may be more than $5 per LLIN.118 We have little information on how reliable the Roll Back Malaria Partnership's estimate is likely to be. Broadly, this estimate is consistent with the data we have seen from AMF. We have used estimates based on data from AMF in our cost per life saved calculations, because we have more information on their reliability. We do not believe our estimate of AMF's cost-per-net ought to be compared directly to the Roll Back Malaria Partnership's estimate, given how uncertain we are about the latter.
Cost per life saved
Using $5.31 as the average cost per net in the countries that AMF is considering future distributions in, we estimate the cost per child life saved through an AMF-funded LLIN distribution at about $2,838.119
Note that our cost-effectiveness analyses are simple models that do not take into account a number of factors. For example, our model does not include other potential benefits of LLINs, such as non-fatal cases of malaria prevented, prevention of deaths in age groups other than under-5 year olds, or prevention of other mosquito-borne diseases. It also does not include possible offsetting impacts or other harms. We do include possible developmental impacts on children who sleep under an LLIN: we estimate that sleeping under an LLIN provides the same developmental impacts that a deworming pill provides. However, most of the benefit provided by LLINs is in the lives that LLINs save, not in their impact on development.
There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.
The full details of our cost-effectiveness analysis are at our report on mass distribution of LLINs.
Is there room for more funding?
In 2015, AMF has not signed any new distribution agreements for distributions in the near future. It signed agreements for four large distributions in Malawi in 2018.120 We are still uncertain about the rate at which AMF is capable of spending its funding on additional distributions. AMF currently has approximately $18.5 million that it could use for future distributions. AMF has told us that it has a pipeline of possible future net distributions that add up to roughly $100 million beyond what it currently holds.121
Rate of funds moved
In November 2013, we wrote in this blog post that we believed AMF did not have room for more funding because it had raised over $10.6 million due to GiveWell's recommendation since 2011 and had not spent the bulk of these funds. In 2014, AMF completed and signed agreements for several large distributions, increasing our confidence that it could productively use the donated funds it held.122 As of October 2015, AMF had not signed any new distribution agreements for the near future, although it signed agreements for four large distributions in Malawi in 2018.123
There is still significant uncertainty as to AMF's ability to find new partners and countries to work with. To find a sufficiently large distribution often requires negotiating with the national malaria control programs of countries in sub-Saharan Africa, which we perceive to have some discretion in which funders they work with, and which we perceive to be choosing funders based on a variety of factors including size and reporting requirements.124 Because AMF is able to fund only a relatively small piece of a given country’s distribution, but has substantial reporting requirements, there may be fundamental reasons for governments to prefer other funders. We believe that this is the most important and fundamental explanation for AMF's difficulties in finalizing distributions in the past.
We believe that other factors may have also played a role. We discuss this topic in detail in this blog post. AMF believes that the factors that have played the largest role in delaying recent distributions include a) the reluctance of a funder AMF has been working closely with to pay for post-distribution check ups and b) the fact that AMF has never previously partnered with the countries it is in discussions with for a large number of nets.125 It is our understanding that there have been two cases in which AMF could have likely secured a deal more quickly if it had agreed to fund post-distribution check ups; we are not sure if AMF's decision to push for a different funder to take on those costs was optimal (the agreements are still being discussed).126
Pipeline of potential distributions
As of October 2015, AMF told us that it was in active discussions about large potential distributions (over 1 million LLINs) in five countries, none of which it has completed a large distribution in before, and in preliminary discussions with one other country. We've briefly summarized some of the details of these potential agreements in the following list (note that AMF prefers to keep the names of countries it is in negotiations with confidential until agreements are signed; we've used the pseudonyms that AMF assigns the countries on its future distributions page):127
- Country K – 2 million LLINs for 2016: AMF has a potential funder for non-net costs. AMF does not yet have a distribution partner for this distribution. It is possible that AMF could work with the country's public health system as a distribution partner (rather than finding an in-country NGO to distribute the nets).
- Country I – 3.3 million LLINs for 2016-2017: AMF has identified a distribution partner in-country that it is enthusiastic about working with. AMF is in discussions with a potential funder of the non-net costs. In addition to the 3.3 million LLINs, AMF would consider offering to fund 3.5 million LLINs for 2018.
- Country J – 3.2 million LLINs for 2016-2017: AMF has identified a distribution partner in-country that it is enthusiastic about working with. AMF is in discussions with a potential funder of the non-net costs. In addition to the 3.2 million LLINs, AMF would like to offer to fund 8.4 million LLINs for this country in 2017-2018.
- Country C – 2 million LLINs for 2016-2017: AMF has a potential funder for non-net costs. AMF does not yet have a distribution partner for this distribution. In addition to the 2 million LLINs, AMF would also like to offer to fund another 2.1 million LLINs for 2016.
- Country Z – 15 million LLINs for 2016-2017: AMF has started preliminary discussions with the UK's Department for International Development (DFID) about funding distributions in Country Z, which needs approximately 13 million nets in 2016 and 15 million nets in 2017.128 AMF has been asked if it could fund 15 million LLINs in this country, which would cost AMF approximately $37.5 million, although AMF expects this figure to be revised over the next several months.129
AMF has told us that the discussions for distributions in countries I, J, and C all involve the same funder for non-net costs.130 We are slightly concerned that AMF's pipeline will be quite short if that funder ultimately decides not to work with AMF. AMF has told us that even if discussions fall through for one of the countries with this funder, arrangements could still work out for one of the other countries.131
Finally, AMF is considering co-funding a "Malaria Unit" with Concern Universal in Malawi (details above); AMF has committed approximately $400,000 over the next three years to this project.132
Current funding situation
AMF currently holds approximately $18.5 million in funds it could allocate to future distributions.133 AMF told us that it expects to receive approximately $1.5 million during the 2015 giving season (approximately October 2015 through December) from non-GiveWell donors and that it is in preliminary conversations with one possibly major donor about additional funding.134
AMF has decided to prioritize funding the distributions in its pipeline in the following order:135
- AMF's first priority is to fund the distributions it is discussing with countries I and C; these distributions require a total of $18.7 million in funding. AMF currently has enough to fund these distributions.
- AMF's next priority would be to fund a distribution it is discussing with Country K ($8.2 million required) or an additional distribution in Country C ($6.3 million required). Cumulative need: $14.5 million.
- AMF's next priority would be to fund the large distribution in Country Z, for which AMF would need roughly $37.5 million. Cumulative need: $52 million.
- AMF's next priority would be to fund a distribution in Country J, for which it would need an additional $9.6 million. Cumulative need: $61.6 million.
- Finally, AMF would look to fund additional distributions in Country I for 2018 ($13.1 million required) or additional distributions in Country J, which has a large net gap (up to $25.2 million required). Cumulative need: Approximately $100 million.
We believe it is unlikely that all discussions will be successful. AMF does not currently have enough funding to move forward with most of these opportunities, thus AMF is considering dropping discussions with Country J and possibly Country K.136 AMF believes that distribution agreements will be signed for countries I and C within the next several months.137 We note that, in the past, negotiations have often taken significantly longer than AMF has estimated.
We believe AMF could effectively utilize $40-75 million more. We note that distributions can take over a year to plan138 and funding that AMF receives in the next year may not be used or committed in that year. We expect to continue to follow AMF's progress of building a track record of finalizing and carrying out distributions over the next year.
Effect of additional funds
AMF has told us that the number of opportunities it considers at one time is limited by its funds and that the effect of having additional funds is largely to increase the size of the distributions it considers, rather than increasing the number of distributions it considered.139 With additional funding, AMF would likely offer to fill a larger portion of the funding gaps for the countries with which it is currently in discussions.140 The type of opportunities it will consider depends on how much funding it has available.141 In 2014, AMF told us that it had had several conversations about potential distributions that have not moved forward because AMF was unable to consider a distribution of sufficient size.142 We do not have more details on these conversations, particularly on whether other issues contributed to the discussions not going forward. We do not know if proposed distribution size contributed at all to delays in signing distributions in 2015; the discussions in 2015 were for larger distributions than AMF previously funded.
Global funding gaps for LLINs
The Roll Back Malaria Partnership's Harmonization Working Group estimates funding gaps for LLINs across all African countries. In May 2014, October 2014, and October 2015, we asked the Co-Chair of the Harmonization Working Group for updates on the size of the funding gap. On all occasions she told us that there remained significant gaps in funding for LLINs.143
It most recently published a gap analysis in October 2015. At that time, it estimated that about 245 million additional LLINs would be needed in 2015-2017.144 There appear to be particularly large gaps for Nigeria, Sudan, and Uganda in 2016. In 2017 the gap analysis shows many countries with a significant gap (over 1 million nets needed).145
While we would guess that other funders will contribute some LLINs, the size of the gaps and the persistence of the 2015 gap late into the year, as occurred in 2013 as well (we don't have a comparable estimate for 2014),146 suggests that there is, and will continue to be, unmet need for LLINs in African countries.
AMF as an organization
We believe the Against Malaria Foundation to be an exceptionally effective organization:
- Track record: We feel AMF has built up a strong track record of raising money, finding smaller distribution partners, and getting partners to report information publicly at an unusual level. More recently, AMF has begun to demonstrate its ability to implement its model at significantly larger scale.
- Communication: AMF has always communicated extremely clearly and directly with us and given thoughtful answers to our critical questions.
- Self-evaluation: AMF has invested heavily in self-evaluation, going above and beyond what’s usual for data collection on bed net distributions: it has demonstrated a commitment to collecting long-term net usage data.
- Transparency: AMF appears to value transparency as much as any organization we’ve encountered. It hasn’t just shared information with us; it publishes significant amounts of useful information publicly on its own, far more than the norm. We have never seen AMF hesitate to share information publicly (unless it had what we consider a good reason).
However, we see some potential room for improvement:
- AMF does not seem to be as aggressive and ambitious about growth and capacity as it could be. It has operated for 11 years and still has only two full-time staff members; it does not appear to us to have built as robust a fundraising operation as it could. Recently, AMF has informed us that it is considering hiring a Head of Operations in 2016, depending on the anticipated workload.147 AMF is also considering hiring someone to assist with administrative work.148
- AMF has not conducted any field visits to distribution partners (as far as we are aware). We see this as a negative since we find such visits valuable.
More on how we think about evaluating organizations at our 2012 blog post.
Sources
- 1
"How will the money raised be spent? On bed nets (mosquito nets). Specifically, long-lasting insecticidal nets (LLINs)." AMF Frequently Asked Questions.
- 2
AMF Distributions Adding up the #LLINs column for the Ntcheu District 2012, Balaka District 2013, Dedza District 2014, Kasaï Occidental 2014, and Dowa District 2015 distributions results in 1,740,852 nets.
- 3
- AMF Distributions Adding up the #LLINs column for the Ntcheu District 2015, Balaka District 2015, and Nord Ubangi 2015, Equateur Pr. distributions results in 1,275,468 nets.
- Rob Mather, AMF Founder, email to GiveWell, September 9, 2015 AMF notes that the Nord Ubangi distribution will run from Nov 2015 through Mar 2016.
- 4
AMF Distributions Adding up the #LLINs column for the Dedza District 2018, and Dowa District 2018, Ntcheu District 2018, and Balaka District 2018 distributions results in 1,480,000 nets.
- 5
AMF Distributions See the North Idjwi Island, South Kivu and South Idjwi Island, South Kivu distributions, each of which are for 62,000 nets.
- 6
- 7
A mop-up distribution for an additional 45,000 nets that were still needed after the initial distribution was completed in May 2015. More here.
- 8
AMF worked with Concern Universal, Médecins Sans Frontières (MSF) Belgium, and GOAL Global, and others. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 9
This understanding is from many conversations with AMF and following AMF's progress over time.
- 10
Note that the process of agreeing on expectations for the distribution often involves additional actors. For example, sometimes the distribution partner is not paying for non-net costs, in which case the non-net costs funder might also be part of the discussions about expectations. Additionally, AMF works closely with countries' National Malaria Control Programmes (NMCPs); NMCPs must approve the net distributions and often are willing to use government resources (e.g., community health workers) to assist with a net distribution.
- 11
- "This is a list of the countries with known gaps and where there are significant contiguous areas without nets, or a significant percentage required, and for which the estimate of need is believed to be reasonably accurate. It does not include countries where there are gaps, typically up to 40% of what the nation needs, but they are spread more uniformly across the country and would therefore require an ‘in-fill campaign’. An in-fill campaign is different from a so-called ‘universal coverage campaign’ because the percentage installed base of nets is higher in the former case and so a pre-distribution registration survey (PDRS) is an absolute requirement to ensure an efficient allocation of nets. Our methodology would lend itself to these campaigns if the relevant National Malaria Control Programme (NMCP) were to embrace a detailed PDRS. The list does not include, in our view, other countries where the need has not yet been quantified. Given there are many countries with needs estimated, we have not chosen to seek out other countries in need of nets. Our assumption is groups like AMP will be a source of reporting on additional countries as quantified needs emerge." Rob Mather, AMF Founder, email to GiveWell, August 8, 2012.
- In September 2015, we checked in with AMF about its process for determining in which countries it works. Rob Mather noted that AMF continues to review malaria prevalence data (where that data exists), although maintains a level of skepticism about that data given that it can be unreliable. Even so, AMF feels comfortable drawing conclusions about which countries have high malaria mortality burdens based on the data it sees and the conversations that it engages in. AMF checks the following sources to keep up-to-date on which countries have a significant malaria burden:
- The Alliance for Malaria Prevention, which sends out a weekly email with malaria-related data and information.
- The African Leaders Malaria Alliance (ALMA), which AMF has conversations with on a quarterly basis.
- In-country partners, who frequently attend malaria task force meetings and have recent news
- Members of malaria advisory groups
- Other connections
Before AMF decides to approach a country to offer funding for nets, it has many conversations to confirm the level of need that country has with the other actors that are working on malaria in the country. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 12
- Rob Mather, AMF Founder, email to GiveWell, November 20, 2012.
- "Initial (directional) information
a) Sources of collated numbers (eg AMP, ALMA)
We liaise with groups that collate statistics on net numbers. AMP’s (Alliance for Malaria Prevention) weekly email is one such source as you correctly identify and ALMA’s (African Leaders Malaria Alliance) monthly update is another. The numbers are not always up to date and do not always agree. However they are useful as a first alert that there is a gap in a national campaign. I have attached two emails that show you the sort of thing we receive. b) Distribution Partners Groups involved in distributions in particular countries may be the first to be aware there is a net gap and they may approach us by email or telephone. c) Individuals/Advisors (to national campaigns). Those involved in national campaigns have also approached us in the past to ask if we are in a position to help close a net gap. This may be a technical malaria advisor who has just come out of a meeting where it has become apparent there is a gap in funding for nets for a near-term program. We refer to this information as ‘directional’ as in all cases detailed information is then required to confirm there is a gap." Rob Mather, AMF Founder, email to GiveWell, June 29, 2011.
- 13
Examples:
- Alliance for Malaria Prevention Conference call minutes (June 22, 2011) Pg 13.
- Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, email to Rob Mather, June 26, 2011.
"GiveWell: Neither of the AMP emails you sent (May 28, 2011 and June 22, 2011) nor the ALMA email (from June 26, 2011) provide a figure for net gap in Malawi. Do you know why this is? Did you use these groups for directional information in the case of Malawi? Has the net situation changed in Malawi recently?
AMF: I thought it did – about 500,000 nets. However, a number we needed further proof on as our assessment was these were indicative numbers and not hard and fast numbers. We needed hard and fast numbers to be able to commit funds for nets as we would not do so if there wasn’t a gap. Liaising with the Global Fund, PMI (the two major funders) and the NMCP was, and is, key to this process. We have yet to receive confirmation of net gap which is why we have not as yet committed funds for nets. In part, this is due to the NMCP being required by the Global Fund (rightly in our view) to re-tender a particular element of the net distribution campaign and that has taken priority over work to finalise numbers. As with us, this has halted Global Fund money for nets until all is resolved satisfactorily. I also believe in part it is due to the NMCP having estimates and now they are being quizzed by us and others, they are having to think what to do – admit they are estimates (which we don’t necessarily have a problem with but we have asked to see the assumptions and logic behind those estimates) and/or because they are now coming round to the viewpoint (which we hold) that given the number of perfectly usable nets in place being above 15% of nets needed country-wide, it is economic and has other benefits, to conduct household by household surveys to determine exactly where (on a household basis) nets are needed. I can run through the $9m (cost of nets saved) vs $2-3m (cost of surveys country-wide) economic argument later." Rob Mather, AMF Founder, email to GiveWell, August 6, 2011. - 14
- "Issues on quantification: it has been raised that LLINs durability in the country is of 2 years rather than 3 years + the country needs to account for existing nets
- GF [Global Fund] has confirmed that an AMP TA [Alliance for Malaria Prevention Technical Assistance] mission should take place at the latest in June 2011
- Strong routine delivery system in place - country will be looking at accounting for existing nets
- Malawi would be a good case study for how continuous distribution works - one of the countries flagged by VCWG [Vector Control Working Group] for case study.
- Malawi achieved UC [universal coverage] by the end of 2010 in 3 of 27 districts"
Alliance for Malaria Prevention Conference call minutes (June 22, 2011) Pg 13.
- 15
- "Malawi...LLIN gap: 2.7 million w/o taking into account any existing LLINs, or 1.5 million if the nets they assume are there are accounted for during HH registration." Alliance for Malaria Prevention Conference call minutes (October 26, 2011) Pg 19.
- As of November 2011, the Alliance for Malaria Prevention had posted minutes from its conference calls since August 3, 2011 at Alliance for Malaria Prevention Conference call minutes
- 16
"We will typically approach the NMCP (National Malaria Control Program) directly and a) ask if there is a gap; b) ask for data on i) the population, number of nets required, number of nets previously distributed, number of nets considered to still be ‘usable nets’, number of nets required to achieve universal coverage (and all of the above on a regional/district basis) and ii) data on existing funding commitments so we can see who else is funding nets and at what level." Rob Mather, AMF Founder, email to GiveWell, June 29, 2011.
Data in AMF Malawi universal coverage calculations (September 26, 2011). In addition, AMF provided the email correspondence it had with the Malawi National Malaria Control Program, which involved extensive back-and-forth about the sources of the data and the assumptions used in the estimates of net needs.
- 17
Tossa Kokou, National Program Manager of the National Malaria Control Program, Togo, email to Rob Mather, May 15, 2012.
- 18
Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.
- 19
Rob Mather, AMF Founder, email to GiveWell, August 8, 2012.
- 20
@AMF LLIN Distribution Proposal Form@. For the Malawi distribution proposal, see Concern Universal Ntcheu 2012 distribution proposal.
- 21
"[GiveWell]: I believe you said on our September call that AMF now requires a planning document to be created with each distribution partner before a distribution. Is this correct? It’s possible I misinterpreted.
[AMF] You are correct. We now require a detailed planning document. See recent Ntcheu and Balaka distributions. This has replaced the previously requested Distribution Proposal which we see as less useful now given: a) we now put in place legal agreement documents, b) each distribution page on the website covers the main points of the distribution, and c) what we seek (generically) in a distribution is covered elsewhere on the website."
Rob Mather, AMF Founder, email to GiveWell, November 9, 2015
- 22
Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.
- 23
Concern Universal Ntcheu 2012 pre-distribution registration survey data
Concern Universal Balaka 2013 pre-distribution registration survey data
Concern Universal Dedza 2014 pre-distribution registration survey data
Concern Universal Dowa 2015 pre-distribution registration survey data - 24
IMA World Health, Kasaï Occidental 2014 distribution data
- 25
- 26
- 27
- AMF Distributions
- "For Balaka and Dedza we have nom photos. CU have told us they lost those files. Procedures are in place to ensure this does not happen again." Rob Mather, AMF Founder, email to GiveWell, November 9, 2015
- "The Dowa photos are on a corrupted hard drive that is no[w] being processed by a tech person. The tech folks (at CU in the UK) have confirmed they can recover the photos so we expect to add them before the end of the year, hopefully before the end of November." Rob Mather, AMF Founder, email to GiveWell, November 9, 2015
- 28
Over the last few years, we have heard conflicting messages from AMF about this documentation:
- AMF previously told us that Concern Universal had forgotten to take photos for the Balaka 2013 and Dedza 2014 distributions.
- "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
[AMF:] No, CU did not take photos of this one. They are aware of this slip and there will be photos and video footage from Dedza. Photos and videos are also included in the agreement for the DRC-Kasaï distribution." Rob Mather, AMF Founder, email to GiveWell, May 22, 2014. - Originally, AMF did not think that photos were taken of the Dedza 2014 distribution. Rob Mather, AMF Founder, email to GiveWell, June 19, 2015
- "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
- In September 2015, AMF informed us that photos were taken. We are not sure what the cause of this confusion was. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- In November 2015, AMF told us that photos were taken, but the files were lost: "For Balaka and Dedza we have no photos. CU have told us they lost those files. Procedures are in place to ensure this does not happen again." Rob Mather, AMF Founder, email to GiveWell, November 9, 2015
- AMF previously told us that Concern Universal had forgotten to take photos for the Balaka 2013 and Dedza 2014 distributions.
- 29
- 30
- AMF told us that future distributions would have the same requirements for photos and video as have past distributions. Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.
- Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- AMF noted that photos were more important when AMF funded smaller distributions - for example, AMF would require a photo of each village sign where nets were distributed as an extra check that nets arrived at their intended destinations. Rob Mather, AMF Founder, phone conversation with GiveWell, November 10, 2015
- 31
AMF page on Ntcheu 2012 distribution
AMF page on Balaka 2013 distribution
AMF page on Dedza 2014 distribution
AMF page on Dowa 2015 distribution - 32
- IMA World Health, Kasaï Occidental 2014 distribution report
- IMA also produced a technology report on the use of smartphones in the distribution. IMA World Health, Kasaï Occidental 2014 technology report
- 33
- "Post-distribution Check-Ups (PDCUs) are carried out every 6 months for 2.5 years (so 5 checks) with 5% of households in each Health Area selected at random and visited unannounced. These data are put in electronic form." AMF Summary features of an AMF distribution
- AMF used to require that distribution partners conducted follow-ups every 6 months for up to 3 or 4 years.
- "[AMF:] Post‐Distribution Surveys (see Note 3). Please confirm you will carry out Post‐Distribution Surveys (PDSs) every 6 months post‐distribution for a period of up to four years to assess the level of net usage (hang‐up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households.
[Concern Universal:] I can confirm Concern Universal and Dedza District Council will conduct a Post-Distribution Survey every six months following the distribution." Concern Universal Balaka 2013 distribution proposal, Pg 2. - Concern Universal Dedza 2014 distribution proposal, Pg 2 contains the same language.
- Response to this requirement from Amani Global Works for North Idjwi Island 2014 distribution: "Yes, we will conduct Post-Distribution Surveys every 6 months for up to 4 years, and provide you with the findings." Amani Global Works North Idjwi Island 2014 distribution proposal, Pg 1.
- IMA World Health agreed to conduct surveys every 6 months for 3 year post-distribution. IMA World Health and AMF distribution agreement for Kasaï Occidental 2014, Pgs 3-4 and IMA World Health and AMF distribution agreement for Nord Ubangi 2015, Pgs 3-4.
- "[AMF:] Post‐Distribution Surveys (see Note 3). Please confirm you will carry out Post‐Distribution Surveys (PDSs) every 6 months post‐distribution for a period of up to four years to assess the level of net usage (hang‐up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households.
- 34
- AMF told us in February 2015 that the 6-month PDCU for the Kasaï Occidental 2014 distribution was scheduled for May 2015. Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015. Note that since the distribution ended in October 2014, the 6-month mark (at which PDCUs are intended to occur) would have been April 2015.
- Generally, PDCUs are expected to occur six months after a distribution. AMF told us that the first Kasaï Occidental 2014 PDCU was completed in June and July 2015, 8 months after the initial distribution. AMF also told us that they have received the PDCU data in mid-August. Rob Mather, AMF Founder, email to GiveWell, September 9, 2015. As of November 2015, we had not seen this data.
- AMF told us that the 3-month PDCU for the flood-affected districts in Malawi had been rescheduled to be a 6-month PDCU and that it would occur in late 2015. Rob Mather, AMF Founder, email to GiveWell, October 12, 2015
- 35
Rob Mather, AMF Founder, email to GiveWell, October 12, 2015
- 36
- We discuss case rate data from Ntcheu in our February 2013 update on AMF.
- "[AMF:] Please confirm you are able to provide monthly malaria case rate data going back at least 12 months for each health centre/clinic in the distribution area and will continue to provide monthly data for a period of four years post‐distribution. This ensures we understand pre‐distribution malaria levels and can monitor them post‐distribution.
[Concern Universal:] Malaria case rate information for each of the 16 Health Centres is available from the Balaka District Health Office using the Malawi Government’s national Health Management Information System (HMIS). This information will be sent to AMF separately." Concern Universal Balaka 2013 distribution proposal, Pg 2. - Concern Universal Dedza 2014 distribution proposal, Pg 2, contains the same language as above
- 37
- Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015
- AMF has asked for malaria case rate data from Concern Universal for 12 months preceding and 4 years following the distributions in Malawi. We discuss case rate data from Ntcheu in our February 2013 update on AMF.
- "[AMF:] Please confirm you are able to provide monthly malaria case rate data going back at least 12 months for each health centre/clinic in the distribution area and will continue to provide monthly data for a period of four years post‐distribution. This ensures we understand pre‐distribution malaria levels and can monitor them post‐distribution.
- [Concern Universal:] Malaria case rate information for each of the 16 Health Centres is available from the Balaka District Health Office using the Malawi Government’s national Health Management Information System (HMIS). This information will be sent to AMF separately."
Concern Universal Balaka 2013 distribution proposal, Pg 2. Concern Universal Dedza 2014 distribution proposal, Pg 2 contains the same language.
- AMF shared data from Ntcheu through through November 2012. AMF page on Ntcheu 2012 distribution We do not believe this data is high quality. In August 2013, it told us that it planned to publish an audit of the data quality by the end of September 2013: “AMF has requested additional data from local health centers and is in the process of
reviewing it. It is planning to publish the results of this audit by the end of September.
At this point, it still believes that the corrected data will show a substantial (~50%+)
reduction in malaria cases in Ntcheu after its distribution there but is still assessing the
level of accuracy and therefore reliability of the malaria case rate data from Ntcheu.” Rob Mather, AMF Founder, phone conversation with GiveWell, August 15, 2013. - Later, AMF told us that it expected to post data in early 2014. See our August 2014 update on AMF
- In November 2014, AMF told us that it expected to share more recent data by the end of January 2015. AMF told us it has been waiting to collect data through to September 2014 to be able to review data for a full year and compare it with the prior year due to changes in the way data is recorded in the health centers. Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.
- 38
Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.
- 39
Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.
- 40
We've tracked the progress of AMF's collection and publication of malaria case rate data for several years. AMF told us several times that it was close to publishing data. Most recently, in June 2015, AMF told us that it currently has malaria case rate data, but has not prioritized preparing the data for publication. AMF also told us that the data is likely to be poor quality, and that it will put little to no weight on the results. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.
- 41
Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.
- 42
Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.
- 43
- 44
- AMF page on non-net costs.
Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015 - GiveWell summary of AMF large-scale distributions
- AMF page on non-net costs.
- 45
- "AMF funding non-net costs can remove the final barrier to a distribution proceeding and avoid potential lengthy delays, of up to a year, given the ideal timing of a distribution relative to other events, for example a rainy season." AMF page on non-net costs.
- AMF has told us that the administrative burden of managing the financials for certain non-net costs (such as post-distribution check ups) are greater for AMF than for partners that have staff in-country. We aren't entirely sure why this is the case, although it seems plausible to us that having staff in-country would be useful for this purpose. AMF mentioned that in-country staff are useful for a) determining if budgets are correct, b) monitoring how money is spent, and c) dealing with small changes that a distribution partner might want to make to the budget. AMF also noted that many of its partners already have financial teams in-country who know how to manage international cashflows; AMF believes it makes sense to make use of these experienced teams. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 46
- Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015
- This research will (a) establish baseline rates of mosquito susceptibility to the insecticide used on most LLINs and (b) compare the effectiveness of LLINs used in most distributions and a newer type of LLIN that is more expensive but may be more effective at killing mosquitoes. Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.
- 47
AMF sent us a draft research proposal for the study in early 2015; some details are in our May 2015 update. AMF sent us what we believe was the final version of the research proposal in June 2015; we have not yet reviewed the final version. AMF insecticide research proposal from the London School of Tropical Medicine
- 48
"We are close to concluding discussion with Concern Universal for AMF and CU to co-fund a Malaria Unit (a larger team of people than currently in place dedicated to malaria control related work) in Malawi led by a senior CU Manager and employing 12 Malawians (10 additional hires)." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 49
"The aim of this unit will be to provide more resource to allow 1. improved efficiency in managing the four contracted three-yearly distributions and associated work and 2. allow us to together innovate and develop additional malaria control support for the four districts and the NMCP, specifically: i) intended close liaising with Health Centre re malaria case data and elements related to the monitoring/recording of malaria data i.e. stock levels of RDTKs, rubber gloves and other diagnosis equipment; qualified staff able to test for malaria, presence at the clinic so diagnosis can happen; systems, capacities etc) ii) research (data), discussion and involvement in ways of ensuring 80% sleeping space coverage throughout the three year net-distribution cycle, including investigation of ‘injection strategy’ net distributions involving mini-mass distributions at two years post-distribution and in the subsequent distribution cycle at one-year post distribution." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 50
The distributions and post-distribution check-ups occur frequently enough that having a trained, consistent staff to manage them is worthwhile. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 51
- "This would be an initial three year commitment from AMF of US$413k...Savings made on non-net costs (with the Malaria Unit funded, some costs included in non-net cost budgets would already be covered) we estimate to be of the order of US$70k, and perhaps as high as US$95k per distribution of which there are four in a three year cycle i.e. US$280-380k in total... If all progresses well we may be able to publish plans in the next six weeks once budgets have been agreed and assuming approval from the Malawi NMCP. We do not anticipate problems, although there is a work visa issue to resolve." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- AMF Malaria Unit draft budget
- 52
Over the last several years, AMF believes that its high net hang-up rates have begun to catch the attention of a number of groups in Malawi, including the NMCP, local organizations that distribute nets, and large international NGOs. AMF also told us that it encouraged Concern Universal to take a more active role in a task force on malaria control in Malawi. This meant that as interest in learning from AMF and Concern Universal grew, Concern Universal was available to explain AMF’s methodologies at the malaria task force meetings, and it was eventually decided to implement some of AMF’s practices for the upcoming distribution. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
- 53
Rob Mather, AMF Founder, email to GiveWell, October 12, 2015
- 54
"Although AMF won’t be managing this process, it is interested to observe how multiple districts will carry out distribution according to this particular data-driven approach. AMF believes that it will be allowed access to that data, either directly or via its in-country partners." GiveWell’s non-verbatim summary of a conversation with Rob Mather and Peter Sherratt on February 6, 2015
- 55
See our summary of post-distribution check up (PDCU) results (the follow-up surveys) here.
See our summary of the timing of AMF's PDCUs here, Sheet "PDCUs".
- 56
- Rob Mather, AMF Founder, phone conversation with GiveWell, July 19, 2012.
- "[GiveWell]: We know that AMF does not distribute bed nets to recipients if they already have an LLIN in good condition, but does AMF distribute bed nets to people if they have an untreated bed net or an ITN in good condition (instead of an LLIN)?
[AMF]: Yes. If it is not an LLIN it is not counted in any pre-distribution assessment of net need."
Rob Mather, AMF Founder, email to GiveWell, September 16, 2015
- 57
Concern Universal Ntcheu 2012 pre-distribution registration survey data.
Concern Universal Balaka 2013 pre-distribution registration survey data. - 58
@GiveWell Notes from Site Visit with Concern Universal in Malawi (October 2011)@.
- 59
Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.
- 60
- "IMA obtained population data for the year 2012 from the Ministry of Health and applied the MOH’s estimate for population growth of 3% per annum." IMA World Health, Kasaï Occidental 2014 distribution report, Pg 6.
- "There were many occasions where the number of nets needed for a community was overestimated. This was discovered in the registration and hang up because of diligent work done. Lesson learned Better population estimates are needed and 1.8 people per nets should be the ratio always used." IMA World Health, Kasaï Occidental 2014 distribution report, Pg 46.
- In cases where too few nets are sent to a location, AMF's distribution partner typically performs a "mop up" exercise in which it orders extra nets and distributes them several months later to the households that didn't receive the nets the first time around. Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015
- "If there are a significant number of nets left over – more than a few hundred for example, the nets are moved to the next distribution. An example would be the 5,770 so nets left over in Balaka (the need was assessed as 149,500 and we had moved in 160,000 nets to ensure we had enough nets to cover any additional need and the final number of nets distributed after mop-up was 154,230) which were moved up to Dedza and distributed as part of that distribution. The number of nets ordered and moved in to Dedza reflected that 5,700. If the number of nets left over is in the hundreds, they are passed to the District Health Officer (DHO) to be included a nets for routine distribution. Sometimes there has been a small, several hundred net shortfall and that gap has in the past been filled by nets in zone from other sources." Rob Mather, AMF Founder, email to GiveWell, September 16, 2015
- 61
"Differences [between the Nord Ubangi distribution and the Kasaï Occidental 2014 distribution]
1. The North Ubangi distribution will be a two-phase distribution - a distinct pre-distribution phase and a subsequent distribution phase. In West Kasaï establishing net need per household and distributing of nets was contemporaneous.
2. Distribution of nets will be done from distribution points rather than at the household
3. The ODK registration form has been re-designed to better capture the required data, to eliminate some types of error and to reduce the chance of others.
4. Operational lessons learned from the West Kasaï distribution will be implemented.
5. An experienced, full time technology manager has been employed by IMA meaning there will be more extensive and easier liaising with AMF on data elements of the distribution. We have met the new technology manger and have spent a week working with him and are impressed.Similarities
1. The same smartphone-based data collection technology will be used as in West Kasaï.Comment
IMA has agreed there are benefits to conducting a separate, earlier pre-distribution phase during which accurate household-level net need is established. IMA are less able to estimate accurately population and net need in North Ubangi compared to what they felt was possible in West Kasaï.IMA believes point distributions rather than at-household distribution will be more cost-effective." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 62
- "For the subsequent verification visits (which Givewell witnessed) HSAs don't go into every household- instead they call a village register at a central location to check with the village leaders and individuals as to whether the data from the house to house survey is accurate. Givewell suggested that we should be doing some randomised checks on individual households at this verification stage- we have agreed and have now incorporated these into our verification model." Robin Todd, Concern Universal Malawi Director, email to Rob Mather, November 18, 2011.
- Concern Universal started this practice in the Ntcheu 2012 distribution at GiveWell's suggestion.
- After the first two weeks of spot checks, the distribution partner reported that it had not found any discrepancies between the number of nets community members said they needed during the verification meeting, and the number of nets found to be needed during the subsequent household visit. Robin Todd, Concern Universal Malawi Director, email to Rob Mather, November 18, 2011.
- AMF has told us that spot checks are now carried out with all distributions. Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.
- 63
"In 2010, Senegal and Cross River State in Nigeria worked on mop-up campaigns following earlier integrated campaigns targeting households with children under five years of age. In both countries, trained health workers or volunteers undertook a household registration to determine:
- the total net need for each household (this was based on one LLIN for two people, rounding up in the case of odd numbers of people in the household, but Senegal also looked at the number of habitual sleeping spaces in each household)
- how many nets each household already had
- how many nets in each household were still viable (in Senegal this number was based on net condition, while in Cross River State, it was based on how long the net had been hanging)
- how many new nets each household would need for full coverage
Both countries had previously undertaken post-distribution surveys which showed high household coverage with LLINs, but during the mop-up exercise they experienced challenges with finding nets in households. Significantly lower numbers of nets were found (50—60 per cent) than would have been expected based on the surveys...
In both countries, it seemed that families often hid nets once word spread that ownership of nets meant no new nets would be received. Despite efforts to encourage families to hang pre-existing nets prior to the household registration in Senegal, people hid nets in order to receive more." Alliance for Malaria Prevention Toolkit (version 2.0) - Chapter 3, Pgs 5-6. - 64
- 65
- 66
- AMF has told us that it is still waiting on photos for the Balaka 2013 and Dedza 2014 distributions, but that it believes that Concern Universal did take photos. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015. This differs from what AMF told us last year, which was that Concern Universal had failed to take photos of the Balaka 2013 distribution:
- "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
- [AMF:] No, CU did not take photos of this one. They are aware of this slip and there will be photos and video footage from Dedza. Photos and videos are also included in the agreement for the DRC-Kasaï distribution."
Rob Mather, AMF Founder, email to GiveWell, May 22, 2014.
- AMF has told us that it is still waiting on photos for the Balaka 2013 and Dedza 2014 distributions, but that it believes that Concern Universal did take photos. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015. This differs from what AMF told us last year, which was that Concern Universal had failed to take photos of the Balaka 2013 distribution:
- 67
- 68
See, for example:
- 69
See our March 2012 update on AMF.
- 70
All weekly reports at AMF page on Balaka 2013 distribution. Examples of problems reported:
- "The exercise despite the verification process and data cleaning faced some duplicates that were discovered during the distributions. The duplicate situation had not been dealt with during data cleaning as it was deemed only when both villages were asked to collect nets from the same distribution point that it became clear that there had been duplicate entries. However, different distribution points at the same time though at the same cluster makes it practically impossible for villagers to collect two nets from different sites by double registering." Concern Universal Balaka 2013 week 1 report.
- "There was a high number of absenteeism during the urban distribution, which resulted in some members receiving the nets on behalf of others. This absenteeism was explained by the unavailability of the beneficiaries due to professional reasons. There were some complaints from beneficiaries whose nets were received by representatives claiming that the nets weren’t handed over. Handing over of nets to representatives was cancelled since it was clear that, unlike in rural areas where all community members know each other and certify the representative’s identification and the nets were handed over to the legitimate beneficiaries, nets were being misappropriated... For the urban distributions we anticipate to conduct them during the weekend to assure that most of the household owners are free from their daily work related activities." Concern Universal Balaka 2013 week 5 report.
- "One health worker assigned to facilitate the distribution process, in the community under his supervision, was caught by the beneficiaries trying to steal about 25 nets. He was reported reported to the authorities and discharged of his duties. This episode disrupted the distribution process and CU staff had to intervene to keep the population from beating the health worker." Concern Universal Balaka 2013 week 5 report.
- 71
All weekly reports at AMF page on Dedza 2014 distribution. Examples of problems reported:
- "The major challenge encountered during this week’s distributions was the misplacing of villages in clusters, which required us to the transfer the nets and distribution registers to the clusters where the villages have presented themselves. This delayed our distribution process but we still managed to reach and carry out the distributions to the affected villages." Concern Universal Dedza 2014 week 1 report.
- "There were 20 villages under Kaphuka that did not receive the nets because of poor communication as their HSAs were attending performance appraisals and failed to communicate the distribution dates to their respective villagers." Concern Universal Dedza 2014 week 3 report.
- "The major challenge during the week was duplication of registration in a way that some beneficiaries seem to have been deliberately registered in more than one village. The registration data was corrected and the affected beneficiaries only received the nets they were entitled to according their respective village data." Concern Universal Dedza 2014 week 3 report.
AMF notes these issue and an additional issue on its blog, "An isolated incident of 300 nets missing from one storage location. This is being investigated and pursued with the police as any nets missing is taken very seriously. 300 nets represents 0.12% of the total nets being distributed. AMF: "Mid-distribution weekly reports for Dedza distribution, Malawi".
- 72
All weekly reports at AMF page on Dowa 2015 distribution. Examples of problems reported:
- "The major challenge we encountered during the distribution was the breaking down of our distribution vehicles, which forced us to interrupt the distribution for almost one month to have them fixed, as some spare parts could not be sourced locally." Concern Universal Dowa 2015 weeks 1-3 report, pg. 2
- "Some beneficiaries didn’t show up as they were attending a clothing items distribution on the same day. The fact that most inhabitants of the said villages are refugees from Rwanda, Burundi or Somalia caused identification challenges, preventing individuals claiming nets on behalf of the beneficiaries from receiving the nets. These beneficiaries will be considered during the mop up exercise." Concern Universal Dowa 2015 weeks 1-3 report, pg. 3
- "Distributions were not conducted in ten of the planned villages under the three health facilities due to funerals hence we deferred distribution and managed to reach for the 493 of the 503 planned villages. However, arrangements will be made at a later date when we will reach them and conclude the distributions in the deferred villages hence their nets have been currently taken back to the warehouse for safe keeping. The above mentioned villages are: Msaka, Kancheri, Chimbalanga Mononga and Nkhota villages from Mtengowanthenga health facility with, respectively, 20, 100, 201, 91 and 49 nets returned; Masiya, Sintala 2, Mulode 2 and Mgoli from Dzoole health facility with a nets requirement of 42, 274, 26 and 84 nets respectively and Mphinda village under Kayembe health facility with a requirement of 140 nets." Concern Universal Dowa 2015 weeks 1-3 report, pg. 5
- 73
“AMF has provided regular, public updates on the large, ongoing net distribution in the Ntcheu district of Malawi. Expected data collection has occurred and the distribution has proceeded close to schedule. AMF's distribution partner, Concern Universal, has been transparent about problems it has encountered, and seems to have a robust process to catch problems (such as attempts to steal nets) when they arise.” See our March 2012 update on AMF.
- 74
Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.
- 75
Concern Universal Ntcheu 2012 distribution report
Concern Universal Balaka 2013 distribution report
IMA World Health, Kasaï Occidental 2014 distribution report - 76
For example, See Dedza 2014 8-month post-distribution check-up data, "Miscellaneous data" tab, where there are 7,718 + 7,111 + 511 = 15,340 nets total, and of those 7,557 + 7,087 + 6 = 14,650 are AMF nets, which is 95.5% of all nets.
- 77
"Our distribution of 676,000 nets in Kasaï Occidental in partnership with IMA World Health (IMA) is our first one using smartphone technology for data collection. We see this as an exciting development with significant potential benefits including:
- Acts against potential theft
- Improved accountability
- Greater transparency
- Greater data accuracy
- Improved cost effectiveness
- Additional data can be collected
- Reduced operational risk
The use of this technology may become a significant determinant of future net distributions that we fund. We will report publicly on our experience with the Kasaï Occidental 2014 distribution and the data gathered…
GPS information can also be gathered helping to locate households and tie the number of nets delivered to each." AMF: "Introduction of smartphone technology to collect distribution data". - 78
AMF: "Initial net distribution verification data for West Kasaï, DRC"
- 79
IMA World Health, Kasaï Occidental 2014 distribution data
- 80
AMF lists the countries it has provided nets to at @AMF Countries Involved@. The Malaria Atlas Project has compiled data on malaria risk by location at Malaria Atlas Project Endemic countries.
- 81
- 82
WHO 2014 Malaria World Report, Pg. 37, Figures 8.6 and 8.7.
- 83
WHO 2014 Malaria World Report, Pg. 37, Figures 8.6 and 8.7. AMF prefers to keep the names of countries it is in negotiations with confidential until agreements are signed, so unfortunately we cannot provide specifics.
- 84
- "Figures from Ntcheu District Hospital show that, in the twelve months to July 2011, 109,972 malaria cases were reported in under five children with a further 133,768 cases reported in the rest of the population." Concern Universal Ntcheu 2012 distribution proposal Pg 1.
- "All of Malawi (except some isolated highland areas) is designated a high risk malaria area. There were 96,453 reported cases of malaria in Balaka District over the past twelve months." Monthly data broken down into cases among under-5s and others also provided. Concern Universal Balaka 2013 distribution proposal, Pg 2.
- 85
- "Continued use of nets is very important. Every six months, a post-distribution survey is carried out to assess net usage and net condition. Approximately 5% of the nets distributed are assessed through visits to randomly selected households. The data collected are used to determine if additional community-level malaria education activities are required. All data are published." AMF information we publish.
- Example from a distribution proposal: "Please confirm you will carry out Post‐Distribution Surveys (PDSs)** every 6 months post‐distribution for a period of up to four years to assess the level of net usage (hang‐up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households." Concern Universal Dedza 2014 distribution proposal, Pg 2.
- AMF has to be flexible about the timing of PDCUs because often there are delays that cannot be prevented. If a PDCU is delayed, the next PDCU will occur on schedule (e.g. even though the Dedza 2014 distribution's first PDCU was done at 8 months, the next distribution will occur at 12 months), unless the delay was severe enough that sticking to the schedule would not make sense (as happened with the Ntcheu 2012 PDCUs). Although AMF originally planned to request PDCUs for four years after a distribution, this doesn't make sense in some contexts where it is required that a mass distribution of LLINs occur at least every 3 years (such as Malawi). Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 86
The methodology used in the Balaka 2013 6-month follow up survey seems similar to that used in the Ntcheu surveys discussed in our August 2014 update on AMF. Specifics from Concern Universal Balaka 2013 6-month post-distribution check-up report:
- Sample selection: "Balaka district has 14 health facilities. It was decided to collect data from 5% of households in each HCCA [health center catchment area], which meant a different number of households in each HCCA as per individual health facility populations. Between 25 and 90 households were randomly selected from each of the selected seven to ten villages, depending on the HCCA, with the villages also selected at random. Villages were randomly selected using the village lists generated from the pre-distribution and distribution work for the October 2013 AMF-funded universal coverage LLIN distribution. A random number table was used to select the villages. Households were randomly selected using the household lists produced during the same campaign. A random number table was used to select the households. Five more households were put on reserve in case no one was at home in the selected households." Pg 5.
- Who carried out the survey: "10 data collectors and 2 supervisors from the District Health Office were involved in the PDCU. The supervisors were responsible for checking the data collection exercise at the same time monitoring how the data was being collected as per requirement." Pg 5. Previous reports were more specific about who the data collectors were (for example, in the Ntcheu 24-month survey they were Health Surveillance Assistants, Malawi's version of community health workers.
- Data quality audits: "Supervisors were required to visit 5% of the households in their area to check the accuracy of the data collectors’ work and had to check all the completed forms submitted to them before submitting them to the Project Manager. The sampled visited households were also chosen at random so the work of all data collectors was checked." Pg 6.
- Cost: "The PDCU cost was US$8,060 equal to US$1.78 per household visited (or $0.052 per net originally distributed which numbered 154,230)." Pg 3.
- Data entry/cleaning: "Improvements in the data entry interface since the last PDCU carried out in Ntcheu (Ntcheu PDCU-24) by AMF meant the data entry proceeded with almost no errors. This reduced the error-checking phase to almost nothing." Pg 6
- 87
The type of technical information we would ideally like to see includes:
- Results of audits on data quality.
- Analysis of the extent to which respondent selection was carried out according to the protocol.
- Analysis of the extent to which selected households were replaced with pre-selected alternate households (i.e. attrition from the sample).
- Survey text used in each survey round and detailed protocols for responses that are directly observed by surveyors.
- 88
Ntcheu 6-month post-distribution check-up data.
Ntcheu 15-month post-distribution check-up data.
Ntcheu 24-month post-distribution check-up data.
Balaka 6-month post-distribution check up data (April 2014). - 89
Rob Mather, AMF Founder, email to GiveWell, June 30, 2014.
- 90
In the data for AMF's post-distribution check ups, it looks to us like "Nets Received" = "Hung" + "Present Not Hung" + "Missing" +" Worn out/not usable", implying that "nets hung" and "nets that are worn out" are exclusive categories. See, for example, Concern Universal Ntcheu 2012 33-month post-distribution check-up data
- 91
"'very good condition' (fewer than 2 holes of up to 2cms in size)." Concern Universal Ntcheu 2012 33-month post-distribution check-up report, Pg 4.
- 92
- "‘good condition’ (fewer than 10 small holes on them)" Concern Universal Ntcheu 2012 33-month post-distribution check-up report. Pg 3.
- Note that nets with fewer than 10 small holes used to be considered in "OK" condition.
"'OK' (fewer than 10 small holes on them)" Concern Universal Ntcheu 2012 24-month post-distribution check-up report, Pg 4.
- 93
- "‘viable condition’, with more than 10 holes or 1 hole larger than 10 cm," Concern Universal Ntcheu 2012 33-month post-distribution check-up report, Pg 3.
- Note that nets with more than 10 small holes used to considered in "poor" condition. "'Poor' condition with more than 10 holes or 1 hole larger than 10 cm." Concern Universal Ntcheu 2012 24-month post-distribution check-up report, Pg 4.
- 94
"Worn out means not usable. Typically, holes will be 10 cms of cms in size and numerous and the net considered effectively unrepairable and/or structural integrity will have gone. If hung the net would not provide protection against mosquitoes given the extent of holes or tears or other loss of structural integrity." Rob Mather, AMF Founder, email to GiveWell, June 30, 2014.
- 95
@AMF how we work with distribution partners@, Pg 4.
- 96
Rob Mather, AMF Founder, email to GiveWell, June 30, 2014.
- 97
For example, consider data from the Ntcheu 2012 33-month PDCU:
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results (https://www.againstmalaria.com/SurveyResults_Live.aspx?ProposalID=184&R…), the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
[AMF]: "This is because the latter figure includes non-AMF nets i.e. there are 462 non-AMF nets included in the ‘Usage of the nets’ tab as we wished to have an overall figure for sleeping space coverage (52%). We may add a note to this tab to indicate there are some non-AMF nets included in the data on this tab." Rob Mather, AMF Founder, email to GiveWell, October 13, 2015. Note that we did not ask AMF where the 462 figure comes from. - 10,664 AMF nets were found to be "very good", "good", or "viable" condition. 10,664 + 462 non-AMF nets = 11,126 nets, which is less than the 11,497 nets "used correctly", implying that some AMF nets that are "worn out" are included in the "used correctly" category. Concern Universal Ntcheu 2012 33-month post-distribution check-up data
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results (https://www.againstmalaria.com/SurveyResults_Live.aspx?ProposalID=184&R…), the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
- 98
For example, in the Concern Universal Ntcheu 2012 33-month post-distribution check-up data, we see that:
- 10,664 AMF nets are hung ("Presence of AMF Nets" tab)
- 10,664 AMF nets are also in "viable", "good", or "very good" condition ("Condition of AMF Nets tab"). It seems odd to us that these numbers match.
- 11,497 nets are "used correctly." ("Usage of the nets" tab) AMF told us 462 non-AMF nets were included in this tab, meaning that 11,035 AMF nets are included.
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results (https://www.againstmalaria.com/SurveyResults_Live.aspx?ProposalID=184&R…), the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
[AMF]: "This is because the latter figure includes non-AMF nets i.e. there are 462 non-AMF nets included in the ‘Usage of the nets’ tab as we wished to have an overall figure for sleeping space coverage (52%). We may add a note to this tab to indicate there are some non-AMF nets included in the data on this tab." Rob Mather, AMF Founder, email to GiveWell, October 13, 2015 Note that we did not ask AMF where the 462 figure comes from.
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results (https://www.againstmalaria.com/SurveyResults_Live.aspx?ProposalID=184&R…), the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
- 11,035 nets "used correctly" is more than the 10,664 nets "hung", implying that nets that were "present but not hung" or "worn out/not usable" could be included. ("Presence of AMF Nets" tab).
- 11,035 nets "used correctly" is also more than the 10,664 nets found in "viable", "good", or "very good" condition, implying that some "worn out" nets are "used correctly."
- 24,980 people were found to be "sleeping under nets" ("Usage of the nets" tab). These are the people who are also labelled as "Covered" by nets.
- 24,980 people / 11,035 nets "used correctly" = 2.26 people per net, which seems reasonable, though higher than we would have expected. 24,980 people / 10,664 nets "hung" = 2.34 people per net, which also seems reasonable but higher than we would have expected. We are uncertain about whether "sleeping under a net" means sleeping under a "used correctly" net, a "hung" net, or a non-"worn out" net.
Concern Universal Ntcheu 2012 33-month post-distribution check-up data
- 99
- "Usage does not appear to have been near-universal. Most studies report usage rates in the range of 60-80%, though some report 90%+ usage." From our page on Long Lasting Insecticide Treated Nets.
- Note that the World Malaria Reports that we looked at showed similar usage rates.
- 100
- "Alliance for Malaria Prevention 2011 lays out a model for estimating the number of LLINs still in use after distributions: 'the number of LLINs already distributed over the last three years and considered to be available in households should be calculated and subtracted from the total need, working with a decay rate of 8 per cent at one year (0-12 months), 20 per cent at two years (13-24 months) and 50 per cent at three years (25-36 months).'…
Bottom line: We believe that the "8%-20%-50%" model is the most widely used and most reasonable approximation available at the moment for capturing the extent to which LLINs remain in use in the years following distribution, accounting for any factors that might cause LLINs to be discarded or additional LLINs to be purchased. It implies an average of 2.22 years of use for each LLIN distributed. Data and analysis on this topic appears extremely thin; we have little sense for how long LLINs last in practice." - "If we assume that an LLIN has a 92% chance of being in use at a given point in the first year after distribution, this implies that for each LLIN delivered, an average of 0.92 LLIN-years of use are obtained in the first year. Assuming 0.92 LLIN-years of use in the first year, 0.8 in the second year and 0.5 in the third year would yield an overall average of 2.22 years of use per LLIN. This is substantially less than the "official life" of an LLIN. As discussed below, we believe this makes sense because the decay function is intended to account for wastage of all kinds, including loss/failed delivery of LLINs, improper use resulting in disrepair, etc."
- "Alliance for Malaria Prevention 2011 lays out a model for estimating the number of LLINs still in use after distributions: 'the number of LLINs already distributed over the last three years and considered to be available in households should be calculated and subtracted from the total need, working with a decay rate of 8 per cent at one year (0-12 months), 20 per cent at two years (13-24 months) and 50 per cent at three years (25-36 months).'…
- 101
AMF Malawi universal coverage calculations (September 26, 2011).
- 102
See our March 2012 update on AMF.
Exact net figure from Rob Mather, AMF Founder, email to GiveWell, November 20, 2012.
- 103
AMF Malawi universal coverage calculations (September 26, 2011).
- 104
"Health Surveillance Assistants (HSAs) are Government extension workers- they are the lowest tier of government presence in the decentralized health system." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.
- 105
"As such they are the first line of response to any public health issues in communities. Their job involves disseminating health related information (such as encouraging people to make use of sanitary facilities, go for immunizations, sleep under mosquito nets etc.), carrying out sanitation and hygiene campaigns and sending data on take-up of facilities to the District Council, conducting basic nutrition support, weighing children and reporting levels of stunting and wasting, detecting common communicable diseases and reporting these to clinicians and other health providers, implementing immunization campaigns etc. As you can see being involved in universal net distribution fits very well with their core public health responsibilities. HSAs need to have a primary school completion certificate as a minimum but the majority of them will have O-Levels (exams sat by pupils aged 16 if they have completed the school system at the recommended pace). Once they have been selected as HSAs they are sent on an initial 9 months intensive training course where they will be trained in many aspects of public health including how to recognize common diseases, how to administer immunizations etc." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.
- 106
- Pre-distribution registration surveys appear to be completed relatively quickly: "Approximately 480 personnel will be involved in the PDRS, with the majority, some 460, involved for 5-7 days over the data collection period." Concern Universal, Dowa 2015 planning document, pg. 4
- In the Ntcheu 2012 distribution, the verification of PDRS data took several weeks: "The verification process took the verification team of 10 members 18 days to complete and in a day 20 clusters were verified with 10 verification sites in the morning and 10 verification sites in the afternoon." Concern Universal Ntcheu 2012 distribution report, pg. 7.
- In the Ntcheu 2012 distribution, distributions were scheduled for several weeks, but covered approximately 10 "clusters" per day: "The distributions were scheduled to have been concluded within 28 days with the team distributing at 10 clusters per day covering five weeks." Concern Universal Ntcheu 2012 distribution report, pg. 7-8
- Concern Universal has recently started to use fewer staff for post-distribution check ups, which causes them to take somewhat longer: "However, in collaboration with the District Environmental Health Office (DEHO) and Malaria Coordinator (MC) and lessons lea[r]nt from 24 month Ntcheu PDCU, it was recommended to have a focused team of 10 data collectors rather than have the HSAs as data collectors from each HCA. This was based on the following reasons. First, this would reduce the number of data collectors that would need to be monitored and trained. Second, we would be able to select reliable individuals whom we could trust to do a diligent and accurate job of collecting the data. Third, it would leave the majority of HSAs to carry on with the normal health tasks and duties. Fourth, by having the same people covering the whole exercise they will get acquainted to the task and reduce errors on data collection. This meant the data collectors would spend thirty seven days collecting data rather than the one or several days if many more data collectors were to be used. This was judged the preferable way of organising and managing the data collection phase." Concern Universal Ntcheu 2012 33-month post-distribution check-up report, pg 5. Other recent post-distribution check-up reports have similar language.
- 107
"i) Field Supervisors (FSs)
22 FSs were selected from permanent and senior health staff in Tshikapa Health District.
ii) Community Health Workers (CHWs) – data collectors
Each of the 22 FSs had the responsibility of recruiting, in each HA they were designated, enough CHWs to gather household data and hang nets. Two primary recruitment criteria were literacy and familiarity with using a mobile phone. The number of CHWs recruited depended on the size of the HA and the number of households to be visited. The aim was to recruit enough CHWs to carry out the entire registration and hang‐up, once it commenced, in a five day period. Between 20 and 40 CHWs were recruited by each FS for a total of 4,000 CHWs across the 8 HZs (8 HZs x 20 HAs x 25 CHWs per HA = 4,000 CHWs)." IMA World Health, Kasaï Occidental 2014 distribution report pg. 13 - 108
GiveWell estimate of AMF cost per net (October 2015). In estimating the overall cost per LLIN in Malawi, we have used an average that equally weighs the cost from each distribution. Another option would be to put more weight on the actual costs (e.g., one could weight the Balaka 2013 distribution at 50%, the Dedza 2014 at 25% and the Dowa 2015 at 25%) to account for the fact that actual costs are often more reliable than budgeted costs.
- 109
Rob Mather, AMF Founder, phone conversation with GiveWell, May 23, 2014. Managerial staff time is listed in the budget.
- 110
See the “Ntcheu 2012 other costs” sheet in GiveWell estimate of AMF cost per net (October 2015) for details.
- 111
See the “AMF org” sheet in in GiveWell estimate of AMF cost per net (October 2015) for details.
- 112
For example:
- The spreadsheet IMA shared of actual costs for the Kasaï Occidental distribution is hard to make sense of and appears to contain several errors. Also, about 20% of the costs are "unconfirmed charges," which are not listed individually.
- For Malawi, Concern Universal shared, in 2012, information on non-monetary costs (such as office space) and the value of time provided by government employees. IMA's actual cost data includes an estimate of indirect costs (as a percentage of total direct costs), but it's unclear if it includes costs paid for by the government or other actors. As a rough approximation, we have added $0.13 per net, which was the estimate of non-monetary partner and local government costs from the 2012 Ntcheu, Malawi distribution.
- 113
- The budget AMF shared with us estimated a cost of $9,000 per PDCU, or $0.01 per net, which is significantly lower than the costs of PDCUs in Malawi. IMA World Health, Kasaï Occidental 2014 non-net costs final budget. AMF told us that this is because not all cost data has been received from IMA yet. Rob Mather, AMF Founder, email to GiveWell, September 9, 2015.
- For the costs of PDCUs in Malawi, see GiveWell estimate of AMF cost per net (October 2015) Sheets: "Summary" and "Schedule for Update of NNCs". The Ntcheu 2012 24-month PDCU is the cheapest of all conducted so far, at $0.03 per net (we aren't sure what caused this PDCU to be so inexpensive). Most PDCU costs are estimated to be between $0.05-$0.10 per net.
- 114
- AMF told us that in DRC net costs and non-net costs are very roughly the same. Rob Mather, AMF Founder, phone conversation with GiveWell, May 23, 2014.
- "In other cases, non-net costs are assumed to $2.50 per LLIN, which is the historical average. Non-net costs can vary significantly across countries. For example, in Angola and DRC, non–net costs may be more than $5." Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, May 23, 2014.
- 115
Some countries prefer to buy conical nets or extra large nets, which cost more than the standard square nets that AMF typically funds. Rob Mather and Peter Sherratt, conversation with GiveWell, October 15, 2015
- 116
See for example Roll Back Malaria Partnership gap analysis (September 2013).
- 117
Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, October 20, 2015
- 118
"The gap analysis includes both the cost to purchase LLINs and non-net costs, such as
shipping and distribution. In cases where national governments provided data on non-net costs, this data was used in the analysis. In other cases, non-net costs are assumed to $2.50
per LLIN, which is the historical average. Non-net costs can vary significantly across countries. For example, in Angola and DRC, non–net costs may be more than $5." Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, May 23, 2014. - 119
See this spreadsheet.
- 120
"AMF and Concern Universal, Malawi (CU) have accepted the Malawi National Malaria Control Programme’s (NMCP) invitation to assume responsibility for the district-wide net distribution re-coverage campaigns in Ntcheu, Balaka, Dedza and Dowa districts in 2018...This timing is consistent with the three year re-coverage cycle needed to keep affected populations protected from malaria-carrying mosquitoes when they sleep at night." AMF: "US$6m commitment to malaria control support in Malawi in 2018"
- 121
AMF funds status, October 10, 2015
- 122
In 2014, we believed that AMF had additional room for more funding because:
- In 2013, AMF had completed two large-scale distributions, both of which were with a single distribution partner, Concern Universal in Malawi. It had not signed agreements with other partners. As of November 2014, AMF had signed agreements for two large distributions with IMA World Health in the DRC (one of which is in progress), helping to demonstrate its ability to finalize distributions with other partners in other locations.
- By the end of 2014, AMF had spent or committed a large portion of the funds it raised prior to 2014. As of November 2014, AMF held $4.9 million in uncommitted funds, of which it raised $2.65 million in 2014. Of committed funds, $1.9 million were for expenses that AMF would incur more than a year later, and AMF was considering reallocating these funds in the near term with the expectation of raising enough to cover these commitments by the time they were due. This brought the total available funds to $6.8 million.
- "Funds in hand as at Nov 2013: $12.5 m
Committed since Nov13: $10.30 m (10.3/12.5 = 82%)- - DRC, West Kasaï 2014 $2.0m (NETS only)
- DRC, Idjwi 2015 $0.37m (NETS + all non-net costs)
- Malawi, Dowa, 2015 $1.49m (NETS + all non-net costs)
- Malawi, Ntcheu 2015 $1.34m (NETS + all non-net costs)
- Malawi, Balaka 2015 $0.94m (NETS + all non-net costs)
- Malawi, Dedza 2017 $1.72m (NETS + all non-net costs)
- DRC, North Ubangi 2015 $2.34m (NETS + shipping)
- Malawi, Dedza, 2014 $0.10m (additional nets)Funds uncommitted $2.2m
New funds raised $2.65m
Total funds uncommitted $4.85m"
Rob Mather, AMF Founder, email to GiveWell, November 26, 2014. - "If ‘make available’ all Dedza17: $1.720m [i.e. if funds committed to the Dedza 2017 distribution were made available for an earlier distribution]
Ditto, Yr2+ PDCU Do/Nt/Ba: $0.2 m [refers to future post-distribution check ups in Dowa, Ntcheu, and Balaka]"
Rob Mather, AMF Founder, email to GiveWell, November 8, 2014.
- "Funds in hand as at Nov 2013: $12.5 m
- AMF's pipeline for potential distributions included several distributions with established partners. Given AMF's history with these partners, we guessed that there were fewer barriers to signing agreements for additional distributions. Funding all such distributions would have required AMF to raise significantly more funds. AMF may also have had opportunities to fund distributions with new partners.
- 123
- Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- "AMF and Concern Universal, Malawi (CU) have accepted the Malawi National Malaria Control Programme’s (NMCP) invitation to assume responsibility for the district-wide net distribution re-coverage campaigns in Ntcheu, Balaka, Dedza and Dowa districts in 2018...This timing is consistent with the three year re-coverage cycle needed to keep affected populations protected from malaria-carrying mosquitoes when they sleep at night." AMF: "US$6m commitment to malaria control support in Malawi in 2018"
- 124
"With or without a co-funding partner, our sense, is NMCPs will first try and achieve funding from:
a) Organisations from whom they have received funding before and with whom they have established relationships (know how the relationship operates, reporting requirements etc)
b) Organisations with high levels of funding to minimise the number of agreements reached (often 2)
c) Organisations who have the least accountability requirements. We do not have hard information to support this but is a view shared by others within the malaria and wider aid community." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015 - 125
- "The main barrier is there are monitoring staff at [redacted] who are currently expressing some reluctance to see [redacted] money being used for post-distribution check-ups (PDCUs). We have indicated that we are prepared to be flexible on the frequency and sample size, and this is being considered within [redacted]." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- "Our experience over the last several years indicates discussions to reach agreement can be as short as 3 months (Malawi/CU, DRC/IMA/DFID) but can extend to 9-12+ months.
Recent discussions have been continuing for close to 9 months. This may be due to:
- The particular co-funding partner [redacted] involved in all the distributions currently far advanced
- AMF is a new partner (at the scale of millions of nets) with the NMCPs.
- AMF is seeking to introduce elements of accountability that are new to some NMCPs and the co-funding partner.Our experience is when an agreement with a co-funding partner is reached quickly, agreements with the relevant NMCP and distribution partner can follow quickly (within several months)."
Rob Mather, AMF Founder, email to GiveWell, September 9, 2015 - AMF notes that because it is a new partner for the countries it is in discussions with, but it not offering to fill the entire net gap these countries face, it has had to deal with a large amount of slow bureaucracy (e.g. from the NMCPs). Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 126
- AMF told us that the discussions about the PDCU costs have only delayed an agreement for 1-2 months so far.
- AMF is reluctant to fund PDCU costs because:
- AMF believes that PDCUs provide an opportunity for its co-funding partner to experiment with funding net accountability. If the PDCUs go well, then the partner may be more willing to fund additional monitoring in the future.
- AMF does not have in-country staff to monitor and manage PDCUs, while its co-funding partners often do. AMF thinks that managing the funding of the PDCUs would place a large administrative burden on AMF, whereas the management would be relatively easy for the co-funding partner.
Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015. We do not have a clear sense of why it is more difficult for AMF to manage the funding of PDCUs, but it seems plausible to us that not having staff in-country would make this more difficult.
- 127
Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
AMF funds status, October 10, 2015 - 128
- Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- AMF funds status, October 10, 2015
- Rob Mather, AMF Founder, conversation with GiveWell, October 22, 2015
- Note that this is a different estimation of Country Z's net need than is indicated by the Roll Back Malaria Partnership gap analysis (October 2015), although AMF has told us that Country Z would like to shift its net distribution schedule earlier since coverage rates have been low in Country Z. Rob Mather, AMF Founder, conversation with GiveWell, October 22, 2015
- 129
Rob Mather and Peter Sherratt, conversation with GiveWell, October 15, 2015
AMF funds status, October 10, 2015 - 130
Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 131
Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 132
- "We are close to concluding discussion with Concern Universal for AMF and CU to co-fund a Malaria Unit (a larger team of people than currently in place dedicated to malaria control related work) in Malawi led by a senior CU Manager and employing 12 Malawians (10 additional hires). The aim of this unit will be to provide more resource to allow 1. improved efficiency in managing the four contracted three-yearly distributions and associated work and 2. allow us to together innovate and develop additional malaria control support for the four districts and the NMCP, specifically: i) intended close liaising with Health Centre re malaria case data and elements related to the monitoring/recording of malaria data i.e. stock levels of RDTKs, rubber gloves and other diagnosis equipment; qualified staff able to test for malaria, presence at the clinic so diagnosis can happen; systems, capacities etc) ii) research (data), discussion and involvement in ways of ensuring 80% sleeping space coverage throughout the three year net-distribution cycle, including investigation of ‘injection strategy’ net distributions involving mini-mass distributions at two years post-distribution and in the subsequent distribution cycle at one-year post distribution.
This would be an initial three year commitment from AMF of US$413k...Savings made on non-net costs (with the Malaria Unit funded, some costs included in non-net cost budgets would already be covered) we estimate to be of the order of US$70k, and perhaps as high as US$95k per distribution of which there are four in a three year cycle i.e. US$280-380k in total... If all progresses well we may be able to publish plans in the next six weeks once budgets have been agreed and assuming approval from the Malawi NMCP. We do not anticipate problems, although there is a work visa issue to resolve." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- AMF Malaria Unit draft budget
- "We are close to concluding discussion with Concern Universal for AMF and CU to co-fund a Malaria Unit (a larger team of people than currently in place dedicated to malaria control related work) in Malawi led by a senior CU Manager and employing 12 Malawians (10 additional hires). The aim of this unit will be to provide more resource to allow 1. improved efficiency in managing the four contracted three-yearly distributions and associated work and 2. allow us to together innovate and develop additional malaria control support for the four districts and the NMCP, specifically: i) intended close liaising with Health Centre re malaria case data and elements related to the monitoring/recording of malaria data i.e. stock levels of RDTKs, rubber gloves and other diagnosis equipment; qualified staff able to test for malaria, presence at the clinic so diagnosis can happen; systems, capacities etc) ii) research (data), discussion and involvement in ways of ensuring 80% sleeping space coverage throughout the three year net-distribution cycle, including investigation of ‘injection strategy’ net distributions involving mini-mass distributions at two years post-distribution and in the subsequent distribution cycle at one-year post distribution.
- 133
- Rob Mather, AMF Founder, email to GiveWell, October 13, 2015
- "Funds in hand: $ 23.1 m
- Allocated: ‐$5.5 m (Malawi 2018)
- Ringfenced: ‐$1.1 m (Malaria Unit, IRR study)
- Balance: $16.5 m
- Release of allocated: $2.0m (part Malawi 2018)
- Revised balance: $ 18.5 m"
- This is more than twice as much as what AMF held in Fall 2014 ($6.8 million that it could allocate to future distributions).
- Rob Mather, AMF Founder, email to GiveWell, October 13, 2015
- 134
- Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015.
- 135
AMF funds status, October 10, 2015
Rob Mather and Peter Sherratt, conversation with GiveWell, October 15, 2015. - 136
Rob Mather and Peter Sherratt, conversation with GiveWell, October 15, 2015 AMF is dropping Country J because it has found it more difficult to reach agreements about AMF's monitoring requirements with contacts in the country, and AMF would prefer to work with partners who are fully supportive of AMF's monitoring requirements. AMF is dropping Country K because the nets the NMCP requires that funders buy are more expensive than the nets required in countries I and C, so AMF can cover more people by working in countries I and C.
- 137
- Rob Mather and Peter Sherratt, conversation with GiveWell, October 15, 2015
- "We are discussing four distributions and are at the final stage with them all. We anticipate a decision with them all in the next several months." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- Note that we think that AMF is typically over-optimistic in its predictions. For example, a distribution that it told us was close to being signed throughout 2015 recently fell through with little warning (Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015).
- 138
"The operational plan for a distribution typically covers 12 months. The planning horizon for a specific distribution is, however, typically 18 to 24 months as discussions to assess and approve a distribution start six months earlier and the funds required (we need to have funds in hand in order to have serious discussions) are aggregated in the preceding months." AMF: "Operational planning (12 months) and planning horizon (18 to 24 months)".
- 139
Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.
- 140
- Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- AMF notes that it is cautious about offering to support more distributions than it currently has the funding fore. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- "We have always made clear to the NMCPs and other partners with whom we are discussing distributions that we have funds in hand currently for the quantity of nets we have offered to fund but that number of nets might reduce if other distributions are funded and that we would communicate any change to them as and when that occurred." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 141
Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.
- 142
Rob Mather and Peter Sherratt, phone conversation with GiveWell, November 6, 2014.
- 143
Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, October 20, 2015
- 144
Roll Back Malaria Partnership gap analysis (October 2015), pg. 3, sum of columns 2015, 2016, and 2017, Totals row. It also estimated that about 886 million additional LLINs would be needed in 2016-2020.
- 145
Roll Back Malaria Partnership gap analysis (October 2015), pg. 3.
- 146
- As of October 2015, there was still a gap of approximately 40 million LLINs for 2015. Roll Back Malaria Partnership gap analysis (October 2015), pg. 3.
- For 2013, see: Roll Back Malaria Partnership gap analysis (September 2013), total in cell N45.
- The December 2014 gap analysis does not include an estimate of the remaining gap for 2014.Roll Back Malaria Partnership gap analysis (December 2014)
- 147
AMF would like this person to be able to travel to the countries where AMF is funding distributions and assist with discussions in-person. AMF notes that it has not started to actively look for this position, although it suspects that once it does it will not take long to find candidates. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 148
Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015