Against Malaria Foundation - September 2012 Update

Published: September 2012

Summary

  • Plans for future distributions
    • AMF currently has $4.6 million in funds, which would allow it to purchase approximately 1.1 million nets for distribution.
    • AMF is currently evaluating potential distributions in Togo and Mali. Were it to decide to move forward with these distributions, they would require all of AMF's current funding.
    • AMF is distributing funds at a slower pace than it anticipated. We will continue to monitor this and provide updates on AMF's progress in purchasing nets.
  • Past distributions
    • AMF completed a distribution of approximately 268,000 nets in Ntcheu, Malawi in April 2012. It has followed through on its plans to carefully monitor and share data from this distribution, having now posted (a) a final distribution report, (b) malaria case rate data for Ntcheu, and (c) post-distribution surveys of net receipt and use. All data is consistent with the notion that this was a successful distribution.
  • Bottom line: We remain confident in the effectiveness of AMF's activities. It remains our #1-ranked organization.

Table of Contents

Plans for future distributions

In early April 2012, AMF’s distribution partner in Malawi, Concern Universal, completed its distribution of about 268,000 LLINs.1 AMF is now in the process of searching for future distribution partners.

AMF has located potential distribution partners in two countries: Plan Togo and Plan Mali. AMF’s discussions with these potential partners are at advanced and intermediate stages, respectively. As of August 2012, AMF had not made any agreements with new distribution partners for large-scale distributions.2

Plan Togo

  • AMF understands that there is a current need for about 500,000 nets in Togo in order to achieve universal coverage.3 Based on conversations with other funders and the Togo National Malaria Control Programme (NMCP), AMF has concluded that funding to cover this gap is not currently available.4
  • Plan Togo previously was involved in an LLIN distribution campaign in 2011, and AMF believes that this campaign was well-conducted.5
  • Plan Togo and the NMCP have been responsive in providing AMF with information. Discussion on elements of the distribution, including some pre-distribution and post-distribution information and activities, is not yet complete, so AMF has not been able to reach a final decision and conclude an agreement. For example: Plan Togo has told AMF that it has a pre-distribution registration survey (PDRS) for the relevant distribution areas, but AMF has not yet seen this data, nor knows how recent and therefore reliable it is. Consequently, AMF does not know whether a new PDRS will be requested and, if it were, what Plan Togo and the NMCP’s response would be.6
  • AMF has spoken with Plan Canada, which has agreed to cover the full non-net costs of the distribution (including shipping, the distribution and monitoring, pre-distribution registration survey and post-distribution follow-up required by AMF) if AMF decides to partner with Plan Togo.7

Plan Mali

  • AMF understands that approximately 4 million additional nets are needed in Mali in order to achieve universal coverage.8
  • AMF is mid-way through the process of evaluating a potential partnership with Plan Mali, and has not yet reached agreement on its requirements or received the information it needs about Plan Mali’s capacity in the country.9
  • Based on the funds it had available as of early August and assuming a 500,000 net distribution in Togo, AMF could fund approximately 600,000 nets for Mali.10
  • AMF may consider increasing the scale of the Mali net distribution up to 4 million nets if it has successful progress with the Mali NMCP and Plan Mali (or other distribution partners), and if the need for 4 million nets can be justified with reliable data, and if there are no other sources of funds. Should it scale up this distribution, AMF would need to raise additional funds.11
  • AMF has spoken with Plan Canada, which has indicated that it would fund the non-net distribution costs if AMF decides to partner with Plan Mali (though AMF has not discussed with Plan Canada the non-net funding for a larger-scale distribution of 4 million nets).12

Next steps

  • AMF is hoping to reach agreement with distribution partners in Togo and Mali by the end of September, if possible.13
  • If partnership agreements are not reached with partners in Togo and Mali, AMF has a list of other countries with significant net gaps it plans to explore, including Cameroon, Ivory Coast, Guinea, Chad, Nigeria and Malawi.14

Room for more funding

In September 2011, AMF told us that if it were to receive $3 million or more from GiveWell, it would use those funds for a distribution with an expected completion date of June-September 2012.15 As of August 2012, AMF had received that level of funding from GiveWell, but had not made sufficient progress to meet its expected completion date for future distributions.

We discussed this slower-than-expected progress with Rob Mather, AMF's founder. He told us that during March and April 2012, AMF had discussions with the National Malaria Control Program (NMCP) in Malawi regarding the country’s upcoming universal coverage distribution. AMF was considering funding the 600,000 nets this campaign required to close the gap in Malawi. However, AMF withdrew from this potential distribution because it could neither obtain the data to fully evaluate the potential distribution nor receive assurances on how the distribution would be carried out. AMF is considering publishing detailed information on its experiences with this distribution.16

AMF currently believes that the earliest date its funds could be spent is October 2012.17 In this scenario, nets would be disbursed in early 2013.

AMF is conscious of not converting funds to nets as quickly as it expected to and has learned it is necessary to consider more potential distributions in parallel to avoid delays it considers suboptimal. Going forward, AMF plans to build a “pipeline” of potential future distributions, so that money can be moved more quickly into buying nets. AMF feels that it was a mistake to place too much stock in a few potential distributions (including the one in Malawi, discussed above), rather than aggressively pursue more potential distribution opportunities back in April.18 AMF has recently provided an explanation and update to donors whose donations are currently unallocated to a distribution. (http://www.againstmalaria.com/futuredistributions.aspx)

Results of the past distribution in Malawi

The distribution in Ntcheu, Malawi was completed in early April 2012;19 we describe details of the distribution in our previous update.

Final distribution report

Concern Universal, AMF's distribution partner in Malawi, published a final report on the Ntcheu distribution that includes a detailed description of each stage of the distribution process (orientation, registration, data entry, verifications, distribution) with photographs.20 This report also lists challenges met throughout the process and recommendations for future distributions.21

In addition to the problems Concern Universal identified that we mentioned in our previous update, the report mentions an anecdote of an identified misuse of 10 nets.22

Malaria case rate data from Ntcheu

AMF requires its distribution partners to collect monthly malaria case rate data from all health centers in the distribution zone for 12 months preceding and 4 years following the distribution.23 AMF has followed through on its plans to collect and share this data and has published data on malaria cases from each of the 37 health facilities in Ntcheu for the months of July 2010 through June 2012.24 This includes data for nearly four months of full net coverage in Ntcheu.25

Based on independent evidence, we believe that net distributions reduce deaths from malaria. (For more, see our page on mass distributions of insecticide-treated nets.) The data AMF collected are consistent with the notion that AMF's net distribution had a significant impact on malaria rates in Ntcheu. However, we do not currently feel that we are able to draw strong conclusions from this data, so it does not significantly increase our confidence in net distributions in general, or AMF's in particular.26

Post-distribution survey data from Ntcheu

AMF requires its distribution partners to administer 4 post-distribution surveys (PDSs) to gather data on net usage and condition.27 In Ntcheu, the first of these surveys was administered six months following the distribution in 200-250 households in each of the 37 health centre areas.28

6-month PDS data is now available for some areas of Ntcheu,29 covering 7,646 households and 15,735 nets (5.9% of the total distributed).30 The results report a 90% usage rate and 99% of nets still in very good condition (fewer than two holes of 2 cms or smaller).31 District by district data is available on AMF's website on its survey summary page at http://www.againstmalaria.com/SurveyForms_JobsSummary.aspx (archived).

AMF told us that Concern Universal used several measures to increase the reliability of PDS data.32

We are glad that AMF is collecting this data because it demonstrates that AMF is following through on its plans post-distribution and because the survey data might identify a serious problem, were one to exist. Concern's methodology for conducting these surveys appears reasonable, but we have not yet seen the full technical details of these surveys or results from Concern Universal's data quality auditing procedures. Thus, after only a limited examination of the data, these survey results are consistent with the notion that those who receive AMF's nets use them.

Sources

  • 1"Th[e final] distribution will be conducted in late March and early April when the project receives the additional 16,574 LLINs from AMF." Concern Universal, "Final Distribution Report," Pg 14.

    “…Concern Universal Malawi and Ntcheu District Council conducted a Universal Long Lasting Insecticide Net (LLIN) Distribution Programme across Ntcheu District as part of the National Malaria Control Programme Universal distribution efforts. Against Malaria Foundation provided the 268,000 LLINs for distribution in Ntcheu District whilst funds for distribution operations have been provided by Irish Aid. This stock of nets will be sufficient to ensure coverage of every sleeping space in Ntcheu District.” Concern Universal, “Final Distribution Report,” Pg 2.

  • 2Rob Mather, phone conversation with GiveWell, July 19, 2012, and Rob Mather, email to GiveWell, September 6, 2012.
  • 3 Rob Mather, phone conversation with GiveWell, July 19, 2012 and Rob Mather, email to GiveWell, September 6, 2012. The email cites sources of the estimate as Alliance for Malaria Prevention and NMCP Togo.
  • 4Rob Mather, phone conversation with GiveWell, July 19, 2012.
    "Then, I talked to the person at the Global Fund to Fight AIDS, Tuberculosis, and Malaria for Togo, and they laughed when I asked if they had money to fill this gap. The clear indication was they did not have the funds. The 500,000 net gap has sat there for many months – 9 or more - and the majority of the nationwide distribution took place last year without these nets funded so that is perhaps another clue." Rob Mather, email to GiveWell, September 6, 2012.
  • 5Rob Mather, phone conversation with GiveWell, July 19, 2012.
  • 6Rob Mather, phone conversation with GiveWell, July 19, 2012.
  • 7Rob Mather, phone conversation with GiveWell, July 19, 2012.
  • 8Rob Mather, email to GiveWell, September 7, 2012. The email cites the source of the estimate as Alliance for Malaria Prevention and NMCP Mali.
  • 9 Rob Mather, phone conversation with GiveWell, July 19, 2012.
  • 10Rob Mather, email to GiveWell, August 8, 2012.
  • 11Rob Mather, email to GiveWell, August 8, 2012.
  • 12 Rob Mather, phone conversation with GiveWell, July 19, 2012.
  • 13 Rob Mather, phone conversation with GiveWell, July 19, 2012.
  • 14 “Beyond Togo and Mali, the countries on our list currently are Cameroon, Ivory Coast, Guinea, Tchad, Nigeria and Malawi." Rob Mather, email to GiveWell, July 23, 2012.
  • 15Against Malaria Foundation, "Room For More Funding Analysis (September 2011)."
  • 16Rob Mather, email to GiveWell, September 6, 2012.
  • 17 Rob Mather, phone conversation with GiveWell, July 19, 2012.
  • 18Rob Mather, phone conversation with GiveWell, September 6, 2012.
  • 19"The report covers the nets distributed at Ntcheu town which were the final part of the Universal Distribution of Long Lasting Insecticide Treated Nets (LLINs) across Ntcheu District. This distribution was conducted in the last week of March and the first weeks of April 2012 when the project received an additional 16,574 LLINs from AMF." Concern Universal, "Final Distribution Report," Pg 27.
  • 20

    See Concern Universal, "Final Distribution Report," Pgs 3-14.

  • 21See Concern Universal, “Final Distribution Report,” Pgs 15-18.
  • 22"However PSI did seize 10 vests from a tailor which had been sewn out of 2 Permanets which were distributed as part of our work. The tailor did not disclose whether the nets had been given to him or whether he purchased them from somewhere else- he did however state that one Permanet could be used to make 5 vests. We will monitor this situation to see whether it is an isolated incident or cause for greater concern." Concern Universal, "Final Distribution Report," Pg 18.
  • 23“Malaria case rate data, pre and post‐distribution: This information is crucial to being able to monitor the impact of the nets over time. We ask for the following information and confirmations: a) Please provide 12 months historic monthly malaria case rate information for each health clinic in the intended distribution area b) Please indicate for each health centre if this malaria data is via Rapid Diagnostic Testing Kit (RDTK) analysis or via clinical observation c) Please indicate if there is a plan to ensure each health centre will always have an adequate stock of RDTKs to ensure malaria case rate information can be continually gathered? d) Please confirm you will be able to gather and provide that information for the next four years?” Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 3.

  • 24

    Against Malaria Foundation, “Malaria Case Rate Data.”

  • 25

    The final distribution phase that occurred in late March/early April 2012 included 16,574 nets out of about 268,000 nets, so the months of March/April are ones in which there was nearly full coverage: "[The final] distribution will be conducted in late March and early April when the project receives the additional 16,574 LLINs from AMF." Concern Universal, "Final Distribution Report," Pg 14.

  • 26

    AMF performed its own analysis of the data and reached a stronger conclusion about the meaning of the results. Below, we summarize AMF's findings, explain our perspective, and present an email Immo Kleinschmidt (bio, archived) sent to AMF about this data. For the full discussion of AMF's analysis, see AMF's "Ntcheu Update" (http://www.againstmalaria.com/update_ntcheu.aspx) and updated malaria case rate data.

    AMF's findings

    AMF performed a preliminary analysis of the malaria case rate data in which it concludes that the data shows a 40-50% decline in the number of malaria cases in March-June 2012, the months immediately after the distribution, compared to the same months in the prior year. (“We now have two sets of post-distribution data following the December 2011 to February 2012 distribution of 270,000 nets in Ntcheu District, Malawi and the results are strong. Summary 1. Malaria rates in March to June 2012 are already 50%, 45%, 40% and 40% lower than in the corresponding months in 2011.” Against Malaria Foundation, “Ntcheu Update.”)

    Our perspective There are two issues that lower our confidence in AMF's conclusions:

      Different malaria diagnosis methods used during the collection period

      The malaria case rate data was collected using different diagnostic mechanisms during the time periods being compared. From July 2010 to June 2011 malaria cases were identified on the basis of clinical observation. From July 2011 onwards, blood slide testing using Rapid Diagnostic Testing (RDT) kits were used.

      • Whereas RDTs use a blood test, ("For these reasons, the development, in the early 1990s, of a simple immunochromatographic test to detect malaria parasite antigens in a fingerprick blood sample was a major advance. RDTs do not require water, electricity or laboratory facilities and can easily be performed in remote rural settings." WHO, "Universal Access to Malaria Diagnostic Testing," Pg 4.) clinical observation relies on a clinician’s judgment. Often, fever is mis-identified as a sign of malaria, when it is actually caused by other illnesses. ("Last year, the WHO Global Malaria Programme issued revised guidelines for the treatment of malaria in which it was recommended that all suspected cases of malaria receive a diagnostic test prior to treatment. In many settings, especially in Africa, this represents a real paradigm change. For far too long, fever has been equated with malaria throughout most of the continent." World Health Organization, "Universal Access to Malaria Diagnostic Testing," Pg vii.)
      • This results in clinical observation overestimating the number of malaria cases, while RDTs are more accurate.

      AMF comments on its website that clinical observation is widely accepted to be prone to over estimate the true level of malaria. In order to estimate the actual drop in malaria cases after the distribution, AMF attempts to determine the over estimation of malaria cases from July 2010 to June 2011. AMF lays out its reasoning in the updated malaria case rate data. We do not find AMF's method robust.

      AMF estimates that during the rainy season months of March-June, clinical observation would yield a 25% higher rate of malaria diagnosis than RDTs, so it decreased the reported cases of malaria in the March-June 2011 period by 25%. AMF then compared recent months’ malaria data with its adjusted estimate of the true levels of malaria in the same months of the prior year. (Rob Mather, email to GiveWell, September 6, 2012.) This lowered AMF’s estimated drop in malaria for March – June 2012 to 40-50% (from an unadjusted 50-60%). (“As the March-June 2012 data falls within the peak malaria season, we estimate the actual malaria fall is some 10 percentage points less or 50%, 45%, 40 and 40% respectively." Against Malaria Foundation, "Ntcheu Update.")

      However, we believe that this adjustment is based on limited evidence, (“Data in the non-rainy season strongly indicates clinical observation overestimates malaria cases by 40-50%. This is concluded from comparing data from Jul to Nov 2011 (RDTK basis) with Jul to Nov 2010 data (clinical observation). In the rainy season there is limited data to draw conclusions but data for Dec 2011-Jan 2012 compared to Dec 2010-Jan 2011 suggests the over-estimate due to clinical observation in these months may be significantly lower and in the range of 5-15%. This has support from anecdotal evidence (we are trying to collect data from other studies) which suggests 'false positives' decline in the rainy season as a high percentage of those with 'malaria and fever type symptoms' do indeed have malaria." Against Malaria Foundation, "Ntcheu Update.") and therefore this data doesn't lead us to significantly adjust our prior view of this distribution's impact.

      Fluctuation in external factors

      Some of the change in malaria cases could be caused by external factors that influence malaria rates, such as rainfall, as opposed to the distribution. ("Rainfall is important for the availability of vector breeding sites. The effect of rainfall depends on the breeding habits of mosquitoes. It may boost the proliferation of species that breed in running water and suppress those that prefer stagnant water and small pools. Most spectacular is the effect of heavy rainfall in semi- arid and arid areas. It may be followed by epidemics that start 3–4 weeks after the rain, an event that has been observed in Khartoum (Sudan) and Djibouti. Not only is the overall amount of precipitation essential, but also the rate. Heavy rainfall produced within a few hours would flush out breeding places, whereas the same amount spread over a few days would facilitate breeding. Thus rainfall should be monitored on a daily basis, and good indicators are the total rainfall and the number of rainy days." World Health Organization, "Guidelines on the Elimination of Residual Foci of Malaria Transmission," Pgs 18-19.)

      Update (Oct. 11, 2012): AMF received rainfall data from the two weather stations in Ntcheu. The total rainfall in the rainy seasons was roughly similar between years in both places, but the month to month patterns were different. We have not attempted to answer the question of whether a variation in the month to month pattern of rainfall could affect malaria rates. See Against Malaria Foundation, "Ntcheu rainfall data," "Ntcheu rainfall data graphs," and Robin Todd, Concern Universal, email to Rob Mather, October 9, 2012.

    Dr. Immo Kleinschmidt, an epidemiological statistician from the London School of Hygiene and Tropical Medicine, assessed AMF's malaria case rate data and came to the following conclusions:

    "

    1. We know from many randomised trials that bednets are effective in protecting against malaria (see review by Lengeler et al, 2004), and
    2. You have evidence that many more people owned and used nets after the distribution compared to before the distribution

    It is therefore reasonable to conclude that the component of the decline in cases which cannot be accounted for by the change in diagnosis method, is very likely to be a result of the bednet distribution. This decline could be between 30% and 50%, but it is hard to put an exact figure on it.

    It is very plausible that the observed decline in malaria cases, after allowing for the change in diagnosis method, is associated with the mass distribution of nets." Dr. Immo Kleinschmidt, email to Rob Mather, September 11, 2012.

  • 27"Post‐Distribution Surveys (PDSs): These occur 6, 18, 30 and 42 months after a distribution. They assess three things: 1. Hang‐up % ‐ are the nets still being used? 2. Correct usage – are the nets being used properly? 3. Net condition – in what state are the nets?” Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 3.
  • 28 “Between 200 and 250 households in each of the 37 health centre catchment areas were surveyed to assess net hang-up and condition." Against Malaria Foundation, “Ntcheu Update.” "The first Post-Distribution Survey (PDS) is after 6 months and then subsequent PDSs would be 12 months apart." Rob Mather, email to GiveWell, August 8, 2012.
  • 29

    “We now have two sets of post-distribution data following the December 2011 to February 2012 distribution of 270,000 nets in Ntcheu District, Malawi.” Against Malaria Foundation, “Ntcheu Update.”

  • 30

    "This would achieve a survey of more than 5% of the covered [households] and nets and we considered this fine (actual numbers: 7,646 [households], 15,735 LLINs assessed = 5.9% of distributed nets)." Rob Mather, e-mail from Rob Mather, August 8, 2012.

  • 31

    “Net condition six months post-distribution is exceptionally strong. 99% are in 'very good' condition (fewer than two holes of 2 cms or smaller) and 1% are in 'OK' condition (fewer than 10 holes). We would expect nets to be in very good condition after six months but this is an exceptionally high level. The condition of the net is a subjective assessment. It is possible there is bias introduced by those carrying out the survey.” Against Malaria Foundation, “Ntcheu Update.”

  • 32"Households are randomly chosen [to be surveyed]. For Ntcheu, [Concern Universal] explained they used a randomly selected child to select randomly a household on a list and they then visited each fifth house on the list thereafter." Rob Mather, email to GiveWell, August 8, 2012.

    ”The following operating elements were put in place to mitigate against inaccurate data collection:

    • training was given to those carrying out the survey to ensure consistency of the counting of the number and size of holes;
    • Concern Universal staff were involved in and supervised the survey work and selected to carry out this work only a subset of the government's health staff, Health Surveillance Assistants (HSAs), known to be the most reliable;
    • emphasis was placed during training on accurate recording of net condition and other data as a) false data would not help our collective understanding of the condition of the net stock and b) spot checks would occur to check data accuracy;
    • those carrying out the survey work had nothing to gain by providing inaccurate assessments, not even significant time saved in carrying out the assessments as households had to sign the survey forms;
    • data verification through spot checks of pre-distribution registration information, gathered by HSAs, showed the data collected was exceptionally accurate.

    We believe the data is reliable. We will however be conducting some spot checks to test the accuracy of the data.” Against Malaria Foundation, “Ntcheu Update.”

    ”Are all the nets being used correctly? Please ask the householder to demonstrate how the nets are used at night.” Excerpted from post-distribution survey form, Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 4.