Against Malaria Foundation - August 2015 Update

Published: August 2015

Updates on completed distributions

Major updates on completed distributions:

• AMF has continued following up long-term on completed distributions. We have seen the results from three recent surveys in Malawi: a 33-month post-distribution check up (PDCU) on the 2011-2012 Ntcheu distribution in Malawi (56% of the people surveyed were sleeping under LLINs), a 14-month PDCU on the 2013 distribution in Balaka (73% under LLINs), and an 8-month PDCU on the 2014 distribution in Dedza (82% under LLINs).
• AMF has published documentation for the Kasaï Occidental distribution (completed in November 2014), which was AMF’s largest distribution to date and its first large-scale distribution outside of Malawi. In general, monitoring of the distribution was similar to that of AMF's previous large-scale distributions, though it appears there were fewer procedures to audit the accuracy of household data. We expect data from the first post-distribution check up to be available soon.

A summary of AMF’s completed distributions, the documentation that is currently available for each of the completed distributions, and the documentation that is expected in the future, is here.

More details on the updates are below.

Post-distribution check up surveys for distributions in Malawi

Ntcheu district

In December 2011 to February 2012, AMF, with Concern Universal, distributed about 270,000 nets in the district of Ntcheu, Malawi.1 Concern Universal has now completed its fourth post-distribution check up (PDCU) in the district 33 months after the distribution.2 Previously, Concern Universal had completed post-distribution check ups at 6, 15, and 24 months post-distribution.3

For this PDCU, a team of ten data collectors travelled to 37 Health Centre Areas (HCAs) and randomly selected 5% of the villages in each area to visit.4 In each village, the data collectors randomly selected 5% of the households that had received nets during the distribution to visit.5 The data collectors filled out a survey form for each household, which included information on the condition of the nets in the household and the number of nets hung over sleeping spaces.6 The survey results include data on net usage.7 The report does not specify how usage data was collected; AMF told us that the data collectors directly observed whether nets were hung and what condition they were in.8 Two supervisors randomly selected 5% of all of the households surveyed and revisited those households to check the accuracy of the data collectors’ work;9 we have not seen any details on who the supervisors were or any results from the supervisors’ random follow-up surveys. Once the data collectors submitted their data, the supervisors checked the data for errors.10 The data was then sent to four data entry clerks, who entered it into a web interface designed by AMF.11

Overall, no major issues were reported.12 Although $14,448 was budgeted for the post-distribution check up, the final cost of the survey was only $12,607,13 due in part to a change in the exchange rate between Malawi kwacha and U.S. dollars.14

Details on the results from the PDCU are below.

Balaka district

From October to December of 2013, AMF, with Concern Universal, distributed about 155,000 nets in the district of Balaka, Malawi.15 Concern Universal has now completed its second post-distribution check up survey in the district, 14 months after the distribution.16 Originally, Concern Universal planned to do the survey 12 months after the distribution, but the PDCU was delayed several months.17 Previously, Concern Universal completed a post-distribution check up 6 months after the distribution.18

The methodology for Balaka’s 14-month PDCU exercise was nearly identical to that of Ntcheu’s 33-month PDCU (see above for details).19 Like the 33-month Ntcheu PDCU, the 14-month Balaka PDCU came in under budget: $9,667 was originally allocated to the exercise, but only $8,337 was ultimately spent.20 This was again due to the favorable exchange rate between Malawi kwacha and U.S. dollars.21

Details on the results from the PDCU are below.

Dedza district

In September and October 2014, AMF, with Concern Universal, distributed about 245,000 nets in the district of Dedza, Malawi.22 Concern Universal has completed its first post-distribution check up survey in the district, 8 months after the distribution.23 Originally, Concern Universal planned to do the survey 6 months after the distribution, but the PDCU was delayed several months.24

The methodology for Dedza’s 8-month PDCU exercise was nearly identical to that of Ntcheu’s 33-month PDCU (see above for details).25 The budget for the PDCU was $12,849; however, the report does not include the actual costs of the PDCU exercise.26 AMF plans to update the report once data on how much the survey cost becomes available.27

Details on the results from the PDCU are below.

Results from the PDCU surveys

The table below shows results from the Ntcheu 33-month PDCU, the Balaka 14-month PDCU, and the Dedza 8-month PDCU (data sources in footnote).28

The 14-month post-distribution survey in Balaka found lower rates of people sleeping under a net (73%) than the previous 15-month survey in Ntcheu (84%). Similarly, the 8-month survey in Dedza found lower rates of people sleeping under a net (82%) than earlier 6-month surveys in Ntcheu and Balaka (90% in both districts). We do not know why recent results have been worse than earlier results.

All results from PDCUs of large distributions can be found here.

	Ntcheu: 33-month	Balaka: 14-month	Dedza: 8-month
# of households surveyed	9,500	4,535	9,100
# of AMF nets checked	20,688	8,540	15,676
% of nets surveyed of total distributed	7.7%	5.5%	6.3%
% of nets hung	52%	82%	93%
% of nets missing	3%	2%	1%
% nets in "very good" condition	13%	35%	57%
% nets in "good" condition	23%	39%	36%
% of nets in "viable" condition	19%	11%	6%
% of nets "worn out"	45%	14%	1%
% of nets used correctly	98%	100%	100%
% of people covered by nets	56%	73%	82%

Definitions:

• Nets hung: We have not seen a precise definition of this term. It is our understanding that interviewers are asked to observe whether nets are hung by entering interviewees' houses (rather than simply asking interviewees if they are hung).29
• "Very good" condition: LLIN has fewer than 2 holes of less than 2 cm in size.30
• "Good" condition: LLIN has fewer than 10 small holes.31
• "Viable" condition: LLIN has more than 10 small holes or has one large hole.32
• "Worn out": Survey reports from Concern Universal did not provide a definition of this term. AMF has told us in the past that an LLIN is considered worn out if it has multiple large holes and the LLIN is unrepairable, such that it would not provide protection against mosquitos.33
• Used correctly: According to a survey form from 2012, interviewers are asked to "ask the householder to demonstrate how the nets are used at night" and to then select yes or no to the question "are the nets being used correctly."34

The "decay model" we use to anticipate the how long an LLIN will last (i.e. how long a net is used and remains in good condition) assumes that:35

• 92% of LLINs last at least 0 to 12 months
• 80% of LLINs last at least 13 to 24 months
• 50% of LLINs last at least 25 to 36 months

AMF found that only 36% of the LLINs in Ntcheu were in "very good" or "good" condition at 33 months, with another 19% in "viable" condition.36 AMF notes that the coverage in Ntcheu is not reaching the levels of protection that AMF would like to provide; as a result, AMF is considering designing "injection strategy" campaigns to replenish nets two years after a distribution (instead of waiting three years).37 In Balaka, 74% LLINs were in "very good" or "good" condition at 14 months, with another 11% in "viable" condition.38 Previously, the 6-month post-distribution survey in Balaka found that 95% of nets were in "very good" or "good" condition.39 In Dedza, 93% of LLINs were in "very good" or "good" condition at 8 months, and most of the rest were in "viable" condition.40

It is not clear to us whether the rates of decay in Ntcheu, Balaka, and Dedza are comparable to the assumptions in the decay model, in part because it is not clear whether the definitions of "good condition" are similar in the model and AMF's data.

Recently completed distributions in Malawi

AMF, with its partner Concern Universal, has recently completed three distributions in Malawi. Updates on these distributions:

• Dedza 2014: AMF and Concern Universal completed a distribution in Dedza district in late 2014.41 AMF told us that photos were taken of the distribution and sent to AMF by mail; AMF expects to post the photos by the end of September.42 Both the distribution proposal and a sample of the pre-distribution registration survey data for the Dedza 2014 distribution have been posted.43 AMF expects to post the distribution report in September.44
• Dowa 2015: AMF and Concern Universal completed a distribution in Dowa district in March to May 2015.45 Sample pre-distribution registration survey data for the distribution has been posted, and AMF expects to receive a report on the distribution by the end of September.46 Additionally, three weekly reports have been posted for the distribution, which describe distribution of the first 47% of the LLINs required in Dowa and challenges encountered.47 (Note that AMF does not require distribution partners to submit weekly reports, but it does require a summary distribution report, which has not yet been posted for the Dowa distribution). The reports include summaries of some distribution activities (e.g., the number of villages visited and the number of nets distributed) and several lessons learned (e.g., the reports mention that the distribution was delayed by a month due to vehicle breakdowns).48
• Flood-affected districts: In February and March 2015, AMF provided emergency nets to people in Malawi who were affected by flooding in several districts. AMF previously told us that a 3-month post-distribution check up would be conducted for this distribution.49 We have not yet seen results from this check up and do not know when results will be available. AMF has told us that it expects to publish a report on this distribution by the end of September.50

Note that you can see the full details of the documentation available for each distribution in this spreadsheet.

Kasaï Occidental

From August to November 2014, AMF, with IMA World Health, distributed 624,532 nets in 9 Health Zones in the province of Kasaï Occidental, DRC.51 This was the first large-scale distribution (i.e., a distribution on the order of hundreds of thousands of nets) AMF had completed with a partner other than Concern Universal and in a country other than Malawi.52 Additionally, it was the first distribution that AMF had funded that used smartphones to collect data.53

Documentation

AMF has published:

• A distribution report54
• A report on the use of smartphones in the distribution55
• A summary of the household registration data by health zone (with a sample of the data collected)56
• Photos or video from six of nine health zones57

AMF has not published a distribution proposal from the distribution.58

Below are details on the distribution.

Methodology

IMA reports that it used the following procedures in the distribution:

• LLIN storage: The LLINs were kept at secure storage sites, and when the nets were moved in or out of a storage site, the number of nets were counted and recorded.59
• Data collection: During the distribution, teams of 3-4 Community Health Workers (CHWs) would visit each household.60 At each household, a team would register the household, establishing the number of sleeping spaces in the household, the number of LLINs the household already had that could last at least an additional 2 years, and other information (for a full list of the information collected, see the footnote).61 The information was entered into the team’s smartphone and was also recorded on paper.62
• Hanging nets: The team of CHWs hung up the appropriate number of nets in the household.63 The team used chalk to mark the house with an identification number, which indicated that the house had already been visited and which was also meant to help identify the house during post-distribution check ups.64 Finally, the team would verbally provide the household members with information about how to care for the nets and would take a picture of the hung nets.65 At the end of each day, each team brought its smartphone to the Health Area’s main health center, where the day’s data was uploaded to a laptop.66 The team’s Field Supervisor would then check the data.67 The report did not specify what these checks involved or what was done if errors were found.

Once the distribution in a given Health Area was complete, all of the data from the smartphones was compared to the paper copies of the data.68 We do not know what was done when errors were found, or how many errors there were. After the data had been checked, it was sent to the Health Zone’s central office to be compiled with other data, and ultimately sent to the Data Manager’s team, where it was analyzed.69 IMA shared the data with AMF, which ran its own analysis.70 AMF offered to share the data with us; we have not yet requested this. Note that because the analysis was conducted after the distribution was complete, the analysis could not impact how the nets were distributed; rather, the analysis was done to check for anomalies or inconsistencies in the data (results of AMF's analysis are below).71

It is our understanding that IMA is still analyzing some of the data (for example data on malaria prevalence) and will eventually produce a report with its analysis.72

Challenges and lessons learned

Before distributing nets in all 9 Health Zones, AMF and IMA ran a pilot distribution in the Health Zone of Nyanga to troubleshoot any problems with the methodology.73 As a result of the pilot distribution in Nyanga, several changes to the methodology were made (details in footnote).74

The Kasaï Occidental distribution report includes lessons learned for each Health Zone.75 Many of the Health Zones experienced similar problems. Some of the challenges encountered include (details are in the footnotes):

• Problems with transportation.76
• Inadequate training and incompetence of some CHWs.77
• Fewer nets were needed than expected: approximately 45,000 leftover nets are being sent to Nord Ubangi, DRC for AMF and IMA’s upcoming distribution there.78
• A few cases of dishonesty, by both CHWs and net recipients, were discovered.79
• Several operational issues, such as an 8-week delay due to an outbreak of Ebola.80
• Issues with the smartphones and other technology.81

Results

IMA was able to collect data for 91% of the nets distributed.82 IMA reports that the remaining 9% of data was lost due to technological issues.83 AMF believes that the 9% of nets for which data is missing were likely distributed appropriately (details in footnote).84 The paper records of the data were not used to fill in the gaps that resulted from technology failure.85

AMF checked the data for inconsistencies (for example, more sleeping spaces than people in a household) and anomalies (for example, 20 children under 5 in a household). 96% of the data that was collected had no errors that AMF could find.86 Taking into account missing data and removing inconsistent and anomalous data, AMF has household data for 87% of the nets distributed.87 Overall, AMF has told us that it believes these results to be quite successful, given the challenging context of the DRC and the fact that this was AMF and IMA’s first distribution using smartphones.88 AMF and IMA plan to move forward with the distribution in Nord Ubangi, DRC, which will also use smartphones and will use a similar methodology to the Kasaï Occidental distribution.89

Comparison: the Kasaï Occidental distribution and previous distributions in Malawi

There are a few key differences between the Kasaï Occidental distribution and previous distributions that AMF has funded in Malawi. In the Malawi distributions, Concern Universal distributed nets from central distribution locations, and villagers had to travel to the distribution site to pick up their nets.90 IMA sent workers house to house to distribute nets, and the workers hung up the nets at each house they visited.91

Additionally, in the Malawi distributions, a pre-distribution registration survey was conducted before the distribution of nets, which allowed the data to be checked by 1) a community verification exercise before the nets were handed out, and 2) a verification team that randomly revisited some of the households registered in the pre-distribution survey to review the accuracy of the data collectors' work.92 In the Kasaï Occidental distribution, IMA did not have similar checks on the data collectors’ work: IMA supervisors only verified the accuracy of the CHWs’ data collection by checking the data for obvious errors at the end of each day (and during data analysis once the distribution was over).93

Other ways in which the distributions differ include:

• In Malawi distributions, it is clear how the number of nets any given household received was determined: the number of sleeping spaces - the number of useable nets already in the household = the number of nets required.94 It was less clear how IMA workers determined how many nets a household needed; there is some evidence to suggest that IMA used a slightly different method (see footnote for details).95
• Concern Universal does not mark houses with an identification number during the distribution.

In-progress distributions

For information on recently completed distributions in Malawi, see above.

Nord Ubangi

Originally, a distribution in Nord Ubangi, DRC with IMA as the distribution partner was scheduled for April-June 2015.96 However, this distribution has been delayed, for the following reasons:97

• AMF required more time to discuss logistics details with IMA before the distribution.
• AMF was waiting for details on the Kasaï Occidental distribution before planning the Nord Ubangi distribution. Once those details were shared, AMF felt comfortable moving forward with the Nord Ubangi distribution. AMF also wanted to apply lessons learned from the Kasaï Occidental distribution to the Nord Ubangi distribution.
• The details for the study on insecticide resistance that is occurring with the Nord Ubangi distribution had to be settled before the distribution could begin.

The distribution is now expected to occur October 2015 to March 2016.98

Updates on potential future distributions

Major updates on potential future distributions are:

• AMF and Concern Universal have officially assumed responsibility for re-coverage campaigns in the districts of Ntcheu, Balaka, Dedza, and Dowa in Malawi in 2018.
• AMF is working to finalize five new distribution agreements. AMF has told us that it is in the late stages of discussion for each of these agreements.

More details below.

Distributions in Malawi 2018

AMF told us previously that the National Malaria Control Program (NMCP) of Malawi had requested AMF and Concern Universal’s assistance with a mass LLIN distribution in 2018.99 AMF announced in a recent blog post that Concern Universal and itself have officially accepted this responsibility.100 This distribution is estimated to cost AMF approximately $6 million, and AMF will be paying for both net and non-net costs.101

AMF’s current pipeline

Our last update noted that AMF was discussing potential future distributions with two countries (Country X and Country Y - AMF prefers not to share the names of countries it is negotiating with).102 Originally, AMF was under the impression that discussions with each of these countries would move forward quickly.103 However, discussions progressed more slowly than expected, so AMF decided to proceed with discussions with several more countries, in order to mitigate the risk of its discussions with Country X and Country Y not progressing.104

Currently, AMF is in discussion with five countries (including Country X and Country Y) about potential future distributions; most of these discussions are for one million to several million nets during 2015-2017.105 AMF told us that it is in fairly advanced stages of discussion with all five of the countries, meaning that:106

• AMF has sent a draft agreement to each country’s NMCP for approval and either a) has already received informal approval from the NMCP and is moving forward with other distribution arrangements or b) has had positive discussions with the NMCP but is still waiting on approval.
• AMF has a promising or committed co-funding partner for the non-net costs in each country.
• AMF has identified one or more promising candidates for its distribution partner in each of the countries, and is currently in the process of investigating the candidate(s).107

AMF does not have enough funding to cover the full net need in all five countries and has told each country that the longer it takes to come to an agreement, the less funding AMF might have (because agreements with other countries could be signed first).108 AMF is not planning to fund non-net costs for any of the countries it is in discussions with.109 AMF hopes that one or more agreements for net distributions will be signed within the next few months, but notes that there is a high degree of uncertainty around predicting when distribution agreements will be signed.110

AMF’s current funding situation

AMF currently holds about $21 million in funding, of which $9.9 million is already committed to future distributions, leaving $11.1 million available for future distributions.111 Of the $9.9 million, $4.1 million is committed to distributions in the near future (within the next 9 months), and the remaining $5.8 million is committed to distributions in Malawi at the end of 2017 and in 2018.112

AMF has mentioned before that it might use funding committed to distributions in the far future for distributions that are occurring sooner, because AMF expects that it will be able to raise the funds for the distributions in the far future at a later date.113 This implies that AMF has $16.9 million that it could use for future distributions in the near term.114

AMF has told us that the total net need for 2015-2017 of the five countries in which it is in discussions is 32 million nets.115 At $3 per net, it would cost $96 million to fully fill the funding gap for nets in these countries.116

Malaria case rate data

In the distribution agreements for the Ntcheu 2011-2012, Balaka 2013, and Dedza 2014 distributions, AMF requested that Concern Universal provide monthly malaria case rate data from before and after the distributions from all health centers in the three districts.117 After this data was delayed multiple times, AMF informed us in February 2015 that Concern Universal had sent the data and that AMF would publish it in a month or two.118 As of the writing of this update, the data has not been published, and when we last spoke to AMF we were told that the publication of the malaria case rate data was not a high priority.119 AMF told us that it has not made this a high priority because it does not believe that the data is high enough quality to reliably indicate trends in the malaria case rate.120

We no longer see the fact that AMF collects malaria case rate data as a consideration in AMF’s favor.

Data collection for other distributions in Malawi

We noted in our last update on AMF that the Malawi NMCP is planning to implement some of AMF’s distribution practices in districts for which AMF does not fund distributions.121 The practices will be implemented in 18 of 28 districts during a large distribution of approximately 7 million nets occurring towards the end of 2015.122

The practices that will be implemented are:123

• Organizations will collect pre-distribution registration data for all households
• Organizations may require that supervisors revisit 5% of the households that were surveyed for the pre-distribution registration to verify that data collectors’ work was accurate124

AMF has also advocated for (a) counting sleeping spaces (as opposed to people) during the pre-distribution registration, and distributing nets according to sleeping spaces;125 (b) putting all data in electronic form;126 and (c) post-distribution check ups.127 These practices have not been made requirements.128

Over the last several years, AMF believes that its high net hang-up rates have begun to catch the attention of a number of groups in Malawi, including the NMCP, local organizations that distribute nets, and large international NGOs.129 AMF also told us that it encouraged Concern Universal to take a more active role in a task force on malaria control in Malawi.130 This meant that as interest in learning from AMF and Concern Universal grew, Concern Universal was available to explain AMF’s methodologies at the malaria task force meetings, and it was eventually decided to implement some of AMF’s practices for the upcoming distribution.131

Sources

Document	Source
AMF all distributions page	Source (archive)
AMF blog post, February 18, 2015	Source (archive)
AMF blog post, July 3, 2015	Source (archive)
AMF dollars per net page	Source (archive)
AMF DRC West Kasai distribution summary numbers	Source
AMF financial information page	Source (archive)
AMF funds history and status	Unpublished
AMF future distributions net need	Unpublished
AMF how we work with distribution partners	Source
AMF page on Balaka 2013 distribution	Source (archive)
AMF page on Dedza 2014 distribution	Source (archive)
AMF page on Dedza 2017 distribution	Source (archive)
AMF page on Dowa 2015 distribution	Source (archive)
AMF page on Kasaï Occidental 2014 distribution	Source (archive)
AMF page on Nord Ubangi 2015 distribution	Source (archive)
AMF page on Ntcheu 2011-2012 distribution	Source (archive)
Balaka 2013 14-month PDCU data	Source (archive)
Balaka 2013 14-month PDCU report	Source (archive)
Balaka 2013 6-month PDCU report	Source (archive)
Balaka 2013 distribution proposal	Source (archive)
Balaka 2013 distribution report	Source (archive)
Dedza 2014 8-month PDCU data	Source (archive)
Dedza 2014 8-month PDCU report	Source (archive)
Dedza 2014 distribution proposal	Source (archive)
Dedza 2014 mid-distribution weekly reports	Source (archive)
Dowa 2015 mid-distribution weekly reports	Source (archive)
GiveWell AMF review 2014	Source
GiveWell AMF update August 2014	Source (archive)
GiveWell AMF update May 2015	Source (archive)
GiveWell estimate of AMF cost per net (November 2014)	Source
Kasaï Occidental 2014 summary and sample data	Source
Kasaï Occidental distribution report	Source (archive)
Kasaï Occidental technology report	Source (archive)
Malawi NMCP invitation for 2018 distribution	Source
Ntcheu 2011-12 24-month PDCU report	Source (archive)
Ntcheu 2011-12 33-month PDCU data	Source (archive)
Ntcheu 2011-12 33-month PDCU report	Source (archive)
Ntcheu 2011-12 distribution report	Source (archive)
Rob Mather and Peter Sherratt, conversation with GiveWell, February 6, 2015	Unpublished
Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015	Unpublished
Rob Mather, conversation with GiveWell, February 24, 2015	Unpublished
Rob Mather, email to GiveWell, February 24, 2015	Unpublished
Rob Mather, email to GiveWell, June 1, 2015	Unpublished
Rob Mather, email to GiveWell, June 19, 2015	Unpublished
Rob Mather, email to GiveWell, September 6, 2015	Unpublished
Rob Mather, email to GiveWell, June 30, 2014	Unpublished
Rob Mather, email to GiveWell, May 22, 2014	Unpublished

• 1

  AMF page on Ntcheu 2011-2012 distribution.

• 2

  AMF page on Ntcheu 2011-2012 distribution.

• 3

  AMF page on Ntcheu 2011-2012 distribution.

• 4

  "Ntcheu district has 37 health facilities. It was recommended to collect data from 5% of households in each HCCA [note: while HCCA is never defined in the report, we believe it is the same as HCA], which meant a different number of households in each HCCA as per individual health facility populations (refer to appendix A). Between 100 and 550 households were randomly selected from each of the selected four to 22 villages, depending on the HCCA. Villages were randomly selected using the village lists generated from the pre-distribution and distribution work for the December 2011-April 2012 AMF-funded universal coverage LLIN distribution. A random number table was used to select the villages." Ntcheu 2011-12 33-month PDCU report, Pg 6.

• 5

  "Between 100 and 550 households were randomly selected from each of the selected four to 22 villages, depending on the HCCA. Villages were randomly selected using the village lists generated from the pre-distribution and distribution work for the December 2011-April 2012 AMF-funded universal coverage LLIN distribution. A random number table was used to select the villages." Ntcheu 2011-12 33-month PDCU report, Pg 6.

• 6

  "The orientation included detailed explanation of the LLIN Distribution project back ground, how the beneficiaries were identified, the process of registration, data verification and LLIN net distribution, survey objectives and the logic behind the survey form (net condition, type of nets, what sleeping spaces are, what is meant by hung nets and noting hung nets against AMF nets received) as well as having the data collectors pre-test exercise in order to fill in sample forms and ask questions to ensure their understanding of what information should be collected and how." Ntcheu 2011-12 33-month PDCU report, Pgs 5-6.

• 7

  Ntcheu 2011-12 33-month PDCU data.

• 8

  "Data was collected by Health Surveillance Assistants (HSAs) via direct observation of whether the nets were hung and inspecting the nets’ condition." Rob Mather, email to GiveWell, September 6, 2015.

• 9
  • "The supervisors were responsible for checking the data collection exercise at the same time monitoring how the data was being collected as per requirement." Ntcheu 2011-12 33-month PDCU report, Pg 6.
  • "Supervisors were required to visit 5% of the households being interviewed to check the accuracy of the data collectors’ work and had to check all the completed forms submitted to them before submitting them to the Project Manager. The sampled visited households were also chosen at random so the work of all data collectors was checked." Ntcheu 2011-12 33-month PDCU report, Pg 6.

• 10

  "Once the data collection was complete, the data collectors submitted completed forms to their assigned supervisor who was responsible for checking the forms for obvious errors or omissions, including a lack of householder signature, before delivering the forms to the data entry team." Ntcheu 2011-12 33-month PDCU report, Pg 6.

• 11

  "There were four data entry clerks with knowledge in basic computing. The data entry clerks were also exposed to a questionnaire orientation where they were briefed on the forms and introduced to the online web links and how to enter the data on the electronic form, make editions and post the data. The data entry clerks were assigned specific health facilities in order to facilitate their performance monitoring. Data was entered into a database via a web interface created by AMF. An internet connection was required for this work." Ntcheu 2011-12 33-month PDCU report, Pgs 6-7.

• 12
  • "The operational elements that went well were:
    • All the selected villages were visited.
    • There was a positive response from the LLIN beneficiaries at community level.
    • The survey form was short with only one page, which was ideal for the data collectors and the respondents
    • Local community leaders and household heads allowed the data collectors to enter their households to see the hung nets and check the condition they were in.
    • Management support and commitment towards the activity by Concern Universal and District Health staff was very encouraging, hence the timely execution of the exercise.
    • The data collectors were committed to collecting the data.

    The elements that did not go so well were:

    • In some selected villages, on a planned data collection day, the team faced challenges due to funerals hence data collection was delayed."

    Ntcheu 2011-12 33-month PDCU report, Pgs 7-8.

  • This was the first time that Concern Universal used a small group of data collectors for the entire survey; in previous PDCUs Concern Universal used different data collectors for each of the 37 HCAs:

    "There are 37 Health Centres (HCs) in Ntcheu District. Each has approximately 20 staff attached to each one, the majority being salaried Health Surveillance Assistants (HSAs). However, in collaboration with the District Environmental Health Office (DEHO) and Malaria Coordinator (MC) and lessons leant from 24 month Ntcheu PDCU, it was recommended to have a focused team of 10 data collectors rather than have the HSAs as data collectors from each HCA. This was based on the following reasons.

    First, this would reduce the number of data collectors that would need to be monitored and trained. Second, we would be able to select reliable individuals whom we could trust to do a diligent and accurate job of collecting the data. Third, it would leave the majority of HSAs to carry on with the normal health tasks and duties. Fourth, by having the same people covering the whole exercise they will get acquainted to the task and reduce errors on data collection.

    This meant the data collectors would spend thirty seven days collecting data rather than the one or several days if many more data collectors were to be used. This was judged the preferable way of organising and managing the data collection phase."
    Ntcheu 2011-12 33-month PDCU report, Pg 5.

  • The new strategy seems to have worked well; Concern Universal recommended that it be used in future PDCUs: "The same data collectors should be hired to collect the data for the whole exercise in the upcoming subsequent PDCU surveys. Likewise the same data entry clerks should be involved in the next subsequent upcoming PDCUs since they are already familiar with the system." Ntcheu 2011-12 33-month PDCU report, Pg 8.
  • Concern Universal also recommended that permanent vehicles be used in future PDCUs to avoid transportation delays: "In order to maintain and follow the timeline and meet the deadlines permanent vehicles should be allocated to the activity." Ntcheu 2011-12 33-month PDCU report, Pg 8.

• 13

  "The budget was MK 5,779,367.00 (US$14,448) and the total PDCU cost was MK 5,774,064.00 (US$12,607). MK = Malawi Kwatcha." Ntcheu 2011-12 33-month PDCU report, Pg 7.

• 14

  Comment from AMF: "The PDCU came in exactly on budget in Malawi Kwatcha. Given the exchange rate MK:USD moved in our favour, in USD terms, the cost was approximately 15% less expensive than expected. CU has managed the budget very well." Ntcheu 2011-12 33-month PDCU report, Pg 7.

• 15

  AMF all distributions page.

• 16
  • AMF page on Balaka 2013 distribution.
  • "This check-up was carried out at 14 months, rather than 12 months, post-distribution due to planning delays." Balaka 2013 14-month PDCU report, Pg 3.

• 17

  "This check-up was carried out at 14 months, rather than 12 months, post-distribution due to planning delays." Balaka 2013 14-month PDCU report, Pg 3.

• 18

  AMF page on Ntcheu 2011-2012 distribution.

• 19
  • See the Ntcheu 2011-12 33-month PDCU report and the Balaka 2013 14-month PDCU report for comparative details on their methodologies.
  • "Data was gathered in all of the district’s 14 Health Centre Areas (HCAs). 4,535 households (HH) were randomly selected and visited unannounced. This check-up was carried out at 14 months, rather than 12 months, post-distribution due to planning delays." Balaka 2013 14-month PDCU report, Pg 3.

• 20

  "The budget was MK 3,866,920.00 (US$ 9,667) and the total PDCU cost was MK 3,818,565.00 (US$ 8,337). MK = Malawi Kwatcha." Balaka 2013 14-month PDCU report, Pg 7.

• 21

  Comment from AMF: "The PDCU came in on budget in Malawi Kwatcha. Given the exchange rate MK:USD moved in our favour, in USD terms, the cost was approximately 15% less expensive than expected. CU has managed the budget very well." Balaka 2013 14-month PDCU report, Pg 7.

• 22
  • AMF page on Dedza 2014 distribution.
  • Note: Since the writing of this update, AMF has updated its website to show that 290,770 nets were distributed in the Dedza 2014 distribution. This is because AMF recently completed a large mop-up distribution for the Dedza 2014 distribution; the mop-up distribution gave out approximately 45,000 nets to recipients who were missed during the initial distribution. More on the the mop-up distribution can be read here.

• 23
  • "This check-up was carried out at 8 months, rather than 6 months, post-distribution due to planning delays." Dedza 2014 8-month PDCU report, Pg 3.
  • "One of the CU vehicles transporting data collectors and the Malaria Coordinator was involved in an accident during the final week of data collection, resulting in the total loss of the vehicle as well as injuries to the staff members and the Malaria Coordinator. This accident delayed the data collection exercise as the injured staff members required time off to recover from the physical and psychological trauma. A replacement vehicle had to be arranged in order to proceed with the activities. Note: The staff injured have all recovered fully. The Malaria Coordinator suffered minor injuries that did not require medical treatment. All were able to resume work after a few days off." Dedza 2014 8-month PDCU report, Pg 7.

• 24
  • "The check-up was carried out at 8 months, rather than 6 months, post-distribution due to planning delays." Dedza 2014 8-month PDCU report, Pg 3.
  • "One of the CU vehicles transporting data collectors and the Malaria Coordinator was involved in an accident during the final week of data collection, resulting in the total loss of the vehicle as well as injuries to the staff members and the Malaria Coordinator. This accident delayed the data collection exercise as the injured staff members required time off to recover from the physical and psychological trauma. A replacement vehicle had to be arranged in order to proceed with the activities. Note: The staff injured have all recovered fully. The Malaria Coordinator suffered minor injuries that did not require medical treatment. All were able to resume work after a few days off." Dedza 2014 8-month PDCU report, Pg 7.

• 25
  • See the Ntcheu 2011-12 33-month PDCU report and the Dedza 2014 8-month PDCU report for comparative details on their methodologies.
  • "Data was gathered in 33 of the district’s 34 Health Centre Areas (HCAs). 9,100 households (HH) were randomly selected and visited unannounced. This check-up was carried out at 8 months, rather than 6 months, post-distribution due to planning delays." Dedza 2014 8-month PDCU report, Pg 3.

• 26

  "The PDCU budget cost was US$12,849 equal to US$1.41 per household visited (or $0.052 per net originally distributed). The actual cost are still being collated and this document will be updated to include them in the coming weeks." Dedza 2014 8-month PDCU report, Pg 3.

• 27
  • "The actual cost are still being collated and this document will be updated to include them in the coming weeks." Dedza 2014 8-month PDCU report, Pg 3.
  • AMF notes that "Actual cost numbers are reported to us typically three months after the end of the activity when all cost items have been gathered by CU’s finance department. We expect to have the actual cost by the end of September. A new dedicated malaria-related accountant started at CU Malawi on 01Sep." Rob Mather, email to GiveWell, September 6, 2015.

• 28
  • Ntcheu 2011-12 33-month PDCU data
  • Ntcheu 2011-12 33-month PDCU report
  • Balaka 2013 14-month PDCU data
  • Balaka 2013 14-month PDCU report
  • Dedza 2014 8-month PDCU data
  • Dedza 2014 8-month PDCU report

• 29
  • AMF told us last year that this was the case. Rob Mather, email to GiveWell, June 30, 2014.
  • "The operational elements that went well were: … Local community leaders and household heads allowed the data collectors to enter their households to see the hung nets and check the condition they were in." Ntcheu 2011-12 33-month PDCU report, Pg 7. The same quote is on Pg 7 of the Balaka 2013 14-month PDCU report and Pgs 7 of the Dedza 2014 8-month PDCU report.

• 30

  "'very good condition' (fewer than 2 holes of up to 2cms in size)." Ntcheu 2011-12 33-month PDCU report, Pg 4.

• 31
  • "‘good condition’ (fewer than 10 small holes on them)" Ntcheu 2011-12 33-month PDCU report. Pg 3.
  • Note that nets with fewer than 10 small holes used to be considered in "OK" condition.
    "'OK' (fewer than 10 small holes on them)" Ntcheu 2011-12 24-month PDCU report, Pg 4.

• 32
  • "‘viable condition’, with more than 10 holes or 1 hole larger than 10 cm," Ntcheu 2011-12 33-month PDCU report, Pg 3.
  • Note that nets with more than 10 small holes used to considered in "poor" condition. "'Poor' condition with more than 10 holes or 1 hole larger than 10 cm." Ntcheu 2011-12 24-month PDCU report, Pg 4.

• 33

  "Worn out means not usable. Typically, holes will be 10 cms of cms in size and numerous and the net considered effectively unrepairable and/or structural integrity will have gone. If hung the net would not provide protection against mosquitoes given the extent of holes or tears or other loss of structural integrity." Rob Mather, email to GiveWell, June 30, 2014.

• 34

  AMF how we work with distribution partners, Pg 4.

• 35

  "Alliance for Malaria Prevention 2011 lays out a model for estimating the number of LLINs still in use after distributions: 'the number of LLINs already distributed over the last three years and considered to be available in households should be calculated and subtracted from the total need, working with a decay rate of 8 per cent at one year (0-12 months), 20 per cent at two years (13-24 months) and 50 per cent at three years (25-36 months).'…
  Bottom line: We believe that the "8%-20%-50%" model is the most widely used and most reasonable approximation available at the moment for capturing the extent to which LLINs remain in use in the years following distribution, accounting for any factors that might cause LLINs to be discarded or additional LLINs to be purchased. It implies an average of 2.22 years of use for each LLIN distributed. Data and analysis on this topic appears extremely thin; we have little sense for how long LLINs last in practice." GiveWell report on the "decay model" for LLINs.

• 36

  See the table above.

• 37

  "This suggests for other distribution areas showing a similar decline in coverage over time other actions are necessary if the target is to maintain a minimum 80% sleeping space coverage throughout the typical three year period between mass universal coverage distributions. We are considering with partners various developments. One possible development is an ‘injection strategy’ that brings in a quantity of nets at 24 months, and perhaps at 12 months, to increase net coverage to a level that ensures any decline over the subsequent year leaves sleeping space coverage at or above 80%. This would involve a mechanism by which sleeping spaces in need of a net, across all households, were accurately tallied at the 24 month, and perhaps 12 month, point. This could involve a district wide ‘pre-distribution registration survey’, similar to that carried out prior to a mass distribution or could perhaps be achieved through engagement of local communities. The practicality of different approaches, associated costs and the coverage likely to be achieved is being assessed. What is clear is that sleeping space coverage is below 80% during a significant part of year three post-distribution and better malaria control is likely to be achieved through maintaining an 80% or above coverage level during this third year." Pg. 4, Ntcheu 2011-12 33-month PDCU report.

• 38

  See the table above.

• 39
  • By the definitions in the PDCUs, "OK" is the same as "good." Balaka 2013 6-month PDCU report, Balaka 2013 14-month PDCU report.
  • See the "Results from follow-up surveys" section of GiveWell AMF update August 2014.

• 40

  See the table above.

• 41

  AMF page on Dedza 2014 distribution.

• 42
  • Originally, AMF did not think that photos were taken of the distribution. Rob Mather, email to GiveWell, June 19, 2015.
  • Later, AMF discovered the photos were being sent by mail: "Photos were taken and they are being sent to AMF (by post). We expect to have a selection added to the website by the end of September." Rob Mather, email to GiveWell, September 6, 2015.

• 43

  AMF page on Dedza 2014 distribution.

• 44

  Rob Mather, email to GiveWell, September 6, 2015.

• 45

  AMF page on Dowa 2015 distribution.

• 46
  • AMF page on Dowa 2015 distribution.
  • Rob Mather, email to GiveWell, September 6, 2015.

• 47

  The distribution occurred March through May 15th. The weekly reports cover March 23 - 27, April 27 - 30, and May 4 - 8. The report from the third week notes, "To date we have distributed 184,554 nets of a total of 391,608 nets required, representing 47% of the whole district’s coverage." Dowa 2015 mid-distribution weekly reports.

• 48
  • "The major challenge we encountered during the distribution was the breaking down of our distribution vehicles, which forced us to interrupt the distribution for almost one month to have them fixed, as some spare parts could not be sourced locally." Dowa 2015 mid-distribution weekly reports, Pg 2.
  • AMF does not require Concern Universal to produce weekly reports on the distribution; Concern Universal does this of its own accord. We believe these reports reflect well on Concern Universal’s commitment to transparency, and they increase our confidence in Concern Universal’s ability to discover problems with a distribution. The weekly reports from the Dowa 2015 distribution had a similar level of detail to previous weekly reports from other distributions (e.g., see the Dedza 2014 mid-distribution weekly reports).

• 49

  See our May 2015 update on AMF.

• 50

  Rob Mather, email to GiveWell, September 6, 2015.

• 51
  • AMF all distributions page.
  • IMA also provided 86,500 nets from its own stocks: "This total included the 676,000 AMF‐funded nets and 86,500 LLINs from IMA stocks." Kasaï Occidental distribution report, Pg 6.

• 52

  AMF all distributions page.

• 53
  • "This distribution is the first that both parties have undertaken using smartphones to collect household-level data. We expect teething problems, as with any new introduction of technology, especially in a country such as DRC. However, we are undeterred by this, as the benefits of improving processes in this region are so significant." AMF blog post, February 18, 2015.
  • Some of the benefits AMF believes come from the use of smartphones include:
    "1. Acts against potential theft Data in electronic form can be shared widely and easily so net numbers can be tracked with precision and visibly. This inhibits theft, making occurrences obvious, quickly. There was no material loss of nets in this distribution.
    2. Improves accountability Detailed data, at the household level, can be shared with partners demonstrating what has taken place. That was the case here.
    3. Increases transparency All partners, including donors and the wider public, have access to detailed data and can see what has taken place. That was the case here.
    4. Increases data accuracy Data can be processed more quickly and errors spotted with time for corrections to be made prior to the net distribution taking place. That was not the case here as the registration and distribution were contemporaneous.
    5. May reduce cost Data processing is simpler compared to paper based data. We are waiting for final details of cost numbers to assess if costs were lower and if so by how much.
    6. Additional data can be collected GPS data and photos can be easily collected. That was the case here.
    7. Reduces operational risk As long as electronic data is backed up, the risk of data loss, compared to data in paper form, can be significantly reduced or eliminated. While the vast majority of data was accurately stored and transferred, some lessons have been learned to improve in this area." Pg. 4, Kasaï Occidental technology report.

• 54

  The distribution report includes an explanation of the methodology used for the distribution, as well as detailed reports of problems encountered during the distribution and plans to address those problems in the future. Kasaï Occidental distribution report.
  

• 55

  Kasaï Occidental technology report.

• 56

  Kasaï Occidental 2014 summary and sample data.

• 57

  AMF page on Kasaï Occidental 2014 distribution.

• 58

  AMF has told us that it has a distribution proposal from IMA but that publishing that publishing it is a lower priority task. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 59
  • "The nets were securely stored at Lofice, the transport company used with the following security: 24 hour guards, secured enclosure and secured internal warehouse. The nets remained in their containers and on the trucks for a week before onward road travel to Tshikapa District." Kasaï Occidental distribution report, Pg 5.
  • "At each net storage site there was a warehouseman who wrote down the quantity of nets entering or leaving. Specifically, the number of bales was counted in/out (rather than a verbal statement of quantity by one person being accepted) and two signatures placed against the number recorded." Kasaï Occidental distribution report, Pg 19.
  • "In each health area there were at least two storage sites for supplies (nets plus other materials). Daily movement of nets was strictly recorded in the daily stock ledger so that it was known how many nets had gone to each Health Area or village. Any nets remaining after the distribution were returned to the secure storage locations. Any movement of nets involved manual counting of the bales of nets with two individuals having to sign for the precise quantity leaving, or being received at, the secure storage area and being loaded on, or unloaded from, the transport vehicle. The same process was required to be done when the nets arrived at a distribution location, with the transport driver and the individual in charge of the distribution location, both counting and signing for the nets." Kasaï Occidental distribution report, Pgs 18-19.

• 60
  • "Teams of three or four CHWs visited households sequentially." Kasaï Occidental distribution report, Pg 16.
  • "CHWs worked in teams of 3 or 4 to ensure mutual peer control." Kasaï Occidental distribution report, Pg 19.

• 61
  • "This was a crucial stage of the net distribution that consisted of collecting relevant household data for every single household in the distribution catchment area including, but not limited to, the following ‘must have’ information:
    • Name of household head
    • Number of people in the household
    • Number of sleeping spaces (‘for the purpose of nets’*)
    • Number of perfectly good nets with at least two years of life left in them
    • Location of household (GPS coordinates)
    • Household identifier"

    Kasaï Occidental distribution report, Pg 16.

  • "The CHW asks for additional information not contained in the registration questionnaire (incidents of malaria, socio‐demographic information, what type of medication they took for malaria and information on the demographic make‐up of the household to enable the collection and integration of the data." Kasaï Occidental distribution report, Pg 17.

• 62
  • "The CHW responsible for entering household data (CHW with the smartphone) gathers the required data with the help of the head of household." Kasaï Occidental distribution report, Pg 17.
  • "The CHWs visiting beneficiaries kept a hard copy list of the household information entered in the smartphone in case of data failure or a need to check data" Kasaï Occidental distribution report, Pg 19.
  • We do not know if IMA will continue to collect data by paper in future distributions or if this was a one-time methodology decision as insurance against the smartphones completely failing.

• 63
  • "The CHW responsible for the hanging of the nets does so with the help of the head of household and/or other household members, using the string and nails brought along if necessary. A net is hung over each sleeping space (two people per sleeping space) and then rolled up out of the way (for example off the ground) during the day to help avoid unnecessary wear and tear or damage. It is then rolled down at night." Kasaï Occidental distribution report, Pg 17.
  • It is our understanding that if a household had a high-quality net that could last an additional 2 years, IMA would hang this net up instead of providing a new one (or leave it in place if it were already hung). We believe this because a) AMF's net distributions in Malawi did not provide new nets if a household already has usable nets and b) IMA collected information on the "number of perfectly good nets with at least two years of life left in them." We think it is likely that IMA collected this information to determine how many nets to give. Kasaï Occidental distribution report, Pg 17.
  • We are not entirely clear on how IMA determined the appropriate number of nets to give to each household, although we believe that IMA relied on the number of sleeping spaces in a household. The following quote is in the distribution report: "When assessing the number of sleeping spaces it is important to avoid confusion between the number of separate areas in which individuals might sleep and, given nets distributed are ‘double’ nets that will accommodate two people, the number of sleeping spaces that is equal to the number of nets that are needed. For example, in a household with mum plus dad, four children aged eight, four, six and two and a grandfather there may be six separate sleeping areas: mum plus dad, one each for the four children and one for the grandfather. However, for the purposes of nets, there would be four sleeping spaces: one for mum and dad, one for the two older children, one for the two younger children and one for the grandfather." Kasaï Occidental distribution report, Pg 16. To continue with the example quoted, we believe that this could mean that if the six separate sleeping areas in the household were spread out in such a way that 4 nets could not cover them, the household would still only receive 4 nets, and would have to move its sleeping areas closer together if it wanted to cover everyone. We are not sure how easy it is for a household to move sleeping areas, or how it was decided where to hang nets in cases like this.
  • On the other hand, when we look at the sample data collected, it does not seem to support this interpretation. For example, row 9 shows 5 people living in a household, and there are 5 sleeping spaces in the household. Because the household already had 2 useable LLINs, it received 3 from IMA. This seems to imply that even if there was only one person using a sleeping space, IMA would still give a net to cover the whole sleeping space, and not require that the sleeping space be moved close to other sleeping spaces so that two people could be covered with one net. Sample Data-Anonymized sheet, Kasaï Occidental 2014 summary and sample data.

• 64
  • "In order to both a) avoid returning to a household that had already received nets and b) be able to identify the household during subsequent post‐distribution follow‐up, a CHW marked a unique household identifier using chalk on the outside of the household in an easily visible place (for example on a doorframe, door or other front area)." Kasaï Occidental distribution report, Pg 17.
  • "Creating a household identifier is necessary as many households do not have clear addresses. For example, it is frequently the case in rural and semi‐rural locations that street names and household numbers do not exist. The household identifier was created as one long number, as follows: Day of distribution + Unique number of the smartphone being used to collect the data + Household number for that day’s distribution + Number of sleeping spaces + Number of nets installed. For example: 171115‐2365714‐015‐04‐04." Kasaï Occidental distribution report, Pgs 16-17.
  • We are not sure if the chalk is easily wiped off or might be washed away when it rains. We are also not sure if there was another way for the CHWs to determine which households had already been covered, although we imagine that since nets were being hung up in households as the distribution went, it was somewhat easy to tell which households had already been visited.

• 65
  • "The final step for the CHWs was to pass on information to as many household members as possible regarding the correct use of the net, including: tucking the net under mats or mattresses at night; not leaving any gaps through which mosquitoes could pass; rolling up the net during the day to minimize damage; avoiding causing holes and rips in the net; the importance of repairing the net if holes develop; and lastly, the ability to wash to net 20 times in the next 3 years." Kasaï Occidental distribution report, Pg 17.
  • "Once hung, the CHW took a photo using the smartphone to record an image of the successfully delivered and hung nets. If the householder did not want to be in the photo, just a photo of the net was taken." Kasaï Occidental distribution report, Pg 25.

• 66

  "At the end of each distribution day each smartphone was brought to the Health Area’s main health centre and, via a WiFi connection, the data collected on the smartphone was uploaded (transferred) to a locally located laptop." Kasaï Occidental distribution report, Pg 17.

• 67

  "At the end of each day, the CHWs returned the smartphones to the Field Supervisor and the data was imported into a computer using a WiFi router connection. The Field Supervisor checked the data to make sure the household registrations had been carried out correctly." Kasaï Occidental distribution report, Pg 25.

• 68

  "After the distribution had been completed in a Health Area and all data uploaded from the smartphones, the data team compared the electronic data with the paper data collected by each distribution site in each health area." Kasaï Occidental distribution report, Pg 18.

• 69

  "Household data went through the following steps: 1. Data collected at the household by a CHW 2. Independent supervisor uploaded the data from the smartphone to a laptop. 3. The supervisor reviewed the data and corrected obvious mistakes or inconsistencies. 4. The data across a Health Area was compiled into one data set 5. The data was transported to the Health Zone central office via a USB memory stick and compiled to give a complete data set for the Health Zone. There were between 17 and 28 Health Areas in a Health Zone. 6. The data was sent by USB memory stick and hard disc from the Health Zone level to the Assistant Data Manager at the central level for overall compilation 7. The data was analysed by the Data Manager and Assistant Data" Kasaï Occidental distribution report, Pg 18.

• 70
  • Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.
  • "The distribution the data was stored in Excel spreadsheets, one for each Health Zone. Data was supplied by IMA Health to AMF in this way." Kasaï Occidental technology report, Pg 13.

• 71
  • "As the data received was after distribution of the nets, analysis of the data would not lead to cleaned and corrected data being sent back to the field to influence the number of nets distributed to a particular household. This was because household registration and distribution of nets – and indeed hang‐up of the nets ‐ was contemporaneous. IMA wished to conduct the distribution in this way to gain information on high hang‐up rates expected to be achieved through immediate hang‐up of the nets. The aim of the analysis was therefore to assess the accuracy of the distribution as judged by: 1. How accurately did the data match records for the number of nets moved to each pre‐ positioning and final distribution point? ...2. How free was the data of ‘inconsistency’ and ‘anomaly’ errors?" Kasaï Occidental technology report, Pgs 13-14.
  • Note that this is different from AMF’s distributions with Concern Universal in Malawi, where pre-registration survey data is checked by a) supervisors randomly revisiting some of the households that were surveyed to ensure that the data is accurate, and b) village meetings where Concern Universal reads off household names and the number of LLINs allocated to that household, and households indicate if errors have been made. All of this happens before the distribution and thus can have an impact on the number of nets a household receives. GiveWell AMF review 2014.

• 72

  "The IMA Monitoring and Evaluation (M&E) team is currently in the process of analysing data for other indicators such as prevalence of malaria and will distribute a separate IMA report of the findings." Kasaï Occidental distribution report, Pg 18.

• 73
  • "For this distribution, Nyanga was considered a pilot area and a separate net distribution was conducted there shortly before the other HZs received nets." Kasaï Occidental distribution report, Pg 4.
  • "A pilot distribution was used to test the technology at a small but sufficient scale so lessons could be learned before larger roll out." Kasaï Occidental technology report, Pg 5.

• 74
  • CHWs were estimated to be able to distribute nets to 50 households per day based on the rate of distribution in Nyanga, which impacted the number of phones purchased for the subsequent distribution.
    • "The number of phones was determined by the number of CHWs needed to perform the distribution. It was estimated that CHWs could register 50 households per day. This estimation was taken from the average number of households RECOs registered in one day during the pilot distribution in Nyanga." Kasaï Occidental distribution report, Pgs 6-7.
    • Note that this seems slightly at odds with another section of the distribution report: "The number of phones required and therefore bought was calculated according to the number of CHW teams needed to perform the distribution and hang‐up in two days per health zone. We estimated that each CHW team would be able to cover 40 households per day. Therefore the total number of households was divided by 80 to determine the number of phones required. The calculation did not specifically include a reserve quantity in case of failure/out of service phones." Kasaï Occidental distribution report, Pg 10.
  • Additional batteries to charge phones were bought after the Field Supervisors in Nyanga found themselves travelling large distances at night to recharge phones for the next day of the distribution.
    • "In the pilot distribution in Nyanga Health Zone, supervisors would swap unused phones during the data collection when a smartphone ran out of charge. Furthermore, once the net distribution team was in the field, the ability to recharge the phones (finding a generator, solar installation, logistics) was reduced drastically. Many supervisors would travel back (for some, a 2 hour roundtrip) each night to recharge the phones because power sources in the villages were non‐existent. Others were able to rent power generators locally. Thus, for this distribution, additional batteries were bought." Kasaï Occidental distribution report, Pg 11.
  • Wifi routers were purchased to upload data more quickly.
    • "Lessons learned during the pilot distribution stage in Nyanga influenced us to change a few things for these distributions. Specifically:

      1. We purchased WiFi routers which made for quicker uploading of data." Kasaï Occidental distribution report, Pg 20.

  • The registration survey on the smartphones was tweaked to make data entry easier.
    • "The modification of the data entry forms made after the pilot distribution in Nyanga HZ led to cleaner data (more accurate, fewer errors). The merger of the two questionnaires (= data forms), namely for registration and for distribution reduced the time of the interview at each household and removed duplicate questions." Kasaï Occidental distribution report, Pg 27.
  • IMA changed its procedures such that registration and distribution activities would occur simultaneously (during the pilot, registration happened on the first day and distribution happened the next day). Generally, IMA observed that the distribution went more quickly when registration and distribution occurred simultaneously.
    • "The new method of carrying out the distribution, i.e. performing the household data entry and the handing out and hanging of the nets at the same time worked very well and saved time. In the pilot distribution in Nyanga the registration took place on the first day and the distribution of the nets required took place on the second day." Kasaï Occidental distribution report, Pgs 26-29.
    • "3. Registering the household (collecting data) and distributing the nets at the same time allowed us to complete the distribution at a much faster pace." Kasaï Occidental distribution report, Pg 20.

• 75

  Kasaï Occidental distribution report.

• 76
  • "There weren’t enough trucks available which made movement of nets more difficult than desirable. Lesson learned More attention needs to be given to this in initial logistics planning." Kasaï Occidental distribution report, Pg 27.
  • "The estimates of distances between the central location, where staff would meet at the start of each day and the health areas to which they would travel, were not accurate leading to delays Lesson learned Double‐checking of estimated distances to ensure they are sufficiently accurate."Kasaï Occidental distribution report, Pgs 33, 46, and 54.
  • "Vehicle breakdowns. This affected the movement of field supervisors and delayed some supervision activities. Private vehicles had to be rented. Lesson learned Consideration of the vehicles and their reliability in the planning process." Kasaï Occidental distribution report, Pgs 33, 47, 50, and 55.
  • "Bad condition of the roads
    This made travel more difficult and caused delays in the distribution. Lesson learned Consideration of the condition of the roads in the planning process." Kasaï Occidental distribution report, Pgs 37, 40, 43, and 50.
  • "The permanent break down of some vehicles and motorbikes slowed down supervisions. Lesson learned Consideration of the vehicles and their reliability in the planning process.

• 77
  • "The level of instruction given to the CHWs was so low that additional training sessions had to be organised, covering familiarity with the equipment and its practical use to ensure the CHWs were fully familiar with the smartphone and the ODK Collect registration programme and what information had to be collected. The additional training included practical use of the smartphone. Lesson learned Greater attention needs to be given to the initial training." Kasaï Occidental distribution report, Pgs 33, and 46.
  • "Some CHWs complained the pay was not enough for the work they were asked to do and this seemed to slow down the work done. Lesson learned Establish clearly the daily pay rate and ask, as CHWs are selected, they accept that daily rate." Kasaï Occidental distribution report, Pgs 33, 47, and 55.
  • "Certain CHWs, even after training, were not up to the level required to carry out the job well. Lesson learned Ensure better initial screening and selection of the CHWs before training starts." Kasaï Occidental distribution report, Pg 37.
  • "There was too much influence over selection by some community leaders and those selected were not good enough to do the job. Lesson learned Greater attention needs to be given to the initial selection to ensure those selected as capable for the training and the job." Kasaï Occidental distribution report, Pgs 46, 50, and 54-55.
  • "Some CHGWs were found to be inebriated during working hours. This was found to be linked to a number of nurses and health staff had recruited family members and friends rather than individuals qualified to be CHWs. The individuals were dismissed, unpaid. Lesson learned Monitoring CHWs is important." Kasaï Occidental distribution report, Pg 29.
  • "A Field Supervisor was found to be sitting under a tree and not working rather than supervising CHWs. He was dismissed and sent back to Tshikapa. Lesson learned Monitoring Field Supervisors’ performance is important." Kasaï Occidental distribution report, Pgs 29.

• 78
  • "IMA obtained population data for the year 2012 from the Ministry of Health and applied the MOH’s estimate for population growth of 3% per annum." Kasaï Occidental distribution report, Pg 6.
  • "There were many occasions where the number of nets needed for a community was overestimated. This was discovered in the registration and hang up because of diligent work done. Lesson learned Better population estimates are needed and 1.8 people per nets should be the ratio always used." Kasaï Occidental distribution report, Pg 46.
  • "In a significant majority of Health Areas more nets were pre‐positioned than were needed. The estimates of population were not accurate enough. Lesson Learned There should be a critical assessment of the population numbers used to determine net numbers." Kasaï Occidental distribution report, Pg 20.
  • "The population was in fact half of that estimated. [Note: this refers to the population in a particular health zone] Lesson learned Greater attention needs to be given to the accuracy of population information and assumptions for developing estimates which are in turn used to order nets and drive logistics costs. A separate pre‐distribution phase should be considered in which detailed numbers (e.g. population, net need) are established. This may be a better solution to allow an accurate number of nets to be moved and distributed." Kasaï Occidental distribution report, Pg 27.
  • Although often too many nets were transported to an area, sometimes too few nets were sent too. "Nets were deployed notably late in many instances. Only 20% of Health Areas had sufficient nets delivered in a timely fashion. Lesson learned Greater attention is needed to proper organisation and logistics management." Kasaï Occidental distribution report, Pg 33.
  • "Under‐estimation and over‐estimation of the number of households in a number of areas This led to too few or too many nets pre‐positioned and led to a need to a re‐movement of nets." Kasaï Occidental distribution report, Pg 40.
  • We aren’t sure if there were any leftover nets from the 86,500 nets that IMA contributed. It could be that the 45,000 figure includes leftover nets from both IMA and AMF’s contributions.
  • It is our impression that AMF and IMA drastically overestimated the population. We think this because IMA and AMF were under the impression that approximately 28% of sleeping spaces would be covered with a useable LLIN (one that did not need to be replaced). However, it turned out that only 2.3% of the population was covered with a useable LLIN. Thus, AMF and IMA had to cover a much greater proportion of the population than they were expecting (an additional 25% or so), but still had a large number leftover nets. This implies that the population was significantly smaller than expected.
    • "7. The suggestion from IMA pre-distribution was that 28% of sleeping spaces were already covered with nets. That 28% figure was used in estimating the number of nets needing to be purchased. The numbers here suggest that figure turned out to be 2.3%. Is our understanding correct?" AMF DRC West Kasai distribution summary numbers, cell B62.

• 79

  "i) Some householders tried to invent sleeping spaces
  Some householders tried to increase unfairly the number of nets they would receive. Lesson learned The registration process finds out these situations.
  i) Some CHWs tried to divert ties, nails and hammers
  This would be theft for personal benefit and the loss of hammers would be a cost and cause inconvenience to the distribution. Lesson learned Counting equipment and ensuring return, and not assuming return, is important."
  Kasaï Occidental distribution report, Pg 43.

• 80
  • "All distribution activities had to be stopped due to an Ebola outbreak. A period of one month after the last Ebola case was identified had to be identified before the distribution could restart and be completed. The pause was for 8 weeks. Lesson learned Relevance of thinking during the planning phase (in the province more widely) of how plans and schedules can be rearranged if a major delay is necessary in one HZ." Kasaï Occidental distribution report, Pg 43.
  • "In certain cases the CHWs complained there wasn’t an appropriate place (an easily visible place on the front of the household) to mark the household identifier. Lesson learned CHWs were writing the household identifier in too large a handwriting and so were running out of space to write it on the door. They were told to use smaller handwriting." Kasaï Occidental distribution report, Pg 29.
  • "Lack of consistent electricity. This caused delays. Lesson learned Allowances needed in the timetable for potential delays that are caused by elements beyond the control of the team. Develop ‘What if?’ scenarios so if there are problems or delays there is a plan of how to respond and/or decisions can be taken more quickly." Kasaï Occidental distribution report, Pg 37.
  • "The coordination between the two was not as good as it should have been at times leading to gaps in information and understanding of work carried out. Lesson learned Better coordination to ensure the central location is fully aware, and in a timely fashion, of the data from the field." Kasaï Occidental distribution report, Pg 51.

• 81
  • "Initially 430 smartphones were bought but 97 of these proved to be faulty and unusable. In retrospect, the smartphones were not bought form a reliable source. Lesson learned A strong recommendation, to avoid duplicate effort, delays and unnecessary cost, is to ensure the phones are bought from a highly reliable source with each one tested out of country prior to shipment to country. Identify vendors that can deal with bulk purchases." Kasaï Occidental distribution report, Pg 20.
  • "The phones were purchased from three sources. Even with the popularity and wide availability of the Samsung brand, it was difficult to source all the phone models from one source. 70 phones proved to be faulty and unusable. In retrospect, this problem occurred because one of the suppliers was an unproven vendor, and that vendor delivered a faulty batch of phones. The distribution campaign continued with 360 smartphones and it was still possible to finish the distribution in a timely fashion." Kasaï Occidental distribution report, Pg 10.
  • Note that in the previous two quotes, there is disagreement about how many phones were found to be "faulty and unusable." In the first quote, it is claimed that 97 phones were found to be faulty, whereas in the second quote, it is claimed that only 70 phones were found to be faulty. We are not sure of the reason for this discrepancy.
  • "i) Problems with smartphones. The two problems experienced with some phones were: i) they were no longer in good working order and data could not be entered into the phones or the ODK programme would not run; ii) they would not retain energy for beyond the first hour of two of the day’s work. This led to the number of teams working on any one day being reduced and increased the number of days over which the work had to take place. Lesson learned Greater attention needs to be paid to how the smartphones are treated and consideration could be given to the robustness of the phones. Consideration should be given in selecting appropriate phones how they perform in high ambient temperature environments." Kasaï Occidental distribution report, Pgs 33-34.
  • "A mistake was made in using an over‐powerful generator to recharge the phone that led to some burning out due to the excessive voltage used. Lesson learned Attention needs to be paid to the voltage used in charging phones/batteries. This is important information that has been noted for inclusion in all future briefing sessions." Kasaï Occidental distribution report, Pg 34.
  • "Not all of the phones worked all of the time causing delays and inefficiencies and some data loss. Lesson learned It is very important the smartphones are reliable and there are backup phones in case they are needed." Kasaï Occidental distribution report, Pgs 40, 43, 47, 51, and 55.
  • "There were problems charging the phones and spare batteries which led to insufficient battery life during the day and some phones not working. Sometimes this was caused by generators, used for recharging, breaking down. Lesson learned Greater attention needs to be given to ensuring phones work, batteries can be charged so running out of battery power does not occur. Consideration should be given to back‐up phones that can be used when particular phones do not work so the important work to be done and data gathered is not disrupted." Kasaï Occidental distribution report, Pgs 40-41, 44, and 56
  • "iv) Inappropriate equipment used in charging phones
    Rather than this being unsuitable ‘phone chargers’, this problem related to generators being used to generate the electricity from which the phones were charged. Unfortunately ‘constant voltage’ transformers were not used and as a result of the fluctuating/surging charging voltage from the generators, some of the phones (and several laptops) ‘burn out’. Lesson learned Care needs to be taken when charging devices where there are power outages and generators may be need. Constant voltage transformers should be given serious consideration." Kasaï Occidental distribution report, Pg 56.

• 82

  "91% of the nets distributed had data recorded electronically (565,306 nets to 231,925 households, out of total of 624,532 distributed)." Kasaï Occidental technology report, Pg 14.

• 83

  "9% of the nets distributed did not have data recorded electronically due to:

  • Failure of individual smartphones with no back up available for data collection
  • Issues with data being corrupted and not retrievable from the smartphones
  • Issues with the data transfer process leading to data loss"

  Kasaï Occidental technology report, Pg 3.

• 84

  "We are broadly confident the 9% of nets for which we do not have data in electronic form were distributed as intended for the following reasons:

  • Signed net movement forms are reported by IMA to show nets moved to the final distribution points as expected
  • Explanations from IMA as to why data was lost or not collected electronically are credible
  • Paper based records are reported by IMA to corroborate net distribution totals."

  Kasaï Occidental technology report, Pg 3.

• 85

  AMF told us: "Our understanding is 1) some data was only collected in electronic form so its loss or corruption meant there was no back up. Other data was duplicated on paper but the task of going back to individual health centres to collect that data was onerous and IMA decided not to do so as it was considered a significant and onerous task. We discussed this with IMA and in the circumstances decided not to press this further. We did however discuss how systems would change to ensure this could not happen in North Ubnagi and have developed improved protocols and requirements." Rob Mather, email to GiveWell, September 6, 2015.

• 86
  • "96% of the household data records were without error." Kasaï Occidental technology report, Pg 15.
  • "4% of the households had one error or more in the data recorded due to:
    • Data inconsistencies within a household record (these can be designed out with improvements to the logic mechanism i.e. acceptable answers during data entry)
    • Data anomalies within a household record suggesting data entry error (these can be reduced with improved training and/or a verification step during data analysis)
    • No data analysis between data collection and nets being distributed (this can be reduced if the distribution follows a separate registration phase)

    Kasaï Occidental technology report, Pg 3.

  • "An inconsistency error is where two pieces of data are not consistent. For example, 4 sleeping spaces in the household but 5 nets given out; or 3 people in the household but 4 under5s recorded. Usually these are data entry errors. The data entry screens are set up with error messages so as not to allow most inconsistency errors but not all were designed out and some were possible. An anomaly error is where information looks wrong. For example, more than 20 under 5s in a household or all in a household are under 5s. The data are typically outliers against the bulk of the data. These are also likely to be data entry errors." Kasaï Occidental technology report, Pg 15.

• 87
  • 91% * 96% = 87%
  • Note that this not not guarantee that the data is reliable; AMF only checked to the data for inconsistencies and anomalies. This does not rule out the possibility of falsified data.

• 88
  • Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
  • "This distribution is the first that both parties have undertaken using smartphones to collect household-level data. We expect teething problems, as with any new introduction of technology, especially in a country such as DRC. However, we are undeterred by this, as the benefits of improving processes in this region are so significant." AMF blog post, February 18, 2015.

• 89
  • AMF told us that it was waiting to receive data from the Kasaï Occidental distribution before starting the Nord Ubangi distribution, so that AMF could feel sufficiently comfortable moving forward. AMF also wanted to apply the lessons learned from the Kasaï Occidental distribution to the Nord Ubangi distribution. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
  • "We intend to use the same data collection process in the next net distribution, adjusted to apply lessons learned." Pg 3, Kasaï Occidental technology report

• 90
  • "To reach out to every one of the 782 villages individually would have been very timely hence the villages were clustered to reduce the contact points. The clusters were formed with the assistance and identification of the HSAs. The clustering was based on the closeness of the villages and one village was identified to be a meeting point. There were 282 clusters that were formed. The cluster has a village composition ranging from one to five depending on the size and geographic proximity of the villages." Ntcheu 2011-12 distribution report, Pg 5. Also see the diagram on Pg 9.
  • "At a distribution point beneficiaries from villages in the cluster gathered together with their local development and elected leaders. The distribution registers which were in duplicates were called at one village after another and the called beneficiaries lined up to receive their nets. Upon completing distribution one copy was collected for office records and the other register was left with the village leaders." Ntcheu 2011-12 distribution report, Pg 8.

• 91

  See the methodology section above.

• 92
  • "Verification of the long lasting insecticide treated nets registers was done at village level where everyone was present. The community was eagerly listening as team members were calling out the name of the household, the number of people in the household, number of LLIN in that household, number of sleeping spaces in the household in question and finally the number of nets the household was going to receive. Upon completing name calling out, those villagers who did not hear their names being called were registered so that they could be included in the data base. One HSA was assigned to cross check the call out from the printout roll with the raw data and, where the printout differed with the raw data this was noted and later sent back to the data center for data updating. The community also played an important role during the verification in that whenever a name was called out that did not exist in their village, they advised the team members to delete such a beneficiary from the list, which was done after consultation with the village head. The other strength of the roll calling during the verification was that whenever the data that the beneficiary gave on registration was different with what the communities’ perception of the household, the community could raise the issue and the beneficiary could be taken to task to justification. Whenever the community disagreed with the justification, they were advising and recommending the verification team members on the total number of nets the household should get. This improved transparency of the net distribution process." Balaka 2013 distribution report, Pg 6.
  • "Based on GiveWell's suggestion, Concern Universal also decided to send its staff to random households prior to the Ntcheu 2012 distribution to spot-check the accuracy of its data. After the first two weeks of spot checks, the distribution partner reported that it had not found any discrepancies between the number of nets community members said they needed during the verification meeting, and the number of nets found to be needed during the subsequent household visit. AMF has told us that spot checks are now carried out with all distributions. We have not seen results from these spot checks." GiveWell AMF review 2014.

• 93
  • "At the end of each day, the CHWs returned the smartphones to the Field Supervisor and the data was imported into a computer using a WiFi router connection. The Field Supervisor checked the data to make sure the household registrations had been carried out correctly."Kasaï Occidental distribution report, Pg 25
  • "After the distribution had been completed in a Health Area and all data uploaded from the smartphones, the data team compared the electronic data with the paper data collected by each distribution site in each health area." Kasaï Occidental distribution report, Pg 18.

• 94

  Balaka 2013 distribution report, Pgs 3-4. Note that both "useable LLIN" and "sleeping space" are fairly well defined on Pg 4.

• 95
  • We are not entirely clear on how IMA determined the appropriate number of nets to give to each household, although we believe that IMA relied on the number of sleeping spaces in a household. The following quote is in the distribution report: "When assessing the number of sleeping spaces it is important to avoid confusion between the number of separate areas in which individuals might sleep and, given nets distributed are ‘double’ nets that will accommodate two people, the number of sleeping spaces that is equal to the number of nets that are needed. For example, in a household with mum plus dad, four children aged eight, four, six and two and a grandfather there may be six separate sleeping areas: mum plus dad, one each for the four children and one for the grandfather. However, for the purposes of nets, there would be four sleeping spaces: one for mum and dad, one for the two older children, one for the two younger children and one for the grandfather." Kasaï Occidental distribution report, Pg 16. To continue with the example quoted, we believe that this could mean that if the six separate sleeping areas in the household were spread out in such a way that 4 nets could not cover them, the household would still only receive 4 nets, and would have to move its sleeping areas closer together if it wanted to cover everyone. We are not sure how easy it is for a household to move sleeping areas, or how it was decided where to hang nets in cases like this.
  • On the other hand, when we look at the sample household data, it does not seem to support this interpretation. For example, row 9 shows 5 people living in a household, and there are 5 sleeping spaces in the household. Because the household already had 2 useable LLINs, it received 3 from IMA. This seems to imply that even if there was only one person using a sleeping space, IMA would still give a net to cover the whole sleeping space, and not require that the sleeping space be moved close to other sleeping spaces so that two people could be covered with one net. Sample Data-Anonymized sheet, Kasaï Occidental 2014 summary and sample data.

• 96

  See here.

• 97
  • Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
  • Rob Mather, email to GiveWell, June 1, 2015

• 98

  AMF page on Nord Ubangi 2015 distribution.

• 99
  • See our May 2015 update of AMF.
  • "2018: In addition, the NMCP through the LLINs task force is grateful for the offer from the CU/AMF partnership to continue to assist in four districts beyond the 2015 distributions, for the 2018 universal coverage distributions.
    In view of the above the NMCP and LLINs task force is requesting AMF/CU to undertake LLINs mass distribution campaign in four district in 2018, which will again be in line with the NMCP’s schedule and policy." Malawi NMCP invitation for 2018 distribution

• 100

  "AMF and Concern Universal, Malawi (CU) have accepted the Malawi National Malaria Control Programme’s (NMCP) invitation to assume responsibility for the district-wide net distribution re-coverage campaigns in Ntcheu, Balaka, Dedza and Dowa districts in 2018...This timing is consistent with the three year re-coverage cycle needed to keep affected populations protected from malaria-carrying mosquitoes when they sleep at night." AMF blog post, July 3, 2015

• 101

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 102

  "AMF is attempting to finalize two new distribution agreements: one for approximately 2.5 million nets in Country X (AMF cannot publicly share the names of countries it is negotiating with), and one for approximately 5.5 million nets over the next three years in Country Y. We do not have a strong sense of how likely it is that AMF will be able to finalize these distribution agreements." GiveWell AMF update May 2015.

• 103

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 104

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 105

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 106

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 107

  AMF investigates distribution partner candidates by (1) having discussions with the candidate to see how aligned the organization is with AMF’s values and (2) discussing the candidate with other independent organizations that work in the country. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
  

• 108
  • AMF has told us that the total net need for 2015-2017 of the five countries in which it is in discussions is 32 million nets. AMF future distributions net need
  • Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
  • Rob Mather, email to GiveWell, June 1, 2015

• 109

  There are a few reasons for this:

  • AMF believes that having a co-funder for non-net costs is one way to mitigate its risks. AMF’s co-funding partners investigate the distributions they are agreeing to using their own methods, and can learn of (or sometimes already know of) information that impacts the distribution. For example, if AMF cannot find a co-funding partner for a specific country, this may be a warning sign about that particular country. AMF can talk to potential co-funding partners to figure out why they won’t work in a particular country.
  • AMF believes that not paying for non-net costs allows its funds to go further.
  • AMF is not set up in countries where distributions occur. Thus, AMF is not in a position where it can easily deliver on the reporting it requires. This makes AMF hesitant to fund an organization for all net and non-net costs; if the organization ultimately fails to deliver on the reporting that AMF requested, AMF will have a hard time resolving the issue. With a co-funding partner, especially a partner that works in-country, AMF has more assurance that the reporting it requires will be delivered.

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 110

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 111
  • In early June 2015, AMF told us that it had about $22 million in funding, of which $10.2 million was already committed to future distributions ($4.4 for distributions in the near future and $5.8 for the 2018 Malawi distributions). Rob Mather, email to GiveWell, June 1, 2015 and AMF funds history and status
  • In July 2015, AMF’s website showed approximately $21 million in hand (see FY 2016), and approximately $4.1 million had been used for purchases or forward commitments. We believe that AMF did not include its future commitments to the 2018 Malawi distributions in its online financials because the distributions are so far in the future. AMF financial information page
  • $4.1 million + $5.8 million = $9.9 million
  • Note that in February, AMF told us that it had approximately $18 million in hand. So, AMF was able to raise approximately $4 million in several months’ time ($22 million - $18 million = $4 million). We know that GiveWell transferred approximately $2 million to AMF in early March (the transferred funds were donations that GiveWell had received for AMF or for our top charities). AMF told us that another approximately $700,000 came from GiveWell-influenced donors. This implies that AMF received approximately $1.3 million from other donors in March through May. Rob Mather, email to GiveWell, June 19, 2015

• 112
  • Note that the $4.1 committed to future distributions does not include the provisional distributions on the AMF all distributions page. AMF funds history and status
  • Rob Mather, email to GiveWell, June 1, 2015

• 113

  For example, AMF is thinking about using the funds for the 2018 distributions in Malawi sooner than 2018. AMF has told us that if it spent the funds from the 2018 Malawi distributions it would need to re-raise the funds by early 2017. Rob Mather, conversation with GiveWell, February 24, 2015

• 114

  $11.1 million + $5.8 million = $16.9 million

• 115

  AMF future distributions net need

• 116
  • AMF uses $3/net as its rough approximation of net costs. AMF dollars per net page
  • Note that this does not include non-net costs. GiveWell has calculated the total costs for nets to be $5.30 in Malawi and $7.50 in the DRC. GiveWell estimate of AMF cost per net (November 2014)

• 117
  • We discuss case rate data from Ntcheu in our February 2013 update on AMF.
  • "[AMF:] Please confirm you are able to provide monthly malaria case rate data going back at least 12 months for each health centre/clinic in the distribution area and will continue to provide monthly data for a period of four years post‐distribution. This ensures we understand pre‐distribution malaria levels and can monitor them post‐distribution.
    [Concern Universal:] Malaria case rate information for each of the 16 Health Centres is available from the Balaka District Health Office using the Malawi Government’s national Health Management Information System (HMIS). This information will be sent to AMF separately." Balaka 2013 distribution proposal, Pg 2.
  • Dedza 2014 distribution proposal, Pg 2, contains the same language as above

• 118

  Rob Mather, conversation with GiveWell, February 24, 2015

• 119

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 120

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 121

  "AMF primarily distributes nets in 4 of Malawi’s 28 districts. However, AMF told us that for the second and third quarter of 2015, the National Malaria Control Program (NMCP) of Malawi is planning to implement some of AMF’s distribution practices in districts for which AMF does not fund distributions." GiveWell AMF update May 2015

• 122

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015

• 123

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 124

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.
  

• 125

  If a data collector visits a house and asks how many people live there and is told "six people", but there are currently only three at the house, the data collector has no way of knowing whether or not this is true. On the other hand, it is our understanding that it is much more difficult to invent sleeping spaces, which are always in the house. This is why AMF considers it best practice to give out nets according to sleeping spaces, rather than people.
  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 126

  AMF notes that the cost of data collection (several hundred thousand) is very small compared to the cost of the overall distributions (tens of millions). AMF also notes that having the data in electronic form allows one to much more easily check that the summary headline numbers for any given distribution are accurate and not inflated.
  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 127

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 128

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 129

  AMF mentioned that, anecdotally, it has heard that its net hang-up rates are 20-40 percentage points higher than other organizations’ net hang-up rates. These numbers are based on small reviews that other organizations have done and the PDCUs that AMF requires for its own distributions.
  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 130

  The task force is attended by representatives from the NMCP and other large and local organizations working on net distributions in Malawi. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.

• 131

  Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015.