Update on co-funding grant to Population Services International Myanmar - January 2016

This page gives an update on the Containment of Artemisinin Resistance in Eastern Myanmar project, which is being carried out by Population Services International (PSI), was recommended to Good Ventures by the Gates Foundation and is also being funded by the UK's Department for International Development (DFID). Good Ventures contributed to this project as part of an effort to learn from major foundations by co-funding projects with them. We previously published updates on this project in April 2013 and November 2013.

Note: This update was written in July 2015 and reflects the progress of the project through July 2015.

Published January 2016

Project progress

• PSI signed a contract with a second private-sector distributor, PolyGold, in July 2014, and PolyGold began distribution of its brand of ACT, Artel+, in August 2014.1 PSI reported that as of November 2014 it had sold around 1.3 million doses of ACT to its two private-sector partners.2 Sales of ACTs from PSI to distributors slowed in 2013 and 2014, which PSI attributes to declining malaria prevalence.3
• PSI now aims to encourage the use of rapid diagnostic tests (RDTs) for all fever cases before dispensing ACTs.4 PSI completed a pilot project to promote the use of RDTs.5 PSI has told us that it is interested in scaling up this program.6 RDT use is expected to have increased benefits in the context of decreasing malaria rates. In particular, appropriate diagnosis can prevent drug wastage, delay in treatment for other causes of fever, and the emergence of resistance to the non-artemisinin drugs in ACTs.7 Approval for the RDT roll out from the Ministry of Health was granted in February 20158 with the provision that PSI distribute the RDTs to the private sector free-of-charge.9 This means that PSI will not be able to use existing private sector supply chains and so distribution may be less efficient than it has been for ACT.10
• The government of Myanmar has banned the importation of most brands of AMTs, but sales of AMT are not banned and in some places sales have continued. It is unclear whether the AMTs on the market are mainly those imported prior to the ban, whether AMTs are being imported illegally, or whether the ban has failed to cover all brands of AMTs.11
• During the field visit for donors in February 2014, a DFID participant noticed that all ACTs encountered in the market were set to expire in March or April. Donors raised concerns that expired drugs could eventually trickle down to hard-to-reach areas, which led PSI to begin its replacement process slightly earlier than it had planned.12 PSI told us that by the end of April 2014, it had removed 100,000 doses of ACT from the market and that it planned to remove the other 100,000 packs by the end of May.13 We have not followed up to confirm that this happened.
• The project had spent less than it originally projected it would by early 2014, due to a delayed start and to a reduced need for ACTs relative to initial expectations.14 PSI therefore requested and received an 18-month no-cost extension (NCE), allowing it to extend its use of grant funds through March 2016. The project was originally scheduled to end in October 2014. PSI plans to conduct an additional round of surveys during the NCE period.15
• The Greater Mekong Sub-region (GMS; an area encompassing 2.6 million square miles in Cambodia, China, Laos, Myanmar, Thailand, and Vietnam)16 has shifted its strategy from resistance containment to malaria elimination due to concerns that the independent rise of artemisinin resistance in multiple locations indicates that efforts to eliminate the parasite in one region may have a limited impact on the emergence of resistance in neighboring regions. As such, GMS leaders have set a target of eliminating malaria from the region by 2030, and, in connection with that effort, PSI will expand the geographic scope of its own project.17

Monitoring and evaluation

In our introductory report on the project, we outlined the ways PSI planned to monitor and evaluate the project. At that time, PSI planned to collect data on the project from outlet surveys, household surveys, a supply chain assessment, mystery client surveys, exit interviews, focus group discussions, and monthly sales data.

Since our last update, we have seen results from:

• 2014 private outlet survey
• 2013 private outlet survey
• 2013 household survey
• RDT pilot project evaluation
• Mystery client surveys
• Site visit by donors in February 2014, which included visits to 10 drug outlets and 6 informal providers. GiveWell/Good Ventures participated in this visit.

In addition, DFID has commissioned an independent evaluation of the project, which Montrose International began in mid-2013. It has since produced several reports.18

Methodology

Outlet surveys

We have not seen a detailed report on the 2013 outlet survey. PSI told us that the 2013 surveys used the same methodology as the baseline surveys (completed in 2012 and discussed in our November 2013 update) and the 2014 outlet survey (discussed below).19

PSI shared a detailed report on the 2014 outlet survey. This survey appears to have used methods that were broadly similar to those of the baseline survey.20

Townships were randomly selected for inclusion in the sample. Within each township, five urban and five rural areas were randomly selected, and all eligible outlets in those areas were surveyed (eligible outlets had malaria blood testing available or reported that they had had antimalarials in stock within the prior three months). The survey involved a provider interview and a structured questionnaire used to record information about all antimalarials and RDTs present at the outlet.21 Providers reported how many antimalarials and RDTs they had sold or distributed in the preceding week.22

The outlet surveys cover outlets in "intervention" and "comparison" sites: both are supplied with ACT by PSI's distributors, but the comparison sites do not receive PSI's promotional activities meant to promote uptake of ACT.23 The intervention areas are townships located along the eastern part of the country bordering with China and Thailand, and include the priority area for national artemisinin resistance containment efforts. The comparison area is near the intervention townships. We do not know how, specifically, the comparison townships were selected.24

In the results section below, we present data from both intervention and comparison groups. We find the change over time in the outlets surveyed more meaningful than differences between intervention and comparison groups. This is because (a) we are more interested in the impact of the program as a whole than the impact of the promotion component of the program, and (b) it is unclear whether the comparison townships are a suitable control group.

Household survey

Our November 2013 update noted that most of the people surveyed in the baseline household survey were unable to identify which type of medication they used during their most recent fever episode. The follow up household survey, completed in 2013, found that individuals were able to remember which medication they had used at much higher rates than in the baseline survey.25 It is unclear to us why the second survey was more successful than the first, and this raises concerns for us about the reliability of these results and their comparability to the baseline results. For that reason, we have not yet analyzed the results.

RDT pilot evaluation

PSI shared details of the methodology used to evaluate the RDT pilot project.26 Six townships were selected for the project and divided into three groups.27 In all three groups, outlets received subsidized RDTs. In group one, this is all they received. In group two, outlets received the subsidized RDTs plus financial incentives to use RDTs in the form of free ACTs or RDTs. In group three, providers received the subsidized RDTs plus intensive support visits involving education and counseling to encourage RDT use.28

Household surveys, mystery client surveys, data from RDTs and from stock resupply, and qualitative interviews with providers and staff were used to assess the project.29 For the household surveys, baseline and follow up surveys were conducted and all households that had had a member with a fever in the preceding three weeks who had taken a malaria drug or had symptoms consistent with malaria were included.30 Mystery client data was collected from randomly selected providers.31 Trained mystery clients visited outlets and said that they had a fever that they thought was like a malaria fever that they had had on a previous occasion. A researcher accompanied each mystery client, posing as a friend, and, after leaving the outlet, completed a survey form on the provider's response.32

Mystery client surveys

In addition to the mystery client surveys conducted as part of the RDT evaluation, PSI has also provided some results from mystery client surveys that aimed to assess whether outlets sell full or partial courses of ACTs and provide clients with instructions for correct use.33 We have not seen the methodology or detailed results from these surveys.

Site visit

We summarize our February 2014 visit to the program in this blog post. Full notes and photos from our visit are available here.

Results

According to these reports:

• Availability of ACTs has increased substantially. In 2012, the Outlet Survey audit found Quality Assured ACT (QAACT) in only 24% of antimalarial-stocking outlets surveyed in the intervention areas. That increased to 70% in 2013 and 80% in 2014. In the comparison areas, QAACT was found at 30% of antimalarial-stocking outlets surveyed in 2012, 46% in 2013, and 50% in 2014.34 Meanwhile, oral AMT was found in 54% of antimalarial-stocking outlets surveyed in the intervention areas in 2012, falling to 37% in 2013 and 10% in 2014. For antimalarial stocking outlets in the comparison areas, oral AMT was found in 50% in 2012, 31% in 2013, and 24% in 2014.35
• Subsidy is largely passed on to consumers. It appears that the subsidy has been successful at keeping the price of ACT similar to that of AMT: the 2013 and 2014 outlet surveys found that, of surveyed outlets selling Supa Arte 4 (the QAACT PSI distributes through AA Medical Products),36 between 79% and 90% of outlets in both the intervention and comparison areas were selling it for less than 500 kyat,37 which PSI reports is the cost of one typical dose of AMT.38
• ACT market share has increased; AMT market share has decreased but AMT remains available. In each outlet survey, outlets were asked to report how much of each type of antimalarial they sold or distributed in the preceding week. The reported market share of QAACT as a percentage of antimalarials sold or distributed in the interventions zones was 18% in 2012, 62% in 2013, and 53% in 2014, while in the comparison zones it was 23% in 2012, 53% in 2013, and 23% in 2014. Meanwhile, the reported market share of oral AMT in intervention zones was 35% in 2012, 14% in 2013, and 12% in 2014, and in the comparison zones was 34% in 2012, 25% in 2013, and 31% in 2014.39
• Other antimalarials are frequently used. The 2014 outlet survey found that non-artemisinin antimalarials make up between 30% and 40% of the antimalarial market based on reported sales by outlets in the week preceding the survey.40 It is unclear whether these are being prescribed appropriately – while they are the correct course of treatment for one type of malaria, in cases where RDTs are not used, providers may be prescribing them inappropriately in place of ACTs.41
• Knowledge among sellers has increased but remains fairly low. The outlet surveys report the percentage of antimalarial providers that were able to correctly report the most effective antimalarial when asked. In 2012, 18% in the intervention zones and 24% in the control zones were able to respond correctly, while in 2013, 42% of intervention zone outlets and 32% of control zone outlets were able to do so, and in 2014, 50% of intervention zone outlets and 47% of control zone outlets were able to do so.42
• RDT pilot found modest gains in RDT use. The household survey conducted to evaluate the RDT pilot found that financial incentives increased the percentage of fever patients who reported using an RDT from 2.7% at baseline to 11.9%, and that education and counseling increased the percentage from 5.4% at baseline to 13% in the townships studied, while RDT usage in the townships with just the subsidy increased from 3% at baseline to 6.4% after the roll-out.43 The "mystery client" survey that PSI conducted at a random sample of outlets in the three intervention arms44 found that 65% of providers proposed performing an RDT test for malaria when a "mystery client" reported that s/he had malaria-like symptoms.45 . 40% of providers both proposed performing an RDT test and also performed all 5 involved steps correctly.46
• Results from mystery client surveys are difficult to interpret. PSI reports that the year 3 mystery client survey found that only 10% of priority outlet providers prescribed a mystery client with malaria-like symptoms a full course of ACT and provided instructions for correct use, down from 29% in the year 2 survey. PSI hypothesizes that this may be due to the fact that malaria is becoming less common and so providers may be less likely to assume that a fever indicates malaria or to stock ACTs.47 We have not seen additional results or methodology from the mystery client surveys, and do not know how to reconcile the low percentage of providers that apparently distribute full courses of ACT with the higher market share of ACT. One possibility is that providers may be selling partial courses of ACTs; we have not seen data on how many providers sold mystery clients partial doses.
• During the 2014 site visit, DFID expressed a "very positive" assessment of the project’s progress.48

Risks to the success of the project

In our initial report on the project, we discussed several risks that we believed could prevent PSI from executing the project as planned. Here we provide updates on those initial risks:

• Targeting of subsidies: In our initial report on the project, we noted the possibility that PSI’s price subsidy could be captured by suppliers or retailers instead of reaching consumers. As noted above, it does not appear that this is occurring.
• Patient compliance with full treatment course of ACT: We noted initial concerns that patients might not purchase or complete a full course of ACT. The 2013 household survey results indicate that a substantial percentage of patients with fevers either buy or consume a partial ACT dose,49 although as discussed above we have concerns about the reliability of the data from this survey.
• Partnership with large supplier: We stated in our original report that a successful relationship with AA Medical Products would be crucial for the success of the project. To date, we are not aware of PSI having any difficulties working with AA, and, as discussed above, PSI has entered into a partnership with a second supplier, which will mitigate this risk further.
• Inherent challenges of behavior change: In our initial report, we noted that changing behavior through public advocacy seemed inherently difficult. We noted, however, that PSI sought to change behavior using both advocacy and other tools: in particular, a ban on existing AMTs and a subsidy for the replacement. As discussed above, while ACT seems to have gained market share relative to AMT, it remains the case that many providers do not prescribe it or use it correctly.

In addition, there were several risks we did not foresee. Issues that PSI has encountered which we did not note in our first report include:

• Unanticipated complexity of banning AMTs: As we noted in a previous update, PSI first believed that the government had banned AMTs and then discovered that only one brand of AMT had been banned. PSI's advocacy to the government led to a second brand being banned. It remains unclear whether all AMTs have been banned.50
• Supply chain management under conditions of limited information: Because of declining malaria rates and a lack of information about the initial state of the market, PSI’s first order of ACTs was larger than necessary. This led to drugs expiring before they could be sold and PSI needing to replace drugs throughout the supply chain.
• Conflict-affected areas: Parts of Eastern Myanmar are inaccessible to PSI’s program and evaluation due to ongoing conflict within the regions. PSI believes that this area has high malaria rates.51 It is possible that ACT resistance could continue to develop and spread from these areas even if the program is successful elsewhere. Some anecdotal evidence indicates that these regions are receiving some subsidized ACTs.52

Sources

Document	Source
Asia Development Bank Overview of the Greater Mekong Subregion	Source (archive)
Chris White, email to GiveWell, April 28, 2014	Unpublished
Chris White, email to GiveWell, May 27, 2014	Unpublished
DFID Annual Review (February 2014)	Source
GiveWell PSI Myanmar site visit notes 2014	Source
GiveWell Reflections on a Site Visit in Myanmar (Burma)	Source
Henrietta Allen, email to GiveWell, September 25, 2015	Unpublished
Montrose Independent Evaluation Case Study 2 Private Providers (June 2014)	Source
Montrose Independent Evaluation Inception Report and Evaluation Framework (August 2013)	Source
Montrose International Project Evaluation Report (June 2014)	Source
PSI AMTR Baseline Household Survey	Source
PSI AMTR Baseline Outlet Survey	Source
PSI Myanmar 2014 Outlet Survey Report	Source
PSI progress report (March to September 2013)	Source
PSI Progress Report (Oct 2014 to March 2015)	Source
PSI Year 3 Progress Report (Oct 2013 to Sept 2014)	Source
RDT Pilot Phase 1 Final Report (PSI and UCSF)	Source
Updated Risk Matrix (June 2015)	Source

• 1

  “A distribution contract with the second largest artemisinin monotherapy distributor, Poly-Gold, was signed in July 2014. This will help increase market penetration and availability of QAACT in the private sector and reduce monopolistic dominance of the first distributor, AA. A new brand of ACT, Artel+, distributed by PolyGold debuted in August.” PSI Year 3 Progress Report (Oct 2013 to Sept 2014), Pg 5.

• 2

  "As of November 2014, more than 1.3 million QAACTs have been sold to the two licensed distributors and more than 3,000 drug shops located in 247 townships. This total does not include the more than 10,000 informal outlets in rural areas reached by PSI’s product promoter team, many of which now also stock QAACTs." PSI Year 3 Progress Report (Oct 2013 to Sept 2014), Pg 8.
  

• 3

  "There has been a significant decline in sales/distribution of QAACTs over the past 2 years (2013 and 2014), due mainly to declining malaria prevalence, itself a result of intensive partners’ efforts and donor support over the past several years. (see Figure 3) As previously documented, this decline has had major implications on the project in terms of commodity procurement and expired ACT and RDT stocks. It will also have a major impact on commodity projections as well as improved case management using RDTs to avoid drug wastage. Improved case management will be critical to the next phase of the project to minimize resistance pressure and maximize value for money." PSI Year 3 Progress Report (Oct 2013 to Sept 2014), Pg 9.

• 4

  "In the first phase of the project the primary objective was to rapidly replace oAMT drugs. Toward this end, PSI developed the Padonmar quality seal, an identifier used for all QAACT. All communications campaigns and efforts were centered on this quality seal, aiming to increase the awareness of and demand for QAACTs in the private sector among the target population. However, in response to rapidly declining malaria prevalence, PSI’s strategy has evolved to focus on improved case management using malaria RDT testing for all fever cases before dispensing QAACTs. This will not only help improve value for money by increasing the rationalized use of QAACTs, the main cost driver of the project, it will also reduce the likelihood of development of parasite resistance to the partner drug." PSI Year 3 Progress Report (Oct 2013 to Sept 2014), Pg 12.

• 5

  RDT Pilot Phase 1 Final Report (PSI and UCSF).

• 6

  Chris White, email to GiveWell, May 27, 2014.

• 7

  “RDT positivity rates will undoubtedly continue to decline. Against this shifting epidemiological landscape, it will be imperative that the private sector is supported in adopting diagnostic tools to ameliorate the threat of drug wastage, inappropriate fever management and, critically in the context of containment, drug resistance emerging to partner drugs used in combination with artemisinin.” PSI progress report (March to September 2013), Pg 9.

• 8

  "PSI re-submitted the RDT scale up plan in Jan 2015 addressing the MOH’s concerns and approval was obtained in late Feb 2015. PSI is working closely with the Ministry of Health to determine exactly what activities/components of the plan are approved/disapproved and to what extent." PSI Progress Report (Oct 2014 to March 2015), Pg 5.

• 9

  "MOH approved RDT scale under the condition that RDT will be distributed into private sector free. The efficiency of RDT distribution in this model will not be comparable to the distribution through the distributors’ private sector supply chain. Therefore, PSI has carry out a careful assessment on the current resources and additional HR needed, territory management plan and the potential outlets for RDT, the anticipated challenges and how to overcome this etc to ensure successful scale up." Updated Risk Matrix (June 2015), Pg 4.

• 10

  "Since RDT distribution was recommended to be free, PSI will not be able to use existing distributors’ supply chain due to lack of incentives at all level of supply chain. Acknowledging that this is not the best scenario that PSI hopes for, and the speed and efficiency of the RDT scale up will not be as comparable to what was done with QAACT distribution, PSI has carried out an operational planning meeting internally with for key persons involved. Anticipated challenges and risks associated, actions/strategies to overcome/mitigate this, and PSI’s sales team’s existing coverage and HR, and the additional resources needed for RDT scale up were discussed. Based on the discussions, the distribution model was developed to be shared with donors by July 2015." PSI Progress Report (Oct 2014 to March 2015), Pg 12.

• 11

  “The imports of two AMTs, artesunate and artemether, were banned in Burma in December 2011 and August 2012 respectively by the Food and Drug Agency (FDA). Stocks of these residual AMTs are still being sold, as there is not a ban on sales of AMTs. Most concerning however is that another AMT distributed by Liberty, appears not to have had its importation banned by the agency, and is being seen within markets.” DFID Annual Review (February 2014), Pg 14.

• 12

  “At the time of the site visit, PSI was waiting to begin selling a new batch of drugs, and replacing expired drugs in the market with new drugs, to minimize wastage. The issue surfaced during the field visit when a DFID participant noticed that all the ACTs we encountered in the market were set to expire in March or April. Donors raised the concern that PSI may be waiting too long to replace the drugs and that expired drugs may trickle down to harder-to-reach areas as a result. This prompted a conversation that led PSI to begin the replacement process slightly earlier than planned, after obtaining approval from the donors.” GiveWell Reflections on a Site Visit in Myanmar (Burma).

• 13

  Chris White, email to GiveWell, April 28, 2014.

• 14

  “The project is under spent due to some extent of the delayed start, but also due to the lower than expected requirement of ACTs (which is one of the main cost drivers).” DFID Annual Review (February 2014), Pg 16.

• 15

  “In May of 2014 DFID informed the evaluation team of a no-cost extension of the AMTR project from October 2014 to March 2016. Based on information obtained from PSI this will imply an additional round of surveys in 2015 with the last data needed for the final evaluation of the project becoming available in approximately October 2015.” Montrose International Project Evaluation Report (June 2014), Pg 5.

• 16

  Asia Development Bank Overview of the Greater Mekong Subregion.

• 17

  "The findings from the recent resistant gene mapping studies that the association of mutations in a kelch protein on chromosome 13 (K13) with the resistance and the independent emergence of K13 mutations in multiple geographic locations suggests that elimination of the parasite from the entire GMS region will be the only way to stop the spread of resistance malaria towards Africa. Efforts to eliminate artemisinin-resistant malarial parasites in one region may have a limited impact on the emergence of resistance in neighboring regions, particularly if selection pressure for resistance is sustained by continued availability of artemisinin monotherapies and counterfeit or substandard medicines. Therefore, the entire Greater Mekong Sub-region (GMS) has shifted its strategy from resistance containment to malaria elimination. All GMS political leaders have endorsed this agenda and agreed to eliminate malaria from the region by 2030. This has implication on the AMTR project. PSI discussed the urgent needs for project area expansion outside current AMTR project townships, especially towards the western border of Myanmar. PSI requested and received donor’s approval during annual donor review visit to expand the geographic scope of the project. Moreover, to contribute to the regional goal of malaria elimination, PSI will revamp its malaria case management strategy in the private sector to ensure utilization of RDT test results and completion of the treatment course." PSI Progress Report (Oct 2014 to March 2015), Pg 6.

• 18

  "In May 2013, DFID contracted an independent evaluation of the project ‘Replacement of malaria monotherapy drugs in the private sector to support the containment of drug resistant malaria in eastern Burma’ shortened to the ‘Artemisinin Monotherapy Replacement’ Project (AMTR). The project started in March 2012 and was originally to end in December 2014, but has now been given an 18 month no cost extension. The independent evaluation team led by Montrose conducted an inception review of the project in two phases, respectively in June and July 2013. Building on the findings from these two field visits, and further discussions with members of the evaluation steering committee, the independent evaluation team identified several thematic issues for case studies and working papers which are intended to highlight and present specific aspects of the evaluation to broader audiences, and complement the monitoring and evaluation work of PSI." Montrose Independent Evaluation Case Study 2 Private Providers (June 2014), Pg 8.

  PSI has shared with us Montrose Independent Evaluation Inception Report and Evaluation Framework (August 2013) and Montrose International Project Evaluation Report (June 2014).

• 19

  Henrietta Allen, email to GiveWell, September 25, 2015.

• 20

  See PSI Myanmar 2014 Outlet Survey Report, Pg 49 and PSI AMTR Baseline Outlet Survey, Pg 16.

• 21

  "A full description of the research design and methods is provided in Annex 2. Briefly, a representative sample of townships was selected in intervention and comparison areas (see sampled townships in Annex 3). Within sampled townships, 5 urban wards and 5 rural village tracts were selected using simple random sampling. Within wards and village tracts, a census of all outlets with the potential to sell or distribute antimalarials and/or provide malaria blood testing was completed. Government health facilities were not included in the study as the AMTR project was designed to replace oral AMT in the private sector. Outlets were screened to determine eligibility. Outlets eligible for the survey met at least one of three criteria: 1) one or more antimalarials were in stock on the day of the survey; 2) one or more antimalarials were in stock in the three months preceding the survey; and/or 3) malaria blood testing (microscopy or RDT) was available. The results of the census are summarized in Figure 1. A structured questionnaire was used to complete an audit of all antimalarials and RDTs as well as a provider interview (see Annex 4). See Annex 5 for detailed summaries of antimalarials and RDTs audited. Data collection was paper-based and the data were entered using CSPro. All data cleaning and analysis was performed using Stata 12.1 (©StataCorp, College Station, TX). Data were weighted to account for variation in probability of outlet selection (see Annex 6), and standard error calculation reflected clustering of outlets at commune and district levels. Standard indicators were constructed according to definitions applied across ACTwatch project countries (see Annex 7)." PSI Myanmar 2014 Outlet Survey Report, Pg 10.
  

• 22

  "Antimalarial audit information included formulation, package size, brand name, active ingredients and strengths, manufacturer, country of manufacture, reported sale/distribution in the week preceding the survey, retail price, and wholesale price. RDT audit information included brand name, manufacturer, country of manufacture, reported sale/distribution in the week preceding the survey, retail price, and wholesale price." PSI Myanmar 2014 Outlet Survey Report, Pg 51.

• 23

  "A counterfactual will be directly available for the promotional activities of PSI at outlet levels as the design of the outlet survey includes intervention (PSI project area) and comparison sites (sample of outlets from tier 3 outside the PSI project area). This comparison will be between outlets that are supplied with QA-ACT by AA Medical Products and any other distributor PSI may enter an agreement with during the project, but without the promotional activities of the PSI product promoters (see Figure 4), and those outlets that have been “primed” by PSI to stimulate the uptake of QA-ACT and later RDT. This design, therefore, allows the assessment of trends over time, as well as the effects of the promotional PSI activities, but must take into account as one of the potential confounders that the actual level of malaria transmission, and hence demand for malaria treatments may be lower in some of the comparison areas." Montrose Independent Evaluation Inception Report and Evaluation Framework (August 2013), Pg 14.

• 24

  "Intervention areas are target project townships located along the eastern part of the country bordering with China and Thailand, including the resistance containment Tier 1 zone—the Myanmar Artemisinin Containment Project (MARC) area. The MARC area is targeted by national partners for implementation of interventions to identify, track, and address the spread of artemisinin drug resistance. A comparison area sample was drawn from locations in proximity to the project townships." PSI Myanmar 2014 Outlet Survey Report, Pg 10.

• 25

  "The key results of the 2013 household survey are shown in Figure 7. While in 2012 none of the fever cases had received an ACT, 61% did in 2013." Montrose International Project Evaluation Report (June 2014), Pg 10.

• 26

  RDT pilot final report: RDT Pilot Phase 1 Final Report (PSI and UCSF).

• 27

  "We included six townships in the study, with two townships per arm. We purposefully selected the townships to include areas where PSI RDT supplies were available among pharmacists and drug vendors, and areas with similar risk of malaria. The selected townships were similar in terms of socioeconomic status, level of migration, access to roads, population size, male-­‐to-­‐female ratio, and presence of health centers." RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 11.

• 28

  "Arm 1

  • RDT resupply at approximately $0.18/RDT, upon receipt of a box of used RDTs

  Arm 2

  • RDT resupply at approximately $0.18/RDT, upon receipt of a box of used RDTs
  • ‘Financial’ incentive in the form of ‘free/promotional’ ACT AETD1 or RDT kits, or every 5 RDTs purchased at resupply

  Arm 2 [sic]

  • RDT resupply at approximately $0.18/RDT, upon receipt of a box of used RDTs
  • Monthly intensive support visit to all target outlets (with one-­‐on-­‐one discussion, behavioral change communication (BCC), information, education, and communication (IEC), provision of materials, and visits to recent patients)"

  RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 9

• 29

  "The 18‐month study included the roll-out of RDTs and two provider incentive interventions to six townships in Myanmar. RDT roll-out was assessed over the course of four months. We used the following methods to assess the aims and objectives: 1) household surveys; 2) mystery clients; 3) RDT-­‐derived data; 4) stock resupply data; and 5) qualitative interviews with providers and Phase One operations and management staff." RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 11.

• 30

  “We conducted pre- and post- roll-out household surveys in the six townships to gauge the change in RDT use in the target population. We collected baseline data one month after the introduction of RDTs in all townships for all 3 Arms. One month later, we rolled out the interventions of IEC (Arm 3) and provider financial incentives (Arm 2) in two townships each in four communities. At the end of the four months, we conducted a follow-up household survey to assess differences in RDT uptake by intervention arm (Aim 2). To reduce potential confounding, the interventions were implemented in townships matched on the community-level characteristics of population size, male-to-female gender ratio, geographic area, and socioeconomic status. For survey implementation, we enumerated and screened all households in the selected townships for inclusion. Inclusion criteria were: 1) having a member of the household who had a fever in the last three weeks, had either taken an antimalarial drug, or who had symptoms consistent with malaria (24); and 2) living in an area where PSI supplies RDTs in pharmacies and drug shops. Once a household was determined eligible for the study, the head of the household answered questions on basic demographic characteristics (including household socioeconomic status), recent episodes of fever among household members, antimalarial drug use, RDT knowledge and use, RDT test results if applicable, and where they obtained the RDT. The main outcome of interest from the household survey data was the proportion of RDT use (defined as the proportion of RDT use per population treated for malaria or fever in the past 2 weeks). In total, we completed 832 household surveys with fever cases. All analyses were carried out in Stata 12MP and weighted by population." RDT Pilot Phase 1 Final Report (PSI and UCSF), Pgs 11-12.

• 31

  “We used stratified random sampling to select provider participants for the mystery client assessment, with stratification by intervention arm and provider type (GRS, IDV, MDR). From the list of all 631 providers (Table 1), we randomly selected 20 providers of each provider type and intervention arm to be enrolled, with the exception of the MDRs where fewer than 20 per cell were to be enrolled. The final study sample included 171 providers as shown in Table 2.” RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 12.

• 32

  "Mystery clients received intensive training by study personnel on the clinical scenario and completing the assessment form. The mystery client assessments were conducted between 26 August and 1 September 2013. The mystery client presented at the outlets saying that s/he had fever that s/he thought was like a malaria fever that s/he had on a previous occasion. The provider could then propose a RDT on site, not propose a RDT at all, or propose a RDT at a different location. If the provider did not automatically propose a RDT at his or her own shop, the mystery client was trained to ask for a RDT [see Appendix 1]. The researcher accompanied the mystery client to the outlet as a friend from town, but did not speak or engage with the provider. The researcher observed everything the outlet provider did and completed the record form after leaving the outlet." RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 13.

• 33

  "The goal for the percentage of target outlet providers who prescribe a "mystery client" with suspected malaria a full course of ACT, including providing instructions for correct use for year 3 was 40%. The project achieved only 10.4% on this goal, according to the mystery client survey. This represents a decline from 29% achieved in the year 2 mystery client survey." PSI Year 3 Progress Report (Oct 2013 to Sept 2014), Pg 11.

• 34

  PSI Myanmar 2014 Outlet Survey Report, Pg 37.

• 35

  PSI Myanmar 2014 Outlet Survey Report, Pg 38.

• 36

  "PSI sells subsidised quality assured ACTs (co-formulated artemether lumefantrine), branded as Supa-Arte and Artel Plus, through two major private drug distributors in Myanmar, AA Medical Products Ltd and PolyGold." PSI Myanmar 2014 Outlet Survey Report, Pg 48.

• 37

  PSI Myanmar 2014 Outlet Survey Report, Pg 40.

• 38

  "In year 2 90.9% of priority outlets sold Supa Arte 4 for 500 mmk or less, the cost of one typical dose of artemisinin monotherapy." PSI Year 3 Progress Report (Oct 2013 to Sept 2014), Pg 8.

• 39

  PSI Myanmar 2014 Outlet Survey Report, Pg 43.

• 40

  29.9% in intervention zones, 40.6% in comparison zones. PSI Myanmar 2014 Outlet Survey Report, Pg 43.

• 41

  “Somewhat worrying, however, is the continued high sale of non-artemisinin medicines, representing a total share of 45%. One can argue that these are needed for the P. vivax infections, but this would be correct only if the level of parasitological diagnosis is high. But RDT availability did not change in the priority outlets from 2012 to 2013, remaining very low, at 6.0% and 6.5% respectively, and higher but also unchanged in the private clinics and among CHW at 61.2% and 58.3% respectively.” Montrose International Project Evaluation Report (June 2014), Pg 7.

• 42

  PSI Myanmar 2014 Outlet Survey Report, Pg 46.

• 43

  RDT Pilot Phase 1 Final Report (PSI and UCSF), Figure 1, Pg 15.

• 44

  “We enrolled 631 owners or workers at private outlets who PSI had trained on providing RDT services across the six townships in the Mon and Shan States. Table 1 shows the number of provider outlets included in the sampling frame of the study. In total, we trained 399 GRS owners/workers, 177 IDVs, and 55 MDRs on using RDTs. Sampling Strategy We used stratified random sampling to select provider participants for the mystery client assessment, with stratification by intervention arm and provider type (GRS, IDV, MDR). From the list of all 631 providers (Table 1), we randomly selected 20 providers of each provider type and intervention arm to be enrolled, with the exception of the MDRs where fewer than 20 per cell were to be enrolled. The final study sample included 171 providers as shown in Table 2." RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 12.

• 45
  • "Mystery clients received intensive training by study personnel on the clinical scenario and completing the assessment form. The mystery client assessments were conducted between 26 August and 1 September 2013. The mystery client presented at the outlets saying that s/he had fever that s/he thought was like a malaria fever that s/he had on a previous occasion. The provider could then propose a RDT on site, not propose a RDT at all, or propose a RDT at a different location. If the provider did not automatically propose a RDT at his or her own shop, the mystery client was trained to ask for a RDT [see Appendix 1]. The researcher accompanied the mystery client to the outlet as a friend from town, but did not speak or engage with the provider. The researcher observed everything the outlet provider did and completed the record form after leaving the outlet. All data were analysed in Stata MP12, stratified by provider type and intervention arm. Results were calculated unweighted, except for the overall score, which was calculated as weighted average." RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 13.
  • ”Overall, 65% of providers proposed a RDT without prompting.” RDT Pilot Phase 1 Final Report (PSI and UCSF), Pg 16.

• 46

  RDT Pilot Phase 1 Final Report (PSI and UCSF), Figure 2, Pg 16.

• 47

  "The goal for the percentage of target outlet providers who prescribe a 'mystery client' with suspected malaria a full course of ACT, including providing instructions for correct use for year 3 was 40%. The project achieved only 10.4% on this goal, according to the mystery client survey. This represents a decline from 29% achieved in the year 2 mystery client survey. There are several potential explanations for the results and the decline: 1) in conditions of rapidly declining malaria prevalence, providers are adjusting by becoming less likely to dispense an antimalarial to a person presenting with fever; 2) antimalarial medicines are becoming slower moving inventory for private sector providers, which makes then less inclined to stock them; and 3) the assumption that a client presenting with malaria-like symptoms has malaria and should be given a QAACT no longer pertains (Sun Quality Health providers had an RDT positive rate of 6% in 2014; this is strong rationale for approval of private sector RDT scale-up)." PSI Year 3 Progress Report (Oct 2013 to Sept 2014), Pg 11

• 48

  "At the end of our visit, DFID staff provided PSI with both overall feedback on how they believed the project was going ('a very positive review')..." GiveWell PSI Myanmar site visit notes 2014.

• 49

  77% of those who reported receiving ACTs said they received a full dose (47% of all respondents with fever), and 35.2% of those who reported receiving a full dose (16.5% of all respondents with fever) said they completed it. Montrose International Project Evaluation Report (June 2014), Figure 7, Pg 10.

• 50

  “The imports of two AMTs, artesunate and artemether, were banned in Burma in December 2011 and August 2012 respectively by the Food and Drug Agency (FDA). Stocks of these residual AMTs are still being sold, as there is not a ban on sales of AMTs. Most concerning however is that another AMT distributed by Liberty, appears not to have had its importation banned by the agency, and is being seen within markets.” DFID Annual Review (February 2014), Pg 14.

• 51

  “The geographical area of eastern Myanmar with the highest burden of disease includes areas of the country that are hard to reach and often politically sensitive… 51 townships (housing approximately 38% of the population in these states) are inaccessible due to security reasons. No program activity takes place in these areas, and they were excluded from the study.” PSI AMTR Baseline Household Survey, Pg 18.

• 52

  "There is some anecdotal evidence that Supa-Arte reaches conflict affected areas which have a high burden of drug resistance.” DFID Annual Review (February 2014), Pg 15.