Deworm the World Initiative, Led by Evidence Action: Supplementary Information - January 2017 Version

We have published a more recent supplementary information page. See our most recent supplementary information page for Evidence Action's Deworm the World Initiative.

This page contains further discussion and information for our review of Deworm the World. This page is intended to provide supplementary information on topics covered in the main review and is not intended to be read independently of the main review. The information on this page is less frequently updated than our main review; there is a note at the top of each section indicating when it was last updated.

Description of programs by country

Section last updated: November 2016

As of September 2016, Deworm the World had supported deworming programs in India, Kenya, Ethiopia, Vietnam, and Nigeria, and it had started support (or was in discussions about starting support) in two other countries (Pakistan and Indonesia).1 Several of these programs started in 2015 or 2016; as of early 2015, Deworm the World had only supported deworming programs in Kenya and India. Each country has or will have a slightly different program model; we expand upon the programs by country below and have summarized some high-level details in this spreadsheet.

India

Deworm the World started working in India in 2009.2 It has assisted at both the state level and the national level with all areas mentioned on the main review page.3 Deworm the World has received funding for its work in India from the Children's Investment Fund Foundation (CIFF), the United States Agency for International Development (USAID), Dubai Cares, the END Fund, and the Michael & Susan Dell Foundation (past).4

In 2014, following advocacy from Deworm the World, the national government decided to implement "National Deworming Day," a single-day school and preschool-based deworming program targeting all at-risk children aged 1-19 for soil-transmitted helminthiasis (STH) treatment.5 The first National Deworming Day was held in February 2015, with 11 states participating that day and one state postponing until April (five of which received direct support from Deworm the World).6 A second National Deworming Day was held in February 2016 in 30 of India's 36 states and union territories; eight of the participating states received support from Deworm the World.7 India held a third National Deworming Day in August 2016 in 26 states with prevalence rates meriting twice-per-year treatments; Deworm the World supported five of these states.8 Deworm the World is now seeking additional funding to expand its work to several additional states (more on the main review page).

One state that Deworm the World previously supported—Delhi—has transitioned to implementing its deworming program with very limited technical support; Deworm the World expects to reduce support to other states as their capacity to operate their programs without Deworm the World's assistance increases over the next few years.9

Kenya

Deworm the World started to work with the national government in Kenya in 2009, supporting a pilot school-based deworming program.10 In 2010, Deworm the World agreed to support Kenya's National School-Based Deworming Program (NSBDP), which started in 2012 under a 5-year plan funded by CIFF and the END Fund.11 Deworm the World has entered its fifth year of work in Kenya under this agreement (2016-2017) and is seeking funding to continue supporting the Kenya program (more on the main review page).12

Deworm the World provides assistance in all of the other areas we described on the main review page.13 Most of Kenya was mapped before Deworm the World started to work there, and the Kenya Medical Research Institute (KEMRI) is funded by CIFF to conduct independent prevalence surveys (with limited sample sizes) before and after each round of deworming.14 Deworm the World has received funding from the END Fund to support treatments for lymphatic filariasis in Kenya, starting in 2017.15

Similar to in India, the program in Kenya has moved towards a model in which deworming occurs on consolidated Deworming Days.16

Currently, Deworm the World has a robust staff presence in Kenya, in part because Deworm the World implements monitoring activities in Kenya itself (rather than hiring consulting firms to implement this) and acts as the fiscal agent for the program.17 The Kenya team occasionally supports and trains staff for Deworm the World's newer programs.18

Ethiopia

In Ethiopia, Deworm the World partners with the Schistosomiasis Control Initiative (SCI)—another of GiveWell's recommended charities—to advise the national government.19 Deworm the World started advising Ethiopia in late 2014, after Ethiopian government officials visited Kenya for a learning exchange and spoke with Deworm the World representatives during the trip.20

In early 2015, Ethiopia launched a national school-based deworming program.21 Deworm the World supported the government in implementing a pilot mass drug administration (MDA) in mid-2015 and the national program's first full MDA in late 2015.22 Deworm the World's work on the pilot was funded in part by its unrestricted funding.23 In mid-2015, CIFF and the END Fund decided to fund a five-year national school-based deworming program in Ethiopia.24

Deworm the World has told us that its role in Ethiopia is similar to its role in Kenya and India (though it has less of a staff presence), while SCI assists with activities that are more scientifically technical, such as conducting prevalence surveys.25

Pakistan

Deworm the World first began to explore the possibility of working in Pakistan in 2015, for two reasons:

• Pakistan is believed to have a high worm burden across a large population, and there is currently no mass treatment of school-age children for STH in Pakistan.26
• An organization that Evidence Action has partnered with on other programs—International Relief & Development (IRD)—works in the health sector in Pakistan.27

In Pakistan, Deworm the World is partnering with two local organizations: IRD and the Institute of Development and Economic Alternatives (IDEAS).28 Neither organization has experience with school-based deworming.29 IRD has implemented other health programs and IDEAS is involved in education policy work.30

Because the prevalence of STH in Pakistan has not previously been mapped, IRD and Deworm the World are conducting a nationally representative prevalence survey there.31 The prevalence survey was initially slated to begin in early 2016, but was rescheduled to start in August 2016 after multiple delays; it is expected to finish in November 2016 (see footnote for detail).32 Results of the prevalence survey will determine the recommended treatment strategy for STH in Pakistan. Though WHO has predicted high prevalence of STH throughout most of the country, preliminary results are suggesting that there may be lower prevalence than originally thought, so it is unlikely a nationwide deworming program will be required.33

Deworm the World and its partners have reached out to provincial governments in Pakistan about implementing a deworming program; it expects conversations to move more quickly once results from the prevalence surveys are available.34 In June 2016, the national Ministry of Health in Pakistan dissolved, so Deworm the World expects to primarily work with individual provinces, similar to how it works closely with states in India; it believes that this will cause working in Pakistan to be somewhat more challenging.35

The prevalence survey in Pakistan is almost entirely supported by Deworm the World's unrestricted funding.36

Nigeria

In 2015, Deworm the World hired a consulting firm to assess the opportunities for working in Nigeria.37 The assessment found that several states with a high burden of STH did not have any NGO partners focused on deworming.38 After meeting with government officials in four of the states, Deworm the World decided to work in Cross River.39

In Nigeria, the national government requests that all programs that target a neglected tropical disease (NTD) be integrated with other NTD programs.40 Deworm the World partners with RTI International in Cross River.41 Deworm the World is supporting the state government to implement school-based treatment for schistosomiasis and STH, while RTI focuses on community-based treatment of other endemic NTDs (more details in footnote).42 The first MDA they supported took place in June 2016.43

Most of Nigeria has already been mapped for STH and schistosomiasis, which means that Deworm the World does not need to assist with mapping surveys.44 Future activities in Nigeria will likely include advocacy and technical assistance to the national government and expanding Deworm the World's support to additional states.45 Deworm the World intends to register in Nigeria and establish a national office there to support its future scale-up.46

Deworm the World is funding its work in Nigeria with unrestricted funding.47

Vietnam

Deworm the World is partnering with the Thrive Networks in Vietnam on an integrated deworming, sanitation, and hygiene education program in four provinces.48 The program is funded primarily by Dubai Cares and includes a randomized controlled trial (RCT) to test the impact of hygiene education on STH reinfection rates.49 The program was approved by the Vietnamese government in late 2015.50

Some deworming already occurs in Vietnam; Deworm the World and Thrive Networks are partnering with the government to improve the existing deworming program in the four provinces targeted by the program.51 Deworm the World hopes to convince the national government to develop an evidence-based, cost-effective national deworming program.52

The first MDA supported by Deworm the World occurred in April 2016; it was conducted by the National Institute of Malaria, Parasitology, and Entomology (NIMPE)—the government agency in Vietnam that implements MDAs.53 After the MDA occurred, Deworm the World received the results from the baseline prevalence survey it conducted pre-MDA; the results indicated that the treatment frequency required for each province was lower than expected.54 Deworm the World is currently advocating for the government to adjust its strategy; the current government policy is to conduct biannual deworming MDAs.55

Deworm the World has also worked with NIMPE to conduct prevalence surveys in an additional 21 provinces.56 This was beyond the scope of the Dubai Cares grant, so Deworm the World supported the prevalence surveys with its unrestricted funding.57 The results from that survey indicated that most provinces only need treatment once per year or once every other year.58

Deworm the World originally became involved in Vietnam through discussions between its former Executive Director, Alix Zwane, and Thrive Networks' former regional director.59 We are unsure who initiated the partnership between the two organizations.

Nepal

In early 2015, Deworm the World was in discussions with Nepal about starting a deworming program there.60 However, in mid-2015 Nepal experienced a large earthquake and those discussions were halted.61 It has not yet re-engaged with those conversations, in part due to capacity constraints.62

Deworm the World was first connected to the government of Nepal via the Abdul Latif Jameel Poverty Action Lab (J-PAL) (similar to how the program in Bihar began).63

Indonesia

Deworm the World has had initial conversations with the government of Indonesia about a potential deworming program there.64 Indonesia would require both technical assistance and funding for the implementation of a deworming program.65 Deworm the World hopes to make progress on conversations during the last quarter of 2016.66

Lymphatic filariasis programs in the countries Deworm the World works Section last updated: November 2016 India. There are active lymphatic filariasis (LF) programs in several of the states that Deworm the World supports.67 We discuss some details of India's LF program in this footnote.68 Kenya. There are LF programs in the coastal-region counties in Kenya.69 Ethiopia. Ethiopia has had an active LF program. Going forward it intends to coordinate LF treatments with treatments for STH to avoid duplication of efforts.70 Nigeria. In Cross River, Deworm the World works with its partner RTI to support an integrated NTD program. Our understanding is that there is coordination to avoid duplication of efforts for LF and STH.71 Vietnam. Vietnam recently stopped its LF program and intends to monitor whether or not it has successfully eliminated LF in the future.72 Pakistan. We do not know whether or not Pakistan has any active LF programs. Indonesia. Deworm the World told us that Indonesia has an active LF treatment program.73 Monitoring process in India Section last updated: November 2016 Bihar 2015: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state. The objective of independent monitoring is to determine whether deworming is being implemented according to planned protocols. Two-stage probability sampling was used to select schools for coverage validation on deworming day and mop-up day. First, 125 blocks were selected from all 38 districts by probability proportional to size sampling, followed by random sampling of schools to provide statewide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students. During coverage validation an additional 748 randomly sampled schools were surveyed after deworming days to check whether deworming occurred, if reporting protocols were followed, and to validate the coverage reporting. Coverage validation data was gathered through interviews with headmasters and 3 students (in 3 different randomly selected classes in each school), and by checking of all class registers and reporting forms."74 Rajasthan 2015: "The process of monitoring and evaluation in each deworming round are performed in three ways: (1) process monitoring, (2) coverage reporting and (3) coverage validation. In Rajasthan, both process monitoring and coverage validation were carried out in schools, while only coverage validation was done at anganwadis. This is because the four-day long (10-13 February) deworming program in anganwadis made process monitoring a challenge. Process Monitoring, Coverage Reporting, and Coverage Validation: Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program. Evidence Action assesses the program preparedness during pre-deworming phase and selected independent monitors observe [sic] the deworming processes on Deworming Day and mop-up day. We conduct process monitoring in two ways: a) telephone monitoring and cross verification and b) physical verification by visiting schools and training venues. Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state. The objective of independent monitoring is to determine whether the program is being implemented according to planned protocols. Two-stage probability sampling was used to select schools for independent monitoring on Deworming Day and mop-up day. First 125 blocks were selected from all 33 districts by probability proportional to size (PPS) sampling, followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively. These 125 monitors visited 125 schools on Deworming Day and an additional 125 on mop-up day to check for adequate drug supplies and awareness materials, to confirm whether teachers had received training, and to assess knowledge of adverse event management protocols and reporting processes. Monitors gathered data through observation of deworming and through interviewing headmasters, teachers, and randomly selected students."75 Madhya Pradesh 2015: "In Madhya Pradesh, as preschool-age deworming was implemented through the BSM, monitoring efforts focused on the school-age program through the NDD. In the future, it will be important to expand monitoring to anganwadis to better understand program preparedness and performance[…] Evidence Action assesses the program preparedness during pre-deworming phase and selected independent monitors observe the deworming processes on Deworming Day and mop-up day. We conduct process monitoring in two ways: a) telephone monitoring and cross verification, and b) physical verification by visiting schools and training venues. A two-stage probability sampling process was followed to select schools for NDD, mop-up day and coverage validation. Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively. The monitors visited 125 randomly selected schools on NDD, and an additional 125 schools on mop-up day (February 14) to check for adequacy of drug supplies and awareness materials, and assess whether teachers had received training, and had knowledge of adverse event management protocols and reporting processes. Monitors gathered data by observing deworming and by interviewing headmasters, teachers, and randomly selected students. An additional 750 randomly sampled schools were surveyed from February 18-26 to check whether deworming occurred and reporting protocols were followed, and to validate the coverage reporting."76 Chhattisgarh 2015 (coverage validation only): "The school database for random sampling in the 11 districts was obtained from the Ministry of Drinking Water and Sanitation, Government of India website. We visited 10 randomly selected schools in each block from the 55 blocks in these 11 districts. Therefore, they visited a total of 550 randomly sampled schools for coverage validation activities. In addition to the headmster's interview and verification of the deworming related documents, three randomly selected children from three different randomly selected classes were interviewed in each school. In addition, we also visited any anganwadis attached to the sampled schools. We could not achieve the targeted sample of 550 schools as two districts, Sukma and Bijapur, could not be covered given the high risk due to insurgency in these areas."77 Delhi 2015: "Evidence Action assesses the program preparedness during the pre-deworming phase and selected independent monitors observe the processes on deworming and mop up days. We conduct process monitoring in two ways: a) telephone monitoring and cross verification, and b) physical verification by visiting schools and training venues. The method of stratified random sampling using proportional allocation approach was followed for selection of schools and anganwadis for deworming day, mop-up day, and coverage validation monitoring to provide state-wide estimates of indicators. We hired an independent research agency, Sigma Research and Consulting Private Limited that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively. These monitors were to visit total of 400 randomly selected schools and 400 randomly selected anganwadis; 80 schools and 80 anganwadis on deworming day and mop up day each (April 16 & April 20); and 240 schools and 240 anganwadis during coverage validation (April 23-27, 2015). The actual number of schools and anganwadis visited on each day is given in annexure (Table SA-1). The monitors visited the selected schools and anganwadis on deworming day, on mop up day to check for adequacy of drug supplies and awareness materials, whether teachers/anganwadi workers had received training, and knowledge of adverse event management protocols and reporting processes. Monitors gathered data through observation during deworming and interviews with headmasters, teachers, and anganwadi workers as well as of randomly selected students from schools. Additional randomly sampled schools and anganwadis were surveyed from April 23-27 to check whether deworming occurred, reporting protocols were followed, and to validate the coverage reporting."78 Bihar 2014: "In its attempt to evaluate the adherence of each process to guideline and time plan, Deworm the World supported a two-way monitoring strategy; a) Telephone monitoring and cross verification, and b) Physical verification by visiting the sites, schools, and training venues."79 Delhi 2013: "In order to carry out robust M&E activities, Deworm the World retained short-term resources comprised of district coordinators and telecaller to a) assess the readiness of the system to implement deworming, and b) to follow-up with districts and nodal officers (for anganwadis) for the return of data post-deworming. In addition, independent monitors were hired and trained by Deworm the World to assess preparedness, visit schools and anganwadis on deworming and mop-up days and carry out coverage validation post mop-up day. The Deworm the World team and officials from SHS and DHFW also made several field visits to monitor the entire deworming program before, during, and after deworming day."80 Rajasthan 2013: "In order to carry out robust M&E activities, Deworm the World hired district coordinators, telecallers and independent monitors as detailed below. In addition, staff and officials from DMHFW, RCEE and DWCD also made several field visits to monitor the entire deworming program before, during, and after deworming day."81 Bihar 2012: Process: "Independent monitors visited a randomly selected sample of schools over four days – Deworming Day, Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 1216 schools (1.75% of 69,299 schools in Bihar) targeted for monitoring. From each of the 38 districts in Bihar, 2 blocks were selected by simple random sampling, for a total of 76 blocks. 2 clusters were randomly selected from each of the 76 blocks, excluding clusters with fewer than 8 schools. In each of these 152 clusters, 8 schools were randomly selected for a total of 1216 schools. Each monitor was assigned a block comprising 16 schools to be monitored. From this list of 16 schools, the monitor could visit any 4 schools on Deworming Day, another 4 on Mop-Up Day, and 8 more schools over the two Coverage Validation days. Out of the total sample of 1216 schools, monitors were able to survey 1196 schools comprising 302 schools on Deworming Day, 296 on Mop-Up Day, and 598 on Coverage Validation days."82 What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results.83 Delhi 2012: Process: "Monitoring visits were conducted by senior government officials. Random site visits by independent auditors occurred at a subset of training sessions and participating institutions. Coverage validation engaged independent auditors who conducted random site visits at a representative sample of schools and anganwadis to validate coverage statistics. This source of information was carefully compared with programme reports collated from each school and anganwadi to arrive at an accurate assessment of programme coverage."84 What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results.85 Rajasthan 2012: Process: "In order to evaluate the efficacy of the deworming protocol and process, independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 990 schools (1.1% of 90,488 schools in Rajasthan) targeted for monitoring. From each of the 33 districts in Rajasthan, 2 blocks were selected by simple random sampling. In each of these 66 blocks, 15 schools were randomly selected for a total of 990 schools. One monitor was assigned to each of these blocks. From the list of 15 schools in a block, the monitor could visit any 3 schools before Deworming Day for Preparation Monitoring, another 2 schools on Deworming Day, 2 more schools on Mop-Up Day, and 4 more schools over the two Coverage Validation days. The remaining 4 schools served as a buffer in case a particular school could not be visited. Hence, the actual sample size was 726 schools (0.8% of all schools)."86 What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results.87 Bihar 2011: Process: "Moreover, independent monitors were mobilized in each block (560 total) to visit 5% of all government schools on deworming and mop-up days to ensure that an adequate quantity of drugs were available, trained teachers were administrating drugs according to protocol, and community sensitization initiatives were successful in mobilizing non-enrolled children to attend. One auditor per district was in charge of supervising monitors to ensure quality monitoring of all schools."88 What we’ve seen: Monitoring forms used for data collection and full monitoring data from deworming day and mop-up day. We have not seen a summary of this data and have not analyzed it ourselves.89 Changes in prevalence over time Section last updated: November 2016 India prevalence surveys Deworm the World has conducted prevalence surveys in several of the states that it assists with in India, and has conducted one follow-up prevalence survey in the state of Bihar.90 The two surveys measuring the prevalence of worm infections in Bihar are consistent with the notion that the deworming program reduced infection rates in target populations. However, there are also plausible, alternative explanations for these results. (Note that the first survey was not designed to measure a subsequent change in infection rates, so our conclusion is not surprising.) The original prevalence study in Bihar measured prevalence in four of Bihar’s 38 districts, which were selected to inform a prevalence model based on climatic and socioeconomic variance across the state.91 We are not sure how representative that survey was of the state as a whole since the districts were not chosen randomly.92 Since the survey found >50% prevalence,93 the government of Bihar planned to follow WHO’s recommendation to conduct deworming MDAs twice per year, with one round each year led by the National Filaria Control Program (NFCP) and one round through schools that would be assisted by Deworm the World.94 However, according to Deworm the World, only four rounds of deworming took place between the two prevalence surveys in 2011 and early 2015, with one community-based round led by the NFCP and three school-based rounds assisted by Deworm the World.95 Deworm the World’s follow-up survey in early 2015 was conducted in randomly selected districts and was designed to be representative of the entire state and the three climatic regions.96 The results of both surveys, broken down by climatic region, are in the table below; no single district was surveyed in both prevalence surveys. Indications of STH prevalence in Bihar’s three agro-climatic zones in 2011 and 201597 Hookworm Ascaris (roundworm) Trichuris (whipworm) Any soil-transmitted helminth NW Alluvial Plains, 2011 (2 districts, non-random) 47.4% 56.8% 4.2% 71.3% NW Alluvial Plains, 2015 (5 districts, random) 21.0% 24.0% 12.0% 43.0% NE Alluvial Plains, 2011 (1 district, non-random) 32.8% 72.1% 11.3% 79.6% NE Alluvial Plains, 2015 (3 districts, random) 17.0% 38.0% 5.0% 50.0% S Alluvial Plains, 2011 (1 district, non-random) 40.2% 25.0% 1.7% 49.0% S Alluvial Plains, 2015 (6 districts, random) 14.0% 6.0% 2.0% 20.0% Statewide 2011 (4 districts, non-random) 42.2% 52.1% 5.2% 67.5% Statewide 2015 (14 districts, random) 17.0% 19.0% 6.0% 35.0% We are unsure whether worm prevalence in Bihar would have increased or decreased in the four years between the two prevalence surveys if the three MDAs that Deworm the World supported had not occurred.98 Here we’ve listed some other factors that we weigh when considering these results: Methodological limitations of these prevalence surveys. It may be that the original prevalence survey took place in districts with unusually high worm prevalence. Since it is unclear whether the original prevalence survey was representative of the state or its climatic zones and the two prevalence surveys did not sample the same districts, it may be that the decrease in prevalence seen above is the result of comparing different populations rather than a fall in prevalence in the same population over time. MDAs for lymphatic filariasis. The NFCP, which is designed to reduce the burden of LF as well as STH, claims it conducted MDAs in Bihar in 2012 and 2014 with albendazole, the drug used to treat STH.99 According to the Indian government's program website, the LF treatment program is designed to be annual, and reached 86% of the population in both 2012 and 2014.100 We would not be surprised if the coverage rates reported by the LF program were inaccurate, but we have not tried to verify them. Deworm the World has said that the 2014 round actually occurred in early 2015 after the 2015 prevalence survey.101 Factors other than MDAs. There are a variety of large-scale changes that could affect worm prevalence, such as improved sanitation infrastructure or general development. According to India’s central bank, inflation-adjusted per-capita income in Bihar increased by a total of 39% over a similar four year period, suggesting that there could have been broad improvements in the region that would be associated with better health and lower worm prevalence.102 Alternatively, it seems possible that population growth combined with poor sanitation infrastructure could have increased worm prevalence over this period if Deworm the World-supported MDAs had not been carried out. Kenya prevalence surveys In Kenya, the Kenya Medical Research Institute (KEMRI) is funded by CIFF to conduct independent prevalence surveys before and after every MDA that Deworm the World supports; Deworm the World is not involved in these surveys.103 We have seen results from the prevalence surveys conducted before and after the Year 1 (2012-13), Year 2 (2013-14), and Year 3 (2014-15) MDAs in Kenya.104 Deworm the World has provided us with details of a) how the baseline survey was conducted and b) pre-registered plans for follow up surveys.105 Note that we have not yet seen detailed descriptions of the methodologies used for the Year 2 and Year 3 surveys, so we have not verified that they were carried out as intended.106 The prevalence surveys are not representative of the full program Deworm the World supports: they are conducted in 20 districts randomly sampled from 66 districts in which STH was assumed to be endemic before Kenya started its national deworming program; we do not believe the 66 districts were selected randomly from all districts in the national program.107 We summarize the key features of the prevalence surveys in this footnote.108 We believe these surveys provide evidence that the deworming program reduced infection rates in target populations, indicating that the program effectively reached targeted children. The tables below show the results of the prevalence surveys for Year 1 (2012-13), Year 2 (2013-14), and Year 3 (2014-15).109 Kenya prevalence survey results - soil-transmitted helminths Hookworm Ascaris (roundworm) Trichuris (whipworm) Any soil-transmitted helminth Year 1 (2012-13) baseline (200 schools) 16.9% 19.2% 5.4% 33.4% Year 1 (2012-13) post-MDA (70 schools)110 3.2% 2.3% 4.3% 8.7% Year 2 (2013-14) pre-MDA (60 schools) 4.5% 12.5% 5.1% 19.0% Year 2 (2013-14) post-MDA (60 schools) 2.2% 1.9% 2.7% 6.0% Year 3 (2014-2015) pre-MDA (200 schools) 2.3% 11.9% 4.5% 16.3% Year 3 (2014 -2015) post-MDA (60 schools) 1.8% 2.8% 2.3% 6.3% Kenya prevalence survey results - Schistosomiasis111 S. haematobium - Coast Province only112 S. mansoni - all provinces Year 1 (2012-13) baseline113 14.8% 2.1% Year 1 (2012-13) post-MDA Unknown No comparable data available Year 2 (2013-14) pre-MDA 10.5% 2.7% Year 2 (2013-14) post-MDA 7.6% 0.6% Year 3 (2014-2015) pre-MDA 8.8% 1.5% Year 3 (2014 -2015) post-MDA 5.8% 0.8% Schools in Kenya do not all undergo MDAs simultaneously, but the surveys are conducted at approximately the same time before and after each MDA; the post-MDA surveys are conducted 5-6 weeks after an MDA.114 Additionally, it seems possible that surveying the same 60 schools each year could introduce bias: if the schools have an incentive to show that the program is working, they may execute a higher-quality deworming program than they otherwise would if they weren't going to be re-surveyed.115 There is no control group for the prevalence surveys. Does Deworm the World increase the likelihood that governments implement deworming programs? Section last updated: November 2016 While we have limited evidence to rely on, we would guess that Deworm the World increases the likelihood that (national and subnational) governments implement deworming programs. In India, state governments seem to heavily utilize Deworm the World when planning, implementing, and monitoring their deworming programs. In several present and future countries, Deworm the World pays (or expects it may pay) the majority of financial program costs, which increases our expectation that the organization is critical to the program happening.116 Below, we look in detail at what we know of Deworm the World's impact in India, as a light case study of Deworm the World's potential impact. We detail what we know with respect to Deworm the World’s experience in India, including: Our limited understanding of how Indian states have historically decided to launch and maintain health programs with NGO assistance; Deworm the World’s role in the mass school-based deworming programs that were in operation in India before the first national deworming day (NDD) in 2015, including our conversations with government officials in Rajasthan about their decision to launch a deworming program with Deworm the World’s assistance; The national government’s interaction with Deworm the World as part of its decision to launch a national deworming day in India in 2015. We then present what we know of Deworm the World's impact in several other countries, where our knowledge is more limited. Reasons new health programs may be started or halted in India Deworm the World and Children’s Investment Fund Foundation (CIFF) have told us that Indian state governments often receive funds earmarked for broader health programs but often fail to spend these funds.117 We asked the Center for Global Development (CGD) for a recommendation of someone to talk to in order to understand states’ decisions to initiate deworming programs, and CGD referred us to Professor Devesh Kapur.118 Dr. Kapur asserted that it takes significant internal political will or external stimulus (such as from a non-profit) to sufficiently overcome general bureaucratic inertia in India to sustain a new health program. This holds true even in cases where a national mandate exists for a program (as it does for school-based deworming). Dr. Kapur felt that in the majority of cases, nonprofit technical assistance was likely to increase the probability of a program’s going forward.119 Deworm the World told us that it believes that some school health programs have been stalled in Indian states due to negative media attention;120 one of Deworm the World's goals is to prevent these reports in the states in which it works.121 The documentation that Deworm the World has sent us supports—but does not fully demonstrate—the possibility that negative media undermined consistent mass deworming in Assam before the National Deworming Day in 2015. We have not attempted to independently verify that possibility because we do not think we would likely be able to do so effectively. Nonetheless, it is plausible that were decision-makers in Indian states to have the impression that negative reports could cause a program to be halted, they might be less willing to move ahead, and Deworm the World's assurances that it would help prevent these reports could increase the likelihood that a state agrees to implement deworming. Early deworming programs in India and Deworm the World's role In 2013 Deworm the World and CIFF told us that the Indian government mandates that all states provide school-based deworming through the larger school-based Weekly Iron and Folic Acid Supplementation program and that states can request funding for deworming through this program.122 At the end of 2013, the only states in India that appear to have been implementing school-based deworming programs were Andhra Pradesh, Bihar, Delhi, Jharkhand, Punjab, Rajasthan, and possibly Assam.123 Deworm the World states that it (or others affiliated with it) played a key role in four of these seven states’ decisions to implement deworming.124 In late 2014, Deworm the World believed that more states were starting to launch their own deworming programs without assistance from Deworm the World but was unsure how many had successfully done so.125 Without involvement from Deworm the World, Assam initiated a vitamin A supplementation and deworming mass drug administration (MDA) in the 2010-2011 school year but reported low and conflicting coverage for that year.126 In the 2012-2013 school year Assam planned to implement a deworming program through the School Health Program, though we are not sure whether that program happened as planned.127 Assam did approach Deworm the World in late 2013 to explore opportunities for the organization to provide assistance to the state, though as of late 2016 Deworm the World has not become directly involved there.128 During our site visit to Deworm the World in Rajasthan, India, we spoke with three government officials who were involved in the deworming program.129 Of these, two stressed the importance of Deworm the World's technical assistance, and the person we perceive as having been most responsible (of the three) for the decision to go forward with deworming gave the impression that the availability of this technical assistance had been a key factor in deciding to go forward. It should be noted that one of the three gave the impression that Deworm the World's help was not needed, and all three conversations took place with multiple Deworm the World representatives present. With that said, the highest ranking of the three officials gave what we felt to be a nuanced and realistic picture of Deworm the World's impact that implied a substantial (while not determinative) role. She stated that a) the immediate availability of technical assistance improved her confidence that the program would proceed quickly and smoothly, b) she wasn’t sure whether the program would have proceeded if not for Deworm the World, and c) she was interested in finding a nonprofit technical assistance partner for at least one other program in a different category.130 This suggests that nonprofit technical assistance can be a key factor in progressing a program. The National Deworming Day and Deworm the World’s role In 2014, Deworm the World told us that the Indian national government was pursuing the idea of having a coordinated national deworming day (NDD), whereby the national government would provide some assistance to states in implementing school-based deworming on a single day to encourage more states to implement the program. Deworm the World told us that it initially proposed this idea.131 Deworm the World said that the government asked it to provide technical assistance to the NDD, including helping develop the implementation strategy, designing and developing training and reference materials, community mobilization strategies, and monitoring and evaluation systems and reporting formats.132 The posters, ads for radio and television, training materials, and other documents that Deworm the World helped create for NDD are available at the website in this footnote.133 Deworm the World told us that, as part of its advisory role in the planning process, it advocated for a delay of the initial program from October 2014 to February 2015 because it believed there weren’t sufficient drug supplies.134 The first NDD occurred in February 2015,135 with 12 states participating (with deworming in one additional state occurring in April).136 A second NDD occurred in February 2016, with 30 of India's 36 states and union territories participating.137 Deworm the World's impact on the existence of deworming programs in other countries In October 2016, we spoke to Deworm the World about whether or not deworming programs would have occurred in some of the other countries it works in without Deworm the World's assistance. Of the three countries we discussed, only one (Nigeria) seemed like a case for Deworm the World causing new deworming programs to exist:138 Vietnam: There was a national deworming program in Vietnam before Deworm the World started working there, although our impression is that this program did not treat all areas that needed a deworming MDA or treated those areas somewhat sporadically. Deworm the World told us that the RCT it is supporting in Vietnam likely would not have been funded if Deworm the World were not involved.139 Nigeria: Deworm the World believes that if it had not partnered with RTI in Cross River, Nigeria, RTI most likely would have supported community-based treatments specifically targeting onchocerciasis and lymphatic filariasis in the state; in those local government areas (LGAs) also endemic for schistosomiasis and/or STH, treatment for those would likely have been provided.140 Such a program would not have treated the LGAs in Cross River that are not endemic for LF and/or onchocerciasis but are endemic for schistosomiasis and/or STH. There are many funders supporting NTD treatments in Nigeria, and Deworm the World noted that another organization may have supported deworming in Cross River if RTI and Deworm the World had not started to work there. However, many funders are focused on integrated NTD programs, which often prioritize LF and onchocerciasis treatments, so Deworm the World is not sure if other organizations would have ensured that all appropriate LGAs were treated for schistosomiasis and STH.141 Furthermore, there are still funding gaps in Nigeria,142 so it is possible that if another funder had supported Cross River, this would have come at the expense of not covering another state. Ethiopia: Ethiopia was already receiving support from SCI before Deworm the World started assisting the national government. SCI and the government were planning a national program;143 this program would have likely occurred without Deworm the World's support. We discuss how Deworm the World may have influenced the quality of these programs below. Does Deworm the World's work increase the quality of deworming programs? Section last updated: November 2016 Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are highly uncertain about this. Deworm the World may improve program quality by: Increasing the chances that the first deworming round in each state begins earlier than it otherwise would have and that subsequent rounds occur on schedule.144 Providing training or increasing the training quality by simplifying training material and creating a more robust training program for those who train representatives from each school.145 Ensuring that support roles are staffed.146 Improving focus and attention to detail, possibly increasing the likelihood that schools receive the materials and instructions necessary to implement the deworming program.147 Expanding the scope of the program to a broader age group.148 Increasing community acceptance of mass treatment and the ability of a program to avoid or withstand publicity associated with related or seemingly-related adverse events.149 Implementing or advocating for monitoring systems. This may improve program quality by creating a mechanism through which implementers are held accountable. Also, monitoring systems could generate lessons that Deworm the World and the government could use in future rounds of treatment.150 Advocating for treatment strategies that align with WHO guidelines, which may help to ensure that children are receiving the necessary dosage of drugs on an annual basis.151 Again, to answer this question we have primarily looked at evidence from Deworm the World's program in India. Evidence that relates to Deworm the World’s impact on the quality of deworming programs includes the following, each discussed in more detail below: Testimony of a government official in Rajasthan in 2013 Calls Deworm the World monitors made in India during the run-up to Deworming Day A training we observed in 2013 We also note what we have learned from talking to Deworm the World about its programs in other countries. Testimony of a government official We believe our strongest piece of evidence in favor of the idea that Deworm the World improves quality is the conversation we had with a Rajasthan nodal officer in 2013,152 who cited many of the points above and made the case that Deworm the World had played an important role in improving the quality of deworming, backing up an intuition that, without external support, such programs would often be of lower quality. On the other hand, (a) this conversation took place with multiple Deworm the World representatives present, and the nodal officer had worked closely with Deworm the World; (b) another government official (on the same visit) stated that he believed Deworm the World's technical help was not needed and did not add value, as the state was accustomed to running school health programs and deworming is a relatively simple one (we do not put strong weight on (b) alone, since the government official may have had other incentives to give the message he did and had not worked as closely with the program as the nodal officer, but in context of the other observations we find (b) worth noting). Monitoring calls before Deworming Day As part of its standard monitoring process in India, Deworm the World has tele-callers place thousands of calls to government employees at different levels of government during the run-up to Deworming Day to ensure that preparations are proceeding as intended.153 Deworm the World reports that these calls helped to uncover problems, which were reported on the same day to the appropriate government officials to address.154 It also seems plausible that these calls reduce the likelihood that problems arise after the calls are placed, due to the signal of importance created by the call itself. We are unaware of how common these monitoring calls are as part of other government programs, nor are we aware of any data that could help quantify the size of the impact they have. Training observed in 2013 We felt that the training we attended in 2013 (the only part of Deworm the World's work that we observed directly) had major limitations in terms of potential to improve program quality.155 Deworm the World has made changes to address these since this visit, though we have not observed a training since that time.156 Deworm the World's impact on program quality in other countries In October 2016, we spoke to Deworm the World about how it has impacted the quality of programs that it has supported. In general, Deworm the World claims to have improved the quality of the programs it has worked with:157 Vietnam: In Vietnam, Deworm the World works with the National Institute of Malaria, Parasitology, and Entomology (NIMPE), which had conducted prevalence surveys and MDAs prior to Deworm the World's support. Deworm the World feels that collaboration with NIMPE in the four provinces Deworm the World is supporting has improved NIMPE's processes, although Deworm the World is unsure if NIMPE will adopt its practices more broadly. For example, prior to Deworm the World's support, NIMPE's prevalence surveys only examined a small number of schools in limited areas that were not fully representative of the area being surveyed.158 Additionally, NIMPE did not have procedures in place to train teachers and health workers on how to implement a Deworming Day; Deworm the World assisted with setting up these processes. Deworm the World has also been advocating for Vietnam to follow WHO guidelines when designing its treatment strategy; it hopes that this will make the program more evidence-based in the future. Nigeria: In Nigeria, Deworm the World provided expertise on school-based MDA programs. Without its assistance, Deworm the World believes that its partner, RTI, may have started a program in Cross River that only supported community-based programs, which would have missed school-age children in areas where community-based programs were not recommended. Ethiopia: Because it works closely with SCI in Ethiopia, Deworm the World believes that it is difficult to assess the impacts attributable to Deworm the World alone. SCI was providing technical assistance to Ethiopia before Deworm the World began to, and Deworm the World believes that SCI was planning to support the treatments of both schistosomiasis and STH prior to Deworm the World's involvement. Deworm the World noted that it leveraged its Kenya team's expertise to assist the Ethiopian government and to facilitate learning exchanges between the two countries. Accuracy of coverage data Section last updated: November 2016 In calculating the cost-per-treatment figures above, we use the number of children treated in the programs, which is obtained from data reported by governments to Deworm the World. We have made adjustments to these figures in our cost-per-treatment analysis based on findings from Deworm the World's independent monitoring of treatment numbers in a sample of schools. India Comparing classroom data to school data Monitors visit a random sample of schools to compare each school’s records of how many children it dewormed to the number of children that were dewormed according to its classroom records. Based on this data, it appears that many schools in 2015 did not follow the recording protocol, but the schools that did had reasonable agreement on the number of children dewormed between the classroom records and the school-wide records (details in this footnote).159 In 2016, the ratio of students recorded as dewormed in schools' records to the students the school reported as dewormed ranged from 53% to 94% (53% means that for every 100 students reported being dewormed, only 53 were actually recorded as dewormed when Deworm the World monitors checked schools' forms).160 We have made an adjustment for this in our cost-per-treatment analysis. Comparing classroom data to state data It is our understanding that Deworm the World does not regularly monitor the accuracy of the aggregation process beyond the school level for each round of deworming, although it periodically checks this.161 The number of children treated in each school is recorded in classrooms and then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (school, node, block, district, and state) to create a reported coverage estimate for the entire state.162 Errors in the aggregation process could occur if those performing the aggregation have an incentive to overreport the number of children dewormed, or if reporting forms are lost (in which case we believe that deworming would be underreported).163 We calculated the portion of children that were dewormed—according to class records in schools which used class records or didn’t do deworming—at the schools monitors visited. Then, we compared this to the fraction of total students covered out of total students enrolled in government schools (details of methodology in this footnote).164 The results of estimating statewide coverage using both government-reported data and monitors’ data are in the below table. Estimates of total statewide coverage (India, 2015)165 2015 deworming round % of students dewormed based on government-reported figures % of students dewormed based on monitors' observations (of class records) Bihar 74% 65% Rajasthan 89% 85% Madhya Pradesh 90% 68% Delhi 82% 77% TOTAL 80% 73% While we don’t put much weight in the comparison above, we note that a) there are cases in which the proportion of children covered is significantly different, which makes us more uncertain how much weight to put in the coverage figures, and b) the fact that we do not see a pattern of the government consistently over-reporting its coverage figures gives us slightly more confidence in using the coverage figures. Kenya We have not looked in-depth at the data that Deworm the World has sent us related to coverage in Kenya. If we take the coverage reporting figures at face value, Kenya was able to treat 83% of its target population of children (in counties participating in the national deworming program).166 According to one recent summary monitoring report from Kenya that we have seen, monitors observed coverage to be 99% in the schools they visited (based on school records).167 As in India, Deworm the World's Kenya team also calculates the ratio of students recorded as being dewormed in classroom records to the students reported as dewormed by the school. In the 2014-15 round (Year 3, the most recent round for which we have data), Deworm the World found that this ratio was 105%, implying that the government had underreported coverage for that round.168 We have made an adjustment using the 105% figure in our cost-per-treatment analysis. Are donations to Deworm the World leveraged? Section last updated: November 2016 We have written before about the complexity involved in trying to understand leverage in charity, and below we lay out the relevant issues worth considering in the case of Deworm the World. Donors to Deworm the World support only expenses paid by Deworm the World. Generally, Deworm the World tries to work with governments and other funders to create funding arrangements where each partner is contributing some portion of the program's costs. Sometimes, governments pay most of the expenses. For example, Deworm the World's costs in India are only approximately 44% of the overall cost of the program (not including the value of teacher time).169 The role that Deworm the World's funds play in the program is an important consideration in determining the cost-effectiveness of donations. This role could range between: High leverage, high cost-effectiveness. Deworm the World's funds could have high leverage by: (a) causing government funds that otherwise would not have been spent or would have been spent on a lower-value program to be used in support of a deworming program; or (b) contributing a relatively small amount of funding to cause a deworming program to run significantly better, thereby reaching many more children than it otherwise would have. No leverage, average cost-effectiveness. Deworm the World's funding may allow the program to reach more children at a similar overall cost-per-treatment as it would have otherwise. Minimal contributions, low cost-effectiveness. If Deworm the World's work leads to few additional children receiving treatment, it may increase the overall cost-per-treatment. In the past, we guessed that Deworm the World’s work in India played a role in increasing the likelihood that state governments conduct deworming programs, so we estimated that its leverage on funds used for scaling up India programs could have reasonably ranged from less than 1x (i.e., programs would have taken place without Deworm the World’s involvement) to 4-5x, though we were highly uncertain about this estimate.170 India may have been an especially promising place to achieve leverage because the Indian national government had made money available for state governments interested in implementing deworming, and states may have been more likely to implement deworming with the addition of technical assistance.171 In the countries Deworm the World is starting to work in, governments may have less funding to support deworming. This may cause Deworm the World to pay a higher fraction of the overall cost of the program, making the upside potential for leverage of future donations more limited. For example, we estimate that Deworm the World bears the majority of total program costs in Kenya.172 Deworm the World has told us that it expects the cost per treatment of its future programs to be more similar to its program in Kenya than India, though we have not explicitly asked about whether it expects to pay a similar share of overall program costs as it does in its Kenya program.173 Deworm the World Initiative and Evidence Action Section last updated: November 2016 Deworm the World Initiative is led by Evidence Action. Evidence Action supports other programs in addition to Deworm the World.174 This has some implications relevant to Deworm the World’s room for more funding: donations to Evidence Action, even if restricted to Deworm the World, might change the actions that staff take to fundraise (i.e., which grants they pursue, what type of funding they ask for). We've seen some evidence that this is the case: In early 2015, Evidence Action’s plan for using unrestricted funds included a relatively high priority to spend $0.8 million on Deworm the World. After receiving funds related to GiveWell’s recommendation that were designated for Deworm the World, Evidence Action allocated unrestricted funding to other programs instead of to Deworm the World. As such, it seems likely that $0.8 million of GiveWell directed funds (70% of GiveWell-directed funds to Deworm the World that year) caused on the margin more funding to Evidence Action’s other programs, rather than more dewormings to take place.175 In early 2016, Evidence Action's plans for the first $1.55 million in unrestricted funding it raised included spending $100,000 in unrestricted funding on Deworm the World, $600,000 on Dispensers for Safe Water, $300,000 on Evidence Action Beta, and the rest on organizational development.176 The next $1.5 million raised after the first $1.55 million was expected to go towards reserves.177 We are not sure how this compares to Evidence Action's plans before the 2015 giving season, which were not shared with us. We note that Deworm the World was slated to receive significantly less than Dispensers for Safe Water, despite being a similarly sized program. It seems plausible to us that had Deworm the World not received so much unrestricted funding from GiveWell-influenced donors over the 2015 giving season, Evidence Action would have planned to allocate more of its expected unrestricted funding to Deworm the World. In late 2016, Evidence Action shared with us a rough estimate of how it planned to allocate (or already had allocated) $1.9 million in unrestricted funding in 2016.178 For the $1.9 million, $163,000 (8%) was expected to go to Deworm the World, which is slightly more than the portion expected in early 2016.179 However, no funding was allocated to Evidence Action Beta.180 While we are not sure why this is the case, it is possible that Evidence Action chose to use its funding on other programs after Good Ventures made a grant to Evidence Action Beta in early 2016.181 Evidence Action also planned to allocate significantly more than expected to Dispensers for Safe Water ($1 million, compared to $0.6 million originally planned).182 The reallocation of Evidence Action Beta funding, which served primarily to increase funding to Dispensers for Safe Water, again indicates that GiveWell-influenced funding may be impacting how Evidence Action chooses to use its unrestricted funding. Evidence Action also shared with us a rough estimate of how it would allocate $1.8 million in unrestricted funding that it expects to have in 2017: $600,000 is expected to be allocated to Deworm the World while $500,000 is expected to go to Dispensers for Safe Water and $100,00 to Evidence Action Beta (the rest is for organizational development).183 Deworm the World also notes that many of Evidence Action's investments in general organizational development have benefited Deworm the World, as well as Evidence Action's other programs, substantially.184 All sources for Deworm the World review Section last updated: November 2016 Document Source Alderman et al. 2006 Source (archive) Allen and Parker 2011 Source (archive) Alix Zwane conversation August 30th 2013 Unpublished Alix Zwane conversation June 4th 2013 Source Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013 Unpublished Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 Unpublished Assam 2010 guidelines for deworming Source Assam midday meal report 2013 Source (archive) Assam reproductive and child health 2011-2012 Source (archive) Assam state programme implementation plan 2011-2012 Source (archive) Baird et al 2012 Source Bleakley 2007 Source (archive) CIFF conversation September 10th 2013 Source Croke 2014 Source (archive) Devesh Kapur conversation October 14th 2013 Source Deworm the World and SCI, Ethiopia coverage survey Source Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Unpublished Deworm the World, Bihar 2016 IMCV report Source Deworm the World, Chhattisgarh 2016 IMCV report Source Deworm the World, Chhattisgarh prevalence survey report, August 2016 Source Deworm the World, Kenya Coverage Reporting data, Year 3 Source Deworm the World, Kenya Deworming Day data, Year 3 Source Deworm the World, 2015 expense summary Unpublished Deworm the World, 2015 expense summary - by funder Unpublished Deworm the World, Ethiopia independent monitoring report, Year 1 Source Deworm the World, Ethiopia prevalence survey report Source Deworm the World, Ethiopia Workplan Unpublished Deworm the World, Vietnam final report for STH survey in 21 provinces Source Deworm the World, Kenya 2014-2015 program report Source Deworm the World, Kenya Narrative Report - Year 1 Source Deworm the World, Kenya Narrative Report - Year 2, Quarter 4 Source Deworm the World, Kenya Narrative Report - Year 3, Quarter 3 Source Deworm the World, Kenya process monitoring report, Year 4 Source Deworm the World, Kenya Year 2, DD - Main instrument Source Deworm the World, Kenya Year 3, DD - Main instrument Source Deworm the World, Kenya Year 2, Pre DD - School instrument Source Deworm the World, Kenya Year 3, Pre TT form Source Deworm the World, Kenya Year 3, Post DD - Coverage instrument Source Deworm the World, Madhya Pradesh 2015 prevalence survey report Source Deworm the World, Madhya Pradesh cost-per-treatment - 2015 Unpublished Deworm the World, Madhya Pradesh 2016 IMCV report Source Deworm the World, National Deworming Day states, August 2016 Source Deworm the World, Rajasthan 2016 IMCV report Source Deworm the World, target populations and parasitology data Source Deworm the World, Telengana 2016 IMCV report Source Deworm the World, Tripura 2016 IMCV report Source Deworm the World, Uttar Pradesh prevalence survey report, December 2015 Source Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day Source Deworm the World, Vietnam 2016 monitoring survey form for coverage validation Source Deworm the World, Vietnam Independent Monitoring Report, 2016 Source Deworm the World, Vietnam baseline prevalence survey - 4 provinces Source Deworm the World 2015 Uttar Pradesh prevalence survey report Source Deworm the World, Uttar Pradesh 2016 IMCV report Source DSW 2012 GiveWell site visit Source DtWI 2013 GiveWell government interviews Source DtWI 2013 GiveWell site visit Source DtWI Assam research 2013 Source DtWI Bihar 2011 cost data Source DtWI Bihar 2011 coverage data Source DtWI Bihar 2011 monitoring data for deworming day Source DtWI Bihar 2011 monitoring data for mop-up day Source DtWI Bihar 2011 Monitoring Form for Deworming Day Source DtWI Bihar 2011 monitoring form for mop-up day Source DtWI Bihar 2011 prevalence survey report Source DtWI Bihar 2011 program report Source DtWI Bihar 2012 cost data Source DtWI Bihar 2012 cost data details Unpublished DtWI Bihar 2012 coverage data Source DtWI Bihar 2012 monitoring data for coverage validation Source DtWI Bihar 2012 monitoring report Source DtWI Bihar 2014 cost data Unpublished DtWI Bihar 2014 program report Source DtWI Bihar 2014 program report annex 1 Source DtWI Bihar 2014 program report annex 2 Source DtWI Bihar 2015 independent monitoring tables Source DtWI Bihar 2015 monitoring data for coverage validation, schools Source DtWI Bihar 2015 monitoring data from deworming day, schools Source DtWI Bihar 2015 monitoring data from mopup day, schools Source DtWI Bihar 2015 monitoring survey for coverage validation, schools Source DtWI Bihar 2015 monitoring survey from deworming day, schools Source DtWI Bihar 2015 monitoring survey from mopup day, schools Source DtWI Bihar 2015 Prevalence Survey report Source DtWI Bihar 2015 Program report Source DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015 Unpublished DtWI Chhattisgarh 2015 coverage validation report Source DtWI Chhattisgarh 2015 coverage validation tables Unpublished DtWI Chhattisgarh 2015 independent monitoring tables Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools Source DtWI class register audits 2013 Source DtWI cost narrative 2013 Source DtWI Cost per treatment blog post January 2015 Source (archive) DtWI cost per treatment summary 2013 Source DtWI coverage data 2013 - 2014 Source DtWI Delhi 2012 cost data Source DtWI Delhi 2012 coverage data by anganwadi Source DtWI Delhi 2012 coverage data by school Source DtWI Delhi 2012 coverage report Source DtWI Delhi 2012 monitoring data Source DtWI Delhi 2012 monitoring form deworming day Source DtWI Delhi 2012 prevalence survey design Source DtWI Delhi 2012 prevalence survey report Source DtWI Delhi 2012 program report Source DtWI Delhi 2013 cost data Unpublished DtWI Delhi 2013 program report Source DtWI Delhi 2015 independent monitoring tables Source DtWI Delhi 2015 monitoring data for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring data for coverage validation, schools Source DtWI Delhi 2015 monitoring data from deworming day, anganwadis Source DtWI Delhi 2015 monitoring data from deworming day, schools Source DtWI Delhi 2015 monitoring data from mopup day, anganwadis Source DtWI Delhi 2015 monitoring data from mopup day, schools Source DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring survey for coverage validation, schools Source DtWI Delhi 2015 monitoring survey from deworming day, anganwadis Source DtWI Delhi 2015 monitoring survey from deworming day, schools Source DtWI Delhi 2015 monitoring survey from mopup day, anganwadis Source DtWI Delhi 2015 monitoring survey from mopup day, schools Source DtWI Delhi 2015 program report Source DtWI Kenya 2013-2014 cost per treatment data Source DtWI Kenya 2013-2014 program report Source DtWI Madhya Pradesh 2015 coverage validation form Unpublished DtWI Madhya Pradesh 2015 deworming day monitoring form Unpublished DtWI Madhya Pradesh 2015 independent monitoring tables Source DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools Source DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools Source DtWI Madhya Pradesh 2015 program report Source DtWI Monitoring Improvements 2014 Source DtWI NDD blog post February 2015 Source (archive) DtWI NDD Year 1 M&E review July 2015 Source DtWI Rajasthan 2012 cost data Source DtWI Rajasthan 2012 cost data details Unpublished DtWI Rajasthan 2012 coverage data for anganwadi Source DtWI Rajasthan 2012 coverage data for schools Source DtWI Rajasthan 2012 monitoring data for coverage validation in anganwadis Source DtWI Rajasthan 2012 monitoring data for coverage validation in schools Source DtWI Rajasthan 2012 monitoring form coverage day Source DtWI Rajasthan 2012 monitoring form pre-deworming day Source DtWI Rajasthan 2012 monitoring report Source DtWI Rajasthan 2012 prevalence survey report Source DtWI Rajasthan 2013 cost data Unpublished DtWI Rajasthan 2013 prevalence survey report Source DtWI Rajasthan 2013 program report Source DtWI Rajasthan 2015 independent monitoring tables Source DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring data for coverage validation, schools Source DtWI Rajasthan 2015 monitoring data from deworming day, schools Source DtWI Rajasthan 2015 monitoring data from mopup day, schools Source DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring survey for coverage validation, schools Source DtWI Rajasthan 2015 monitoring survey from deworming day, schools Source DtWI Rajasthan 2015 monitoring survey from mopup day, schools Source DtWI Rajasthan 2015 program report Source Evidence Action 2014 budget Unpublished Evidence Action, 2015 financials by program Unpublished Evidence Action, blog post, January 8, 2015 Source (archive) Evidence Action, blog post, January 16, 2015 Source (archive) Evidence Action, blog post, June 12, 2015 Source (archive) Evidence Action, blog post, December 21, 2015 Source (archive) Evidence Action, blog post, April 27, 2016 Source (archive) Evidence Action, blog post, June 30, 2016 Source (archive) Evidence Action, blog post, July 5, 2016 Source (archive) Evidence Action, blog post, August 1, 2016 Source Evidence Action, Projected allocation of unrestricted funds, 2016 Source Evidence Action 2015 draft budget Unpublished Evidence Action 2015 funding gap analysis Source Evidence Action cover letter 2013 Source Evidence Action launch announcement 2013 Source (archive) Evidence Action Q1 financials, 2016 Unpublished Evidence Action website 2013 Source (archive) Evidence Action website announcement April 2014 Source (archive) Evidence Action website, Deworm the World Initiative (October 2015) Source (archive) Evidence Action website, Deworm the World Initiative (March 2016) Source (archive) Evidence Action website, Deworm the World Initiative (December 2016) Source (archive) Evidence Action website, Evidence Action Beta (October 2015) Source (archive) Evidence Action website, Who we are (November 2016) Source (archive) GiveWell analysis of Deworm the World 2014 Financial summary Source GiveWell analysis of Deworm the World cost per treatment Source GiveWell analysis of Deworm the World cost-per-treatment, 2016 Source GiveWell analysis of Deworm the World cost-per-treatment, October 2016 Source GiveWell analysis of Deworm the World financials - 2016 Source GiveWell DtWI 2013-2014 cost data summary Source GiveWell enrollment-based student coverage check 2015 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013 Source GiveWell's notes from site visit to India, October 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014 Unpublished GiveWell's non-verbatim summary of a conversation with END Fund staff, October 17, 2016 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014 Source GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014 Source GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 Source Global Alliance to Eliminate Lymphatic Filariasis - Prevention Source (archive) Grace Hollister conversation June 19th 2013 Source Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Unpublished Grace Hollister, conversation with GiveWell, April 20, 2016 Unpublished Grace Hollister, conversation with GiveWell, May 5, 2016 Unpublished Grace Hollister, conversation with GiveWell, June 13, 2016 Unpublished Grace Hollister, conversation with GiveWell, July 25, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 11, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 24, 2016 Unpublished Grace Hollister, conversation with GiveWell, September 1, 2016 Unpublished Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, March 23, 2015 Unpublished Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 Unpublished Grace Hollister, email to GiveWell, March 6, 2016 Unpublished Grace Hollister, email to GiveWell, June 9, 2016 Unpublished Grace Hollister, email to GiveWell, September 13, 2016 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 Source Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Unpublished Grace Hollister, edits to GiveWell's review, November 7, 2016 Unpublished Grace Hollister, edits to GiveWell's review, November 20, 2016 Unpublished Harvard Business School Kenya Case Study A 2010 Unpublished India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015 Source India NDD documents 2015 Source (archive) Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014 Unpublished JPAL CEAs in education 2011 Source Kabatereine et al. 2001 Source (archive) KEMRI prevalence report - Year 2 Unpublished LF treatment coverage 2015 Source (archive) LF treatment drugs 2012 Source (archive) Miguel and Kremer 2004 Source Mwandawiro et al. 2013 Source (archive) Neetu Chandra Sharma, Daily Mail - India article, August 8, 2016 Source (archive) Paul Monaghan, conversation with GiveWell, September 8, 2016 Unpublished Paul Byatta, conversation with GiveWell, September 20, 2016 Unpublished Paul Byatta, attachments to email to GiveWell, September 23, 2016 Source Preventive chemotherapy in human helminthiasis 2006 Source (archive) Professor Devesh Kapur Biography 2013 Source (archive) Reserve Bank of India, GDP per capita, Table 10, September 16, 2015 Source (archive) SCI Malawi coverage survey 2012 Source STH coalition framework for action November 2014 Source (archive) U-DISE Elementary Thematic Maps 2015 Source (archive) U-DISE Secondary Flash Statistics 2015 Source (archive) U-DISE Secondary Thematic Maps 2015 Source (archive) WHO, Helminth control in school-age children Source WHO, Helminth control in school-age children second edition Source WHO soil-transmitted helminthiases 2012 Source (archive) WHO, Summary of global update on preventive chemotherapy implementation in 2015 Source WHO STH factsheet Source (archive) WHO STH treatment report Source (archive) WHO Weekly epidemiological record, 6 March 2015 Source (archive) WHO Weekly epidemiological record, 18 December 2015 Source World Schistosomiasis Risk Chart 2012 Source 1 Grace Hollister, conversation with GiveWell, August 11, 2016 and Grace Hollister, conversation with GiveWell, August 24, 2016 2 "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program….In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J- PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there." Grace Hollister conversation June 19th 2013, Pgs 1-2. See the same set of conversation notes for descriptions of how Deworm the World's programs in Delhi and Rajasthan started as well. 3 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016. For example, see DtWI Madhya Pradesh 2015 program report: Prevalence surveys: "To develop an appropriate STH treatment strategy, Evidence Action obtained support and approvals from State NHM [National Health Mission] and Education Department to conduct an STH prevalence and intensity survey among children enrolled in government primary schools. Evidence Action, in partnership with National Institute of Epidemiology - Chennai (NIE), Post Graduate Institute of Medical Education and Research - Chandigarh (PGIMER), and GfK Mode, a market research firm conducted a STH prevalence and intensity survey among children aged 5 to 10, studying in government primary schools in the state." Pg 7. High level planning (e.g. obtaining drugs): "Evidence Action worked with the nodal officer to complete the drug requisition for 2015-16 in August 2014. The drugs for next deworming round (2016) were received at the state in form of two consignments on June 13 and June 18, 2015." Pg 12. Training and distribution cascades: "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." Pg 11. "Evidence Action supported the implementation of the training cascade as depicted below, to orient various levels of functionaries in the key departments… Evidence Action helped contextualize materials from the NDD resource kit according to state requirements, including training presentations, handouts for frontline workers, and frequently asked questions (FAQs)... Evidence Action's regional and district coordinators participated in all 51 district-level trainings under NDD. Additionally the team attended a sample of clock-level trainings to provide support and assess quality (Annexure H2)." Pg 14. Community sensitization: "Evidence Action also briefed media representatives about the program and the event, resulting in widespread coverage. Media kits included key information on the program such as a concept note shared by the state NHM. District-level launch events were held widely across the state. The events were led by local district administration and supported by Evidence Action district coordinators (Annexure G2)." Pg 13. Troubleshooting with telecallers: "Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." Pg 17. Monitoring: "In order to fulfil this need, Evidence Action worked intensively with the state health, and education, departments to ensure quality planning and implementation of the deworming program." Pg 15. 4 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, email to GiveWell, June 9, 2016 5 "Additionally the national government of India has expressed interest in possibly implementing a deworming program nationwide, rather than waiting for each state to launch separate programs. They have expressed interest in receiving technical assistance from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2. "On National Deworming Day, 12 states in India selected for inclusion in the program’s first phase will be required to administer deworming drugs to schoolchildren, regardless of measures of worm prevalence. National Deworming Day was originally planned for October 2014, but was delayed after Deworm the World staff alerted the government that drug supplies were insufficient. The Indian government has publically announced that National Deworming Day will be in February 2015, but it is possible that there will be further delays. "The target of the National Deworming Day is children who are 1-19 years old. In the first year, the national government is not advising states to highly prioritize deworming treatment in preschools, though some states were prepared and have already procured deworming medication." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1. "Deworm the World worked closely with the ministry to develop operational and financial guidelines, public awareness campaigns, and monitoring strategies for the program, based on the experiences of state-level deworming programs." Grace Hollister, email to GiveWell, June 9, 2016. The National Deworming Day was announced by India's national Ministry of Health and Family Welfare. 6 "India held its first National Deworming Day in February. Eleven states participated." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2. "11 dewormed in February; the 12th, Delhi, dewormed in April. This delay was due to conflicts of the February timing with state elections, which require the participation of teachers." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Grace Hollister, email to GiveWell, June 9, 2016 7 "February 10, 2015 was the first National Deworming Day in India, when over 89 million preschool and school-aged [sic] children were dewormed with our support. This led to many states recognising the importance of mass school-based deworming as a safe, cost-effective, and scalable health intervention. Since then, we have had an incredible year, with agreements signed with the states of Uttar Pradesh and Chhattisgarh, and renewed with the states of Bihar and Rajasthan. Our India program now extends technical assistance to seven states: Bihar, Rajasthan, Madhya Pradesh, Uttar Pradesh, Delhi, Chhattisgarh, and – the latest – Telangana." Evidence Action, blog post, December 21, 2015 Ms. Hollister told us that Deworm the World had also assisted Tripura during the National Deworming Day in 2016. Note that the level of support Deworm the World provides differs from state to state. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, email to GiveWell, June 9, 2016 8 "Recently completed worm prevalence surveys conducted by the National Center for Disease Control and Evidence Action revealed that 26 Indian states have such high worm prevalence to warrant twice annual treatment. The Government of India has decided to double up on the successes achieved during this year’s National Deworming Day with our support. "The next deworming day will take place in Indian schools and anganwadis on August 10." Evidence Action, blog post, August 1, 2016 Neetu Chandra Sharma, Daily Mail - India article, August 8, 2016 28 states were supposed to participate in the second round of deworming in 2016, but only 26 did. Deworm the World was supporting one of the states that did not implement deworming - Bihar. Bihar was unable to obtain drugs in time for the MDA. Deworm the World, National Deworming Day states, August 2016 9 "Deworm the World has phased out its comprehensive technical assistance to Delhi, but the national office will continue to provide minimal support, including attending meetings and doing some policy advocacy work. Deworm the World no longer has dedicated staff for Delhi." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 2 10 Grace Hollister, email to GiveWell, June 9, 2016 11 Note that Deworm the World is the fiscal agent controlling funding for the deworming program in Kenya (which is different from its role in India). This means that Deworm the World has additional responsibility in Kenya for ensuring that funds are used correctly. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Kenya’s National School-Based Deworming Program started in 2009. In 2012, it expanded to a nationwide program aimed at treating all at-risk Kenyan children each year for at least five years." Evidence Action website, Deworm the World Initiative (March 2016) "This actually happened in 2010, but took awhile for funding to come through, and the partnership was formalized at the end of 2011. The first round of the NSBDP took place in 2012." Grace Hollister, edits to GiveWell's review, November 7, 2016 Funders of the program discussed in Deworm the World staff, conversations with GiveWell, October 3-4, 2016 12 Grace Hollister, conversation with GiveWell, July 25, 2016 and Grace Hollister, conversation with GiveWell, August 11, 2016 13 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "We supported surveys for schisto with END Fund resources. Also, there is some additional mapping required in areas not previously surveyed that we are working with the NTD Unit to support and resource, to guide revised treatment strategy beginning in Year 6." Grace Hollister, edits to GiveWell's review, November 7, 2016 14 CIFF uses these prevalence surveys as a way to assess Deworm the World's work. We discuss the methodology and results of these prevalence surveys more here. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, conversation with GiveWell, June 13, 2016 Deworm the World has helped with prevalence surveys looking at schistosomes: "...we have funded schisto surveying through END Fund resources, and have helped develop a treatment strategy for schisto as well." Grace Hollister, email to GiveWell, June 9, 2016 15 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 16 In the first year that Deworm the World worked with Kenya, the MDAs were implemented in 12 different waves. In 2016, there will only be two waves - one in February and one in May. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Currently, implementation of the Kenya program is funded by CIFF and the END Fund, while the government provides support via its personnel. Deworm the World is in discussions about how the program will be funded once the current grant ends in 2017. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 17 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 18 For example, the Kenya team has supported (or will support) the Ethiopia, Nigeria, and Pakistan teams. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 19 Although the agreement that governs how SCI and Deworm the World assist with the MDAs is with the Federal Ministry of Health, SCI and Deworm the World also work to some extent with regional governments. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 20 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 21 "We are proud and excited to announce that Ethiopia's Federal Ministry of Health is launching a national deworming program targeting school-age children. Ethiopia has one of the highest burdens of neglected tropical diseases in the world, with over 10M children at risk for schistosomiasis and 18M children at risk for soil-transmitted helminths." Evidence Action, blog post, January 8, 2015 See Evidence Action, blog post, June 12, 2015 for some description of the roles Deworm the World and SCI played in this decision. The program is led by the Federal Ministry of Health with technical inputs from the Ethiopian Public Health Institute. Grace Hollister, email to GiveWell, June 9, 2016 22 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 23 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 24 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Implementation funds are provided to the Federal Ministry of Health, and funds for technical assistance provided to SCI, of which a portion funds Deworm the World's engagement. Grace Hollister, email to GiveWell, June 9, 2016 25 SCI works with the Expanded Programme on Immunization (EPI) to conduct the prevalence surveys. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Additionally, Deworm the World's Kenya team supports Ethiopia by leading training sessions and developing monitoring tools for Ethiopia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Note that Deworm the World's staff structure in Ethiopia is significantly different from that in Kenya or India: in Ethiopia, it only has one staff member, a consultant, who works closely with the SCI team. Grace Hollister, email to GiveWell, June 9, 2016 26 Grace Hollister, email to GiveWell, June 9, 2016 27 In the case of Pakistan, it was important to find a competent organization that Deworm the World could partner with, for two reasons: first, when entering a new country, it is easiest if Deworm the World can use its partner's financial systems and permissions to move funds to the deworming program (as opposed to Deworm the World attempting to set up an independent financial entity in the country); second, in Pakistan it is unclear how much capacity the government will have to offer to assist with deworming, so it may be important for partner organizations to provide that capacity. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 28 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 29 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 30 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 31 Deworm the World does not plan to test for prevalence in areas where there are security issues, it is logistically difficult to test for prevalence, or the agro-climatic region wouldn't support worms (e.g., the region is mountainous). Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, conversation with GiveWell, August 11, 2016 32 As of March 2016, Deworm the World was still in the process of obtaining government approval for the prevalence surveys, but hoped to start them in April and have results by June. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Factors causing the additional delays included: First, the Kato Katz kits required for the prevalence survey were held up at customs in Pakistan. Then, many schools in Pakistan closed early for the summer. Deworm the World had to wait until the schools opened again before it could begin the survey. As of mid-August, Deworm the World felt that prospects for beginning the survey by the end of the month looked promising. Grace Hollister, conversation with GiveWell, August 11, 2016 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 33 Grace Hollister, edits to GiveWell's review, November 7, 2016 34 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 35 Deworm the World expects this to make working in Pakistan more challenging, in part because the WHO (one of the largest donors of deworming drugs) is not able to work directly with non-national government representatives. Grace Hollister, conversation with GiveWell, August 11, 2016 Deworm the World expected this to occur. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 36 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Diagnostic kits were donated by the WHO. Grace Hollister, email to GiveWell, June 9, 2016 37 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 38 "In Nigeria, programs like deworming are typically implemented at the state level, with various donors and partner organizations supporting different states and targeting different NTDs. USAID and the Department for International Development are major donors. Partner organizations include RTI and Sightsavers, which often implement large-scale integrated NTD programs targeting several diseases." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg. 4. Because Nigeria is a country in which NTDs, including STH, are highly endemic, there are many NGOs already working there. However, because there is a heavy focus on integrated programs that address multiple NTDs at once, the states in Nigeria that are not as heavily afflicted by multiple NTDs are less likely to have an NGO partner. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016. 39 As of March 2016, Deworm the World had hired two staff members to work in Cross River. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 40 Grace Hollister, conversation with GiveWell, August 24, 2016. Note: Deworm the World has been discussing this request with the Federal Ministry of Health and has received permission to support school-based deworming programs in places where other NTD treatment programs are not yet occurring. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 41 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Initially, Deworm the World did not intend to open an office in Nigeria, so partnering with RTI also provided an official entity to work through and use for financial purposes, because RTI had established offices and financial systems in Nigeria. "Deworm the World does not plan to open its own office in Nigeria, but to work through a partner with an on-the-ground presence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg. 4. Grace Hollister, conversation with GiveWell, August 24, 2016 42 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, email to GiveWell, June 9, 2016 As of June 2016, the two organizations had established a contract governing their partnership, and memoranda of understanding were in place with the health and education sectors of the state government. Grace Hollister, email to GiveWell, June 9, 2016 Deworm the World has committed to work in Cross River with RTI until 2019; as of March 2016 it had allocated $1.36 million to this project for four rounds of deworming. Grace Hollister, conversation with GiveWell, August 24, 2016 and Evidence Action Q1 financials, 2016 If a local government area (LGA) is endemic for lymphatic filariasis (LF), then RTI will support treatment of LF in the LGA with a community-based distribution of albendazole. Subsequently, Deworm the World will not support a school-based deworming program in the same LGA, because albendazole will already have been distributed to the population. It can be difficult to achieve high coverage rates with community-based MDAs, so Deworm the World and RTI plan to examine their monitoring results to inform future treatment strategies. If coverage rates are low, they may intensify community sensitization efforts; it is highly unlikely that they will switch to a school-based deworming program in an LGA that has a community-based program for LF. Grace Hollister, conversation with GiveWell, August 24, 2016 If Deworm the World and RTI are both supporting programs in the same LGA, the community-based MDA will typically occur several weeks after the school-based MDA. Grace Hollister, conversation with GiveWell, August 24, 2016 43 "This week, the Cross River State Ministry of Health’s Neglected Tropical Diseases (NTD) unit launched its inaugural statewide school-based deworming exercise that will treat against two neglected tropical diseases that are particularly common in children: schistosomiasis and soil-transmitted helminthiasis (STH). The school-based deworming exercise will cover 11 of the 18 local government areas in Cross River for the first time, and is targeting 600,000 at-risk school-aged children in primary and junior secondary public and private schools. Other NTDs endemic to the state (lymphatic filariasis and onchocerciasis) will be treated through a community-based approach, according to standard practice." Evidence Action, blog post, June 30, 2016 44 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 45 Grace Hollister, conversation with GiveWell, August 11, 2016 and Grace Hollister, conversation with GiveWell, August 24, 2016 As of March 2016 Deworm the World had hired a consultant to work in Abuja to build federal relationships and start working on business development activities. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Deworm the World has started discussions with national government about creating an STH and schistosomiasis action plan, and it is considering expanding its support to 1-2 additional states next year. Grace Hollister, conversation with GiveWell, August 11, 2016 46 There are several advantages to registering in Nigeria, including: (1) it will be easier for Deworm the World to sign MOUs with states and the national government, (2) it will allow Deworm the World to bring funding into the country without working through another organization (Deworm the World has been working financially through its partner, RTI), and (3) registering signals to other organizations that Deworm the World intends to commit to working in Nigeria for a long time, which makes it more likely that Deworm the World will be included in NGO coalitions and discussions. The office will only serve Deworm the World; it is not intended to serve Evidence Action's other programs. The office will probably include one program manager per state that Deworm the World supports and may include a larger operations staff if Deworm the World decides to operate as its own fiscal agent (instead of operating through RTI). Grace Hollister, conversation with GiveWell, August 11, 2016 and Grace Hollister, conversation with GiveWell, August 24, 2016 47 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 48 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Thrive Networks has extensive experience in Vietnam in the water and sanitation, health and education sectors." Grace Hollister, email to GiveWell, June 9, 2016 Grace Hollister, conversation with GiveWell, August 11, 2016 49 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, email to GiveWell, June 9, 2016 50 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 51 Note that in the RCT evaluating the program, the control group will be dewormed along with the treatment group. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 52 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 and Grace Hollister, edits to GiveWell's review, November 7, 2016 53 "On April 28 and 29, more than 700,000 primary school children across four provinces in northern Vietnam will line up in their classrooms to receive a deworming tablet. 8.5 million children in Vietnam are at risk of parasitic worm infections that can harm their health, development, and school participation. Evidence Action’s Deworm the World Initiative supports the Government of Vietnam as it strengthens and improves school-based deworming to keep children healthy and in school. Between now and 2018, the program will distribute more than four million treatments to combat worms in the four provinces." Evidence Action, blog post, April 27, 2016 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 54 Grace Hollister, conversation with GiveWell, August 24, 2016 Deworm the World had originally predicted that all four provinces would need treatments two times per year. The survey results indicated that three of the provinces only need treatment one time per year, and one of the provinces only need treatment one time per every two years. Deworm the World, target populations and parasitology data 55 Grace Hollister, conversation with GiveWell, August 24, 2016 Deworm the World plans to support a second round of treatment in November 2016, as per the government's approved work plan, but going forward does not intend to support MDAs beyond what is recommended according to the WHO guidelines. Paul Monaghan, conversation with GiveWell, September 8, 2016 56 Deworm the World, Vietnam final report for STH survey in 21 provinces NIMPE has conducted prevalence surveys in Vietnam before, but the surveys that Deworm the World commissioned were designed in line with WHO guidelines and representative within each province. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "These surveys are taking place in areas of the country that are not currently treated for STH, and where albendazole administration for lymphatic filariasis ceased several years ago. The intent of the surveys is to determine whether there are areas of the country where STH treatment is required, and will [sic] contribute to developing an evidence-based treatment strategy for the country." Grace Hollister, email to GiveWell, June 9, 2016 57 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 58 "The control and prevention of STH in the 21 provinces will be used the MDA deworming guideline number 1932/QD-BYT date 19 May, 2016 of Vietnamese Ministry Of Health. If the prevalence of STH infection are > = 20% the MDA will be conducted two time per year as in Quang Ninh province. If the prevalences of STH infection are from 10% up to 20% the MDA deworming will be condcuted 1 time per year with Tra Vinh, Ninh Binh, Binh Thuan and Hung Yen provinces. The rest provinces, the prevalence of the STH infection < 10% the MDA will be carried out one time in two year." Deworm the World, Vietnam final report for STH survey in 21 provinces, Pg 151. 59 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 60 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 61 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 62 Deworm the World told us that it has heard that Nepal may already have a functioning deworming program that achieves high coverage. However, its previous conversations with the government in Nepal had indicated that the government was eager for technical assistance. Deworm the World still intends to resume the conversations eventually to clarify the situation, but it has recently been capacity-constrained due to the scale-up of National Deworming Day in India. Grace Hollister, conversation with GiveWell, August 11, 2016 63 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016. See the footnotes in our section on India above. 64 There is some chance that Deworm the World could partner with RTI again; RTI currently works on a lymphatic filariasis program in Indonesia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Deworm the World believes that only a low percentage of children at risk for STH are being treated in Indonesia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, conversation with GiveWell, August 11, 2016 65 Deworm the World provides both types of support in Nigeria and can imagine doing the same in Indonesia. Grace Hollister, conversation with GiveWell, August 11, 2016 66 Grace Hollister, conversation with GiveWell, August 11, 2016 67 "It is likely that >5% of the population receives LF treatment in India." Deworm the World, target populations and parasitology data. Also, see next footnote. 68 The intended treatment for LF in India is annual administration of DEC [Diethylcarbamazine] and Albendazole. Grace Hollister, edits to GiveWell's review, November 7, 2016 "Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease. Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except: - children below 2 years - pregnant women - seriously ill persons (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012. The WHO recommends annual albendazole treatment for areas with 20-50% prevalence of STH. Preventive chemotherapy in human helminthiasis 2006, Pg 41. According to the Indian government's program website, the LF treatment program has high coverage across many states (e.g., typically 80-90% in states carrying out the program), including Bihar, Madhya Pradesh, Uttar Pradesh, and Chhattisgarh. The LF program has recently been active in Bihar, Madhya Pradesh, Uttar Pradesh, and Chhattisgarh, but not Rajasthan or Delhi. LF treatment coverage 2015 "neither Rajasthan nor Delhi are endemic for LF; no LF MDA is necessary" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Deworm the World believes that LF treatments in Bihar and other states have not been delivered as effectively as planned; we do not have additional information about LF treatment delivery in other states beyond limited publicly available data. "The Bihar program was originally designed so that one round of albendazole administration would come from community-based lymphatic filariasis (LF) treatment (LF is endemic throughout the state) and a second round would come through the school-based deworming program. For a variety of reasons, LF treatment has not been consistently implemented. It is possible that more consistent albendazole administration as part of LF treatment would have resulted in a more significant drop in STH prevalence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 2. "Kenya and India are the countries where Deworm the World operates two of its biggest programs and where Ms. Hollister has the greatest familiarity with the state of LF treatment. In both countries, LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015, Pg 1 "Deworm the World believes that LF treatments in Bihar and other states have not been delivered as effectively as planned" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Data on reported coverage of LF treatment programs here: LF treatment coverage 2015 69 "LF is endemic in 6 coastal-region counties in Kenya, all of which are treated under the NSBDP. This accounts for >5% of the total target population. However, LF MDA has been highly irregular and has not achieved significant coverage (50% or less in the operating years of the NSBDP)." Deworm the World, target populations and parasitology data 70 "As of 2015, an estimated 112 woredas were endemic for LF. 102 of these are co-endemic for STH. However, only 53 woredas were treated for LF in 2015. The new (2016-2020) NTD Strategic Plan for Ethiopia indicates that MDA in LF-endemic districts 'will be integrated with STH so as to increase efficiency and reduce the quantity of albendazole tablets required.'" Deworm the World, target populations and parasitology data 71 "The integrated NTD program in Cross River is designed to leverage LF treatment in LGAs co-endemic for STH; this applies to 3 LGAs." Deworm the World, target populations and parasitology data. The spreadsheet shows which types of MDAs are planned to occur in each LGA (the type of MDAs planned depend on which NTDs are endemic). Notice that where an LGA is endemic for LF, albendazole is distributed in a community-based MDA and there is no treatment for STH. However, in LGAs where STH is endemic but LF is not, the plan is to support school-based MDAs that distribute mebendazole. If an LGA is endemic for LF, treatment for LF will occur in that LGA and a school-based MDA to treat for STH is not required (because LF treatments also treat for STH). Grace Hollister, conversation with GiveWell, August 24, 2016 72 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "LF MDA has stopped in Vietnam, with TAS expected later this year. None of the provinces where we work is LF endemic." Deworm the World, target populations and parasitology data 73 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 74 DtWI Bihar 2015 Program report, Pg. 15 75 DtWI Rajasthan 2015 program report, Pg 14. 76 DtWI Madhya Pradesh 2015 program report, Pg 15. 77 DtWI Chhattisgarh 2015 coverage validation report, Pg. 3 78 DtWI Delhi 2015 program report, Pg 14 79 DtWI Bihar 2014 program report, Pg 22. Pgs 23-30 describe activities over the various days. 80 DtWI Delhi 2013 program report, Pg 12 81 DtWI Rajasthan 2013 program report, Pg 11 82 DtWI Bihar 2012 monitoring report, Pg 1. 83 DtWI Bihar 2012 monitoring report 84 DtWI Delhi 2012 program report, Pgs 7-8 85 DtWI Delhi 2012 program report, Pgs 17-22 86 DtWI Rajasthan 2012 monitoring report, Pg 1. 87 DtWI Rajasthan 2012 monitoring report 88 DtWI Bihar 2011 program report, Pg 9. 89 DtWI Bihar 2011 program report 90 DtWI Bihar 2011 prevalence survey report DtWI Bihar 2015 Prevalence Survey report DtWI Rajasthan 2012 prevalence survey report DtWI Rajasthan 2013 prevalence survey report (Deworm the World re-surveyed Rajasthan because there were methodological weaknesses in the first prevalence survey) DtWI Delhi 2012 prevalence survey report Deworm the World notes: "Follow up surveys should take place after multiple rounds of treatment (WHO guidance is between three and six years of treatment), hence why only one follow-up has taken place to date." Grace Hollister, email to GiveWell, June 9, 2016 91 Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected." DtWI Bihar 2011 prevalence survey report, Pg 3. "Predicted prevalence is modelled using survey data from 2010-2011, point-level environmental data (annual temperature and rainfall) and a district level proxy for socio-economic status (literacy)." DtWI Bihar 2011 prevalence survey report, Pg 5. 92 "The prevalence of STH (including hookworm, A. lumbricoides, and T. trichiura) across these states in Bihar was 67.5% with district prevalence ranging from 49.0% to 79.6%." DtWI Bihar 2011 prevalence survey report, Pg 4. 67% = 782/1159. From "Table Three: Cumulative prevalence of each species by district": "Total - Number Students" = "1159"; "Total - % Any infection" = "(782) 67.5%". DtWI Bihar 2011 prevalence survey report, Pg 4. Selection of districts and schools: "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected from the state schools database." Pg 9. Selection of students: "Within each school 65 children aged 6 and above representing both sexes equally from class 1 to 6 were randomly selected, class by class, using random number tables." Pg 9. Technique: "Screening of infection for STH was based on a double Kato-Katz smear of 41.7 mg prepared from fresh stool samples." Pg 11. Participation rate: "From a total of 1,281 school children registered in the survey and provided with pots, 1,159 returned samples." (90.5%) Pg 11. Note on two stages of first prevalence survey based on DtWI Bihar 2011 prevalence survey report and Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 Based on the timing of the decision to deworm in Bihar, there was only time to collect prevalence surveys from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet DtWI's typical standard of quality. Before the DtWI-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report, Pg 9. Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness. (Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013) 93 "The prevalence of STH (including hookworm, A. lumbricoides, and T. trichiura) across these states in Bihar was 67.5% with district prevalence ranging from 49.0% to 79.6%." DtWI Bihar 2011 prevalence survey report, Pg 4. 94 Albendazole is used for treatment of both lymphatic filariasis (where it is used in combination with either DEC [Diethylcarbamazine] (as in India) or ivermectin) and of STH. Therefore, a round of treatment for lymphatic filariasis also treats STH. In this review, the term "deworming round" includes any MDA of albendazole. "A Memorandum of Understanding (MOU) was signed on March 5, 2010 among State Health Society Bihar, Bihar Education Project Council, and Deworm the World Initiative to implement the school-based deworming program in the state for treatment of STH. Based on Prevalence Survey findings, which suggested treatment recommendation of WHO, the Government of Bihar decided to implement biannual state-wide deworming beginning in 2011. Since then, Evidence Action has extended technical assistance to an annual round of deworming for all school-age children through a school-based model. The second round of deworming treatment was provided through the National Filaria Control Program (NFCP), which conducts annual mass drug administration of albendazole to the 2 years and above population at the community. In 2011, a total of 16.7 million children were dewormed at schools, earning the distinction of being the world's largest school-based deworming program. In 2012, 16.33 million children were dewormed in Round 2. 17.47 million, including 16.2 school-age children, were dewormed as part of Round 3 in 2014." DtWI Bihar 2015 Program report, Pg 6. "Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012. Four deworming rounds or distribution of albendazole during LF-MDA were missed during the period" DtWI Bihar 2015 Prevalence Survey report, Pg 5. 95 "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5. "The ‘2014’ [LF-MDA] occurred in February/March 2015. There was no LF MDA in the calendar year of 2014. But, since the LF MDA occurred prior to the closing of the 2014 - 2015 financial year (March 31, 2015). This is apparently common practice and acceptable to the government of India, and that is what is reported on the website. There was no LF MDA in the calendar year of 2014, as there was no DEC [Diethylcarbamazine] in Bihar until February 2015. Therefore a round did not occur prior to the second survey." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 96 Prevalence figures "On the basis of the collected and analyzed data, the overall weighted prevalence of any STH in Bihar was calculated as 35%. Prevalence in different agro-climatic zones ranged from 20% to 50%" DtWI Bihar 2015 Prevalence Survey report, Pg 2. Representative survey design "Per the 2011 census, there were 18,884,945 children aged 5-10 years in Bihar. Assuming one sentinel school is required for 300,000 targeted children, we needed at least 63 schools (rounded to 65) to monitor the performance of the deworming program. Collecting samples from 50 children per school, the minimum sample size required for estimation of prevalence using the sentinel site method was 3,250. Bihar is divided into three agro-climatic zones (Figure 2) – the North West Alluvial Plains (Zone 1) consisting of 12 districts, the North East Alluvial Plains (Zone 2) consisting of 9 districts and South Bihar Alluvial Plains consisting of 17 districts (Zone 3). The three zones respectively accounted for 42%, 22% and 36% of 5-10 years population in the state. NIE randomly selected 65 schools from the three zones, proportionate to the percentage of 5-10 year population in each of the zones. Thus, the study needed 28, 14 and 23 schools from zones 1, 2 and 3 respectively. NIE followed a 2-stage sampling procedure for selecting sentinel schools. In the first stage, NIE randomly selected 14 districts from Bihar (six from zone 1, three from zone 2 and six from zone 3) to meet logistical (teams could not spend too much time traveling and setting up temporary field laboratories, because it reduced the number of samples the teams could analyze), geographic dispersion (the survey needed to be geographically dispersed to ensure the best estimates of prevalence), and time constraints (there were only 20 days available for the survey). In the second stage, NIE line-listed all the primary schools (with the total strength of ≥ 60 children) of the districts selected from each zone. They then selected the required number of schools for each zone randomly from the list of schools in the selected districts. To select the required number of children, we assigned a random number (between one and five) to classes of the selected school. The survey was initiated from the class corresponding to the random number assigned for the school and field teams enumerated the children present in the class starting from roll number one on the attendance register. If the number of children in the selected class was &lt 50, children from the next class were selected. This procedure was followed until 50 children from each school were selected." DtWI Bihar 2015 Prevalence Survey report, Page 8. 97DtWI Bihar 2015 Prevalence Survey report 98 Deworm the World has supported three MDAs in Bihar over the last four years: "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5. 99 The National Vector Borne Disease Control Programme LF treatment coverage 2015 The treatment for LF in India is annual administration of DEC [Diethylcarbamazine] or DEC and Albendazole: "Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease. Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except: - children below 2 years - pregnant women - seriously ill persons (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012. "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5. 100 "Revised Strategy Annual Mass Drug Administration with single dose of DEC [Diethylcarbamazine] was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease. Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except: - children below 2 years - pregnant women - seriously ill persons (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012 The National Vector Borne Disease Control Programme LF treatment coverage 2015 101 "The ‘2014’ MDA occurred in February/March 2015. There was no LF MDA in the calendar year of 2014. But, since the LF MDA occurred prior to the closing of the 2014 - 2015 financial year (March 31, 2015). This is apparently common practice and acceptable to the government of India, and that is what is reported on the website. There was no LF MDA in the calendar year of 2014, as there was no DEC [Diethylcarbamazine] in Bihar until February 2015. Therefore a round did not occur prior to the second survey." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 "Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012. Four deworming rounds or distribution of albendazole during LF-MDA were missed during the period" DtWI Bihar 2015 Prevalence Survey report, Pg 5. "The Bihar program was originally designed so that one round of albendazole administration would come from community-based lymphatic filariasis (LF) treatment (LF is endemic throughout the state) and a second round would come through the school-based deworming program. For a variety of reasons, LF treatment has not been consistently implemented. It is possible that more consistent albendazole administration as part of LF treatment would have resulted in a more significant drop in STH prevalence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 2 "Kenya and India are the countries where Deworm the World operates two of its biggest programs and where Ms. Hollister has the greatest familiarity with the state of LF treatment. In both countries, LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015, Pg 1 102 39% = 16801/12090 - 1 2010-2011 "PER CAPITA NET STATE DOMESTIC PRODUCT AT FACTOR COST - Constant price... Base : 2004-05)": 12090 billion Rupee 2014-2015 "PER CAPITA NET STATE DOMESTIC PRODUCT AT FACTOR COST - Constant price... Base : 2004-05)": 16801 billion Rupee Reserve Bank of India, GDP per capita, Table 10, September 16, 2015 103 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "The M&E includes a series of pre- and post-intervention, repeat cross-sectional surveys in a representative, stratified, two-stage sample of schools across Kenya. District stratification was based on both geography and anticipated infection prevalence. The programme contains three tiers of monitoring: i) a national baseline survey including 200 schools in 20 districts, which aims to establish an accurate national measurement of infection levels; ii) surveys conducted pre and post intervention (pre-post surveys), which monitor 60 of the 200 schools before and immediately after the deworming activity to evaluate reductions in infections that can be directly attributed to programme implementation; and iii) high frequency surveys in 10 schools, distinct from the 60 pre-post schools, at four time points in a single year, before, during, and after treatment (Figure 1). Two hundred schools were examined at baseline and will be re-examined in year 3 and 5 in order to monitor long-term changes in worm infection at a national level both in terms of prevalence and intensity of infection. This sample size was chosen in order to be able to detect a five-percentage-point change in prevalence across years, assuming power β = 0.80 and test size α = 0.05, and considering the anticipated variance in prevalence. Sixty schools (a subset of the 200) will be surveyed every year for 5 years, before each treatment round to evaluate programme impact and 3–5 weeks post-treatment to evaluate treatment efficacy [2]. The same schools will be surveyed each year in the 60 pre-post survey schools, whereas in the remaining 130 schools, a different random sample of schools will be undertaken each year. In the 10 high frequency schools, a cohort of children will be followed-up longitudinally and assessed for haemoglobin concentration in addition to parasitological outcomes." Mwandawiro et al. 2013 Pgs 1-2. 104 Mwandawiro et al. 2013 KEMRI prevalence report - Year 2 Deworm the World, Kenya 2014-2015 program report, Pgs 13-14 105 See Mwandawiro et al. 2013. Because this is a published paper that describes the methodologies of the prevalence surveys (sampling, timing, outcomes measured, etc.) over the course of Kenya's five-year deworming program, we consider this to effectively be a pre-registration. 106 Deworm the World does not have full details about KEMRI's prevalence surveys easily available: to avoid the possibility of introducing bias, KEMRI has kept its methodologies private (so that, e.g., Deworm the World would not learn which schools KEMRI is evaluating). We intend to pursue additional details in the second half of 2016. Grace Hollister, conversation with GiveWell, June 13, 2016 107 "Based on available data and predictive maps [9,10], STH was assumed to be endemic in 66 districts. From these districts, grouped into strata, 20 districts were randomly selected for M&E in the first sampling stage, with number of districts per province proportional to population: six districts from Western Province, three from the Rift Valley, five from the Coast, and six from Nyanza (Table 1)." Mwandawiro et al. 2013 Pg 3. All 66 districts in which STH was assumed to be endemic are supported by Deworm the World: "From the year 2012, the ministries of health and of education of Kenya plan to deworm all school –age children who live in 66 districts identified as having a high prevalence of soil-transmitted helminth (STH) infection and schistosomiasis in four provinces." KEMRI prevalence report - Year 2 Pg 2 Deworm the World told us that the KEMRI surveys are only occurring in districts funded by CIFF. CIFF funds most of the districts in the national deworming program, but the END Fund also funds several districts. Grace Hollister, conversation with GiveWell, June 13, 2016 Kenya's national deworming program supports more than 66 districts: Deworm the World told us that "districts" were recently re-divided into "sub-counties" (with each sub-county being approximately the same size as districts were previously). Grace Hollister, conversation with GiveWell, June 13, 2016 In Year 3 of the Kenya program, Deworm the World supported MDAs in 111 sub-counties. Deworm the World, Kenya 2014-2015 program report, Pg 10. "[GiveWell]: How were the 66 districts (from which the 20 districts used in the survey were randomly selected) initially selected? [Charles Mwandawiro]: They 66 districts were picked from a map developed using historical data (research studies and MoH data). These were given by the MoH and confirmed by a team in a meeting that they were the districts which CIFF wanted to be covered (where worm infections were definitely known to occur)" Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 If the 66 surveys were not selected randomly, this could cause the prevalence surveys to be somewhat biased. For example: If only districts with high prevalence were chosen, it could be the case that the schools surveyed are the schools most incentivized to implement deworming effectively. Or, if only districts with high prevalence were chosen, perhaps children in such districts would be more willing to take deworming pills (it is plausible that children in higher prevalence districts might feel noticeably better after deworming, and therefore be more cooperative when taking the deworming pills each year). If the 66 districts were chosen based on ease of access to schools (e.g., to make surveying easier), the surveys might only be evaluating the locations in which deworming is easiest to implement. "[GiveWell]: Are there ways in which these surveys might not accurately reflect the impact of the program that we might not currently understand? [Charles Mwandawiro]: Yes. For example there could be other areas (out of the 66 districts) that have infections but not covered by the programme. Such areas could be Nairobi (unprogrammed deworming) and parts of Rift Valley. Otherwise, by and large, the surveys reflect the impact of the programme as designed." Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 108 200 schools were randomly selected from the 20 districts for the baseline survey in early 2012, and 60 of those schools are re-surveyed before and after their MDAs each year. See Figure 1 of Mwandawiro et al. 2013, Pg 3. Note the post-MDA survey is supposed to occur 3-5 weeks after the treatment. We do not know if the surveys occurred on the timeframes they were intended to, and we do not know how much time passes between the pre-MDA survey and the MDA itself. "At the second sampling stage, primary schools were randomly selected from within the chosen 20 districts." Mwandawiro et al. 2013 Pg 3. "The article describes the M&E design of the Kenya national SBD programme and presents results from the baseline survey conducted in early 2012." Mwandawiro et al. 2013 Pg 1. Of the remaining 140 schools, 10 will be surveyed both pre- and post- MDA in Years 1, 3, and 5, while 130 are surveyed only pre-MDA in Years 1, 3, and 5. See Figure 1 of Mwandawiro et al. 2013, Pg 3. Note that the 130 schools surveyed pre-MDA in Years 1, 3, and 5 are not the same schools each time; they are re-selected randomly: "The same schools will be surveyed each year in the 60 pre-post survey schools, whereas in the remaining 130 schools, a different random sample of schools will be undertaken each year." Mwandawiro et al. 2013 Pgs 2-3. In the 60 schools surveyed each year, the sampled children are randomly selected each year, meaning the surveys do not follow precisely the same population year-to-year. "In each school, 18 children (9 girls and 9 boys) were sampled randomly from each of six classes - one Early Childhood Development (ECD) class and classes 2–6 - using computer generated random number tables, for a total of approximately 108 per school. The sampling within these specified classes aimed to target children aged 5–16 years. " Mwandawiro et al. 2013 Pg 3. Also see Figure 1 of Mwandawiro et al. 2013, Pg 3. We believe the populations will be very similar, so we are unconcerned that different children are surveyed each year. The surveys measure the prevalence and intensity of both soil-transmitted helminths and schistosomes. "Stool samples were obtained for each child and two slides prepared and examined for the presence and intensity of STH species and S. mansoni using the Kato Katz method, with the concentration of eggs expressed as eggs per gram (epg) of faeces. Urine samples were obtained only from children in Coast Province (where Schistosoma haematobium is widespread) and investigated for presence and intensity of S. haematobium using the urine filtration method, with the concentration of S. haematobium eggs estimated in eggs per 10 ml urine. Egg counts were performed only up to 24,000 epg and 1,000 eggs/ 10 ml urine, respectively. Infection intensities above these values were, therefore, not further quantified." Mwandawiro et al. 2013, Pg 3. Additionally, in the 10 schools surveyed pre- and post-MDA every odd year, anaemia and educational outcomes are measured. We have not seen any results from the anaemia and educational outcome tests, but we only have results from Year 1 and Year 2 and we are not sure if these outcomes were measured in the first year. From Mwandawiro et al. 2013: "In the 10 high frequency schools, a cohort of children will be followed-up longitudinally and assessed for haemoglobin concentration in addition to parasitological outcomes." Pg 3. See Figure 1, Pg 3. "In the 10 “high frequency” schools, finger-prick blood samples were obtained and analysed using a HemoCue photometer (HemoCue, Angelhom, Sweden) to estimate haemoglobin concentration." Pgs 3-4. The survey protocol does not require researchers to provide treatment to students immediately after they are tested (we have seen this requirement in some of SCI's prevalence surveys): "[GiveWell]: When children are surveyed pre-MDA and found to have worms, are they then ethically required to be treated? Or does treatment occur during the MDA? [Charles Mwandawiro]: Ethically they are required to be treated but because of the number and geographical distribution they are all treated during the MDA and not separately." Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 109 The data in the table for soil-transmitted helminths is from KEMRI prevalence report - Year 2, Pg 4 and Deworm the World, Kenya 2014-2015 program report, Pgs 12-14. The data for schistosomiasis is from Table 4 of Mwandawiro et al. 2013, Pg 6 and Box 5 of KEMRI prevalence report - Year 2, Pg 15. Note that the two sources we've seen describing the results from the prevalence surveys do not exactly match. In Mwandawiro et al. 2013, the prevalence at baseline was 15.6% for hookworm, 18.0% for Ascaris, and 6.6% for Trichuris, whereas the KEMRI prevalence report - Year 2 shows the baseline prevalence being 16.9% for hookworm, 19.2% for Ascaris, and 5.4% for Trichuris. (Pg 5 and Pg 4, respectively) We are not sure where this discrepancy comes from, but the numbers are close enough that we don't find it too concerning. 110In years 1, 3, and 5, 10 schools (randomly selected from the 200 schools at baseline but distinct from the 60 schools that will be followed each year) will be surveyed both pre-MDA and post-MDA. These schools will be surveyed intensively, with three post-MDA surveys measuring for worm prevalence and additional variables, such as anaemia and educational outcomes. The same 10 schools will be surveyed for each of Year 1, 3, and 5, and the same students from each of the 10 schools will be surveyed. Mwandawiro et al. 2013, Pgs 2-3. 111 The Year 1 baseline data in the table is from Table 4, Mwandawiro et al. 2013, Pg 6. The Year 1 post-MDA data is from Table 11 of KEMRI prevalence report - Year 2, Pg 20. The report only gave prevalences by county, so we took the median of all counties and the median of counties within the Coast Province (see this Wikipedia article) for a rough comparison. The data pre- and post- the Year 2 MDA is from KEMRI prevalence report - Year 2, Pgs 15 and 18. The data for Year 3 is from Deworm the World, Kenya 2014-2015 program report, Pgs 12-14. Note that the two sources we've seen describing the results from the prevalence surveys do not exactly match. In Mwandawiro et al. 2013, the baseline Year 1 prevalence for S. Mansoni at baseline was 0% in the Coast Province and 2.1% in all provinces, while the prevalence for S. haematobium was 14.8% in the Coast Province (Table 4, Pg 6). However, in KEMRI prevalence report - Year 2, the baseline Year 1 prevalence is reported as 0.1% for S. mansoni in the Coast Province and 1.8% in all provinces (Box 5, Pg 15), while the prevalence for S. haematobium is reported as 18.0% (Pg 18). We are not sure where these discrepancies comes from. 112 S. haematobium was only measured in the Coast Province: "Urine samples were obtained only from children in Coast Province (where Schistosoma haematobium is widespread) and investigated for presence and intensity of S. haematobium using the urine filtration method, with the concentration of S. haematobium eggs estimated in eggs per 10 ml urine." Mwandawiro et al. 2013, Pg 3. "Urine samples were examined for S. haematobium infections in 9 schools in Coast Province in Kilifi and Kwale Counties." KEMRI prevalence report - Year 2, Pg 18. 113Table 4, Mwandawiro et al. 2013, Pg 6. 114 We were told that in the first year of Kenya's deworming program, the MDAs occurred in 12 different waves, as opposed to all on the same day. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "[GiveWell]: How much time passes between the initial survey and the MDA? How much time passes between the MDA and the follow-up survey? [Charles Mwandawiro]: We usually do 3-5 weeks pre-MDA and 5-6 weeks post-MDA because we do both schistosomiasis and STH" Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 It might be possible that the timing of the surveys could also affect observed outcomes; for example, worm burden might be especially high during some seasons in Kenya, so if pre-MDA surveys were conducted during high-burden seasons, while post-MDAs were conducted later in low-burden seasons, the effect of deworming might appear to be greater than it actually is. 115 See Figure 1 of Mwandawiro et al. 2013, Pg 3. This might be less of an issue in Year 3 and Year 5, when KEMRI plans to survey an additional 130 schools randomly selected from the same 20 districts. Note that the schools are not told that they are being monitored to assess the program: "[GiveWell]: Do teachers and administrators know that they are staffing the schools that are being used to assess the program? It seems possible that in the 60 schools that are re-surveyed each year, teachers may become aware of this. [Charle Mwandawiro]: The teachers were not told so. But they possibly know that their schools are being used as examples to monitor decline of infection. This however is yet to be determined and they is no reason so far to think so since we have not seen this reflected in what we observe and get." Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 116 For example, Deworm the World is using its unrestricted funding to support its program in Nigeria and expects to financially support MDAs in Pakistan. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 117 "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2. "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2. 118 Professor Devesh Kapur Biography 2013 119 Devesh Kapur conversation October 14th 2013 120 "Some school health programs, such as the WIFS program in Delhi, are rolled out too quickly and their trainings are carried out poorly. Then they receive negative media attention because of students’ adverse reactions to treatments. Negative media can cause delays or cancellation of school health programs. The WIFS program has also received negative media attention in the states of Haryana and Odisha." DtWI 2013 GiveWell site visit, Pg 10. 121 "There have been very few adverse reactions to deworming treatments in DtWI-supported states because of the quality of Deworm the World's trainings." DtWI 2013 GiveWell site visit, Pg 10. 122 "National funds for deworming are part of the [Weekly Iron Fortification Supplements] WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12. "Wherever possible, Deworm the World works to get deworming added as a line item in the budget. Otherwise, it is difficult to ensure that long-term funding will exist for the program." DtWI 2013 GiveWell site visit, Pg 10. "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2. 123 Alix Zwane conversation August 30th 2013 "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3. Assam has also announced its intentions to move forward with its deworming program in March 2013: "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. As reported by NRHM, Assam any [sic] school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam midday meal report 2013, Pg 11. According to Deworm the World, "No schools in Assam have been covered under the School Health Program to provide deworming in 2012/13. The programme was then re-scheduled to begin in March 2013. No evidence of this actually having taken place." DtWI Assam research 2013, Pg 1. "[Deworm the World is] also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2. Originally Deworm the World didn’t believe that Jharkhand conducted a school-based deworming MDA in 2013, but learned of it by early 2014. Alix Zwane conversation August 30th 2013 GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3. "Note that not all of these programs would have been operating at scale (the ones we supported were)." Grace Hollister, email to GiveWell, June 9, 2016 124 Deworm the World was not involved in Punjab, Jharkhand, nor Assam launching their statewide school-based deworming programs. "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3. "DtW has been involved in deworming programs in four different states, and is still actively involved of three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement." Grace Hollister conversation June 19th 2013, Pg 1. "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. "In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J-PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there. In August a memorandum of understanding (MoU) was formalized between DtW and the relevant players in Bihar (School Health Society Bihar and Bihar Education Project Council) for program implementation." Grace Hollister conversation June 19th 2013, Pg 2. "DtW leveraged its networks to engage the support of the Minister of Health, Women, and Child Welfare [in Delhi]. Education stakeholders were also brought in so that the program could reach beyond preschool to older grades as well. DtW conducted a great deal of advocacy, maintaining continuous interactions and significant support among all relevant departments to bring the program to fruition Due to the complicated agency system, there were many other government stakeholders as well, requiring a high degree of coordination by DtW to ensure program objectives could be achieved." Grace Hollister conversation June 19th 2013, Pg 3. "After the deworming program launched in Delhi, Rajasthan saw the results generated by the DtW- supported program in Delhi and committed to doing a deworming program, allocating funding for it in their budget. In March 2012 they brought DtW in to help. In this case, the state already knew what it wanted and already had a school health program. They sought DtW's technical expertise, mapping ability, general program support, stakeholder coordination, etc. Deworm the World coordinated signature of a MoU between the Departments of Women and Child Development, Education, and Health, UNICEF and DtW to guide program implementation, and helped establish of a technical secretariat housed within the Education Department." Grace Hollister conversation June 19th 2013, Pg 3. "DtW has worked in Delhi, Bihar, Rajasthan, and Andhra Pradesh (AP). • In Bihar, according to documentation provided by DtW, the program began because of previous Poverty Action Lab (J-PAL) work in Bihar. DtW also noted that Rajasthan approached them because of the success (as highlighted in the media) of the Delhi program. DtW supported a pilot program and prevalence survey in AP, and though DtW didn't continue to engage with AP, the AP government seems to have recently conducted a school-based deworming program. • More recently, Punjab approached DtW, but DtW didn't have the funding to move forward with Punjab, and Punjab is now implementing deworming on its own. It's not possible to be certain whether or not DtW has caused deworming to happen that otherwise would not have, but it's important to note that DtW took the opportunities presented to help run better programs where possible as well as improve data quality and did so on a limited budget. They are also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2. 125 GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 While 11 states and Delhi participated in the first NDD in 2015, all states are supposed to participate in the second NDD in 2016. "The NDD slated for February 2016 is supposed to cover all states in India, per the Ministry of Health and Family Welfare’s issuance of correspondence to state governments." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 126 "Keeping in view the problem of worm infestation of children in Assam it has been decided to conduct two rounds of De-worming every year in the month of March and September. The first round will be conducted during March’ 2010. On every Wednesday (VHND/Immunization Day) medicine for De-worming will be given to all children between the age group 1 to 5 years along with vitamin A supplementation." Assam 2010 guidelines for deworming, Pg 1. 976,192 "children below 5 years provided vitamin A syrup" as of November/December 2010 Assam reproductive and child health 2011-2012, Pg 73. The National Rural Health Mission reported the progress for 2010-2011 as 92,957 "Students given IFA/ de-worming tablets". Assam state programme implementation plan 2011-2012, Pg 21. Assam is "supposed to provide deworming to all children, but AWC services provided in Assam are some of the worst performers for deworming coverage." DtWI Assam research 2013, Pg 1. 127 "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. NB. As reported by NRHM, Assam any school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam midday meal report 2013, Pg 11. 128 Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013. "Deworm the World continues to support the four states it was working in previously (Rajasthan, Delhi, Bihar, and Madhya Pradesh) and has added programs in Chhattisgarh and Uttar Pradesh. [...] Deworm the World is in discussions with Telangana [...] Deworm the World has phased out its comprehensive technical assistance to Delhi, but the national office will continue to provide minimal support, including attending meetings and doing some policy advocacy work. Deworm the World no longer has dedicated staff for Delhi." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pgs 1-2 129 DtWI 2013 GiveWell government interviews. 130 "If Deworm the World had not been there, how would the deworming program be different? Since Deworm the World have already implemented deworming programs in Delhi and Tamil Nadu, they brought experience. If a state government decides to do something, nothing is impossible. However, Deworm the World's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have Deworm the World's experience, there could have been more problems. The government received important support from DtWI, particularly with distributing the deworming tablets, implementing the trainings, and developing training materials." "Have you ever wanted to do a program but been unable to find a technical partner? Ms. Gupta is still looking for a technical partner on specific issues. For example, Rajasthan has a large population with special needs. It has generally been able to find partners for helping the visually impaired, but very few organizations work with autistic children, children with cerebral palsy, or children with mental retardation. If there were a partner to support the Rajasthan government in this area, this would be very welcome. The Rajasthan government has funds for this type of program, but are looking for a technical partner. An NGO called Sight Savers works with blind children. Before Sight Savers, she was not aware of problems with low vision. Many children need magnifying glasses and large print books. Sight Savers helped them to identify doctors and hospitals to work with such children." DtWI 2013 GiveWell government interviews, Pg 7-9. 131 "Additionally the national government of India has expressed interest in possibly implementing a deworming program nationwide, rather than waiting for each state to launch separate programs. They have expressed interest in receiving technical assistance from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2 Grace Hollister, email to GiveWell, June 9, 2016 132 "Deworm the World Initiative at Evidence Action will be the technical assistance partner to MoHFW, Government of India. The specific responsibilities are as follows: Support in development of National Deworming Day implementation strategy Design and develop training and reference materials, community mobilization strategies for increased awareness and coverage of target beneficiaries, Monitoring and Evaluation (M&E) Systems and reporting formats" India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015, Pg 10. "Deworm the World has three primary roles in National Deworming Day: • Agenda setting: Deworm the World has played a large role in getting deworming on the national government’s health agenda, which led to the creation of National Deworming Day. Before Deworm the World began working with the national government, deworming was officially a part of some health programs, but was inconsistently implemented outside of those states where Deworm the World provides technical assistance. • Developing materials and advising the national government: Deworm the World has also developed operational guidelines for program implementation, including training materials, public awareness materials, monitoring forms, guidance for teachers and health workers, and a Frequently Asked Questions guide about National Deworming Day, in partnership with the national government. The national government will distribute these materials to state governments for use on National Deworming Day; some materials will be adapted to state-specific contexts. Deworm the World’s work plan also advised the national government on appropriate timing for workshops and trainings, and plans to conduct a training for state-level functionaries once the date of deworming is announced. Deworm the World’s India Country Director Priya Jha speaks with Dr. Khera of the Child Health Division at the Indian Ministry of Health and Family Welfare on a weekly basis about plans for National Deworming Day. • Direct work with state governments: Deworm the World will work individually with the states Bihar, Rajasthan, Delhi, and Madhya Pradesh to adapt National Deworming Day guidance into those programs. Deworm the World has been working in the first three of those states for multiple rounds of deworming. Funding for working in Madhya Pradesh comes from a USAID grant, for which this will be the first round of deworming support. Deworm the World also hopes to work in Chhattisgarh with additional funds from USAID, and Odisha and Uttar Pradesh with funding from the Children's Investment Fund Foundation. Other than developing operational guidelines and materials, and providing highlevel training for key state functionaries, Deworm the World does not have the capacity to be involved in the first phase of National Deworming Day in other states in India. Deworm the World had originally planned to create a national implementation and monitoring workshop for representatives from each Indian state prior to National Deworming Day. However, it is too late to host the workshop before the first National Deworming Day in February 2015. Editor’s note (based on updates after this conversation): Deworm the World did provide some support to Chhattisgarh and supported the national government in hosting a workshop." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pgs 1-2 "Additionally the national government of India has expressed interest in possibly implementing a deworming program nationwide, rather than waiting for each state to launch separate programs. They have expressed interest in receiving technical assistance from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2 "The Indian national government has recently demonstrated an increased interest in deworming programs. The Child Health Division of the Ministry of Health & Family Welfare (MoHFW) has been particularly interested in deworming. Deworm the World Initiative (DtWI)’s program in India is working with Ajay Khera, the leader of the Child Health Division, and other staff members in the division to encourage the division to continue promoting deworming. DtWI has also been meeting with the Joint Secretary of MoHFW and requesting appointments with the Minister of Health & Family Welfare. Deworming has been proposed to be included in the 100 Day Agenda of the new administration in India. DtWI is a stakeholder supporting this effort. DtWI has proposed to assist with several aspects of the program including planning, guideline development and other technical issues." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014, Pg 1 "States which did not receive technical support from Deworm the World may not have had adequate time or support for planning the February deworming round. This may have resulted in lower coverage and weaker monitoring. Some states would likely appreciate assistance from Deworm the World or another similar organization." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2 133 India NDD documents 2015 134 "National Deworming Day was originally planned for October 2014, but was delayed after Deworm the World staff alerted the government that drug supplies were insufficient. The Indian government has publically announced that National Deworming Day will be in February 2015, but it is possible that there will be further delays." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1 "Deworm the World had originally planned to create a national implementation and monitoring workshop for representatives from each Indian state prior to National Deworming Day. However, it is too late to host the workshop before the first National Deworming Day in February 2015. Editor’s note (based on updates after this conversation): Deworm the World did provide some support to Chhattisgarh and supported the national government in hosting a workshop." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 2 135 "A program of the Government of India, the February 10 event in eleven states targeted 140 million children with school-based deworming treatment." DtWI NDD blog post February 2015 136 "On National Deworming Day, 12 states in India selected for inclusion in the program’s first phase will be required to administer deworming drugs to schoolchildren, regardless of measures of worm prevalence." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1 "India held its first National Deworming Day in February. Eleven states participated." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2. "11 dewormed in February; the 12th, Delhi, dewormed in April. This delay was due to conflicts of the February timing with state elections, which require the participation of teachers." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 137 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Grace Hollister, email to GiveWell, June 9, 2016 138 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 139 Grace Hollister, edits to GiveWell's review, November 7, 2016 140 RTI is funded primarily by ENVISION (of USAID), which does not typically fund STH and schistosomiasis treatments unless they are integrated with other NTD treatments. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 141 This understanding is from several conversation with different organizations. 142 Deworm the World, SCI, Sightsavers, and END Fund have all told us that they would could additional funding to expand deworming in Nigeria. 143 This understanding is from several conversation with different organizations. 144 For example, by helping governments obtain drugs in a timely manner: "in some cases we have improved the likelihood that sufficient drugs were available in a timely manner for program rollout; some of that is through support to govts in accessing the WHO donation program, other is due to pushing govts to procure needed drugs in a timely manner." Grace Hollister, edits to GiveWell's review, November 7, 2016 "DtWI's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have Deworm the World's experience, there could have been more problems." Veenu Gupta, Principal Secretary to School Education Department, Government of Rajasthan DtWI 2013 GiveWell government interviews, Pg 7. 145 "DtWI has helped to improve deworming trainings. Last year, Deworm the World helped to develop the content for the trainings. Deworm the World made the content more concise and easy to understand than the government would have done on its own. Deworm the World also improves trainings by tracking when people do not show up to trainings and following up with them and by determining aspects of training that could be improved in future years." Girish Bharbwag, Nodal Officer in Rajasthan DtWI 2013 GiveWell government interviews, Pg 4. "And also to note that some programs administering deworming medication do not provide any training to teachers and/or health workers prior to drug distribution" Grace Hollister, edits to GiveWell's review, November 7, 2016 146 The District Coordinator position, for example, is not always staffed in other school-based health programs. "The state government paid for District Coordinators for the hand washing program in 2008. However, the payment for this position was low. In the first year, there were 15-16 independent monitors. This year, there are only 7 District Coordinators. The government is finding it difficult to fill these vacancies." DtWI 2013 GiveWell government interviews, Pg 5. "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 4. 147 "The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 5. 148 In Bihar, Deworm the World says that it has convinced the state to expand its program to include preschool-aged children in the next round of deworming, set for early 2015. GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 The MDA that Jharkhand implemented before it was in contact with Deworm the World only targeted 10-19 year olds: "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3. 149 Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 150 DtWI Monitoring Improvements 2014 DtWI Bihar 2014 program report DtWI Delhi 2013 program report DtWI Rajasthan 2013 program report GiveWell DtWI 2013-2014 cost data summary 151 Grace Hollister, edits to GiveWell's review, November 7, 2016 152 Conversation notes here. 153 Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16 Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15 Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15 Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17 154 See citations in previous footnote. "These calls helped to uncover problems, which are reported on the same day to the appropriate government officials to address" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 We have not seen any detailed data of the calls made or the issues uncovered or whether they are resolved. We are only aware of one specific example of the type and magnitude of problems uncovered: In the Bihar 2014 program, Deworm the World-led monitoring uncovered that 18% of schools that were called had not received the drugs at the scheduled time, and it seems plausible that the issue was largely addressed before deworming day (though we have not vetted detailed data supporting this). "BEO [Block education officer] offices distributed the drugs among the school headmasters at block level trainings. In monitoring phone calls, 82% of the 354 schools contacted by Deworm the World tele-callers across 27 districts confirmed receiving drugs two days prior to Deworming Day. When this potentially problematic information came to light, the tele-calling team and DCs hired by Deworm the World contacted all the BEO offices to ensure delivery of drugs to all the schools before Deworming Day along with instructions issued from the BEPC nodal officer. Subsequent independent monitoring data (from visits to schools during Deworming Day and Mop-Up Day) shows that approximately 96% schools had received drugs by Deworming Day. This was a significant jump from the 82% polled only two days earlier." DtWI Bihar 2014 program report, Pg 13. 155 Our observations are noted here: DtWI 2013 GiveWell site visit. 156 In 2014, Deworm the World reported improvements to the training process, including focusing on areas where problems were identified in previous rounds, simplifying materials, and testing if participants are learning key messages. DtWI Monitoring Improvements 2014, Pgs 1-2. 157 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 158 The prevalence surveys also examined more worm species than were necessary from a public health perspective. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 159 Monitors find that the number of students dewormed according to school records was greater than the number according to those schools' classroom records for each state that Deworm the World supported in 2015; the median over-report amount was 15%. Process: For each state for which we have 2015 deworming coverage validation survey data, we calculated a state-level inflation rate according to the surveyors' observations. We calculated inflation rate as the difference between reported treatments and recorded treatments, out of the total recorded treatments: (treatments reported on the school reporting form​ minus treatments recorded as tick marks in class records)/(treatments recorded as tick marks in class records). In the raw dataset, some entries are marked as missing data using error codes described in the associated code sheet; these were treated as contributing zero tickmarks or zero reported treatments (as appropriate) to the total count. Bihar: 22% inflation rate: (141,377-115,889)/115,889 DtWI Bihar 2015 monitoring data for coverage validation, schools (Deworm the World also reports this figure as 22%: "The state level verification factor for Bihar was found to be 0.81972, indicating that for every 82 enrolled children who were recorded as deworming in the schools, the school reported that 100 enrolled children had been dewormed. This corresponds to an overall 22% inflation of reporting in the state, meaning that reported numbers appear to be approximately 22% higher than the numbers recorded in attendance registers." DtWI Bihar 2015 Program report, Pg 19) Rajasthan: 15.0% inflation rate: (42,845-37,256)/37,256 DtWI Rajasthan 2015 monitoring data for coverage validation, schools Delhi: 1.9% inflation rate: (170,060-166,956)/166,956 DtWI Delhi 2015 monitoring data for coverage validation, schools Madhya Pradesh: 11.3% inflation rate: (39,983-35,912)/35,912 DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools Chhattisgarh: 100.0% inflation rate: (63,162-31,582)/31,582 DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools This over-reporting results from schools that reported deworming students in the school summary but did not mark any classroom records as the program protocol instructed. After excluding schools in which no classroom records were marked, the number of students dewormed according to the school records was similar to those schools' classroom records for the same states (the median over-report amount was 0.4%). Process: For each state for which we have 2015 deworming coverage validation survey data, we calculated a state-level inflation rate ​excluding schools at which no tick marks were recorded​ according to the surveyors' observations. We calculated inflation rate as the difference between reported treatments and recorded treatments, out of the total recorded treatments: (treatments reported on the school reporting form, in schools with any tick marks recorded in any class records​ minus treatments recorded as tick marks in class records)/(treatments recorded as tick marks in class records). In the raw dataset, some entries are marked as missing data using error codes described in the associated code sheet; these were treated as contributing zero tickmarks or zero reported treatments (as appropriate) to the total count. Bihar: -2.8% adjusted inflation rate: (112,653-115,889)/115,889. DtWI Bihar 2015 monitoring data for coverage validation, schools Rajasthan: 2.9% adjusted inflation rate: (38,318-37,256)/37,256 DtWI Rajasthan 2015 monitoring data for coverage validation, schools Delhi: 1.7% adjusted inflation rate: (169,748-166,956)/166,956 DtWI Delhi 2015 monitoring data for coverage validation, schools Madhya Pradesh: -5.8% adjusted inflation rate: (33,844-35,912)/35,912 DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools Chhattisgarh: 0.4% adjusted inflation rate: (31,694-31,582)/31,582 DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools 160 This ratio is called the "verification factor." Deworm the World calculates it for each of the MDAs it supports. Bihar: 76% Deworm the World, Bihar 2016 IMCV report, Pg. 21 Chhattisgarh: 76% Deworm the World, Chhattisgarh 2016 IMCV report, Pg. 22 Madhya Pradesh: 94% Deworm the World, Madhya Pradesh 2016 IMCV report, Pg. 22 Rajasthan: 91% Deworm the World, Rajasthan 2016 IMCV report, Pg. 25 Telengana: 53% Deworm the World, Telengana 2016 IMCV report, Pg. 13 Tripura: 88% Deworm the World, Tripura 2016 IMCV report, Pg. 27 Uttar Pradesh: 64% Deworm the World, Uttar Pradesh 2016 IMCV report, Pg. 24 161 "We don’t have a process that is part of every round. Data quality assessments are undertaken periodically in collaboration with the government – we have completed these in a few states and in other states DQAs are in process/planned. Sampling as part of the assessment includes schools, anganwadis, blocks, and districts. One of the key issues we have found to date is the lack of available documentation at each level." Grace Hollister, edits to GiveWell's review, November 7, 2016. Note that we have not yet asked Deworm the World to share results from any of its data quality assessments with us. 162 For example, the nodal headmasters add the school totals from their respective jurisdictions and report them to the block-level officials, who use those to calculate the total children dewormed in their block and report that figure to the district, who do the same in reporting to the state level, where the grand total is calculated. Bihar 2015: "With close support from our teams, the State Health Society Bihar and Bihar Education Project Council collected and compiled the coverage report for the round within the reporting timelines in the prescribed reporting format (Annexure H.1). Coverage reporting structure and timeline is shown below in Figure 4:" (see source for annexure and figure) DtWI Bihar 2015 Program report, Pg 16. Delhi 2015: "In this round, each school and anganwadi was supposed to fill a one-page reporting form (annexure III). In order to improve the accuracy of coverage reporting by the schools and anganwadis, every participating school and anganwadi was instructed to follow a recording protocol for deworming. Every teacher and anganwadi worker was required to put a single tick (√) next to a child’s name in the school/anganwadi register if they were administered albendazole on deworming day and double-tick mark (√√) if dewormed on mop up day. School headmasters and anganwadi workers were responsible to compile the number of dewormed children, fill the reporting format and submit it to the next level. Reporting structure of coverage data from schools and anganwadis and timelines are given in the below flow chart:" (see source for chart) DtWI Delhi 2015 program report, Pg 15. Rajasthan 2013: "School headmasters were required to hand over the completed reporting form to their respective nodal headmasters – a senior headmaster of a school looking after a cluster of schools – who in turn would submit these forms to the respective BEO office. Apart from the forms, the nodal headmasters also submitted the collated information. This information further gets consolidated first at block level, then at district level and finally at state level. Education department shared this compiled information with all the stakeholders." DtWI Rajasthan 2013 program report, Pg 14. Bihar 2014 was an exception; aggregation of data for that round of treatment was done centrally, by an independent firm. "In Round 3, each school was supposed to fill a one-page, simple school summary form (Form S), capturing only the essential details on the school such as total enrollment, total number of dewormed children by date and by enrollment status, number of adults dewormed, availability of drugs, drug usage and wastage, remaining drug stocks and contact details of the headmaster. In order to ensure that coverage reporting by the schools is accurate, every participating school was instructed to follow a special recording protocol for deworming. Every teacher was required to put a single tick mark (9) next to a child’s name in the attendance register if they had consumed the tablet on Deworming Day. The teachers were instructed to put a double-tick mark (99) next to a child’s name if s/he had been administered the tablet on Mop-Up Day. These tick marks are intended to be the basis for the numbers reported by every school in the S forms. Schools were supposed to provide the number of enrolled children dewormed by counting the single and double tick marks in the attendance registers. In addition, the provision for dewormed non-enrolled children was to be maintained along with the details of adults dewormed. School headmasters were supposed to submit the filled summary form to BRP by January 30 2014. Blocks were to submit all the collected forms, without any consolidation or compilation, at DEO office by February 5 2014. Districts were instructed to submit these forms at BEPC by February 20 2014. [...] Simplification of consolidation process by replacing data consolidation at block and district level with only collection of forms and submission at higher level. [...] Further, DtWI changed its prior strategy of relying on a single and small data entry partner, and selected a reputed agency with significant experience in large scale surveys across India to do data entry. This data entry partner subsequently dedicated significant data entry resources to Bihar form entry." DtWI Bihar 2014 program report, Pgs 26-27 163 In Bihar's 2015 program, 99.8% of schools reported deworming data, so we believe minimal school-level data was missing from the aggregation process. DtWI Bihar 2015 Program report, Pg 4 (Table 1) In Rajasthan's 2015 program, 94.0% of schools reported deworming coverage, so it seems possible that any deworming conducted in the remaining 6% of schools was left out of the reported coverage. @DtWI Rajasthan 2015 Program report@, Pg 4 (Table 1) In Delhi's 2015 program, 100% of schools and AWCs [anganwadi child-care centres] reported deworming. @DtWI Delhi 2015 Program report@, Pg 4 (Table 1) In Madhya Pradesh's 2015 program, 100% of schools and AWCs reported deworming. @DtWI Madhya Pradesh 2015 Program report@, Pg 5 (Table 1) In Bihar’s 2014 program, roughly 8% of schools’ summary reports were missing ("In total, 64,724 schools out of the 70,675 targeted schools submitted their summary forms."), and it sounds as if these were simply left out of the reported coverage data ("The result of this modified data cleaning and data entry process was that the coverage data was available to share from the 64,724 schools within 3 months of the deworming date. This cleaned data indicated that 16,225,546 children were dewormed in Bihar out of which 15,489,334 were enrolled children and 736,212 were non-enrolled children.") DtWI Bihar 2014 program report, Pg. 27. In Delhi’s 2013 program, Deworm the World reports that missing reports were excluded from the reported coverage figures: "The program targeted 3,032 schools and 10,500anganwadis. As on the cutoff date for report collection, 15 December 2013, data from 603 schools was pending. The above data is based on a dataset comprising 2,417 schools and 10,591 anganwadis." DtWI Delhi 2013 program report, Pg. 2. We have not seen information about missing data for Rajasthan’s 2013 program. 164 Methodology: Schools included: all schools monitors visited during coverage validation day, except those for which no classes records showed any dewormings (because we believe many schools just didn’t use that protocol despite deworming students) unless the principal said that no deworming had taken place in the school. We also excluded schools with missing total enrollment data. Students dewormed in included schools: total ‘tick marks’ in school register for all classes in included schools. Note that we are unsure whether this measure sometimes includes unenrolled students as well. Enrollment in included schools: total enrollment as reported by the monitor's check of the attendance register. Total students enrolled in each state (public and private): Sum of primary, upper primary, secondary, and higher secondary enrollment figures for the 2014-2015 school year from India’s District Information System for Education (DISE). Primary and upper primary total enrollment: U-DISE Elementary Thematic Maps 2015, Pgs 60-61. Secondary and upper secondary total enrollment: 2014-15 totals U-DISE Secondary Flash Statistics 2015, Pg 34. Total students enrolled in government and government-aided schools: Sum of (total enrollment * percentage of enrollment in government and government-aided schools) for primary, upper primary, secondary, and upper secondary schools. Percentage of students enrolled in government and government-aided schools by state: U-DISE Elementary Thematic Maps 2015, Pgs 62-63 and U-DISE Secondary Thematic Maps 2015, Pgs 33-34. U-DISE Secondary Thematic Maps 2015, Pg 34 reports the percentage of students who are enrolled in private unaided managements. We are assuming that the remainder of students are in government or government-aided schools. U-DISE Elementary Thematic Maps 2015, Pgs 62-63 reports the percentage of students enrolled in "government management schools." We are uncertain whether "government management schools" includes government-aided private schools, but we are assuming so for these figures. According to the table of contents, the map on Pg 33 of U-DISE Secondary Thematic Maps 2015 reports data on "Percentage of Secondary Enrollment by Private Unaided Management." However, the title of the map on Pg 33 is "Percentage of Professionally Qualified Teachers: Secondary Level." Based on the context in which the map appears, and because there is another map in the document with the title "Percentage of Professionally Qualified Teachers: Secondary Level" our best guess is that the table of contents is correct and the map title on Pg. 33 is an error. Estimated statewide enrolled students dewormed: (Dewormed enrolled students / Enrolled students) * Statewide enrollment in government and government-aided schools. Numbers Bihar (DtWI Bihar 2015 monitoring data for coverage validation, schools) Included schools: 557/748 schools surveyed: 234 schools without tick marks, but in 43 of those those the principal said no deworming happened; 748 - 234 + 43 = 557. Enrolled students dewormed in sample: 115,815 Enrollment in sample: 177,464 Statewide enrollment in government and government-aided schools: 23,902,897 Estimated statewide enrolled students dewormed: 15,599,299 Rajasthan (DtWI Rajasthan 2015 monitoring data for coverage validation, schools) Excluded schools: 46 Enrolled students dewormed in sample: 37,256 Enrollment in sample: 43,939 Statewide enrollment in government and government-aided schools: 7,289,229 Estimated statewide enrolled students dewormed: 6,180,557 Delhi (DtWI Delhi 2015 monitoring data for coverage validation, schools) Note that Delhi didn’t deworm students in grade 11 due to examinations. We didn’t adjust any of the numbers from the raw data because it appears that grade 11 classes were still sampled by monitors, so the monitored coverage should still reflect the low participation from that grade. "Due to examinations of 11 grade in April, the SHS decided to cover these children later in the year." DtWI Delhi 2015 program report, Pg 4 In coverage validation data, some of the reports are from class 11. DtWI Delhi 2015 monitoring data for coverage validation, schools Excluded schools: 3 Enrolled students dewormed in sample: 166,956 Enrollment in sample: 218,098 Statewide enrollment in government and government-aided schools: 2,564,953 Estimated statewide enrolled students dewormed: 1,963,495 Madhya Pradesh (DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools) Excluded schools: 178 Enrolled students dewormed in sample: 35,834 Enrollment in sample: 52,472 Statewide enrollment in government and government-aided schools: 11,151,527 Estimated statewide enrolled students dewormed: 7,615,563 Chhattisgarh (DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools) Excluded schools: 247 Enrolled students dewormed in sample: 31,535 Enrollment in sample: 40,575 Statewide enrollment in government and government-aided schools: 4,644,179 Only students 10-19 were dewormed so we are not able to generate a similar estimate of students dewormed in this state. Our calculations in this spreadsheet: GiveWell enrollment-based student coverage check 2015 165 Chhattisgarh is excluded because only some districts in the state were covered, so we are not able to apply the same methodology. The calculations for this table are in this spreadsheet: GiveWell enrollment-based student coverage check 2015 Reported enrolled student coverage Bihar: 17,600,122 ("Number of enrolled children dewormed (age 6-19 years)") DtWI Bihar 2015 Program report, Pg 4 Rajasthan: 6,463,898 ("Total enrolled children (6-19 years) dewormed at schools") DtWI Rajasthan 2015 program report, Pg 4 Delhi: "The coverage data from the schools in Delhi indicated that 1,828,562 enrolled children were dewormed in the state during deworming day and mop up day against the total target of 2,240,573 enrolled children from class 1 to 12." DtWI Delhi 2015 program report, Pg 18. Madhya Pradesh: 10,073,830 ("Number of enrolled children (Class 1 to 12) dewormed at schools") DtWI Madhya Pradesh 2015 program report, Pg 5 Chhattisgarh: "The coverage data from the state indicated that 916,596 children in the age group 10-19 years were dewormed against the target of 978,008. These include 849,797 enrolled children at schools and 128,211 out-of-school children." DtWI Chhattisgarh 2015 coverage validation report, Pg 7 166 Deworm the World, Kenya Coverage Reporting data, Year 3, "County Level-STH" sheet, cell B24. 167 "Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17. 168 Paul Byatta, attachments to email to GiveWell, September 23, 2016 169 GiveWell analysis of Deworm the World cost-per-treatment, October 2016, "Summary" sheet, cell H36. 170 See our 2014 review. 171 "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2. "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2. "National funds for deworming are part of the [Weekly Iron Fortification Supplements] WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12. 172 Excluding estimates of the costs of teacher and government staff time, Deworm the World bears approximately 90% of costs in Kenya. See GiveWell analysis of Deworm the World cost-per-treatment, October 2016, "Summary" sheet, cell Y36. 173 "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 Deworm the World noted that the costs in Kenya are unusually high (partly due to higher quality M&E): "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 Our understanding is that costs are higher in Kenya than India primarily due to cultural differences, including paying significantly higher allowances to teachers for participating in the program. For more details, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell analysis of Deworm the World cost per treatment. 174 "Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage. GiveWell conducted a site visit to DSW in November 2012 and published notes. DSW 2012 GiveWell site visit 175 GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and Evidence Action 2015 funding gap analysis 176 Evidence Action, Projected allocation of unrestricted funds, 2016. 177 Evidence Action, Projected allocation of unrestricted funds, 2016 178 GiveWell analysis of Deworm the World financials - 2016, Sheet: "EA Unres Commit - guess" 179 Deworm the World's early 2016 plan indicated spending $100,000 on Deworm the World if it raised $1.9 million. Evidence Action, Projected allocation of unrestricted funds, 2016 180 Originally $300,000 was expected to be used for Evidence Action Beta. Evidence Action, Projected allocation of unrestricted funds, 2016 181 It's possible that support for Evidence Action Beta is included under the organizational development category. It is also possible that Evidence Action did not need to support Evidence Action Beta after Good Ventures provided Evidence Action a grant at the recommendation of GiveWell as part of GiveWell's experimental work. 182 The rest was allocated to organizational development. Evidence Action, Projected allocation of unrestricted funds, 2016 and GiveWell analysis of Deworm the World financials - 2016, Sheet: "EA Unres Commit - guess" 183 GiveWell analysis of Deworm the World financials - 2016, Sheet: "EA Unres Commit - guess" 184 "...many of the investments of unrestricted have substantial positive benefit for DtW, even as they are not specifically programmatic in nature. These include investment in financial capacity, and the transition of the Indian entity (which at the moment is exclusively working on deworming, though this may not be the case in the future)." Grace Hollister, edits to GiveWell's review, November 20, 2016