Deworm the World Initiative, Led By Evidence Action - June 2016 review

We have published a more recent review of this organization. See our most recent report on Deworm the World.

---

Published: June 2016

Summary

What do they do? The Deworm the World Initiative (evidenceaction.org/#deworm-the-world), led by Evidence Action, advocates for, supports, and evaluates government-run school-based deworming programs. (More)

Does it work? We believe there is relatively strong evidence for the positive impact of deworming. Deworm the World sends monitors to schools during and after deworming to determine whether the programs it supports have reached a large proportion of children targeted. We have reviewed data from its two largest programs, which overall indicate strong results, though we do not have full details on the methodology used in Kenya. (More)

What do you get for your dollar? We estimate that children will be dewormed for a total of about $0.80 per child, or $0.56 per child excluding the value of teachers’ and principals’ time spent on the program. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)

Is there room for more funding? We believe that Deworm the World is not constrained by funding. As of March 2016, Deworm the World held $17.9 million in funding, $11.2 million of which was unrestricted. Donors should note that Deworm the World is part of a larger organization, Evidence Action, that conducts other work we have not evaluated, and receives unrestricted funding that it allocates across programs. (More)

Deworm the World is recommended because of its:

• Focus on a program with a strong track record and excellent cost-effectiveness. (More)
• Strong process for assessing whether the deworming programs it supports are successfully deworming children. (More)
• Standout transparency – it has shared significant, detailed information about its programs with us.

Major unresolved issues include:

• We have a limited understanding of Deworm the World’s past use of unrestricted funds at a detailed level.
• We are unsure how additional donations to Deworm the World will impact how Evidence Action, Deworm the World’s parent organization, chooses to allocate its unrestricted funding.
• We do not know how well Deworm the World’s success and cost-effectiveness in new countries will compare to its track record in India and Kenya.

Our review process

Our review process has consisted of:

• Conversations with Deworm the World Director Grace Hollister and other Deworm the World and Evidence Action staff since 2012.1
• A conversation with the Children's Investment Fund Foundation (CIFF), a funder of Deworm the World.2
• Reviewing documents Deworm the World sent in response to our queries.
• Following Deworm the World's progress and plans for funds raised as a result of GiveWell's recommendation (as well as Evidence Action's overall progress and plans).
• Site visits:
  • In November 2012, we visited Deworm the World's office in Nairobi, Kenya and met its staff there. (Notes from our visit)
  • In September 2013, we visited Deworm the World's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with Deworm the World. (Notes from our visit)
• In 2015, we retained two journalists to visit areas served by Deworm the World in Kenya. We published their report on our blog.

All content on Deworm the World, including past reviews, updates, blog posts and conversation notes, is available here.

A note about this review

Deworm the World's longest running programs are in Kenya and India. Our review goes into more depth on Deworm the World’s work in India than its work in Kenya. When we first engaged with Deworm the World in 2013, Deworm the World expected to use additional unrestricted funding in India. Because we expected the donations we would recommend to be used in India, we focused our review there.

As of the writing of this review, both India and Kenya are almost entirely funded by restricted funding, and unrestricted funds will most likely either support Deworm the World’s work in new countries or deworming in Kenya post-2017 (more below). We would now guess that information about Deworm the World’s work in all locations is relevant to evaluating its future prospects. Therefore, this review includes some information on its programs in countries other than India, but for primarily historical reasons still has somewhat more of a focus on Deworm the World's work in India.

What do they do?

The Deworm the World Initiative, led by Evidence Action, advocates for and supports the implementation of government-run deworming programs for preschool- and school-aged children.3 The support that Deworm the World provides is of two types: 1) Deworm the World offers "technical assistance" (we expand more on what this includes below) to governments implementing deworming, and 2) Deworm the World sometimes funds components of deworming programs.4

The deworming programs that Deworm the World supports are focused on executing school-based mass drug administrations (MDAs), in which deworming pills are distributed without testing each individual for worms and the aim is to treat the entire population of children within a geographic area; this strategy is used because it is more cost-effective than testing each individual.5 Deworm the World focuses on MDAs that treat children infected with soil-transmitted helminthiasis (STH).6 Where needed, these programs may also include treatment for schistosomiasis.7

In India and Kenya, Deworm the World has worked primarily with government staff to implement deworming programs. As Deworm the World has expanded to new countries, it has started to collaborate with local NGO partners to support governments.8 Additionally, Deworm the World has also recently considered taking on a few projects that fall outside of its purview of supporting the direct implementation of deworming program (see footnote).9

Deworm the World was founded in 2007,10 and as of March 2016 had supported deworming treatments in India, Kenya, and Ethiopia, and had started preliminary support for deworming programs in Pakistan, Nigeria, and Vietnam.11

Below, we discuss:

• Deworm the World's role in government-led deworming programs
• The status of Deworm the World's work by country
• A breakdown of Deworm the World's spending
• Deworm the World's relationship to Evidence Action

Deworm the World’s role in government-led deworming programs

The deworming programs that Deworm the World supports are implemented by the governments it works with.12 Below, we expand on Deworm the World's role in the programs it supports. Note that we use "Deworming Day" to mean the day on which the MDA takes place. Similarly, we use "Mop-Up Day" to refer to the day that occurs several days after Deworming Day and is when students who were absent or sick on Deworming Day are given their deworming pills (although note that not all countries include a Mop-Up Day in their program).13

The assistance that Deworm the World provides in each country varies based on what each partnering government needs.14 Historically, Deworm the World's role has included the following (note that our categorization of Deworm the World's activities slightly differs from Deworm the World's standard categorization):

1. Advocacy: Deworm the World actively encourages national and large sub-national governments to implement mass school-based deworming programs.15 Our impression is that Deworm the World's advocacy primarily consists of meeting with health officials in a government to discuss the benefits of deworming and how a deworming program might be implemented.16 Deworm the World has told us that it will not work with a government on a national deworming program until it has built a strong working relationship with that government via its advocacy.17 Deworm the World also participates in the broader "STH community"; that is, it works with other organizations advocating for and implementing activities that will further reduce or eliminate STH globally.18
2. Prevalence surveys. Before Deworm the World helps launch a deworming program in a new area, it evaluates whether the prevalence of worm infections is sufficient to justify an MDA for the school-aged population.19 If no prevalence surveys have been conducted recently, it generally commissions one.20 It also plans to conduct follow-up prevalence surveys periodically, so that it can track the impact of the MDAs.21 Deworm the World generally contracts out work on prevalence surveys.22
3. High-level planning. Deworm the World has told us that it often assists governments with high-level operational decisions, such as how to budget for the program or obtain deworming drugs.23 For example, Deworm the World has helped governments submit requests for deworming drugs (albendazole or praziquantel) to the WHO.24
4. Program preparation: Trainings, distribution of materials, and troubleshooting. Deworm the World has helped governments design and organize what it calls a "training cascade" (more detail in the footnote).25 Through the training cascade, teachers and other government staff learn how to implement a Deworming Day and receive materials necessary for implementation (such as reporting forms and drugs).26 In the past, Deworm the World has hired or trained staff to lead the trainings and developed materials for the trainings.27 In India, Deworm the World has also arranged tele-callers to reach out to schools to assess their preparedness and notify government officials of any problems before Deworming Day.28
5. Community sensitization. Deworm the World supports community sensitization efforts, which aim to make local communities aware of Deworming Day and the benefits of deworming children.29 For example, via the training cascade, teachers are instructed to spread the word about Deworming Day to their communities.30 Deworm the World has also developed text message campaigns, organized public announcement events, and edited mass media materials to be more appropriate for local contexts.31
6. Monitoring and evaluation. Deworm the World told us that it helps governments design or improve reporting and monitoring systems. It also collects monitoring data independently.32 Deworm the World focuses on assisting with the collection of three main types of monitoring data:33
   • Monitoring before and during deworming: Monitors hired by Deworm the World visit schools before and during Deworming Day and Mop-Up Day. It is meant to assess both a) how prepared schools and health systems are to implement deworming and b) the extent to which proper procedures are followed.34 Monitoring visits may include assessments of the quality of trainings, community sensitization efforts, and Deworming Day activities, depending on what Deworm the World and the government agree to monitor.35
   • Coverage reporting: On Deworming Day and Mop-Up Day, teachers are asked to mark the number of children that they deworm. This data is then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (e.g., in India: school, node, block, district, and state) to create a reported coverage estimate for a region.36
   • Coverage validation: Approximately one week after Mop-Up Day, Deworm the World sends independent monitors back to schools to check the coverage data recorded by schools and ask students about whether or not they were dewormed.37 This data can then be compared to the coverage data reported by the government.

   Typically, Deworm the World hires and trains third-party monitors to collect process monitoring and coverage validation data; this footnote includes Deworm the World’s descriptions of the monitor selection process used in four states in India in 2015.38 It is our understanding that in Kenya, Evidence Action (Deworm the World's parent organization) maintains a monitoring team year-round that Deworm the World makes use of.39

   Deworm the World also assists with additional monitoring when needed.40

Description of programs by country

As of March 2016, Deworm the World had supported deworming treatments in India, Kenya, and Ethiopia, and it had started support (or was in discussions about starting support) in five other countries.41 Each country has or will have a slightly different program model; we expand upon the programs by country below and have summarized the differences in this spreadsheet.

India

Deworm the World started working in India in 2009.42 It has assisted at both the state level and the national level with all areas mentioned above.43 Currently, Deworm the World's work in India is fully funded, primarily by the Children's Investment Fund Foundation (CIFF), the United States Agency for International Development (USAID), Dubai Cares, and the END Fund.44

In 2014, following advocacy from Deworm the World, the national government decided to implement "National Deworming Day," a single-day deworming program targeting all children aged 1-19 for STH treatment.45 The first National Deworming Day was held in February 2015, with 12 states participating (five of which received direct support from Deworm the World).46 A second National Deworming Day was held in February 2016 in 30 of India's 36 states and union territories; eight of the participating states received support from Deworm the World.47

One state that Deworm the World previously supported - Delhi - has transitioned to implementing its deworming program with very limited technical support; Deworm the World expects to reduce support to other states as their capacity to operate their programs without Deworm the World's assistance increases over the next few years.48

Kenya

Deworm the World started to work with the national government in Kenya in 2009, supporting a pilot deworming program.49 In 2012, Deworm the World agreed to support Kenya's national deworming program, under a 5-year plan funded by CIFF and the END Fund.50 Deworm the World does not assist much with prevalence surveys in Kenya, because (a) Kenya was mapped before Deworm the World started to work there, and (b) the Kenya Medical Research Institute (KEMRI) is funded by CIFF to conduct independent prevalence surveys before and after each round of deworming.51 Deworm the World provides assistance in all of the other areas we described above.52

Similar to in India, the program in Kenya has moved towards a model in which deworming occurs on consolidated Deworming Days. Deworm the World hopes to eventually reduce the amount of support it provides in Kenya.53

Currently, Deworm the World has a robust staff presence in Kenya, in part because Deworm the World implements monitoring activities in Kenya itself (rather than hiring consulting firms to implement this).54 The Kenya team occasionally supports and trains staff for Deworm the World's newer programs.55

Ethiopia

In Ethiopia, Deworm the World partners with the Schistosomiasis Control Initiative (SCI) - another of GiveWell's recommended charities - to advise the national government.56 Deworm the World started advising Ethiopia in late 2014, after Ethiopian government officials visited Kenya for a learning exchange and spoke with Deworm the World representatives during the trip.57

In early 2015, Ethiopia launched a national school-based deworming program.58 Deworm the World supported the government in implementing a pilot MDA in mid-2015 and the national program's first full MDA in late 2015.59 Deworm the World's work on the pilot was funded in part by its unrestricted funding.60 In mid-2015, CIFF and the END Fund decided to fund a five-year national school-based deworming program in Ethiopia.61

Deworm the World has told us that its role in Ethiopia is similar to its role in Kenya and India (though it has less of a staff presence), while SCI assists with activities that are more scientifically technical, such as conducting prevalence surveys.62

Pakistan

Deworm the World first began to explore the possibility of working in Pakistan in 2015, for two reasons:

• Pakistan is believed to have a high worm burden across a large population, and there is currently no mass treatment of school-age children for STH in Pakistan.63
• An organization that Evidence Action partners with - International Relief & Development (IRD) - works in the health sector in Pakistan.64

In Pakistan, Deworm the World is partnering with two local organizations: IRD and the Institute of Development and Economic Alternatives (IDEAS).65 Neither organization has experience with school-based deworming.66 IRD has implemented other health programs and IDEAS is involved in education policy work.67

Because the prevalence of STH in Pakistan has not previously been mapped, IRD and Deworm the World are, as of early 2016, in the process of conducting prevalence surveys.68 Deworm the World and its partners have reached out to provincial governments in Pakistan about implementing a deworming program; it expects conversations to move more quickly once results from the prevalence surveys are available.69 Given the devolution of health policy and programming to provinces in Pakistan, Deworm the World expects to primarily work with individual provinces, similar to how it works closely with states in India.70

The prevalence survey in Pakistan is almost entirely supported by Deworm the World's unrestricted funding.71

Nigeria

In 2015, Deworm the World hired a consulting firm to assess the opportunities for working in Nigeria.72 The assessment found that several states with a high burden of STH did not have any NGO partners focused on deworming.73 After meeting with government officials in four of the states, Deworm the World decided to work in Cross River state.74

Deworm the World is partnering with RTI International, an NGO with established offices and financial systems in Nigeria.75 Deworm the World is supporting the state government to implement school-based treatment for schistosomiasis and STH, while RTI focuses on community-based treatment of other endemic NTDs.76 As of June 2016, the two organizations had established a contract governing their partnership, and memoranda of understanding were in place with the health and education sectors of the state government.77 The first MDA will take place in June.78

Most of Nigeria has already been mapped for STH and schistosomiasis, which means that Deworm the World does not need to assist with mapping surveys.79 Deworm the World mainly expects to assist with developing trainings, high level planning (e.g., standardizing timelines and processes), and monitoring activities.80

Deworm the World is funding its work in Nigeria with unrestricted funding.81

Future activities in Nigeria might include advocacy and technical assistance to the national government on and/or additional states.82

Vietnam

Deworm the World is partnering with the Thrive Network in Vietnam on an integrated deworming, sanitation, and hygiene education program.83 The program is funded primarily by Dubai Cares and includes a randomized controlled trial to test the impact of hygiene education on STH reinfection rates.84 The program was approved by the Vietnamese government in late 2015.85

Some deworming already occurs in Vietnam; Deworm the World and Thrive Networks are partnering with the government to improve the existing deworming program in the four provinces targeted by the program.86 Deworm the World hopes to a) expand its operations to other provinces that need deworming assistance and to b) convince the national government to develop a comprehensive national deworming program.87

Currently, Deworm the World is working with the National Institute of Malaria, Parasitology, and Entomology (NIMPE) – the organization in Vietnam that implements MDAs – to conduct prevalence surveys in approximately twenty-one provinces.88 This is beyond the scope of the Dubai Cares grant, so Deworm the World is supporting the prevalence surveys with its unrestricted funding.89 In addition to assisting with prevalence surveys, Deworm the World expects to help NIMPE develop an integrated training and distribution cascade, design new materials for training and community sensitization efforts, procure drugs, develop an adverse events protocol, and establish an independent monitoring system.90

Deworm the World originally became involved in Vietnam through discussions between its former Executive Director, Alix Zwane, and Thrive Networks' former regional director.91 We are unsure who initiated the partnership between the two organizations.

Nepal

In early 2015, Deworm the World was in discussions with Nepal about starting a deworming program there.92 However, in mid-2015 Nepal experienced a large earthquake and those discussions were halted.93 Deworm the World plans to restart its conversation with Nepal in the first half of 2016.94

Deworm the World was first connected to the government of Nepal via J-PAL (similar to how the program in Bihar began).95

Indonesia

Deworm the World has had initial conversations with Indonesia about a potential deworming program there and is planning a scoping trip in mid-2016.96 It believes that only a low percentage of children at risk for STH are being treated in Indonesia.97

Breakdown of Deworm the World’s spending

We have seen high-level summaries of Deworm the World's spending in 2015.98 The vast majority of this spending was funded by restricted funding.99 Restricted funding from the Children's Investment Fund Foundation (CIFF) and the END Fund fully fund Deworm the World’s work in Kenya, where it has supported a nationwide deworming program since 2012.100 In 2014 and 2012 respectively, Deworm the World received funding commitments from CIFF/Dubai Cares and USAID to fully support its work in India; in 2015 a grant from the END Fund was received to support work in Rajasthan.101

Deworm the World’s spending on deworming programs is supplemented by the spending of partner organizations and the implementing governments. Below is a breakdown of how Deworm the World spent its funding in 2015 globally as well as in Kenya and India.102 Deworm the World has told us that the majority of its personnel costs fall within the "program management" category.103

Deworm the World’s expenses - 2015104

	India	Kenya	All locations
Policy	7%	6%	8%
Awareness	7%	5%	6%
Drugs	1%	1%	1%
Training	7%	29%	15%
Monitoring and evaluation	11%	10%	9%
Prevalence survey	25%	0%	12%
Program management	26%	36%	35%
Exploratory	0%	0%	0%
Indirect costs	15%	13%	14%
Total	100%	100%	100%

Deworm the World and Evidence Action

In early 2013, Innovations for Poverty Action announced the formation of Evidence Action to scale cost-effective and evidence-based programs. Evidence Action took over two such programs from Innovations for Poverty Action - Deworm the World and Dispensers for Safe Water - and has a program for investigating other programs to consider scaling up in the future called Evidence Action Beta.105 We focus this review on Deworm the World and discuss the room for more funding implications of Deworm the World’s being a program of a larger organization below.

Does it work?

Deworm the World-assisted mass drug administration programs are focused on delivering treatments that have been independently studied in rigorous trials and found to be effective. Evidence from Deworm the World's monitoring makes a relatively strong case that the programs Deworm the World has supported have successfully dewormed children.

While Deworm the World's track record in Kenya and India is strong, it is currently expanding to several new countries and working with new partner organizations. As with any major scale-up, there is a risk that it may not achieve as strong results as it has in the past or may not produce similarly strong evidence of its impact.

Here we focus on the following questions to understand whether Deworm the World’s activities are having the intended impact (details in the sections that follow).

• Are mass school-based deworming programs effective when implemented well?
• Have infection rates decreased in target populations?
• Are programs operating as intended?
• How does Deworm the World affect program outcomes?
• Are there any negative or offsetting impacts?

Are mass school-based deworming programs effective when implemented well?

Deworm the World supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.

There are some important differences between the type and severity of worm infections in the places Deworm the World works and the places where the key studies on improved life outcomes from deworming took place, which we discuss below. In particular, Deworm the World primarily provides support to mass drug administrations (MDAs) that treat populations with fewer infections from soil-transmitted helminths, and very low or no infections from schistosomes (including many locations where the programs Deworm the World supports do not treat for schistosomes because prevalence is too low to justify treatment).106

Have infection rates decreased in target populations?

India prevalence surveys

Deworm the World has conducted prevalence surveys in several of the states that it assists with in India, and has conducted one follow-up prevalence survey in the state of Bihar.107 The two surveys measuring the prevalence of worm infections in Bihar are consistent with the notion that the deworming program reduced infection rates in target populations. However, there are also plausible, alternative explanations for these results. (Note that the first survey was not designed to measure a subsequent change in infection rates, so our conclusion is not surprising.)

The original prevalence study in Bihar measured prevalence in four of Bihar’s 38 districts, which were selected to inform a prevalence model based on climatic and socioeconomic variance across the state.108 We are not sure how representative that survey was of the state as a whole since the districts were not chosen randomly.109 Since the survey found >50% prevalence,110 the Government of Bihar planned to follow the WHO’s recommendation to conduct deworming MDAs twice per year, with one round each year led by the National Filaria Control Program (NFCP) and one round through schools that would be assisted by Deworm the World.111 However, according to Deworm the World, only four rounds of deworming took place between the two prevalence surveys in 2011 and early 2015, with one community-based round led by the NFCP and three school-based rounds assisted by Deworm the World.112

Deworm the World’s follow-up survey in early 2015 was conducted in randomly selected districts and was designed to be representative of the entire state and the three climatic regions.113 The results of both surveys, broken down by climatic region, are in the table below; no single district was surveyed in both prevalence surveys.

Indications of STH prevalence in Bihar’s three agro-climatic zones in 2011 and 2015114

	Hookworm	Ascaris (roundworm)	Trichuris (whipworm)	Any soil-transmitted helminth
NW Alluvial Plains, 2011 (2 districts, non-random)	47.4%	56.8%	4.2%	71.3%
NW Alluvial Plains, 2015 (5 districts, random)	21.0%	24.0%	12.0%	43.0%
NE Alluvial Plains, 2011 (1 district, non-random)	32.8%	72.1%	11.3%	79.6%
NE Alluvial Plains, 2015 (3 districts, random)	17.0%	38.0%	5.0%	50.0%
S Alluvial Plains, 2011 (1 district, non-random)	40.2%	25.0%	1.7%	49.0%
S Alluvial Plains, 2015 (6 districts, random)	14.0%	6.0%	2.0%	20.0%
Statewide 2011 (4 districts, non-random)	42.2%	52.1%	5.2%	67.5%
Statewide 2015 (14 districts, random)	17.0%	19.0%	6.0%	35.0%

We are unsure whether worm prevalence in Bihar would have increased or decreased in the four years between the two prevalence surveys if the three MDAs that Deworm the World supported had not occurred.115 Here we’ve listed some other factors that we weigh when considering these results:

• Methodological limitations of these prevalence surveys. It may be that the original prevalence survey took place in districts with unusually high worm prevalence. Since it is unclear whether the original prevalence survey was representative of the state or its climatic zones and the two prevalence surveys did not sample the same districts, it may be that the decrease in prevalence seen above is the result of comparing different populations rather than a fall in prevalence in the same population over time.
• MDAs for lymphatic filariasis. The NFCP’s program, which is designed to reduce the burden of lymphatic filariasis (LF) as well as STH, claims it conducted MDAs in Bihar in 2012 and 2014 with albendazole, the drug used to treat STH.116 According to the Indian government's program website, the LF treatment program is designed to be annual, and reached 86% of the population in both 2012 and 2014.117 We would not be surprised if the coverage rates reported by the LF program were inaccurate, but we have not tried to verify them. Deworm the World has said that the 2014 round actually occurred in early 2015 after the 2015 prevalence survey.118
• Factors other than MDAs. There are a variety of large-scale changes that could affect worm prevalence, such as improved sanitation infrastructure or general development. According to India’s central bank, inflation-adjusted per-capita income in Bihar increased by a total of 39% over a similar four year period, suggesting that there could have been broad improvements in the region that would be associated with better health and lower worm prevalence.119 Alternatively, it seems possible that population growth combined with poor sanitation infrastructure could have increased worm prevalence over this period if Deworm the World-supported MDAs had not been carried out.

Kenya prevalence surveys

In Kenya, the Kenya Medical Research Institute (KEMRI) is funded by CIFF to conduct independent prevalence surveys before and after every MDA that Deworm the World supports; Deworm the World is not involved in these surveys.120 We have seen results from the prevalence surveys conducted before and after the Year 1 (2012-13), Year 2 (2013-14), and Year 3 (2014-15) MDAs in Kenya.121 Deworm the World has provided us with details of a) how the baseline survey was conducted and b) pre-registered plans for follow up surveys.122 Note that we have not yet seen detailed descriptions of the methodologies used for the Year 2 and Year 3 surveys, so we have not verified that they were carried out as intended.123

The prevalence surveys are not representative of the full program Deworm the World supports: they are conducted in 20 districts randomly sampled from 66 districts in which STH was assumed to be endemic before Kenya started its national deworming program; we do not believe the 66 districts were selected randomly from all districts in the national program.124 We summarize the key features of the prevalence surveys in this footnote.125

We believe these surveys provide evidence that the deworming program reduced infection rates in target populations. The tables below show the results of the prevalence surveys for Year 1 (2012-13), Year 2 (2013-14), and Year 3 (2014-15).126

Kenya prevalence survey results - soil-transmitted helminths

	Hookworm	Ascaris (roundworm)	Trichuris (whipworm)	Any soil-transmitted helminth
Year 1 (2012-13) baseline (200 schools)	16.9%	19.2%	5.4%	33.4%
Year 1 (2012-13) post-MDA (70 schools)127	3.2%	2.3%	4.3%	8.7%
Year 2 (2013-14) pre-MDA (60 schools)	4.5%	12.5%	5.1%	19.0%
Year 2 (2013-14) post-MDA (60 schools)	2.2%	1.9%	2.7%	6.0%
Year 3 (2014-2015) pre-MDA (200 schools)	2.3%	11.9%	4.5%	16.3%
Year 3 (2014 -2015) post-MDA (60 schools)	1.8%	2.8%	2.3%	6.3%

Kenya prevalence survey results - Schistosomiasis128

	S. haematobium - Coast Province only129	S. mansoni - all provinces
Year 1 (2012-13) baseline130	14.8%	2.1%
Year 1 (2012-13) post-MDA	Unknown	No comparable data available
Year 2 (2013-14) pre-MDA	10.5%	2.7%
Year 2 (2013-14) post-MDA	7.6%	0.6%
Year 3 (2014-2015) pre-MDA	8.8%	1.5%
Year 3 (2014 -2015) post-MDA	5.8%	0.8%

We are uncertain what timing schedule the prevalence surveys follow. Schools in Kenya do not all undergo MDAs simultaneously, and we do not know if the pre-MDA surveys are conducted at the same time (e.g., in February) or if they occur at a standard time before an MDA (e.g., 3 weeks before the MDA).131 It is possible that the timing of the surveys could affect observed outcomes; for example, worm burden might be especially high during some seasons in Kenya, so if pre-MDA surveys were conducted during high-burden seasons, while post-MDAs were conducted later in low-burden seasons, the effect of deworming might appear to be greater than it actually is.132

Additionally, it seems possible that surveying the same 60 schools each year could introduce bias: if the schools have an incentive to show that the program is working, they may execute a higher-quality deworming program than they otherwise would if they weren't going to be re-surveyed.133 Similarly, if the survey protocol requires researchers to provide treatment to students immediately after they are tested (as we have seen in some of SCI's prevalence surveys), the results would not be representative of the quality of the program overall; we do not know for certain if this was the case in Kenya, though we would guess that it is not because the survey was designed to assess program impact.134

There is no control group for the prevalence surveys. However, we believe the short amount of time between the pre-MDA surveys and the post-MDA surveys makes it very unlikely that factors other than the MDAs caused the decrease in worm prevalence.135

Are programs operating as intended?

Administration of deworming drugs is a relatively straightforward program, though any MDA could encounter many challenges when operating at a large scale. The information we have seen from third-party monitors hired and trained by Deworm the World in India and Kenya strongly suggests that the programs are generally operating as intended.

Monitors visit schools on and after Deworming Day (and sometimes before), and record their observations.136 They also interview principals, teachers, students, and sometimes parents, recording the responses to a large number of interview questions.137 In the rest of this section, we use monitors' assessments to answer the following questions:138

• Are schools prepared for Deworming Day? Do teachers have the knowledge and materials necessary to implement deworming? Before and during Deworming Day, Deworm the World evaluates whether teachers are knowledgeable on topics relevant to deworming and drugs are in adequate supply.
• Are schools deworming children? Are Deworming Days being implemented? Do a high proportion of children receive treatment? During Deworming Day visits, monitors observe whether children are chewing and swallowing the pills. During coverage validation, monitors interview children to ask whether deworming took place and whether they swallowed the pills.
• Are schools following proper procedures? Are the appropriate number of pills being given to children? Are safety procedures being followed? Do teachers record information correctly? During Deworming Day visits, monitors observe whether teachers are adhering to deworming procedures (such as administering the drugs after the students have eaten and not giving drugs to children who appear sick).139

We describe at a high level some of the processes that monitors use in this footnote.140 As far as we know, Deworm the World does not audit monitors’ work.

Are schools prepared for Deworming Day?

Based on the data and reports we have seen, we feel fairly confident that most schools in India and Kenya are prepared to implement Deworming Days. Most schools seem to have sufficient number of deworming pills available for children; fewer appear to have adequately trained teachers.

India
In India, telecallers phone schools ahead of Deworming Day to assess the school's preparedness.141 We have not yet requested data from these calls.

Deworm the World's monitors also ask school personnel questions about their preparedness when visiting schools on Deworming Day. Below, we present data related to preparedness from several programs in India that Deworm the World supported in 2015.142

Sample preparedness monitoring results (India, 2015)143

	Bihar 2015	Rajasthan 2015	Delhi 2015	Madhya Pradesh 2015	Chhattisgarh 2015	Sample of question asked
Schools that had a representative attend deworming training (according to principal/headmaster)	85.8%144	63.8%145	83.0%146	39.8%147	40.6%148	"Did you or any teachers from your school attend official training in the last 4 months for deworming?"149
Schools that received deworming tablets (according to principal/headmaster)	90.3%150	98.0%151	98.6%152	82.8%153	Not reported	"Did your school receive the deworming tablets?"154
Schools with sufficient drugs for deworming (according to principal/headmaster)	86.1%155	96.0%156	97.2%157	79.2%158	95.5%159	"Did you have the sufficient drugs for deworming?"160

(Note: Previously, we summarized monitoring from the 2013-2014 programs and the 2012 programs).

We think this data generally shows a fairly high level of preparedness in the states that Deworm the World assisted (note that Deworm the World provided "light" technical assistance to Chhattisgarh in 2015).161 The fact that the surveys identified low training rates in several cases increases our confidence in the reliability of the surveys.

Kenya
In Kenya, Deworm the World's monitors visit schools before Deworming Day to assess their level of preparedness for the upcoming MDA.162 We have not yet asked how the schools are selected.163 The most recent monitoring report we have from Kenya (for the 2014-15 round, or Year 3) notes that 97% of schools visited pre-Deworming Day had a teacher who had attended a training in the last fifteen days.164 While 95% of schools had received some deworming tablets, only 87% of schools believed that they had received enough to cover the children at their school on Deworming Day.165 Note that Deworm the World shared the pre-Deworming Day data that informs the monitoring report and we have not yet analyzed it. There appears to be a large number of questions in the survey that were not included in the report.166

Monitors also ask some questions regarding preparedness during Deworming Day. For example, during Deworming Day visits, monitors check the number of drugs ordered against the number of students enrolled.167 The monitors then observe a randomly selected class, which includes checking to see if drugs run out during deworming, and interview teachers after the deworming has occurred about whether or not there were sufficient materials.168 (We are not sure if the schools selected for observation on Deworming Day are the same schools visited before Deworming Day and if the schools are selected randomly). We have reviewed data collected from interviews and observations on Deworming Day related to schools' preparedness; sample results are presented in the table below:169

Sample preparedness monitoring results (Kenya, Year 3: 2014-15)170

	Percentage	Question asked on survey form
Schools with a head teacher who had recently attended deworming training (according to the head teacher)	92%	"Have you attended any training session on deworming in the past 15 days?"171
Schools that received deworming tablets (according to principal/headmaster)	98%	"Have you received the Albendazole tablets?"172
Schools with sufficient drugs for deworming (according to principal/headmaster)	90%	"Do you have sufficient Albendazole tablets for your school?"173

Our analysis provides slightly different results than what was presented in the monitoring report.174 For example, only 90% of schools reported sufficient Albendazole tablets, which contradicts the monitoring report's claim that 98% of schools had sufficient drugs for Deworming Day.175 We do not know the reasons for this discrepancy. We also see some reason to interpret these data cautiously, as some of it was self-reported by people who could be biased to want favorable monitoring results.

Both the summary report's results and the results from our own data analysis indicate a high level of school preparedness for Deworming Day.

Are schools deworming children?

Monitors’ direct observations on Deworming Day and Mop-Up Day and interviews with children during coverage validation strongly suggest that a high portion of schools implement Deworming Days.

India
In India, Deworm the World's monitors survey randomly selected schools (intended to be representative at the state level) on Deworming Day and Mop-Up Day to check whether deworming is occurring. They also visit another set of randomly sampled schools within a few weeks of Deworming Day, to check schools' records for evidence of deworming and interview children and teachers about the Deworming Day.176 If the chosen school is closed on the day of the deworming, they are instructed to go to an assigned backup school instead.177 It is our impression that most schools were open when monitors visited.178 The results of some of the interviews and observations from Deworming Day and coverage validation are in the table below.179

Sample monitoring results on whether deworming occurred (India, 2015)

Evidence of deworming occurring (timing of observation)	Bihar	Rajasthan	Delhi	Madhya Pradesh
Monitors observe deworming activities on Deworming Day (DD)	83%	94%	88%	70%
Principal reports conducting deworming on DD (same day)	82%	98%	99%	70%
Present children report being dewormed, at schools where principal reported deworming happened (coverage validation day) - GiveWell calculation	97%	96%	94%	93%

We see some reason to interpret these data cautiously (for example, some of it was self-reported by people who could be biased to want favorable monitoring results).

Kenya

In Kenya, monitors observe deworming activities on Deworming Days and Mop-Up Days, and also conduct coverage validation activities within a week of deworming occurring.180 We are not sure if the schools observed on Deworming Day are randomly selected.181

During coverage validation, it is our understanding that monitors interview three randomly selected children from three separate classes, for a total of nine students, at each school they visit.182 We have analyzed some of the data from these interviews, and from monitors' observations and interviews on Deworming Day. Sample results are in the table below:183

Sample monitoring results on whether deworming occurred (Kenya, Year 3: 2014-15)

Evidence of deworming occurring (timing of observation)	Percentage	Question asked on survey form
Monitors observe deworming activities on Deworming Day (DD)184	98%	"For Albendazole treatment: a. Circle gender of child (refer to class teacher if required). b. circle if tablet taken/refused for albendazole"185
Head teacher reports conducting deworming on DD (DD)	97%	"Are there any events or special programs happening at this school today?"186
Children identify Albendazole as the pill they were given on DD (coverage validation)	95%	"Show student 3 tablets, ask: Which one of these three tablets did you take?"187

Note that these results broadly match up with the information from the monitoring summary report (details in footnote).188 We believe the results we have seen are consistent with the view that deworming is occurring, although we note that the results from Kenya seem quite high to us when compared to the results we have seen from India. We are not sure if this is due to some bias in the way that the Kenya data is collected, or if Deworm the World's program in Kenya is simply higher-quality.189

Are schools following proper procedures?

The data that we have seen indicates that schools are, by and large, following the correct processes to deworm children, although teachers do not always follow proper recording procedures.

India
In India, when monitors visit schools on Deworming Day, they make observations and ask questions related to the process of deworming, to check whether or not it is being implemented correctly. For example, monitors observe whether or not teachers ask their students if they are sick before giving them their pills and how many pills teachers give students.190 In the table below, we present the results from some of the monitors' observations.191

Sample monitoring results on whether proper procedures were followed (India, 2015)192

	Bihar 2015	Rajasthan 2015	Delhi 2015	Madhya Pradesh 2015	Sample of question asked
Classes where teachers followed the correct recording protocol (monitors' observations)	68%193	66%194	85%195	62%196	"Did the teacher tick each child's name/roll no. in the attendance register after giving them deworming medicine?"197
Classes where teachers told children to chew the pill before swallowing it (monitors' observations)	89%198	83%199	96%200	82%201	"If children are getting deworming tablets, is the teacher telling the students to chew the tablet before swallowing it?"202
Classes where teachers identified sick children before administering the tablet (monitors' observations)	80%203	72%204	93%205	73%206	"Did the teacher ask the children if they are sick/under medication before giving the medicine?"207

Kenya

In Kenya, monitors conduct extensive interviews with head teachers and teachers on Deworming Day and observe deworming procedures in classrooms. We have analyzed some of the data from these activities; sample results are in the table below.208

Sample monitoring results on whether proper procedures were followed (Kenya, Year 3: 2014-15)

	Percentage	Question asked on survey form
Teachers recorded administering the deworming pill (monitors' observations)	89%209	"Did the teacher mark on the monitoring form as the tablet was administered?"210
Students that teachers observed swallowing Albendazole (monitors' observations)	96%	"For Albendazole treatment: ...c. Circle if the teacher observed the child taking the tablets"211
Of children given tablets, the % who were not sick (monitors' interviews with students on Deworming Day)	95%	"How were you feeling today morning?"212

We believe these results are consistent with the claim that teachers are following proper procedures during implementation. Note that these results broadly match up with the information from the monitoring summary report (details in footnote).213

How does Deworm the World affect program outcomes?

Deworm the World may be having an impact in the following ways:

1. It may increase the likelihood that a government implements a deworming program, by advocating for deworming programs, by offering to provide technical assistance, and/or by funding implementation.
2. It may improve the quality of a deworming program that would have been implemented without Deworm the World (leading to more children dewormed effectively or improved cost-effectiveness).

While we have limited evidence to rely on, we would guess that Deworm the World increases the likelihood that (national and subnational) governments implement deworming, particularly when it pays for the majority of financial program costs, as it expects to do in the countries to which it expands. Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are uncertain about this.

Note that evaluating the impact of a technical assistance and advocacy organization such as Deworm the World is not as straightforward as evaluating a direct service organization. There are substantial potential advantages to supporting such an organization, as it may be able to have a greater impact per dollar by influencing government policy than it would have by simply carrying out programs on its own. However, the additional steps between the organization’s actions and the desired outcomes complicate impact assessment.

Does Deworm the World increase the likelihood that governments implement deworming programs?

While we have limited evidence to rely on, we would guess that Deworm the World increases the likelihood that (national and subnational) governments implement deworming programs. In India, governments seem to heavily utilize Deworm the World when planning, implementing, and monitoring their deworming programs. In future countries, Deworm the World plans to pay the majority of financial program costs, which increases our expectation that the organization is critical to the program happening.

Below, we detail what we know with respect to Deworm the World’s experience in India, including:

1. Our limited understanding of how Indian states have historically decided to launch and maintain health programs with NGO assistance;
2. Deworm the World’s role in the mass school-based deworming programs that were in operation in India before the first national deworming day (NDD) in 2015, including our conversations with government officials in Rajasthan about their decision to launch a deworming program with Deworm the World’s assistance;
3. The national government’s interaction with Deworm the World as part of its decision to launch a national deworming day in India in 2015.

• Reasons new health programs may be started or halted in India
  • Deworm the World and Children’s Investment Fund Foundation (CIFF) have told us that Indian state governments often receive funds earmarked for broader health programs but often fail to spend these funds.214
  • We asked the Center for Global Development for a recommendation of someone to talk to in order to understand states’ decisions to initiate deworming programs, and CGD referred us to Professor Devesh Kapur.215 Dr. Kapur asserted that it takes significant internal political will or external stimulus (such as from a non-profit) to sufficiently overcome general bureaucratic inertia in India to sustain a new health program. This holds true even in cases where a national mandate exists for a program (as it does for school-based deworming). Dr. Kapur felt that in the majority of cases, nonprofit technical assistance was likely to increase the probability of a program’s going forward.216
  • Deworm the World told us that it believes that some school health programs have been stalled in Indian states due to negative media attention;217 one of Deworm the World's goals is to prevent these reports in the states in which it works.218 The documentation that Deworm the World has sent us supports – but does not fully demonstrate – the possibility that negative media undermined consistent mass deworming in Assam before the National Deworming Day in 2015. We have not attempted to independently verify that possibility because we do not think we would likely be able to do so effectively. Nonetheless, it is plausible that were decision-makers in Indian states to have the impression that negative reports could cause a program to be halted, they might be less willing to move ahead, and Deworm the World's assurances that it would help prevent these reports could increase the likelihood that a state agrees to implement deworming.
• Early deworming programs in India and Deworm the World's role
  • In 2013 Deworm the World and CIFF told us that the Indian government mandates that all states provide school-based deworming through the larger school-based Weekly Iron and Folic Acid Supplementation program and that states can request funding for deworming through this program.219 At the end of 2013, the only states in India that appear to have been implementing school-based deworming programs were Andhra Pradesh, Bihar, Delhi, Jharkhand, Punjab, Rajasthan, and possibly Assam.220 Deworm the World states that it (or others affiliated with it) played a key role in four of these seven states’ decisions to implement deworming.221 In late 2014, Deworm the World believed that more states were starting to launch their own deworming programs without assistance from Deworm the World but was unsure how many had successfully done so.222
  • Without involvement from Deworm the World, Assam initiated a vitamin A supplementation and deworming mass drug administration (MDA) in the 2010-2011 school year but reported low and conflicting coverage for that year.223 In the 2012-2013 school year Assam planned to implement a deworming program through the School Health Program, though we are not sure whether that program happened as planned.224 Assam did approach Deworm the World in late 2013 to explore opportunities for the organization to provide assistance to the state, though as of late 2015 Deworm the World has not become directly involved there.225
  • During our site visit to Deworm the World in Rajasthan, India, we spoke with three government officials who were involved in the deworming program.226 Of these, two stressed the importance of Deworm the World's technical assistance, and the person we perceive as having been most responsible (of the three) for the decision to go forward with deworming gave the impression that the availability of this technical assistance had been a key factor in deciding to go forward. It should be noted that one of the three gave the impression that Deworm the World's help was not needed, and all three conversations took place with multiple Deworm the World representatives present. With that said, the highest ranking of the three officials gave what we felt to be a nuanced and realistic picture of Deworm the World's impact that implied a substantial (while not determinative) role. She stated that a) the immediate availability of technical assistance improved her confidence that the program would proceed quickly and smoothly, b) she wasn’t sure whether the program would have proceeded if not for Deworm the World, and c) she was interested in finding a nonprofit technical assistance partner for at least one other program in a different category.227 This suggests that nonprofit technical assistance can be a key factor in progressing a program.
• The National Deworming Day and Deworm the World’s role
  • In 2014, Deworm the World told us that the Indian national government was pursuing the idea of having a coordinated national deworming day (NDD), whereby the national government would provide some assistance to states in implementing school-based deworming on a single day to encourage more states to implement the program (note that Deworm the World initially proposed this idea).228 Deworm the World said that the government asked it to provide technical assistance to the NDD, including helping develop the implementation strategy, designing and developing training and reference materials, community mobilization strategies, and monitoring and evaluation systems and reporting formats.229 The posters, ads for radio and television, training materials, and other documents that Deworm the World helped create for NDD are available at the website in this footnote.230
  • Deworm the World told us that, as part of its advisory role in the planning process, it advocated for a delay of the initial program from October 2014 to February 2015 because it believed there weren’t sufficient drug supplies.231
  • The first NDD occurred in February 2015,232 with 12 states participating (although deworming in one state was delayed until April).233 A second NDD occurred in February 2016, with 30 of India's 36 states and union territories participating.234

Does Deworm the World's work increase the quality of deworming programs?

Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are highly uncertain about this.

Deworm the World may improve program quality by:

• Increasing the chances that the first deworming round in each state begins earlier than it otherwise would have and that subsequent rounds occur on schedule.235
• Increasing the training quality by simplifying training material and creating a more robust training program for those who train representatives from each school.236
• Ensuring that support roles are staffed.237
• Improving focus and attention to detail, possibly increasing the likelihood that schools receive the materials and instructions necessary to implement the deworming program.238
• Expanding the scope of the program to a broader age group.239
• Increasing community acceptance of mass treatment and the ability of a program to avoid or withstand publicity associated with adverse events or seemingly related adverse events.240
• Implementing or advocating for monitoring systems. This may improve program quality by creating a mechanism through which implementers are held accountable. Also, monitoring systems could generate lessons that Deworm the World and the government could use in future rounds of treatment.241

Evidence that relates to Deworm the World’s impact on the quality of deworming programs includes the following, each discussed in more detail below:

• Testimony of a government official in Rajasthan in 2013
• The number of calls Deworm the World says its monitors made in India during the run-up to Deworming Day
• A training we observed in 2013

Testimony of a government official
We believe our strongest piece of evidence in favor of the idea that Deworm the World improves quality is the conversation we had with a Rajasthan nodal officer in 2013, who cited many of the points above and made the case that Deworm the World had played an important role in improving the quality of deworming, backing up an intuition that without external support, such programs would often be of lower quality. On the other hand, (a) this conversation took place with multiple Deworm the World representatives present, and the nodal officer had worked closely with Deworm the World; (b) another government official (on the same visit) stated that he believed Deworm the World's technical help was not needed and did not add value, as the state was accustomed to running school health programs and deworming is a relatively simple one (we do not put strong weight on (b) alone, since the government official may have had other incentives to give the message he did and had not worked as closely with the program as the Nodal Officer, but in context of the other observations we find (b) worth noting).

Monitoring calls before Deworming Day
As part of its standard monitoring process in India, Deworm the World has tele-callers place thousands of calls to government employees at different levels of government during the run-up to Deworming Day to ensure that preparations are proceeding as intended.242 Deworm the World reports that these calls helped to uncover problems, which were reported on the same day to the appropriate government officials to address.243

It also seems plausible that these calls reduce the likelihood that problems arise after the calls are placed, due to the signal of importance created by the call itself. We are unaware of how common these monitoring calls are as part of other government programs, nor are we aware of any data that could help quantify the size of the impact they have.

Training observed in 2013
We felt that the training we attended in 2013 (the only part of Deworm the World's work that we observed directly) had major limitations in terms of potential to improve program quality.244 Deworm the World has made changes to address these since this visit, though we have not observed a training since that time.245

Are there any negative or offsetting impacts?

We discuss several possible considerations but do not see significant concerns.

Administering deworming drugs seems to be a relatively straightforward program.246 However, there are potential issues that could reduce the effectiveness of some treatments, such as:

• Drug quality: For example, if drugs are not stored properly they may lose effectiveness or expire.247 Our understanding is that Deworm the World generally tests the quality of drugs and monitored storage conditions in each of its recent programs, and this information suggests there have been minimal issues.248
• Dosage: If the incorrect dosage is given, the drugs may not have the intended effect and/or children may experience additional side effects.249 It appears that for STH treatment, all children of a given age group are given the same dose of albendazole and that the dose is generally a single tablet for children 2 years old and above, and half a tablet for those between the ages of 1 and 2.250 Deworm the World monitors reported that, in recent programs in India, up to 9% of schools observed gave children less than the prescribed dose of albendazole and up to 5% gave more than the prescribed dose of albendazole.251
• Replacement of government funding: We have limited information about whether governments would pay for the parts of the program paid for by Deworm the World in its absence, though our impression is that they would not.252 We also have little information about what governments would use deworming funds for if they did not choose to implement deworming programs.
• Diversion of skilled labor: Drug distribution occurs only once or twice per year and is conducted by teachers in schools. Based on our site visit in Rajasthan, our impression is that the Nodal Officer (the state official who manages all state school-based programs), the Nodal Officer’s staff, and the people that the Nodal Officer manages throughout the state (Resource Persons and Community Development Project Officers) have significant capacity to take on additional programs, so their taking on this program doesn't impose a significant burden on their time.253 On the other hand, a principal we spoke with commented that he would prefer fewer school-based health programs because they take focus away from the school day, and teachers may not have as much capacity to add deworming to their school schedule in the countries to which Deworm the World expands.254
• Adverse effects and unintended consequences of taking deworming drugs: Our understanding is that expected side effects are minimal and there is little reason to be concerned that drug resistance is currently a major issue (more information from our report on deworming).

What do you get for your dollar?

This section examines the data that we have to inform our estimate of the expected cost-effectiveness of additional donations to Deworm the World.

Note that the number of lives significantly improved is a function of a number of difficult-to-estimate factors, many of which we discuss below. We incorporate these into a cost-effectiveness model which is available here.

We focus on the following questions:

• What is the cost per child treated of the deworming programs, and what percentage of this does Deworm the World bear? We estimate that in India children are dewormed for a total of about $0.27 per child, or $0.07 per child excluding the value of teachers’ and principals’ time spent on the program.255 In Kenya, we estimate the total cost per treatment at about $0.80 per treatment or $0.56 excluding the value of teachers’ and principals’ time spent on the program.256 We expect the cost per treatment to be closer to Kenya's costs in Deworm the World’s potential new countries; however, we are highly uncertain about this.
• How accurate are Deworm the World's reported coverage figures? Deworm the World uses reported coverage figures from the government when calculating its cost-per-treatment. We remain uncertain of the accuracy of these figures.
• How much impact do Deworm the World-supported programs have (per child treated) compared with the ones on which the evidence for deworming's positive impact on life outcomes is based? Because the key deworming studies provided treatments in areas of unusually high prevalence and, in some cases, areas with significant burdens of schistosomiasis infections, we believe that the programs that Deworm the World supports are likely to have less impact on a per-person basis.
• Does Deworm the World "leverage" government funds, such that its activities mobilize resources from other actors? We could imagine that Deworm the World's funds have substantial leverage but could also imagine that other actors’ involvement is causing Deworm the World to pay for things for which other actors would otherwise have paid. Deworm the World may have less leverage in its future programs than it has in past programs.

Our full cost-effectiveness model is available here.

What is the cost per treatment?

We are not sure where additional unrestricted funds given to Deworm the World will be spent (more below). Thus, when considering the cost-effectiveness of additional donations, we have chosen to consider the cost per treatment for the Deworm the World programs for which we have data: India and Kenya. Deworm the World told us that the cost per treatment in India is unusually low; it expects future programs' cost per treatment to be more similar to (though likely less than) the program that it supports in Kenya.257

Note that in our cost-per-treatment analyses below, we use data that Deworm the World has sent us for its most recent rounds of treatment in Kenya and India for which it has cost data. It is possible that this makes Deworm the World's program look more cost-effective than it actually is; for example, this may exclude start up costs from our analysis. Deworm the World has told us that it tries to capture all costs of its program, regardless of who pays for the cost (though we know that it does not includes the value of teacher time during trainings and on Deworming Day).258 To see cost-per-treatment figures across multiple years, see our previous reviews or GiveWell analysis of Deworm the World cost-per-treatment, 2016.259

India

As of April 2016, Deworm the World had estimated the total cost of nine deworming rounds in India.260 These estimates include the costs listed in the following tables. Each of the costs were paid by a combination of government, Deworm the World, and other partners. The following table shows the percentage breakdown for the most recent round of deworming in Bihar, Rajasthan, and Madhya Pradesh:261 Note that it does not include the value of school and government staff time during training, deworming day, and mop-up day.262

Deworm the World and others’ costs in Indian states’ deworming programs: 2014-15

Cost category	Percentage of total costs	% paid by DtWI
Policy & advocacy	1%	100%
Prevalence surveys	6%	100%
Drug procurement & management	38%	0%
Training & distribution costs	11%	36%
Public mobilization & community sensitization	16%	22%
Monitoring & evaluation	7%	86%
Program management	20%	100%
Total	100%	41%

Deworm the World has estimated some of the above costs because it did not have access to full cost data for governments and other partners, and boundaries between the cost categories are not always clear.263

The table below shows the costs to Deworm the World, its partners, and the governments, as well as estimates of the cost per child treated, for the most recent rounds of treatment in Bihar, Rajasthan, Madhya Pradesh.264 About two thirds of the total cost comes from contributions of time from government employees – which have been monetized according to salary levels – rather than financial costs. We would guess that appropriately valuing that time spent, and estimating how much time is spent by teachers and others due to the deworming program, is the largest source of uncertainty in the cost-per-child-treated calculation.265

Total cost per child treated - India

Expense category	Recent deworming rounds	Cost per child
Deworm the World	$1,473,141	$0.03
Partners	$1,214,409	$0.02
Government financial costs	$864,870	$0.02
Government staff time value	$9,791,115	$0.20
Subtotal without staff time	$3,553,185	$0.07
Total costs	$13,344,300	$0.27

Kenya

We estimate the total cost per treatment in Kenya to be about $0.80 per treatment (details below).266

In the third round of treatment in Kenya, Deworm the World estimated that the total cost per treatment was $0.56 (not including the value of teachers' time).267 Deworm the World's cost per treatment in Kenya is in the same range as our estimate of SCI's cost per treatment, which averages data across nine programs in Africa and three years.268 Our estimate of Deworm the World’s total cost per treatment in Kenya attempts to include all partners’ costs so that it represents everything required to deliver the treatments.269 Our estimate includes the value of teachers' time, even though this does not represent an additional financial cost to the program (because the government pays the teachers with or without the program). In our analysis of SCI, we estimate that 30% of the total program costs are in-kind support from the government.270 Applying the same approach to Deworm the World (for simplicity and consistency), we estimate that the total cost per treatment in Kenya is $0.80.271

Total cost per child treated - Kenya272

Expense category	Recent deworming rounds	Cost per child
Deworm the World	$3,034,625	$0.50
Partners	$329,688	$0.05
Government financial costs	$14,918	$0.00
Government staff time value	$1,448,241	$0.24
Subtotal without staff time	$3,379,230	$0.56
Total costs	$4,827,472	$0.80

Accuracy of coverage data

In calculating the cost-per-treatment figures above, we divide Deworm the World's costs for a program by the number of children reported to have been treated in the program. The number of children treated is obtained from coverage data, which is reported by governments to Deworm the World. While the coverage data reported by the government seems plausible based on monitors' observations, we remain unsure how accurate the government data is.

India

In India, the government's estimates – that about 80% of students enrolled in primary and secondary schools were dewormed in 2015 in states supported by Deworm the World – seem plausible based on monitoring results.273

The number of children treated in each school is recorded in classrooms and then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (school, node, block, district, and state) to create a reported coverage estimate for the entire state.274 The raw numbers produced by this process and reported by the state governments are here.

Monitors visit a random sample of schools to compare each school’s records of how many children it dewormed to the number of children that were dewormed according to its classroom records.275 Based on this data, it appears that many schools did not follow the recording protocol, but the schools that did had reasonable agreement on the number of children dewormed between the classroom records and the school-wide records (details in this footnote).276

It is our understanding that Deworm the World does not have a monitoring process to measure the accuracy of the aggregation process beyond the school level. Errors in the aggregation process could occur if those performing the aggregation have an incentive to overreport the number of children dewormed, or if reporting forms are lost (in which case we believe that deworming would be underreported).277

We calculated the portion of children that were dewormed – according to class records in schools which used class records or didn’t do deworming – at the schools monitors visited. Then, we compared this to the fraction of total students covered out of total students enrolled in government schools (details of methodology in this footnote).278

The results of estimating statewide coverage using both government-reported data and monitors’ data are in the below table.

Estimates of total statewide coverage (India, 2015)279

2015 deworming round	% of students dewormed based on government-reported figures	% of students dewormed based on monitors' observations (of class records)
Bihar	74%	65%
Rajasthan	89%	85%
Madhya Pradesh	90%	68%
Delhi	69%	77%
TOTAL (may not sum due to rounding)	80%	NA

While we don’t put much weight in the comparison above, the fact that we do not see a pattern of the government consistently over-reporting its coverage figures gives us more confidence that the reported figures are not grossly inaccurate.

Kenya
We have not looked in-depth at the data that Deworm the World has sent us related to coverage in Kenya. If we take the coverage reporting figures at face value, Kenya was able to treat 83% of its population of children (in counties participating in the national deworming program).280 According to the most recent summary monitoring report from Kenya that we have seen, monitors observed coverage to be 99% in the schools they visited.281 We are not sure what definition of "coverage" is being used by the report.282 We will place very low weight on these figures until we have vetted them further.

Baseline infection status

In this section, we discuss two key differences between Deworm the World's programs in India and Kenya and the deworming programs discussed in our intervention report on deworming: (a) the prevalence of particular worms and the prevalence of high intensity infections and (b) the existence of other neglected tropical disease programs that may treat the same condition. Given these differences, we are unsure how similar Deworm the World's impacts are to the impacts of the deworming program we have studied.

We are uncertain how these considerations apply to programs that Deworm the World will support with additional funding (more below). We would guess that future programs would also treat in areas with different worm types and significantly lower prevalence than those in the key deworming studies, largely because prevalence in those studies was unusually high.283

Differences in worm prevalence and intensity

Key pieces of evidence that we discuss in our report on deworming (Miguel and Kremer 2004, Baird et al 2012, and Croke 2014) are from deworming experiments conducted in Kenya and Uganda in the late 1990s and early 2000s. Prior to receiving deworming treatment, the participants in those studies had relatively high rates of moderate-to-heavy infections of schistosomes or hookworm.284

When we examine the baseline prevalence and intensity of infections in participants in Deworm the World's Kenya and India programs, we find:

• STH: Baseline STH prevalence in India and Kenya was generally lower than that in the studies providing the best evidence for the benefits of deworming (Bleakley 2007, Croke 2014, and Miguel and Kremer 2004).285
• Schistosomiasis: Schistosomiasis prevalence was also lower in Kenya than it was in the key studies. Schistosomiasis is not present in the vast majority of India (and Deworm the World does not treat for it there).286

In the table below, we compare the prevalence of Kenya and the Indian states that Deworm the World currently supports for which we have prevalence surveys (Rajasthan, Bihar, Madhya Pradesh, and Uttar Pradesh) to the populations in the Croke study and the Miguel and Kremer study.287 Deworm the World notes that there are relevant methodological differences between the prevalence surveys, which makes them difficult to compare (see our previous footnote for more detail on methodology); we do not fully understand which methodological differences Deworm the World is referring to and we believe the surveys are similar enough that comparison remains a useful exercise.288

Prevalence comparison

Worm (intensity)	Rajasthan (post-MDA)	Bihar (post-MDA)	Madhya Pradesh (post-MDA)	Uttar Pradesh (post-MDA)	Kenya (pre-MDA)	Croke (pre-MDA)	Miguel and Kremer (pre-MDA)
Schistosome prevalence (moderate/high)	0%	0%	0%	0%	Not reported	0%	7%
Hookworm prevalence (moderate/high)	Not measured	3.70%	Not reported	Not reported	0.4%	Not reported	15%
Ascaris prevalence (moderate/high)	Not measured	8.80%	Not reported	Not reported	7.9%	Not reported	16%
Trichuris prevalence (moderate/high)	Not measured	0.40%	Not reported	Not reported	0.1%	Not reported	10%
Schistosome prevalence (any)	0%	0%	0%	0%	1.8%	0.03%	22%
Hookworm prevalence (any)	1.0%	42.2%	12.0%	22.7%	16.9%	55.1%	77%
Ascaris prevalence (any)	20.2%	52.1%	0.2%	69.6%	19.2%	3.5%	42%
Trichuris prevalence (any)	0.2%	5.2%	0.0%	4.6%	5.4%	1.4%	55%

Note that the WHO does not recommend MDA in areas where less than 20% of children are infected with one or more types of soil-transmitted helminth.289 Similarly, after multiple (5-6) years of treatments, the WHO recommends further MDAs for areas with at least 10% STH prevalence.290 In Bihar, Rajasthan, Madhya Pradesh, and Uttar Pradesh Deworm the World recommended statewide deworming programs; in Kenya, all 66 districts surveyed are also being treated.291

The prevalences in the table above for Bihar and Madhya Pradesh were measured after multiple rounds of treatment, while Rajasthan's prevalence was measured after one round of treatment. This makes it difficult to compare the prevalences in a meaningful way. It is likely that prevalence was higher at baseline.

Deworm the World is currently in the process of conducting prevalence surveys in Vietnam and Pakistan.292 We have not yet asked Deworm the World for prevalence data from where it plans to work in Nigeria.293

We have seen some prevalence survey data for Ethiopia, from SCI (Deworm the World's partner there). We have not examined how representative the schools surveyed are of the full treatment area or fully vetted the methodology used for the prevalence survey. We have also made several large assumptions to make the data somewhat comparable to the data from other countries.294 We roughly estimate that the average prevalence for each type of STH in Ethiopia (at any intensity of infection) is around 7%, while the average prevalence for schistosomiasis (at any intensity of infection) is approximately 13%.295 We put little weight on these numbers.

We believe it is likely that the other countries in which Deworm the World is working or may work in the future will also differ from the locations in which the studies that provide the evidence-base for deworming were conducted. For example, our understanding is that, as in India, there is no schistosomiasis in Pakistan (one of the countries Deworm the World is starting to work in).

Treatment for lymphatic filariasis

In some of the countries that Deworm the World works, there are existing programs to treat lymphatic filariasis (LF).296 Albendazole, the same drug used to treat STH, is usually used in combination with one additional drug to treat LF (and the same dosage is used for both treatments).297 We discuss some details of India's LF program in this footnote.298 Deworm the World told us that Vietnam recently stopped its LF treatments, and that Indonesia and Cross River, Nigeria have active LF treatment programs.299 We do not know whether Pakistan or Ethiopia have active LF treatment programs in the areas that Deworm the World works or plans to work.

For areas that have existing LF treatment programs, the effect of Deworm the World’s support may be to transition an area from once-per-year deworming treatment to twice-per-year treatment.300 We have not yet seen prevalence data for several of the locations in which Deworm the World works now and may work in the future, so we are unsure (a) whether the prevalence in those areas is sufficiently high to justify additional treatment and (b) how the impact of a such a transition would compare to the impact of transitioning from no treatment to some treatment.

Are donations to Deworm the World leveraged?

We have written before about the complexity involved in trying to understand leverage in charity, and below we lay out the relevant issues worth considering in the case of Deworm the World.

Donors to Deworm the World support only expenses paid by Deworm the World. Generally, Deworm the World tries to work with governments and other funders to create funding arrangements where each partner is contributing some portion of the program's costs. Sometimes, governments pay most of the expenses. For example, Deworm the World's costs in India are only approximately 41% of the overall cost of the program (not including the value of teacher time).301

The role that Deworm the World's funds play in the program is an important consideration in determining the cost-effectiveness of donations. This role could range between:

• High leverage, high cost-effectiveness. Deworm the World's funds could have high leverage by: (a) causing government funds that otherwise would not have been spent or would have been spent on a lower-value program to be used in support of a deworming program; or (b) contributing a relatively small amount of funding to cause a deworming program to run significantly better, thereby reaching many more children than it otherwise would have.
• No leverage, average cost-effectiveness. Deworm the World's funding may allow the program to reach more children at a similar overall cost-per-treatment as it would have otherwise.
• Minimal contributions, low cost-effectiveness. If Deworm the World's work leads to few additional children receiving treatment, it may increase the overall cost-per-treatment.

In the past, we guessed that Deworm the World’s work in India played a role in increasing the likelihood that state governments conduct deworming programs, so we estimated that its leverage on funds used for scaling up India programs could have reasonably ranged from less than 1x (i.e., programs would have taken place without Deworm the World’s involvement) to 4-5x, though we were highly uncertain about this estimate.302 India may have been an especially promising place to achieve leverage because the Indian national government had made money available for state governments interested in implementing deworming, and states may have been more likely to implement deworming with only the additional of technical assistance.303

Future donations to Deworm the World are unlikely to be used for the India program (more). In the countries Deworm the World is starting to work in, governments may have less funding to support deworming. This may cause Deworm the World to pay a higher fraction of the overall cost of the program, making the upside potential for leverage of future donations more limited. For example, we estimate that Deworm the World bears the majority of total program costs in Kenya.304 Deworm the World has told us that it expects the cost per treatment of its future programs to be more similar to its program in Kenya than India, though we have not explicitly asked about whether it expects to pay a similar share of overall program costs as it does in its Kenya program.305

Is there room for more funding?

In short:

• Estimated needs: Deworm the World told us that it is not currently constrained by funding and does not expect to be constrained by funding before our next update (in November 2016).
• Cash on hand: As of the end of March 2016, Deworm the World held $17.9 million, of which $11.2 million was officially uncommitted. However, Deworm the World's budget indicates a plan to spend $13.3 million in 2016.
• Other sources of funds: As of the end of March 2016, Evidence Action, Deworm the World’s parent organization, held roughly $22.1 million, all of which was committed, with the exception of the uncommitted funding Deworm the World held.
• Past spending: We have limited information about how Deworm the World used unrestricted funds it received in the past.
• Additional considerations: Below, we discuss three issues that are material to a determination of Deworm the World’s room for more funding: (a) its preference for multi-year commitments from donors, (b) its expectation that it will utilize a new operating model when expanding into new countries, and (c) its relationship with Evidence Action which also has unrestricted funding available.

Available and expected funds

At the end of March 2016, Deworm the World held approximately $17.9 million, of which:306

• $1.4 million was restricted
• $5.3 million was unrestricted, but committed to projects
• $11.2 million was unrestricted and uncommitted

Of the amount committed to projects, $1.3 million was to be set aside for reserves.307 Although only $6.7 million of Deworm the World's funding was officially committed to projects, Deworm the World's 2016 budget indicates a plan to spend $13.3 million.308 Deworm the World has told us that it expects to receive significant additional restricted funding for Kenya, India, Vietnam, and Ethiopia during 2016; we have not yet asked for additional detail.309

Evidence Action held nearly $22.1 million by the end of March 2016; other than the $11.2 million uncommitted within Deworm the World, all of Evidence Action's funding was already committed.310 Of the $22.1 million, $2.5 million was unrestricted funding; most of this funding was committed to reserves or organizational development, although approximately $268,000 raised in the first quarter of 2016 will be committed to Dispensers for Safe Water, in accordance with Evidence Action's plans for 2016 unrestricted funds (see footnote).311

Uses of additional funding

Currently, Deworm the World holds enough unrestricted funding to move forward with all of the projects in new countries that it is interested in pursuing.312 In the future, Deworm the World may be looking for funding to put towards continuing its Kenya program beyond 2017, but this is highly uncertain.313

It is possible that Deworm the World could use additional funding to build out its reserves, although we are highly uncertain about this. Deworm the World has told us that neither it nor Evidence Action has finalized a reserves policy and that it is unsure how its reserves will work in conjunction with the reserves that Evidence Action holds or will hold (details in footnote).314

Past uses of unrestricted funds

In the past, Deworm the World has often spent unrestricted funding on different activities than it predicted before receiving this funding and more slowly than we expected. However, we believe that the alternative uses of unrestricted funding have been reasonable.

Activities funded vs. expectations

• 2013 expectations vs. 2014 actual: We detailed Deworm the World's use (and planned use) of the funding it raised in 2013-2014 in this November 2014 blog post.315 Overall, Deworm the World's funding decisions were roughly in line with our previous expectations.316
• 2014 expectations vs. 2015 actual: At the end of 2014, Deworm the World told us that it would spend additional unrestricted funding on new staff, overhead, expansion to Vietnam, and evaluation of new evidence-based programs related to deworming (details in footnote).317 We have limited information from the 2015 financial documents Deworm the World has sent us, and we are unable to tell if it spent funding as it claimed it would; we can see that it spent significantly less on expanding its program to Vietnam, more on expanding in India, and more on overhead than expected.318 Deworm the World told us by email that it allocated slightly over $1 million in unrestricted funding in 2015 (details in footnote).319 While we believe all of the uses of funds are reasonable, they did not closely match what Deworm the World had told us it would use the unrestricted funding for.320
• 2015 expectations vs. 2016 budget: At the end of 2015, Deworm the World told us that it intended to spend additional unrestricted funding primarily on multi-year deworming programs in new countries and on reserves.321 We are not sure if Deworm the World's 2016 budget closely follows this plan: many of the projected expenses in new countries look lower to us than we would expect, but Deworm the World expects to significantly increase its spending in new countries in subsequent years to meet its multi-year targets.322 For example, no new country (other than Pakistan) has an expected budget greater than $600,000.323 However, Deworm the World intends to spend almost $1.8 million in Pakistan this year, which is close to, although less than, one third of its original 3-year plan.324 Note that we are unable to tell from the 2016 budget how much of each activity will be supported by restricted funding or unrestricted funding.

Given Deworm the World's track record, we feel that we cannot be sure that Deworm the World will spend unrestricted funding in the ways it tells us it expects to. Additionally, we currently have a weak understanding of what expenses are covered by restricted funding and how Deworm the World determines additional activities to fund with its unrestricted funding. We intend to discuss this more with Deworm the World in the latter half of 2016.

Speed of spending down unrestricted funding

As of late 2015, Deworm the World had spent or allocated approximately half of the unrestricted funds it held over the past two years.325 Specifically, it had spent approximately one quarter of the unrestricted funds it held, and allocated another quarter to future activities.326 Very roughly, we would guess that Deworm the World spends its unrestricted funding approximately one to three years after receiving it.327

Considerations relevant to assessing Deworm the World’s room for more funding

Multi-year commitments

Deworm the World has told us that it prefers to have enough funding to make 3-5 year commitments when attempting to launch a new program. It told us that governments typically ask for multi-year commitments because a) deworming programs must be sustained over time to cause the desired impacts and b) governments want assurance that support will be sustained so that they can better plan how to use their funding.328 As a result, Deworm the World expects that having multiple years of allocated funding makes partnerships with governments (formalized by signed memoranda of understanding (MOUs)) more likely and reduces the amount of time before programs are launched.329

Additionally, Deworm the World has told us that having multiple years of funding for its programs allows staff to spend less of their time fundraising and more time on other aspects of their work.

We have not yet investigated this topic thoroughly, so we do not feel confident that 3-5 year commitments are necessary nor do we feel that Deworm the World’s preference is unreasonable.

Challenges of expanding to new countries using a new operating model

Deworm the World’s track record largely consists of work in Kenya and India. Launching programs in new countries may introduce challenges that are hard to predict, such as differences in cultural and bureaucratic expectations, increased difficulty of predicting program costs and success, and political and economic instability.

Deworm the World Initiative and Evidence Action

The Deworm the World Initiative is an initiative led by Evidence Action. Evidence Action includes other programs, of which the only other large program is Dispensers for Safe Water (DSW).330

This has some implications relevant to Deworm the World’s room for more funding: donations to Evidence Action, even if restricted to Deworm the World, might change the actions that staff take to fundraise (i.e., which grants they pursue, what type of funding they ask for). We've seen mixed evidence that this is the case:

• In early 2015, Evidence Action’s plan for using unrestricted funds included a relatively high priority to spend $0.8 million on Deworm the World. After receiving funds related to GiveWell’s recommendation that were designated for Deworm the World, Evidence Action allocated unrestricted funding to other programs instead of to Deworm the World. As such, it seems likely that $0.8 million of GiveWell directed funds (70% of GiveWell-directed funds to Deworm the World) caused on the margin more funding to Evidence Action’s other programs, rather than more dewormings to take place.331
• In 2016, Evidence Action's plans for its unrestricted funding included spending 32% of the funding on Deworm the World, 52% on Dispensers for Safe Water, 11% on Evidence Action Beta, and 5% on other expenses.332 We are not sure how this compares to Evidence Action's plans before the 2015 giving season, which were not shared with us. However, we note that this distribution mirrors the distribution of Evidence Action's total expenses in 2015.333

Global need for treatment

There appears to be a substantial unmet need for STH and schistosomiasis treatment globally.

A WHO report estimates that in 2014 about 47% of all children in need of STH deworming received treatment.334 We have not vetted this data, but it is consistent with our impression that there is a large global unmet need for STH deworming.

The Schistosomiasis Control Initiative shared an unpublished World Health Organization (WHO) estimate of the number of school-aged children and adults who require treatment for schistosomiasis and those who received treatment in 2014. We do not have permission to publish this data, but we note that only a small portion of those who WHO believes need treatment received it in 2014.335

Deworm the World as an organization

We believe that the Deworm the World Initiative, led by Evidence Action, is a strong organization:

• Track record: Deworm the World has a track record helping to start deworming programs where they do not already exist and providing ongoing support to programs in India and Kenya. Deworm the World has minimal experience starting new programs outside of India and Kenya, as it now hopes to do.
• Self-evaluation: Deworm the World collects a large amount of relevant data about its programs demonstrating a commitment to self-evaluation.
• Communication: Deworm the World has generally communicated clearly and directly with us, given thoughtful answers to our critical questions, and shared significant, substantive information.
• Transparency: Deworm the World is very transparent.

More on how we think about evaluating organizations at our 2012 blog post.

Sources

Document	Source
Alderman et al. 2006	Source (archive)
Alix Zwane conversation August 30th 2013	Unpublished
Alix Zwane conversation June 4th 2013	Source
Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013	Unpublished
Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013	Unpublished
Assam 2010 guidelines for deworming	Source
Assam midday meal report 2013	Source (archive)
Assam reproductive and child health 2011-2012	Source (archive)
Assam state programme implementation plan 2011-2012	Source (archive)
Baird et al 2012	Source
Bleakley 2007	Source (archive)
CIFF conversation September 10th 2013	Source
Croke 2014	Source (archive)
Devesh Kapur conversation October 14th 2013	Source
Deworm the World budget - 2016	Unpublished
Deworm the World, Kenya Coverage Reporting data, Year 3	Source
Deworm the World, Kenya Deworming Day data, Year 3	Source
Deworm the World, 2015 expense summary	Unpublished
Deworm the World, 2015 expense summary - by funder	Unpublished
Deworm the World, Ethiopia Workplan	Unpublished
Deworm the World, Kenya 2014-2015 program report	Source
Deworm the World, Kenya Narrative Report - Year 1	Source
Deworm the World, Kenya Narrative Report - Year 2, Quarter 4	Source
Deworm the World, Kenya Narrative Report - Year 3, Quarter 3	Source
Deworm the World, Kenya Year 2, DD - Main instrument	Source
Deworm the World, Kenya Year 3, DD - Main instrument	Source
Deworm the World, Kenya Year 2, Pre DD - School instrument	Source
Deworm the World, Kenya Year 3, Post DD - Coverage instrument	Source
Deworm the World, Madhya Pradesh cost-per-treatment - 2015	Unpublished
Deworm the World 2015 Uttar Pradesh prevalence survey report	Source
DSW 2012 GiveWell site visit	Source
DtWI 2013 GiveWell government interviews	Source
DtWI 2013 GiveWell site visit	Source
DtWI Assam research 2013	Source
DtWI Bihar 2011 cost data	Source
DtWI Bihar 2011 coverage data	Source
DtWI Bihar 2011 monitoring data for deworming day	Source
DtWI Bihar 2011 monitoring data for mop-up day	Source
DtWI Bihar 2011 Monitoring Form for Deworming Day	Source
DtWI Bihar 2011 monitoring form for mop-up day	Source
DtWI Bihar 2011 prevalence survey report	Source
DtWI Bihar 2011 program report	Source
DtWI Bihar 2012 cost data	Source
DtWI Bihar 2012 cost data details	Unpublished
DtWI Bihar 2012 coverage data	Source
DtWI Bihar 2012 monitoring data for coverage validation	Source
DtWI Bihar 2012 monitoring report	Source
DtWI Bihar 2014 cost data	Unpublished
DtWI Bihar 2014 program report	Source
DtWI Bihar 2014 program report annex 1	Source
DtWI Bihar 2014 program report annex 2	Source
DtWI Bihar 2015 independent monitoring tables	Source
DtWI Bihar 2015 monitoring data for coverage validation, schools	Source
DtWI Bihar 2015 monitoring data from deworming day, schools	Source
DtWI Bihar 2015 monitoring data from mopup day, schools	Source
DtWI Bihar 2015 monitoring survey for coverage validation, schools	Source
DtWI Bihar 2015 monitoring survey from deworming day, schools	Source
DtWI Bihar 2015 monitoring survey from mopup day, schools	Source
DtWI Bihar 2015 Prevalence Survey report	Source
DtWI Bihar 2015 Program report	Source
DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015	Unpublished
DtWI Chhattisgarh 2015 coverage validation report	Source
DtWI Chhattisgarh 2015 coverage validation tables	Unpublished
DtWI Chhattisgarh 2015 independent monitoring tables	Source
DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis	Source
DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools	Source
DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis	Source
DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools	Source
DtWI class register audits 2013	Source
DtWI cost narrative 2013	Source
DtWI Cost per treatment blog post January 2015	Source (archive)
DtWI cost per treatment summary 2013	Source
DtWI coverage data 2013 - 2014	Source
DtWI Delhi 2012 cost data	Source
DtWI Delhi 2012 coverage data by anganwadi	Source
DtWI Delhi 2012 coverage data by school	Source
DtWI Delhi 2012 coverage report	Source
DtWI Delhi 2012 monitoring data	Source
DtWI Delhi 2012 monitoring form deworming day	Source
DtWI Delhi 2012 prevalence survey design	Source
DtWI Delhi 2012 prevalence survey report	Source
DtWI Delhi 2012 program report	Source
DtWI Delhi 2013 cost data	Unpublished
DtWI Delhi 2013 program report	Source
DtWI Delhi 2015 independent monitoring tables	Source
DtWI Delhi 2015 monitoring data for coverage validation, anganwadis	Source
DtWI Delhi 2015 monitoring data for coverage validation, schools	Source
DtWI Delhi 2015 monitoring data from deworming day, anganwadis	Source
DtWI Delhi 2015 monitoring data from deworming day, schools	Source
DtWI Delhi 2015 monitoring data from mopup day, anganwadis	Source
DtWI Delhi 2015 monitoring data from mopup day, schools	Source
DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis	Source
DtWI Delhi 2015 monitoring survey for coverage validation, schools	Source
DtWI Delhi 2015 monitoring survey from deworming day, anganwadis	Source
DtWI Delhi 2015 monitoring survey from deworming day, schools	Source
DtWI Delhi 2015 monitoring survey from mopup day, anganwadis	Source
DtWI Delhi 2015 monitoring survey from mopup day, schools	Source
DtWI Delhi 2015 program report	Source
DtWI Kenya 2013-2014 cost per treatment data	Source
DtWI Kenya 2013-2014 program report	Source
DtWI Madhya Pradesh 2015 coverage validation form	Unpublished
DtWI Madhya Pradesh 2015 deworming day monitoring form	Unpublished
DtWI Madhya Pradesh 2015 independent monitoring tables	Source
DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools	Source
DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools	Source
DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools	Source
DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools	Source
DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools	Source
DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools	Source
DtWI Madhya Pradesh 2015 program report	Source
DtWI Monitoring Improvements 2014	Source
DtWI NDD blog post February 2015	Source (archive)
DtWI NDD Year 1 M&E review July 2015	Source
DtWI Rajasthan 2012 cost data	Source
DtWI Rajasthan 2012 cost data details	Unpublished
DtWI Rajasthan 2012 coverage data for anganwadi	Source
DtWI Rajasthan 2012 coverage data for schools	Source
DtWI Rajasthan 2012 monitoring data for coverage validation in anganwadis	Source
DtWI Rajasthan 2012 monitoring data for coverage validation in schools	Source
DtWI Rajasthan 2012 monitoring form coverage day	Source
DtWI Rajasthan 2012 monitoring form pre-deworming day	Source
DtWI Rajasthan 2012 monitoring report	Source
DtWI Rajasthan 2012 prevalence survey report	Source
DtWI Rajasthan 2013 cost data	Unpublished
DtWI Rajasthan 2013 prevalence survey report	Source
DtWI Rajasthan 2013 program report	Source
DtWI Rajasthan 2015 independent monitoring tables	Source
DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis	Source
DtWI Rajasthan 2015 monitoring data for coverage validation, schools	Source
DtWI Rajasthan 2015 monitoring data from deworming day, schools	Source
DtWI Rajasthan 2015 monitoring data from mopup day, schools	Source
DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis	Source
DtWI Rajasthan 2015 monitoring survey for coverage validation, schools	Source
DtWI Rajasthan 2015 monitoring survey from deworming day, schools	Source
DtWI Rajasthan 2015 monitoring survey from mopup day, schools	Source
DtWI Rajasthan 2015 program report	Source
Evidence Action 2014 budget	Unpublished
Evidence Action, 2015 financials by program	Unpublished
Evidence Action, blog post, January 8, 2015	Source (archive)
Evidence Action, blog post, January 16, 2015	Source (archive)
Evidence Action, blog post, June 12, 2015	Source (archive)
Evidence Action, blog post, December 21, 2015	Source (archive)
Evidence Action, Projected allocation of unrestricted funds, 2016	Source (archive)
Evidence Action 2015 draft budget	Unpublished
Evidence Action 2015 funding gap analysis	Source
Evidence Action cover letter 2013	Source
Evidence Action launch announcement 2013	Source (archive)
Evidence Action Q1 financials, 2016	Unpublished
Evidence Action website 2013	Source (archive)
Evidence Action website announcement April 2014	Source (archive)
Evidence Action website, Deworm the World Initiative (October 2015)	Source (archive)
Evidence Action website, Deworm the World Initiative (March 2016)	Source (archive)
Evidence Action website, Evidence Action Beta (October 2015)	Source (archive)
GiveWell analysis of Deworm the World 2014 Financial summary	Source
GiveWell analysis of Deworm the World cost per treatment	Source
GiveWell analysis of Deworm the World cost-per-treatment, 2016	Source
GiveWell DtWI 2013-2014 cost data summary	Source
GiveWell enrollment-based student coverage check 2015	Source
GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014	Source
GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013	Source
GiveWell's notes from site visit to India, October 2013	Source
GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013	Source
GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014	Source
GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014	Unpublished
GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015	Source
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015	Unpublished
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015	Source
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015	Unpublished
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014	Source
GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014	Source
GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015	Source
Global Alliance to Eliminate Lymphatic Filariasis - Prevention	Source (archive)
Grace Hollister conversation June 19th 2013	Source
Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016	Unpublished
Grace Hollister, conversation with GiveWell, April 20, 2016	Unpublished
Grace Hollister, conversation with GiveWell, May 5, 2016	Unpublished
Grace Hollister, conversation with GiveWell, June 13, 2016	Unpublished
Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015	Unpublished
Grace Hollister, Deworm the World Director, email to GiveWell, March 23, 2015	Unpublished
Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013	Unpublished
Grace Hollister, email to GiveWell, March 6, 2016	Unpublished
Grace Hollister, email to GiveWell, June 9, 2016	Unpublished
Harvard Business School Kenya Case Study A 2010	Unpublished
India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015	Source
India NDD documents 2015	Source (archive)
Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014	Unpublished
JPAL CEAs in education 2011	Source
Kabatereine et al. 2001	Source (archive)
KEMRI prevalence report - Year 2	Unpublished
LF treatment coverage 2015	Source (archive)
LF treatment drugs 2012	Source (archive)
Miguel and Kremer 2004	Source
Mwandawiro et al. 2013	Source (archive)
Preventive chemotherapy in human helminthiasis 2006	Source (archive)
Professor Devesh Kapur Biography 2013	Source (archive)
Reserve Bank of India, GDP per capita, Table 10, September 16, 2015	Source (archive)
SCI Ethiopia mapping of SCH and STH 2014	Unpublished
STH coalition framework for action November 2014	Source (archive)
U-DISE Elementary Thematic Maps 2015	Source (archive)
U-DISE Secondary Flash Statistics 2015	Source (archive)
U-DISE Secondary Thematic Maps 2015	Source (archive)
WHO, Helminth control in school-age children	Source
WHO Preventive chemotherapy in human helminthiasis	Source (archive)
WHO schistosomiasis treatment gap data	Unpublished
WHO soil-transmitted helminthiases 2012	Source (archive)
WHO STH factsheet	Source (archive)
WHO STH treatment report	Source (archive)
WHO Weekly epidemiological record, 3 April 2015	Source (archive)
WHO Weekly epidemiological record, 6 March 2015	Source (archive)
WHO Weekly epidemiological record, 18 December 2015	Source
World Schistosomiasis Risk Chart 2012	Source

Uncited supplementary materials

Document	Source
DtWI Bihar 2011 monitoring forms phase I	Source
DtWI Bihar 2012 monitoring data deworming day anon	Source
DtWI Bihar 2012 monitoring data mop-up anon	Source
DtWI Bihar 2014 program report annex 3a	Source
DtWI Bihar 2014 program report annex 3b	Source
DtWI Bihar 2014 program report annex 3c	Source
DtWI Bihar 2014 program report annex 3d	Source
DtWI Bihar 2014 program report annex 4a	Source
DtWI Bihar 2014 program report annex 4b	Source
DtWI Bihar 2014 program report annex 4c	Source
DtWI Bihar 2014 program report annex 5a	Source
DtWI Bihar 2014 program report annex 5b	Source
DtWI Bihar 2014 program report annex 6a	Source
DtWI Bihar 2014 program report annex 6b	Source
DtWI Bihar 2014 program report annex 7	Source
DtWI Bihar 2014 program report annex 8	Source
DtWI cost per child description	Source
DtWI Kenya site visit 2012	Source
DtWI Rajasthan 2012 monitoring data anganwadi deworming day anon	Source
DtWI Rajasthan 2012 monitoring data anganwadi mop-up anon	Source
DtWI Rajasthan 2012 monitoring data anganwadi prep anon	Source
DtWI Rajasthan 2012 monitoring data schools deworming day anon	Source
DtWI Rajasthan 2012 monitoring data schools mop-up anon	Source
DtWI Rajasthan 2012 monitoring data schools prep anon	Source

• 1
  • GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013
  • Alix Zwane conversation June 4th 2013
  • Grace Hollister conversation June 19th 2013
  • Alix Zwane conversation August 30th 2013
  • Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013
  • GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013
  • GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014
  • GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014
  • GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014
  • GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015
  • GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015

• 2

  CIFF conversation September 10th 2013

• 3

  This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.

• 4
  • "More specifically, Evidence Action advocates for school-based deworming to policymakers and provides technical assistance to launch, strengthen and sustain school-based deworming programs." Evidence Action website, Deworm the World Initiative (March 2016)
  • An example of Deworm the World supporting a program with funding: "Deworm the World is working in partnership with Thrive Networks in Vietnam on an integrated program of both deworming and water, sanitation, and hygiene (WASH) education, and this includes an RCT to explore the impact of hygiene education in combination with deworming. This program is funded primarily by Dubai Cares, with Deworm the World slated to provide unrestricted funding for certain program components." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 5.

• 5
  • "Is mass treatment justified? On cost-effectiveness grounds we believe that it clearly is, as the cost of treatment is cheaper than individual screening. The WHO states that the cost of screening is four to ten times that of the treatment itself. Because the drugs are very safe and has no side effects for the uninfected, the WHO does not recommend individual screening. The WHO instead recommends mass drug administration in areas where more than 20% of children are infected." Evidence Action website, Deworm the World Initiative (March 2016)
  • Deworm the World focuses on school-based programs because the highest burdens for STH and schistosomiasis (the two diseases that Deworm the World targets) tend to be observed in children. For this reason, a significant decrease in the worm burden in children for these diseases translates to a significant decrease in the burden across an entire community. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 6
  • "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis. Deworm the World is one of the only organizations focused on STH (while there are other programs that focus on schistosomiasis), and there is significant room to scale-up STH programs. In some ways, schistosomiasis has an even larger gap to fill than STH. If Deworm the World shifted its focus to include schistosomiasis, it might widen the existing STH gap. Additionally, Deworm the World specializes in school-based deworming, which is not the ideal approach in every situation." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5.
  • This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.

• 7

  "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5.

• 8

  For example, Deworm the World is partnering, or planning to partner, with local organizations in Pakistan, Vietnam, and Nigeria:

  • Nigeria: "Deworm the World is in discussions with a potential partner which plans to work in Cross River on other integrated NTD treatment. The need to scale up treatment for schistosomiasis and STH among school-age children has not yet been addressed, and Deworm the World has been in discussions with this partner, the state NTD coordinator, and other state officials about creating a school-based deworming program to treat both STH and schistosomiasis beginning in 2016." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, pg. 6.
  • Vietnam: "[In Vietnam] Deworm the World’s partnership with both the government and Thrive Networks is a new working model for Deworm the World; elsewhere, it has supported government implementation or run its program independently. Deworm the World has only one staff member in the country. Dubai Cares provides most of the program funding. All funding goes to Thrive Networks, which provides money to the government for implementation expenses. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 3.
  • Pakistan: "[In Pakistan, Deworm the World] plans to contract with a local organization to do these [prevalence] surveys, but does not yet have a signed agreement...Deworm the World plans to work in partnership with the same local organization to provide technical support, likely beginning in Punjab and later expanding to Sindh." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.

• 9

  For example, Deworm the World has considered supporting evaluations or monitoring of different deworming-related programs:

  • "If funding permits, the Kenyan government may begin a lymphatic filariasis treatment program along its coast. If it does so, Deworm the World will provide process monitoring and coverage validation for the treatments." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 3.
  • "We are contributing approx $111k to the TUMIKIA and TakeUp studies, complementary studies leveraging the Kenya program to look at the potential for breaking STH transmission." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Deworm the World has told us that it is also interested in opportunistically evaluating new evidence-based programs that may efficiently complement deworming. GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014
  • Integrated deworming and sanitation or neglected tropical disease (NTD) programs are popular, and in some cases country governments are primarily enthusiastic about partnerships with NGOs that are working on integrated programs. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 10

  "In January 2007, the [Young Global Leaders] launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs." Harvard Business School Kenya Case Study A 2010 Pg 7.

• 11
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Deworm the World has supported deworming activities in India since 2009, in Kenya since 2012, and in Ethiopia since 2014:
    • "[Where We Work, Bihar State, India]: 1st deworming round in 2011 reached 17 million children." Evidence Action website, Deworm the World Initiative (March 2016)
    • "[Where We Work, Kenya] With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively." Evidence Action website, Deworm the World Initiative (March 2016)
    • Deworm the World supported a pilot deworming program in Ethiopia in April 2015 and another deworming program in October and November of 2015. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Updates on activities in India, Kenya, and Ethiopia were discussed in various conversations; for example, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015.
  • Plans for Nigeria, Pakistan, and Vietnam were discussed here: GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015

• 12

  "The government is fully responsible for program implementation, and … these programs leverage thousands of govt personnel from health and education to be able to run." Grace Hollister, email to GiveWell, June 9, 2016

• 13

  "Note that there are not single deworming days in all countries, nor do all countries have a mop-up day. In all cases there are mass campaigns, but the structure varies by country." Grace Hollister, email to GiveWell, June 9, 2016

• 14
  • This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.
  • For example, Deworm the World has assisted in a number of areas in Andhra Pradesh and Bihar:
    • "Andhra Pradesh...
      Deworm the World’s contributions
      • Prevalence survey...
      • Operational support
        • Helped government develop operational plans and budgets
        • Coordinated cross-sectoral partners through the establishment of a State School Health Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS)
        • Coordinated drug donation made by Feed the Children
        • Designed a monitoring and evaluation (M&E) system
        • Created government tableau for community awareness
      • Trainings
        • Conducted a master training session for program
        • Designed training cascade for the master trainees to train the rest of the implementers
        • Designed training materials
        • Developed materials and campaigns for community sensitization
    • Bihar…
      Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andhra Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO…

      Delhi…
      In addition to the standard contributions (see Andhra Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children.

      Rajasthan…
      DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO."

    @Grace Hollister Conversation June 19th 2013@, Pg 1-4.

• 15
  • "As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program."
    @Alix Zwane Conversation June 4th 2013@, Pg 2.
  • "The Nepalese government’s planning commission initiated contact with Deworm the World , which sent its India country director, Priya Jha, to Kathmandu for a meeting, which J-PAL facilitated. The conversation revealed that school-based health programs are not fully operating in all areas of the country, and Nepalese officials have expressed interest in getting technical assistance from Deworm the World, particularly with scaling up their programs. Deworm the World has contacted the government about further steps and is awaiting a response from health and education officials." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pg 1
  • "Deworm the World does not yet have an agreement with the government in Pakistan to conduct the surveys, but hopes to accomplish this in the next month, and anticipates that the prevalence surveys will be conducted beginning in January or February of 2016. It is expected that a clear articulation of need will be an important factor in building a strong case to the government in favor of deworming programs, and it may be best to wait until the results of the surveys are available in the second quarter of 2016 before beginning discussions with the government on a scaled school-based program. Treatment may not begin until 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7
  • Deworm the World's advocacy often occurs side-by-side with Deworm the World's technical assistance; once Deworm the World proves that a deworming program can be well-executed, it is easier to interest national governments in funding deworming programs. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 16

  For example:

  • "DtW has been involved in deworming programs in four different states, and is still actively involved of three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement...In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. [Note: these notes contain additional examples of Deworm the World's advocacy for other states in India]
  • "Together, we suggested to the Federal Ministry of Health that they expand this initial work. What would it take to treat at least 75% of all at-risk school-age children in the country and to launch a truly national program?

    SCI helped Oumer Shafi, the committed and action-oriented Coordinator for Neglected Tropical Diseases in the Federal Ministry of Health, develop a detailed action plan. This entailed sophisticated statistical analysis to determine how many deworming sites would be required to reach at least 80% of kids at risk.

    Meanwhile, I worked closely with Birhan Mengistu, an up-and-coming leader seconded from the World Health Organization, and with other Ministry of Health staff. We sat for hours hunched over laptop screens to develop detailed five-year budgets, talking through row after row of spreadsheets and reviewing everything from the cost of fuel for drug transport to the needs of teachers.

    Together with the Federal Ministry of Health, we were able to think and act boldly. We are excited to continue to partner with SCI and are seeking other partners who also share common goals and values to rapidly scale school-based deworming in endemic countries.

    ...When we floated the idea of vastly increasing the scope of the originally proposed deworming rounds to be a truly national plan treating upwards of 75% of all children at-risk, Shafi didn’t flinch."
    Evidence Action, blog post, June 12, 2015

  • Other advocacy activities can include discussing: "how deworming can fit into the current policy environment and policy priorities of a government, how such a program can/should be financed, the robust evidence of impact, how a country can best take advantage of WHO drug donations, encourage program champions within government, help establish program governance structures. Once a program is established, advocacy doesn’t end – we work with govts to ensure the continuation of the above. Typically we refer to this group of activities as policy and advocacy, because there is a heavy emphasis on the former." Grace Hollister, email to GiveWell, June 9, 2016

  These discussions also provide opportunities for Deworm the World to assess how well a deworming program with the government might run. For example, if Deworm the World discovered from its advocacy discussions that there were high rates of teacher or student absenteeism, then a school-based deworming program may not work in the country. Deworm the World assesses risks like this through a diagnostic survey of the country’s capacity, including school attendance rates, which must be sufficiently high if a school-based deworming program is to succeed. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 17

  Deworm the World builds strong working relationships with governments to try to ensure that its programs will be effective, and it will not commit to a program if it does not foresee success in that country. It can decide to abandon plans for a program before a memorandum of understanding (MoU) is signed. In one state in India, Jharkhand, Deworm the World explored a program, but did not build a strong working relationship with the government, so Deworm the World pulled out of discussions before discussing an MoU or investing much money. The discussion stage with governments is important for helping Deworm the World assess the government's position and viability as a partner. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 18

  "The global STH community has changed significantly in the last couple of years, especially due to the formation of the STH Coalition. The community is now prioritizing STH (in a way similar to how LF became prioritized with the formation of the Global Alliance to Eliminate Lymphatic Filariasis, which has seen significant success).
  The STH community is developing plans to scale-up treatment, especially in high burden countries such as:

  • Ethiopia
  • Nigeria
  • India
  • Pakistan
  • The Democratic Republic of the Congo
  • Indonesia
  • The Philippines
  • Tanzania

  As part of the STH Coalition, Evidence Action is chairing a working group on school-aged children. It has used some of its unrestricted funding to hire consultants to create “snapshots” of each country, including obstacles, gaps, potential strategies, and financial needs.

  Deworm the World expects to see an increase in partnerships between the various groups in the STH community. Deworm the World hopes to leverage partnerships with existing organizations in, e.g., Nigeria, Pakistan, Ethiopia, etc., to provide catalytic support (rather than opening its own offices in those places)." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 6.

• 19
  • "We work with epidemiologists and local partners to assess worm prevalence and intensity, obtaining data to develop a targeted treatment strategy and to determine parasitological impact once programs are in place." Evidence Action website, Deworm the World Initiative (March 2016)
  • A few examples of Deworm the World assisting with prevalence surveys include:
    • "Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children." Grace Hollister conversation June 19th 2013, Pg 1.
    • "DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment." Grace Hollister conversation June 19th 2013, Pg 2.
    • "In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city." Grace Hollister conversation June 19th 2013, Pg 3.
    • "DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year." Grace Hollister conversation June 19th 2013, Pg 4.
  • For example, Deworm the World is currently supporting prevalence surveys in Pakistan because Pakistan has not yet been "mapped" (i.e., prevalence surveys have not yet been conducted in Pakistan), so nobody knows how heavy the worm burden is in Pakistan or where deworming efforts should be focused. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 20
  • "The WHO reports that Pakistan is endemic for STH, but there is not yet sufficient evidence of prevalence and intensity to develop an evidence-based treatment strategy. Deworm the World has committed unrestricted funding to fund prevalence surveys in two large provinces, Punjab and Sindh. It is targeting these provinces because their school enrollment rates are high, the areas are fairly secure, and they contain a significant percentage of the population of Pakistan. Deworm the World plans to contract with a local organization to do these surveys, but does not yet have a signed agreement." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.
  • "Deworm the World has estimated very roughly that its three-year program in Nepal would cost about $6 million ($2 million per year). This is based on a cost per child of no more than $0.50 and a target population of about 6 million children. However, the latter estimate is based on the outdated prevalence survey data mentioned above. Deworm the World will need to conduct a new survey to determine an exact target population and a more accurate budget." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pgs 1-2.
  • "Deworm the World will likely wait to expand its Vietnam activities until further mapping and impact evaluation have been completed." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pg 7.

• 21
  • Deworm the World originally planned to do prevalence surveys every few years but may do them less frequently going forward.
  • "DtWI would like to do prevalence surveys after every 3 years or so. Ideally, prevalence surveys would be carried out after every third round of treatment immediately prior to the following round." GiveWell's notes from site visit to India, October 2013
  • "Note: this [follow-up prevalence survey] strategy is evolving; WHO recommendations are to conduct sentinel site surveys after 5-6 rounds of treatment, and we are moving in that direction. Key is how a new survey would impact the treatment strategy" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Key M&E activities in India include "post-round 3 impact measurement" prevalence surveys. DtWI NDD Year 1 M&E review July 2015, Pg 3.
  • Deworm the World has said that the Rajasthan follow-up prevalence survey is tentatively planned for late 2017. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015

• 22

  Examples of Deworm the World working with partners on prevalence surveys:

  • "The WHO reports that Pakistan is endemic for STH, but there is not yet sufficient evidence of prevalence and intensity to develop an evidence-based treatment strategy. Deworm the World has committed unrestricted funding to fund prevalence surveys in two large provinces, Punjab and Sindh. It is targeting these provinces because their school enrollment rates are high, the areas are fairly secure, and they contain a significant percentage of the population of Pakistan. Deworm the World plans to contract with a local organization to do these surveys, but does not yet have a signed agreement." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.
  • "Following three rounds of school-based deworming, Evidence Action - Deworm the World Initiative recommended a second prevalence survey to the Bihar government, in order to understand the effect of deworming in Bihar on STH infection levels. With approvals from the State Government, in January and February 2015, Evidence Action - Deworm the World Initiative, conducted an STH prevalence survey among school-age children in government primary schools in Bihar. The survey took place in 65 schools in 14 districts, covering all three agro-climatic zones in the state. The National Institute of Epidemiology – Chennai (NIE) designed the survey, and analyzed the dataset to produce epidemiological findings. Field teams hired through GfK Mode (an agency with prior experience in sample collection for STH prevalence surveys), visited the households of children in the selected schools to collect stool samples and information related to school, household, deworming, and sanitation, to better understand infection patterns and allow for sample weighting. The 2 Post Graduate Institute of Medical Education and Research – Chandigarh (PGIMER) analyzed stool samples in field laboratories, which were set up in district and block health facilities, using the WHO recommended Kato-Katz method." DtWI Bihar 2015 Prevalence Survey report, Pgs 1-2.

• 23
  • For example, Deworm the World told us that before it started conversations with the Indian government, the government was not aware that it could obtain albendazole for free from the World Health Organization. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Deworm the World also claims that it assists with other high-level tasks, such as creating a program budget: "We work closely with the Ministries of Education and Health to design a program with joint ownership, develop operational plans and budgets, coordinate logistics, and provide on-the-ground support to ensure a high quality outcome." Evidence Action website, Deworm the World Initiative (March 2016).
  • Note that we do not feel like we have a strong understanding of Deworm the World's activities in this area; for example, we have not asked Deworm the World what it has brought to the planning, budgeting, or logistics processes that would not have otherwise been included. We do not have a strong sense from Deworm the World's website about what these activities involve (e.g., we do not know what it means for Deworm the World to have "coordinated logistics").

• 24

  For example: "DtWI provided support to the state government in submitting the drug requisition to WHO in March 2013, as well as in shipping, custom clearances and transportation upon arrival in India." DtWI Rajasthan 2013 program report, Pg 6.

• 25
  • Training cascades involve a series of trainings that start at high levels in the government, and proceed in a step-wise fashion down to local levels, where teachers are eventually trained. At each training, materials (such as deworming drugs and posters that notify the community about deworming day) are passed down from the staff member leading the training to the staff members attending the training, until materials eventually reach teachers.
  • "Transportation of tablets to all districts was managed and supported by DtWI, in coordination with SHS and DHFW to the district level dispensary, from where they were collected by the respective teachers for their school. For anganwadis, the syrups were sent to the nodal officer who arranged further transportation to the supervisors who in turn handed over to the anganwadis. Training about health education on types of STH, need for deworming, transmission of worms through open defecation and other practices and how to safely administer deworming drug was conducted through a cascade model. In the first step of the cascade, training of trainers was conducted at the state level where a total of 1040 participants attended (390 WCD + 650 education department) in 60 sessions. These trainers further trained 3032 headmasters and 3032 teachers and 10,500anganwadi workers in groups of 30 participants. The training on deworming was integrated with training for the WIFS program to effectively utilize time of participants and trainers and reduce training costs. A simplified training manual was developed that included content on deworming and WIFS into a single document. At the state level training sessions, training videos on three types of soil transmitted helminths (ascaris, trichuris and hookworm) and worm infestation cycle were also used. DtWI hired district coordinators (DCs) to provide short term support to the deworming program. They played a key role in ensuring that drugs in adequate quantities were available with the district-level dispensaries and nodal offices for further distribution. The DCs collated information on shortfall or surplus of drugs at district level and shared information with the DtWI state team, who coordinated with SHS to ensured-distribution or fresh supplies to districts facing drug deficits." DtWI Delhi 2013 program report, Pg 7.
  • See Pgs. 12-13 of DtWI Delhi 2015 program report for a visual representation and description of a recent training and distribution cascade.

• 26
  • "We consistently design and support training through an efficient multi-tier cascade approach that is tailored to the local context, ensuring knowledge reaches from the national level all the way to the teachers responsible for administering deworming medication." Evidence Action website, Deworm the World Initiative (March 2016).
  • For example, a description of the training cascade in Kenya: "The National School-Based Deworming Programme uses a cascade implementation model that efficiently and cost-effectively delivers training materials, deworming tablets, monitoring forms, funds, trainings other programme materials and resources from the national level to schools. At the national level, the Programme trains a team of MoEST and MoH officials as master trainers, requisitions deworming tablets through the MoH, and develops treatment and implementation strategies, training materials and monitoring tools. Thereafter, an initial planning meeting is held with county and sub-county leadership. This meeting is followed by two levels of trainings on how to successfully implement the Deworming Programme: Sub-County Training and Teacher Trainings. These trainings prepare sub-county and division officials to plan subsequent programme activities within the cascade, distribution of materials, planning of deworming and community mobilization and sensitization. After these trainings and community mobilization, the critical day of implementation occurs – Deworming Day – where teachers administer deworming tablets to millions of children in over 11,000 schools across Kenya and fill in monitoring forms to capture treatment data. These forms and any unused deworming tablets are moved up through a “Reverse Cascade” as described below. The cascade model helps to manage the national scale of the NSBDP, and therefore, builds capacity for successful implementation at various levels. Additionally, the cascade brings together MoEST and MoH personnel through collaborative leadership responsibilities for the planning, implementation and monitoring of programme activities at all levels. The cascade is outlined in the infographic below." DtWI Kenya 2013-2014 program report, Pg 5.
  • "Drug distribution: As per NDD operational guidelines, and established best practice, drug distribution was integrated with the training cascade (as detailed in the training section below), whereby NDD kits were provided to health functionaries at the district level trainings for onward distribution. The kits included drugs, IEC materials, and reporting forms." DtWI Madhya Pradesh 2015 program report, Pg 12.

• 27

  For example:

  • "DtWI provided technical expertise across all program components, and served as the primary coordinating body among implementing agencies. DtWI facilitated drug donations from WHO, provided professional master trainers for training of trainers, developed training materials such as flipcharts, modified training booklet and reporting forms, designed and carried out independent monitoring, and developed adverse event protocols." DtWI Rajasthan 2013 program report, Pg 5.
  • "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to districte medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materilas, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." DtWI Madhya Pradesh 2015 program report, Pg 11.
  • "We hire master trainers, or train govt staff to be master trainers." Grace Hollister, email to GiveWell, June 9, 2016

• 28
  • Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16
  • Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15
  • Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15
  • Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17

• 29

  "We work with governments and communications experts to design locally appropriate awareness campaigns to communicate messages through a wide variety of channels to increase public acceptance and effectiveness of deworming programs." Evidence Action website, Deworm the World Initiative (March 2016)

• 30

  "As part of their training, school headmasters/teachers were instructed to share information on the deworming program in the morning prayer sessions at their respective schools on a daily basis from October 6, 2013 onwards. They were also advised to convene school management committee meetings to communicate about the benefits of deworming and the schedule of deworming program. School headmasters were also advised to carry out student rallies / processions (prabhat pheri) to create awareness in the communities." DtWI Rajasthan 2013 program report, Pgs 8-9.

• 31
  • "One other key strategy adopted by DtWI to spread awareness was through text (SMS) reminders over mobile phones to school teachers, headmasters, Child Development Project Officers (CDPOs) and lady supervisors as a reminder about deworming day. SMSs were also used to reinforce precautions on drug administration, such as not giving drugs on an empty stomach, but only after midday meals and not giving drugs to sick children. In all, about 80,000 text messages were sent to school teachers and headmasters three times – a total of 2,40,000 messages. These messages were sent a day before deworming day, on mop-up day and after mop-up day. About 1400 such messages were sent twice to lady supervisors and CDPOs on a day before deworming day and on mop-up day. Similarly, five rounds of around 2400 text messages were sent to block level officials to expedite coverage reporting. This was an example of ensuring last-mile communication at low cost of about 12 paisa per message (or roughly 1/5th of a cent)." DtWI Rajasthan 2015 program report, Pg 9.
  • "Additionally, mike announcements were made at public places in blocks and districte headquarters by Evidence Action for 5 days, closer to deworming day (annexure E.4)." DtWI Bihar 2015 Program report, Pg 13.
  • "The state Health Society Bihar and Evidence Action rolled out a media mix to generate community awareness and increase program visibility to improve coverage in the state (Annexure E.1). We supported teh adaptation and contextualization of prototypes from the National Deworming Day IEC resource toolkit. At the state level, State Health Society Bihar, in coordination with the Department of Public Relation, Government of Bihar, published newspaper advertisement in four dailies one day prior to deworming and mop up day, i.e., on 20 and 25 February (Annexure E.2). Radio jingles, customized into three local dialects, were aired from 15 to 26 February on the All India Radio to maximize outreach to the community. For additional visibility of the program at the community level, State Health Society Bihar printed 513,625 posters (7 for each school, including distribution in the local community), 1068 banners for Primary Health Centers, hoardings at 38 district headquarter. All of these were adapted and contextualized by Evidence Action." DtWI Bihar 2015 Program report, Pg 12.
  • Other community awareness activities include creating posters to display at schools or advertising the deworming day in the newspaper: "Activities designed to enhance community awareness on deworming were rolled out to improve overall program coverage. The awareness activities included newspaper advertisements a day prior to the deworming day; a 60-second radio jingle aired on 3 FM channels from April 7 to 15 by School Health Scheme, and banners displayed at schools. Evidence Action was part of the committee formed by the state government for contextualization of the radio jingle. Evidence Action extended support to the state in contextualizing IEC materials from the National Deworming Day guidelines. The Directorate of Family Welfare also independently developed and printed handbills for the distribution at anganwadis to mobilize people on deworming day. The School Health Scheme provided banners to the schools, the distribution of which was integrated in trainings for teachers. The Delhi state government also used an e-portal to disseminate key information , including dates for deworming and mop up days, benefits of deworming, and details of the launch event. " DtWI Delhi 2015 program report, Pgs 11-12.

• 32

  "We help governments design monitoring systems to measure effectiveness in achieving intended program results. We also conduct independent monitoring to validate program results, and evaluate the impact of programs in reducing worm prevalence and intensity." Evidence Action website, Deworm the World Initiative (March 2016)

• 33

  Note that Deworm the World hires monitors for the first and third type of monitoring data collected below, but that the second is collected entirely by government staff: "Coverage reporting is done by the government- we sometimes assist in the data analysis, designing reporting forms, and ensuring that the 'reverse cascade' is appropriately designed." Grace Hollister, email to GiveWell, June 9, 2016

• 34

  "Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program." DtWI Madhya Pradesh 2015 program report, Pg 16.

• 35
  • In India, on Deworming Day and Mop Up Day, Deworm the World commissions independent monitors to go schools who gather data on whether principals and teachers are prepared for deworming day, the availability of drugs and supplementary materials, whether students are being dewormed, whether proper procedures are being followed, and more. For example, see Deworming Day monitoring data from Rajasthan in 2015: DtWI Rajasthan 2015 monitoring survey from deworming day, schools (shows which questions were asked) and DtWI Rajasthan 2015 monitoring data from deworming day, schools (shows the survey responses).
  • Kenya:
    • Trainings monitoring: "PMCV [Process Monitoring and Coverage Validation] officers observed 36 CHEW [Community Health Extension Worker] Forums aimed at introducing the deworming sensitization message and materials/methods as well as asigning CHEWs to schools for monitoring. A successful community health extension worker forum is one that starts on time and where all the materials were present. Overall, 63% of participants arrived before training, whereas 22% arrived 1hr after the forum had begun and 15% of participants arrived more than 1hr after the forum’s commencement. Lateness appears to be a commonality to all training sessions. Materials required for CHEW training include a powerpoint printout, CHEW checklist and Severe Adverse Event (SAE; side-effects of the drugs) protocol.
      In 51% of forums, ALL of the Materials Pack was distributed at the start of the forum. In 13% of forums, SOME of the Materials Pack was distributed at the start of the forum. In 36% of forums, NONE of the Materials Pack was distributed at the start of the forum." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 9.
    • Community sensitization monitoring: "A number of parents were also interview at schools on Deworming Day regarding their knowledge of deworming and the source of that knowledge. The intention behind this exercise was to compare the information source to those interviewed prior to deworming as a measure of consistency. In Figure 4, the results of the interviews pre-deworming day are compared with those parents interviewed on deworming day. The results remain largely similar, however more parents reported getting their information from ‘other’ sources (51%) when interviewed on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 14.
    • School preparedness monitoring: "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.
    • Deworming Day and Mop Up Day monitoring: "PMCV field officers visit schools on Deworming Day to observe procedure and interview teachers/head teachers regarding deworming. The number of schools observed on Deworming Day treating for STH in Year 3 was 247. The combined population of registered children at the observed schools was 88,820 children. It is estimated that 7,485 children were directly observed being treated for STH. Seven schools treating for both STH and SCH were observed. The total registered population of children in these schools was 3,198 children and 352 children were directly observed by field officers participating in Deworming Day .A quality Deworming Day is regarded to be one where:
      Deworming occurs within 1 week before teacher training
      The school would have the correct materials (including sufficient drugs) in place before commencement
      Children of the appropriate ages are treated (ages 2-14 years)
      Non-enrolled and ECD aged children are prioritized for treatment within the schools
      The correct dosage of drugs is given to all children"
      Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16.

• 36

  This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. For example, see DtWI Delhi 2012 coverage data by school.

• 37

  Grace Hollister, email to GiveWell, March 6, 2016

• 38
  • "A competitive RFP process is now used to identify a professional survey organization to provide independent monitors. There are requirements placed on the experience of these monitors." DtWI Monitoring Improvements 2014, Pg 1.
  • Bihar: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state… Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively." DtWI Bihar 2015 Program report, Pg. 15.
  • Rajasthan: "Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state... Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively." DtWI Rajasthan 2015 program report, Pg 14.
  • Madhya Pradesh: "Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively." DtWI Madhya Pradesh 2015 program report, Pg 15.
  • Delhi: "[Evidence Action] hired an independent research agency, Sigma Research and Consulting Private Limited that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively." DtWI Delhi 2015 program report, Pg 14.
  • Deworm the World was also involved in Chhattisgarh, but as it was engaged relatively late in the process, it did not conduct all of its standard monitoring activities in the state: "Although we place great emphasis on understanding the extent to which the school and health systems are ready to implement deworming, the extent to which deworming processes are being followed, and the extent to which coverage has occurred as planned, in Chhattisgarh we supported only with the coverage validation activity at schools due to time constraints." DtWI Chhattisgarh 2015 coverage validation report, Pgs 2-3.

• 39
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "Evidence Action has a permanent Monitoring, Learning, and Information Systems team. Deworm the World leverages this team for M&E." Grace Hollister, email to GiveWell, June 9, 2016

• 40
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • For example: "...we conducted coverage validation of the LF program in a few states in India and Kenya, all at the request of the government." Grace Hollister, email to GiveWell, June 9, 2016

• 41

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 42

  "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program….In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J- PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there." Grace Hollister conversation June 19th 2013, Pgs 1-2. See the same set of conversation notes for descriptions of how Deworm the World's programs in Delhi and Rajasthan started as well.

• 43

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016. For example, see DtWI Madhya Pradesh 2015 program report:

  • Prevalence surveys: "To develop an appropriate STH treatment strategy, Evidence Action obtained support and approvals from State NHM and Education Department to conduct an STH prevalence and intensity survey among children enrolled in government primary schools. Evidence Action, in partnership with National Institute of Epidemiology - Chennai (NIE), Post Graduate Institute of Medical Education and Research - Chandigarh (PGIMER), and GfK Mode, a market research firm conducted a STH prevalence and intensity survey among children aged 5 to 10, studying in government primary schools in the state." Pg 7.
  • High level planning (e.g. obtaining drugs): "Evidence Action worked with the nodal officer to complete the drug requisition for 2015-16 in August 2014. The drugs for next deworming round (2016) were received at the state in form of two consignments on June 13 and June 18, 2015." Pg 12.
  • Training and distribution cascades:
    • "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." Pg 11.
    • "Evidence Action supported the implementation of the training cascade as depicted below, to orient various levels of functionaries in the key departments… Evidence Action helped contextualize materials from the NDD resource kit according to state requirements, including training presentations, handouts for frontline workers, and frequently asked questions (FAQs)... Evidence Action's regional and district coordinators participated in all 51 district-level trainings under NDD. Additionally the team attended a sample of clock-level trainings to provide support and assess quality (Annexure H2)." Pg 14.
  • Community sensitization: "Evidence Action also briefed media representatives about the program and the event, resulting in widespread coverage. Media kits included key information on the program such as a concept note shared by the state NHM. District-level launch events were held widely across the state. The events were led by local district administration and supported vey Evidence Action district coordinators (Annexure G2)." Pg 13.
  • Troubleshooting with telecallers: "Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." Pg 17.
  • Monitoring: "In order to fulfil this need, Evidence Action worked intensively with the state health, and education, departments to ensure quality planning and implementation of the deworming program." Pg 15.

• 44
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 45
  • "Additionally the national government of India has expressed interest in possibly implementing a deworming program nationwide, rather than waiting for each state to launch separate programs. They have expressed interest in receiving technical assistance from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2.
  • "On National Deworming Day, 12 states in India selected for inclusion in the program’s first phase will be required to administer deworming drugs to schoolchildren, regardless of measures of worm prevalence. National Deworming Day was originally planned for October 2014, but was delayed after Deworm the World staff alerted the government that drug supplies were insufficient. The Indian government has publically announced that National Deworming Day will be in February 2015, but it is possible that there will be further delays.
    The target of the National Deworming Day is children who are 1-19 years old. In the first year, the national government is not advising states to highly prioritize deworming treatment in preschools, though some states were prepared and have already procured deworming medication." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1.
  • "Deworm the World worked closely with the ministry to develop operational and financial guidelines, public awareness campaigns, and monitoring strategies for the program, based on the experiences of state-level deworming programs." Grace Hollister, email to GiveWell, June 9, 2016. The National Deworming Day was announced by India's national Ministry of Health and Family Welfare.

• 46
  • "India held its first National Deworming Day in February. Eleven states participated." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2.
  • "11 dewormed in February; the 12th, Delhi, dewormed in April. This delay was due to conflicts of the February timing with state elections, which require the participation of teachers." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Grace Hollister, email to GiveWell, June 9, 2016

• 47
  • "February 10, 2015 was the first National Deworming Day in India, when over 89 million preschool and school-aged children were dewormed with our support. This led to many states recognising the importance of mass school-based deworming as a safe, cost-effective, and scalable health intervention. Since then, we have had an incredible year, with agreements signed with the states of Uttar Pradesh and Chhattisgarh, and renewed with the states of Bihar and Rajasthan. Our India program now extends technical assistance to seven states: Bihar, Rajasthan, Madhya Pradesh, Uttar Pradesh, Delhi, Chhattisgarh, and – the latest – Telangana." Evidence Action, blog post, December 21, 2015
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Ms. Hollister told us that Deworm the World had also assisted Tripura during the National Deworming Day in 2016. Note that the level of support Deworm the World provides differs from state to state.
  • Grace Hollister, email to GiveWell, June 9, 2016

• 48

  "Deworm the World has phased out its comprehensive technical assistance to Delhi, but the national office will continue to provide minimal support, including attending meetings and doing some policy advocacy work. Deworm the World no longer has dedicated staff for Delhi." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 2

• 49

  Grace Hollister, email to GiveWell, June 9, 2016

• 50
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Note that Deworm the World is the fiscal agent controlling funding for the deworming program in Kenya (which is different from its role in India). This means that Deworm the World has additional responsibility in Kenya for ensuring that funds are used correctly.
  • "Kenya’s National School-Based Deworming Program started in 2009. In 2012, it expanded to a nationwide program aimed at treating all at-risk Kenyan children each year for at least five years." Evidence Action website, Deworm the World Initiative (March 2016)

• 51
  • CIFF uses these prevalence surveys as a way to assess Deworm the World's work. We discuss the methodology and results of these prevalence surveys more below. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, conversation with GiveWell, June 13, 2016
  • Deworm the World has helped with prevalence surveys looking at schistosomes: "...we have funded schisto surveying through END Fund resources, and have helped develop a treatment strategy for schisto as well." Grace Hollister, email to GiveWell, June 9, 2016

• 52

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 53
  • In the first year that Deworm the World worked with Kenya, the MDAs were implemented in 12 different waves. In 2016, there are only be two waves - one in February and one in May. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Currently, implementation of the Kenya program is funded by CIFF and the END fund, while the government provides support via its personnel. Deworm the World is in discussions about how the program will be funded once the current grant ends in 2017. Deworm the World has started conversations with the government about how to transition the program from full support from Deworm the World to operating without that support. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 54

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 55

  For example, the Kenya team has (or will) support the Ethiopia, Nigeria, and Pakistan teams.

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 56

  Although the agreement that governs how SCI and Deworm the World assist with the MDAs is with the Federal Ministry of Health, SCI and Deworm the World also work to some extent with regional governments. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 57

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 58
  • "We are proud and excited to announce that Ethiopia's Federal Ministry of Health is launching a national deworming program targeting school-age children. Ethiopia has one of the highest burdens of neglected tropical diseases in the world, with over 10M children at risk for schistosomiasis and 18M children at risk for soil-transmitted helminths." Evidence Action, blog post, January 8, 2015
  • See Evidence Action, blog post, June 12, 2015 for some description of the roles Deworm the World and SCI played in this decision.
  • The program is led by the Federal Ministry of Health with technical inputs from the Ethiopian Public Health Institute. Grace Hollister, email to GiveWell, June 9, 2016

• 59

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 60

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 61
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Implementation funds are provided to the Federal Ministry of Health, and funds for technical assistance provided to SCI, of which a portion funds Deworm the World's engagement. Grace Hollister, email to GiveWell, June 9, 2016

• 62
  • We do not believe that we fully understand the distinction between Deworm the World's role and SCI's role in Ethiopia, given that SCI also appears to help with non-scientifically technical activities. For example: "SCI/Evidence Action assessment of quality of working group meetings, as identified by criteria: attendence and active participation in meetings from all members, relevant agendas developed for meetings, quality and amount of high-impact decision-making undertaken during meeting, demonstrated ability of working group to troubleshoot challenges to the program as they arise." Deworm the World, Ethiopia Workplan Sheet: "TA Programme KPIs"
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016. SCI works with the Expanded Programme on Immunization (EPI) to conduct the prevalence surveys.
  • Additionally, Deworm the World's Kenya team supports Ethiopia by leading training sessions and developing monitoring tools for Ethiopia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Note that Deworm the World's staff structure in Ethiopia is significantly different from in Kenya or India: in Ethiopia, it only has one staff member, who works closely with the SCI team. Grace Hollister, email to GiveWell, June 9, 2016

• 63

  Grace Hollister, email to GiveWell, June 9, 2016

• 64

  In the case of Pakistan, it was important to find a competent organization that Deworm the World could partner with, for two reasons: first, when entering a new country, it is easiest if Deworm the World can use its partner's financial systems and permissions to move funds to the deworming program (as opposed to Deworm the World attempting to set up an independent financial entity in the country). Second, in Pakistan it is unclear how much capacity the government will have to offer to assist with deworming, so it may be important for partner organizations to provide that capacity. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 65

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 66

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 67

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 68
  • As of March 2016, Deworm the World was still in the process of obtaining government approval for the prevalence surveys, but hoped to start them in April and have results by June. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Deworm the World does not plan to test for prevalence in areas where there are security issues, it is logistically difficult to test for prevalence, or the agro climatic region wouldn't support worms (e.g., the region is mountainous). Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 69

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 70

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 71
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Diagnostic kits were donated by the WHO. Grace Hollister, email to GiveWell, June 9, 2016

• 72

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 73
  • "In Nigeria, programs like deworming are typically implemented at the state level, with various donors and partner organizations supporting different states and targeting different NTDs. USAID and the Department for International Development are major donors. Partner organizations include RTI and Sightsavers, which often implement large-scale integrated NTD programs targeting several diseases." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg. 4
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016. Because Nigeria is a country in which NTDs, including STH, are highly endemic, there are many NGOs already working there. However, because there is a heavy focus on integrated programs that address multiple NTDs at once, the states in Nigeria that are not as heavily afflicted by multiple NTDs are less likely to have an NGO partner.

• 74

  As of March 2016, Deworm the World had hired two staff members to work in Cross River. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 75
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "Deworm the World does not plan to open its own office in Nigeria, but to work through a partner with an on-the-ground presence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg. 4

• 76
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 77

  Grace Hollister, email to GiveWell, June 9, 2016

• 78

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 79

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 80

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 81

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 82

  For example, as of March 2016 Deworm the World had hired a consultant to work in Abuja to build federal relationships and start working on business development activities. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 83
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "Thrive Networks has extensive experience in Vietnam in the water and sanitation, health and education sectors." Grace Hollister, email to GiveWell, June 9, 2016

• 84
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 85

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 86

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Note that in the RCT evaluating the program, the control group will be dewormed along with the treatment group.

• 87

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 88
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 NIMPE has conducted prevalence surveys in Vietnam before, but they were not has high-quality as the surveys that Deworm the World and NIMPE will conduct together.
  • "These surveys are taking place in areas of the country that are not currently treated for STH, and where albendazole administration for lymphatic filariasis ceased several years ago. The intent of the surveys is to determine whether there are areas of the country where STH treatment is required, and will contribute to developing an evidence-based treatment strategy for the country." Grace Hollister, email to GiveWell, June 9, 2016

• 89

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 90

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 91

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 92

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 93

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 94

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 95

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 See the footnotes in our section on India above.

• 96

  There is some chance that Deworm the World could partner with RTI again; RTI currently works on a lymphatic filariasis program in Indonesia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 97

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 98

  See GiveWell analysis of Deworm the World 2014 Financial summary for 2014 financial data. Note that these figures include central costs, so this reflects Deworm the World’s full budget.

• 99

  See Deworm the World, 2015 expense summary - by funder. Of $5,481,207 spent in 2015, it is our guess that only $814,504 (~15%) was unrestricted (we have counted Good Ventures 2014, Retail Fund Raising, Anonymous, and Marshall as unrestricted).

• 100
  • "Kenya’s National School-Based Deworming Program started in 2009. In 2012, it expanded to a nationwide program aimed at treating all at-risk Kenyan children each year for at least five years. Implemented by the Ministry of Education, Science, and Technology and the Ministry of Health. With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively. School year 2014/15 results will be released in October. Technical and operational assistance to the program will continue through 2017." Evidence Action website, Deworm the World Initiative (October 2015)
  • "Deworm the World is planning a strategy to sustain and institutionalize the program after its current grants expire in June 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 4

• 101
  • "DtWI’s funding gap for 2015 will be small because DtWI recently received a large grant from CIFF. The CIFF grant will cover DtWI’s expenses in India, even with the large growth that DtWI is planning." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 2.
  • Grace Hollister, email to GiveWell, June 9, 2016

• 102

  Some of the differences between spending patterns in Kenya and India are due to the facts that a) Deworm the World pays for direct implementation costs in Kenya that it doesn't pay in India, b) Deworm the World does not pay for prevalence surveys in Kenya, and c) in Kenya, teachers are paid significantly higher allowances (per diems) for participating in the program:

  • "It is important to note that there are fundamental differences between the Kenya and India programs, as funding to Evidence Acton supports direct implementation costs in Kenya (where the government does not pay those costs, but rather provides in-kind support), and India, where the government pays the lion’s share of implementation costs." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 103

  Grace Hollister, email to GiveWell, June 9, 2016

• 104
  • Deworm the World, 2015 expense summary Note that Deworm the World sent us several different breakdowns of its 2015 expenses. The versions do not all match, but we suspect this is because a) some time passed between creating and sending us the different versions and b) Deworm the World was still in the process of finalizing its financials when the expense summaries were sent.
  • Deworm the World, 2015 expense summary - by funder shows Deworm the World's costs broken down by funder.
  • See our 2014 review for a summary of Deworm the World's costs pre-2014 and our 2015 review for a summary of Deworm the World's costs in 2014.

• 105
  • "Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success.

    Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action." Evidence Action launch announcement 2013

  • "Evidence Action Beta investigates what interventions might be suitable for massive scale up – finding the next thing that works." Evidence Action website, Evidence Action Beta (October 2015)

• 106
  • India: "…absent from most of the country, [schistosomiasis] risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1.
  • "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3.
  • See section on worm prevalence and intensity in India and Kenya below.

• 107
  • DtWI Bihar 2011 prevalence survey report
  • DtWI Bihar 2015 Prevalence Survey report
  • DtWI Rajasthan 2012 prevalence survey report
  • DtWI Rajasthan 2013 prevalence survey report (Deworm the World re-surveyed Rajasthan because there were methodological weaknesses in the first prevalence survey)
  • DtWI Delhi 2012 prevalence survey report
  • Deworm the World notes: "Follow up surveys should take place after multiple rounds of treatment (WHO guidance is between three and six years of treatment), hence why only one follow-up has taken place to date." Grace Hollister, email to GiveWell, June 9, 2016

• 108
  • Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013
  • "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected." DtWI Bihar 2011 prevalence survey report, Pg 3.
  • "Predicted prevalence is modelled using survey data from 2010-2011, point-level environmental data (annual temperature and rainfall) and a district level proxy for socio-economic status (literacy)." DtWI Bihar 2011 prevalence survey report, Pg 5.

• 109
  • "The prevalence of STH (including hookworm, A. lumbricoides, and T. trichiura) across these states in Bihar was 67.5% with district prevalence ranging from 49.0% to 79.6%." DtWI Bihar 2011 prevalence survey report, Pg 4.
  • 67% = 782/1159. From "Table Three: Cumulative prevalence of each species by district": "Total - Number Students" = "1159"; "Total - % Any infection" = "(782) 67.5%". DtWI Bihar 2011 prevalence survey report, Pg 4.
  • Selection of districts and schools: "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected from the state schools database." Pg 9.
  • Selection of students: "Within each school 65 children aged 6 and above representing both sexes equally from class 1 to 6 were randomly selected, class by class, using random number tables." Pg 9.
  • Technique: "Screening of infection for STH was based on a double Kato-Katz smear of 41.7 mg prepared from fresh stool samples." Pg 11.
  • Participation rate: "From a total of 1,281 school children registered in the survey and provided with pots, 1,159 returned samples." (90.5%) Pg 11.
  • Note on two stages of first prevalence survey based on DtWI Bihar 2011 prevalence survey report and Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013
    • Based on the timing of the decision to deworm in Bihar, there was only time to collect prevalence surveys from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet DtWI's typical standard of quality.
    • Before the DtWI-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report, Pg 9.
    • Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness. (Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013)

• 110

  "The prevalence of STH (including hookworm, A. lumbricoides, and T. trichiura) across these states in Bihar was 67.5% with district prevalence ranging from 49.0% to 79.6%." DtWI Bihar 2011 prevalence survey report, Pg 4.

• 111
  • Albendazole is used for treatment of both lymphatic filariasis (where it is used in combination with either DEC [Diethylcarbamazine] (as in India) or ivermectin) and of STH. Therefore, a round of treatment for lymphatic filariasis also treats STH. In this review, the term "deworming round" includes any MDA of albendazole.
  • "A Memorandum of Understanding (MOU) was signed on March 5, 2010 among State Health Society Bihar, Bihar Education Project Council, and Deworm the World Initiative to implement the school-based deworming program in the state for treatment of STH. Based on Prevalence Survey findings, which suggested treatment recommendation of WHO, the Government of Bihar decided to implement biannual state-wide deworming beginning in 2011. Since then, Evidence Action has extended technical assistance to an annual round of deworming for all school-age children through a school-based model. The second round of deworming treatment was provided through the National Filaria Control Program (NFCP), which conducts annual mass drug administration of albendazole to the 2 years and above population at the community. In 2011, a total of 16.7 million children were dewormed at schools, earning the distinction of being the world's largest school-based deworming program. In 2012, 16.33 million children were dewormed in Round 2. 17.47 million, including 16.2 school-age children, were dewormed as part of Round 3 in 2014." DtWI Bihar 2015 Program report, Pg 6.
  • "Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012. Four deworming rounds or distribution of albendazole during LF-MDA were missed during the period" DtWI Bihar 2015 Prevalence Survey report, Pg 5.

• 112
  • "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5.
  • "The ‘2014’ [LF-MDA] occurred in February/March 2015. There was no LF MDA in the calendar year of 2014. But, since the LF MDA occurred prior to the closing of the 2014 - 2015 financial year (March 31, 2015). This is apparently common practice and acceptable to the government of India, and that is what is reported on the website. There was no LF MDA in the calendar year of 2014, as there was no DEC [Diethylcarbamazine] in Bihar until February 2015. Therefore a round did not occur prior to the second survey." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015

• 113
  • Prevalence figures
    "On the basis of the collected and analyzed data, the overall weighted prevalence of any STH in Bihar was calculated as 35%. Prevalence in different agro-climatic zones ranged from 20% to 50%" DtWI Bihar 2015 Prevalence Survey report, Pg 2.
  • Representative survey design
    "Per the 2011 census, there were 18,884,945 children aged 5-10 years in Bihar. Assuming one sentinel school is required for 300,000 targeted children, we needed at least 63 schools (rounded to 65) to monitor the performance of the deworming program. Collecting samples from 50 children per school, the minimum sample size required for estimation of prevalence using the sentinel site method was 3,250.
    Bihar is divided into three agro-climatic zones (Figure 2) – the North West Alluvial Plains (Zone 1) consisting of 12 districts, the North East Alluvial Plains (Zone 2) consisting of 9 districts and South Bihar Alluvial Plains consisting of 17 districts (Zone 3). The three zones respectively accounted for 42%, 22% and 36% of 5-10 years population in the state. NIE randomly selected 65 schools from the three zones, proportionate to the percentage of 5-10 year population in each of the zones. Thus, the study needed 28, 14 and 23 schools from zones 1, 2 and 3 respectively.
    NIE followed a 2-stage sampling procedure for selecting sentinel schools. In the first stage, NIE randomly selected 14 districts from Bihar (six from zone 1, three from zone 2 and six from zone 3) to meet logistical (teams could not spend too much time traveling and setting up temporary field laboratories, because it reduced the number of samples the teams could analyze), geographic dispersion (the survey needed to be geographically dispersed to ensure the best estimates of prevalence), and time constraints (there were only 20 days available for the survey). In the second stage, NIE line-listed all the primary schools (with the total strength of ≥ 60 children) of the districts selected from each zone. They then selected the required number of schools for each zone randomly from the list of schools in the selected districts. To select the required number of children, we assigned a random number (between one and five) to classes of the selected school. The survey was initiated from the class corresponding to the random number assigned for the school and field teams enumerated the children present in the class starting from roll number one on the attendance register. If the number of children in the selected class was &lt 50, children from the next class were selected. This procedure was followed until 50 children from each school were selected." DtWI Bihar 2015 Prevalence Survey report, Page 8.

• 114DtWI Bihar 2015 Prevalence Survey report
• 115

  Deworm the World has supported three MDAs in Bihar over the last four years: "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5.

• 116
  • The National Vector Borne Disease Control Programme LF treatment coverage 2015
  • The treatment for LF in India is annual administration of DEC [Diethylcarbamazine] or DEC and Albendazole:
    "Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
    Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except:
    - children below 2 years
    - pregnant women
    - seriously ill persons
    (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012.
  • "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5.

• 117
  • "Revised Strategy
    Annual Mass Drug Administration with single dose of DEC [Diethylcarbamazine] was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
    Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except: - children below 2 years - pregnant women - seriously ill persons (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012
  • The National Vector Borne Disease Control Programme LF treatment coverage 2015

• 118
  • "The ‘2014’ MDA occurred in February/March 2015. There was no LF MDA in the calendar year of 2014. But, since the LF MDA occurred prior to the closing of the 2014 - 2015 financial year (March 31, 2015). This is apparently common practice and acceptable to the government of India, and that is what is reported on the website. There was no LF MDA in the calendar year of 2014, as there was no DEC [Diethylcarbamazine] in Bihar until February 2015. Therefore a round did not occur prior to the second survey." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • "Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012. Four deworming rounds or distribution of albendazole during LF-MDA were missed during the period" DtWI Bihar 2015 Prevalence Survey report, Pg 5.
  • "The Bihar program was originally designed so that one round of albendazole administration would come from community-based lymphatic filariasis (LF) treatment (LF is endemic throughout the state) and a second round would come through the school-based deworming program. For a variety of reasons, LF treatment has not been consistently implemented. It is possible that more consistent albendazole administration as part of LF treatment would have resulted in a more significant drop in STH prevalence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 2
  • "Kenya and India are the countries where Deworm the World operates two of its biggest programs and where Ms. Hollister has the greatest familiarity with the state of LF treatment. In both countries, LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015, Pg 1

• 119
  • 39% = 16801/12090 - 1
  • 2010-2011 "PER CAPITA NET STATE DOMESTIC PRODUCT AT FACTOR COST - Constant price... Base : 2004-05)": 12090 billion Rupee
  • 2014-2015 "PER CAPITA NET STATE DOMESTIC PRODUCT AT FACTOR COST - Constant price... Base : 2004-05)": 16801 billion Rupee
  • Reserve Bank of India, GDP per capita, Table 10, September 16, 2015

• 120
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "The M&E includes a series of pre- and post-intervention, repeat cross-sectional surveys in a representative, stratified, two-stage sample of schools across Kenya. District stratification was based on both geography and anticipated infection prevalence. The programme contains three tiers of monitoring: i) a national baseline survey including 200 schools in 20 districts, which aims to establish an accurate national measurement of infection levels; ii) surveys conducted pre and post intervention (pre-post surveys), which monitor 60 of the 200 schools before and immediately after the deworming activity to evaluate reductions in infections that can be directly attributed to programme implementation; and iii) high frequency surveys in 10 schools, distinct from the 60 pre-post schools, at four time points in a single year, before, during, and after treatment (Figure 1).
    Two hundred schools were examined at baseline and will be re-examined in year 3 and 5 in order to monitor long-term changes in worm infection at a national level both in terms of prevalence and intensity of infection. This sample size was chosen in order to be able to detect a five-percentage-point change in prevalence across years, assuming power β = 0.80 and test size α = 0.05, and considering the anticipated variance in prevalence. Sixty schools (a subset of the 200) will be surveyed every year for 5 years, before each treatment round to evaluate programme impact and 3–5 weeks post-treatment to evaluate treatment efficacy [2]. The same schools will be surveyed each year in the 60 pre-post survey schools, whereas in the remaining 130 schools, a different random sample of schools will be undertaken each year. In the 10 high frequency schools, a cohort of children will be followed-up longitudinally and assessed for haemoglobin concentration in addition to parasitological outcomes." Mwandawiro et al. 2013 Pgs 1-2.

• 121
  • Mwandawiro et al. 2013
  • KEMRI prevalence report - Year 2
  • Deworm the World, Kenya 2014-2015 program report, Pgs 13-14

• 122

  See Mwandawiro et al. 2013. Because this is a published paper that describes the methodologies of the prevalence surveys (sampling, timing, outcomes measured, etc.) over the course of Kenya's five-year deworming program, we consider this to effectively be a pre-registration.

• 123

  Deworm the World does not have full details about KEMRI's prevalence surveys easily available: to avoid the possibility of introducing bias, KEMRI has kept its methodologies private (so that, e.g., Deworm the World would not learn which schools KEMRI is evaluating). We intend to pursue additional details in the second half of 2016. Grace Hollister, conversation with GiveWell, June 13, 2016

• 124
  • "Based on available data and predictive maps [9,10], STH was assumed to be endemic in 66 districts. From these districts, grouped into strata, 20 districts were randomly selected for M&E in the first sampling stage, with number of districts per province proportional to population: six districts from Western Province, three from the Rift Valley, five from the Coast, and six from Nyanza (Table 1)." Mwandawiro et al. 2013 Pg 3.
  • All 66 districts in which STH was assumed to be endemic are supported by Deworm the World: "From the year 2012, the ministries of health and of education of Kenya plan to deworm all school –age children who live in 66 districts identified as having a high prevalence of soil-transmitted helminth (STH) infection and schistosomiasis in four provinces." KEMRI prevalence report - Year 2 Pg 2
  • Deworm the World told us that the KEMRI surveys are only occurring in districts funded by CIFF. CIFF funds most of the districts in the national deworming program, but the END Fund also funds several districts. Grace Hollister, conversation with GiveWell, June 13, 2016
  • Kenya's national deworming program supports more than 66 districts: Deworm the World told us that "districts" were recently re-divided into "sub-counties" (with each sub-county being approximately the same size as districts were previously). Grace Hollister, conversation with GiveWell, June 13, 2016 In Year 3 of the Kenya program, Deworm the World supported MDAs in 111 sub-counties. Deworm the World, Kenya 2014-2015 program report, Pg 10.
  • Deworm the World told us that it did not have enough detail about the KEMRI prevalence surveys to know whether or not the 66 districts had been selected randomly. Grace Hollister, conversation with GiveWell, June 13, 2016
  • If the 66 surveys were not selected randomly, this could cause the prevalence surveys to be somewhat biased. For example:
    • If only districts with high prevalence were chosen, it could be the case that the schools surveyed are the schools most incentivized to implement deworming effectively.
    • Or, if only districts with high prevalence were chosen, perhaps children in such districts would be more willing to take deworming pills (it is plausible that children in higher prevalence districts might feel noticeably better after deworming, and therefore be more cooperative when taking the deworming pills each year).
    • If the 66 districts were chosen based on ease of access to schools (e.g., to make surveying easier), the surveys might only be evaluating the locations in which deworming is easiest to implement.

• 125
  • 200 schools were randomly selected from the 20 districts for the baseline survey in early 2012, and 60 of those schools are re-surveyed before and after their MDAs each year.
    • See Figure 1 of Mwandawiro et al. 2013, Pg 3. Note the post-MDA survey is supposed to occur 3-5 weeks after the treatment. We do not know if the surveys occurred on the timeframes they were intended to, and we do not know how much time passes between the pre-MDA survey and the MDA itself.
    • "At the second sampling stage, primary schools were randomly selected from within the chosen 20 districts." Mwandawiro et al. 2013 Pg 3.
    • "The article describes the M&E design of the Kenya national SBD programme and presents results from the baseline survey conducted in early 2012." Mwandawiro et al. 2013 Pg 1.
  • Of the remaining 140 schools, 10 will be surveyed both pre- and post- MDA in Years 1, 3, and 5, while 130 are surveyed only pre-MDA in Years 1, 3, and 5.
    • See Figure 1 of Mwandawiro et al. 2013, Pg 3.
    • Note that the 130 schools surveyed pre-MDA in Years 1, 3, and 5 are not the same schools each time; they are re-selected randomly: "The same schools will be surveyed each year in the 60 pre-post survey schools, whereas in the remaining 130 schools, a different random sample of schools will be undertaken each year." Mwandawiro et al. 2013 Pgs 2-3.
  • In the 60 schools surveyed each year, the sampled children are randomly selected each year, meaning the surveys do not follow precisely the same population year-to-year.
    • "In each school, 18 children (9 girls and 9 boys) were sampled randomly from each of six classes - one Early Childhood Development (ECD) class and classes 2–6 - using computer generated random number tables, for a total of approximately 108 per school. The sampling within these specified classes aimed to target children aged 5–16 years. " Mwandawiro et al. 2013 Pg 3. Also see Figure 1 of Mwandawiro et al. 2013, Pg 3. We believe the populations will be very similar, so we are unconcerned that different children are surveyed each year.
  • The surveys measure the prevalence and intensity of both soil-transmitted helminths and schistosomes.
    • "Stool samples were obtained for each child and two slides prepared and examined for the presence and intensity of STH species and S. mansoni using the Kato Katz method, with the concentration of eggs expressed as eggs per gram (epg) of faeces. Urine samples were obtained only from children in Coast Province (where Schistosoma haematobium is widespread) and investigated for presence and intensity of S. haematobium using the urine filtration method, with the concentration of S. haematobium eggs estimated in eggs per 10 ml urine. Egg counts were performed only up to 24,000 epg and 1,000 eggs/ 10 ml urine, respectively. Infection intensities above these values were, therefore, not further quantified." Mwandawiro et al. 2013, Pg 3.
  • Additionally, in the 10 schools surveyed pre- and post-MDA every odd year, anaemia and educational outcomes are measured. We have not seen any results from the anaemia and educational outcome tests, but we only have results from Year 1 and Year 2 and we are not sure if these outcomes were measured in the first year. From Mwandawiro et al. 2013:
    • "In the 10 high frequency schools, a cohort of children will be followed-up longitudinally and assessed for haemoglobin concentration in addition to parasitological outcomes." Pg 3.
    • See Figure 1, Pg 3.
    • "In the 10 “high frequency” schools, finger-prick blood samples were obtained and analysed using a HemoCue photometer (HemoCue, Angelhom, Sweden) to estimate haemoglobin concentration." Pgs 3-4.

• 126
  • The data in the table for soil-transmitted helminths is from KEMRI prevalence report - Year 2, Pg 4 and Deworm the World, Kenya 2014-2015 program report, Pgs 12-14.
  • The data for schistosomiasis is from Table 4 of Mwandawiro et al. 2013, Pg 6 and Box 5 of KEMRI prevalence report - Year 2, Pg 15.
  • Note that the two sources we've seen describing the results from the prevalence surveys do not exactly match. In Mwandawiro et al. 2013, the prevalence at baseline was 15.6% for hookworm, 18.0% for Ascaris, and 6.6% for Trichuris, whereas the KEMRI prevalence report - Year 2 shows the baseline prevalence being 16.9% for hookworm, 19.2% for Ascaris, and 5.4% for Trichuris. (Pg 5 and Pg 4, respectively) We are not sure where this discrepancy comes from, but the numbers are close enough that we don't find it too concerning.

• 127In years 1, 3, and 5, 10 schools (randomly selected from the 200 schools at baseline but distinct from the 60 schools that will be followed each year) will be surveyed both pre-MDA and post-MDA. These schools will be surveyed intensively, with three post-MDA surveys measuring for worm prevalence and additional variables, such as anaemia and educational outcomes. The same 10 schools will be surveyed for each of Year 1, 3, and 5, and the same students from each of the 10 schools will be surveyed. Mwandawiro et al. 2013, Pgs 2-3.
• 128
  • The Year 1 baseline data in the table is from Table 4, Mwandawiro et al. 2013, Pg 6.
  • The Year 1 post-MDA data is from Table 11 of KEMRI prevalence report - Year 2, Pg 20. The report only gave prevalences by county, so we took the median of all counties and the median of counties within the Coast Province (see this Wikipedia article) for a rough comparison.
  • The data pre- and post- the Year 2 MDA is from KEMRI prevalence report - Year 2, Pgs 15 and 18.
  • The data for Year 3 is from Deworm the World, Kenya 2014-2015 program report, Pgs 12-14.
  • Note that the two sources we've seen describing the results from the prevalence surveys do not exactly match. In Mwandawiro et al. 2013, the baseline Year 1 prevalence for S. Mansoni at baseline was 0% in the Coast Province and 2.1% in all provinces, while the prevalence for S. haematobium was 14.8% in the Coast Province (Table 4, Pg 6). However, in KEMRI prevalence report - Year 2, the baseline Year 1 prevalence is reported as 0.1% for S. mansoni in the Coast Province and 1.8% in all provinces (Box 5, Pg 15), while the prevalence for S. haematobium is reported as 18.0% (Pg 18). We are not sure where these discrepancies comes from.

• 129
  • S. haematobium was only measured in the Coast Province: "Urine samples were obtained only from children in Coast Province (where Schistosoma haematobium is widespread) and investigated for pre- sence and intensity of S. haematobium using the urine filtration method, with the concentration of S. haema- tobium eggs estimated in eggs per 10 ml urine." Mwandawiro et al. 2013, Pg 3.
  • "Urine samples were examined for S. haematobium infections in 9 schools in Coast Province in Kilifi and Kwale Counties." KEMRI prevalence report - Year 2, Pg 18.

• 130Table 4, Mwandawiro et al. 2013, Pg 6.
• 131

  We were told that in the first year of Kenya's deworming program, the MDAs occurred in 12 different waves, as opposed to all on the same day. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 If the pre-MDA surveys are conducted together (e.g., if all occur in February), then post-MDA surveys will vary as to how long after the pre-MDA survey they are conducted, since they occur 3-5 weeks after treatment. If pre-MDA surveys are always conducted ~X days before the treatment, then the delay between the pre-MDA surveys and post-MDA surveys will be more consistent.

• 132

  Or, if some pre-MDA and post-MDA surveys are separated by a long time-lag (e.g., 6 months or more), it's possible that the effect of deworming would appear to be smaller than it actually is, since some children will likely become reinfected with worms during the lag.

• 133

  See Figure 1 of Mwandawiro et al. 2013, Pg 3. This might be less of an issue in Year 3 and Year 5, when KEMRI plans to survey an additional 130 schools randomly selected from the same 20 districts.

• 134
  • "This survey provides an up-to-date assessment of STH infections in the regions of Kenya targeted for school-based deworming, and provides a rigorous basis for evaluating programme impact." Mwandawiro et al. 2013,Pg 8.
  • Deworm the World told us that it did not believe the surveyors provided treatments after testing. However, Deworm the World also told us that it only had limited information on the methodology of the KEMRI prevalence surveys. Grace Hollister, conversation with GiveWell, June 13, 2016
    
  • 135

    That is, unlike with the prevalence surveys conducted in Bihar, we doubt that are large-scale changes that could affect worm prevalence so quickly, such as improved sanitation infrastructure or general development, which would probably take much longer to show such large effects. Additionally, the fact that the worm prevalence spikes again before the Year 2 MDA makes it less likely that large-scale changes are the cause of these changes (since we think it is unlikely that large-scale changes would cause such impermanent, quick shifts).

  • 136

    This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.

  • 137

    This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.

  • 138

    In some deworming programs assisted by Deworm the World, monitors also assess the quality of community sensitization efforts (via interviews with community members to see if they are aware of deworming) and of trainings (via visiting the trainings to observe what it taught and administering before and after tests to participants in the training). We have not prioritized reviewing the data from these activities, although we think it could also provide insight as to the quality of Deworm the World's program.

  • 139

    Note that it is not actually dangerous to give the deworming pills to sick children; Deworm the World simply prefers not to deworm sick children to avoid causing people to associate the deworming pills with illness.

    • "Deworming pills should not be distributed to sick children...This is not because deworming pills could harm sick children. It is because DtWI wants to avoid people (and potentially the media) blaming the deworming pill for a child’s illness." DtWI 2013 GiveWell site visit

  • 140

    Monitoring procedures vary somewhat among the countries and rounds of deworming, but in most cases the process includes:

    
    The monitors use field observations at a sample of schools and, in some instances, training sessions to gather information on the program in advance of treatment days, during treatment days, and after treatment days. Some details are below and additional detail can be found in the cited documents.

    KENYA
    We do not have as detailed descriptions for Kenya's monitoring procedures as we do for India.

    • Year 3 (2014-2015): "Process monitoring and coverage validation (PMCV) activities were carried prior to, during, and following all deworming dates. PMCV field officers observed events according to a pre-determined sample sizes. The table below shows the Year 3 deworming activities to date. All monitoring or observed events were randomly sampled from a list of planned events." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pgs 4-5.
    • Year 2 (2013-2014): "PMCV field officers observed a proportion of events within the programme according to a pre-determined sample size. The table shows the activities to date." Deworm the World, Kenya Narrative Report - Year 2, Quarter 4, Pg 1.
    • Year 1 (2012-2013): "Process Monitoring and Coverage Validation Data, (PMCV) is utilized to assess the programme quality and to verify coverage data. PMCV information is collected by a group of field monitors who are deployed to observe a sample of the following events:
      1. Regional trainings
      2. Teacher trainings
      3. Pre deworming day preparation
      4. Deworming day
      5. Post deworming day for data confirmation

      This evaluates the success of the training cascade at moving information, materials, funds, and drugs from the national to the school level. The sampling plan was developed at three levels in the cascade for each type of event and is detailed below for each level. Given the nature of the program and roll-out, the sampling strategy is designed to be agile and responsive to program changes and last-minute scheduling. The sample is structured such that the each selection is a subset of the selection before. For example the selection of teacher training events is made from within those regional trainings which were observed….To mitigate the effect of being observed, on the results, there is then a second limited selection outside of this cascade at each level. Both Schistosomiasis and STH only areas are covered by the PMCV data selection. Table 1 provides a summary of the sample planned and executed under PMCV." Deworm the World, Kenya Narrative Report - Year 1, Pgs 2-3.

    INDIA

    • Bihar 2015: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state. The objective of independent monitoring is to determine whether deworming is being implemented according to planned protocols. Two-stage probability sampling was used to select schools for coverage validation on deworming day and mop-up day. First, 125 blocks were selected from all 38 districts by probability proportional to size sampling, followed by random sampling of schools to provide statewide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students.
      During coverage validation an additional 748 randomly sampled schools were surveyed after deworming days to check whether deworming occurred, if reporting protocols were followed, and to validate the coverage reporting. Coverage validation data was gathered through interviews with headmasters and 3 students (in 3 different randomly selected classes in each school), and by checking of all class registers and reporting forms." DtWI Bihar 2015 Program report, Pg. 15.
    • Rajasthan 2015: "The process of monitoring and evaluation in each deworming round are performed in three ways: (1) process monitoring, (2) coverage reporting and (3) coverage validation. In Rajasthan, both process monitoring and coverage validation were carried out in schools, while only coverage validation was done at anganwadis. This is because the four-day long (10-13 February) deworming program in anganwadis made process monitoring a challenge.
      Process Monitoring, Coverage Reporting, and Coverage Validation: Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program. Evidence Action assesses the program preparedness during pre-deworming phase and selected independent monitors observe [sic] the deworming processes on Deworming Day and mop-up day. We conduct process monitoring in two ways: a) telephone monitoring and cross verification and b) physical verification by visiting schools and training venues. Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state. The objective of independent monitoring is to determine whether the program is being implemented according to planned protocols. Two-stage probability sampling was used to select schools for independent monitoring on Deworming Day and mop-up day. First 125 blocks were selected from all 33 districts by probability proportional to size (PPS) sampling, followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively. These 125 monitors visited 125 schools on Deworming Day and an additional 125 on mop-up day to check for adequate drug supplies and awareness materials, to confirm whether teachers had received training, and to assess knowledge of adverse event management protocols and reporting processes. Monitors gathered data through observation of deworming and through interviewing headmasters, teachers, and randomly selected students." DtWI Rajasthan 2015 program report, Pg 14.
    • Madhya Pradesh 2015: "In Madhya Pradesh, as preschool-age deworming was implemented through the BSM, monitoring efforts focused on the school-age program through the NDD. In the future, it will be important to expand monitoring to anganwadis to better understand program preparedness and performance…
      Evidence Action assesses the program preparedness during pre-deworming phase and selected independent monitors observe the deworming processes on Deworming Day and mop-up day. We conduct process monitoring in two ways: a) telephone monitoring and cross verification, and b) physical verification by visiting schools and training venues.
      A two-stage probability sampling process was followed to select schools for NDD, mop-up day and coverage validation. Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively. The monitors visited 125 randomly selected schools on NDD, and an additional 125 schools on mop-up day (February 14) to check for adequacy of drug supplies and awareness materials, and assess whether teachers had received training, and had knowledge of adverse event management protocols and reporting processes. Monitors gathered data by observing deworming and by interviewing headmasters, teachers, and randomly selected students. An additional 750 randomly sampled schools were surveyed from February 18-26 to check whether deworming occurred and reporting protocols were followed, and to validate the coverage reporting." DtWI Madhya Pradesh 2015 program report, Pg 15.
    • Chhattisgarh 2015 (coverage validation only): "The school database for random sampling in the 11 districts was obtained from the Ministry of Drinking Water and Sanitation, Government of India website. We visited 10 randomly selected schools in each block from the 55 blocks in these 11 districts. Therefore, they visited a total of 550 randomly sampled schools for coverage validation activities. In addition to the headmster's interview and verification of the deworming related documents, three randomly selected children from three different randomly selected classes were interviewed in each school. In addition, we also visited any anganwadis attached to the sampled schools. We could not achieve the targeted sample of 550 schools as two districts, Sukma and Bijapur, could not be covered given the high risk due to insurgency in these areas." DtWI Chhattisgarh 2015 coverage validation report, Pg. 3.
    • Delhi 2015: "Evidence Action assesses the program preparedness during the pre-deworming phase and selected independent monitors observe the processes on deworming and mop up days. We conduct process monitoring in two ways: a) telephone monitoring and cross verification, and b) physical verification by visiting schools and training venues.
      The method of stratified random sampling using proportional allocation approach was followed for selection of schools and anganwadis for deworming day, mop-up day, and coverage validation monitoring to provide state-wide estimates of indicators. We hired an independent research agency, Sigma Research and Consulting Private Limited that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively. These monitors were to visit total of 400 randomly selected schools and 400 randomly selected anganwadis; 80 schools and 80 anganwadis on deworming day and mop up day each (April 16 & April 20); and 240 schools and 240 anganwadis during coverage validation (April 23-27, 2015). The actual number of schools and anganwadis visited on each day is given in annexure (Table SA-1). The monitors visited the selected schools and anganwadis on deworming day, on mop up day to check for adequacy of drug supplies and awareness materials, whether teachers/anganwadi workers had received training, and knowledge of adverse event management protocols and reporting processes. Monitors gathered data through observation during deworming and interviews with headmasters, teachers, and anganwadi workers as well as of randomly selected students from schools. Additional randomly sampled schools and anganwadis were surveyed from April 23-27 to check whether deworming occurred, reporting protocols were followed, and to validate the coverage reporting." DtWI Delhi 2015 program report, Pg 14.
    • Bihar 2014: "In its attempt to evaluate the adherence of each process to guideline and time plan, Deworm the World supported a two-way monitoring strategy; a) Telephone monitoring and cross verification, and b) Physical verification by visiting the sites, schools, and training venues." DtWI Bihar 2014 program report, Pg 22. Pgs 23-30 describe activities over the various days.
    • Delhi 2013: "In order to carry out robust M&E activities, Deworm the World retained short-term resources comprised of district coordinators and telecaller to a) assess the readiness of the system to implement deworming, and b) to follow-up with districts and nodal officers (for anganwadis) for the return of data post-deworming. In addition, independent monitors were hired and trained by Deworm the World to assess preparedness, visit schools and anganwadis on deworming and mop-up days and carry out coverage validation post mop-up day. The Deworm the World team and officials from SHS and DHFW also made several field visits to monitor the entire deworming program before, during, and after deworming day." DtWI Delhi 2013 program report, Pg 12.
    • Rajasthan 2013: "In order to carry out robust M&E activities, Deworm the World hired district coordinators, telecallers and independent monitors as detailed below. In addition, staff and officials from DMHFW, RCEE and DWCD also made several field visits to monitor the entire deworming program before, during, and after deworming day." DtWI Rajasthan 2013 program report, Pg 11.
    • Bihar 2012:
      • Process: "Independent monitors visited a randomly selected sample of schools over four days – Deworming Day, Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 1216 schools (1.75% of 69,299 schools in Bihar) targeted for monitoring. From each of the 38 districts in Bihar, 2 blocks were selected by simple random sampling, for a total of 76 blocks. 2 clusters were randomly selected from each of the 76 blocks, excluding clusters with fewer than 8 schools. In each of these 152 clusters, 8 schools were randomly selected for a total of 1216 schools. Each monitor was assigned a block comprising 16 schools to be monitored. From this list of 16 schools, the monitor could visit any 4 schools on Deworming Day, another 4 on Mop-Up Day, and 8 more schools over the two Coverage Validation days. Out of the total sample of 1216 schools, monitors were able to survey 1196 schools comprising 302 schools on Deworming Day, 296 on Mop-Up Day, and 598 on Coverage Validation days." DtWI Bihar 2012 monitoring report, Pg 1.
      • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Bihar 2012 monitoring report).
    • Delhi 2012:
      • Process: "Monitoring visits were conducted by senior government officials. Random site visits by independent auditors occurred at a subset of training sessions and participating institutions. Coverage validation engaged independent auditors who conducted random site visits at a representative sample of schools and anganwadis to validate coverage statistics. This source of information was carefully compared with programme reports collated from each school and anganwadi to arrive at an accurate assessment of programme coverage." DtWI Delhi 2012 program report, Pgs 7-8.
      • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Delhi 2012 program report, Pgs 17-22).
    • Rajasthan 2012:
      • Process: "In order to evaluate the efficacy of the deworming protocol and process, independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 990 schools (1.1% of 90,488 schools in Rajasthan) targeted for monitoring. From each of the 33 districts in Rajasthan, 2 blocks were selected by simple random sampling. In each of these 66 blocks, 15 schools were randomly selected for a total of 990 schools. One monitor was assigned to each of these blocks. From the list of 15 schools in a block, the monitor could visit any 3 schools before Deworming Day for Preparation Monitoring, another 2 schools on Deworming Day, 2 more schools on Mop-Up Day, and 4 more schools over the two Coverage Validation days. The remaining 4 schools served as a buffer in case a particular school could not be visited. Hence, the actual sample size was 726 schools (0.8% of all schools)." DtWI Rajasthan 2012 monitoring report, Pg 1.
      • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Rajasthan 2012 monitoring report).
    • Bihar 2011:
      • Process: "Moreover, independent monitors were mobilized in each block (560 total) to visit 5% of all government schools on deworming and mop-up days to ensure that an adequate quantity of drugs were available, trained teachers were administrating drugs according to protocol, and community sensitization initiatives were successful in mobilizing non-enrolled children to attend. One auditor per district was in charge of supervising monitors to ensure quality monitoring of all schools." DtWI Bihar 2011 program report, Pg 9.
      • What we’ve seen: Monitoring forms used for data collection and full monitoring data from deworming day and mop-up day. We have not seen a summary of this data and have not analyzed it ourselves. (DtWI Bihar 2011 program report)

• 141

  For example: "Our tele-callers place phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17.

• 142

  We present results that we judged were fairly easy to interpret.

• 143

  From schools visited by monitors on Deworming Day.

• 144Aggregate level analysis (N = 995 schools) DtWI Bihar 2015 independent monitoring tables, Pg 10
• 145Aggregate level analysis (N = 625) DtWI Rajasthan 2015 independent monitoring tables, Pg 10
• 146Aggregate level analysis (N = 400 schools), DtWI Delhi 2015 independent monitoring tables, Pg 26
• 147Aggregate level analysis (N = 1000 schools) DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 12
• 148Coverage validation survey (N = 490 schools), DtWI Chhattisgarh 2015 coverage validation tables, Pg 1
• 149DtWI Madhya Pradesh 2015 coverage validation form, Pg 3
• 150Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 3
• 151Deworming Day and Mop-up Day (N = 250) DtWI Rajasthan 2015 independent monitoring tables, Pg 3
• 152Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 4
• 153Deworming Day and Mop-up Day (N = 250 schools) DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 3
• 154DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 7
• 155Coverage Validation (N = 748 schools) DtWI Bihar 2015 independent monitoring tables, Pg 5
• 156Coverage validation (N = 375) DtWI Rajasthan 2015 independent monitoring tables, Pg 5
• 157Coverage validation (N = 253 schools), DtWI Delhi 2015 independent monitoring tables, Pg 9
• 158Coverage validation (N = 750 schools) DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 5
• 159Coverage validation survey (N = 490 schools), DtWI Chhattisgarh 2015 coverage validation tables, Pg 1
• 160DtWI Madhya Pradesh 2015 coverage validation form, Pg 5
• 161

  "Following the Government of India's decision to launch phase one of the National Deworming Day in 12 states, including Chhattisgarh, and release of the National Deworming Day operational guidelines, we reached out to the Government of Chhattisgarh in early February 2015 to explore whether a light technical assistance could assist in kick starting National Deworming Day implementation. In response to our proposal, the director of the National Health Mission agreed to retain Evidence Action's support for National Deworming Day implementation, including: (i( reinforcement of key training messages to functionaries of health, education, and women and child development departments through bulk messaging (SMS); (i) coverage validation of deworming day and mop-up day implementation; and (iii) data entry and analysis of National Deworming Day field monitoring forms. The details are shared below. The technical assistance from Evidence Action to the state of Chhattisgarh is supported through the United States Agency for INternational Development (USAID)" DtWI Chhattisgarh 2015 coverage validation report, Pg 2.

• 162

  "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.

• 163

  "Use the PRE-DD-SCHOOL instrument to conduct interviews and note observations at the selected primary school before deworming day. In case selected school is closed, does not exist or has already dewormed move to the next nearby primary school. Allow Head teacher or Rep to consult with Head teacher/ REP when necessary. PRE-DD-SCHOOL should be completed along with PRE-DD-ECD or PRE-DD-COMMUNITY." Deworm the World, Kenya Year 2, Pre DD - School instrument, Pg 1.

• 164

  "According to interviews with head teachers, 97% of these schools had a teacher who had attended training in the past 15 days. A further 86% of trained teachers had trained or sensitized other teachers on how to administer drugs and conduct deworming day. Almost all teachers (99%) found the Teacher Training Booklet to be ‘very’ or ‘somewhat’ useful in this process and 95% reported to use it often." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.

• 165

  "At the time of PMCV visits, 95% of schools reported having received deworming tablets, with 97% of those schools having received them at the time of TT. This percentage is higher than those originally observed by PMCV officers with only 92% of schools reported to receive their drugs during TT. Upon further investigation, 87% of schools considered that they had received a sufficient supply of drugs for their current enrolled and ECD populations. It is likely that these schools requested additional drugs, because 98% of schools were observed to have sufficient drugs in place on deworming day. Only 5% of schools did not have all monitoring forms present prior to deworming day. Such schools have always sought support from the sub-county offices that organize additional prints or photocopying to ensure they have the forms on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16.

• 166

  The survey instruments that Deworm the World's monitors use are long (e.g., see Deworm the World, Kenya Year 2, Pre DD - School instrument, in which both teachers and randomly selected students are interviewed). The monitoring reports only include a few summary metrics, and we are unsure how the metrics included in the report are chosen. It is possible that the report's metrics are those that reflect most favorably on Deworm the World and that vetting the full data could reveal that schools are less adequately prepared.

• 167
  • "Approximately how many Albendazole tablets have you received? WRITE NUMBER (check physical drugs)" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 7.
  • "How many ECD children are enrolled at the attached ECD centre?WRITE NUMBER" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 10.
  • "How many primary schoolstudents are enrolled in this school? (Exclude attached ECD)WRITE NUMBER" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 2.

• 168
  • "Randomly Selected Class and Stream: (To be assigned before school visit)" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 1.
  • "Answer the following questions at the end of the deworming exercise (for STH):...Did the school seem to have enough tablets of Albendazole for the day? SINGLE CODE" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 23.
  • "HEAD TEACHER POST-INTERVIEW...Was the availability of drugs sufficient for today? SINGLE CODE" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 24.

• 169

  Deworm the World, Kenya Deworming Day data, Year 3

• 170
  • Note that we focus on questions related to Albendazole in our analysis. Deworm the World also supports some treatments for schistosomiasis (treated with Praziquantel) in Kenya, but the large majority of schools surveyed were only treating with Albendazole. Of the five schools monitors observed that were also treating with Praziquantel, all five reported that they had sufficient Praziquantel. Deworm the World, Kenya Deworming Day data, Year 3
  • We chose sample questions to present here based on a) what we thought was easy to understand, and b) our sense of what information would be most likely to change our view about Deworm the World.

• 171Deworm the World, Kenya Year 3, DD - Main instrument, Pg 5
• 172Deworm the World, Kenya Year 3, DD - Main instrument, Pg 6
• 173Deworm the World, Kenya Year 3, DD - Main instrument, Pg 9
• 174

  For example, the most recent summary report from Kenya reported that 98% of schools visited on Deworming Day had ordered sufficient drugs, 9% of schools were observed to run out of drugs during Deworming Day, and 92% of head teachers reported that their school had sufficient drugs:

  • "Pre-Deworming: The percentage of target schools with the appropriate drugs (according to the number ordered and enrolment numbers) - albendazole (ALB) for treating STH and praziquantel (PZQ) where required for treating SCH - in place on Deworming Day was 98% or 243 out of 247 sampled schools.
  • Deworming: Only 9% percent of sampled schools were observed to run out of drugs on Deworming Day. However it was noted that only 87% of schools had enough PZQ for treating SCH in non-enrolled children.
  • Post-Deworming: According to interviews with head teachers post-Deworming Day, 92% of schools recorded a sufficient supply of drugs. This marginal difference between pre and post-reports of sufficient drug supply may indicate the underestimation of the drugs required to treat enrolled, non-enrolled, and ECD children. Indeed, this assumption is further supported by reports by PMCV field teams regarding an insufficient supply of drugs in 9% of sampled schools."

  Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pgs 16 - 17. See the table above for results from our analysis, or Deworm the World, Kenya Deworming Day data, Year 3.
  Note:We are not sure what to make of the fact that some of the figures reported seem to slightly contradict each other. If 9% of schools were observed to run out of drugs, then 98% couldn't have ordered a sufficient number of drugs.

• 175

  Compare the table above to: "The percentage of target schools with the appropriate drugs (according to the number ordered and enrolment numbers) - albendazole (ALB) for treating STH and praziquantel (PZQ) where required for treating SCH - in place on Deworming Day was 98% or 243 out of 247 sampled schools." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16.

• 176
  • Bihar: "Two-stage probability sampling was used to select schools for coverage validation on deworming day and mop-up day. First, 125 blocks were selected from all 38 districts by probability proportional to size sampling (Probability proportional to size sampling (PPS) selected blocks in Bihar, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. Schools were then randomly selected from the selected blocks.), followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students.
    During coverage validation an additional 748 randomly sampled schools were surveyed after deworming days to check whether deworming occurred, if reporting protocols were followed, and to validate the coverage reporting. Coverage validation data was gathered through interviews with headmasters and 3 students (in 3 different randomly selected classes in each school), and by checking of all class registers and reporting forms." DtWI Bihar 2015 Program report, Pg. 15.
  • Rajasthan: "Two-stage probability sampling was used to select schools for independent monitoring on Deworming Day and mop-up day. First, 125 blocks were selected from all 33 districts by probability proportional to size (PPS) sampling (Blocks were selected by Probability proportional to size sampling (PPS) in Rajasthan, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. Schools were then randomly selected from the selected blocks.), followed by random sampling of schools to provide state-wide estimates of indicators." @DtWI Rajasthan 2015 Program report@, Pg. 14.
  • Madhya Pradesh: "A two-stage probability sampling process (Blocks were selected by Probability Proportional to Size (PPS) sampling, followed by random sampling of schools to provide state-wide estimates of indicators. We used PPS sampling to select blocks in Madhya Pradesh, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. After selecting blocks, we randomly selected schools from within these blocks.) was followed to select schools for NDD, mop-up day and coverage validation." @DtWI Madhya Pradesh 2015 Program report@, Pg. 16.
  • Delhi: "The method of stratified random sampling using proportional allocation approach (A random sample of population in which the population is first divided into distinct subgroups or strata, and random samples are then taken separately for each stratum proportional to the size of the subgroup.) was followed for selection of schools and anganwadis for deworming day, mop-up day, and coverage validation monitoring to provide state-wide estimates of indicators." @DtWI Delhi 2015 Program report@, Pg. 14.

• 177

  For example, in Delhi, monitors are instructed in the survey instrument, "Is the school open? [If not,] Did you go to the buffer school?" DtWI Delhi 2015 monitoring survey from deworming day, schools Pg 3.

• 178

  We spot-checked several states' data:

  • DtWI Bihar 2015 monitoring data from deworming day, schools, Column L shows that all schools monitors visited were open.
  • DtWI Rajasthan 2015 monitoring data from deworming day, schools, Column M shows that 124 of the 125 schools monitors visited were open.

• 179
  • "Please choose a class Randomly for questions from class teacher." DtWI Bihar 2015 monitoring survey for coverage validation, schools, Pg. 12.
  • "Select a child randomly and Take the child outside the class/away from the teacher." DtWI Bihar 2015 monitoring survey for coverage validation, schools, Pg. 14.
  • Bihar "First, 125 blocks were selected from all 38 districts by probability proportional to size sampling, followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students." DtWI Bihar 2015 Program report Pgs 15-16
  • Chhattisgarh "We visited 10 randomly selected schools in each block from the 55 blocks in these 11 districts. Therefore, they visited a total of 550 randomly sampled schools for coverage validation activities. In addition to the headmaster’s interview and verification of the deworming related documents, three randomly selected children from three different randomly selected classes were interviewed in each school." DtWI Chhattisgarh 2015 coverage validation report, Pg 3
  Selected results from monitoring
  • Definitions
    • Monitors observe deworming activities on DD: At the schools monitors observed (not closed), monitors, randomly chose one classroom in which to observe deworming activities taking place. E.g. in the Delhi survey instrument, "Check if students in the class are being given or preparing to be given deworming medicine. Can you see any deworming activity taking place in the class? [Yes or No]" DtWI Delhi 2015 monitoring survey from deworming day, schools, Pg 17.
    • Principal reports having received medicine (on DD or MUD): asked of principals by monitors when they visit schools on deworming day and mop-up day; this figure counts all answers on either day, though each principal was only asked on at most one day. E.g. in the Delhi survey instrument, "Did your school receive the deworming tablets/syrups? [Yes or No]" DtWI Delhi 2015 monitoring survey from deworming day, schools, Pg 11.
    • Principal reports conducting deworming on DD (same day): asked of principals on deworming day. E.g. in the Delhi survey instrument, "Are students getting deworming drug in your school today?" DtWI Delhi 2015 monitoring survey from deworming day, schools, Pg 13.
    • Present children report being dewormed, at schools where principal reported deworming happened (coverage validation day) - GiveWell calculation: On coverage validation day, monitors went to schools and asked principals whether their school had conducted a deworming. If they said yes, the monitor randomly chose three classes, and in each class, interviewed one student. For each student, if they were marked as present on the class attendance register for either deworming day or mop-up day, the monitor took them to a place to interview them privately and asked them - among other things - (a) whether they consumed a deworming pill, and (b) if so, how they consumed it. This metric is the portion of present students interviewed that said yes to (a) and didn’t say, in response to (b), that they [actually] didn’t consume it. E.g. in the Delhi survey instrument, "Did you consume the medicine that was given to you? [Yes or No]... If you got a tablet, how did you consume it? [Prompt with options 1. Chewed tablet before swallowing, 2. Swallowed tablet directly, 3. Didn’t consume it, 4. Don’t know / don’t remember, 5. Other, specify, 6. No response.]" DtWI Delhi 2015 monitoring survey for coverage validation, schools, Pgs 21, 23.
    • We believe coverage validation was a few days or up to a couple weeks after deworming days. For example:
      • Madhya Pradesh 2015: "The monitors visited 125 randomly selected schools on NDD, and an additional 125 schools on mop-up day (February 14) to check for adequacy of drug supplies and awareness materials, and assess whether teachers had received training, and had knowledge of adverse event management protocols and reporting processes. Monitors gathered data by observing deworming and by interviewing headmasters, teachers, and randomly selected students. An additional 750 randomly sampled schools were surveyed from February 18-26 to check whether deworming occurred and reporting protocols were followed, and to validate the coverage reporting." DtWI Madhya Pradesh 2015 program report, Pg 16
      • Delhi 2015: "These monitors were to visit total of 400 randomly selected schools and 400 randomly selected anganwadis; 80 schools and 80 anganwadis on deworming day and mop up day each (April 16 & April 20); and 240 schools and 240 anganwadis during coverage validation (April 23-27, 2015)." DtWI Delhi 2015 program report, Pg 14
  • Sources
    • Deworming day data from each state: DtWI Bihar 2015 monitoring data from deworming day, schools, DtWI Delhi 2015 monitoring data from deworming day, schools, DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools, DtWI Rajasthan 2015 monitoring data from deworming day, schools, DtWI Delhi 2015 monitoring data from deworming day, anganwadis
    • Mopup day data from each state: DtWI Bihar 2015 monitoring data from mopup day, schools, DtWI Delhi 2015 monitoring data from mopup day, schools, DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools, DtWI Rajasthan 2015 monitoring data from mopup day, schools, DtWI Delhi 2015 monitoring data from mopup day, anganwadis
    • Coverage validation data from each state: DtWI Bihar 2015 monitoring data for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools, DtWI Delhi 2015 monitoring data for coverage validation, schools, DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools, DtWI Rajasthan 2015 monitoring data for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis, DtWI Delhi 2015 monitoring data for coverage validation, anganwadis, DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis
    • Deworming day survey instrument from each state: DtWI Bihar 2015 monitoring survey from deworming day, schools, DtWI Delhi 2015 monitoring survey from deworming day, schools, DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools, DtWI Rajasthan 2015 monitoring survey from deworming day, schools, DtWI Delhi 2015 monitoring survey from deworming day, anganwadis
    • Mopup day survey instrument from each state: DtWI Bihar 2015 monitoring survey from mopup day, schools, DtWI Delhi 2015 monitoring survey from mopup day, schools, DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools, DtWI Rajasthan 2015 monitoring survey from mopup day, schools, DtWI Delhi 2015 monitoring survey from mopup day, anganwadis
    • Coverage validation survey instrument from each state: DtWI Bihar 2015 monitoring survey for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools, DtWI Delhi 2015 monitoring survey for coverage validation, schools, DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools, DtWI Rajasthan 2015 monitoring survey for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis, DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis, DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis
  • We did not include Chhattisgarh in this table since Deworm the World conducted a coverage validation process but not the other elements of the monitoring process on which this table draws.

• 180

  Grace Hollister, email to GiveWell, March 6, 2016

• 181

  "Use the DD-MAIN instrument to conduct interviews and note observations at the selected primary school on deworming day. In case selected school is closed or does not exist or has already dewormed, move to the next nearby primary school. DD-MAIN should be completed along with DD-INT Maintain detailed notes in field diary on observations from this visit." Deworm the World, Kenya Year 3, DD - Main instrument, Pg 1.

• 182

  See Deworm the World, Kenya Year 3, Post DD - Coverage instrument. On Pgs 1-3 there are three spaces for randomly selected classes. The instructions read: "Thank the Head Teacher or designate and request to speak to pupils of the randomized class...CHOOSE CHILD 5, 10 AND 15TH ON FORM E. IF LESS THAN 15 CHILDREN, SELECT THE LAST CHILD. ENSURE TO INTERVIEW AT LEAST THREE CHILDREN. ASK THE TEACHER FOR PERMISSION TO SPEAK TO THEM ONE AT A TIME Ask questions in multiple ways for interviews with students, use local language if possible. Don’t rush responses. Try to make them feel at ease. Speak to one child at a time at a place where they are comfortable.... [interview questions]...END, MOVE TO THE NEXT SAMPLED CLASS"

• 183
  • Deworm the World, Kenya Deworming Day data, Year 3
  • We chose sample questions to present here based on a) what we thought was easy to understand, and b) our sense of what information would be most likely to change our view about Deworm the World, and c) comparability to the questions we looked at for Deworm the World's program in India.
  • Note that we include several schools in our analysis that monitors did not record observations for (reasons include that the school was closed or did not exist).

• 184This number is calculated by looking at monitors' observations of classroom activities on Deworming Day. It is the number of classroom observations in which at least one child was observed to take a deworming pill over the total number of observations. See column ZD of Deworm the World, Kenya Deworming Day data, Year 3.
• 185Deworm the World, Kenya Year 3, DD - Main instrument, Pg 14
• 186Deworm the World, Kenya Year 3, DD - Main instrument, Pg 2
• 187Deworm the World, Kenya Year 3, Post DD - Coverage instrument, Pg 2
• 188

  The monitoring reports from Kenya do not include strongly compelling evidence in support of the claim that Deworming Days widely occurred. The most recent summary monitoring report notes that "deworming was considered to be 'systematic' in 98% of schools." We are not sure what definition of "systematic" is being used when calculating these figures.

  • "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment.

    Deworming was reported to occur inside classes in 47% of observed schools and outside in 53% of schools. Deworming was considered to be ‘systematic’ in 98% of schools. The correct dosage for albendazole is one tablet per child and the correct age is 2-14 years. These procedures were observed to be followed correctly by 86% of teachers observed by field officers.

    Coverage: Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools. Teachers were reported to correctly observe children swallowing PZQ in 99% of schools and ALB in 96% of schools. Observing children swallowing is most important when treating for SCH as the tablet does not taste pleasant and there are high chances of children spitting if not observed."
    Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17.

• 189

  Deworm the World notes that an important difference between the two programs is scale: The Kenya program is significantly smaller than the India program, thus it may be easier to execute the program more effectively. Grace Hollister, email to GiveWell, June 9, 2016

• 190

  Monitors are prompted to answer:

  • "Did the teacher ask the children if they are sick/under medication before giving the medicine?"
  • "Did you see any child being given less than one deworming tablet?"
  • "Did you see any child being given more than one deworming tablet?"

  DtWI Rajasthan 2015 monitoring survey from mopup day, schools, Pgs 13-14.

• 191

  The metrics we choose to present are based on how easy we believe they are to understand and how relevant we think they are to one's view of Deworm the World.

• 192
  • Monitors only did coverage validation in Chhattisgarh, and did not conduct monitoring activities on deworming day or mop-up day. The results in the table below are only from classes where monitors observed deworming activities.
  • We chose sample questions to present here based on a) what we thought was easy to understand, and b) our sense of what information would be most likely to change our view about Deworm the World.

• 193Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 2
• 194Deworming Day and Mop-up Day (N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 2
• 195Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 2
• 196Deworming Day and Mop-up Day (N = 250 schools) DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2
• 197DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 13
• 198Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 2
• 199Deworming Day and Mop-up Day (N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 1
• 200Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 2
• 201Deworming Day and Mop-up Day (N = 250 schools) DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2
• 202DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 13
• 203Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 1
• 204Deworming Day and Mop-up Day (N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 1
• 205Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 2
• 206Deworming Day and Mop-up Day (N = 250 schools) DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 1
• 207DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 13
• 208

  We present metrics that are similar to the metrics we chose to present for India, to allow for easier comparison.

• 209This figure only includes schools that conducted Deworming Day. Deworm the World, Kenya Deworming Day data, Year 3
• 210Deworm the World, Kenya Year 3, DD - Main instrument, Pg 19
• 211Deworm the World, Kenya Year 3, DD - Main instrument, Pg 14
• 212Deworm the World, Kenya Year 3, DD - Main instrument, Pg 19
• 213

  The monitoring reports from Kenya do not include strongly compelling evidence in support of the claim that proper procedures were followed:

  • "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment.

    Deworming was reported to occur inside classes in 47% of observed schools and outside in 53% of schools. Deworming was considered to be ‘systematic’ in 98% of schools. The correct dosage for albendazole is one tablet per child and the correct age is 2-14 years. These procedures were observed to be followed correctly by 86% of teachers observed by field officers.

    Coverage: Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools. Teachers were reported to correctly observe children swallowing PZQ in 99% of schools and ALB in 96% of schools. Observing children swallowing is most important when treating for SCH as the tablet does not taste pleasant and there are high chances of children spitting if not observed."
    Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17.

• 214

  "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2.

  "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.

• 215

  Professor Devesh Kapur Biography 2013

• 216

  Devesh Kapur conversation October 14th 2013

• 217

  "Some school health programs, such as the WIFS program in Delhi, are rolled out too quickly and their trainings are carried out poorly. Then they receive negative media attention because of students’ adverse reactions to treatments. Negative media can cause delays or cancellation of school health programs. The WIFS program has also received negative media attention in the states of Haryana and Odisha." DtWI 2013 GiveWell site visit, Pg 10.

• 218

  "There have been very few adverse reactions to deworming treatments in DtWI-supported states because of the quality of Deworm the World's trainings." DtWI 2013 GiveWell site visit, Pg 10.

• 219
  • "National funds for deworming are part of the [Weekly Iron Fortification Supplements] WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12.
  • "Wherever possible, Deworm the World works to get deworming added as a line item in the budget. Otherwise, it is difficult to ensure that long-term funding will exist for the program." DtWI 2013 GiveWell site visit, Pg 10.
  • "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.

• 220
  • Alix Zwane conversation August 30th 2013
  • "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.
  • Assam has also announced its intentions to move forward with its deworming program in March 2013: "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. As reported by NRHM, Assam any [sic] school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam midday meal report 2013, Pg 11.
  • According to Deworm the World, "No schools in Assam have been covered under the School Health Program to provide deworming in 2012/13. The programme was then re-scheduled to begin in March 2013. No evidence of this actually having taken place." DtWI Assam research 2013, Pg 1.
  • "[Deworm the World is] also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2.
  • Originally Deworm the World didn’t believe that Jharkhand conducted a school-based deworming MDA in 2013, but learned of it by early 2014.
    • Alix Zwane conversation August 30th 2013
    • GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.
  • "Note that not all of these programs would have been operating at scale (the ones we supported were)." Grace Hollister, email to GiveWell, June 9, 2016

• 221
  • Deworm the World was not involved in Punjab, Jharkhand, nor Assam launching their statewide school-based deworming programs.
  • "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.
  • "DtW has been involved in deworming programs in four different states, and is still actively involved of three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement." Grace Hollister conversation June 19th 2013, Pg 1.
  • "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1.
  • "In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J-PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there. In August a memorandum of understanding (MoU) was formalized between DtW and the relevant players in Bihar (School Health Society Bihar and Bihar Education Project Council) for program implementation." Grace Hollister conversation June 19th 2013, Pg 2.
  • "DtW leveraged its networks to engage the support of the Minister of Health, Women, and Child Welfare [in Delhi]. Education stakeholders were also brought in so that the program could reach beyond preschool to older grades as well. DtW conducted a great deal of advocacy, maintaining continuous interactions and significant support among all relevant departments to bring the program to fruition Due to the complicated agency system, there were many other government stakeholders as well, requiring a high degree of coordination by DtW to ensure program objectives could be achieved." Grace Hollister conversation June 19th 2013, Pg 3.
  • "After the deworming program launched in Delhi, Rajasthan saw the results generated by the DtW- supported program in Delhi and committed to doing a deworming program, allocating funding for it in their budget. In March 2012 they brought DtW in to help. In this case, the state already knew what it wanted and already had a school health program. They sought DtW's technical expertise, mapping ability, general program support, stakeholder coordination, etc. Deworm the World coordinated signature of a MoU between the Departments of Women and Child Development, Education, and Health, UNICEF and DtW to guide program implementation, and helped establish of a technical secretariat housed within the Education Department." Grace Hollister conversation June 19th 2013, Pg 3.
  • "DtW has worked in Delhi, Bihar, Rajasthan, and Andhra Pradesh (AP).
    • In Bihar, according to documentation provided by DtW, the program began because of previous Poverty Action Lab (J-PAL) work in Bihar. DtW also noted that Rajasthan approached them because of the success (as highlighted in the media) of the Delhi program. DtW supported a pilot program and prevalence survey in AP, and though DtW didn't continue to engage with AP, the AP government seems to have recently conducted a school-based deworming program.
    • More recently, Punjab approached DtW, but DtW didn't have the funding to move forward with Punjab, and Punjab is now implementing deworming on its own.

    It's not possible to be certain whether or not DtW has caused deworming to happen that otherwise would not have, but it's important to note that DtW took the opportunities presented to help run better programs where possible as well as improve data quality and did so on a limited budget. They are also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2.

• 222
  • GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
  • While 11 states and Delhi participated in the first NDD in 2015, all states are supposed to participate in the second NDD in 2016. "The NDD slated for February 2016 is supposed to cover all states in India, per the Ministry of Health and Family Welfare’s issuance of correspondence to state governments." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015

• 223

  "Keeping in view the problem of worm infestation of children in Assam it has been decided to conduct two rounds of De-worming every year in the month of March and September. The first round will be conducted during March’ 2010. On every Wednesday (VHND/Immunization Day) medicine for De-worming will be given to all children between the age group 1 to 5 years along with vitamin A supplementation." Assam 2010 guidelines for deworming, Pg 1.

  976,192 "children below 5 years provided vitamin A syrup" as of November/December 2010 Assam reproductive and child health 2011-2012, Pg 73.

  The National Rural Health Mission reported the progress for 2010-2011 as 92,957 "Students given IFA/ de-worming tablets". Assam state programme implementation plan 2011-2012, Pg 21.

  Assam is "supposed to provide deworming to all children, but AWC services provided in Assam are some of the worst performers for deworming coverage." DtWI Assam research 2013, Pg 1.

• 224

  "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. NB. As reported by NRHM, Assam any school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam midday meal report 2013, Pg 11.

• 225
  • Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013.
  • "Deworm the World continues to support the four states it was working in previously (Rajasthan, Delhi, Bihar, and Madhya Pradesh) and has added programs in Chhattisgarh and Uttar Pradesh. [...] Deworm the World is in discussions with Telangana [...] Deworm the World has phased out its comprehensive technical assistance to Delhi, but the national office will continue to provide minimal support, including attending meetings and doing some policy advocacy work. Deworm the World no longer has dedicated staff for Delhi." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pgs 1-2

• 226

  DtWI 2013 GiveWell government interviews.

• 227

  "If Deworm the World had not been there, how would the deworming program be different?

  • Since Deworm the World have already implemented deworming programs in Delhi and Tamil Nadu, they brought experience.
  • If a state government decides to do something, nothing is impossible. However, Deworm the World's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have Deworm the World's experience, there could have been more problems.
  • The government received important support from DtWI, particularly with distributing the deworming tablets, implementing the trainings, and developing training materials."

  "Have you ever wanted to do a program but been unable to find a technical partner?

  • Ms. Gupta is still looking for a technical partner on specific issues. For example, Rajasthan has a large population with special needs. It has generally been able to find partners for helping the visually impaired, but very few organizations work with autistic children, children with cerebral palsy, or children with mental retardation. If there were a partner to support the Rajasthan government in this area, this would be very welcome. The Rajasthan government has funds for this type of program, but are looking for a technical partner.
  • An NGO called Sight Savers works with blind children. Before Sight Savers, she was not aware of problems with low vision. Many children need magnifying glasses and large print books. Sight Savers helped them to identify doctors and hospitals to work with such children."

  DtWI 2013 GiveWell government interviews, Pg 7-9.

• 228
  • "Additionally the national government of India has expressed interest in possibly
    implementing a deworming program nationwide, rather than waiting for each state to
    launch separate programs. They have expressed interest in receiving technical assistance
    from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2
  • Grace Hollister, email to GiveWell, June 9, 2016

• 229
  • "Deworm the World Initiative at Evidence Action will be the technical assistance partner to MoHFW, Government of India. The specific responsibilities are as follows:
    Support in development of National Deworming Day implementation strategy
    Design and develop training and reference materials, community mobilization strategies for increased awareness and coverage of target beneficiaries, Monitoring and Evaluation (M&E) Systems and reporting formats" India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015, Pg 10.
  • "Deworm the World has three primary roles in National Deworming Day: • Agenda setting: Deworm the World has played a large role in getting deworming on the national government’s health agenda, which led to the creation of National Deworming Day. Before Deworm the World began working with the national government, deworming was officially a part of some health programs, but was inconsistently implemented outside of those states where Deworm the World provides technical assistance. • Developing materials and advising the national government: Deworm the World has also developed operational guidelines for program implementation, including training materials, public awareness materials, monitoring forms, guidance for teachers and health workers, and a Frequently Asked Questions guide about National Deworming Day, in partnership with the national government. The national government will distribute these materials to state governments for use on National Deworming Day; some materials will be adapted to state-specific contexts. Deworm the World’s work plan also advised the national government on appropriate timing for workshops and trainings, and plans to conduct a training for state-level functionaries once the date of deworming is announced. Deworm the World’s India Country Director Priya Jha speaks with Dr. Khera of the Child Health Division at the Indian Ministry of Health and Family Welfare on a weekly basis about plans for National Deworming Day. • Direct work with state governments: Deworm the World will work individually with the states Bihar, Rajasthan, Delhi, and Madhya Pradesh to adapt National Deworming Day guidance into those programs. Deworm the World has been working in the first three of those states for multiple rounds of deworming. Funding for working in Madhya Pradesh comes from a USAID grant, for which this will be the first round of deworming support. Deworm the World also hopes to work in Chhattisgarh with additional funds from USAID, and Odisha and Uttar Pradesh with funding from the Children's Investment Fund Foundation. Other than developing operational guidelines and materials, and providing highlevel training for key state functionaries, Deworm the World does not have the capacity to be involved in the first phase of National Deworming Day in other states in India. Deworm the World had originally planned to create a national implementation and monitoring workshop for representatives from each Indian state prior to National Deworming Day. However, it is too late to host the workshop before the first National Deworming Day in February 2015. Editor’s note (based on updates after this conversation): Deworm the World did provide some support to Chhattisgarh and supported the national government in hosting a workshop." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pgs 1-2
  • "Additionally the national government of India has expressed interest in possibly
    implementing a deworming program nationwide, rather than waiting for each state to
    launch separate programs. They have expressed interest in receiving technical assistance
    from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2
  • "The Indian national government has recently demonstrated an increased interest in deworming programs. The Child Health Division of the Ministry of Health & Family Welfare (MoHFW) has been particularly interested in deworming. Deworm the World Initiative (DtWI)’s program in India is working with Ajay Khera, the leader of the Child Health Division, and other staff members in the division to encourage the division to continue promoting deworming. DtWI has also been meeting with the Joint Secretary of MoHFW and requesting appointments with the Minister of Health & Family Welfare. Deworming has been proposed to be included in the 100 Day Agenda of the new administration in India. DtWI is a stakeholder supporting this effort. DtWI has proposed to assist with several aspects of the program including planning, guideline development and other technical issues." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014, Pg 1
  • "States which did not receive technical support from Deworm the World may not have had adequate time or support for planning the February deworming round. This may have resulted in lower coverage and weaker monitoring. Some states would likely appreciate assistance from Deworm the World or another similar organization." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2

• 230

  India NDD documents 2015

• 231
  • "National Deworming Day was originally planned for October 2014, but was delayed after Deworm the World staff alerted the government that drug supplies were insufficient. The Indian government has publically announced that National Deworming Day will be in February 2015, but it is possible that there will be further delays." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1
  • "Deworm the World had originally planned to create a national implementation and monitoring workshop for representatives from each Indian state prior to National Deworming Day. However, it is too late to host the workshop before the first National Deworming Day in February 2015.
    Editor’s note (based on updates after this conversation): Deworm the World did provide some support to Chhattisgarh and supported the national government in hosting a workshop." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 2

• 232

  "A program of the Government of India, the February 10 event in eleven states targeted 140 million children with school-based deworming treatment." DtWI NDD blog post February 2015

• 233
  • "On National Deworming Day, 12 states in India selected for inclusion in the program’s first phase will be required to administer deworming drugs to schoolchildren, regardless of measures of worm prevalence." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1
  • "India held its first National Deworming Day in February. Eleven states participated." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2.
  • "11 dewormed in February; the 12th, Delhi, dewormed in April. This delay was due to conflicts of the February timing with state elections, which require the participation of teachers." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015

• 234
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 235

  "DtWI's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have Deworm the World's experience, there could have been more problems." Veenu Gupta, Principal Secretary to School Education Department, Government of Rajasthan DtWI 2013 GiveWell government interviews, Pg 7.

• 236

  "DtWI has helped to improve deworming trainings. Last year, Deworm the World helped to develop the content for the trainings. Deworm the World made the content more concise and easy to understand than the government would have done on its own. Deworm the World also improves trainings by tracking when people do not show up to trainings and following up with them and by determining aspects of training that could be improved in future years." Girish Bharbwag, Nodal Officer in Rajasthan DtWI 2013 GiveWell government interviews, Pg 4.

• 237

  The District Coordinator position, for example, is not always staffed in other school-based health programs. "The state government paid for District Coordinators for the hand washing program in 2008. However, the payment for this position was low. In the first year, there were 15-16 independent monitors. This year, there are only 7 District Coordinators. The government is finding it difficult to fill these vacancies." DtWI 2013 GiveWell government interviews, Pg 5.

  "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems.
  The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 4.

• 238

  "The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 5.

• 239
  • In Bihar, Deworm the World says that it has convinced the state to expand its program to include preschool-aged children in the next round of deworming, set for early 2015. GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
  • The MDA that Jharkhand implemented before it was in contact with Deworm the World only targeted 10-19 year olds: "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.

• 240

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 241
  • DtWI Monitoring Improvements 2014
  • DtWI Bihar 2014 program report
  • DtWI Delhi 2013 program report
  • DtWI Rajasthan 2013 program report
  • GiveWell DtWI 2013-2014 cost data summary

• 242
  • Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16
  • Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness.

    Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15

  • Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15
  • Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials.

    Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17

• 243
  • See citations in previous footnote.
  • "These calls helped to uncover problems, which are reported on the same day to the appropriate government officials to address" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • We have not seen any detailed data of the calls made or the issues uncovered or whether they are resolved.
  • We are only aware of one specific example of the type and magnitude of problems uncovered: In the Bihar 2014 program, Deworm the World-led monitoring uncovered that 18% of schools that were called had not received the drugs at the scheduled time, and it seems plausible that the issue was largely addressed before deworming day (though we have not vetted detailed data supporting this).
    "BEO [Block education officer] offices distributed the drugs among the school headmasters at block level trainings. In monitoring phone calls, 82% of the 354 schools contacted by Deworm the World tele-callers across 27 districts confirmed receiving drugs two days prior to Deworming Day. When this potentially problematic information came to light, the tele-calling team and DCs hired by Deworm the World contacted all the BEO offices to ensure delivery of drugs to all the schools before Deworming Day along with instructions issued from the BEPC nodal officer. Subsequent independent monitoring data (from visits to schools during Deworming Day and Mop-Up Day) shows that approximately 96% schools had received drugs by Deworming Day. This was a significant jump from the 82% polled only two days earlier." DtWI Bihar 2014 program report, Pg 13.

• 244

  Our observations are noted here: DtWI 2013 GiveWell site visit.

• 245

  In 2014, Deworm the World reported improvements to the training process, including focusing on areas where problems were identified in previous rounds, simplifying materials, and testing if participants are learning key messages. DtWI Monitoring Improvements 2014, Pgs 1-2.

• 246
  • Our intervention report discusses this briefly
  • Other conversations and observations have reinforced our impression that administering deworming drugs is fairly straightforward.
  • The WHO factsheet on STH: "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet

• 247

  Deworm the World notes that: "albendazole and mebendazole are quite hardy (neither requires special storage conditions) and have a long shelf life." Grace Hollister, email to GiveWell, June 9, 2016

• 248

  For example, drug quality was tested in each program that Deworm the World supported in India in 2013-2014.

  • Bihar 2014:
    • "The drugs were safely stored in State Health Depot in Patna until November 2013. At that point, the districts began to pick up their share of the drugs from the depot. In October, Deworm the World had coordinated for lab testing of the stored drugs via ASCHO NIBULA INDUSTRIES LTD, an independent lab which approved the quality of the drugs." DtWI Bihar 2014 program report, Pg 13.
    • "The quality of drug storage was satisfactory in most schools that were monitored. 98.7% of them stored the drugs in a clean location, 91.1% of them were stored away from direct sunlight and 97.4% of them were stored away from the direct reach of children." DtWI Bihar 2014 program report annex 1, Pg 7.
  • Delhi 2013:
    • "Once the procured syrups and donated tablets were delivered to Directorate of Health Services central storage room, they were tested in a government-accredited laboratory to ensure drug quality prior to administration." DtWI Delhi 2013 program report, Pg 6.
    • From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Delhi 2013 program report, Pg 24. A table of results is also presented on the same page.
  • Rajasthan 2013:
    • "To instill confidence among the stakeholders that the drugs were of good quality, Deworm the World arranged for sample testing of the donated drugs by two independent labs7. Similarly, Rajasthan Medical Services Corporation sample tested the syrups they procured." DtWI Rajasthan 2013 program report, Pg 6.
    • From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Rajasthan 2013 program report, Pg 40. A table of results is also presented on the following page.

• 249

  Of classes where monitors observed deworming activities in India, there were low numbers of adverse events (see table below). We aren't sure what portion of the adverse events may be caused by incorrect dosages.

  	Bihar 2015	Rajasthan 2015	Delhi 2015	Madhya Pradesh 2015	Sample of question asked
  Classes where there were adverse events (monitors' observations)	5% (vomiting), 0% (diarrhea)	2% (vomiting), 0% (diarrhea)	5% (vomiting), 0.8% (diarrhea)	6% (vomiting), 0% (diarrhea)	"Did you see any child with adverse effects (nausea, vomiting, stomachache, etc.) after taking the medicine?"

  Sources for the information in the table:

  • Bihard 2015: Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 4
  • Rajasthan 2015: Deworming Day and Mop-up Day ( N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 4
  • Delhi 2015: Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 6
  • Madhya Pradesh 2015: Deworming Day and Mop-up Day (N = 250 schools), DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 4
  • Sample question: DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 14

• 250
  • "Note that National Deworming Day operational guidelines state that only tablets should be used. Albendazole dosage is the same for all children aged 2 and above; it is only children aged 1-2 that require a different (half) dose." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • From Deworm the World's report on the Delhi 2013 program: "A key discussion from round one was the administration of drug to younger children in the pre-school age for whom chewing a tablet was seen as a difficulty. Hence keeping in mind the scale of the program, with an objective of making it a safe public health initiative the committee decided that deworming would be implemented across all districts following the World Health Organization (WHO) sanction in administering albendazole 400 mg tablets and the GOI guidelines under the WIFS program for administering the syrup vial. They also decided to opt for Albendazole 200mg dose suspension for 2-6 year children as it is a single dose for this age group which leads to lesser error in administration, while Albendazole 400mg tablets for older children." DtWI Delhi 2013 program report, Pg 6.
  • The WHO factsheet on STH cites only a single recommended dosage (depending on which drug is used): "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet

• 251
  • Bihar 2015: In 8.8% of trained schools and 5.4% of untrained schools, monitors observed children given less than one tablet; in 2.6% of trained schools and 4.7% of untrained schools, monitors observed children given more than one tablet. DtWI Bihar 2015 independent monitoring tables, Pg 9 (Table 10)
  • Rajasthan 2015: In 6.0% of schools, monitors observed children given less than one tablet; in 1.6% of schools, monitors observed children given more than one tablet. DtWI Rajasthan 2015 independent monitoring tables, Pg 2 (Table 2)
  • Madhya Pradesh 2015: In 9.3% of schools, monitors observed children given less than one tablet; in 2.7% of schools, monitors observed children given more than one tablet. DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2 (Table 2)
  • Delhi 2015: In 2.5% of schools and 10.6% of anganwadis, monitors observed children given less than the prescribed dose of albendazole; in 0.8% of schools and 6.0% of anganwadis, monitors observed children given more than the prescribed dose of albendazole. DtWI Delhi 2015 independent monitoring tables, Pgs 2, 16 (Tables S1 and A1)
  • Bihar’s 2014: 8% of schools observed gave children less than one tablet and 2% gave more than one tablet. DtWI Bihar 2014 program report annex 2, Pgs 2 and 4.
  • "In 3.1% of schools and anganwadis, monitors observed children being given more than one tablet/syrup bottle. As per protocol, the children should not have been given more than one tablet. Therefore in cases such as these, the monitors were trained to intervene and prevent the administration of an additional dose." (The percentage of schools using less than one pill/bottle per child was not reported.) DtWI Rajasthan 2013 program report, Pg 38.
  • Monitors in Delhi in 2013 did not report on either of these observations. See DtWI Delhi 2013 program report, Pgs 21-25.

• 252
  • "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2.
  • "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and Deworm the World's tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. Deworm the World called a random sample of 8,000 schools.
    The prevalence survey would not have happened without Deworm the World's support." DtWI 2013 GiveWell government interviews, Pg 5.

• 253

  "RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3.

• 254

  "[The Nodal Headmaster said] that most aspects of the program are excellent, but he had 2 suggestions:

  • Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so)
  • Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6.

• 255

  GiveWell DtWI 2013-2014 cost data summary shows costs of $2,628,511 ($0.089 per treatment), excluding school staff time. Unpublished data show the imputed cost of tablets (which we assume were donated) as $43,200 in Delhi, $373,383 in Bihar, and $147,467 in Rajasthan. Removing these brings the total cost to $2,064,461, or $0.07 per treatment.

• 256

  GiveWell analysis of Deworm the World cost per treatment

• 257
  • "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
  • Deworm the World noted that the costs in Kenya are high, partly due to higher quality M&E:
    • "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
    • "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
    • In the second round of treatment in Kenya, prevalence surveys were 11% of the total costs and other M&E was 6%. Per treatment, these costs are $0.059 ($379,523/6,405,462) and $0.035 ($222,750/6,405,462) respectively DtWI Kenya 2013-2014 cost per treatment data
  • Our understanding is that costs are higher in Kenya than India primarily due to cultural differences, including paying significantly higher allowances (per diems)to teachers for participating in the program. For more details, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell analysis of Deworm the World cost per treatment.
  • Deworm the World has also noted that treating schistosomiasis in Kenya increases the costs: "The extra costs of treating schistosomiasis in addition to STH. Schistosomiasis drugs tend to be more expensive than STH drugs and, in Kenya, schistosomiasis treatment sites are sometimes much more remote. The treatment strategy for schistosomiasis also differs from STH because schistosomiasis is more localized (e.g., it is not necessarily ideal to treat an entire sub-county). This also makes mapping schistosomiasis more expensive." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 3.

• 258

  See Evidence Action, blog post, January 16, 2015 for a description of how Deworm the World calculates the cost of deworming.

• 259

  In general, cost-per-treatment figures have stayed fairly consistent over the last several years in Kenya and India:

  • In the Kenya 2013-2014 treatment round, the total cost-per-treatment was $0.56 (not including staff time). It was the same in the 2014-2015 treatment round. GiveWell analysis of Deworm the World cost-per-treatment, 2016
  • In the India 2012 treatment rounds for which we have information, the average cost-per-treatment was $0.36 (including staff time). In the 2013-2014 treatment rounds, the average cost-per-treatment was $0.30 (see our 2015 review. In the 2014-2015 treatment rounds, the average cost-per-treatment was $0.27. GiveWell analysis of Deworm the World cost-per-treatment, 2016

• 260
  • DtWI cost per treatment summary 2013
  • GiveWell DtWI 2013-2014 cost data summary
  • GiveWell analysis of Deworm the World cost-per-treatment, 2016
  • The rounds for which we have seen cost-per-treatment data are:
    • Bihar: 2012, 2014, and 2015
    • Delhi: 2012 and 2013
    • Rajasthan: 2012, 2013, and 2015
    • Madhya Pradesh: 2015

• 261

  GiveWell analysis of Deworm the World cost-per-treatment, 2016 Note: We focus on these three states because Deworm the World did not send us 2015 cost-per-treatment data for any other states in India.

• 262

  For example, see our 2015 review. We estimated the staff time costs based on DtWI’s estimate of similar costs from the same states in 2012. DtWI had imputed those costs based on estimates of government employee salaries; for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50) (DtWI Rajasthan 2013 cost data).

• 263
  • Alix Zwane conversation August 30th 2013
  • For example, for Madhya Pradesh's 2015 costs: "Drug costs are included in this model as an imputed cost to the government. Unlike other programs supported by Evidence Action in 2014/2015, Madhya Pradesh did not receive Albendazole under the WHO drug donation program. Instead, they procured the drugs locally. In order to ensure that the program had sufficent drug supplies, the government of India purchased roughly $35,000 in Albendazole and Syrups to supplement their existing stocks. Given that the number of drugs disseminated to districts for use on National Deworming Day is unavailable, and the true value of drugs used by the program far exceeds the $35,000 the government spent to supplement existing stocks, this model relies on treatment figures to estimate the value of drugs. This method of calculation is a deviation from the method used to calculate the value of drugs in models representing the cost of other geographies within India for the 2014/2015 round." Deworm the World, Madhya Pradesh cost-per-treatment - 2015

• 264
  • We have not adjusted these data for inflation or changes in exchange rates over time.
  • Deworm the World's 2013 and 2014 budgets stopped estimating the financial value of government employees’ time spent on the deworming program after the deworming rounds in 2012: "We have consulted with JPAL on the costing model approach and they have suggested, in line with papers they've done, not to quantify teachers' time nor principals' time for the actual deworming day; we still have included the cost of teachers attending the training since there is a direct cash transfer to the teachers for their coming to the training." Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014
  • Deworm the World cited JPAL CEAs in education 2011 for this approach.
  • The change is most relevant for teachers and principals, large numbers of whom attend a training and administer the pills to children. While we have seen direct costs of paying teachers and principals that attend deworming training, we are unsure whether those costs should be counted as per-diems (perhaps necessary for some trainees to pay for transport, but not accounting for the lost work time at school) or as optional extra wages (that don’t result in lost work time at school and are sufficient to incentivize the necessary labor), or something in between.
  • In order to include the value of teachers and principals’ time for the three recent rounds, we have copied the relevant expenses from the prior year’s budget of the same state (which were imputed costs) and replaced the allowances that were paid directly to teachers and principals during trainings. We believe this methodology is likely to result in some inaccuracies and we may revise it as we become confident of a more accurate method.
    • GiveWell DtWI 2013-2014 cost data summary
    • DtWI cost per treatment summary 2013, Summary.

• 265

  We estimated the staff time costs based on Deworm the World's estimate of similar costs from the same states in 2012. It had imputed those costs based on estimates of government employee salaries, for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50).

• 266

  See GiveWell analysis of Deworm the World cost-per-treatment, 2016.

• 267
  • GiveWell analysis of Deworm the World cost-per-treatment, 2016
  • Note that this is precisely the same cost estimate Deworm the World developed last year; see our 2015 review.
  • Previously, Deworm the World estimated that the deworming program in Kenya cost around $0.40 per child treated: "We note that preliminary estimates of program costs in Kenya, which we can estimate with far greater precision, suggest programming costs there of about $0.40 per child." Evidence Action cover letter 2013, Pg 2.

• 268

  Excluding the value of teachers' and principals' time, we estimate that Deworm the World's cost per treatment is $0.56 and that SCI's cost per treatment is $0.57. Supporting data and calculations are shown in GiveWell analysis of Deworm the World cost-per-treatment, 2016. Additional explanation follows:

  • Our discussion of SCI's cost per treatment is here.
  • Our best guess is that the total cost per treatment for SCI-supported programs is $1.26, split between 3 categories:
    1. 42% ($0.53) are SCI's program costs
    2. 28% ($0.35) are drug costs
    3. 30% ($0.38) are an estimate of the governments' contributions (which we believe are primarily the value of time spent by teachers and principals supporting the program.)
  • We estimate that Deworm the World's drug costs in Kenya are $0.05 per treatment. We believe that drug costs are lower for the program in Kenya than most of SCI's programs because SCI generally treats for schistosomiasis, but a relatively small proportion of deworming in Kenya includes schistosomiasis treatment, because relatively few areas have high enough prevalence.
  • Deworm the World does more extensive prevalence surveying in Kenya than it does in other countries and than we believe SCI does; this may cause an overestimate of Deworm the World’s costs in other countries.
    • "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
    • "For the NSBDP, three prevalence and intensity surveys for STH and one for schistosomiasis are expected to be or have been completed by KEMRI. The implementation costs of these surveys were divided among the program's expected duration of five years. Therefore, this model includes 1/5 of the total survey-associated costs." DtWI Kenya 2013-2014 cost per treatment data, Introduction sheet.
  • We have not adjusted these data for inflation or changes in exchange rates over time.

• 269
  • See further discussion here.
  • Deworm the World adopted a different approach, citing JPAL CEAs in education 2011.

• 270
  • See our discussion of the rationale and limitations of this estimate here.
  • For comparison, Deworm the World previously estimated the value of in-kind support from the governments in Bihar and Rajasthan. Our understanding is that these estimates suggested in-kind cost of $0.31 per treatment in Bihar 2012 and $0.20 in Rajasthan 2012.
    • For both states, Deworm the World shared detailed data behind its cost per treatment data. This included a sheet for imputed costs. We aggregated these line items, excluding expenses from non-government partners (e.g. WHO and UNICEF), and we believe the result largely accounts for teachers' time during training and deworming day. Treatments from DtWI cost per treatment summary 2013
    • Bihar 2012: DtWI Bihar 2012 cost data details shows $5.21 million, which implies $0.31 per treatment, given 16,867,388 treatments. $5.21 million is converted from INR 276.1 million at 53 INR per USD (based on the source), and is a total of these line items:
      • Master Trainers as Trainers in Block Level Trainings
      • Master Trainers as Trainers in Sector Level Trainings
      • Block Level Training
      • Sector Level Training
      • Honorarium: Teachers and Headmasters
      • Honorarium: Health Department (Doctors and ANMs)
    • Rajasthan 2012: DtWI Rajasthan 2012 cost data details shows $2.06 million, which implies $0.20 per treatment, given 10,132,535 treatments. $2.06 million is converted from INR 109.1 million at 53 INR per USD (based on the source), and is a total of 23 line items, with these 5 largest line items accounting for over 90% of the total:
      • Teachers and Headmasters as Trainees
      • Lady Supervisors and Anganwadi Workers as Trainees
      • Block Level Training
      • Honorarium: Teachers and Headmasters
      • Honorarium: Anganwadi Workers

• 271

  See GiveWell analysis of Deworm the World cost per treatment, sheet "GiveWell estimate"

• 272

  GiveWell analysis of Deworm the World cost-per-treatment, 2016

• 273
  • We have more confidence in the quantity of data on enrolled students dewormed (in contrast with unenrolled students and pre-school children) since Deworm the World collects higher quality monitoring data on this population, as discussed above.
  • In this estimate we are not including the deworming programs in Indian states that launched as part of the National Deworming Day (NDD) in 2015, even though Deworm the World believes that it was critical to the NDD happening. We do not have detailed data on the number of children reached in states that launched programs for NDD, do not expect those numbers to be as reliable as the ones for which Deworm the World has run its coverage validation process (discussed below), and we are unsure how to credit Deworm the World for its role in causing those deworming programs to happen.
  • 80% figure: total 36.0 million enrolled children reported dewormed by government, divided by 44.9 million children enrolled statewide (See “Estimates of total statewide coverage” table in main text.)

• 274
  • For example, the nodal headmasters add the school totals from their respective jurisdictions and report them to the block-level officials, who use those to calculate the total children dewormed in their block and report that figure to the district, who do the same in reporting to the state level, where the grand total is calculated.
  • Bihar 2015: "With close support from our teams, the State Health Society Bihar and Bihar Education Project Council collected and compiled the coverage report for the round within the reporting timelines in the prescribed reporting format (Annexure H.1). Coverage reporting structure and timeline is shown below in Figure 4:" (see source for annexure and figure) DtWI Bihar 2015 Program report, Pg 16.
  • Delhi 2015: "In this round, each school and anganwadi was supposed to fill a one-page reporting form (annexure III). In order to improve the accuracy of coverage reporting by the schools and anganwadis, every participating school and anganwadi was instructed to follow a recording protocol for deworming. Every teacher and anganwadi worker was required to put a single tick (√) next to a child’s name in the school/anganwadi register if they were administered albendazole on deworming day and double-tick mark (√√) if dewormed on mop up day. School headmasters and anganwadi workers were responsible to compile the number of dewormed children, fill the reporting format and submit it to the next level. Reporting structure of coverage data from schools and anganwadis and timelines are given in the below flow chart:" (see source for chart) DtWI Delhi 2015 program report, Pg 15.
  • Rajasthan 2013: "School headmasters were required to hand over the completed reporting form to their respective nodal headmasters – a senior headmaster of a school looking after a cluster of schools – who in turn would submit these forms to the respective BEO office. Apart from the forms, the nodal headmasters also submitted the collated information. This information further gets consolidated first at block level, then at district level and finally at state level. Education department shared this compiled information with all the stakeholders." DtWI Rajasthan 2013 program report, Pg 14.
  • Bihar 2014 was an exception; aggregation of data for that round of treatment was done centrally, by an independent firm. "In Round 3, each school was supposed to fill a one-page, simple school summary form (Form S), capturing only the essential details on the school such as total enrollment, total number of dewormed children by date and by enrollment status, number of adults dewormed, availability of drugs, drug usage and wastage, remaining drug stocks and contact details of the headmaster. In order to ensure that coverage reporting by the schools is accurate, every participating school was instructed to follow a special recording protocol for deworming. Every teacher was required to put a single tick mark (9) next to a child’s name in the attendance register if they had consumed the tablet on Deworming Day. The teachers were instructed to put a double-tick mark (99) next to a child’s name if s/he had been administered the tablet on Mop-Up Day. These tick marks are intended to be the basis for the numbers reported by every school in the S forms. Schools were supposed to provide the number of enrolled children dewormed by counting the single and double tick marks in the attendance registers. In addition, the provision for dewormed non-enrolled children was to be maintained along with the details of adults dewormed. School headmasters were supposed to submit the filled summary form to BRP by January 30 2014. Blocks were to submit all the collected forms, without any consolidation or compilation, at DEO office by February 5 2014. Districts were instructed to submit these forms at BEPC by February 20 2014. [...] Simplification of consolidation process by replacing data consolidation at block and district level with only collection of forms and submission at higher level. [...] Further, DtWI changed its prior strategy of relying on a single and small data entry partner, and selected a reputed agency with significant experience in large scale surveys across India to do data entry. This data entry partner subsequently dedicated significant data entry resources to Bihar form entry." DtWI Bihar 2014 program report, Pgs 26-27

• 275

  Note that a slightly larger sample of schools is surveyed for coverage validation than for Deworming Day and Mop-Up Day.

• 276
  • Monitors find that the number of students dewormed according to school records was greater than the number according to those schools' classroom records for each state that Deworm the World supported in 2015; the median over-report amount was 15%.
    • Process: For each state for which we have 2015 deworming coverage validation survey data, we calculated a state-level inflation rate according to the surveyors' observations. We calculated inflation rate as the difference between reported treatments and recorded treatments, out of the total recorded treatments: (treatments reported on the school reporting form​ minus treatments recorded as tick marks in class records)/(treatments recorded as tick marks in class records). In the raw dataset, some entries are marked as missing data using error codes described in the associated code sheet; these were treated as contributing zero tickmarks or zero reported treatments (as appropriate) to the total count.
    • Bihar: 22% inflation rate: (141,377-115,889)/115,889 DtWI Bihar 2015 monitoring data for coverage validation, schools (Deworm the World also reports this figure as 22%: "The state level verification factor for Bihar was found to be 0.81972, indicating that for every 82 enrolled children who were recorded as deworming in the schools, the school reported that 100 enrolled children had been dewormed. This corresponds to an overall 22% inflation of reporting in the state, meaning that reported numbers appear to be approximately 22% higher than the numbers recorded in attendance registers." DtWI Bihar 2015 Program report, Pg 19)
    • Rajasthan: 15.0% inflation rate: (42,845-37,256)/37,256 DtWI Rajasthan 2015 monitoring data for coverage validation, schools
    • Delhi: 1.9% inflation rate: (170,060-166,956)/166,956 DtWI Delhi 2015 monitoring data for coverage validation, schools
    • Madhya Pradesh: 11.3% inflation rate: (39,983-35,912)/35,912 DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools
    • Chhattisgarh: 100.0% inflation rate: (63,162-31,582)/31,582 DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools
  • This over-reporting results from schools that reported deworming students in the school summary but did not mark any classroom records as the program protocol instructed. After excluding schools in which no classroom records were marked, the number of students dewormed according to the school records was similar to those schools' classroom records for the same states (the median over-report amount was 0.4%).
    • Process: For each state for which we have 2015 deworming coverage validation survey data, we calculated a state-level inflation rate ​excluding schools at which no tick marks were recorded​ according to the surveyors' observations. We calculated inflation rate as the difference between reported treatments and recorded treatments, out of the total recorded treatments: (treatments reported on the school reporting form, in schools with any tick marks recorded in any class records​ minus treatments recorded as tick marks in class records)/(treatments recorded as tick marks in class records). In the raw dataset, some entries are marked as missing data using error codes described in the associated code sheet; these were treated as contributing zero tickmarks or zero reported treatments (as appropriate) to the total count.
    • Bihar: -2.8% adjusted inflation rate: (112,653-115,889)/115,889. DtWI Bihar 2015 monitoring data for coverage validation, schools
    • Rajasthan: 2.9% adjusted inflation rate: (38,318-37,256)/37,256 DtWI Rajasthan 2015 monitoring data for coverage validation, schools
    • Delhi: 1.7% adjusted inflation rate: (169,748-166,956)/166,956 DtWI Delhi 2015 monitoring data for coverage validation, schools
    • Madhya Pradesh: -5.8% adjusted inflation rate: (33,844-35,912)/35,912 DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools
    • Chhattisgarh: 0.4% adjusted inflation rate: (31,694-31,582)/31,582 DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools

• 277
  • In Bihar's 2015 program, 99.8% of schools reported deworming data, so we believe minimal school-level data was missing from the aggregation process. DtWI Bihar 2015 Program report, Pg 4 (Table 1)
  • In Rajasthan's 2015 program, 94.0% of schools reported deworming coverage, so it seems possible that any deworming conducted in the remaining 6% of schools was left out of the reported coverage. @DtWI Rajasthan 2015 Program report@, Pg 4 (Table 1)
  • In Delhi's 2015 program, 100% of schools and AWCs [anganwadi child-care centres] reported deworming. @DtWI Delhi 2015 Program report@, Pg 4 (Table 1)
  • In Madhya Pradesh's 2015 program, 100% of schools and AWCs reported deworming. @DtWI Madhya Pradesh 2015 Program report@, Pg 5 (Table 1)
  • In Bihar’s 2014 program, roughly 10% of schools’ summary reports were missing ("In total, 64,724 schools out of the 70,675 targeted schools submitted their summary forms."), and it sounds as if these were simply left out of the reported coverage data ("The result of this modified data cleaning and data entry process was that the coverage data was available to share from the 64,724 schools within 3 months of the deworming date. This cleaned data indicated that 16,225,546 children were dewormed in Bihar out of which 15,489,334 were enrolled children and 736,212 were non-enrolled children.") DtWI Bihar 2014 program report, Pg. 27.
  • In Delhi’s 2013 program, Deworm the World reports that missing reports were excluded from the reported coverage figures: "The program targeted 3,032 schools and 10,500anganwadis. As on the cutoff date for report collection, 15 December 2013, data from 603 schools was pending. The above data is based on a dataset comprising 2,417 schools and 10,591 anganwadis." DtWI Delhi 2013 program report, Pg. 2.
  • We have not seen information about missing data for Rajasthan’s 2013 program.

• 278
  • Methodology:
    • Schools included: all schools monitors visited during coverage validation day, except those for which no classes records showed any dewormings (because we believe many schools just didn’t use that protocol despite deworming students) unless the principal said that no deworming had taken place in the school. We also excluded schools with missing total enrollment data.
    • Students dewormed in included schools: total ‘tick marks’ in school register for all classes in included schools. Note that we are unsure whether this measure sometimes includes unenrolled students as well.
    • Enrollment in included schools: total enrollment as reported by the monitor's check of the attendance register.
    • Total students enrolled in each state (public and private): Sum of primary, upper primary, secondary, and higher secondary enrollment figures for the 2014-2015 school year from India’s District Information System for Education (DISE). Primary and upper primary total enrollment: U-DISE Elementary Thematic Maps 2015, Pgs 60-61. Secondary and upper secondary total enrollment: 2014-15 totals U-DISE Secondary Flash Statistics 2015, Pg 34.
    • Total students enrolled in government and government-aided schools: Sum of (total enrollment * percentage of enrollment in government and government-aided schools) for primary, upper primary, secondary, and upper secondary schools. Percentage of students enrolled in government and government-aided schools by state: U-DISE Elementary Thematic Maps 2015, Pgs 62-63 and U-DISE Secondary Thematic Maps 2015, Pgs 33-34.
      • U-DISE Secondary Thematic Maps 2015, Pg 34 reports the percentage of students who are enrolled in private unaided managements. We are assuming that the remainder of students are in government or government-aided schools. U-DISE Elementary Thematic Maps 2015, Pgs 62-63 reports the percentage of students enrolled in "government management schools." We are uncertain whether "government management schools" includes government-aided private schools, but we are assuming so for these figures.
      • According to the table of contents, the map on Pg 33 of U-DISE Secondary Thematic Maps 2015 reports data on "Percentage of Secondary Enrollment by Private Unaided Management." However, the title of the map on Pg 33 is "Percentage of Professionally Qualified Teachers: Secondary Level." Based on the context in which the map appears, and because there is another map in the document with the title "Percentage of Professionally Qualified Teachers: Secondary Level" our best guess is that the table of contents is correct and the map title on Pg. 33 is an error.
    • Estimated statewide enrolled students dewormed: (Dewormed enrolled students / Enrolled students) * Statewide enrollment in government and government-aided schools.
  • Numbers
    • Bihar (DtWI Bihar 2015 monitoring data for coverage validation, schools)
      • Included schools: 557/748 schools surveyed: 234 schools without tick marks, but in 43 of those those the principal said no deworming happened; 748 - 234 + 43 = 557.
      • Enrolled students dewormed in sample: 115,815
      • Enrollment in sample: 177,464
      • Statewide enrollment in government and government-aided schools: 23,902,897
      • Estimated statewide enrolled students dewormed: 15,599,299
    • Rajasthan (DtWI Rajasthan 2015 monitoring data for coverage validation, schools)
      • Excluded schools: 46
      • Enrolled students dewormed in sample: 37256
      • Enrollment in sample: 44316
      • Statewide enrollment in government and government-aided schools: 7,289,229
      • Estimated statewide enrolled students dewormed: 6,180,557
    • Delhi (DtWI Delhi 2015 monitoring data for coverage validation, schools)
      • Note that Delhi didn’t deworm students in grade 11 due to examinations. We didn’t adjust any of the numbers from the raw data because it appears that grade 11 classes were still sampled by monitors, so the monitored coverage should still reflect the low participation from that grade.
        • "Due to examinations of 11 grade in April, the SHS decided to cover these children later in the year." DtWI Delhi 2015 program report, Pg 4
        • In coverage validation data, some of the reports are from class 11. DtWI Delhi 2015 monitoring data for coverage validation, schools
      • Excluded schools: 3
      • Enrolled students dewormed in sample: 166956
      • Enrollment in sample: 218098
      • Statewide enrollment in government and government-aided schools: 2,564,953
      • Estimated statewide enrolled students dewormed: 1,963,495
    • Madhya Pradesh (DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools)
      • Excluded schools: 178
      • Enrolled students dewormed in sample: 35834
      • Enrollment in sample: 52472
      • Statewide enrollment in government and government-aided schools: 11,151,527
      • Estimated statewide enrolled students dewormed: 7,615,563
    • Chhattisgarh (DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools)
      • Excluded schools: 247
      • Enrolled students dewormed in sample: 31535
      • Enrollment in sample: 40575
      • Statewide enrollment in government and government-aided schools: 4,644,179
      • Only students 10-19 were dewormed so we are not able to generate a similar estimate of students dewormed in this state.
  • Our calculations in this spreadsheet: GiveWell enrollment-based student coverage check 2015

• 279
  • Chhattisgarh is excluded because only some districts in the state were covered, so we are not able to apply the same methodology.
  • The calculations for the first three columns of this table are in this spreadsheet: GiveWell enrollment-based student coverage check 2015
  • Reported enrolled student coverage
    • Bihar: 17,600,122 ("Number of enrolled children dewormed (age 6-19 years)") DtWI Bihar 2015 Program report, Pg 4
    • Rajasthan: 6,463,898 ("Total enrolled children (6-19 years) dewormed at schools") DtWI Rajasthan 2015 program report, Pg 4
    • Delhi: "The coverage data from the schools in Delhi indicated that 1,828,562 enrolled children were dewormed in the state during deworming day and mop up day against the total target of 2,240,573 enrolled children from class 1 to 12." DtWI Delhi 2015 program report, Pg 18.
    • Madhya Pradesh: 10,073,830 ("Number of enrolled children (Class 1 to 12) dewormed at schools") DtWI Madhya Pradesh 2015 program report, Pg 5
    • Chhattisgarh: "The coverage data from the state indicated that 916,596 children in the age group 10-19 years were dewormed against the target of 978,008. These include 849,797 enrolled children at schools and 128,211 out-of-school children." DtWI Chhattisgarh 2015 coverage validation report, Pg 7

• 280

  Deworm the World, Kenya Coverage Reporting data, Year 3

• 281

  "Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17.

• 282

  The statement could mean that 99% of students present on Deworming Day were observed to swallow pills at schools where Deworming was systematically occurring. It might also mean that 99% of students enrolled at schools were marked as having received a deworming pill on either Deworming Day or Mop-Up Day. There are other plausible definitions, and the report does not make it clear which is being used.

• 283

  Deworm the World tends to focus on treating STH, but the deworming programs we reviewed focused primarily on schistosomiasis.

• 284
  • Details of the Kenya studies (Miguel and Kremer 2004, Baird et al 2012) in our Reanalysis of the Miguel and Kremer deworming experiment page.
  • Details of the Uganda study (Croke 2014) here.

• 285

  Deworm the World notes: "...baseline STH prevalence has been vastly different across geographies even within India. Keep in mind that WHO recommends mass deworming in areas where prevalence exceeds 20% - we would consider any prevalence above 20% to therefore not be 'low.' And above 50% prevalence is high." Grace Hollister, email to GiveWell, June 9, 2016

• 286

  "…absent from most of the country, risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1.

  "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3.

• 287

  Notes on the data we have included:

  • Baseline prevalence data for populations studied in Miguel and Kremer 2004 and Croke 2014 are provided in this table for comparison, as these papers represent what we believe to be the strongest pieces of evidence of long term damages from intestinal parasites (see discussion in this section).
  • The table uses Miguel and Kremer 2004 definitions of moderate intensity, which are not the same epg [eggs per gram] levels as WHO definitions for all worms. "Following Brooker, Miguel, et al. (2000), thresholds for moderate infection are 250 epg for Schistosomiasis. mansoni and 5,000 epg for Roundworm, the WHO standards, and 750 epg for Hookworm and 400 epg for Whipworm, both somewhat lower than the WHO standard." Miguel and Kremer 2004, Pg. 167.
  • Bihar, Madhya Pradesh, Rajasthan, and Uttar Pradesh schistosome prevalence was not measured by Deworm the World. We assume 0% based on World Schistosomiasis Risk Chart 2012.
  • We also have worm infection data from Delhi, but do not include it here as Deworm the World has told us that Delhi had lower prevalence than a region it would normally prioritize for mass school-based deworming, but that it has particular political and cultural significance as the capital of India. Additionally, the deworming round in 2013 is the last for which Deworm the World is expecting to assist Delhi significantly. See DtWI Delhi 2012 prevalence survey report.
  • We have seen more recent prevalence data for Bihar and Kenya (discussed here). We do not include that data here because it is unclear how it should affect our expectation of the cost-effectiveness of deworming in Bihar and Kenya. Intuitively, we would expect that, since the prevalence has dropped, the benefits of future deworming will be lower than earlier rounds of deworming. However, our cost-effectiveness analysis is generally not this precise, and (roughly) aims to estimate the average cost-effectiveness over an entire program.

  Sources and comments for the data:

  • Miguel and Kremer 2004
    • Miguel and Kremer 2004, Table II, Pg 168.
    • We provide data from year one of the Miguel and Kremer study, which is the time period before El Niño weather conditions set in and increased the prevalence and intensity of local infections.
  • Croke 2014:
    • Kabatereine et al. 2001, Pg 284, Table 1, Average for eastern districts.
    • Kabatereine et al. 2001 was a disease mapping study conducted in southern Uganda in 1998, two years before the study underlying Croke 2014 began, i.e. these are not results from the individuals enrolled in the study underlying Croke 2014.
    • Note that the districts in the eastern district average in Kabatereine et al. 2001 are not an exact match for the districts included in Croke 2014, though four of the five districts included in Croke 2014 (Busia, Iganga, Mbale, and Tororo) are included in the Eastern district average.
    • The districts for Croke 2014 were selected because Kabatereine et al. 2001 had found high worm prevalence in eastern districts: "Five districts in the eastern region of Uganda were selected (Busia, Iganga, Mbale, Palissa, and Tororo) because a survey had indicated that about 60% of children aged 5-10 years were infected with intestinal nematodes, most commonly hookworm." Alderman et al. 2006, pg 1.
  • Bihar
    • DtWI Bihar 2011 prevalence survey report
    • There is a pre-existing LF program that provides albendazole to a large portion of the population in Bihar annually.
    • Based on the timing of the decision to deworm in Bihar, there was only time to collect prevalence surveys from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet Deworm the World's typical standard of quality.
    • Before the Deworm the World-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report, Pg 9.
    • Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness. (Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013)
  • Rajasthan
    • DtWI Rajasthan 2013 prevalence survey report
    • The results presented here are from Deworm the World's second prevalence survey, one year after the first statewide round of deworming. Deworm the World conducted this prevalence survey in September 2013; Deworm the World believes it is higher quality and addresses the issues (for example, with measuring hookworm prevalence) in the initial survey. GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
    • The first prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence was expected to be significant in Rajasthan. DtWI Rajasthan 2012 prevalence survey report, Pgs 15-16.
  • Madhya Pradesh
    • DtWI Madhya Pradesh 2015 program report, Pg 37
    • We have not seen a detailed prevalence survey report or protocol for Madhya Pradesh.
  • Uttar Pradesh
    • Deworm the World 2015 Uttar Pradesh prevalence survey report
    • There is a pre-existing LF program that provides albendazole to a large portion of the population in Uttar Pradesh annually; this program implemented an MDA in March 2015. Deworm the World 2015 Uttar Pradesh prevalence survey report, Pg 9
    • Districts without LF treatment programs were surveyed in May 2015, while districts with LF treatment programs were not surveyed until July and August. Deworm the World 2015 Uttar Pradesh prevalence survey report, Pg 9
  • Kenya
    • KEMRI prevalence report - Year 2, Pgs 4, 10, and 15.
    • It is not clear to us what definition was used for medium and high intensity infections.
    • For schistosomiasis, we used the measures of S. Mansoni, which was measured in the full sample of 200 schools using the Kato Katz method. In 9 schools, S. haematobin was also measured via a urine test, and found a much higher prevalence of 18.0% at baseline (compared to 1.8% for S. Mansoni). KEMRI prevalence report - Year 2 Pgs 15 and 18, and Mwandawiro et al. 2013 Pgs 3-4.

• 288

  Deworm the World noted that the KEMRI surveys in Kenya are designed for "impact assessment", while the surveys in India are designed for "mapping" - as far as we can tell, this simply means that the schools in each survey might be selected differently (e.g., in a "mapping" survey, schools will be selected in part based on which agro-climatic region they are in, whereas in an "impact assessment" survey, schools might be selected randomly or according to some other criteria). Additionally, Deworm the World noted that the surveys in India are looking at a much larger population than the Miguel and Kremer 2004 and Croke 2014 studies examined. We do not believe that these details make a comparison unuseful. Grace Hollister, conversation with GiveWell, June 13, 2016

• 289

  "When prevalence of any STH infection is less than 20%, large-scale preventive chemotherapy interventions are not recommended. Affected individuals should be dealt with on a
  case-by-case basis." Preventive chemotherapy in human helminthiasis 2006, Pg 41.

• 290
  • WHO, Helminth control in school-age children, Pg 85
  • "Based on the findings of the prevalence survey and WHO guidelines, Evidence Action recommends an annual school based deworming program for school-age children in the state. [...] Given the pre-existing deworming treatments described above, this prevalence survey cannot be considered a baseline survey to assess STH infection rates in a treated population, to determine an optimal treatment strategy." DtWI Madhya Pradesh 2015 program report, Pg 38.
  • "Our recommendation is explained in the prevalence survey report. The prevalence and intensity rates from the survey are not "baseline" data, given that there has been relatively regular administration of albendazole in MP through the BSM program that treated PSAC since 2005, and the LF program which provided community-wide treatment of 11 districts of MP (the number of endemic districts had fallen to 8 by 2014). As a result, these deworming efforts have likely had an impact on STH prevalence and MP could not be considered an untreated baseline population. We therefore did not apply the WHO guidelines for baseline STH prevalence.
    Annexure 10 of the WHO guidelines suggest continuing annual treatment for populations which have received deworming for several years, and prevalence is still greater than 10%. In addition, the high rate of open defecation in the state, and the planned ending of the LF program in MP, increased the risk of infection and a potential resurgence in prevalence." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Grace Hollister, email to GiveWell, June 9, 2016

• 291
  • Bihar: "In this context, biannual treatment is recommended throughout Bihar." DtWI Bihar 2011 prevalence survey report, Pg 5.
  • Rajasthan: "WHO guidelines recommend that an annual mass deworming program be conducted in areas where prevalence falls in the range of 20% to 50% (WHO, 2011). Since the prevalence of STH in Rajasthan falls within this range, it is recommended that Rajasthan continue conduct an annual mass deworming program in the state." DtWI Rajasthan 2013 prevalence survey report, Pg 7.
  • Madhya Pradesh: "Based on the findings of the prevalence survey and WHO guidelines, Evidence Action recommends an annual school based deworming program for school-age children in the state." DtWI Madhya Pradesh 2015 program report, Pg 38.
  • Uttar Pradesh: "On the basis of this study, Evidence Action makes the following recommendations to the Government of Uttar Pradesh: 1. Implementation of biannual deworming for school-age and preschool-age children in all districts of the state, given the high prevalence observed in the state in the school-age population." Deworm the World 2015 Uttar Pradesh prevalence survey report, Pg 26
  • Kenya: "From the year 2012, the ministries of health and of education of Kenya plan to deworm all school –age children who live in 66 districts identified as having a high prevalence of soil-transmitted helminth (STH) infection and schistosomiasis in four provinces." KEMRI prevalence report - Year 2, Pg. 2

• 292

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 293

  We are under the impression Deworm the World took into account existing data or partners' data when determining which areas to target. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 294
  • For schistosomiasis, mapping data was given in the form of number of districts with schistosomiasis prevalence ≥50%, ≥10% and <50%, and ≥1% and <10%. As an estimate, we assumed that these ranges correspond to average prevalences of the midpoint of the range: 75%, 30%, and 5.5%, respectively.
  • For STH, mapping data was given in the form of number of districts with STH prevalence ≥50%, ≥20% and <50%, and ≥1% and <20%. As an estimate, we assumed that these ranges correspond to average prevalences of 75%, 35%, and 10.5%, respectively.
  • We assumed that treatment would be provided in all areas, including low risk STH areas. The mapping report notes that WHO does not recommend MDA for STH in low risk areas (Pg 9), but it is our understanding that SCI generally treats for STH wherever it treats for schistosomiasis, which may include some districts at low risk for STH.
  • We have assumed that no child has more than one type of STH and that prevalence of each type of STH is the same. This is unlikely to be accurate, but serves as a lower bound.

  SCI Ethiopia mapping of SCH and STH 2014.

• 295

  See our cost-effectiveness model, Sheet 'Intensity of worms.'

• 296
  • For example, India has such a program: The National Vector Borne Disease Control Programme LF treatment coverage 2015
  • In Kenya, the LF program is housed within the country’s neglected tropical disease (NTD) unit, which has asked Deworm the World if it might support its process monitoring and coverage validation (PMCV) operations for LF. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • Deworm the World has told us that in both countries (Kenya and India), LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 297
  • "DEC [Diethylcarbamazine] + Albendazole in selected distt & DEC in other distt" LF treatment drugs 2012.
  • "Albendazole, the same drug used to treat STH, is usually used to treat LF." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Assam 2010 guidelines for deworming recommends 400mg of albendazole for children being dewormed, and Global Alliance to Eliminate Lymphatic Filariasis - Prevention also recommends 400mg doses.

• 298
  • The intended treatment for LF in India is annual administration of DEC [Diethylcarbamazine] or DEC and Albendazole.
    • "Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
      Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except:
      - children below 2 years
      - pregnant women
      - seriously ill persons
      (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012.
  • The WHO recommends annual albendazole treatment for areas with 20-50% prevalence of STH.
    • Preventive chemotherapy in human helminthiasis 2006, Pg 41.
  • According to the Indian government's program website, the LF treatment program has high coverage across many states (e.g., typically 80-90% in states carrying out the program), including Bihar, Madhya Pradesh, Uttar Pradesh, and Chhattisgarh.
    The LF program has recently been active in Bihar, Madhya Pradesh, Uttar Pradesh, and Chhattisgarh, but not Rajasthan or Delhi.
    • LF treatment coverage 2015
    • "neither Rajasthan nor Delhi are endemic for LF; no LF MDA is necessary" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Deworm the World believes that LF treatments in Bihar and other states have not been delivered as effectively as planned; we do not have additional information about LF treatment delivery in other states beyond limited publicly available data.
    • "The Bihar program was originally designed so that one round of albendazole administration would come from community-based lymphatic filariasis (LF) treatment (LF is endemic throughout the state) and a second round would come through the school-based deworming program. For a variety of reasons, LF treatment has not been consistently implemented. It is possible that more consistent albendazole administration as part of LF treatment would have resulted in a more significant drop in STH prevalence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 2.
    • "Kenya and India are the countries where Deworm the World operates two of its biggest programs and where Ms. Hollister has the greatest familiarity with the state of LF treatment. In both countries, LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015, Pg 1
    • "Deworm the World believes that LF treatments in Bihar and other states have not been delivered as effectively as planned" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
    • Data on reported coverage of LF treatment programs here: LF treatment coverage 2015

• 299

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 300
  • Or from twice-per-year to thrice-per-year.
  • Note that community-based treatment, such as is typically used for LF, involves enlisting several people to travel from house to house to administer treatment, making it much more time-consuming and costly than school-based programs. Because its goal is to treat every person in a community, multiple trips to a single area may be required to ensure total coverage (e.g., if a household member is not at home during the first visit). GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • Deworm the World also claims that it generally knows where there is overlap between areas that are endemic for STH or schistosomiasis and areas that are endemic for LF. For example, in Kenya, only the coastal area is LF-endemic, but worm infections are more widespread, so LF treatment efforts involve a smaller number of subcounties. Community-based treatment programs might be cost-effective in places endemic with many NTDs but might not be in places only endemic for STH or schistosomiasis. Deworm the World has told us that many places do not have much overlap between different NTDs so school-based deworming programs can provide a cost-effective alternative to community-based treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 301

  GiveWell analysis of Deworm the World cost per treatment

• 302

  See our 2014 review.

• 303
  • "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2.
  • "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.
  • "National funds for deworming are part of the [Weekly Iron Fortification Supplements] WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12.

• 304

  Excluding estimates of the costs of teacher and government staff time, Deworm the World bears approximately 90% of costs in Kenya. See GiveWell analysis of Deworm the World cost-per-treatment, 2016.

• 305
  • "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
  • Deworm the World noted that the costs in Kenya are unusually high (partly due to higher quality M&E): "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
  • Our understanding is that costs are higher in Kenya than India primarily due to cultural differences, including paying significantly higher allowances to teachers for participating in the program. For more details, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell analysis of Deworm the World cost per treatment.

• 306

  Evidence Action Q1 financials, 2016

• 307

  Evidence Action Q1 financials, 2016

• 308

  Deworm the World budget - 2016

• 309

  Grace Hollister, email to GiveWell, June 9, 2016

• 310

  Evidence Action Q1 financials, 2016

• 311
  • The $2.5 million in unrestricted funding is allocated as follows: $1.5 million for reserves, $268,223 for Evidence Action's 2016 allocation priorities, and the remaining $683,837 ($952,060 - $268,223 = $638,837) for "organizational development and/or reserves allocation in 2016." Evidence Action Q1 financials, 2016 and Grace Hollister, conversation with GiveWell, June 13, 2016
  • Evidence Action Q1 financials, 2016, which notes for the $268,223 raised in Q1 2016 that "2016 Evidence Action Unrestricted funds will be allocated in accordance with the 'waterfall' chart previously shared."
  • See Evidence Action, Projected allocation of unrestricted funds, 2016 for Evidence Action's plans for 2016 unrestricted funds. This indicates that the first $600,00 raised will be used to fund Dispensers for Safe Water.

• 312

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 313

  Grace Hollister, conversation with GiveWell, April 20, 2016

• 314
  • Deworm the World told us in late 2015 that it aimed to have reserve funds equivalent to three months of operating expenses.
    • Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
    • Note that in past years, Deworm the World has informally aimed to have reserves of ~10% of its budget.
      • "DtWI now projects a budget of approximately $8 million for DtWI in 2014, and is
        expecting to reach its target of holding cash reserves equivalent to 10% of its annual
        expenses." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 1
      • For example, Evidence Action 2014 budget shows "DtWI target unrestricted funds, working capital" as 10% of expenses.
  • In April 2016, Deworm the World told us that it was considering a 6-month reserves target, but that its reserves policy still needed to be finalized. It also noted that it is unsure what the future process will be for programs to access Evidence Action's reserves. Grace Hollister, conversation with GiveWell, May 5, 2016
  • We have not discussed this reserves target in depth with Deworm the World, given that it is not finalized, so we are unsure (a) about the circumstances under which Deworm the World would make use of those reserves and how much of a reserves budget it might need, given that its budget is largely supported by multi-year restricted grants, (b) why it changed its reserve target from previous years, and (c) whether or not it is currently hitting its reserves target. Additionally, it is our understanding that Evidence Action intends to allocate a large portion of its unrestricted funding to reserves in 2016. We are not sure if Deworm the World needs any reserves in addition to Evidence Action's reserves, but our guess is no.

• 315

  Note that this post details more than $1.2 million in spending and allocated spending. Our impression is that Deworm the World has not spent all of the funds that this post describes as allocated (see below).

• 316

  See the blog post linked above. Additionally, Deworm the World sent us details regarding Good Ventures' 2013-2014 giving season grant to Deworm the World. Deworm the World's budget for the grant approximately matched what it ultimately spent the grant on; the primary deviation was that Deworm the World spent less in Vietnam than anticipated and more on exploration of other new country opportunities. However, Deworm the World's budget for the grant did not completely match what we had discussed unrestricted funding would be used for; the main difference was that Deworm the World budgeted funding for studies of how to break worm transmission. DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015

• 317

  See our November 2014 review:

  "The Deworm the World Initiative seeks an additional $1.3 million to support its activities in 2015 and 2016. DtWI expects to spend $377,000 of the $1.3 million (29%) it seeks on work related to expanding school-based mass deworming programs and funding related operating expenses (including impact evaluation related expenses). More specifically, these activities would be:

  • $230,000: staff to support expansion in India, new countries, and related operating and evaluation expenses.
  • $144,000: DtWI overhead. These funds support DtWI as an organization but are not directly programmed (e.g., a portion of Alix Zwane's, the Executive Director of Evidence Action salary, Evidence Action financial staff, etc.).
  • $170,000: implementation support for the integrated deworming, sanitation and hygiene education program in Vietnam, in partnership with Thrive Networks.
  • $500,000: evaluation of new evidence-based programs that leverage deworming. We have limited detail about what this would entail. One idea that DtWI has investigated is the possibility of distributing bednets along with deworming pills in schools as an alternative distribution mechanism to national net distributions. Another is including hand-washing educational programming alongside deworming days. This line item includes $50,000 to support DtWI's evaluation of its hygiene and deworming program funded by Dubai Cares and $50,000 to enable DtWI to hire a senior epidemiologist.
  • $230,000: staff to support evaluation of DtWI’s work in Kenya. This work is primarily funded by CIFF. DtWI believes that additional resources can improved significantly the quality of the analysis done regarding the cost effectiveness of breaking transmission."

• 318
  • Deworm the World, 2015 expense summary - by funder. The following funders are "unrestricted": "Good Ventures 2014", "Retail Fund Raising", "Anonymous", and "Marshall". Unfortunately, the expense summary does not have enough detail for us to know, for example, whether or not Deworm the World spent any funding on "evaluations of new evidence-based programs that leverage deworming" (which was one of the activities it told us it would use additional unrestricted funding for at the end of 2014).
  • Deworm the World had told us it would spend $170,000 in unrestricted funding on Vietnam if it met its funding goals (GiveWell-influenced donors alone nearly fulfilled its funding goals), but instead it only spent $32,613.
  • Deworm the World planned to only spend $230,000 in unrestricted funding on expansion in India and other countries, but it spent $293,915 on expansion in India alone.
  • Deworm the World intended to use $144,000 in unrestricted funding for overhead. Instead, it spent $289,375 on overhead (i.e., it's "Global" category).

• 319

  Deworm the World told us that it had allocated unrestricted donations in the following way:

  • $200,000: A prevalence survey in Pakistan
  • $111,000: Support for the TUMIKIA and TakeUp studies
  • $93,000: Support for a deworming program in Vietnam
  • $12,000: A training materials project in India
  • $100,000: Salaries for staff exploring work in new countries (intended to extend to 2016)
  • $~550,000: Support for a program in Cross River, Nigeria.

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 320

  For example:

  • Only $111,000 was allocated to the TUMIKIA and TakeUp studies, but Deworm the World had told us it would spend $500,000 in unrestricted funding on evaluations if it met its funding goals (GiveWell-influenced donors alone nearly fulfilled its funding goals).
  • $750,000 was allocated to activities in Pakistan, Nigeria, and exploratory work in other new countries, but Deworm the World had not included Pakistan or Nigeria in its original plan for unrestricted funding. It had originally only expected to spend only a portion of $230,000 on "staff to support expansion in … new countries" ( See our November 2014 review.)

• 321

  See our November 2015 review. Deworm the World intended to spend unrestricted funding on:

  • A deworming program in Pakistan (~$8.4 million for a 3-year program: $2 million per year in Punjab and $0.8 million per year in Sindh)
  • A deworming program in a new country (~$6 million for a 3-year program)
  • A deworming program in Vietnam (~$2.6 million, unclear how many years)
  • Reserves ($2.8 million)

• 322
  • Deworm the World has told us that the speed with which programs can scale depends on the government's structure and preferences. A government might prefer not to start with a full-scale program, and in some places, operations are more decentralized, which makes it more practical to start with only a few regions or districts. Ethiopia, for instance, is highly decentralized, so only a few regions participated in the first round of mass drug administration. The scale at a program’s inception may also be affected by the availability of drugs. If drugs for treatment are already available, but the mechanism to treat the whole country has not yet been established, it makes sense to proceed with deworming immediately in a few areas and try to get the entire country on the same schedule later. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • Grace Hollister, email to GiveWell, June 9, 2016

• 323

  Deworm the World budget - 2016

• 324

  In late 2015, Deworm the World had discussed spending up to $8.4 million in Pakistan over the next three years. $8.4 million / 3 = $2.8 million. We expect that Deworm the World will spend more in subsequent years as the program in Pakistan scales.

• 325
  • Our estimate of Deworm the World’s unrestricted funds over the past two years:
    • Pre-GiveWell funds: Deworm the World had at least $0.4 million in unrestricted funding at the end of 2013. Deworm the World previously existed as a 501(c)3 and had an account that still held funds. As of October 2014, there was about $0.4 million available. Evidence Action 2015 draft budget. We are unsure if there were additional funds in that account at the end of 2013 that were spent down by October 2014.
    • GiveWell-influenced funds: Since 2013, GiveWell has tracked approximately $3.7 million in donations to Deworm the World due to GiveWell’s influence. ($2.1 million in 2013, $1.1 million in 2014 (see our impact), and approximately $0.5 million in 2015 (as of October 2015, according to our internal records)).
    • Recent funds not influenced by GiveWell: In 2014, Evidence Action received $1.7 million in unrestricted, retail donations not attributed to GiveWell, of which roughly 10% was designated for Deworm the World. In 2015, Evidence Action estimates that it is on track to receive about $3 million in unrestricted, retail donations including donations influenced by GiveWell. Since GiveWell has tracked $0.5 million in donations influenced by GiveWell in 2015, we estimate a total of roughly $4.2 in non-GiveWell influenced donations in 2014 and 2015. We estimate that 10% of that total is restricted to Deworm the World based on that ratio from 2014, or about $0.4 million. (GiveWell asked: "Additional insight, if any, into the source of the $1.7M retail donors to Evidence Action last year. Does Evidence Action expect similar donations this year?"
      Deworm the World answered: "Major donors and small online donors. 90% or so of that was not designated to Deworm the World. We believe we're on track to receive ~$3m this year.") Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015"
    • Cash on hand: As of October 2015, Deworm the World told us it had available funding of about $3.3 million. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015. We are not including the $0.2 million in receivables in this figure.
    • Allocations of cash on hand: Deworm the World has allocated $1.1 million of unrestricted funding to future activities, including support intended for new programs in Nigeria and Pakistan. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Summary of total unrestricted funds, spent funds, and allocated funds over the past two years:
    • Estimated total unrestricted funds over the past two years = $0.4M + $3.7M + $0.4M = $4.5M
    • Estimated total spent unrestricted funds over the past two years = $4.5M - $3.3M = $1.2M
    • Total allocated unrestricted funds = $1.1M

• 326

  Deworm the World had allocated $1.1 million of unrestricted funding to future activities, including support intended for new programs in Nigeria, Pakistan, and Vietnam.

  • Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Our understanding of the $1.1 million allocation breakdown is:
    1. Cross River, Nigeria: $0.55 million (Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015)
    2. Pakistan prevalence survey: $0.2 million (Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015)
    3. Staff salaries in 2016: $0.1 million (Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015)
    4. Breaking transmission research: $0.1 million (DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015
    5. Vietnam prevalence surveys and other: $0.1 million (DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015)
    6. India training materials design: $12,000 (DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015)
    7. Ethiopia development of proposal and other: $11,000 (DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015)
  • See our November 2014 blog post for Deworm the World's then-expected allocation of $1.4 million of $2.3 million it had received due to our influence.

• 327

  In the following, we provide some examples of Deworm the World's speed of spending.

  • Spending in 2014: From late 2013 to late 2014, GiveWell-influenced donors donated ~$2.3 million to Deworm the World. At the end of 2014, Deworm the World had spent ~$1.6 million of that funding and had allocated ~$700k to future activities in 2015: ~500k to exploring deworming programs in new countries and ~$200k to research. See this blog post.
    • The financial documents we have seen for 2015 do not include how much was spent on research activities.
    • Deworm the World spent ~$170k on exploring or supporting new countries' deworming programs in 2015. Deworm the World, 2015 expense summary - by funder This is significantly less than the ~$500k allocated, and may include restricted funding. We guess that at the end of 2015, Deworm the World still had at the very least ~$330k of the initial $2.3 million that it had not spent.
  • Spending in 2015: In late 2014, Deworm the World expected:
    • To spend $170,000 of unrestricted funding it received on activities in Vietnam. By the end of 2015, it had only spent ~$33,000 of unrestricted funding in Vietnam and still had $93,000 allocated for future spending in Vietnam. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 and Deworm the World, 2015 expense summary - by funder
    • To spend $500,000 on research. We are not sure how much it spent on research over the course of 2015, but it still had ~$110,000 allocated to spending on research by the end of 2015.
  • Spending in 2016: Deworm the World received ~$11.9 million in unrestricted funding from GiveWell-influenced donors alone between February 2015 and January 2016. Deworm the World's 2016 budget shows a total spend of $13,256,738. When excluding expenses for India and Kenya, which are almost entirely funded by restricted funding, Deworm the World is only expecting to spend $3,405,677 (and this figure likely includes some restricted funding too, e.g., for Deworm the World's program in Vietnam). So, we believe that most of the unrestricted funding donors gave 2015-2016 will not be spent until 2017 or later. Note that we had expected much of the funding to go to three-year projects, so this is not very surprising.

• 328

  Grace Hollister, email to GiveWell, June 9, 2016

• 329
  • "[In Nigeria] Deworm the World would ideally like to make a 3-year commitment in Cross River to increase the chance of government approval, increase stability, enable Deworm the World to establish a partnership with the government, and take steps toward institutionalization of deworming programs.
    There is some inherent risk in multi-year commitments, but one year is not always enough time to build a new program that runs effectively, and governments would be reluctant to work with Deworm the World if they were limited to one year. [...] One-year commitments can be costly for Deworm the World because governments typically expect that programs will continue past the first year, and it can be difficult to find funding." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7
  • "[In Pakistan] A multi-year funding commitment will be important to encourage the government to work with Deworm the World. Ms. Hollister would like to aim for a 5-year commitment and has had a 5-year budget created. While it is unlikely that Deworm the World will get an up-front 5-year funding commitment, it may be difficult to reach an agreement with the government without having funding for all 5 years. Deworm the World could begin with a 3-year commitment and indicate its intention to stay for 5 years and find funding to fill the gap." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 8

• 330

  "Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage.

  GiveWell conducted a site visit to DSW in November 2012 and published notes. DSW 2012 GiveWell site visit

  See GiveWell's review on water quality interventions.

• 331

  GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and Evidence Action 2015 funding gap analysis

• 332

  Deworm the World, 2015 expense summary and Evidence Action, Projected allocation of unrestricted funds, 2016.

• 333

  Evidence Action, 2015 financials by program

• 334
  • WHO Weekly epidemiological record, 18 December 2015, Pg 707 (Table 1)
  • In 2013 about 34% of all children in need of STH deworming received treatment. WHO Weekly epidemiological record, 6 March 2015, Pg 91 (Table 1)
  • Older data: "In 2009, a total of 314.6 million children were reportedly dewormed, of whom
    274 million required preventive chemotherapy. This corresponds to approximately
    31% of all children in need of treatment." WHO STH treatment report, Pg. 34.

• 335

  WHO schistosomiasis treatment gap data.