Deworm the World Initiative, Led By Evidence Action – November 2016 Version

We have published a more recent review of this organization. See our most recent report on Deworm the World.

Published: November 2016

Summary

What do they do? The Deworm the World Initiative (evidenceaction.org/#deworm-the-world), led by Evidence Action, advocates for, supports, and evaluates government-run school-based deworming programs. (More)

Does it work? We believe there is relatively strong evidence for the positive impact of deworming. Deworm the World sends monitors to schools during and after deworming to determine whether the programs it supports have reached a large proportion of children targeted. We have reviewed data from its two largest programs and one new program, which overall indicate strong results. (More)

What do you get for your dollar? We estimate that, in most of Deworm the World's programs, children will be dewormed for a total of about $0.79 per child, or $0.55 per child excluding the value of teachers’ and principals’ time spent on the program. The cost per child treated in India is less than half of that. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)

Is there room for more funding? We believe that Deworm the World is likely to be slightly constrained by funding in the next year and that additional funds would increase the chances that it is able to take advantage of any high-value opportunities it encounters. We expect Deworm the World to have $11 million to spend or commit in the next year (it has about $8.9 million on hand and we expect it will receive about $2.2 in additional funding). We estimate that if it received an additional $13.4 million (allowing it to spend or commit a total of $24.4 million) its chances of being constrained by funding would be reduced to 5%. (More)

Deworm the World is recommended because of its:

• Focus on a program with a strong track record and excellent cost-effectiveness. (More)
• Strong process for assessing whether the deworming programs it supports are successfully deworming children. (More)
• Standout transparency – it has shared significant, detailed information about its programs with us.
• Room for more funding – we believe Deworm the World will be able to use additional funds to start or maintain deworming programs.

Major open questions include:

• Deworm the World's spending has lagged behind revenues as it has worked to start programs in new locations. It is unclear whether it will be able to scale up as quickly as it intends to.
• We are unsure how additional donations to Deworm the World will impact how Evidence Action, Deworm the World’s parent organization, chooses to allocate its unrestricted funding.
• We are not sure how Deworm the World’s success and cost-effectiveness in new countries will compare to its track record in India and Kenya.

Our review process

Our review process has consisted of:

• Conversations with Deworm the World Director Grace Hollister and other Deworm the World and Evidence Action staff since 2012.1
• A conversation with the Children's Investment Fund Foundation (CIFF), a funder of Deworm the World.2
• Reviewing documents Deworm the World sent in response to our queries.
• Following Deworm the World's progress and plans for funds raised as a result of GiveWell's recommendation (as well as Evidence Action's overall progress and plans).
• Site visits:
  • In November 2012, we visited Deworm the World's office in Nairobi, Kenya and met its staff there. (Notes from our visit)
  • In October 2013, we visited Deworm the World's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with Deworm the World. (Notes from our visit)
• In October 2016, we visited Evidence Action's offices in Washington, D.C. We met with five Deworm the World staff members, including leadership and program managers. We also met with several members of the Evidence Action leadership and finance teams.
• In 2015, we retained two journalists to visit areas served by Deworm the World in Kenya. We published their report on our blog.

All content on Deworm the World, including past reviews, updates, blog posts and conversation notes, is available here.

What do they do?

The Deworm the World Initiative, led by Evidence Action, advocates for and supports the implementation of government-run deworming programs for preschool- and school-age children.3 The support that Deworm the World provides is of two types: 1) Deworm the World offers technical assistance to governments implementing deworming, and 2) Deworm the World may fund components of deworming programs.4

The deworming programs that Deworm the World supports are focused on executing school-based mass drug administrations (MDAs), in which the aim is to treat the entire population of children within a geographic area by distributing deworming pills.5 Deworm the World focuses on MDAs that treat children infected with soil-transmitted helminthiasis (STH).6 Where needed, these programs may also include treatment for schistosomiasis.7

In the countries it works in, Deworm the World works primarily with government staff to implement deworming programs; as it has expanded to new countries, it has started to also collaborate with NGO partners to support governments.8 Additionally, Deworm the World has also funded or is considering working on a few projects that fall outside of its purview of supporting the direct implementation of deworming programs, specifically around monitoring and research (see footnote).9

Deworm the World was founded in 2007,10 and as of September 2016 had supported deworming treatments in India, Kenya, Ethiopia, Nigeria, and Vietnam and had started preliminary support for a deworming program in Pakistan.11 Many of these programs are recent and represent progress on Deworm the World's efforts to scale; as of early 2015, Deworm the World had only supported treatments in India and Kenya.12

Below, we discuss:

• Deworm the World's role in government-led deworming programs
• The status of Deworm the World's work by country
• A breakdown of Deworm the World’s spending
• Deworm the World's relationship to Evidence Action

Deworm the World’s role in government-led deworming programs

The deworming programs that Deworm the World supports are implemented by the governments it works with.13 Below, we expand on Deworm the World's role in the programs it supports. Note that we use "Deworming Day" to mean the day on which the MDA takes place. Similarly, we use "Mop-Up Day" to refer to the day that occurs several days after Deworming Day and is when students who were absent or sick on Deworming Day are given their deworming pills (although note that not all countries have just one Deworming Day or include a Mop-Up Day in their program).14

The assistance that Deworm the World provides in each country varies based on what each partnering government needs.15 Historically, Deworm the World's role has included the following:16

1. Advocacy. Deworm the World actively encourages national and large sub-national governments to implement mass school-based deworming programs.17 Our impression is that Deworm the World's advocacy consists of meeting with health and education officials in a government to discuss the benefits of deworming and how a deworming program might be implemented.18 Deworm the World has told us that it will not work with a government on a national deworming program until it has built a strong working relationship with that government via its advocacy.19 Deworm the World also participates in the broader "STH community"; that is, it works with other organizations advocating for and implementing activities that will further reduce or eliminate STH globally.20
2. Prevalence surveys. Before Deworm the World helps launch a deworming program in a new area, it evaluates whether the prevalence of worm infections is sufficient to justify an MDA for the school-age population.21 If no prevalence surveys have been conducted recently, it generally commissions one.22 It also plans to conduct follow-up prevalence surveys periodically, so that it can track the impact of the MDAs and refine treatment strategies as needed, in accordance with WHO guidelines.23 Deworm the World generally contracts out work on prevalence surveys.24
3. High-level program planning. Deworm the World has told us that it often assists governments with high-level operational decisions, such as developing the country's treatment strategy and how to budget for the program.25
   
4. Drug procurement and protocols. Deworm the World assists governments in obtaining drugs, designing drug distribution and tracking processes, and developing adverse event protocols for cases where children react poorly to treatment.26 For example, Deworm the World has helped governments submit requests for deworming drugs (albendazole or praziquantel) to the World Health Organization (WHO) global drug donation program.27
5. Program preparation: trainings and distribution of materials. Deworm the World has helped governments design and organize what it calls a "training cascade" (more detail in the footnote).28 Through the training cascade, teachers and other government staff learn how to implement a Deworming Day and receive materials necessary for implementation (such as reporting forms and drugs).29 In the past, Deworm the World has hired or trained staff to lead the trainings and developed materials for the trainings.30 In India, Deworm the World has also arranged tele-callers to reach out to schools to assess their preparedness and notify government officials of any problems before Deworming Day.31
6. Community sensitization. Deworm the World supports community sensitization efforts, which aim to make local communities aware of Deworming Day and the benefits of deworming children.32 For example, via the training cascade, teachers are instructed to spread the word about Deworming Day to their communities.33 Deworm the World has also developed text message campaigns, organized public announcement events, and edited mass media materials to be more appropriate for local contexts.34
7. Monitoring and evaluation. Deworm the World told us that it helps governments design or improve reporting and monitoring systems. It also collects monitoring data independently.35 Deworm the World focuses on assisting with the collection of three main types of monitoring data:36
   • Monitoring before and during deworming: Monitors hired by Deworm the World visit schools before and during Deworming Day and Mop-Up Day. They are meant to assess both a) how prepared schools and health systems are to implement deworming and b) the extent to which proper procedures are followed.37 Monitoring visits may include assessments of the quality of trainings, community sensitization efforts, and Deworming Day activities, depending on what Deworm the World and the government agree to monitor.38
   • Coverage reporting: On Deworming Day and Mop-Up Day, teachers are asked to mark the number of children that they deworm and schools complete specially designed reporting forms to tally the number of children treated. This data is then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (e.g., in India: school, node, block, district, and state) to create a reported coverage estimate for a region.39
   • Coverage validation: Approximately one week after Mop-Up Day, Deworm the World sends independent monitors back to schools to check the coverage data recorded at schools against the data submitted and ask students about whether or not they were dewormed.40 This data can then be compared to the coverage data reported by the government.

   Typically, Deworm the World hires and trains third-party monitors to collect process monitoring and coverage validation data; this footnote includes Deworm the World’s descriptions of the monitor selection process used in four states in India in 2015.41 In Kenya, Evidence Action (Deworm the World's parent organization) maintains a monitoring team year-round that Deworm the World makes use of.42

Description of programs by country

As of September 2016, Deworm the World had supported deworming programs in India, Kenya, Ethiopia, Vietnam, and Nigeria, and it had started support (or was in discussions about starting support) in two other countries (Pakistan and Indonesia).43 Several of these programs started in 2015 or 2016; as of early 2015, Deworm the World had only supported deworming programs in Kenya and India. Each country has or will have a slightly different program model; we expand upon the programs by country below and have summarized some high-level details in this spreadsheet.

India

Deworm the World started working in India in 2009.44 It has assisted at both the state level and the national level with all areas mentioned above.45 Deworm the World has received funding for its work in India from the Children's Investment Fund Foundation (CIFF), the United States Agency for International Development (USAID), Dubai Cares, the END Fund, and the Michael & Susan Dell Foundation (past).46

In 2014, following advocacy from Deworm the World, the national government decided to implement "National Deworming Day," a single-day school and preschool-based deworming program targeting all at-risk children aged 1-19 for STH treatment.47 The first National Deworming Day was held in February 2015, with 11 states participating that day and one state postponing until April (five of which received direct support from Deworm the World).48 A second National Deworming Day was held in February 2016 in 30 of India's 36 states and union territories; eight of the participating states received support from Deworm the World.49 India held a third National Deworming Day in August 2016 in 26 states with prevalence rates meriting twice-per-year treatments; Deworm the World supported five of these states.50 Deworm the World is now seeking additional funding to expand its work to several additional states (more below).

One state that Deworm the World previously supported—Delhi—has transitioned to implementing its deworming program with very limited technical support; Deworm the World expects to reduce support to other states as their capacity to operate their programs without Deworm the World's assistance increases over the next few years.51

Kenya

Deworm the World started to work with the national government in Kenya in 2009, supporting a pilot school-based deworming program.52 In 2010, Deworm the World agreed to support Kenya's National School-Based Deworming Program (NSBDP), which started in 2012 under a 5-year plan funded by CIFF and the END Fund.53 Deworm the World has entered its fifth year of work in Kenya under this agreement (2016-2017) and is seeking funding to continue supporting the Kenya program (more below).54

Deworm the World provides assistance in all of the other areas we described above.55 Most of Kenya was mapped before Deworm the World started to work there, and the Kenya Medical Research Institute (KEMRI) is funded by CIFF to conduct independent prevalence surveys (with limited sample sizes) before and after each round of deworming.56 Deworm the World has received funding from the END Fund to support treatments for lymphatic filariasis in Kenya, starting in 2017.57

Similar to in India, the program in Kenya has moved towards a model in which deworming occurs on consolidated Deworming Days.58

Currently, Deworm the World has a robust staff presence in Kenya, in part because Deworm the World implements monitoring activities in Kenya itself (rather than hiring consulting firms to implement this) and acts as the fiscal agent for the program.59 The Kenya team occasionally supports and trains staff for Deworm the World's newer programs.60

Ethiopia

In Ethiopia, Deworm the World partners with the Schistosomiasis Control Initiative (SCI)—another of GiveWell's recommended charities—to advise the national government.61 Deworm the World started advising Ethiopia in late 2014, after Ethiopian government officials visited Kenya for a learning exchange and spoke with Deworm the World representatives during the trip.62

In early 2015, Ethiopia launched a national school-based deworming program.63 Deworm the World supported the government in implementing a pilot MDA in mid-2015 and the national program's first full MDA in late 2015.64 Deworm the World's work on the pilot was funded in part by its unrestricted funding.65 In mid-2015, CIFF and the END Fund decided to fund a five-year national school-based deworming program in Ethiopia.66

Deworm the World has told us that its role in Ethiopia is similar to its role in Kenya and India (though it has less of a staff presence), while SCI assists with activities that are more scientifically technical, such as conducting prevalence surveys.67

Pakistan

Deworm the World first began to explore the possibility of working in Pakistan in 2015, for two reasons:

• Pakistan is believed to have a high worm burden across a large population, and there is currently no mass treatment of school-age children for STH in Pakistan.68
• An organization that Evidence Action has partnered with on other programs—International Relief & Development (IRD)—works in the health sector in Pakistan.69

In Pakistan, Deworm the World is partnering with two local organizations: IRD and the Institute of Development and Economic Alternatives (IDEAS).70 Neither organization has experience with school-based deworming.71 IRD has implemented other health programs and IDEAS is involved in education policy work.72

Because the prevalence of STH in Pakistan has not previously been mapped, IRD and Deworm the World are conducting a nationally representative prevalence survey there.73 The prevalence survey was initially slated to begin in early 2016, but was rescheduled to start in August 2016 after multiple delays; it is expected to finish in November 2016 (see footnote for detail).74 Results of the prevalence survey will determine the recommended treatment strategy for STH in Pakistan. Though WHO has predicted high prevalence of STH throughout most of the country, preliminary results are suggesting that there may be lower prevalence than originally thought, so it is unlikely a nationwide deworming program will be required.75

Deworm the World and its partners have reached out to provincial governments in Pakistan about implementing a deworming program; it expects conversations to move more quickly once results from the prevalence surveys are available.76 In June 2016, the national Ministry of Health in Pakistan dissolved, so Deworm the World expects to primarily work with individual provinces, similar to how it works closely with states in India; it believes that this will cause working in Pakistan to be somewhat more challenging.77

The prevalence survey in Pakistan is almost entirely supported by Deworm the World's unrestricted funding.78

Nigeria

In 2015, Deworm the World hired a consulting firm to assess the opportunities for working in Nigeria.79 The assessment found that several states with a high burden of STH did not have any NGO partners focused on deworming.80 After meeting with government officials in four of the states, Deworm the World decided to work in Cross River.81

In Nigeria, the national government requests that all programs that target a neglected tropical disease (NTD) be integrated with other NTD programs.82 Deworm the World partners with RTI International in Cross River.83 Deworm the World is supporting the state government to implement school-based treatment for schistosomiasis and STH, while RTI focuses on community-based treatment of other endemic NTDs (more details in footnote).84 The first MDA they supported took place in June 2016.85

Most of Nigeria has already been mapped for STH and schistosomiasis, which means that Deworm the World does not need to assist with mapping surveys.86 Future activities in Nigeria will likely include advocacy and technical assistance to the national government and expanding Deworm the World's support to additional states.87 Deworm the World intends to register in Nigeria and establish a national office there to support its future scale-up.88

Deworm the World is funding its work in Nigeria with unrestricted funding.89

Vietnam

Deworm the World is partnering with the Thrive Networks in Vietnam on an integrated deworming, sanitation, and hygiene education program in four provinces.90 The program is funded primarily by Dubai Cares and includes a randomized controlled trial to test the impact of hygiene education on STH reinfection rates.91 The program was approved by the Vietnamese government in late 2015.92

Some deworming already occurs in Vietnam; Deworm the World and Thrive Networks are partnering with the government to improve the existing deworming program in the four provinces targeted by the program.93 Deworm the World hopes to convince the national government to develop an evidence-based, cost-effective national deworming program.94

The first MDA supported by Deworm the World occurred in April 2016; it was conducted by the National Institute of Malaria, Parasitology, and Entomology (NIMPE)—the government agency in Vietnam that implements MDAs.95 After the MDA occurred, Deworm the World received the results from the baseline prevalence survey it conducted pre-MDA; the results indicated that the treatment frequency required for each province was lower than expected.96 Deworm the World is currently advocating for the government to adjust its strategy; the current government policy is to conduct biannual deworming MDAs.97

Deworm the World has also worked with NIMPE to conduct prevalence surveys in an additional 21 provinces.98 This was beyond the scope of the Dubai Cares grant, so Deworm the World supported the prevalence surveys with its unrestricted funding.99 The results from that survey indicated that most provinces only need treatment once per year or once every other year.100

Deworm the World originally became involved in Vietnam through discussions between its former Executive Director, Alix Zwane, and Thrive Networks' former regional director.101 We are unsure who initiated the partnership between the two organizations.

Nepal

In early 2015, Deworm the World was in discussions with Nepal about starting a deworming program there.102 However, in mid-2015 Nepal experienced a large earthquake and those discussions were halted.103 It has not yet re-engaged with those conversations, in part due to capacity constraints.104

Deworm the World was first connected to the government of Nepal via the Abdul Latif Jameel Poverty Action Lab (J-PAL) (similar to how the program in Bihar began).105

Indonesia

Deworm the World has had initial conversations with the government of Indonesia about a potential deworming program there.106 Indonesia would require both technical assistance and funding for the implementation of a deworming program.107 Deworm the World hopes to make progress on conversations during the last quarter of 2016.108

Breakdown of Deworm the World’s spending

We have seen detailed breakdowns of Deworm the World's spending for 2014, 2015, and the first half of 2016.109 The vast majority of this spending (about 89%) was funded by restricted funding.110 Restricted funding from the Children's Investment Fund Foundation (CIFF) and the END Fund fully fund Deworm the World’s work in Kenya, where it has supported a nationwide deworming program since 2012.111 In 2014 and 2012 respectively, Deworm the World received funding commitments from CIFF/Dubai Cares and USAID to fully support its work in India; in 2015 a grant from the END Fund was received to support work in Rajasthan.112

Deworm the World’s spending on deworming programs is supplemented by the spending of partner organizations and the implementing governments. Below is a breakdown of how Deworm the World spent its funding from January 2015 through June 2016 globally as well as in Kenya and India.113 In 2015, Deworm the World spent about $5.2 million on all expenses.114 Deworm the World has told us that the majority of its personnel costs fall within the "program management" category.115

Deworm the World’s expenses - January 2015 through June 2016116

	India	Kenya	All locations
Program management	39%	32%	45%
Training	7%	45%	21%
Monitoring and evaluation	18%	10%	11%
Prevalence surveys	17%	0%	9%
Policy	6%	5%	6%
Awareness	10%	6%	6%
Drugs	3%	1%	2%
Total	100%	100%	100%

Deworm the World and Evidence Action

In early 2013, Innovations for Poverty Action (IPA) announced the formation of Evidence Action to scale cost-effective and evidence-based programs. Two IPA initiatives, Deworm the World and Dispensers for Safe Water, were spun off from IPA to be managed by Evidence Action. It has since built a department for investigating, testing, and considering new programs for scaling up called Evidence Action Beta; one program in the Beta portfolio is No Lean Season, which GiveWell has recommended grants to.117 We focus this review on Deworm the World and discuss the room for more funding implications of Deworm the World being a program of a larger organization below.

Does it work?

Deworm the World-assisted mass drug administration programs are focused on delivering treatments that have been independently studied in rigorous trials and found to be effective. Evidence from Deworm the World's monitoring makes a relatively strong case that the programs Deworm the World has supported have successfully dewormed children.

While Deworm the World's track record in Kenya and India is strong, it has recently expanded to several new countries. As with any major scale-up, there is a risk that it may not achieve as strong results as it has in the past or may not produce similarly strong evidence of its impact. We have seen preliminary monitoring from one of its new countries of operation (Vietnam); these results indicate that Deworm the World is using similar monitoring processes in new countries as it has in Kenya and India and that results in Vietnam have been reasonably strong.

Here we focus on the following questions to understand whether Deworm the World’s activities are having the intended impact (details in the sections that follow).

• Are mass school-based deworming programs effective when implemented well?
• Are Deworm the World's programs targeted at areas of need?
• Are deworming pills delivered to and ingested by recipients?
• How does Deworm the World affect program outcomes?
• Are there any negative or offsetting impacts?

Are mass school-based deworming programs effective when implemented well?

Deworm the World supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.

There are some important differences between the type and severity of worm infections in the places Deworm the World works and the places where the key studies on improved life outcomes from deworming took place (which we discuss below). In particular, Deworm the World primarily provides support to mass drug administrations (MDAs) that treat populations where fewer children are infected with soil-transmitted helminths and where the severity of infections tends to be lower. In addition, several of the programs Deworm the World supports do not treat schistosomiasis because it is not endemic in the areas the programs support.118

Is the program targeted at areas of need?

Deworm the World has told us that it advocates for governments to follow WHO guidelines for treatment of STH (the guidelines are based on baseline prevalence):119

• If prevalence is below 20%, MDA is not needed;
• If prevalence is between 20% and 50%, MDAs should occur once per year;
• If prevalence is greater than 50%, MDAs should occur twice per year.
• After multiple (5-6) years of treatments, WHO recommends further annual MDAs for areas with at least 10% STH prevalence and MDAs every two years for areas with 1-10% prevalence.120

Deworm the World also advocates for governments to follow WHO guidelines for treatment of schistosomiasis, in the areas it works where schistosomiasis is endemic.121 We have seen evidence indicating that Deworm the World primarily supports annual MDAs in areas with prevalence greater than 20% and biannual MDAs in areas with prevalence greater than 50%; we have not seen any indication that Deworm the World systematically treats populations with low infection rates.122

Baseline infection status: Deworm the World programs vs. rigorously studied programs

Although Deworm the World only supports MDAs in areas where the prevalence is high enough to warrant treatment according to WHO, most of the populations that Deworm the World's program targets have a lower prevalence and intensity of STH and schistosomiasis than the populations treated by the deworming programs discussed in our intervention report on deworming. Given this, we assume that the programs Deworm the World supports have less impact per child treated than the deworming programs that have been rigorously studied.

Key pieces of evidence that we discuss in our report on deworming (Miguel and Kremer 2004, Baird et al 2012, and Croke 2014) are from deworming experiments conducted in Kenya and Uganda in the late 1990s and early 2000s. Prior to receiving deworming treatment, the participants in those studies had relatively high rates of moderate-to-heavy infections of schistosomes or hookworm.123

In this spreadsheet, we compare the prevalence of the places in which Deworm the World currently supports a program to the prevalence from the Croke study and the Miguel and Kremer study. The prevalences in the table for Madhya Pradesh and Chhattisgarh were measured after multiple rounds of treatment, while Bihar's, Rajasthan's, and Uttar Pradesh's prevalences were measured after one round of treatment. This makes it difficult to compare the prevalences in a meaningful way. It is likely that prevalence was higher at baseline for these regions.124 Note that we have not yet prioritized including recently received prevalence data for Ethiopia, because we do not expect GiveWell-directed funding to support programs in Ethiopia.125

Deworm the World notes that there are relevant methodological differences between the prevalence surveys, which makes them difficult to compare (see our previous footnote for more detail on methodology); we do not fully understand which methodological differences Deworm the World is referring to and we believe the surveys are similar enough that comparison remains a useful exercise.126

We would guess that future programs Deworm the World supports will also treat in areas with different worm types and significantly lower prevalence than those in the key deworming studies, largely because prevalence in those studies was unusually high.127

Treatment for lymphatic filariasis

In some of the countries where Deworm the World works, there are existing programs to treat lymphatic filariasis (LF).128 Albendazole, the same drug used to treat STH, is usually used in combination with one additional drug to treat LF (and the same dosage is used for both treatments).129 For areas that have existing LF treatment programs, the effect of Deworm the World’s support may be to transition an area from once-per-year deworming treatment to twice-per-year treatment.130

We detail what we know about the status of LF programs in the areas in which Deworm the World works below:

• India. There are active LF programs in several of the states that Deworm the World supports.131 We discuss some details of India's LF program in this footnote.132
• Kenya. There are LF programs in the coastal-region counties in Kenya.133
• Ethiopia. Ethiopia has had an active LF program. Going forward it intends to coordinate LF treatments with treatments for STH to avoid duplication of efforts.134
• Nigeria. In Cross River, Deworm the World works with its partner RTI to support an integrated NTD program. Our understanding is that there is coordination to avoid duplication of efforts for LF and STH.135
• Vietnam. Vietnam recently stopped its LF program and intends to monitor whether or not it has successfully eliminated LF in the future.136
• Pakistan. We do not know whether or not Pakistan has any active LF programs.
• Indonesia. Deworm the World told us that Indonesia has an active LF treatment program.137

Are deworming pills delivered to and ingested by recipients?

Administration of deworming drugs is a relatively straightforward program, though any MDA could encounter many challenges when operating at a large scale. The information we have seen from monitors hired and trained by Deworm the World in India, Kenya, Vietnam, and Ethiopia suggests that the programs successfully deliver pills to children, who then swallow them. We have also seen one coverage survey from Ethiopia's national deworming program pilot in April 2015, which was supported by Deworm the World and SCI and indicates that pills were delivered and ingested. Additionally, prevalence surveys in Kenya, and to a lesser degree Bihar, India, show that the prevalence rates of STH and schistosomiasis have declined substantially since Deworm the World started supporting MDAs in those areas, providing additional evidence that the treatments are reaching recipients.

Evidence from monitoring

For each of its programs, Deworm the World hires monitors (who are not associated with the government implementing the program) to:138

• [Sometimes] Observe trainings. For some of its programs, Deworm the World sends monitors to visit a random selection of trainings.139 The visits are unannounced, and the monitors use a standardized checklist to track if the training covers all required topics.140 Monitors also test how the training increases the knowledge of training attendees by selecting several attendees before the training and administering a short quiz to them, then selecting several attendees after the training to again take the quiz.141
• [Sometimes] Visits or calls to communities and schools before Deworming Day. For some of its programs, Deworm the World asks monitors to visit or call, unannounced, a random selection of schools and/or communities before Deworming Day.142 At the schools, monitors will interview teachers and/or headmasters, asking a variety of questions to assess preparedness, such as whether or not the school has enough treatments for Deworming Day and if a representative from the school attended training.143 In communities, monitors select a sample of adults to interview, asking questions to assess their awareness of Deworming Day, which helps Deworm the World determine how successful its community sensitization efforts were.144
• Observe activities on Deworming Day and Mop-Up Day. In all of its programs, Deworm the World sends its monitors, unannounced, to observe a random sample of schools on Deworming Day and Mop-Up Day.145 If the chosen school is closed on the day of the deworming, they are instructed to go to an assigned backup school instead.146 At the schools, monitors interview teachers and school administrators to assess how prepared the school was for Deworming Day. For example, monitors ask if the school has sufficient drugs for Deworming Day, whether or not a school representative attended training, and a variety of questions to test teachers' knowledge about the proper procedures for the MDA campaign, like what the teacher should do if a child is feeling sick.147 Then, the monitors observe randomly selected classes, recording details about the Deworming Day activities, such as whether deworming is in progress, teachers are documenting who is dewormed, and teachers are watching to make sure that students swallow the pills.148
• Conduct a coverage validation exercise. In most of its programs, Deworm the World sends out monitors within 1-2 weeks of Deworming Day and Mop-Up Day to conduct a coverage validation exercise at a randomly selected sample of schools.149 At the schools, monitors ask teachers questions about Deworming Day, such as if the school had sufficient tablets.150 They also check class registers and record the number of students that were dewormed according to the school's records.151 Finally, they randomly select a small sample of students to interview, asking the children several questions about their experience on Deworming Day.152 For example, they might ask if the child received a pill, if the child swallowed the pill, and how the child was feeling on Deworming Day.153

We believe that the last two types of monitoring are especially valuable. However, Deworm the World is not supporting an coverage validation exercise in schools in Ethiopia, because household coverage surveys conducted after the Deworming Days are occurring there instead. Deworm the World also did not support coverage validation monitoring for the first MDA it supported in Cross River, Nigeria, although it hopes to in subsequent rounds.154

While we believe that Deworm the World's monitoring is fairly high-quality, we have a few concerns about its methods:

• Deworm the World does follow-up calls to audit monitors' work in Kenya. It also has some checks on monitors' work in India. We are not aware of audits of monitors' work in other locations.155
• Deworm the World's program and monitoring are done through schools, which makes it difficult to know how many non-enrolled children are potentially being missed by the program. Schools are supposed to target and record the treatments they distribute to non-enrolled children, but most locations do not have reliable data on how many non-enrolled children there are.156
• Even though school staff are not aware ahead of time that a monitor is coming to visit, once a monitor shows up, they may be motivated to execute the program in a more rigorous fashion than they would have otherwise.
• It is possible that children feel pressure to say that they took the deworming pill on coverage validation day. Deworm the World tries to ask students away from their teachers, to reduce pressure, and also has some checks on students' answers.157

Results from monitoring

We find the evidence produced by Deworm the World from observations on Deworming Day and during the coverage validation exercise to be compelling support for the claim that children receive and ingest pills during Deworm the World-supported MDAs; we have laid out this evidence in this spreadsheet.158 We also see some reason to interpret these data cautiously, as some of it was self-reported by people who could be biased to want favorable monitoring results. We note that some of the results from Kenya seem quite high to us when compared to the results we have seen from India. We are not sure if this is due to some bias in the way that the Kenya data is collected, or if Deworm the World's program in Kenya is simply higher-quality.159

See this spreadsheet for a sample of methods used in and results from recent monitoring in Kenya, India, and Vietnam (the three countries from which we've seen this data).160 We have not yet seen Deworm the World's standard monitoring data from Nigeria or Ethiopia, but Deworm the World told us that it should be similar to the type of monitoring that we've seen in the past.161

The spreadsheet does not include all data the monitors collected. We have selected questions that we found particularly relevant to assessing the quality of the programs and easy to interpret. (Note: Previously, we summarized monitoring from the 2013-2014 programs and the 2012 programs).

We have not prioritized reviewing the data from the monitoring Deworm the World conducts before Deworming Day, although we think it could also provide insight as to the quality of Deworm the World's program. However, we have included some metrics in our table related to the preparedness of schools and their adherence to proper protocols (from questions that are asked on Deworming Day). Based on the data and reports we have seen, we feel fairly confident that most schools in India and Kenya are prepared to implement Deworming Days: most schools seem to have sufficient number of deworming pills available for children; fewer appear to have adequately trained teachers (the fact that the surveys identified low training rates in several cases increases our confidence in the reliability of the surveys). We feel less confident that schools are following proper procedures, especially with regards to accurate reporting.

Ethiopia coverage survey

Although we have not yet seen results from Deworm the World's standard monitoring for the MDAs conducted in Ethiopia in mid and late 2015, we have seen a coverage survey conducted after the April 2015 MDA.162 It is our impression that the coverage survey was primarily supported by Deworm the World's partner in Ethiopia, SCI. We write about the methods used for SCI's coverage surveys here and lay out the results and methods used for the Ethiopia coverage survey in this spreadsheet. We believe the coverage survey provides relatively strong evidence that deworming pills were distributed and ingested by children in Ethiopia.

Evidence from prevalence surveys over time

Deworm the World or its partners have conducted several surveys to track changes in schistosomiasis and STH prevalence and intensity rates following Deworm the World-supported treatment programs. In general, prevalence and intensity of the parasites decreased over time in each of the countries studied. This increases our confidence that Deworm the World's program is effectively delivering treatments to children. We note several methodological limitations of these surveys below.

India prevalence surveys

Deworm the World has conducted prevalence surveys in several of the states that it assists with in India, and has conducted one follow-up prevalence survey in the state of Bihar.163 The two surveys measuring the prevalence of worm infections in Bihar are consistent with the notion that the deworming program reduced infection rates in target populations. However, there are also plausible, alternative explanations for these results. (Note that the first survey was not designed to measure a subsequent change in infection rates, so our conclusion is not surprising.)

The original prevalence study in Bihar measured prevalence in four of Bihar’s 38 districts, which were selected to inform a prevalence model based on climatic and socioeconomic variance across the state.164 We are not sure how representative that survey was of the state as a whole since the districts were not chosen randomly.165 Since the survey found >50% prevalence,166 the government of Bihar planned to follow WHO’s recommendation to conduct deworming MDAs twice per year, with one round each year led by the National Filaria Control Program (NFCP) and one round through schools that would be assisted by Deworm the World.167 However, according to Deworm the World, only four rounds of deworming took place between the two prevalence surveys in 2011 and early 2015, with one community-based round led by the NFCP and three school-based rounds assisted by Deworm the World.168

Deworm the World’s follow-up survey in early 2015 was conducted in randomly selected districts and was designed to be representative of the entire state and the three climatic regions.169 The results of both surveys, broken down by climatic region, are in the table below; no single district was surveyed in both prevalence surveys.

Indications of STH prevalence in Bihar’s three agro-climatic zones in 2011 and 2015170

	Hookworm	Ascaris (roundworm)	Trichuris (whipworm)	Any soil-transmitted helminth
NW Alluvial Plains, 2011 (2 districts, non-random)	47.4%	56.8%	4.2%	71.3%
NW Alluvial Plains, 2015 (5 districts, random)	21.0%	24.0%	12.0%	43.0%
NE Alluvial Plains, 2011 (1 district, non-random)	32.8%	72.1%	11.3%	79.6%
NE Alluvial Plains, 2015 (3 districts, random)	17.0%	38.0%	5.0%	50.0%
S Alluvial Plains, 2011 (1 district, non-random)	40.2%	25.0%	1.7%	49.0%
S Alluvial Plains, 2015 (6 districts, random)	14.0%	6.0%	2.0%	20.0%
Statewide 2011 (4 districts, non-random)	42.2%	52.1%	5.2%	67.5%
Statewide 2015 (14 districts, random)	17.0%	19.0%	6.0%	35.0%

We are unsure whether worm prevalence in Bihar would have increased or decreased in the four years between the two prevalence surveys if the three MDAs that Deworm the World supported had not occurred.171 Here we’ve listed some other factors that we weigh when considering these results:

• Methodological limitations of these prevalence surveys. It may be that the original prevalence survey took place in districts with unusually high worm prevalence. Since it is unclear whether the original prevalence survey was representative of the state or its climatic zones and the two prevalence surveys did not sample the same districts, it may be that the decrease in prevalence seen above is the result of comparing different populations rather than a fall in prevalence in the same population over time.
• MDAs for lymphatic filariasis. The NFCP, which is designed to reduce the burden of lymphatic filariasis (LF) as well as STH, claims it conducted MDAs in Bihar in 2012 and 2014 with albendazole, the drug used to treat STH.172 According to the Indian government's program website, the LF treatment program is designed to be annual, and reached 86% of the population in both 2012 and 2014.173 We would not be surprised if the coverage rates reported by the LF program were inaccurate, but we have not tried to verify them. Deworm the World has said that the 2014 round actually occurred in early 2015 after the 2015 prevalence survey.174
• Factors other than MDAs. There are a variety of large-scale changes that could affect worm prevalence, such as improved sanitation infrastructure or general development. According to India’s central bank, inflation-adjusted per-capita income in Bihar increased by a total of 39% over a similar four year period, suggesting that there could have been broad improvements in the region that would be associated with better health and lower worm prevalence.175 Alternatively, it seems possible that population growth combined with poor sanitation infrastructure could have increased worm prevalence over this period if Deworm the World-supported MDAs had not been carried out.

Kenya prevalence surveys

In Kenya, the Kenya Medical Research Institute (KEMRI) is funded by CIFF to conduct independent prevalence surveys before and after every MDA that Deworm the World supports; Deworm the World is not involved in these surveys.176 We have seen results from the prevalence surveys conducted before and after the Year 1 (2012-13), Year 2 (2013-14), and Year 3 (2014-15) MDAs in Kenya.177 Deworm the World has provided us with details of a) how the baseline survey was conducted and b) pre-registered plans for follow up surveys.178 Note that we have not yet seen detailed descriptions of the methodologies used for the Year 2 and Year 3 surveys, so we have not verified that they were carried out as intended.179

The prevalence surveys are not representative of the full program Deworm the World supports: they are conducted in 20 districts randomly sampled from 66 districts in which STH was assumed to be endemic before Kenya started its national deworming program; we do not believe the 66 districts were selected randomly from all districts in the national program.180 We summarize the key features of the prevalence surveys in this footnote.181

We believe these surveys provide evidence that the deworming program reduced infection rates in target populations, indicating that the program effectively reached targeted children. The tables below show the results of the prevalence surveys for Year 1 (2012-13), Year 2 (2013-14), and Year 3 (2014-15).182

Kenya prevalence survey results - soil-transmitted helminths

	Hookworm	Ascaris (roundworm)	Trichuris (whipworm)	Any soil-transmitted helminth
Year 1 (2012-13) baseline (200 schools)	16.9%	19.2%	5.4%	33.4%
Year 1 (2012-13) post-MDA (70 schools)183	3.2%	2.3%	4.3%	8.7%
Year 2 (2013-14) pre-MDA (60 schools)	4.5%	12.5%	5.1%	19.0%
Year 2 (2013-14) post-MDA (60 schools)	2.2%	1.9%	2.7%	6.0%
Year 3 (2014-2015) pre-MDA (200 schools)	2.3%	11.9%	4.5%	16.3%
Year 3 (2014 -2015) post-MDA (60 schools)	1.8%	2.8%	2.3%	6.3%

Kenya prevalence survey results - Schistosomiasis184

	S. haematobium - Coast Province only185	S. mansoni - all provinces
Year 1 (2012-13) baseline186	14.8%	2.1%
Year 1 (2012-13) post-MDA	Unknown	No comparable data available
Year 2 (2013-14) pre-MDA	10.5%	2.7%
Year 2 (2013-14) post-MDA	7.6%	0.6%
Year 3 (2014-2015) pre-MDA	8.8%	1.5%
Year 3 (2014 -2015) post-MDA	5.8%	0.8%

Schools in Kenya do not all undergo MDAs simultaneously, but the surveys are conducted at approximately the same time before and after each MDA; the post-MDA surveys are conducted 5-6 weeks after an MDA.187 Additionally, it seems possible that surveying the same 60 schools each year could introduce bias: if the schools have an incentive to show that the program is working, they may execute a higher-quality deworming program than they otherwise would if they weren't going to be re-surveyed.188 There is no control group for the prevalence surveys.

How does Deworm the World affect program outcomes?

Deworm the World may be having an impact in the following ways:

1. It may increase the likelihood that a government implements a deworming program, by advocating for deworming programs, by offering to provide technical assistance, and/or by funding implementation.
2. It may improve the quality of a deworming program that would have been implemented without Deworm the World (leading to more children dewormed effectively or improved cost-effectiveness).

Deworm the World expects to pay for the majority of financial program costs in many of the new countries to which it has expanded or intends to expand.189 In situations where Deworm the World is funding a deworming program, we believe it's likely that Deworm the World plays an instrumental role in causing the program to happen.190

Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are uncertain about this.

Does Deworm the World increase the likelihood that governments implement deworming programs?

While we have limited evidence to rely on, we would guess that Deworm the World increases the likelihood that (national and subnational) governments implement deworming programs. In India, state governments seem to heavily utilize Deworm the World when planning, implementing, and monitoring their deworming programs.

In several present and future countries, Deworm the World pays (or expects it may pay) the majority of financial program costs, which increases our expectation that the organization is critical to the program happening.191

Below, we look in detail at what we know of Deworm the World's impact in India, as a light case study of Deworm the World's potential impact. We detail what we know with respect to Deworm the World’s experience in India, including:

1. Our limited understanding of how Indian states have historically decided to launch and maintain health programs with NGO assistance;
2. Deworm the World’s role in the mass school-based deworming programs that were in operation in India before the first national deworming day (NDD) in 2015, including our conversations with government officials in Rajasthan about their decision to launch a deworming program with Deworm the World’s assistance;
3. The national government’s interaction with Deworm the World as part of its decision to launch a national deworming day in India in 2015.

We then present what we know of Deworm the World's impact in several other countries, where our knowledge is more limited.

Reasons new health programs may be started or halted in India

Deworm the World and Children’s Investment Fund Foundation (CIFF) have told us that Indian state governments often receive funds earmarked for broader health programs but often fail to spend these funds.192 We asked the Center for Global Development (CGD) for a recommendation of someone to talk to in order to understand states’ decisions to initiate deworming programs, and CGD referred us to Professor Devesh Kapur.193 Dr. Kapur asserted that it takes significant internal political will or external stimulus (such as from a non-profit) to sufficiently overcome general bureaucratic inertia in India to sustain a new health program. This holds true even in cases where a national mandate exists for a program (as it does for school-based deworming). Dr. Kapur felt that in the majority of cases, nonprofit technical assistance was likely to increase the probability of a program’s going forward.194

Deworm the World told us that it believes that some school health programs have been stalled in Indian states due to negative media attention;195 one of Deworm the World's goals is to prevent these reports in the states in which it works.196 The documentation that Deworm the World has sent us supports—but does not fully demonstrate—the possibility that negative media undermined consistent mass deworming in Assam before the National Deworming Day in 2015. We have not attempted to independently verify that possibility because we do not think we would likely be able to do so effectively. Nonetheless, it is plausible that were decision-makers in Indian states to have the impression that negative reports could cause a program to be halted, they might be less willing to move ahead, and Deworm the World's assurances that it would help prevent these reports could increase the likelihood that a state agrees to implement deworming.

Early deworming programs in India and Deworm the World's role

In 2013 Deworm the World and CIFF told us that the Indian government mandates that all states provide school-based deworming through the larger school-based Weekly Iron and Folic Acid Supplementation program and that states can request funding for deworming through this program.197 At the end of 2013, the only states in India that appear to have been implementing school-based deworming programs were Andhra Pradesh, Bihar, Delhi, Jharkhand, Punjab, Rajasthan, and possibly Assam.198 Deworm the World states that it (or others affiliated with it) played a key role in four of these seven states’ decisions to implement deworming.199 In late 2014, Deworm the World believed that more states were starting to launch their own deworming programs without assistance from Deworm the World but was unsure how many had successfully done so.200

Without involvement from Deworm the World, Assam initiated a vitamin A supplementation and deworming mass drug administration (MDA) in the 2010-2011 school year but reported low and conflicting coverage for that year.201 In the 2012-2013 school year Assam planned to implement a deworming program through the School Health Program, though we are not sure whether that program happened as planned.202 Assam did approach Deworm the World in late 2013 to explore opportunities for the organization to provide assistance to the state, though as of late 2016 Deworm the World has not become directly involved there.203

During our site visit to Deworm the World in Rajasthan, India, we spoke with three government officials who were involved in the deworming program.204 Of these, two stressed the importance of Deworm the World's technical assistance, and the person we perceive as having been most responsible (of the three) for the decision to go forward with deworming gave the impression that the availability of this technical assistance had been a key factor in deciding to go forward. It should be noted that one of the three gave the impression that Deworm the World's help was not needed, and all three conversations took place with multiple Deworm the World representatives present. With that said, the highest ranking of the three officials gave what we felt to be a nuanced and realistic picture of Deworm the World's impact that implied a substantial (while not determinative) role. She stated that a) the immediate availability of technical assistance improved her confidence that the program would proceed quickly and smoothly, b) she wasn’t sure whether the program would have proceeded if not for Deworm the World, and c) she was interested in finding a nonprofit technical assistance partner for at least one other program in a different category.205 This suggests that nonprofit technical assistance can be a key factor in progressing a program.

The National Deworming Day and Deworm the World’s role

In 2014, Deworm the World told us that the Indian national government was pursuing the idea of having a coordinated national deworming day (NDD), whereby the national government would provide some assistance to states in implementing school-based deworming on a single day to encourage more states to implement the program. Deworm the World told us that it initially proposed this idea.206 Deworm the World said that the government asked it to provide technical assistance to the NDD, including helping develop the implementation strategy, designing and developing training and reference materials, community mobilization strategies, and monitoring and evaluation systems and reporting formats.207 The posters, ads for radio and television, training materials, and other documents that Deworm the World helped create for NDD are available at the website in this footnote.208

Deworm the World told us that, as part of its advisory role in the planning process, it advocated for a delay of the initial program from October 2014 to February 2015 because it believed there weren’t sufficient drug supplies.209 The first NDD occurred in February 2015,210 with 12 states participating (with deworming in one additional state occurring in April).211 A second NDD occurred in February 2016, with 30 of India's 36 states and union territories participating.212

Deworm the World's impact on the existence of deworming programs in other countries

In October 2016, we spoke to Deworm the World about whether or not deworming programs would have occurred in some of the other countries it works in without Deworm the World's assistance. Of the three countries we discussed, only one (Nigeria) seemed like a case for Deworm the World causing new deworming programs to exist:213

• Vietnam: There was a national deworming program in Vietnam before Deworm the World started working there, although our impression is that this program did not treat all areas that needed a deworming MDA or treated those areas somewhat sporadically. Deworm the World told us that the RCT it is supporting in Vietnam likely would not have been funded if Deworm the World were not involved.214
• Nigeria: Deworm the World believes that if it had not partnered with RTI in Cross River, Nigeria, RTI most likely would have supported community-based treatments specifically targeting onchocerciasis and lymphatic filariasis in the state; in those local government areas (LGAs) also endemic for schistosomiasis and/or STH, treatment for those would likely have been provided.215 Such a program would not have treated the LGAs in Cross River that are not endemic for LF and/or onchocerciasis but are endemic for schistosomiasis and/or STH.

  There are many funders supporting NTD treatments in Nigeria, and Deworm the World noted that another organization may have supported deworming in Cross River if RTI and Deworm the World had not started to work there. However, many funders are focused on integrated NTD programs, which often prioritize LF and onchocerciasis treatments, so Deworm the World is not sure if other organizations would have ensured that all appropriate LGAs were treated for schistosomiasis and STH.216 Furthermore, there are still funding gaps in Nigeria,217 so it is possible that if another funder had supported Cross River, this would have come at the expense of not covering another state.

• Ethiopia: Ethiopia was already receiving support from SCI before Deworm the World started assisting the national government. SCI and the government were planning a national program;218 this program would have likely occurred without Deworm the World's support.

We discuss how Deworm the World may have influenced the quality of these programs below.

Does Deworm the World's work increase the quality of deworming programs?

Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are highly uncertain about this.

Deworm the World may improve program quality by:

• Increasing the chances that the first deworming round in each state begins earlier than it otherwise would have and that subsequent rounds occur on schedule.219
• Providing training or increasing the training quality by simplifying training material and creating a more robust training program for those who train representatives from each school.220
• Ensuring that support roles are staffed.221
• Improving focus and attention to detail, possibly increasing the likelihood that schools receive the materials and instructions necessary to implement the deworming program.222
• Expanding the scope of the program to a broader age group.223
• Increasing community acceptance of mass treatment and the ability of a program to avoid or withstand publicity associated with related or seemingly-related adverse events.224
• Implementing or advocating for monitoring systems. This may improve program quality by creating a mechanism through which implementers are held accountable. Also, monitoring systems could generate lessons that Deworm the World and the government could use in future rounds of treatment.225
• Advocating for treatment strategies that align with WHO guidelines, which may help to ensure that children are receiving the necessary dosage of drugs on an annual basis.226

Again, to answer this question we have primarily looked at evidence from Deworm the World's program in India. Evidence that relates to Deworm the World’s impact on the quality of deworming programs includes the following, each discussed in more detail below:

• Testimony of a government official in Rajasthan in 2013
• Calls Deworm the World monitors made in India during the run-up to Deworming Day
• A training we observed in 2013

We also note what we have learned from talking to Deworm the World about its programs in other countries.

Testimony of a government official

We believe our strongest piece of evidence in favor of the idea that Deworm the World improves quality is the conversation we had with a Rajasthan nodal officer in 2013,227 who cited many of the points above and made the case that Deworm the World had played an important role in improving the quality of deworming, backing up an intuition that, without external support, such programs would often be of lower quality. On the other hand, (a) this conversation took place with multiple Deworm the World representatives present, and the nodal officer had worked closely with Deworm the World; (b) another government official (on the same visit) stated that he believed Deworm the World's technical help was not needed and did not add value, as the state was accustomed to running school health programs and deworming is a relatively simple one (we do not put strong weight on (b) alone, since the government official may have had other incentives to give the message he did and had not worked as closely with the program as the nodal officer, but in context of the other observations we find (b) worth noting).

Monitoring calls before Deworming Day

As part of its standard monitoring process in India, Deworm the World has tele-callers place thousands of calls to government employees at different levels of government during the run-up to Deworming Day to ensure that preparations are proceeding as intended.228 Deworm the World reports that these calls helped to uncover problems, which were reported on the same day to the appropriate government officials to address.229

It also seems plausible that these calls reduce the likelihood that problems arise after the calls are placed, due to the signal of importance created by the call itself. We are unaware of how common these monitoring calls are as part of other government programs, nor are we aware of any data that could help quantify the size of the impact they have.

Training observed in 2013

We felt that the training we attended in 2013 (the only part of Deworm the World's work that we observed directly) had major limitations in terms of potential to improve program quality.230 Deworm the World has made changes to address these since this visit, though we have not observed a training since that time.231

Deworm the World's impact on program quality in other countries

In October 2016, we spoke to Deworm the World about how it has impacted the quality of programs that it has supported. In general, Deworm the World claims to have improved the quality of the programs it has worked with:232

• Vietnam: In Vietnam, Deworm the World works with the National Institute of Malaria, Parasitology, and Entomology (NIMPE), which had conducted prevalence surveys and MDAs prior to Deworm the World's support. Deworm the World feels that collaboration with NIMPE in the four provinces Deworm the World is supporting has improved NIMPE's processes, although Deworm the World is unsure if NIMPE will adopt its practices more broadly. For example, prior to Deworm the World's support, NIMPE's prevalence surveys only examined a small number of schools in limited areas that were not fully representative of the area being surveyed.233 Additionally, NIMPE did not have procedures in place to train teachers and health workers on how to implement a Deworming Day; Deworm the World assisted with setting up these processes. Deworm the World has also been advocating for Vietnam to follow WHO guidelines when designing its treatment strategy; it hopes that this will make the program more evidence-based in the future.
• Nigeria: In Nigeria, Deworm the World provided expertise on school-based MDA programs. Without its assistance, Deworm the World believes that its partner, RTI, may have started a program in Cross River that only supported community-based programs, which would have missed school-age children in areas where community-based programs were not recommended.
• Ethiopia: Because it works closely with SCI in Ethiopia, Deworm the World believes that it is difficult to assess the impacts attributable to Deworm the World alone. SCI was providing technical assistance to Ethiopia before Deworm the World began to, and Deworm the World believes that SCI was planning to support the treatments of both schistosomiasis and STH prior to Deworm the World's involvement. Deworm the World noted that it leveraged its Kenya team's expertise to assist the Ethiopian government and to facilitate learning exchanges between the two countries.

Are there any negative or offsetting impacts?

We discuss several possible considerations but do not see significant concerns.

Administering deworming drugs seems to be a relatively straightforward program.234 However, there are potential issues that could reduce the effectiveness of some treatments, such as:

• Drug quality: For example, if drugs are not stored properly they may lose effectiveness or expire.235 Our understanding is that Deworm the World periodically tests the quality of drugs and has monitored storage conditions in each of its recent programs, and this information suggests there have been minimal issues.236 In India, state governments are responsible for testing the drugs they use.237
• Dosage: If the incorrect dosage is given, the drugs may not have the intended effect and/or children may experience additional side effects.238 It appears that for STH treatment, all children of a given age group are given the same dose of albendazole and that the dose is generally a single tablet for children 2 years old and above, and half a tablet for those between the ages of 1 and 2.239 Deworm the World monitors reported that, in recent programs in India, up to 9% of schools observed gave children less than the prescribed dose of albendazole and up to 5% gave more than the prescribed dose of albendazole.240
• Concerns over whether treatment is sustained: We believe it is important that deworming programs are sustained over time, as re-infection is rapid and a one-time treatment may have little long-term effect.241
• Replacement of government funding: We have limited information about whether governments would pay for the parts of the program paid for by Deworm the World in its absence, though our impression is that they would not.242 We also have little information about what governments would use deworming funds for if they did not choose to implement deworming programs.
• Diversion of skilled labor: Drug distribution occurs only once or twice per year and is conducted by teachers in schools. Based on our site visit in Rajasthan, our impression is that the Nodal Officer (the state official who manages all state school-based programs), the Nodal Officer’s staff, and the people that the Nodal Officer manages throughout the state (Resource Persons and Community Development Project Officers) have significant capacity to take on additional programs, so their taking on this program doesn't impose a significant burden on their time.243 On the other hand, a principal we spoke with commented that he would prefer fewer school-based health programs because they take focus away from the school day.244
• Adverse effects and unintended consequences of taking deworming drugs: Our understanding is that expected side effects are minimal and there is little reason to be concerned that drug resistance is currently a major issue (more information from our report on deworming).
• Popular discontent: We have heard a couple of accounts of discontent in response to mass drug administration campaigns, including one case that led to riots.245 While the accounts we have heard are from programs supported by the Schistosomiasis Control Initiative, we think it is possible that other deworming programs could cause similar discontent.

What do you get for your dollar?

This section examines the data that we have to inform our estimate of the expected cost-effectiveness of additional donations to Deworm the World. Note that the number of lives significantly improved is a function of a number of difficult-to-estimate factors, many of which we discuss below. We incorporate these into a cost-effectiveness model which is available here.

We focus on the following questions:

• What is the cost per child treated of the deworming programs, and what percentage of this does Deworm the World bear? We estimate that in India children are dewormed for a total of about $0.32 per child, or $0.09 per child excluding the value of teachers’ and principals’ time spent on the program. In Kenya, we estimate the total cost per treatment at about $0.79 per treatment or $0.55 excluding the value of teachers’ and principals’ time spent on the program. We expect the cost per treatment in Deworm the World’s potential new countries to be closer to Kenya's costs; however, we are highly uncertain about this.
• How accurate are Deworm the World's reported coverage figures? Deworm the World uses reported coverage figures from the government when calculating its cost-per-treatment. We remain uncertain of the accuracy of these figures.
• Does Deworm the World "leverage" government funds, such that its activities mobilize resources from other actors? We could imagine that Deworm the World's funds have substantial leverage but could also imagine that other actors’ involvement is causing Deworm the World to pay for things for which other actors would otherwise have paid. Deworm the World may have less leverage in its future programs than it has had in past programs.

Our full cost-effectiveness model is available here.

What is the cost per treatment?

When considering the cost-effectiveness of additional donations, we consider the cost per treatment for the Deworm the World programs for which we have data: India and Kenya. Deworm the World told us that the cost per treatment in India is unusually low; it expects other programs' cost per treatment to be more similar to the program that it supports in Kenya.246 Deworm the World told us that its initial, rough estimates of the cost per treatment in Vietnam and Nigeria look very similar to the costs in Kenya, although these estimates could change significantly as more data comes in.247

Note that in our cost-per-treatment analyses below, we use data that Deworm the World has sent us for its most recent rounds of treatment in Kenya and India for which it has cost data. It is possible that this makes Deworm the World's program look more cost-effective than it actually is; for example, this may exclude start-up costs from our analysis. Deworm the World has told us that it tries to capture all costs of its program, regardless of who pays for the cost. However, we know that it does not include the value of teacher time during trainings and on Deworming Day248 and that there are several high-level costs not directly attributable to programs that Deworm the World does not include (such as exploratory work in new geographies that does not lead to a new program).249 We have included these costs in our analyses.

To see cost-per-treatment figures across multiple years, see our previous reviews or GiveWell analysis of Deworm the World cost-per-treatment, October 2016.250

India

As of September 2016, Deworm the World had estimated the total cost of nine deworming rounds in India.251 These estimates include the costs listed in the following tables. Each of the costs was paid by a combination of government, Deworm the World, and other partners. The following table shows the percentage breakdown for the most recent round of deworming in Bihar, Rajasthan, and Madhya Pradesh.252 Note that it does not include the value of school and government staff time during training, deworming day, and mop-up day.253

Deworm the World and others’ costs in Indian states’ deworming programs: 2014-15254

Cost category	Percentage of total costs	% paid by DtWI
Policy & advocacy	1%	100%
Prevalence surveys	6%	100%
Drug procurement & management	37%	0%
Training & distribution costs	10%	36%
Public mobilization & community sensitization	16%	22%
Monitoring & evaluation	7%	86%
Program management	20%	100%
Additional high-level costs (GiveWell assumption)	4%	100%
Total	100%	44%

Deworm the World has estimated some of the above costs because it did not have access to full cost data for governments and other partners, and boundaries between the cost categories are not always clear.255

The table below shows the costs to Deworm the World, its partners, and the governments, as well as estimates of the cost per child treated, for the most recent rounds of treatment in Bihar, Rajasthan, and Madhya Pradesh.256 About two thirds of the total cost comes from contributions of time from government employees—which have been monetized according to salary levels—rather than financial costs. We would guess that appropriately valuing that time spent, and estimating how much time is spent by teachers and others due to the deworming program, is the largest source of uncertainty in the cost-per-child-treated calculation.257 In its own calculations, Deworm the World excludes the value of government employees' time because the government would have incurred these costs in the absence of the program.258

Total cost per child treated - India259

Expense category	Recent deworming rounds	Cost per child
Deworm the World	$1,602,778	$0.04
Partners	$1,214,409	$0.03
Government financial costs	$864,870	$0.02
Government staff time value	$9,791,115	$0.23
Subtotal without staff time	$3,682,057	$0.09
Total costs	$13,473,172	$0.32

Kenya

We estimate the total cost per treatment in Kenya to be about $0.79 (details below).260

In the third round of treatment in Kenya, we estimate that the total cost per treatment was $0.55 (not including the value of teachers' time).261 Deworm the World's cost per treatment in Kenya is in the same range as our estimate of SCI's cost per treatment, which averages data across nine programs in Africa and three years.262 Our estimate of Deworm the World’s total cost per treatment in Kenya attempts to include all partners’ costs so that it represents everything required to deliver the treatments.263 Our estimate includes the value of teachers' time, even though this does not represent an additional financial cost to the program (because the government pays the teachers with or without the program). In our analysis of SCI, we estimate that 30% of the total program costs are in-kind support from the government.264 Applying the same approach to Deworm the World (because we do not have comparable data for Deworm the World), we estimate that the total cost per treatment in Kenya is $0.79.265

Total cost per child treated - Kenya266

Expense category	Recent deworming rounds	Cost per child
Deworm the World	$3,157,187	$0.50
Partners	$329,688	$0.05
Government financial costs	$14,918	$0.00
Government staff time value	$1,485,851	$0.23
Subtotal without staff time	$3,501,793	$0.55
Total costs	$4,987,644	$0.79

Accuracy of coverage data

In calculating the cost-per-treatment figures above, we use the number of children treated in the programs, which is obtained from data reported by governments to Deworm the World. We have made adjustments to these figures in our cost-per-treatment analysis based on findings from Deworm the World's independent monitoring of treatment numbers in a sample of schools.

India

Comparing classroom data to school data

Monitors visit a random sample of schools to compare each school’s records of how many children it dewormed to the number of children that were dewormed according to its classroom records. Based on this data, it appears that many schools in 2015 did not follow the recording protocol, but the schools that did had reasonable agreement on the number of children dewormed between the classroom records and the school-wide records (details in this footnote).267

In 2016, the ratio of students recorded as dewormed in schools' records to the students the school reported as dewormed ranged from 53% to 94% (53% means that for every 100 students reported being dewormed, only 53 were actually recorded as dewormed when Deworm the World monitors checked schools' forms).268

We have made an adjustment for this in our cost-per-treatment analysis.

Comparing classroom data to state data

It is our understanding that Deworm the World does not regularly monitor the accuracy of the aggregation process beyond the school level for each round of deworming, although it periodically checks this.269 The number of children treated in each school is recorded in classrooms and then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (school, node, block, district, and state) to create a reported coverage estimate for the entire state.270 Errors in the aggregation process could occur if those performing the aggregation have an incentive to overreport the number of children dewormed, or if reporting forms are lost (in which case we believe that deworming would be underreported).271

We calculated the portion of children that were dewormed—according to class records in schools which used class records or didn’t do deworming—at the schools monitors visited. Then, we compared this to the fraction of total students covered out of total students enrolled in government schools (details of methodology in this footnote).272

The results of estimating statewide coverage using both government-reported data and monitors’ data are in the below table.

Estimates of total statewide coverage (India, 2015)273

2015 deworming round	% of students dewormed based on government-reported figures	% of students dewormed based on monitors' observations (of class records)
Bihar	74%	65%
Rajasthan	89%	85%
Madhya Pradesh	90%	68%
Delhi	82%	77%
TOTAL	80%	73%

While we don’t put much weight in the comparison above, we note that a) there are cases in which the proportion of children covered is significantly different, which makes us more uncertain how much weight to put in the coverage figures, and b) the fact that we do not see a pattern of the government consistently over-reporting its coverage figures gives us slightly more confidence in using the coverage figures.

Kenya

We have not looked in-depth at the data that Deworm the World has sent us related to coverage in Kenya. If we take the coverage reporting figures at face value, Kenya was able to treat 83% of its target population of children (in counties participating in the national deworming program).274 According to one recent summary monitoring report from Kenya that we have seen, monitors observed coverage to be 99% in the schools they visited (based on school records).275

As in India, Deworm the World's Kenya team also calculates the ratio of students recorded as being dewormed in classroom records to the students reported as dewormed by the school. In the 2014-15 round (Year 3, the most recent round for which we have data), Deworm the World found that this ratio was 105%, implying that the government had underreported coverage for that round.276 We have made an adjustment using the 105% figure in our cost-per-treatment analysis.

Are donations to Deworm the World leveraged?

We have written before about the complexity involved in trying to understand leverage in charity, and below we lay out the relevant issues worth considering in the case of Deworm the World.

Donors to Deworm the World support only expenses paid by Deworm the World. Generally, Deworm the World tries to work with governments and other funders to create funding arrangements where each partner is contributing some portion of the program's costs. Sometimes, governments pay most of the expenses. For example, Deworm the World's costs in India are only approximately 44% of the overall cost of the program (not including the value of teacher time).277

The role that Deworm the World's funds play in the program is an important consideration in determining the cost-effectiveness of donations. This role could range between:

• High leverage, high cost-effectiveness. Deworm the World's funds could have high leverage by: (a) causing government funds that otherwise would not have been spent or would have been spent on a lower-value program to be used in support of a deworming program; or (b) contributing a relatively small amount of funding to cause a deworming program to run significantly better, thereby reaching many more children than it otherwise would have.
• No leverage, average cost-effectiveness. Deworm the World's funding may allow the program to reach more children at a similar overall cost-per-treatment as it would have otherwise.
• Minimal contributions, low cost-effectiveness. If Deworm the World's work leads to few additional children receiving treatment, it may increase the overall cost-per-treatment.

In the past, we guessed that Deworm the World’s work in India played a role in increasing the likelihood that state governments conduct deworming programs, so we estimated that its leverage on funds used for scaling up India programs could have reasonably ranged from less than 1x (i.e., programs would have taken place without Deworm the World’s involvement) to 4-5x, though we were highly uncertain about this estimate.278 India may have been an especially promising place to achieve leverage because the Indian national government had made money available for state governments interested in implementing deworming, and states may have been more likely to implement deworming with the addition of technical assistance.279

In the countries Deworm the World is starting to work in, governments may have less funding to support deworming. This may cause Deworm the World to pay a higher fraction of the overall cost of the program, making the upside potential for leverage of future donations more limited. For example, we estimate that Deworm the World bears the majority of total program costs in Kenya.280 Deworm the World has told us that it expects the cost per treatment of its future programs to be more similar to its program in Kenya than India, though we have not explicitly asked about whether it expects to pay a similar share of overall program costs as it does in its Kenya program.281

Is there room for more funding?

We believe that there is a 50% chance that Deworm the World will be slightly constrained by funding in the next year and that additional funds would increase the chances that it is able to take advantage of any high-value opportunities it encounters. We estimate that if it received an additional $13.4 million its chances of being constrained by funding would be reduced to 5%.

In short, we calculate this from (more details in the sections below):

• Total opportunities to spend funds productively: We estimate that Deworm the World could productively use or commit between $11.4 million (50% confidence) and $24.4 million (5% confidence) in unrestricted funding in its next budget year. This excludes $6.6 million that is restricted or already allocated to its programs.
• Cash on hand: As of the end of July 2016, Deworm the World held $8.9 million that will be available in 2017: $17.6 million in cash on hand, of which $6.6 million was committed and $2.1 million that it expects to set aside for reserves.
• Expected additional funding: We estimate that Deworm the World will receive an additional $2.2 million in unrestricted funding for its work in 2017.

Below, we also discuss:

• Past spending: To date, Deworm the World's spending has lagged behind revenues as it has worked to start programs in new locations. We see this as an ongoing risk, but not a major reason to reduce support. While it has often deviated from previous plans, we believe its spending decisions have been reasonable and loosely in accordance with our expectations.
• Additional considerations: We have found errors in Deworm the World's financial statements that reduce our confidence that we have a complete, accurate understanding of Deworm the World's financial situation. We also discuss four issues that are material to a determination of Deworm the World’s room for more funding: (a) the possibility of alternative funders of its work in Kenya, (b) its preference for multi-year commitments from donors, (c) its expectation that it will utilize a new operating model when expanding into new countries, and (d) its relationship with Evidence Action, which also has unrestricted funding available.

Available and expected funds

At the end of July 2016, Deworm the World held approximately $17.6 million, of which:282

• $1.5 million was restricted by donors to specific programs
• $5.1 million was unrestricted and committed to projects or set aside for reserves ($1.3 million)
• $11 million was unrestricted and uncommitted, of which Deworm the World expects to set aside $2.1 million for reserves, leaving $8.9 million available

We expect that Deworm the World will receive additional donations over the remainder of 2016 and in 2017 from:

• Donors who are not influenced by GiveWell's research: Deworm the World expects to receive roughly $1.2 million in unrestricted funding.283
• Donors who give based on GiveWell's top charity list, but do not follow our recommendation for marginal funding: GiveWell maintains both a list of all top charities that meet our criteria and a recommendation for which charity or charities to give to to maximize the impact of additional donations, given cost-effectiveness of remaining funding gaps. We estimate that Deworm the World will receive about $1 million from donors who use our top charity list but don't follow our recommendation for marginal donations.284
• Donors who follow GiveWell's recommendation for marginal donations: Our estimate of room for more funding is used to make a recommendation to these donors.

With $8.9 million in available funds on hand and $2.2 million expected in additional funding ($1.2 and $1 million from the first two groups, respectively), we estimate that Deworm the World will have about $11 million available in 2017.285

Deworm the World expects to receive additional restricted funding during the remainder of 2016 and in 2017 for some of its programs; we have not asked for additional detail about this restricted funding because Deworm the World does not expect the funding gaps we are considering to be filled by restricted funding.286

Uses of additional funding

In the table below, we've briefly summarized the details of Deworm the World's funding gaps; further detail follows the table. We discuss our prioritization of these funding gaps more below.

Deworm the World's funding gaps287

Opportunity	Total additional cost (millions USD)	Cumulative funding need (millions USD)	GiveWell's prioritization
Supporting 50% of the costs for 2 years of (a) continuing the national deworming program in Kenya and (b) additional prevalence surveys in Kenya	2.4	Deworm the World will have sufficient funds	Execution level 1
Expansion to 2 new states in India for 3 years	2.9	Deworm the World will have sufficient funds	Execution level 1
Expansion to 3 new states in Nigeria for 3 years	6.1	0.4	Execution level 1
Supporting 50% of the Kenya program and additional surveys for an additional 2 years (4 years total)	2.4	2.8	Execution level 2
Expansion to 1 additional state in India for 3 years	1.2	4.0	Execution level 2
Expansion to 1 province in Pakistan for 2 years	0.5	4.5	Execution level 2
Expansion to Indonesia	4.0	8.5	Execution level 3
Supporting 50% of the Kenya program and additional surveys for an additional year (5 years total)	1.2	9.7	Execution level 3
Expansion to 1 additional province in Pakistan	3.7	13.4	Execution level 3
Total	24.4	13.4

More detail:288

• Kenya: Deworm the World is seeking funding to support an additional five years of the national school-based deworming program in Kenya. The first five years of the program were supported primarily by CIFF, and we discuss the possibility of CIFF continuing funding for the program below. The total cost of the five-year program is projected to be $10.5 million, and Deworm the World would like an additional $1.5 million (total over 5 years) to support prevalence surveys to track the program's impact. It expects other funders to support approximately half of these costs and is looking to fill the remaining half of this gap.
• India: Currently, all states and territories in India participate in National Deworming Day. However, Deworm the World believes that some states could benefit from additional technical assistance to achieve greater coverage and better monitor programs. It believes it can scale up its program to support three additional states in 2017.289 Each state would require approximately $300,000 per year, and Deworm the World would like to be able to commit to three years of support for each state. Additionally, Deworm the World would need funding to support the scale up of its national team to support the state teams.290
• Nigeria: Deworm the World would like to expand its support in Nigeria to three additional states. As of late 2016, it received permission from the government of Nigeria to move forward with its support for three states.291 We estimate that it will need approximately $6.1 million to support all three states for three years.292 These estimates include some national-level costs to support the state programs.
• Pakistan: Deworm the World would like to fund a deworming program in Pakistan, but is waiting on the results of the prevalence survey it recently supported to determine what size of a program Pakistan needs. Preliminary results indicate that Pakistan may have a lower worm prevalence than initially thought, which could lead to a smaller program than expected in Pakistan.293 Conservatively, Deworm the World believes it may only commit to supporting one province in Pakistan in 2017, which would require about $500,000 for two years.294 In the past, Deworm the World told us that being able to commit to multiple years of funding would make negotiations with provinces in Pakistan easier, and that it might be difficult to reach an agreement without having five full years of funding available.295 If Deworm the World were to commit to two provinces in Pakistan, it might need up to $4.2 million in 2017.296
• Indonesia: Deworm the World has started preliminary conversations with Indonesia's government about supporting a deworming program there. It believes that conversations might move forward quickly in 2017 because, unlike in Pakistan, the Indonesian government has an NTD department to coordinate with. Deworm the World is very unsure how large of a program it might support in Indonesia; it estimates between $1.6 and $4 million, but these figures could easily change.
• Reserves: In its room for more funding analysis, Deworm the World noted that it expected to set aside an additional $2.1 million, to add to the $1.32 million it currently holds in reserve. This amount may vary with its total 2017 budget.297 We are unsure about the circumstances under which Deworm the World would make use of those reserves, given that its budget is largely supported by multi-year restricted grants or multi-year allocations from unrestricted funds. We do not believe Deworm the World needs reserves in cases where it already has the funding on hand for a multi-year program, but we believe reserves might be needed for (a) global costs and (b) cases where Deworm the World receives its funding for a program in installments, because historically Deworm the World has experienced cases of installments not being sent in a timely manner.

GiveWell's prioritization of Deworm the World's funding gaps

We have broken down our our top charities' funding gaps and ranked them based on:

• Capacity relevance: how important the funding is for the charity's development and future success.
• Execution relevance: how likely the charity's activities will be constrained if it does not receive the funding.

We believe that "capacity-relevant" gaps are the most important to fill, and "execution"-related gaps vary in importance. More explanation of this model is in this blog post.

In the table above, we have not ranked any of the funding gaps "capacity relevant" because we do not see a strong case that filling those gaps would increase our confidence in Deworm the World's performance (we already believe Deworm the World has a strong track record) or would have an outsized impact on unlocking additional funding opportunities in the near future (in part because Deworm the World makes multi-year agreements).

We consider the funding gaps to be "execution" gaps and assign them a level (1, 2 or 3) by how likely we believe it is that Deworm the World would be constrained by funding (rather than other factors, such as an inability to grow staff capacity quickly enough) if it is unable to fill the funding gap. Level 1 is 50% chance of funding being the constraint, level 2 is 20% chance, and level 3 is 5% chance. These judgements are rough and largely based on a) what Deworm the World has told us about the progress on the various opportunities it is pursuing and b) intuitions formed from following Deworm the World's progress over several years.

Deworm the World has worked in Kenya and India for over 5 years, and we feel confident that Deworm the World will be able to scale up or continue its support in those countries with additional funding. We consider a large portion of the India gap and the first two years of the Kenya gap to be "Execution Level 1" gaps. We also feel moderately confident in Deworm the World making significant progress in Nigeria in 2017; Deworm the World has already successfully supported one state there and has progressed to a late stage in discussions with another three states. So, we consider the funding required to scale up to three additional states in Nigeria to also be "Execution Level 1."

We are less confident in Deworm the World's scale-up in Indonesia and Pakistan. We believe it is somewhat more likely Deworm the World will start to support a program in Pakistan because it has already completed a prevalence survey there, so we have labelled a small portion of the Pakistan gap "Execution Level 2" (i.e., there is only a 20% chance that Deworm the World will be constrained if it does not receive funding for this opportunity).

We think Deworm the World's ambitious growth opportunities are the most uncertain, so we have labelled the full amount of these opportunities (scaling expansively in Pakistan and Indonesia, and funding Kenya for a full 5 additional years) to be "Execution Level 3" gaps.

Past uses of unrestricted funds

In the past, Deworm the World has spent and committed unrestricted funding more slowly than we expected and on different activities than it predicted it would. However, we believe that the alternative uses of unrestricted funding have been reasonable.

Speed of spending

GiveWell started recommending Deworm the World in November 2013; since then (through July 2016), we have influenced about $15.3 million in unrestricted funding to Deworm the World, about $12 million of which was in the last year.298 We estimate that, in the same time period, Deworm the World has raised $2.2 million from other sources.299 So far, Deworm the World has spent $2.3 million of this funding (13&#37) and committed an additional $5.1 million to future projects (total 42&#37 spent or committed).

More details below (we've also laid out the information in this spreadsheet):

• 2013 - 2014: GiveWell first recommended Deworm the World in fall 2013. Over the 2013 giving season and through June 2014, GiveWell-influenced donors gave about $2.3 million to Deworm the World.300 In 2014, Deworm the World only spent $336,000 in unrestricted funding, although in late 2014 it told us it had plans for how it would spend the remainder in the coming years.301
• 2014 - 2015: Between July 2014 and June 2015, GiveWell-influenced donors gave slightly over $1 million to Deworm the World.302 In 2015, Deworm the World spent $743,975 in unrestricted funding.303 In late 2015, it told us that $1.1 million of its unrestricted funding on hand was committed to future projects.304 We estimate that this means that by the end of 2015, Deworm the World had spent or committed approximately 46&#37 of the total unrestricted funding it had raised from late 2013 through June 2015.305
• 2015 - 2016: Between July 2015 and June 2016, GiveWell-influenced donors gave approximately $12 million to Deworm the World.306 As of June 2016, Deworm the World expected that by the end of the year it would have spent about $1.2 million of unrestricted funding in 2016.307 As of July 2016, Deworm the World had committed $5.1 million of its unrestricted funding on hand for future projects.308

Activities funded vs. expectations

Because Deworm the World has spent its funding significantly more slowly than expected, it can be difficult to match up how its spending relates to its past plans. In general, we do not believe Deworm the World has closely followed the plans for spending it has shared with us; however, we believe the nature of Deworm the World's work can require significant shifts in planned spending. We believe that Deworm the World's spending choices have been reasonable.

• 2013 expectations vs. 2014 actual: We detailed Deworm the World's use (and planned use) of the funding it raised in 2013-2014 in this November 2014 blog post. Overall, Deworm the World's funding decisions were roughly in line with our previous expectations.309
• 2014 expectations vs. 2015 actual: At the end of 2014, Deworm the World told us that it would spend additional unrestricted funding on new staff, central costs, expansion to Vietnam, and evaluation of new evidence-based programs related to deworming (details in footnote).310 Including commitments, our impression is that Deworm the World roughly followed this plan.311 It spent less on Vietnam, Kenya, and evaluations, and more on central costs, than expected (details in footnote).312
• 2015 expectations vs. 2016 budget: At the end of 2015, Deworm the World told us that it intended to spend additional unrestricted funding primarily on multi-year deworming programs in Pakistan, Vietnam, and one other new country and on reserves.313 As of late 2016, Deworm the World had committed more than expected to a multi-year program in Nigeria and less than expected to Vietnam, and it had not yet committed to any multi-year programs in Pakistan or another new country.314 We believe this is primarily because Deworm the World's progress has been somewhat slower than expected, not because Deworm the World has significantly changed its plans.315

Considerations relevant to assessing Deworm the World’s room for more funding

Uncertainty on accuracy of financial information

Deworm the World's parent organization, Evidence Action, was created in 2013 to take over two programs incubated at Innovations for Poverty Action (IPA): Deworm the World and Dispensers for Safe Water.316 Our understanding is that there were and still are challenges coordinating grant management across the two organizations. As of October 2016, Evidence Action was in the process of overhauling its financial system.317

In 2016, we spent more time than we had previously on trying to understand Evidence Action's financials, and we found several errors (details in footnote).318 Once these mistakes had been corrected, Deworm the World's records indicated that it held approximately $4.6 million more in available unrestricted funding than the original financial documents had shown.319

While we are more confident in the updated numbers, we are not fully confident that all errors have been corrected.

Alternative funders for Kenya

Deworm the World is seeking funding to support the next five years of a national deworming program in Kenya. Deworm the World currently has a five-year grant from CIFF (and a smaller amount from the END Fund) to support the program; the CIFF grant is ending in June 2017.320 It is unclear to us whether CIFF will continue providing funding for the program and, if so, for how long. Due to this uncertainty, we are not sure if GiveWell-influenced funding is needed to support the Kenya program. However, Deworm the World has told us that the government of Kenya has requested its support in fundraising for the next five years of the program.321

Deworm the World has also told us that the END Fund and other donors are interested in funding a portion of the program. Deworm the World expects these funders could provide about half of the funding for the program. We have some uncertainty about whether 50% is the right division because (a) the END Fund told us that it was interested in providing $1.5 million for the first year of the program (out of $2.4 million), but it might not have enough funding to do so,322 and (b) based on private conversations with donors, we think that the support of some of the "other donors" may hinge on GiveWell's recommendation and therefore should possibly be included in our estimate of room for more funding.

Due to the possibility that Deworm the World's unrestricted funding may displace funding from CIFF, and, to a lesser extent, the END Fund and other donors, we consider the opportunity to fund the Kenya program to be less cost-effective in expectation than it would be if we were confident in the size of the gap. We have included the first two years of the program as an execution level 1 funding gap because our understanding is that Deworm the World will prioritize funding this program over most other opportunities.

Multi-year commitments

Deworm the World has told us that it prefers to have enough funding to make 3-5 year commitments when attempting to launch a new program. It told us that governments typically ask for multi-year commitments because a) deworming programs must be sustained over time to cause the desired impacts and b) governments want assurance that support will be sustained so that they can better plan how to use their funding.323 As a result, Deworm the World expects that having multiple years of allocated funding makes partnerships with governments (formalized by signed memoranda of understanding (MOUs)) more likely and reduces the amount of time before programs are launched.324

Additionally, Deworm the World has told us that having multiple years of funding for its programs allows staff to spend less of their time fundraising and more time on other aspects of their work.

We are not sure that 3-5 year commitments are necessary, but we do not feel that Deworm the World’s preference is unreasonable.

Challenges of expanding to new countries

Launching programs in new countries may introduce challenges that are hard to predict, such as differences in cultural and bureaucratic expectations, increased difficulty of predicting program costs and success, and political and economic instability. So far, we believe that Deworm the World has done reasonably well supporting programs in new geographies, although, because these programs are new, we have only seen limited data from them. We believe Pakistan may be a more challenging and costly environment to work in than the other countries Deworm the World has previously entered.

Deworm the World Initiative and Evidence Action

Deworm the World Initiative is led by Evidence Action. Evidence Action supports other programs in addition to Deworm the World.325

This has some implications relevant to Deworm the World’s room for more funding: donations to Evidence Action, even if restricted to Deworm the World, might change the actions that staff take to fundraise (i.e., which grants they pursue, what type of funding they ask for). We've seen some evidence that this is the case:

• In early 2015, Evidence Action’s plan for using unrestricted funds included a relatively high priority to spend $0.8 million on Deworm the World. After receiving funds related to GiveWell’s recommendation that were designated for Deworm the World, Evidence Action allocated unrestricted funding to other programs instead of to Deworm the World. As such, it seems likely that $0.8 million of GiveWell directed funds (70% of GiveWell-directed funds to Deworm the World that year) caused on the margin more funding to Evidence Action’s other programs, rather than more dewormings to take place.326
• In early 2016, Evidence Action's plans for the first $1.55 million in unrestricted funding it raised included spending $100,000 in unrestricted funding on Deworm the World, $600,000 on Dispensers for Safe Water, $300,000 on Evidence Action Beta, and the rest on organizational development.327 The next $1.5 million raised after the first $1.55 million was expected to go towards reserves.328 We are not sure how this compares to Evidence Action's plans before the 2015 giving season, which were not shared with us. We note that Deworm the World was slated to receive significantly less than Dispensers for Safe Water, despite being a similarly sized program. It seems plausible to us that had Deworm the World not received so much unrestricted funding from GiveWell-influenced donors over the 2015 giving season, Evidence Action would have planned to allocate more of its expected unrestricted funding to Deworm the World.
• In late 2016, Evidence Action shared with us a rough estimate of how it planned to allocate (or already had allocated) $1.9 million in unrestricted funding in 2016.329 For the $1.9 million, $163,000 (8%) was expected to go to Deworm the World, which is slightly more than the portion expected in early 2016.330 However, no funding was allocated to Evidence Action Beta.331 While we are not sure why this is the case, it is possible that Evidence Action chose to use its funding on other programs after Good Ventures made a grant to Evidence Action Beta in early 2016.332 Evidence Action also planned to allocate significantly more than expected to Dispensers for Safe Water ($1 million, compared to $0.6 million originally planned).333 The reallocation of Evidence Action Beta funding, which served primarily to increase funding to Dispensers for Safe Water, again indicates that GiveWell-influenced funding may be impacting how Evidence Action chooses to use its unrestricted funding.

Evidence Action also shared with us a rough estimate of how it would allocate $1.8 million in unrestricted funding that it expects to have in 2017: $600,000 is expected to be allocated to Deworm the World while $500,000 is expected to go to Dispensers for Safe Water and $100,00 to Evidence Action Beta (the rest is for organizational development).334 Deworm the World also notes that many of Evidence Action's investments in general organizational development have benefited Deworm the World, as well as Evidence Action's other programs, substantially.335

Global need for treatment

There appears to be a substantial unmet need for STH and schistosomiasis treatment globally.

In 2016, the WHO released a report on 2015 treatments stating that:336

• 63% of school-age children in need of treatment were treated for STH in 2015. This is a large increase over WHO's report for 2014, which reported 45% coverage.337
• 42% of school-age children in need of treatment were treated for schistosomiasis in 2015.

We have not vetted this data.

Deworm the World as an organization

We believe that the Deworm the World Initiative, led by Evidence Action, is a strong organization:

• Track record: Deworm the World has a track record of assisting governments with deworming programs.
• Self-evaluation: Deworm the World collects a large amount of relevant data about its programs, demonstrating a commitment to self-evaluation.
• Communication: Deworm the World has generally communicated clearly and directly with us, given thoughtful answers to our critical questions, and shared significant, substantive information.
• Transparency: Deworm the World is very transparent.

More on how we think about evaluating organizations at our 2012 blog post.

Sources

Document	Source
Alderman et al. 2006	Source (archive)
Allen and Parker 2011	Source (archive)
Alix Zwane conversation August 30th 2013	Unpublished
Alix Zwane conversation June 4th 2013	Source
Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013	Unpublished
Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013	Unpublished
Assam 2010 guidelines for deworming	Source
Assam midday meal report 2013	Source (archive)
Assam reproductive and child health 2011-2012	Source (archive)
Assam state programme implementation plan 2011-2012	Source (archive)
Baird et al 2012	Source
Bleakley 2007	Source (archive)
CIFF conversation September 10th 2013	Source
Croke 2014	Source (archive)
Devesh Kapur conversation October 14th 2013	Source
Deworm the World and SCI, Ethiopia coverage survey	Source
Deworm the World staff, conversations with GiveWell, October 3-4, 2016	Unpublished
Deworm the World, Bihar 2016 IMCV report	Source
Deworm the World, Chhattisgarh 2016 IMCV report	Source
Deworm the World, Chhattisgarh prevalence survey report, August 2016	Source
Deworm the World, Kenya Coverage Reporting data, Year 3	Source
Deworm the World, Kenya Deworming Day data, Year 3	Source
Deworm the World, 2015 expense summary	Unpublished
Deworm the World, 2015 expense summary - by funder	Unpublished
Deworm the World, Ethiopia independent monitoring report, Year 1	Source
Deworm the World, Ethiopia prevalence survey report	Source
Deworm the World, Ethiopia Workplan	Unpublished
Deworm the World, Vietnam final report for STH survey in 21 provinces	Source
Deworm the World, Kenya 2014-2015 program report	Source
Deworm the World, Kenya Narrative Report - Year 1	Source
Deworm the World, Kenya Narrative Report - Year 2, Quarter 4	Source
Deworm the World, Kenya Narrative Report - Year 3, Quarter 3	Source
Deworm the World, Kenya process monitoring report, Year 4	Source
Deworm the World, Kenya Year 2, DD - Main instrument	Source
Deworm the World, Kenya Year 3, DD - Main instrument	Source
Deworm the World, Kenya Year 2, Pre DD - School instrument	Source
Deworm the World, Kenya Year 3, Pre TT form	Source
Deworm the World, Kenya Year 3, Post DD - Coverage instrument	Source
Deworm the World, Madhya Pradesh 2015 prevalence survey report	Source
Deworm the World, Madhya Pradesh cost-per-treatment - 2015	Unpublished
Deworm the World, Madhya Pradesh 2016 IMCV report	Source
Deworm the World, National Deworming Day states, August 2016	Source
Deworm the World, Rajasthan 2016 IMCV report	Source
Deworm the World, target populations and parasitology data	Source
Deworm the World, Telengana 2016 IMCV report	Source
Deworm the World, Tripura 2016 IMCV report	Source
Deworm the World, Uttar Pradesh prevalence survey report, December 2015	Source
Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day	Source
Deworm the World, Vietnam 2016 monitoring survey form for coverage validation	Source
Deworm the World, Vietnam Independent Monitoring Report, 2016	Source
Deworm the World, Vietnam baseline prevalence survey - 4 provinces	Source
Deworm the World 2015 Uttar Pradesh prevalence survey report	Source
Deworm the World, Uttar Pradesh 2016 IMCV report	Source
DSW 2012 GiveWell site visit	Source
DtWI 2013 GiveWell government interviews	Source
DtWI 2013 GiveWell site visit	Source
DtWI Assam research 2013	Source
DtWI Bihar 2011 cost data	Source
DtWI Bihar 2011 coverage data	Source
DtWI Bihar 2011 monitoring data for deworming day	Source
DtWI Bihar 2011 monitoring data for mop-up day	Source
DtWI Bihar 2011 Monitoring Form for Deworming Day	Source
DtWI Bihar 2011 monitoring form for mop-up day	Source
DtWI Bihar 2011 prevalence survey report	Source
DtWI Bihar 2011 program report	Source
DtWI Bihar 2012 cost data	Source
DtWI Bihar 2012 cost data details	Unpublished
DtWI Bihar 2012 coverage data	Source
DtWI Bihar 2012 monitoring data for coverage validation	Source
DtWI Bihar 2012 monitoring report	Source
DtWI Bihar 2014 cost data	Unpublished
DtWI Bihar 2014 program report	Source
DtWI Bihar 2014 program report annex 1	Source
DtWI Bihar 2014 program report annex 2	Source
DtWI Bihar 2015 independent monitoring tables	Source
DtWI Bihar 2015 monitoring data for coverage validation, schools	Source
DtWI Bihar 2015 monitoring data from deworming day, schools	Source
DtWI Bihar 2015 monitoring data from mopup day, schools	Source
DtWI Bihar 2015 monitoring survey for coverage validation, schools	Source
DtWI Bihar 2015 monitoring survey from deworming day, schools	Source
DtWI Bihar 2015 monitoring survey from mopup day, schools	Source
DtWI Bihar 2015 Prevalence Survey report	Source
DtWI Bihar 2015 Program report	Source
DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015	Unpublished
DtWI Chhattisgarh 2015 coverage validation report	Source
DtWI Chhattisgarh 2015 coverage validation tables	Unpublished
DtWI Chhattisgarh 2015 independent monitoring tables	Source
DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis	Source
DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools	Source
DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis	Source
DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools	Source
DtWI class register audits 2013	Source
DtWI cost narrative 2013	Source
DtWI Cost per treatment blog post January 2015	Source (archive)
DtWI cost per treatment summary 2013	Source
DtWI coverage data 2013 - 2014	Source
DtWI Delhi 2012 cost data	Source
DtWI Delhi 2012 coverage data by anganwadi	Source
DtWI Delhi 2012 coverage data by school	Source
DtWI Delhi 2012 coverage report	Source
DtWI Delhi 2012 monitoring data	Source
DtWI Delhi 2012 monitoring form deworming day	Source
DtWI Delhi 2012 prevalence survey design	Source
DtWI Delhi 2012 prevalence survey report	Source
DtWI Delhi 2012 program report	Source
DtWI Delhi 2013 cost data	Unpublished
DtWI Delhi 2013 program report	Source
DtWI Delhi 2015 independent monitoring tables	Source
DtWI Delhi 2015 monitoring data for coverage validation, anganwadis	Source
DtWI Delhi 2015 monitoring data for coverage validation, schools	Source
DtWI Delhi 2015 monitoring data from deworming day, anganwadis	Source
DtWI Delhi 2015 monitoring data from deworming day, schools	Source
DtWI Delhi 2015 monitoring data from mopup day, anganwadis	Source
DtWI Delhi 2015 monitoring data from mopup day, schools	Source
DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis	Source
DtWI Delhi 2015 monitoring survey for coverage validation, schools	Source
DtWI Delhi 2015 monitoring survey from deworming day, anganwadis	Source
DtWI Delhi 2015 monitoring survey from deworming day, schools	Source
DtWI Delhi 2015 monitoring survey from mopup day, anganwadis	Source
DtWI Delhi 2015 monitoring survey from mopup day, schools	Source
DtWI Delhi 2015 program report	Source
DtWI Kenya 2013-2014 cost per treatment data	Source
DtWI Kenya 2013-2014 program report	Source
DtWI Madhya Pradesh 2015 coverage validation form	Unpublished
DtWI Madhya Pradesh 2015 deworming day monitoring form	Unpublished
DtWI Madhya Pradesh 2015 independent monitoring tables	Source
DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools	Source
DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools	Source
DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools	Source
DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools	Source
DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools	Source
DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools	Source
DtWI Madhya Pradesh 2015 program report	Source
DtWI Monitoring Improvements 2014	Source
DtWI NDD blog post February 2015	Source (archive)
DtWI NDD Year 1 M&E review July 2015	Source
DtWI Rajasthan 2012 cost data	Source
DtWI Rajasthan 2012 cost data details	Unpublished
DtWI Rajasthan 2012 coverage data for anganwadi	Source
DtWI Rajasthan 2012 coverage data for schools	Source
DtWI Rajasthan 2012 monitoring data for coverage validation in anganwadis	Source
DtWI Rajasthan 2012 monitoring data for coverage validation in schools	Source
DtWI Rajasthan 2012 monitoring form coverage day	Source
DtWI Rajasthan 2012 monitoring form pre-deworming day	Source
DtWI Rajasthan 2012 monitoring report	Source
DtWI Rajasthan 2012 prevalence survey report	Source
DtWI Rajasthan 2013 cost data	Unpublished
DtWI Rajasthan 2013 prevalence survey report	Source
DtWI Rajasthan 2013 program report	Source
DtWI Rajasthan 2015 independent monitoring tables	Source
DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis	Source
DtWI Rajasthan 2015 monitoring data for coverage validation, schools	Source
DtWI Rajasthan 2015 monitoring data from deworming day, schools	Source
DtWI Rajasthan 2015 monitoring data from mopup day, schools	Source
DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis	Source
DtWI Rajasthan 2015 monitoring survey for coverage validation, schools	Source
DtWI Rajasthan 2015 monitoring survey from deworming day, schools	Source
DtWI Rajasthan 2015 monitoring survey from mopup day, schools	Source
DtWI Rajasthan 2015 program report	Source
Evidence Action 2014 budget	Unpublished
Evidence Action, 2015 financials by program	Unpublished
Evidence Action, blog post, January 8, 2015	Source (archive)
Evidence Action, blog post, January 16, 2015	Source (archive)
Evidence Action, blog post, June 12, 2015	Source (archive)
Evidence Action, blog post, December 21, 2015	Source (archive)
Evidence Action, blog post, April 27, 2016	Source (archive)
Evidence Action, blog post, June 30, 2016	Source (archive)
Evidence Action, blog post, July 5, 2016	Source (archive)
Evidence Action, blog post, August 1, 2016	Source
Evidence Action, Projected allocation of unrestricted funds, 2016	Source
Evidence Action 2015 draft budget	Unpublished
Evidence Action 2015 funding gap analysis	Source
Evidence Action cover letter 2013	Source
Evidence Action launch announcement 2013	Source (archive)
Evidence Action Q1 financials, 2016	Unpublished
Evidence Action website 2013	Source (archive)
Evidence Action website announcement April 2014	Source (archive)
Evidence Action website, Deworm the World Initiative (October 2015)	Source (archive)
Evidence Action website, Deworm the World Initiative (March 2016)	Source (archive)
Evidence Action website, Deworm the World Initiative (December 2016)	Source (archive)
Evidence Action website, Evidence Action Beta (October 2015)	Source (archive)
Evidence Action website, Who we are (November 2016)	Source (archive)
GiveWell analysis of Deworm the World 2014 Financial summary	Source
GiveWell analysis of Deworm the World cost per treatment	Source
GiveWell analysis of Deworm the World cost-per-treatment, 2016	Source
GiveWell analysis of Deworm the World cost-per-treatment, October 2016	Source
GiveWell analysis of Deworm the World financials - 2016	Source
GiveWell DtWI 2013-2014 cost data summary	Source
GiveWell enrollment-based student coverage check 2015	Source
GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014	Source
GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013	Source
GiveWell's notes from site visit to India, October 2013	Source
GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013	Source
GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014	Source
GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014	Unpublished
GiveWell's non-verbatim summary of a conversation with END Fund staff, October 17, 2016	Source
GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015	Source
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015	Unpublished
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015	Source
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015	Unpublished
GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014	Source
GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014	Source
GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015	Source
Global Alliance to Eliminate Lymphatic Filariasis - Prevention	Source (archive)
Grace Hollister conversation June 19th 2013	Source
Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016	Unpublished
Grace Hollister, conversation with GiveWell, April 20, 2016	Unpublished
Grace Hollister, conversation with GiveWell, May 5, 2016	Unpublished
Grace Hollister, conversation with GiveWell, June 13, 2016	Unpublished
Grace Hollister, conversation with GiveWell, July 25, 2016	Unpublished
Grace Hollister, conversation with GiveWell, August 11, 2016	Unpublished
Grace Hollister, conversation with GiveWell, August 24, 2016	Unpublished
Grace Hollister, conversation with GiveWell, September 1, 2016	Unpublished
Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015	Unpublished
Grace Hollister, Deworm the World Director, email to GiveWell, March 23, 2015	Unpublished
Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013	Unpublished
Grace Hollister, email to GiveWell, March 6, 2016	Unpublished
Grace Hollister, email to GiveWell, June 9, 2016	Unpublished
Grace Hollister, email to GiveWell, September 13, 2016	Unpublished
Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016	Source
Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016	Unpublished
Grace Hollister, edits to GiveWell's review, November 7, 2016	Unpublished
Grace Hollister, edits to GiveWell's review, November 20, 2016	Unpublished
Harvard Business School Kenya Case Study A 2010	Unpublished
India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015	Source
India NDD documents 2015	Source (archive)
Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014	Unpublished
JPAL CEAs in education 2011	Source
Kabatereine et al. 2001	Source (archive)
KEMRI prevalence report - Year 2	Unpublished
LF treatment coverage 2015	Source (archive)
LF treatment drugs 2012	Source (archive)
Miguel and Kremer 2004	Source
Mwandawiro et al. 2013	Source (archive)
Neetu Chandra Sharma, Daily Mail - India article, August 8, 2016	Source (archive)
Paul Monaghan, conversation with GiveWell, September 8, 2016	Unpublished
Paul Byatta, conversation with GiveWell, September 20, 2016	Unpublished
Paul Byatta, attachments to email to GiveWell, September 23, 2016	Source
Preventive chemotherapy in human helminthiasis 2006	Source (archive)
Professor Devesh Kapur Biography 2013	Source (archive)
Reserve Bank of India, GDP per capita, Table 10, September 16, 2015	Source (archive)
SCI Malawi coverage survey 2012	Source
STH coalition framework for action November 2014	Source (archive)
U-DISE Elementary Thematic Maps 2015	Source (archive)
U-DISE Secondary Flash Statistics 2015	Source (archive)
U-DISE Secondary Thematic Maps 2015	Source (archive)
WHO, Helminth control in school-age children	Source
WHO, Helminth control in school-age children second edition	Source
WHO soil-transmitted helminthiases 2012	Source (archive)
WHO, Summary of global update on preventive chemotherapy implementation in 2015	Source
WHO STH factsheet	Source (archive)
WHO STH treatment report	Source (archive)
WHO Weekly epidemiological record, 6 March 2015	Source (archive)
WHO Weekly epidemiological record, 18 December 2015	Source
World Schistosomiasis Risk Chart 2012	Source

• 1
  • GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013
  • Alix Zwane conversation June 4th 2013
  • Grace Hollister conversation June 19th 2013
  • Alix Zwane conversation August 30th 2013
  • Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013
  • GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013
  • GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014
  • GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014
  • GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014
  • GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015
  • GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015
  • In 2016, we deprioritized publishing notes from our conversations with Deworm the World.

• 2

  CIFF conversation September 10th 2013

• 3

  This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.

• 4
  • "More specifically, Evidence Action advocates for school-based deworming to policymakers and provides technical assistance to launch, strengthen and sustain school-based deworming programs." Evidence Action website, Deworm the World Initiative (March 2016)
  • An example of Deworm the World supporting a program with funding: "Deworm the World is working in partnership with Thrive Networks in Vietnam on an integrated program of both deworming and water, sanitation, and hygiene (WASH) education, and this includes an RCT to explore the impact of hygiene education in combination with deworming. This program is funded primarily by Dubai Cares, with Deworm the World slated to provide unrestricted funding for certain program components." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 5.

• 5
  • "Is mass treatment justified? On cost-effectiveness grounds we believe that it clearly is, as the cost of treatment is cheaper than individual screening. The WHO states that the cost of screening is four to ten times that of the treatment itself. Because the drugs are very safe and has [sic] no side effects for the uninfected, the WHO does not recommend individual screening. The WHO instead recommends mass drug administration in areas where more than 20% of children are infected." Evidence Action website, Deworm the World Initiative (March 2016)
  • Deworm the World focuses on school-based programs because the highest burdens for STH and schistosomiasis (the two diseases that Deworm the World targets) tend to be observed in children. For this reason, a significant decrease in the worm burden in children for these diseases translates to a significant decrease in the burden across an entire community. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 6
  • "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis. Deworm the World is one of the only organizations focused on STH (while there are other programs that focus on schistosomiasis), and there is significant room to scale-up [sic] STH programs. In some ways, schistosomiasis has an even larger gap to fill than STH. If Deworm the World shifted its focus to include schistosomiasis, it might widen the existing STH gap. Additionally, Deworm the World specializes in school-based deworming, which is not the ideal approach in every situation." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5.
  • This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.

• 7

  "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5.

• 8
  • "We work with governments around the world to eliminate the public health threat of worms through scaling up school-based mass deworming programs." Evidence Action website, Deworm the World Initiative (December 2016)
  • For example, Deworm the World is partnering, or planning to partner, with local organizations in Pakistan, Vietnam, and Nigeria:
    • Nigeria: "Deworm the World is in discussions with a potential partner which plans to work in Cross River on other integrated NTD treatment. The need to scale up treatment for schistosomiasis and STH among school-age children has not yet been addressed, and Deworm the World has been in discussions with this partner, the state NTD coordinator, and other state officials about creating a school-based deworming program to treat both STH and schistosomiasis beginning in 2016." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, pg. 6.
    • Vietnam: "[In Vietnam] Deworm the World’s partnership with both the government and Thrive Networks is a new working model for Deworm the World; elsewhere, it has supported government implementation or run its program independently. Deworm the World has only one staff member in the country. Dubai Cares provides most of the program funding. All funding goes to Thrive Networks, which provides money to the government for implementation expenses." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 3.
    • Pakistan: "[In Pakistan, Deworm the World] plans to contract with a local organization to do these [prevalence] surveys, but does not yet have a signed agreement...Deworm the World plans to work in partnership with the same local organization to provide technical support, likely beginning in Punjab and later expanding to Sindh." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.

• 9

  For example, Deworm the World has considered supporting evaluations or monitoring of different deworming-related programs:

  • "If funding permits, the Kenyan government may begin a lymphatic filariasis treatment program along its coast. If it does so, Deworm the World will provide process monitoring and coverage validation for the treatments." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 3.
  • "We are contributing approx $111k to the TUMIKIA and TakeUp studies, complementary studies leveraging the Kenya program to look at the potential for breaking STH transmission." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Deworm the World has told us that it is also interested in opportunistically evaluating new evidence-based programs that may efficiently complement deworming. GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014

• 10

  "In January 2007, the [Young Global Leaders] launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs." Harvard Business School Kenya Case Study A 2010, Pg 7.

• 11
  • Deworm the World has supported deworming activities in India since 2009, in Kenya since 2012, and in Ethiopia since 2014:
    • "[Where We Work, Bihar State, India]: 1st deworming round in 2011 reached 17 million children." Evidence Action website, Deworm the World Initiative (March 2016)
    • "[Where We Work, Kenya] With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively." Evidence Action website, Deworm the World Initiative (March 2016)
    • Deworm the World supported a pilot deworming program in Ethiopia in April 2015 and another deworming program in October and November of 2015. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Deworm the World supported treatments in Cross River, Nigeria in mid 2016: "This week, the Cross River State Ministry of Health’s Neglected Tropical Diseases (NTD) unit launched its inaugural statewide school-based deworming exercise that will treat against two neglected tropical diseases that are particularly common in children: schistosomiasis and soil-transmitted helminthiasis (STH). The school-based deworming exercise will cover 11 of the 18 local government areas in Cross River for the first time, and is targeting 600,000 at-risk school-aged children in primary and junior secondary public and private schools. Other NTDs endemic to the state (lymphatic filariasis and onchocerciasis) will be treated through a community-based approach, according to standard practice." Evidence Action, blog post, June 30, 2016
  • Deworm the World supported treatments in Vietnam in mid 2016: "On April 28 and 29, more than 700,000 primary school children across four provinces in northern Vietnam will line up in their classrooms to receive a deworming tablet. 8.5 million children in Vietnam are at risk of parasitic worm infections that can harm their health, development, and school participation. Evidence Action’s Deworm the World Initiative supports the Government of Vietnam as it strengthens and improves school-based deworming to keep children healthy and in school. Between now and 2018, the program will distribute more than four million treatments to combat worms in the four provinces." Evidence Action, blog post, April 27, 2016
  • Updates on activities in India, Kenya, and Ethiopia were discussed in various conversations; for example, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015
  • Plans for Nigeria, Pakistan, and Vietnam were discussed here: GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015
  • Progress in Pakistan was delayed after Kato Katz kits for the planned prevalence survey did not arrive in time. The prevalence survey was rescheduled to begin in August 2016. Deworm the World is waiting on the results of the upcoming prevalence survey to move forward with a treatment strategy. Grace Hollister, conversation with GiveWell, August 11, 2016

• 12

  See the descriptions of programs by country below.

• 13

  "The government is fully responsible for program implementation, and … these programs leverage thousands of govt personnel from health and education to be able to run." Grace Hollister, email to GiveWell, June 9, 2016

• 14

  "Note that there are not single deworming days in all countries, nor do all countries have a mop-up day. In all cases there are mass campaigns, but the structure varies by country." Grace Hollister, email to GiveWell, June 9, 2016

• 15
  • This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.
  • For example, Deworm the World has assisted in a number of areas in India:
    • "Andhra Pradesh...
      Deworm the World’s contributions
      • Prevalence survey...
      • Operational support
        • Helped government develop operational plans and budgets
        • Coordinated cross-sectoral partners through the establishment of a State School Health Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS)
        • Coordinated drug donation made by Feed the Children
        • Designed a monitoring and evaluation (M&E) system
        • Created government tableau for community awareness
      • Trainings
        • Conducted a master training session for program
        • Designed training cascade for the master trainees to train the rest of the implementers
        • Designed training materials
        • Developed materials and campaigns for community sensitization
    • Bihar…
      Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andhra Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO…
    • Delhi…
      In addition to the standard contributions (see Andhra Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children.
    • Rajasthan…
      DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO."

    @Grace Hollister Conversation June 19th 2013@, Pg 1-4.

• 16

  We have matched our descriptions to Deworm the World's standard categorization, albeit in a slightly different order.

• 17
  • "As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program."
    @Alix Zwane Conversation June 4th 2013@, Pg 2.
  • "Deworm the World does not yet have an agreement with the government in Pakistan to conduct the surveys, but hopes to accomplish this in the next month, and anticipates that the prevalence surveys will be conducted beginning in January or February of 2016. It is expected that a clear articulation of need will be an important factor in building a strong case to the government in favor of deworming programs, and it may be best to wait until the results of the surveys are available in the second quarter of 2016 before beginning discussions with the government on a scaled school-based program. Treatment may not begin until 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.
  • Deworm the World's advocacy often occurs side-by-side with Deworm the World's technical assistance; once Deworm the World proves that a deworming program can be well-executed, it is easier to interest national governments in funding deworming programs. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 18

  For example:

  • "DtW has been involved in deworming programs in four different states, and is still actively involved of [sic] three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement...In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. [Note: these notes contain additional examples of Deworm the World's advocacy for other states in India]
  • "Together, we suggested to the Federal Ministry of Health that they expand this initial work. What would it take to treat at least 75% of all at-risk school-age children in the country and to launch a truly national program?

    SCI helped Oumer Shafi, the committed and action-oriented Coordinator for Neglected Tropical Diseases in the Federal Ministry of Health, develop a detailed action plan. This entailed sophisticated statistical analysis to determine how many deworming sites would be required to reach at least 80% of kids at risk.

    Meanwhile, I worked closely with Birhan Mengistu, an up-and-coming leader seconded from the World Health Organization, and with other Ministry of Health staff. We sat for hours hunched over laptop screens to develop detailed five-year budgets, talking through row after row of spreadsheets and reviewing everything from the cost of fuel for drug transport to the needs of teachers.

    Together with the Federal Ministry of Health, we were able to think and act boldly. We are excited to continue to partner with SCI and are seeking other partners who also share common goals and values to rapidly scale school-based deworming in endemic countries.

    ...When we floated the idea of vastly increasing the scope of the originally proposed deworming rounds to be a truly national plan treating upwards of 75% of all children at-risk, Shafi didn’t flinch."
    Evidence Action, blog post, June 12, 2015

  • Other advocacy activities can include discussing: "how deworming can fit into the current policy environment and policy priorities of a government, how such a program can/should be financed, the robust evidence of impact, how a country can best take advantage of WHO drug donations, encourage program champions within government, help establish program governance structures. Once a program is established, advocacy doesn’t end – we work with govts to ensure the continuation of the above. Typically we refer to this group of activities as policy and advocacy, because there is a heavy emphasis on the former." Grace Hollister, email to GiveWell, June 9, 2016
  • These discussions also provide opportunities for Deworm the World to assess how well a deworming program with the government might run. If Deworm the World discovered from its advocacy discussions that there were high rates of teacher or student absenteeism, then it might conclude that a school-based deworming program may not work in the country. Deworm the World assesses risks like this through a diagnostic survey of the country’s capacity, including school attendance rates, which must be sufficiently high if a school-based deworming program is to succeed. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 19

  Deworm the World builds strong working relationships with governments to try to ensure that its programs will be effective, and it will not commit to a program if it does not foresee success in that country. It can decide to abandon plans for a program before a memorandum of understanding (MoU) is signed. In one state in India, Jharkhand, Deworm the World explored a program, but did not build a strong working relationship with the government, so Deworm the World pulled out of discussions before discussing an MoU or investing much money. The discussion stage with governments is important for helping Deworm the World assess the government's position and viability as a partner. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 20

  "The global STH community has changed significantly in the last couple of years, especially due to the formation of the STH Coalition. The community is now prioritizing STH (in a way similar to how LF became prioritized with the formation of the Global Alliance to Eliminate Lymphatic Filariasis, which has seen significant success).
  The STH community is developing plans to scale-up [sic] treatment, especially in high burden countries such as:

  • Ethiopia
  • Nigeria
  • India
  • Pakistan
  • The Democratic Republic of the Congo
  • Indonesia
  • The Philippines
  • Tanzania

  As part of the STH Coalition, Evidence Action is chairing a working group on school-age children. It has used some of its unrestricted funding to hire consultants to create ‘snapshots’ of each country, including obstacles, gaps, potential strategies, and financial needs.

  Deworm the World expects to see an increase in partnerships between the various groups in the STH community. Deworm the World hopes to leverage partnerships with existing organizations in, e.g., Nigeria, Pakistan, Ethiopia, etc., to provide catalytic support (rather than opening its own offices in those places)." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 6.

• 21
  • "We work with epidemiologists and local partners to assess worm prevalence and intensity, obtaining data to develop a targeted treatment strategy and to determine parasitological impact once programs are in place." Evidence Action website, Deworm the World Initiative (March 2016)
  • A few examples of Deworm the World assisting with prevalence surveys include:
    • "Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children." Grace Hollister conversation June 19th 2013, Pg 1.
    • "DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged [sic] children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment." Grace Hollister conversation June 19th 2013, Pg 2.
    • "In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city." Grace Hollister conversation June 19th 2013, Pg 3.
    • "DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year." Grace Hollister conversation June 19th 2013, Pg 4.

• 22
  • For example, Deworm the World is currently supporting prevalence surveys in Pakistan because Pakistan has not yet been "mapped" (i.e., prevalence surveys have not yet been conducted in Pakistan), so nobody knows how heavy the worm burden is in Pakistan or where deworming efforts should be focused. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "Deworm the World has estimated very roughly that its three-year program in Nepal would cost about $6 million ($2 million per year). This is based on a cost per child of no more than $0.50 and a target population of about 6 million children. However, the latter estimate is based on the outdated prevalence survey data mentioned above. Deworm the World will need to conduct a new survey to determine an exact target population and a more accurate budget." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pgs 1-2.
  • "Deworm the World will likely wait to expand its Vietnam activities until further mapping and impact evaluation have been completed." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pg 7.

• 23
  • Deworm the World originally planned to do prevalence surveys every few years but may do them less frequently going forward.
  • "DtWI would like to do prevalence surveys after every 3 years or so. Ideally, prevalence surveys would be carried out after every third round of treatment immediately prior to the following round." GiveWell's notes from site visit to India, October 2013
  • "Note: this [follow-up prevalence survey] strategy is evolving; WHO recommendations are to conduct sentinel site surveys after 5-6 rounds of treatment, and we are moving in that direction. Key is how a new survey would impact the treatment strategy" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Key M&E activities in India include "post-round 3 impact measurement" prevalence surveys. DtWI NDD Year 1 M&E review July 2015, Pg 3.
  • Deworm the World has said that a Rajasthan follow-up prevalence survey is tentatively planned for late 2017. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Grace Hollister, edits to GiveWell's review, November 7, 2016

• 24

  Deworm the World told us that it works "with partners with expertise in STH parasitiology and epidemiology." Grace Hollister, edits to GiveWell's review, November 7, 2016. Examples of Deworm the World working with partners on prevalence surveys:

  • "The WHO reports that Pakistan is endemic for STH, but there is not yet sufficient evidence of prevalence and intensity to develop an evidence-based treatment strategy. Deworm the World has committed unrestricted funding to fund prevalence surveys in two large provinces, Punjab and Sindh. It is targeting these provinces because their school enrollment rates are high, the areas are fairly secure, and they contain a significant percentage of the population of Pakistan. Deworm the World plans to contract with a local organization to do these surveys, but does not yet have a signed agreement." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.
  • "Following three rounds of school-based deworming, Evidence Action - Deworm the World Initiative recommended a second prevalence survey to the Bihar government, in order to understand the effect of deworming in Bihar on STH infection levels. With approvals from the State Government, in January and February 2015, Evidence Action - Deworm the World Initiative, conducted an STH prevalence survey among school-age children in government primary schools in Bihar. The survey took place in 65 schools in 14 districts, covering all three agro-climatic zones in the state. The National Institute of Epidemiology – Chennai (NIE) designed the survey, and analyzed the dataset to produce epidemiological findings. Field teams hired through GfK Mode (an agency with prior experience in sample collection for STH prevalence surveys), visited the households of children in the selected schools to collect stool samples and information related to school, household, deworming, and sanitation, to better understand infection patterns and allow for sample weighting. The 2 Post Graduate Institute of Medical Education and Research – Chandigarh (PGIMER) analyzed stool samples in field laboratories, which were set up in district and block health facilities, using the WHO recommended Kato-Katz method." DtWI Bihar 2015 Prevalence Survey report, Pgs 1-2.

• 25
  • "We work closely with the Ministries of Education and Health to design a program with joint ownership, develop operational plans and budgets, coordinate logistics, and provide on-the-ground support to ensure a high quality outcome." Evidence Action website, Deworm the World Initiative (March 2016)
  • Note that we do not feel like we have a strong understanding of Deworm the World's activities in this area; for example, we have not asked Deworm the World what it has brought to the planning, budgeting, or logistics processes that would not have otherwise been included. We do not have a strong sense from Deworm the World's website about what these activities involve (e.g., we do not know what it means for Deworm the World to have "coordinated logistics").

• 26
  • "We help governments evaluate appropriate drug treatment strategies and dosage, support drug procurement including through global pharmaceutical donation programs, and design robust serious adverse event protocols and drug tracking systems." Evidence Action website, Deworm the World Initiative (December 2016)
  • Grace Hollister, edits to GiveWell's review, November 7, 2016

• 27
  • "DtWI provided support to the state government in submitting the drug requisition to WHO in March 2013, as well as in shipping, custom clearances and transportation upon arrival in India." DtWI Rajasthan 2013 program report, Pg 6.
  • Deworm the World told us that before it started conversations with the Indian government, the government was not aware that it could obtain albendazole for free from the World Health Organization. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 28
  • "We consistently design and support training through an efficient multi-tier cascade approach that is tailored to the local context, ensuring knowledge reaches from the national level all the way to the teachers responsible for administering deworming medication." Evidence Action website, Deworm the World Initiative (March 2016)
    • GiveWell's understanding is that training cascades involve a series of trainings that start at high levels in the government, and proceed in a step-wise fashion down to local levels, where teachers are eventually trained. At each training, materials (such as deworming drugs and posters that notify the community about deworming day) are passed down from the staff member leading the training to the staff members attending the training, until materials eventually reach teachers.
  • A description of a training and distribution cascade: "Transportation of tablets to all districts was managed and supported by DtWI, in coordination with SHS and DHFW to the district level dispensary, from where they were collected by the respective teachers for their school. For anganwadis, the syrups were sent to the nodal officer who arranged further transportation to the supervisors who in turn handed over to the anganwadis. Training about health education on types of STH, need for deworming, transmission of worms through open defecation and other practices and how to safely administer deworming drug was conducted through a cascade model. In the first step of the cascade, training of trainers was conducted at the state level where a total of 1040 participants attended (390 WCD + 650 education department) in 60 sessions. These trainers further trained 3032 headmasters and 3032 teachers and 10,500anganwadi workers in groups of 30 participants. The training on deworming was integrated with training for the WIFS program to effectively utilize time of participants and trainers and reduce training costs. A simplified training manual was developed that included content on deworming and WIFS into a single document. At the state level training sessions, training videos on three types of soil transmitted [sic] helminths (ascaris, trichuris and hookworm) and worm infestation cycle were also used. DtWI hired district coordinators (DCs) to provide short term support to the deworming program. They played a key role in ensuring that drugs in adequate quantities were available with the district-level dispensaries and nodal offices for further distribution. The DCs collated information on shortfall or surplus of drugs at district level and shared information with the DtWI state team, who coordinated with SHS to ensured-distribution [sic] or fresh supplies to districts facing drug deficits." DtWI Delhi 2013 program report, Pg 7.
  • See Pgs. 12-13 of DtWI Delhi 2015 program report for a visual representation and description of a recent training and distribution cascade.

• 29
  • For example, a description of the training cascade in Kenya: "The National School-Based Deworming Programme uses a cascade implementation model that efficiently and cost-effectively delivers training materials, deworming tablets, monitoring forms, funds, trainings other programme materials and resources from the national level to schools. At the national level, the Programme trains a team of MoEST and MoH officials as master trainers, requisitions deworming tablets through the MoH, and develops treatment and implementation strategies, training materials and monitoring tools. Thereafter, an initial planning meeting is held with county and sub-county leadership. This meeting is followed by two levels of trainings on how to successfully implement the Deworming Programme: Sub-County Training and Teacher Trainings. These trainings prepare sub-county and division officials to plan subsequent programme activities within the cascade, distribution of materials, planning of deworming and community mobilization and sensitization. After these trainings and community mobilization, the critical day of implementation occurs – Deworming Day – where teachers administer deworming tablets to millions of children in over 11,000 schools across Kenya and fill in monitoring forms to capture treatment data. These forms and any unused deworming tablets are moved up through a “Reverse Cascade” as described below. The cascade model helps to manage the national scale of the NSBDP, and therefore, builds capacity for successful implementation at various levels. Additionally, the cascade brings together MoEST and MoH personnel through collaborative leadership responsibilities for the planning, implementation and monitoring of programme activities at all levels. The cascade is outlined in the infographic below." DtWI Kenya 2013-2014 program report, Pgs 4-5.
  • "Drug distribution: As per NDD operational guidelines, and established best practice, drug distribution was integrated with the training cascade (as detailed in the training section below), whereby NDD kits were provided to health functionaries at the district level trainings for onward distribution. The kits included drugs, IEC materials, and reporting forms." DtWI Madhya Pradesh 2015 program report, Pg 12.

• 30

  For example:

  • "DtWI provided technical expertise across all program components, and served as the primary coordinating body among implementing agencies. DtWI facilitated drug donations from WHO, provided professional master trainers for training of trainers, developed training materials such as flipcharts, modified training booklet and reporting forms, designed and carried out independent monitoring, and developed adverse event protocols." DtWI Rajasthan 2013 program report, Pg 5.
  • "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." DtWI Madhya Pradesh 2015 program report, Pg 11.
  • "We hire master trainers, or train govt staff to be master trainers." Grace Hollister, email to GiveWell, June 9, 2016

• 31
  • Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16.
  • Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15.
  • Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15.
  • Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17.

• 32

  "We work with governments and communications experts to design locally appropriate awareness campaigns to communicate messages through a wide variety of channels to increase public acceptance and effectiveness of deworming programs." Evidence Action website, Deworm the World Initiative (March 2016)

• 33

  For example: "As part of their training, school headmasters/teachers were instructed to share information on the deworming program in the morning prayer sessions at their respective schools on a daily basis from October 6, 2013 onwards. They were also advised to convene school management committee meetings to communicate about the benefits of deworming and the schedule of deworming program. School headmasters were also advised to carry out student rallies / processions (prabhat pheri) to create awareness in the communities." DtWI Rajasthan 2013 program report, Pgs 8-9.

• 34
  • "One other key strategy adopted by DtWI to spread awareness was through text (SMS) reminders over mobile phones to school teachers, headmasters, Child Development Project Officers (CDPOs) and lady supervisors as a reminder about deworming day. SMSs were also used to reinforce precautions on drug administration, such as not giving drugs on an empty stomach, but only after midday meals and not giving drugs to sick children. In all, about 80,000 text messages were sent to school teachers and headmasters three times – a total of 2,40,000 [sic] messages. These messages were sent a day before deworming day, on mop-up day and after mop-up day. About 1400 such messages were sent twice to lady supervisors and CDPOs on a day before deworming day and on mop-up day. Similarly, five rounds of around 2400 text messages were sent to block level officials to expedite coverage reporting. This was an example of ensuring last-mile communication at low cost of about 12 paisa per message (or roughly 1/5th of a cent)." DtWI Rajasthan 2013 program report, Pg 9.
  • "Additionally, mike announcements were made at public places in blocks and district headquarters by Evidence Action for 5 days, closer to deworming day (Annexure E.4)." DtWI Bihar 2015 Program report, Pg 13.
  • "The State Health Society Bihar and Evidence Action rolled out a media mix to generate community awareness and increase program visibility to improve coverage in the state (Annexure E.1). We supported the adaptation and contextualization of prototypes from the National Deworming Day IEC resource toolkit. At the state level, State Health Society Bihar, in coordination with the Department of Public Relation, Government of Bihar, published newspaper advertisement in four dailies one day prior to deworming and mop up [sic] day, i.e., on 20 and 25 February (Annexure E.2). Radio jingles, customized into three local dialects, were aired from 15 to 26 February on the All India Radio to maximize outreach to the community. For additional visibility of the program at the community level, State Health Society Bihar printed 513,625 posters (7 for each school, including distribution in the local community), 1068 banners for Primary Health Centers, hoardings at 38 district headquarter [sic]. All of these were adapted and contextualized by Evidence Action." DtWI Bihar 2015 Program report, Pg 12.
  • Other community awareness activities include creating posters to display at schools or advertising the deworming day in the newspaper.
    • "Activities designed to enhance community awareness on deworming were rolled out to improve overall program coverage. The awareness activities included newspaper advertisements a day prior to the deworming day; a 60-second radio jingle aired on 3 FM channels from April 7 to 15 by School Health Scheme, and banners displayed at schools. Evidence Action was part of the committee formed by the state government for contextualization of the radio jingle. Evidence Action extended support to the state in contextualizing IEC materials from the National Deworming Day guidelines. The Directorate of Family Welfare also independently developed and printed handbills for the distribution at anganwadis to mobilize people on deworming day. The School Health Scheme provided banners to the schools, the distribution of which was integrated in trainings for teachers. The Delhi state government also used an e-portal to disseminate key information, including dates for deworming and mop up [sic] days, benefits of deworming, and details of the launch event." DtWI Delhi 2015 program report, Pgs 11-12.

• 35

  "We help governments design monitoring systems to measure effectiveness in achieving intended program results. We also conduct independent monitoring to validate program results, and evaluate the impact of programs in reducing worm prevalence and intensity." Evidence Action website, Deworm the World Initiative (March 2016)

• 36

  Note that Deworm the World hires monitors for the first and third type of monitoring data collected, but that the second is collected entirely by government staff: "Coverage reporting is done by the government- we sometimes assist in the data analysis, designing reporting forms, and ensuring that the 'reverse cascade' is appropriately designed." Grace Hollister, email to GiveWell, June 9, 2016

• 37

  "Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program." DtWI Madhya Pradesh 2015 program report, Pg 16.

• 38
  • In India, on Deworming Day and Mop-Up Day, Deworm the World commissions independent monitors who go to schools to gather data on whether principals and teachers are prepared for Deworming Day, the availability of drugs and supplementary materials, whether students are being dewormed, whether proper procedures are being followed, and more. For example, see Deworming Day monitoring data from Rajasthan in 2015: DtWI Rajasthan 2015 monitoring survey from deworming day, schools (shows which questions were asked) and DtWI Rajasthan 2015 monitoring data from deworming day, schools (shows the survey responses).
  • Kenya:
    • Trainings monitoring: "PMCV [Process Monitoring and Coverage Validation] officers observed 36 CHEW [Community Health Extension Worker] Forums aimed at introducing the deworming sensitization message and materials/methods as well as asigning [sic] CHEWs to schools for monitoring. A successful community health extension worker forum is one that starts on time and where all the materials were present. Overall, 63% of participants arrived before training, whereas 22% arrived 1hr after the forum had begun and 15% of participants arrived more than 1hr after the forum’s commencement. Lateness appears to be a commonality to all training sessions. Materials required for CHEW training include a powerpoint printout, CHEW checklist and Severe Adverse Event (SAE; side-effects of the drugs) protocol.
      In 51% of forums, ALL of the Materials Pack was distributed at the start of the forum. In 13% of forums, SOME of the Materials Pack was distributed at the start of the forum. In 36% of forums, NONE of the Materials Pack was distributed at the start of the forum." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 9.
    • Community sensitization monitoring: "A number of parents were also interview [sic] at schools on Deworming Day regarding their knowledge of deworming and the source of that knowledge. The intention behind this exercise was to compare the information source to those interviewed prior to deworming as a measure of consistency. In Figure 4, the results of the interviews pre-deworming day are compared with those parents interviewed on deworming day. The results remain largely similar, however more parents reported getting their information from ‘other’ sources (51%) when interviewed on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 14.
    • School preparedness monitoring: "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.
    • Deworming Day and Mop-Up Day monitoring: "PMCV field officers visit schools on Deworming Day to observe procedure and interview teachers/head teachers regarding deworming. The number of schools observed on Deworming Day treating for STH in Year 3 was 247. The combined population of registered children at the observed schools was 88,820 children. It is estimated that 7,485 children were directly observed being treated for STH. Seven schools treating for both STH and SCH were observed. The total registered population of children in these schools was 3,198 children and 352 children were directly observed by field officers participating in Deworming Day. A quality Deworming Day is regarded to be one where:
      • Deworming occurs within 1 week before [sic] teacher training
      • The school would have the correct materials (including sufficient drugs) in place before commencement
      • Children of the appropriate ages are treated (ages 2-14 years)
      • Non-enrolled and ECD aged children are prioritized for treatment within the schools
      • The correct dosage of drugs is given to all children"

      Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16.

• 39

  This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. For example, see DtWI Delhi 2012 coverage data by school.

• 40

  Grace Hollister, email to GiveWell, March 6, 2016

• 41
  • "A competitive RFP [request for proposal] process is now used to identify a professional survey organization to provide independent monitors. There are requirements placed on the experience of these monitors." DtWI Monitoring Improvements 2014, Pg 1.
  • Bihar: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state… Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively." DtWI Bihar 2015 Program report, Pg. 15.
  • Rajasthan: "Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state... Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively." DtWI Rajasthan 2015 program report, Pg 14.
  • Madhya Pradesh: "Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively." DtWI Madhya Pradesh 2015 program report, Pg 15.
  • Delhi: "[Evidence Action] hired an independent research agency, Sigma Research and Consulting Private Limited [sic] that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively." DtWI Delhi 2015 program report, Pg 14.
  • Deworm the World was also involved in Chhattisgarh, but as it was engaged relatively late in the process, it did not conduct all of its standard monitoring activities in the state: "Although we place great emphasis on understanding the extent to which the school and health systems are ready to implement deworming, the extent to which deworming processes are being followed, and the extent to which coverage has occurred as planned, in Chhattisgarh we supported only with the coverage validation activity at schools due to time constraints." DtWI Chhattisgarh 2015 coverage validation report, Pgs 2-3.

• 42
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "Evidence Action has a permanent Monitoring, Learning, and Information Systems team. Deworm the World leverages this team for M&E." Grace Hollister, email to GiveWell, June 9, 2016
  • "The field officers that collect the data in the field are short term hires who come from the counties in which the program is implemented" Grace Hollister, edits to GiveWell's review, November 7, 2016

• 43

  Grace Hollister, conversation with GiveWell, August 11, 2016 and Grace Hollister, conversation with GiveWell, August 24, 2016

• 44

  "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program….In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J- PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there." Grace Hollister conversation June 19th 2013, Pgs 1-2. See the same set of conversation notes for descriptions of how Deworm the World's programs in Delhi and Rajasthan started as well.

• 45
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016.
  • For example, see DtWI Madhya Pradesh 2015 program report:
    • Prevalence surveys: "To develop an appropriate STH treatment strategy, Evidence Action obtained support and approvals from State NHM [National Health Mission] and Education Department to conduct an STH prevalence and intensity survey among children enrolled in government primary schools. Evidence Action, in partnership with National Institute of Epidemiology - Chennai (NIE), Post Graduate Institute of Medical Education and Research - Chandigarh (PGIMER), and GfK Mode, a market research firm conducted a STH prevalence and intensity survey among children aged 5 to 10, studying in government primary schools in the state." Pg 7.
    • High level planning (e.g. obtaining drugs): "Evidence Action worked with the nodal officer to complete the drug requisition for 2015-16 in August 2014. The drugs for next deworming round (2016) were received at the state in form of two consignments on June 13 and June 18, 2015." Pg 12.
    • Training and distribution cascades:
      • "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." Pg 11.
      • "Evidence Action supported the implementation of the training cascade as depicted below, to orient various levels of functionaries in the key departments… Evidence Action helped contextualize materials from the NDD resource kit according to state requirements, including training presentations, handouts for frontline workers, and frequently asked questions (FAQs)... Evidence Action's regional and district coordinators participated in all 51 district-level trainings under NDD. Additionally the team attended a sample of clock-level trainings to provide support and assess quality (Annexure H2)." Pg 14.
    • Community sensitization: "Evidence Action also briefed media representatives about the program and the event, resulting in widespread coverage. Media kits included key information on the program such as a concept note shared by the state NHM. District-level launch events were held widely across the state. The events were led by local district administration and supported by Evidence Action district coordinators (Annexure G2)." Pg 13.
    • Troubleshooting with telecallers: "Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." Pg 17.
    • Monitoring: "In order to fulfil this need, Evidence Action worked intensively with the state health, and education, departments to ensure quality planning and implementation of the deworming program." Pg 15.

• 46
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 47
  • "Additionally the national government of India has expressed interest in possibly implementing a deworming program nationwide, rather than waiting for each state to launch separate programs. They have expressed interest in receiving technical assistance from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2.
  • "On National Deworming Day, 12 states in India selected for inclusion in the program’s first phase will be required to administer deworming drugs to schoolchildren, regardless of measures of worm prevalence. National Deworming Day was originally planned for October 2014, but was delayed after Deworm the World staff alerted the government that drug supplies were insufficient. The Indian government has publically announced that National Deworming Day will be in February 2015, but it is possible that there will be further delays.
    "The target of the National Deworming Day is children who are 1-19 years old. In the first year, the national government is not advising states to highly prioritize deworming treatment in preschools, though some states were prepared and have already procured deworming medication." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1.
  • "Deworm the World worked closely with the ministry to develop operational and financial guidelines, public awareness campaigns, and monitoring strategies for the program, based on the experiences of state-level deworming programs." Grace Hollister, email to GiveWell, June 9, 2016. The National Deworming Day was announced by India's national Ministry of Health and Family Welfare.

• 48
  • "India held its first National Deworming Day in February. Eleven states participated." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2.
  • "11 dewormed in February; the 12th, Delhi, dewormed in April. This delay was due to conflicts of the February timing with state elections, which require the participation of teachers." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Grace Hollister, email to GiveWell, June 9, 2016

• 49
  • "February 10, 2015 was the first National Deworming Day in India, when over 89 million preschool and school-aged [sic] children were dewormed with our support. This led to many states recognising the importance of mass school-based deworming as a safe, cost-effective, and scalable health intervention. Since then, we have had an incredible year, with agreements signed with the states of Uttar Pradesh and Chhattisgarh, and renewed with the states of Bihar and Rajasthan. Our India program now extends technical assistance to seven states: Bihar, Rajasthan, Madhya Pradesh, Uttar Pradesh, Delhi, Chhattisgarh, and – the latest – Telangana." Evidence Action, blog post, December 21, 2015
  • Ms. Hollister told us that Deworm the World had also assisted Tripura during the National Deworming Day in 2016. Note that the level of support Deworm the World provides differs from state to state. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 50
  • "Recently completed worm prevalence surveys conducted by the National Center for Disease Control and Evidence Action revealed that 26 Indian states have such high worm prevalence to warrant twice annual treatment. The Government of India has decided to double up on the successes achieved during this year’s National Deworming Day with our support.
    "The next deworming day will take place in Indian schools and anganwadis on August 10."
    Evidence Action, blog post, August 1, 2016
  • Neetu Chandra Sharma, Daily Mail - India article, August 8, 2016
  • 28 states were supposed to participate in the second round of deworming in 2016, but only 26 did. Deworm the World was supporting one of the states that did not implement deworming - Bihar. Bihar was unable to obtain drugs in time for the MDA. Deworm the World, National Deworming Day states, August 2016

• 51

  "Deworm the World has phased out its comprehensive technical assistance to Delhi, but the national office will continue to provide minimal support, including attending meetings and doing some policy advocacy work. Deworm the World no longer has dedicated staff for Delhi." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 2

• 52

  Grace Hollister, email to GiveWell, June 9, 2016

• 53
  • Note that Deworm the World is the fiscal agent controlling funding for the deworming program in Kenya (which is different from its role in India). This means that Deworm the World has additional responsibility in Kenya for ensuring that funds are used correctly. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "Kenya’s National School-Based Deworming Program started in 2009. In 2012, it expanded to a nationwide program aimed at treating all at-risk Kenyan children each year for at least five years." Evidence Action website, Deworm the World Initiative (March 2016)
  • "This actually happened in 2010, but took awhile for funding to come through, and the partnership was formalized at the end of 2011. The first round of the NSBDP took place in 2012." Grace Hollister, edits to GiveWell's review, November 7, 2016
  • Funders of the program discussed in Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 54

  Grace Hollister, conversation with GiveWell, July 25, 2016 and Grace Hollister, conversation with GiveWell, August 11, 2016

• 55
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "We supported surveys for schisto with END Fund resources. Also, there is some additional mapping required in areas not previously surveyed that we are working with the NTD Unit to support and resource, to guide revised treatment strategy beginning in Year 6." Grace Hollister, edits to GiveWell's review, November 7, 2016

• 56
  • CIFF uses these prevalence surveys as a way to assess Deworm the World's work. We discuss the methodology and results of these prevalence surveys more here. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, conversation with GiveWell, June 13, 2016
  • Deworm the World has helped with prevalence surveys looking at schistosomes: "...we have funded schisto surveying through END Fund resources, and have helped develop a treatment strategy for schisto as well." Grace Hollister, email to GiveWell, June 9, 2016

• 57

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 58
  • In the first year that Deworm the World worked with Kenya, the MDAs were implemented in 12 different waves. In 2016, there will only be two waves - one in February and one in May. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Currently, implementation of the Kenya program is funded by CIFF and the END Fund, while the government provides support via its personnel. Deworm the World is in discussions about how the program will be funded once the current grant ends in 2017. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 59

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 60

  For example, the Kenya team has supported (or will support) the Ethiopia, Nigeria, and Pakistan teams.

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 61

  Although the agreement that governs how SCI and Deworm the World assist with the MDAs is with the Federal Ministry of Health, SCI and Deworm the World also work to some extent with regional governments. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 62

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 63
  • "We are proud and excited to announce that Ethiopia's Federal Ministry of Health is launching a national deworming program targeting school-age children. Ethiopia has one of the highest burdens of neglected tropical diseases in the world, with over 10M children at risk for schistosomiasis and 18M children at risk for soil-transmitted helminths." Evidence Action, blog post, January 8, 2015
  • See Evidence Action, blog post, June 12, 2015 for some description of the roles Deworm the World and SCI played in this decision.
  • The program is led by the Federal Ministry of Health with technical inputs from the Ethiopian Public Health Institute. Grace Hollister, email to GiveWell, June 9, 2016

• 64

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 65

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 66
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Implementation funds are provided to the Federal Ministry of Health, and funds for technical assistance provided to SCI, of which a portion funds Deworm the World's engagement. Grace Hollister, email to GiveWell, June 9, 2016

• 67
  • SCI works with the Expanded Programme on Immunization (EPI) to conduct the prevalence surveys. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Additionally, Deworm the World's Kenya team supports Ethiopia by leading training sessions and developing monitoring tools for Ethiopia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Note that Deworm the World's staff structure in Ethiopia is significantly different from that in Kenya or India: in Ethiopia, it only has one staff member, a consultant, who works closely with the SCI team. Grace Hollister, email to GiveWell, June 9, 2016

• 68

  Grace Hollister, email to GiveWell, June 9, 2016

• 69

  In the case of Pakistan, it was important to find a competent organization that Deworm the World could partner with, for two reasons: first, when entering a new country, it is easiest if Deworm the World can use its partner's financial systems and permissions to move funds to the deworming program (as opposed to Deworm the World attempting to set up an independent financial entity in the country); second, in Pakistan it is unclear how much capacity the government will have to offer to assist with deworming, so it may be important for partner organizations to provide that capacity. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 70

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 71

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 72

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 73
  • Deworm the World does not plan to test for prevalence in areas where there are security issues, it is logistically difficult to test for prevalence, or the agro-climatic region wouldn't support worms (e.g., the region is mountainous). Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, conversation with GiveWell, August 11, 2016

• 74
  • As of March 2016, Deworm the World was still in the process of obtaining government approval for the prevalence surveys, but hoped to start them in April and have results by June. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Factors causing the additional delays included: First, the Kato Katz kits required for the prevalence survey were held up at customs in Pakistan. Then, many schools in Pakistan closed early for the summer. Deworm the World had to wait until the schools opened again before it could begin the survey. As of mid-August, Deworm the World felt that prospects for beginning the survey by the end of the month looked promising. Grace Hollister, conversation with GiveWell, August 11, 2016
  • Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 75

  Grace Hollister, edits to GiveWell's review, November 7, 2016

• 76

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 77
  • Deworm the World expects this to make working in Pakistan more challenging, in part because the WHO (one of the largest donors of deworming drugs) is not able to work directly with non-national government representatives. Grace Hollister, conversation with GiveWell, August 11, 2016
  • Deworm the World expected this to occur. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 78
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Diagnostic kits were donated by the WHO. Grace Hollister, email to GiveWell, June 9, 2016

• 79

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 80
  • "In Nigeria, programs like deworming are typically implemented at the state level, with various donors and partner organizations supporting different states and targeting different NTDs. USAID and the Department for International Development are major donors. Partner organizations include RTI and Sightsavers, which often implement large-scale integrated NTD programs targeting several diseases." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg. 4.
  • Because Nigeria is a country in which NTDs, including STH, are highly endemic, there are many NGOs already working there. However, because there is a heavy focus on integrated programs that address multiple NTDs at once, the states in Nigeria that are not as heavily afflicted by multiple NTDs are less likely to have an NGO partner. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016.

• 81

  As of March 2016, Deworm the World had hired two staff members to work in Cross River. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 82
  • Grace Hollister, conversation with GiveWell, August 24, 2016.
  • Note: Deworm the World has been discussing this request with the Federal Ministry of Health and has received permission to support school-based deworming programs in places where other NTD treatment programs are not yet occurring. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 83
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Initially, Deworm the World did not intend to open an office in Nigeria, so partnering with RTI also provided an official entity to work through and use for financial purposes, because RTI had established offices and financial systems in Nigeria.
    • "Deworm the World does not plan to open its own office in Nigeria, but to work through a partner with an on-the-ground presence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg. 4.
    • Grace Hollister, conversation with GiveWell, August 24, 2016

• 84
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016
  • As of June 2016, the two organizations had established a contract governing their partnership, and memoranda of understanding were in place with the health and education sectors of the state government. Grace Hollister, email to GiveWell, June 9, 2016
  • Deworm the World has committed to work in Cross River with RTI until 2019; as of March 2016 it had allocated $1.36 million to this project for four rounds of deworming. Grace Hollister, conversation with GiveWell, August 24, 2016 and Evidence Action Q1 financials, 2016
  • If a local government area (LGA) is endemic for lymphatic filariasis (LF), then RTI will support treatment of LF in the LGA with a community-based distribution of albendazole. Subsequently, Deworm the World will not support a school-based deworming program in the same LGA, because albendazole will already have been distributed to the population. It can be difficult to achieve high coverage rates with community-based MDAs, so Deworm the World and RTI plan to examine their monitoring results to inform future treatment strategies. If coverage rates are low, they may intensify community sensitization efforts; it is highly unlikely that they will switch to a school-based deworming program in an LGA that has a community-based program for LF. Grace Hollister, conversation with GiveWell, August 24, 2016
  • If Deworm the World and RTI are both supporting programs in the same LGA, the community-based MDA will typically occur several weeks after the school-based MDA. Grace Hollister, conversation with GiveWell, August 24, 2016

• 85

  "This week, the Cross River State Ministry of Health’s Neglected Tropical Diseases (NTD) unit launched its inaugural statewide school-based deworming exercise that will treat against two neglected tropical diseases that are particularly common in children: schistosomiasis and soil-transmitted helminthiasis (STH). The school-based deworming exercise will cover 11 of the 18 local government areas in Cross River for the first time, and is targeting 600,000 at-risk school-aged children in primary and junior secondary public and private schools. Other NTDs endemic to the state (lymphatic filariasis and onchocerciasis) will be treated through a community-based approach, according to standard practice." Evidence Action, blog post, June 30, 2016

• 86

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 87
  • Grace Hollister, conversation with GiveWell, August 11, 2016 and Grace Hollister, conversation with GiveWell, August 24, 2016
  • As of March 2016 Deworm the World had hired a consultant to work in Abuja to build federal relationships and start working on business development activities. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Deworm the World has started discussions with national government about creating an STH and schistosomiasis action plan, and it is considering expanding its support to 1-2 additional states next year. Grace Hollister, conversation with GiveWell, August 11, 2016

• 88
  • There are several advantages to registering in Nigeria, including: (1) it will be easier for Deworm the World to sign MOUs with states and the national government, (2) it will allow Deworm the World to bring funding into the country without working through another organization (Deworm the World has been working financially through its partner, RTI), and (3) registering signals to other organizations that Deworm the World intends to commit to working in Nigeria for a long time, which makes it more likely that Deworm the World will be included in NGO coalitions and discussions.
  • The office will only serve Deworm the World; it is not intended to serve Evidence Action's other programs.
  • The office will probably include one program manager per state that Deworm the World supports and may include a larger operations staff if Deworm the World decides to operate as its own fiscal agent (instead of operating through RTI).

  Grace Hollister, conversation with GiveWell, August 11, 2016 and Grace Hollister, conversation with GiveWell, August 24, 2016
  

• 89

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 90
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "Thrive Networks has extensive experience in Vietnam in the water and sanitation, health and education sectors." Grace Hollister, email to GiveWell, June 9, 2016
  • Grace Hollister, conversation with GiveWell, August 11, 2016

• 91
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 92

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 93

  Note that in the RCT evaluating the program, the control group will be dewormed along with the treatment group. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 94

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 and Grace Hollister, edits to GiveWell's review, November 7, 2016

• 95
  • "On April 28 and 29, more than 700,000 primary school children across four provinces in northern Vietnam will line up in their classrooms to receive a deworming tablet. 8.5 million children in Vietnam are at risk of parasitic worm infections that can harm their health, development, and school participation. Evidence Action’s Deworm the World Initiative supports the Government of Vietnam as it strengthens and improves school-based deworming to keep children healthy and in school. Between now and 2018, the program will distribute more than four million treatments to combat worms in the four provinces." Evidence Action, blog post, April 27, 2016
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 96
  • Grace Hollister, conversation with GiveWell, August 24, 2016
  • Deworm the World had originally predicted that all four provinces would need treatments two times per year. The survey results indicated that three of the provinces only need treatment one time per year, and one of the provinces only need treatment one time per every two years. Deworm the World, target populations and parasitology data

• 97
  • Grace Hollister, conversation with GiveWell, August 24, 2016
  • Deworm the World plans to support a second round of treatment in November 2016, as per the government's approved work plan, but going forward does not intend to support MDAs beyond what is recommended according to the WHO guidelines. Paul Monaghan, conversation with GiveWell, September 8, 2016

• 98
  • Deworm the World, Vietnam final report for STH survey in 21 provinces
  • NIMPE has conducted prevalence surveys in Vietnam before, but the surveys that Deworm the World commissioned were designed in line with WHO guidelines and representative within each province. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "These surveys are taking place in areas of the country that are not currently treated for STH, and where albendazole administration for lymphatic filariasis ceased several years ago. The intent of the surveys is to determine whether there are areas of the country where STH treatment is required, and will [sic] contribute to developing an evidence-based treatment strategy for the country." Grace Hollister, email to GiveWell, June 9, 2016

• 99

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 100

  "The control and prevention of STH in the 21 provinces will be used the MDA deworming guideline number 1932/QD-BYT date 19 May, 2016 of Vietnamese Ministry Of Health. If the prevalence of STH infection are > = 20% the MDA will be conducted two time per year as in Quang Ninh province. If the prevalences of STH infection are from 10% up to 20% the MDA deworming will be condcuted 1 time per year with Tra Vinh, Ninh Binh, Binh Thuan and Hung Yen provinces. The rest provinces, the prevalence of the STH infection < 10% the MDA will be carried out one time in two year." Deworm the World, Vietnam final report for STH survey in 21 provinces, Pg 151.

• 101

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 102

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 103

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 104

  Deworm the World told us that it has heard that Nepal may already have a functioning deworming program that achieves high coverage. However, its previous conversations with the government in Nepal had indicated that the government was eager for technical assistance. Deworm the World still intends to resume the conversations eventually to clarify the situation, but it has recently been capacity-constrained due to the scale-up of National Deworming Day in India. Grace Hollister, conversation with GiveWell, August 11, 2016

• 105

  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016. See the footnotes in our section on India above.

• 106
  • There is some chance that Deworm the World could partner with RTI again; RTI currently works on a lymphatic filariasis program in Indonesia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Deworm the World believes that only a low percentage of children at risk for STH are being treated in Indonesia. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, conversation with GiveWell, August 11, 2016

• 107

  Deworm the World provides both types of support in Nigeria and can imagine doing the same in Indonesia. Grace Hollister, conversation with GiveWell, August 11, 2016

• 108

  Grace Hollister, conversation with GiveWell, August 11, 2016

• 109

  See GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses. Note that these figures include central costs, so this reflects Deworm the World’s full budget.

• 110

  See GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses.

• 111
  • "Kenya’s National School-Based Deworming Program started in 2009. In 2012, it expanded to a nationwide program aimed at treating all at-risk Kenyan children each year for at least five years. Implemented by the Ministry of Education, Science, and Technology and the Ministry of Health. With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively. School year 2014/15 results will be released in October. Technical and operational assistance to the program will continue through 2017." Evidence Action website, Deworm the World Initiative (October 2015)
  • "Deworm the World is planning a strategy to sustain and institutionalize the program after its current grants expire in June 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 4.

• 112
  • "DtWI’s funding gap for 2015 will be small because DtWI recently received a large grant from CIFF. The CIFF grant will cover DtWI’s expenses in India, even with the large growth that DtWI is planning." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 2.
  • Grace Hollister, email to GiveWell, June 9, 2016

• 113

  Some of the differences between spending patterns in Kenya and India are due to the facts that a) Deworm the World pays for direct implementation costs in Kenya that it doesn't pay in India, b) Deworm the World does not pay for prevalence surveys in Kenya, and c) in Kenya, teachers are paid significantly higher allowances (per diems) for participating in the program:

  • "It is important to note that there are fundamental differences between the Kenya and India programs, as funding to Evidence Acton supports direct implementation costs in Kenya (where the government does not pay those costs, but rather provides in-kind support), and India, where the government pays the lion’s share of implementation costs." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 114

  GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses

• 115

  Grace Hollister, email to GiveWell, June 9, 2016

• 116
  • GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses
  • See our 2014 review for a summary of Deworm the World's costs pre-2014.

• 117
  • "Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success.

    "Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action." Evidence Action launch announcement 2013

  • "Evidence Action Beta investigates what interventions might be suitable for massive scale up [sic] – finding the next thing that works." Evidence Action website, Evidence Action Beta (October 2015)

• 118
  • India: "…absent from most of the country, [schistosomiasis] risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1.
  • "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3.
  • See section on worm prevalence and intensity in India and Kenya below.

• 119

  See the tables at WHO, Helminth control in school-age children second edition, Pg 18. "When the prevalence of any STH infection is under 20%, large-scale preventive chemotherapy interventions are not recommended. Affected individuals should be treated on a case-by-case basis."

• 120
  • WHO, Helminth control in school-age children, Pg 74
  • "Based on the findings of the prevalence survey and WHO guidelines, Evidence Action recommends an annual school based deworming program for school-age children in the state. [...] Given the pre-existing deworming treatments described above, this prevalence survey cannot be considered a baseline survey of an untreated population, but is rather a survey to assess STH infection rates in a treated population, to determine an optimal treatment strategy." DtWI Madhya Pradesh 2015 program report, Pg 38.
  • "Our recommendation is explained in the prevalence survey report. The prevalence and intensity rates from the survey are not "baseline" data, given that there has been relatively regular administration of albendazole in MP through the BSM program that treated PSAC since 2005, and the LF program which provided community-wide treatment of 11 districts of MP (the number of endemic districts had fallen to 8 by 2014). As a result, these deworming efforts have likely had an impact on STH prevalence and MP could not be considered an untreated baseline population. We therefore did not apply the WHO guidelines for baseline STH prevalence.
    "Annexure 10 of the WHO guidelines suggest continuing annual treatment for populations which have received deworming for several years, and prevalence is still greater than 10%. In addition, the high rate of open defecation in the state, and the planned ending of the LF program in MP, increased the risk of infection and a potential resurgence in prevalence." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Grace Hollister, email to GiveWell, June 9, 2016

• 121

  See the tables at WHO, Helminth control in school-age children second edition, Pg 18, for the WHO guidelines for treatment of schistosomiasis.

• 122
  • See Deworm the World, target populations and parasitology data, which shows the prevalence rates for STH and schistosomiasis in Kenya, India, Vietnam, Ethiopia, and Nigeria, as well as the treatment frequency in each of those countries for Deworm the World supported programs. Note that several of the countries in which Deworm the World works have programs to treat lymphatic filariasis, which we discuss more below.
  • Note that in 2016 Deworm the World might support biannual treatments for the four provinces in Vietnam in which it works, even though recent prevalence surveys revealed that the provinces only needed once per year or once per two years treatments. This is because Deworm the World and its partners had originally developed a biannual treatment strategy and now need to adjust the strategy according to the recent prevalence survey results. Deworm the World does not intend to support biannual treatments in Vietnam post-2016. Paul Monaghan, conversation with GiveWell, September 8, 2016

• 123
  • Details of the Kenya studies (Miguel and Kremer 2004, Baird et al 2012) in our Reanalysis of the Miguel and Kremer deworming experiment page.
  • Details of the Uganda study (Croke 2014) here.

• 124

  Notes on the data we have included:

  • Baseline prevalence data for populations studied in Miguel and Kremer 2004 and Croke 2014 are provided in this table for comparison, as these papers represent what we believe to be the strongest pieces of evidence of long term damages from intestinal parasites (see discussion in this section).
  • The table uses Miguel and Kremer 2004 definitions of moderate intensity, which are not the same epg [eggs per gram] levels as WHO definitions for all worms. "Following Brooker, Miguel, et al. (2000), thresholds for moderate infection are 250 epg for Schistosomiasis. mansoni and 5,000 epg for Roundworm, the WHO standards, and 750 epg for Hookworm and 400 epg for Whipworm, both somewhat lower than the WHO standard." Miguel and Kremer 2004, Pg. 167.
  • Bihar, Madhya Pradesh, Rajasthan, and Uttar Pradesh schistosome prevalence was not measured by Deworm the World. We assume 0% based on World Schistosomiasis Risk Chart 2012.
  • We also have worm infection data from Delhi, but do not include it here as Deworm the World has told us that Delhi had lower prevalence than a region it would normally prioritize for mass school-based deworming, but that it has particular political and cultural significance as the capital of India. Additionally, the deworming round in 2013 is the last for which Deworm the World is expecting to assist Delhi significantly. See DtWI Delhi 2012 prevalence survey report.
  • We have seen more recent prevalence data for Bihar and Kenya (discussed below). We do not include that data here because it is unclear how it should affect our expectation of the cost-effectiveness of deworming in Bihar and Kenya. Intuitively, we would expect that, since the prevalence has dropped, the benefits of future deworming will be lower than earlier rounds of deworming. However, our cost-effectiveness analysis is generally not this precise.

  Sources and comments for the data:

  • Deworm the World, target populations and parasitology data. Note that some of this data conflicts with data we have seen from other sources; we have tried to note in the table where this is the case.
  • Miguel and Kremer 2004
    • Miguel and Kremer 2004, Table II, Pg 168.
    • We provide data from year one of the Miguel and Kremer study, which is the time period before El Niño weather conditions set in and increased the prevalence and intensity of local infections.
  • Croke 2014:
    • Kabatereine et al. 2001, Pg 284, Table 1, Average for eastern districts.
    • Kabatereine et al. 2001 was a disease mapping study conducted in southern Uganda in 1998, two years before the study underlying Croke 2014 began, i.e. these are not results from the individuals enrolled in the study underlying Croke 2014.
    • Note that the districts in the eastern district average in Kabatereine et al. 2001 are not an exact match for the districts included in Croke 2014, though four of the five districts included in Croke 2014 (Busia, Iganga, Mbale, and Tororo) are included in the Eastern district average.
    • The districts for Croke 2014 were selected because Kabatereine et al. 2001 had found high worm prevalence in eastern districts: "Five districts in the eastern region of Uganda were selected (Busia, Iganga, Mbale, Palissa, and Tororo) because a survey had indicated that about 60% of children aged 5-10 years were infected with intestinal nematodes, most commonly hookworm." Alderman et al. 2006, pg 1.
  • Bihar
    • We discuss this data above.
    • DtWI Bihar 2011 prevalence survey report
    • There was a pre-existing LF program that provides albendazole to a large portion of the population in Bihar annually.
    • Based on the timing of the decision to deworm in Bihar, there was only time to collect prevalence surveys from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet Deworm the World's typical standard of quality.
    • Before the Deworm the World-managed deworming program in early 2011, a second stage prevalence survey was conducted in four districts. These districts were selected based in part on their complementarity with the original two districts. DtWI Bihar 2011 prevalence survey report, Pg 9.
    • Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness. (Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013)
  • Rajasthan
    • DtWI Rajasthan 2013 prevalence survey report
    • The results presented here are from Deworm the World's second prevalence survey, one year after the first statewide round of deworming. Deworm the World conducted this prevalence survey in September 2013; Deworm the World believes it is higher quality and addresses the issues (for example, with measuring hookworm prevalence) in the initial survey. GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
    • The first prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence was expected to be significant in Rajasthan. DtWI Rajasthan 2012 prevalence survey report, Pgs 15-16.
  • Madhya Pradesh
    • DtWI Madhya Pradesh 2015 program report, Pg 37 and Deworm the World, Madhya Pradesh 2015 prevalence survey report
    • Note that Madhya Pradesh had a program treating for lymphatic filariasis in the state since 2004: "The NFCP was launched in 2004 by the National Vector Borne Disease Control Unit, with the goal of eliminating lymphatic filariasis in endemic districts of Madhya Pradesh by 2015. Under this program, diethylcarbamazine citrate and albendazole are distributed by health workers to the whole community, except for children under one year and pregnant women." Deworm the World, Madhya Pradesh 2015 prevalence survey report, Pg 20
    • We believe the intensities listed on pg 18 of Deworm the World, Madhya Pradesh 2015 prevalence survey report are reported incorrectly; earlier in the report overall STH prevalence is reported to be 12.2%; this seems at odds with the figures reported on pg 18.
  • Uttar Pradesh
    • Deworm the World 2015 Uttar Pradesh prevalence survey report
    • There is a pre-existing LF program that provides albendazole to a large portion of the population in Uttar Pradesh annually; this program implemented an MDA in March 2015. Deworm the World 2015 Uttar Pradesh prevalence survey report, Pg 9
    • Districts without LF treatment programs were surveyed in May 2015, while districts with LF treatment programs were not surveyed until July and August. Deworm the World 2015 Uttar Pradesh prevalence survey report, Pg 9
  • Chhattisgarh
    • Deworm the World, Chhattisgarh prevalence survey report, August 2016. The report is of a prevalence survey conducted in late 2015, after Deworm the World had supported a statewide MDA in Chhattisgarh in early 2015 (Pg 1). Chhattisgarh had also supported deworming treatments through a number of programs since 2004 (Pgs 4-5).
  • Kenya
    • KEMRI prevalence report - Year 2, Pgs 4, 10, and 15.
    • It is not clear to us what definition was used for medium and high intensity infections.
    • For schistosomiasis, we used the measures of S. Mansoni, which was measured in the full sample of 200 schools using the Kato Katz method. In 9 schools, S. haematobium was also measured via a urine test, and found a much higher prevalence of 18.0% at baseline (compared to 1.8% for S. Mansoni). KEMRI prevalence report - Year 2 Pgs 15 and 18, and Mwandawiro et al. 2013 Pgs 3-4.
  • Ethiopia
    • Deworm the World, target populations and parasitology data
    • The prevalence survey took place in two phases: the first phase was 2013-2014, the second phase was in mid-2015. Deworm the World, Ethiopia prevalence survey report, Pg 3.
  • Vietnam
    • The prevalence data we have seen from Vietnam for the four provinces in which Deworm the World works is from Deworm the World, target populations and parasitology data and Deworm the World, Vietnam baseline prevalence survey - 4 provinces. It is our understanding that Vietnam has run a deworming program for multiple years, so the prevalence surveys were conducted with populations that had already been exposed to an MDA: "The prevalences of soil-transmitted helminthiasis were still high in Thanh Hoa and Phu Tho provinces event they have MDA deworming for SAC in every 10 years ago (2 times per year with 3 province Hoa Binh, Thanh Hoa, Nghe An and one per year for Phu Tho province). The prevalences of STH of SAC in Hoa Binh and Nghe An provinces were low infection 6.4% and 10.0%. The last MDA deworming in the four provinces were in November 2015, the coverage of MDA more than 95%. We use Albendazole for the MDA deworming." Deworm the World, Vietnam baseline prevalence survey - 4 provinces, Pg 35
  • Nigeria
    • The only data we have seen from Nigeria so far is from Deworm the World, target populations and parasitology data.

• 125

  In 2016, Deworm the World shared with us baseline prevalence data for Ethiopia: Deworm the World, Ethiopia prevalence survey report. The prevalence survey took place in two phases: the first phase was 2013-2014, the second phase was in mid-2015 (Pg 3). Also see Deworm the World, target populations and parasitology data

• 126

  Deworm the World noted that the KEMRI surveys in Kenya are designed for impact assessment, while the surveys in India are designed for mapping. This means that the schools selected in KEMRI's surveys are all from places where treatment is required or taking place. In mapping surveys, schools are selected to be representative of a larger geographic area (e.g., they may be selected in part based on which agro-climatic region they are in). Additionally, Deworm the World noted that the surveys in India are looking at a much larger population than the Miguel and Kremer 2004 and Croke 2014 studies examined and there may be substantial variation in prevalence across a given area. However, we still believe that making the comparisons we do is somewhat useful. Grace Hollister, conversation with GiveWell, June 13, 2016 and Grace Hollister, edits to GiveWell's review, November 7, 2016

• 127
  • Deworm the World tends to focus on treating STH, but the deworming programs we reviewed focused primarily on schistosomiasis.
  • For example, our understanding is that, as in India, there is no schistosomiasis in Pakistan (one of the countries Deworm the World is starting to work in).

• 128
  • For example, India has such a program: The National Vector Borne Disease Control Programme LF treatment coverage 2015
  • In Kenya, the LF program is housed within the country’s neglected tropical disease (NTD) unit, which has asked Deworm the World if it might support its process monitoring and coverage validation (PMCV) operations for LF. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • Deworm the World has told us that in both countries (Kenya and India), LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • Additionally, our understanding from a number of conversations with Deworm the World and others is that many countries are beginning to shut down their LF programs as they eliminate the disease.

• 129
  • "DEC [Diethylcarbamazine] + Albendazole in selected distt & DEC in other distt" LF treatment drugs 2012.
  • "Albendazole, the same drug used to treat STH, is usually used to treat LF." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Assam 2010 guidelines for deworming recommends 400mg of albendazole for children being dewormed, and Global Alliance to Eliminate Lymphatic Filariasis - Prevention also recommends 400mg doses.

• 130
  • Or from twice-per-year to thrice-per-year.
  • Note that community-based treatment, such as is typically used for LF, involves enlisting several people to travel from house to house to administer treatment, making it much more time-consuming and costly than school-based programs. Because its goal is to treat every person in a community, multiple trips to a single area may be required to ensure total coverage (e.g., if a household member is not at home during the first visit). GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
  • Deworm the World also claims that it generally knows where there is overlap between areas that are endemic for STH or schistosomiasis and areas that are endemic for LF. For example, in Kenya, only the coastal area is LF-endemic, but worm infections are more widespread, so LF treatment efforts involve a smaller number of subcounties. Community-based treatment programs might be cost-effective in places endemic with many NTDs but might not be in places only endemic for STH or schistosomiasis. Deworm the World has told us that many places do not have much overlap between different NTDs so school-based deworming programs can provide a cost-effective alternative to community-based treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015

• 131

  "It is likely that >5% of the population receives LF treatment in India." Deworm the World, target populations and parasitology data. Also, see next footnote.

• 132
  • The intended treatment for LF in India is annual administration of DEC [Diethylcarbamazine] and Albendazole.
    • Grace Hollister, edits to GiveWell's review, November 7, 2016
    • "Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
      Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except:
      - children below 2 years
      - pregnant women
      - seriously ill persons
      (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012.
  • The WHO recommends annual albendazole treatment for areas with 20-50% prevalence of STH.
    • Preventive chemotherapy in human helminthiasis 2006, Pg 41.
  • According to the Indian government's program website, the LF treatment program has high coverage across many states (e.g., typically 80-90% in states carrying out the program), including Bihar, Madhya Pradesh, Uttar Pradesh, and Chhattisgarh.
    The LF program has recently been active in Bihar, Madhya Pradesh, Uttar Pradesh, and Chhattisgarh, but not Rajasthan or Delhi.
    • LF treatment coverage 2015
    • "neither Rajasthan nor Delhi are endemic for LF; no LF MDA is necessary" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • Deworm the World believes that LF treatments in Bihar and other states have not been delivered as effectively as planned; we do not have additional information about LF treatment delivery in other states beyond limited publicly available data.
    • "The Bihar program was originally designed so that one round of albendazole administration would come from community-based lymphatic filariasis (LF) treatment (LF is endemic throughout the state) and a second round would come through the school-based deworming program. For a variety of reasons, LF treatment has not been consistently implemented. It is possible that more consistent albendazole administration as part of LF treatment would have resulted in a more significant drop in STH prevalence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 2.
    • "Kenya and India are the countries where Deworm the World operates two of its biggest programs and where Ms. Hollister has the greatest familiarity with the state of LF treatment. In both countries, LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015, Pg 1
    • "Deworm the World believes that LF treatments in Bihar and other states have not been delivered as effectively as planned" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
    • Data on reported coverage of LF treatment programs here: LF treatment coverage 2015

• 133

  "LF is endemic in 6 coastal-region counties in Kenya, all of which are treated under the NSBDP. This accounts for >5% of the total target population. However, LF MDA has been highly irregular and has not achieved significant coverage (50% or less in the operating years of the NSBDP)." Deworm the World, target populations and parasitology data

• 134

  "As of 2015, an estimated 112 woredas were endemic for LF. 102 of these are co-endemic for STH. However, only 53 woredas were treated for LF in 2015. The new (2016-2020) NTD Strategic Plan for Ethiopia indicates that MDA in LF-endemic districts 'will be integrated with STH so as to increase efficiency and reduce the quantity of albendazole tablets required.'" Deworm the World, target populations and parasitology data

• 135
  • "The integrated NTD program in Cross River is designed to leverage LF treatment in LGAs co-endemic for STH; this applies to 3 LGAs." Deworm the World, target populations and parasitology data. The spreadsheet shows which types of MDAs are planned to occur in each LGA (the type of MDAs planned depend on which NTDs are endemic). Notice that where an LGA is endemic for LF, albendazole is distributed in a community-based MDA and there is no treatment for STH. However, in LGAs where STH is endemic but LF is not, the plan is to support school-based MDAs that distribute mebendazole.
  • If an LGA is endemic for LF, treatment for LF will occur in that LGA and a school-based MDA to treat for STH is not required (because LF treatments also treat for STH). Grace Hollister, conversation with GiveWell, August 24, 2016

• 136
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "LF MDA has stopped in Vietnam, with TAS expected later this year. None of the provinces where we work is LF endemic." Deworm the World, target populations and parasitology data

• 137
  Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  
• 138

  We have gained this understanding through many conversations and across multiple years of reviewing Deworm the World. See our review process.

  Note that the methods Deworm the World uses in each country, for each round of MDA, might not be the same as those outlined here; Deworm the World adjusts its monitoring based on past learning or new contextual constraints.

  In the rest of this footnote, we lay out some broad descriptions of the monitoring processes we have seen in Deworm the World's monitoring reports for states in India. This is to provide an example for the reader of the type of monitoring that Deworm the World conducts and to support the subsequent claims in our report. We have not included excerpts from Deworm the World's 2016 reports from India; we skimmed the reports to see if they appeared to be substantially different from the 2015 reports and felt that they were of similar quality.

  • Bihar 2015: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state. The objective of independent monitoring is to determine whether deworming is being implemented according to planned protocols. Two-stage probability sampling was used to select schools for coverage validation on deworming day and mop-up day. First, 125 blocks were selected from all 38 districts by probability proportional to size sampling, followed by random sampling of schools to provide statewide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students.
    During coverage validation an additional 748 randomly sampled schools were surveyed after deworming days to check whether deworming occurred, if reporting protocols were followed, and to validate the coverage reporting. Coverage validation data was gathered through interviews with headmasters and 3 students (in 3 different randomly selected classes in each school), and by checking of all class registers and reporting forms." DtWI Bihar 2015 Program report, Pg. 15.
  • Rajasthan 2015: "The process of monitoring and evaluation in each deworming round are performed in three ways: (1) process monitoring, (2) coverage reporting and (3) coverage validation. In Rajasthan, both process monitoring and coverage validation were carried out in schools, while only coverage validation was done at anganwadis. This is because the four-day long (10-13 February) deworming program in anganwadis made process monitoring a challenge.
    Process Monitoring, Coverage Reporting, and Coverage Validation: Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program. Evidence Action assesses the program preparedness during pre-deworming phase and selected independent monitors observe [sic] the deworming processes on Deworming Day and mop-up day. We conduct process monitoring in two ways: a) telephone monitoring and cross verification and b) physical verification by visiting schools and training venues. Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state. The objective of independent monitoring is to determine whether the program is being implemented according to planned protocols. Two-stage probability sampling was used to select schools for independent monitoring on Deworming Day and mop-up day. First 125 blocks were selected from all 33 districts by probability proportional to size (PPS) sampling, followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively. These 125 monitors visited 125 schools on Deworming Day and an additional 125 on mop-up day to check for adequate drug supplies and awareness materials, to confirm whether teachers had received training, and to assess knowledge of adverse event management protocols and reporting processes. Monitors gathered data through observation of deworming and through interviewing headmasters, teachers, and randomly selected students." DtWI Rajasthan 2015 program report, Pg 14.
  • Madhya Pradesh 2015: "In Madhya Pradesh, as preschool-age deworming was implemented through the BSM, monitoring efforts focused on the school-age program through the NDD. In the future, it will be important to expand monitoring to anganwadis to better understand program preparedness and performance…
    Evidence Action assesses the program preparedness during pre-deworming phase and selected independent monitors observe the deworming processes on Deworming Day and mop-up day. We conduct process monitoring in two ways: a) telephone monitoring and cross verification, and b) physical verification by visiting schools and training venues.
    A two-stage probability sampling process was followed to select schools for NDD, mop-up day and coverage validation. Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively. The monitors visited 125 randomly selected schools on NDD, and an additional 125 schools on mop-up day (February 14) to check for adequacy of drug supplies and awareness materials, and assess whether teachers had received training, and had knowledge of adverse event management protocols and reporting processes. Monitors gathered data by observing deworming and by interviewing headmasters, teachers, and randomly selected students. An additional 750 randomly sampled schools were surveyed from February 18-26 to check whether deworming occurred and reporting protocols were followed, and to validate the coverage reporting." DtWI Madhya Pradesh 2015 program report, Pg 15.
  • Chhattisgarh 2015 (coverage validation only): "The school database for random sampling in the 11 districts was obtained from the Ministry of Drinking Water and Sanitation, Government of India website. We visited 10 randomly selected schools in each block from the 55 blocks in these 11 districts. Therefore, they visited a total of 550 randomly sampled schools for coverage validation activities. In addition to the headmster's interview and verification of the deworming related documents, three randomly selected children from three different randomly selected classes were interviewed in each school. In addition, we also visited any anganwadis attached to the sampled schools. We could not achieve the targeted sample of 550 schools as two districts, Sukma and Bijapur, could not be covered given the high risk due to insurgency in these areas." DtWI Chhattisgarh 2015 coverage validation report, Pg. 3.
  • Delhi 2015: "Evidence Action assesses the program preparedness during the pre-deworming phase and selected independent monitors observe the processes on deworming and mop up days. We conduct process monitoring in two ways: a) telephone monitoring and cross verification, and b) physical verification by visiting schools and training venues.
    The method of stratified random sampling using proportional allocation approach was followed for selection of schools and anganwadis for deworming day, mop-up day, and coverage validation monitoring to provide state-wide estimates of indicators. We hired an independent research agency, Sigma Research and Consulting Private Limited that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively. These monitors were to visit total of 400 randomly selected schools and 400 randomly selected anganwadis; 80 schools and 80 anganwadis on deworming day and mop up day each (April 16 & April 20); and 240 schools and 240 anganwadis during coverage validation (April 23-27, 2015). The actual number of schools and anganwadis visited on each day is given in annexure (Table SA-1). The monitors visited the selected schools and anganwadis on deworming day, on mop up day to check for adequacy of drug supplies and awareness materials, whether teachers/anganwadi workers had received training, and knowledge of adverse event management protocols and reporting processes. Monitors gathered data through observation during deworming and interviews with headmasters, teachers, and anganwadi workers as well as of randomly selected students from schools. Additional randomly sampled schools and anganwadis were surveyed from April 23-27 to check whether deworming occurred, reporting protocols were followed, and to validate the coverage reporting." DtWI Delhi 2015 program report, Pg 14.
  • Bihar 2014: "In its attempt to evaluate the adherence of each process to guideline and time plan, Deworm the World supported a two-way monitoring strategy; a) Telephone monitoring and cross verification, and b) Physical verification by visiting the sites, schools, and training venues." DtWI Bihar 2014 program report, Pg 22. Pgs 23-30 describe activities over the various days.
  • Delhi 2013: "In order to carry out robust M&E activities, Deworm the World retained short-term resources comprised of district coordinators and telecaller to a) assess the readiness of the system to implement deworming, and b) to follow-up with districts and nodal officers (for anganwadis) for the return of data post-deworming. In addition, independent monitors were hired and trained by Deworm the World to assess preparedness, visit schools and anganwadis on deworming and mop-up days and carry out coverage validation post mop-up day. The Deworm the World team and officials from SHS and DHFW also made several field visits to monitor the entire deworming program before, during, and after deworming day." DtWI Delhi 2013 program report, Pg 12.
  • Rajasthan 2013: "In order to carry out robust M&E activities, Deworm the World hired district coordinators, telecallers and independent monitors as detailed below. In addition, staff and officials from DMHFW, RCEE and DWCD also made several field visits to monitor the entire deworming program before, during, and after deworming day." DtWI Rajasthan 2013 program report, Pg 11.
  • Bihar 2012:
    • Process: "Independent monitors visited a randomly selected sample of schools over four days – Deworming Day, Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 1216 schools (1.75% of 69,299 schools in Bihar) targeted for monitoring. From each of the 38 districts in Bihar, 2 blocks were selected by simple random sampling, for a total of 76 blocks. 2 clusters were randomly selected from each of the 76 blocks, excluding clusters with fewer than 8 schools. In each of these 152 clusters, 8 schools were randomly selected for a total of 1216 schools. Each monitor was assigned a block comprising 16 schools to be monitored. From this list of 16 schools, the monitor could visit any 4 schools on Deworming Day, another 4 on Mop-Up Day, and 8 more schools over the two Coverage Validation days. Out of the total sample of 1216 schools, monitors were able to survey 1196 schools comprising 302 schools on Deworming Day, 296 on Mop-Up Day, and 598 on Coverage Validation days." DtWI Bihar 2012 monitoring report, Pg 1.
    • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Bihar 2012 monitoring report).
  • Delhi 2012:
    • Process: "Monitoring visits were conducted by senior government officials. Random site visits by independent auditors occurred at a subset of training sessions and participating institutions. Coverage validation engaged independent auditors who conducted random site visits at a representative sample of schools and anganwadis to validate coverage statistics. This source of information was carefully compared with programme reports collated from each school and anganwadi to arrive at an accurate assessment of programme coverage." DtWI Delhi 2012 program report, Pgs 7-8.
    • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Delhi 2012 program report, Pgs 17-22).
  • Rajasthan 2012:
    • Process: "In order to evaluate the efficacy of the deworming protocol and process, independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 990 schools (1.1% of 90,488 schools in Rajasthan) targeted for monitoring. From each of the 33 districts in Rajasthan, 2 blocks were selected by simple random sampling. In each of these 66 blocks, 15 schools were randomly selected for a total of 990 schools. One monitor was assigned to each of these blocks. From the list of 15 schools in a block, the monitor could visit any 3 schools before Deworming Day for Preparation Monitoring, another 2 schools on Deworming Day, 2 more schools on Mop-Up Day, and 4 more schools over the two Coverage Validation days. The remaining 4 schools served as a buffer in case a particular school could not be visited. Hence, the actual sample size was 726 schools (0.8% of all schools)." DtWI Rajasthan 2012 monitoring report, Pg 1.
    • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Rajasthan 2012 monitoring report).
  • Bihar 2011:
    • Process: "Moreover, independent monitors were mobilized in each block (560 total) to visit 5% of all government schools on deworming and mop-up days to ensure that an adequate quantity of drugs were available, trained teachers were administrating drugs according to protocol, and community sensitization initiatives were successful in mobilizing non-enrolled children to attend. One auditor per district was in charge of supervising monitors to ensure quality monitoring of all schools." DtWI Bihar 2011 program report, Pg 9.
    • What we’ve seen: Monitoring forms used for data collection and full monitoring data from deworming day and mop-up day. We have not seen a summary of this data and have not analyzed it ourselves. (DtWI Bihar 2011 program report)

• 139

  For example, as part of its monitoring in Kenya, monitors visit sub-county trainings and teacher trainings:

  • "The intent of Sub-Country Training (SCT) sessions is to ensure that Sub-county and division-level trainers understand the purpose and procedure of deworming. The successful completion of this activity allows the division trainers to then conduct the same activity with teachers in their sub-counties.
    "PMCV teams attended 38 out of the total 111 SCTs conducted in the third year of the program. Field officers interview participants before and after the training and completed observations during the course of the activity."
    Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 5.
  • "Teacher Trainings (TT) are conducted by the division trainers using a “Teacher Training Booklet” as reference material. A quality TT session is considered to be one where the necessary content is covered and retained by participants. TTs also serve the function of distribution of drugs and materials (monitoring forms, posters) to teachers. It is the aim of the program to provide all schools with their required drugs and materials at teacher training sessions. Teachers are expected to use the “Deworming Day Checklist” to conduct operations on the day. They are also expected to sensitize other teachers at their schools who did not attend the training on deworming day procedures (see the section on deworming day contained in this report).
    "A total of 76 TT sessions were observed by PMCV field officers in Year 3. Of those training sessions, 13 were specifically SCH trainings."
    Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 7.
  • The trainings that monitors are sent to are randomly selected by a computer program. Paul Byatta, conversation with GiveWell, September 20, 2016

• 140

  Paul Byatta, conversation with GiveWell, September 20, 2016
  

• 141

  We believe that monitors are supposed to select every third teacher that arrives at the training until they have interviewed four teachers:
  "SURVEY INSTRUCTIONS [...]

  • Select every third participant that arrives before the start of the training.
  • Interview at least four participants."

  Deworm the World, Kenya Year 3, Pre TT form, Pg. 1

• 142
  For example:
  • Kenya. In Kenya, Deworm the World's monitors visit schools and communities before Deworming Day to assess their level of preparedness for the upcoming MDA. Schools are selected randomly, and then monitors visit the communities near the schools that are selected. Paul Byatta, conversation with GiveWell, September 20, 2016
    • "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.
    • "Use the PRE-DD-SCHOOL instrument to conduct interviews and note observations at the selected primary school before deworming day. In case selected school is closed, does not exist or has already dewormed move to the next nearby primary school. Allow Head teacher or Rep to consult with Head teacher/ REP when necessary. PRE-DD-SCHOOL should be completed along with PRE-DD-ECD or PRE-DD-COMMUNITY." Deworm the World, Kenya Year 2, Pre DD - School instrument, Pg 1.
    • "PMCV Field officers interviewed a total of 716 parents with children. Of these, 379 were parents of enrolled children across 130 different schools and 337 were parents of non-enrolled children. The number of parents with at least one child enrolled in early childhood development (ECD) was 283 of the sample population, or 65%. Just over one third or 35% of parents had no child enrolled in ECD. The average age of those children reported to be enrolled (by their parents) was 7.7 years, whereas the average age of non-enrolled children was 3.7 years.
      "PMCV field officers observed parents’ level of awareness of Deworming Day, their intentions regarding taking children to be dewormed and documented the primary source by which parents were receiving such information." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 11. The next section is titled "4.1. Pre-Deworming Interviews - Parents."
  • India: In India, telecallers phone schools ahead of Deworming Day to assess the school's preparedness. We have not yet requested data from these calls.
    • For example: "Our tele-callers place phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17.

• 143

  For example, the most recent monitoring report we have from Kenya (for the 2014-15 round, or Year 3) includes some sample results from the pre-Deworming Day visits:

  • The report notes that 97% of schools visited pre-Deworming Day had a teacher who had attended a training in the last fifteen days: "According to interviews with head teachers, 97% of these schools had a teacher who had attended training in the past 15 days. A further 86% of trained teachers had trained or sensitized other teachers on how to administer drugs and conduct deworming day. Almost all teachers (99%) found the Teacher Training Booklet to be ‘very’ or ‘somewhat’ useful in this process and 95% reported to use it often." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.
  • While 95% of schools had received some deworming tablets, only 87% of schools believed that they had received enough to cover the children at their school on Deworming Day: "At the time of PMCV visits, 95% of schools reported having received deworming tablets, with 97% of those schools having received them at the time of TT. This percentage is higher than those originally observed by PMCV officers with only 92% of schools reported to receive their drugs during TT. Upon further investigation, 87% of schools considered that they had received a sufficient supply of drugs for their current enrolled and ECD populations. It is likely that these schools requested additional drugs, because 98% of schools were observed to have sufficient drugs in place on deworming day. Only 5% of schools did not have all monitoring forms present prior to deworming day. Such schools have always sought support from the sub-county offices that organize additional prints or photocopying to ensure they have the forms on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16.
  • Note that Deworm the World shared the pre-Deworming Day data that informs the monitoring report and we have not yet analyzed it. There appears to be a large number of questions in the survey that were not included in the report. The survey instruments that Deworm the World's monitors use are long (e.g., see Deworm the World, Kenya Year 2, Pre DD - School instrument, in which both teachers and randomly selected students are interviewed). The monitoring reports only include a few summary metrics, and we are unsure how the metrics included in the report are chosen. It is possible that the report's metrics are those that reflect most favorably on Deworm the World and that vetting the full data could reveal that schools are less adequately prepared.

• 144
  • When selecting parents to interview in the community, monitors ask the school headmaster to point to a student's house near the school. The monitor begins by interviewing that student's family, then walks to subsequent households, skipping every other household until the monitor has interviewed at least 6 families with an enrolled school-age child and 6 families without. Paul Byatta, conversation with GiveWell, September 20, 2016
  • In Kenya, monitors are also asked to interview the Community Health Extension Worker (CHEW) in the community, since they assist with Deworming Day: "On selecting the CHEW to interview, there is only one CHEW per community. A few communities do not have CHEWs at all." Paul Byatta, attachments to email to GiveWell, September 23, 2016
  • For example, from Kenya: "Of those parents aware of deworming, only 41% knew the correct Deworming Day date, 81% knew the correct target population, and 48% knew the correct age group. These results indicate that although parents report being aware of deworming, almost half do not have the information required to attend (date). There is the scope to find a more robust method of ensuring information retention in awareness of deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 12.

• 145
  • India: In India, Deworm the World's monitors survey randomly selected schools (intended to be representative at the state level) on Deworming Day and Mop-Up Day to check whether deworming is occurring. They also visit another set of randomly sampled schools within a few weeks of Deworming Day, to check schools' records for evidence of deworming and interview children and teachers about the Deworming Day. Examples from several Deworm the World-supported states:
    • Bihar: "Two-stage probability sampling was used to select schools for coverage validation on deworming day and mop-up day. First, 125 blocks were selected from all 38 districts by probability proportional to size sampling (Probability proportional to size sampling (PPS) selected blocks in Bihar, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. Schools were then randomly selected from the selected blocks.), followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students.
      During coverage validation an additional 748 randomly sampled schools were surveyed after deworming days to check whether deworming occurred, if reporting protocols were followed, and to validate the coverage reporting. Coverage validation data was gathered through interviews with headmasters and 3 students (in 3 different randomly selected classes in each school), and by checking of all class registers and reporting forms." DtWI Bihar 2015 Program report, Pg. 15.
    • Rajasthan: "Two-stage probability sampling was used to select schools for independent monitoring on Deworming Day and mop-up day. First, 125 blocks were selected from all 33 districts by probability proportional to size (PPS) sampling (Blocks were selected by Probability proportional to size sampling (PPS) in Rajasthan, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. Schools were then randomly selected from the selected blocks.), followed by random sampling of schools to provide state-wide estimates of indicators." @DtWI Rajasthan 2015 Program report@, Pg. 14.
    • Madhya Pradesh: "A two-stage probability sampling process (Blocks were selected by Probability Proportional to Size (PPS) sampling, followed by random sampling of schools to provide state-wide estimates of indicators. We used PPS sampling to select blocks in Madhya Pradesh, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. After selecting blocks, we randomly selected schools from within these blocks.) was followed to select schools for NDD, mop-up day and coverage validation." @DtWI Madhya Pradesh 2015 Program report@, Pg. 16.
    • Delhi: "The method of stratified random sampling using proportional allocation approach (A random sample of population in which the population is first divided into distinct subgroups or strata, and random samples are then taken separately for each stratum proportional to the size of the subgroup.) was followed for selection of schools and anganwadis for deworming day, mop-up day, and coverage validation monitoring to provide state-wide estimates of indicators." @DtWI Delhi 2015 Program report@, Pg. 14.
  • Kenya: The schools observed on Deworming Day are randomly selected. Paul Byatta, conversation with GiveWell, September 20, 2016
    • "There is no direct sampling link between specific schools monitored on the pre-deworming day and those monitored on the deworming day. However, 20% of schools that are monitored for Pre-deworming, Deworming and Post-Deworming are randomly selected within the same sub-counties in which we monitored sub-county and teacher training events, while 80% of the schools that are monitored are randomly selected from the full pool of targeted schools, but stratified by county." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pg. 2
  • Vietnam: For the first round of monitoring, main schools were randomly selected, but satellite schools were not, because Deworm the World did not have a full list of satellite schools when the monitoring began. Going forward, all schools should be randomly selected. Additionally, during the first round, monitors called ahead to districts several days before Deworming Day to confirm that Deworming Day would be occurring in the schools they were assigned to monitor. It is possible that this notified schools ahead of time that a monitor would be coming, although Deworm the World's Paul Monaghan doubted it. Paul Monaghan, conversation with GiveWell, September 8, 2016

• 146
  • India: For example, in Delhi, monitors are instructed in the survey instrument, "Is the school open? [If not,] Did you go to the buffer school?" DtWI Delhi 2015 monitoring survey from deworming day, schools Pg 3. It is our impression that most schools were open when monitors visited. We spot-checked several states' data:
    • DtWI Bihar 2015 monitoring data from deworming day, schools, Column L shows that all schools monitors visited were open.
    • DtWI Rajasthan 2015 monitoring data from deworming day, schools, Column M shows that 124 of the 125 schools monitors visited were open.
  • Kenya: In Kenya, it is rare that monitors find schools to be closed or missing because Deworm the World has operated in Kenya for several years. However, monitors will try to go to another school if their school is closed. Paul Byatta, conversation with GiveWell, September 20, 2016
    • "When the monitor is unable to monitor the assigned school (because of closure, etc) we ask the monitor to i) to communicate the same with his supervisor and, ii) to visit the nearest school if the nearest school was not part of the randomly picked school." Paul Byatta, attachments to email to GiveWell, September 23, 2016
  • Vietnam: The backup schools are also randomly selected. Paul Monaghan, conversation with GiveWell, September 8, 2016

• 147
  See, for example, DtWI Delhi 2015 monitoring survey from deworming day, schools, Deworm the World, Kenya Year 3, DD - Main instrument, and Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day
  • Note that it is not actually dangerous to give the deworming pills to sick children; Deworm the World simply prefers not to deworm sick children to avoid causing people to associate the deworming pills with illness.
    • "Deworming pills should not be distributed to sick children...This is not because deworming pills could harm sick children. It is because DtWI wants to avoid people (and potentially the media) blaming the deworming pill for a child’s illness." DtWI 2013 GiveWell site visit

• 148
  • For example, in Kenya: "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment.

    Deworming was reported to occur inside classes in 47% of observed schools and outside in 53% of schools. Deworming was considered to be ‘systematic’ in 98% of schools. The correct dosage for albendazole is one tablet per child and the correct age is 2-14 years. These procedures were observed to be followed correctly by 86% of teachers observed by field officers.

    Coverage: Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools. Teachers were reported to correctly observe children swallowing PZQ in 99% of schools and ALB in 96% of schools. Observing children swallowing is most important when treating for SCH as the tablet does not taste pleasant and there are high chances of children spitting if not observed."
    Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17.

  • For example, in India: When monitors visit schools on Deworming Day, they make observations and ask questions related to the process of deworming, to check whether or not it is being implemented correctly. For example, monitors observe whether or not teachers ask their students if they are sick before giving them their pills and how many pills teachers give students. Monitors are prompted to answer:
    • "Did the teacher ask the children if they are sick/under medication before giving the medicine?"
    • "Did you see any child being given less than one deworming tablet?"
    • "Did you see any child being given more than one deworming tablet?"

    DtWI Rajasthan 2015 monitoring survey from mopup day, schools, Pgs 13-14.

• 149
  • Deworm the World is not supporting this monitoring in Ethiopia, because SCI and Deworm the World are supporting coverage surveys to be conducted after the Deworming Days there instead. Deworm the World also did not support this coverage validation monitoring for the first MDA it supported in Cross River, Nigeria, although it hopes to in subsequent rounds. Grace Hollister, edits to GiveWell's review, November 7, 2016
  • For example, this is what we heard about the monitoring conducted in Vietnam. Paul Monaghan, conversation with GiveWell, September 8, 2016. Mr. Monaghan noted that he had based the monitoring methods in Vietnam off of Deworm the World's methods used in its other countries.
  • In India: We believe coverage validation was a few days or up to a couple weeks after deworming days. For example:
    • Madhya Pradesh 2015: "The monitors visited 125 randomly selected schools on NDD, and an additional 125 schools on mop-up day (February 14) to check for adequacy of drug supplies and awareness materials, and assess whether teachers had received training, and had knowledge of adverse event management protocols and reporting processes. Monitors gathered data by observing deworming and by interviewing headmasters, teachers, and randomly selected students. An additional 750 randomly sampled schools were surveyed from February 18-26 to check whether deworming occurred and reporting protocols were followed, and to validate the coverage reporting." DtWI Madhya Pradesh 2015 program report, Pg 16
    • Delhi 2015: "These monitors were to visit total of 400 randomly selected schools and 400 randomly selected anganwadis; 80 schools and 80 anganwadis on deworming day and mop up day each (April 16 & April 20); and 240 schools and 240 anganwadis during coverage validation (April 23-27, 2015)." DtWI Delhi 2015 program report, Pg 14

• 150

  For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools

• 151

  For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools

• 152

  For example, in Kenya, it is our understanding that monitors interview three randomly selected children from three separate classes, for a total of nine students, at each school they visit. See Deworm the World, Kenya Year 3, Post DD - Coverage instrument. On Pgs 1-3 there are three spaces for randomly selected classes. The instructions read: "Thank the Head Teacher or designate and request to speak to pupils of the randomized class...CHOOSE CHILD 5, 10 AND 15TH ON FORM E. IF LESS THAN 15 CHILDREN, SELECT THE LAST CHILD. ENSURE TO INTERVIEW AT LEAST THREE CHILDREN. ASK THE TEACHER FOR PERMISSION TO SPEAK TO THEM ONE AT A TIME Ask questions in multiple ways for interviews with students, use local language if possible. Don’t rush responses. Try to make them feel at ease. Speak to one child at a time at a place where they are comfortable.... [interview questions]...END, MOVE TO THE NEXT SAMPLED CLASS"

• 153

  For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools

• 154

  Grace Hollister, edits to GiveWell's review, November 7, 2016

• 155
  • Vietnam: There were no checks on the monitors' work for the first round of treatment. Paul Monaghan, conversation with GiveWell, September 8, 2016
  • Kenya: In Kenya, Deworm the World does have some checks on monitors' work. A randomly selected 10% of the schools that are being visited by monitors are called the same day that the monitors visit, to ensure that the monitor is actually at the school. Paul Byatta, conversation with GiveWell, September 20, 2016
    • "We do back-checks for the pre-deworming, de-worming and post-deworming day… Generally, we seek to establish that the monitor did actually visit the school assigned to him/her, interviewed at least the headteacher of the school and that there is broad consistency on some of the data they collect by the monitor and the back-checker. For instance, the back-checker also asks about the availability of forms and drugs at schools, which is also information that is asked by the monitor. We share back-checks results with the short term monitors before we make the last payment to them. The other purpose is to improve our data collection training for the subsequent waves." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pgs 2-3
  • India: "In India, the following steps are implemented to serve as check on the IMCV work. Some of these are detailed in the contracts with IM firms – let me know if you’d like to see those:
    1. We collect photographs of all the schools and anganwadis (with the names) visited during PMCV in the states.
    2. We also collect signatures and mobile numbers of headmasters/school teachers and anganwadi workers, which is the part of PMCV format itself.
    3. Together with signature of the teachers/head masters, school stamp is also taken on signature sheet which is part of PMCV tools. Since we collect CAPI based data, monitors carry hard copy of signature sheet to get the stamp of the schools.
    4. Additionally, random calls are also made by state team/tele-callers to confirm that monitors visited to designated school and anganwadis.
    5. Evidence Action staff (including RCs & DCs) visits in selected schools and anganwadis on NDD and mop-up day to check if monitors visited the selected sites."
    Grace Hollister, edits to GiveWell's review, November 7, 2016

• 156
  • Grace Hollister, conversation with GiveWell, September 1, 2016
  • For example, in Kenya: "Does the monitoring team have an estimate of how many non-enrolled children the deworming program reaches? We get this from treatment forms that schools submit back via “the reverse cascade”. For PMCV, we monitor that teachers are aware they should be treating non-enrolled, and they are aware of age categories targeted for non-enrolled. When we do data audit on treatment forms, we also check that data for on non-enrolled is entered accurately." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pg 3

• 157
  • Kenya: "Show student 3 tablets, ask: Which one of these three tablets did you take? Circle the indicated tablet." Deworm the World, Kenya Year 3, Post DD - Coverage instrument
  • Vietnam: "What type of medicine was given out? (prompt the child with options)" and "Do you remember the color of the tablet? (do not prompt)" are questions on the coverage validation survey form for student interviews. Deworm the World, Vietnam 2016 monitoring survey form for coverage validation, Pg 7.
  • India: In India, monitors show children the deworming tablet and ask if it's the tablet that they took, which we do not believe is a rigorous method of verifying student answers. For example, see Pgs 20-23 of DtWI Bihar 2015 monitoring survey for coverage validation, schools

• 158

  Note: We have not included examples from Deworm the World's 2016 reports from India; we skimmed the reports to see if they appeared to be substantially different from the 2015 reports and felt that they were of similar quality. We have also not included results from Year 4 of Kenya's program; we have only seen a report on monitoring done before and during Deworming Day for Year 4 (Deworm the World, Kenya process monitoring report, Year 4)

• 159

  Deworm the World notes that an important difference between the two programs is scale: The Kenya program is significantly smaller than the India program, thus it may be easier to execute the program more effectively. Grace Hollister, email to GiveWell, June 9, 2016

• 160
  • Note that Deworm the World provided "light" technical assistance to Chhattisgarh in 2015.
  • Vietnam: Deworm the World, Vietnam Independent Monitoring Report, 2016 and Paul Monaghan, conversation with GiveWell, September 8, 2016.
  • Kenya:
    • See Deworm the World, Kenya Deworming Day data, Year 3
      • Note that we focus on questions related to Albendazole in our analysis. Deworm the World also supports some treatments for schistosomiasis (treated with Praziquantel) in Kenya, but the large majority of schools surveyed were only treating with Albendazole. Of the five schools monitors observed that were also treating with Praziquantel, all five reported that they had sufficient Praziquantel. Deworm the World, Kenya Deworming Day data, Year 3
      • We chose sample questions to present based on a) what we thought was easy to understand, b) our sense of what information would be most likely to change our view about Deworm the World, and c) the metrics that we felt were most comparable to the metrics we had chosen for India's program, so that we could compare the two programs.
    • Monitors ask some questions regarding preparedness during Deworming Day. For example, during Deworming Day visits, monitors check the number of drugs ordered against the number of students enrolled.
      • "Approximately how many Albendazole tablets have you received? WRITE NUMBER (check physical drugs)" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 7.
      • "How many ECD children are enrolled at the attached ECD centre?WRITE NUMBER" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 10.
      • "How many primary schoolstudents are enrolled in this school? (Exclude attached ECD)WRITE NUMBER" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 2.
    • The monitors then observe a randomly selected class, which includes checking to see if drugs run out during deworming, and interview teachers after the deworming has occurred about whether or not there were sufficient materials.
      • "Randomly Selected Class and Stream: (To be assigned before school visit)" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 1.
      • "Answer the following questions at the end of the deworming exercise (for STH):...Did the school seem to have enough tablets of Albendazole for the day? SINGLE CODE" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 23.
      • "HEAD TEACHER POST-INTERVIEW...Was the availability of drugs sufficient for today? SINGLE CODE" Deworm the World, Kenya Year 2, DD - Main instrument, Pg 24.
    • Our analysis provides slightly different results than what was presented in the monitoring report. For example, the most recent summary report from Kenya reported that 98% of schools visited on Deworming Day had ordered sufficient drugs, 9% of schools were observed to run out of drugs during Deworming Day, and 92% of head teachers reported that their school had sufficient drugs. We do not know the reasons for this discrepancy. Both the summary report's results and the results from our own data analysis indicate a high level of school preparedness for Deworming Day.
      • "Pre-Deworming: The percentage of target schools with the appropriate drugs (according to the number ordered and enrolment numbers) - albendazole (ALB) for treating STH and praziquantel (PZQ) where required for treating SCH - in place on Deworming Day was 98% or 243 out of 247 sampled schools.
      • Deworming: Only 9% percent of sampled schools were observed to run out of drugs on Deworming Day. However it was noted that only 87% of schools had enough PZQ for treating SCH in non-enrolled children.
      • Post-Deworming: According to interviews with head teachers post-Deworming Day, 92% of schools recorded a sufficient supply of drugs. This marginal difference between pre and post-reports of sufficient drug supply may indicate the underestimation of the drugs required to treat enrolled, non-enrolled, and ECD children. Indeed, this assumption is further supported by reports by PMCV field teams regarding an insufficient supply of drugs in 9% of sampled schools."

      Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pgs 16 - 17. See the table for results from our analysis, or Deworm the World, Kenya Deworming Day data, Year 3.
      Note:We are not sure what to make of the fact that some of the figures reported seem to slightly contradict each other. If 9% of schools were observed to run out of drugs, then 98% couldn't have ordered a sufficient number of drugs.

    • Note that we include several schools in our analysis that monitors did not record observations for (reasons include that the school was closed or did not exist).
  • India: The results of some of the interviews and observations from Deworming Day and coverage validation are described below.
    • "Please choose a class Randomly for questions from class teacher." DtWI Bihar 2015 monitoring survey for coverage validation, schools, Pg. 12.
    • "Select a child randomly and Take the child outside the class/away from the teacher." DtWI Bihar 2015 monitoring survey for coverage validation, schools, Pg. 14.
    • Bihar "First, 125 blocks were selected from all 38 districts by probability proportional to size sampling, followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students." DtWI Bihar 2015 Program report Pgs 15-16
    • Chhattisgarh "We visited 10 randomly selected schools in each block from the 55 blocks in these 11 districts. Therefore, they visited a total of 550 randomly sampled schools for coverage validation activities. In addition to the headmaster’s interview and verification of the deworming related documents, three randomly selected children from three different randomly selected classes were interviewed in each school." DtWI Chhattisgarh 2015 coverage validation report, Pg 3
    • Sources
      • Deworming day data from each state: DtWI Bihar 2015 monitoring data from deworming day, schools, DtWI Delhi 2015 monitoring data from deworming day, schools, DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools, DtWI Rajasthan 2015 monitoring data from deworming day, schools, DtWI Delhi 2015 monitoring data from deworming day, anganwadis
      • Mopup day data from each state: DtWI Bihar 2015 monitoring data from mopup day, schools, DtWI Delhi 2015 monitoring data from mopup day, schools, DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools, DtWI Rajasthan 2015 monitoring data from mopup day, schools, DtWI Delhi 2015 monitoring data from mopup day, anganwadis
      • Coverage validation data from each state: DtWI Bihar 2015 monitoring data for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools, DtWI Delhi 2015 monitoring data for coverage validation, schools, DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools, DtWI Rajasthan 2015 monitoring data for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis, DtWI Delhi 2015 monitoring data for coverage validation, anganwadis, DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis
      • Deworming day survey instrument from each state: DtWI Bihar 2015 monitoring survey from deworming day, schools, DtWI Delhi 2015 monitoring survey from deworming day, schools, DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools, DtWI Rajasthan 2015 monitoring survey from deworming day, schools, DtWI Delhi 2015 monitoring survey from deworming day, anganwadis
      • Mopup day survey instrument from each state: DtWI Bihar 2015 monitoring survey from mopup day, schools, DtWI Delhi 2015 monitoring survey from mopup day, schools, DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools, DtWI Rajasthan 2015 monitoring survey from mopup day, schools, DtWI Delhi 2015 monitoring survey from mopup day, anganwadis
      • Coverage validation survey instrument from each state: DtWI Bihar 2015 monitoring survey for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools, DtWI Delhi 2015 monitoring survey for coverage validation, schools, DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools, DtWI Rajasthan 2015 monitoring survey for coverage validation, schools, DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis, DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis, DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis

• 161

  "[GiveWell]:I forgot to ask on the call if there's anything significantly different we should expect for the monitoring from Deworm the World in the future. For example, should we expect the Ethiopia and Nigeria monitoring to look similar to the type of monitoring Deworm the World has conducted in other places?
  [Deworm the World]: Yes, the monitoring will be relatively similar in other locations but does get customized based on the needs/interest of the government and time/budget availability. For instance, in Vietnam we were not able to conduct training monitoring during the MDA earlier this year." Grace Hollister, email to GiveWell, September 13, 2016

• 162

  Deworm the World and SCI, Ethiopia coverage survey

• 163
  • DtWI Bihar 2011 prevalence survey report
  • DtWI Bihar 2015 Prevalence Survey report
  • DtWI Rajasthan 2012 prevalence survey report
  • DtWI Rajasthan 2013 prevalence survey report (Deworm the World re-surveyed Rajasthan because there were methodological weaknesses in the first prevalence survey)
  • DtWI Delhi 2012 prevalence survey report
  • Deworm the World notes: "Follow up surveys should take place after multiple rounds of treatment (WHO guidance is between three and six years of treatment), hence why only one follow-up has taken place to date." Grace Hollister, email to GiveWell, June 9, 2016

• 164
  • Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013
  • "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected." DtWI Bihar 2011 prevalence survey report, Pg 3.
  • "Predicted prevalence is modelled using survey data from 2010-2011, point-level environmental data (annual temperature and rainfall) and a district level proxy for socio-economic status (literacy)." DtWI Bihar 2011 prevalence survey report, Pg 5.

• 165
  • "The prevalence of STH (including hookworm, A. lumbricoides, and T. trichiura) across these states in Bihar was 67.5% with district prevalence ranging from 49.0% to 79.6%." DtWI Bihar 2011 prevalence survey report, Pg 4.
  • 67% = 782/1159. From "Table Three: Cumulative prevalence of each species by district": "Total - Number Students" = "1159"; "Total - % Any infection" = "(782) 67.5%". DtWI Bihar 2011 prevalence survey report, Pg 4.
  • Selection of districts and schools: "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected from the state schools database." Pg 9.
  • Selection of students: "Within each school 65 children aged 6 and above representing both sexes equally from class 1 to 6 were randomly selected, class by class, using random number tables." Pg 9.
  • Technique: "Screening of infection for STH was based on a double Kato-Katz smear of 41.7 mg prepared from fresh stool samples." Pg 11.
  • Participation rate: "From a total of 1,281 school children registered in the survey and provided with pots, 1,159 returned samples." (90.5%) Pg 11.
  • Note on two stages of first prevalence survey based on DtWI Bihar 2011 prevalence survey report and Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013
    • Based on the timing of the decision to deworm in Bihar, there was only time to collect prevalence surveys from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet DtWI's typical standard of quality.
    • Before the DtWI-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report, Pg 9.
    • Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness. (Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013)

• 166

  "The prevalence of STH (including hookworm, A. lumbricoides, and T. trichiura) across these states in Bihar was 67.5% with district prevalence ranging from 49.0% to 79.6%." DtWI Bihar 2011 prevalence survey report, Pg 4.

• 167
  • Albendazole is used for treatment of both lymphatic filariasis (where it is used in combination with either DEC [Diethylcarbamazine] (as in India) or ivermectin) and of STH. Therefore, a round of treatment for lymphatic filariasis also treats STH. In this review, the term "deworming round" includes any MDA of albendazole.
  • "A Memorandum of Understanding (MOU) was signed on March 5, 2010 among State Health Society Bihar, Bihar Education Project Council, and Deworm the World Initiative to implement the school-based deworming program in the state for treatment of STH. Based on Prevalence Survey findings, which suggested treatment recommendation of WHO, the Government of Bihar decided to implement biannual state-wide deworming beginning in 2011. Since then, Evidence Action has extended technical assistance to an annual round of deworming for all school-age children through a school-based model. The second round of deworming treatment was provided through the National Filaria Control Program (NFCP), which conducts annual mass drug administration of albendazole to the 2 years and above population at the community. In 2011, a total of 16.7 million children were dewormed at schools, earning the distinction of being the world's largest school-based deworming program. In 2012, 16.33 million children were dewormed in Round 2. 17.47 million, including 16.2 school-age children, were dewormed as part of Round 3 in 2014." DtWI Bihar 2015 Program report, Pg 6.
  • "Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012. Four deworming rounds or distribution of albendazole during LF-MDA were missed during the period" DtWI Bihar 2015 Prevalence Survey report, Pg 5.

• 168
  • "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5.
  • "The ‘2014’ [LF-MDA] occurred in February/March 2015. There was no LF MDA in the calendar year of 2014. But, since the LF MDA occurred prior to the closing of the 2014 - 2015 financial year (March 31, 2015). This is apparently common practice and acceptable to the government of India, and that is what is reported on the website. There was no LF MDA in the calendar year of 2014, as there was no DEC [Diethylcarbamazine] in Bihar until February 2015. Therefore a round did not occur prior to the second survey." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015

• 169
  • Prevalence figures
    "On the basis of the collected and analyzed data, the overall weighted prevalence of any STH in Bihar was calculated as 35%. Prevalence in different agro-climatic zones ranged from 20% to 50%" DtWI Bihar 2015 Prevalence Survey report, Pg 2.
  • Representative survey design
    "Per the 2011 census, there were 18,884,945 children aged 5-10 years in Bihar. Assuming one sentinel school is required for 300,000 targeted children, we needed at least 63 schools (rounded to 65) to monitor the performance of the deworming program. Collecting samples from 50 children per school, the minimum sample size required for estimation of prevalence using the sentinel site method was 3,250.
    Bihar is divided into three agro-climatic zones (Figure 2) – the North West Alluvial Plains (Zone 1) consisting of 12 districts, the North East Alluvial Plains (Zone 2) consisting of 9 districts and South Bihar Alluvial Plains consisting of 17 districts (Zone 3). The three zones respectively accounted for 42%, 22% and 36% of 5-10 years population in the state. NIE randomly selected 65 schools from the three zones, proportionate to the percentage of 5-10 year population in each of the zones. Thus, the study needed 28, 14 and 23 schools from zones 1, 2 and 3 respectively.
    NIE followed a 2-stage sampling procedure for selecting sentinel schools. In the first stage, NIE randomly selected 14 districts from Bihar (six from zone 1, three from zone 2 and six from zone 3) to meet logistical (teams could not spend too much time traveling and setting up temporary field laboratories, because it reduced the number of samples the teams could analyze), geographic dispersion (the survey needed to be geographically dispersed to ensure the best estimates of prevalence), and time constraints (there were only 20 days available for the survey). In the second stage, NIE line-listed all the primary schools (with the total strength of ≥ 60 children) of the districts selected from each zone. They then selected the required number of schools for each zone randomly from the list of schools in the selected districts. To select the required number of children, we assigned a random number (between one and five) to classes of the selected school. The survey was initiated from the class corresponding to the random number assigned for the school and field teams enumerated the children present in the class starting from roll number one on the attendance register. If the number of children in the selected class was &lt 50, children from the next class were selected. This procedure was followed until 50 children from each school were selected." DtWI Bihar 2015 Prevalence Survey report, Page 8.

• 170DtWI Bihar 2015 Prevalence Survey report
• 171

  Deworm the World has supported three MDAs in Bihar over the last four years: "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5.

• 172
  • The National Vector Borne Disease Control Programme LF treatment coverage 2015
  • The treatment for LF in India is annual administration of DEC [Diethylcarbamazine] or DEC and Albendazole:
    "Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
    Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except:
    - children below 2 years
    - pregnant women
    - seriously ill persons
    (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012.
  • "Evidence Action - Deworm the World provided technical assistance for Round 1 in February 2011, followed by second and third rounds in September 2012 and January 2014 respectively. The National Filaria Control Program, which co-administers albendazole and diethylcarbamazine citrate annually to all people in the community older than 2 years (excluding pregnant women and the seriously ill), targets all 38 districts in Bihar. The treatment for lymphatic filariasis was therefore intended to serve as the second annual dose of albendazole for school-age children, ideally timed to take place six months apart. Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012." DtWI Bihar 2015 Prevalence Survey report, Pg 5.

• 173
  • "Revised Strategy
    Annual Mass Drug Administration with single dose of DEC [Diethylcarbamazine] was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
    Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except: - children below 2 years - pregnant women - seriously ill persons (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012
  • The National Vector Borne Disease Control Programme LF treatment coverage 2015

• 174
  • "The ‘2014’ MDA occurred in February/March 2015. There was no LF MDA in the calendar year of 2014. But, since the LF MDA occurred prior to the closing of the 2014 - 2015 financial year (March 31, 2015). This is apparently common practice and acceptable to the government of India, and that is what is reported on the website. There was no LF MDA in the calendar year of 2014, as there was no DEC [Diethylcarbamazine] in Bihar until February 2015. Therefore a round did not occur prior to the second survey." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • "Since the launch of the school-based deworming program, the National Filaria Control Program has taken place once, in April 2012. Four deworming rounds or distribution of albendazole during LF-MDA were missed during the period" DtWI Bihar 2015 Prevalence Survey report, Pg 5.
  • "The Bihar program was originally designed so that one round of albendazole administration would come from community-based lymphatic filariasis (LF) treatment (LF is endemic throughout the state) and a second round would come through the school-based deworming program. For a variety of reasons, LF treatment has not been consistently implemented. It is possible that more consistent albendazole administration as part of LF treatment would have resulted in a more significant drop in STH prevalence." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 2
  • "Kenya and India are the countries where Deworm the World operates two of its biggest programs and where Ms. Hollister has the greatest familiarity with the state of LF treatment. In both countries, LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015, Pg 1

• 175
  • 39% = 16801/12090 - 1
  • 2010-2011 "PER CAPITA NET STATE DOMESTIC PRODUCT AT FACTOR COST - Constant price... Base : 2004-05)": 12090 billion Rupee
  • 2014-2015 "PER CAPITA NET STATE DOMESTIC PRODUCT AT FACTOR COST - Constant price... Base : 2004-05)": 16801 billion Rupee
  • Reserve Bank of India, GDP per capita, Table 10, September 16, 2015

• 176
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "The M&E includes a series of pre- and post-intervention, repeat cross-sectional surveys in a representative, stratified, two-stage sample of schools across Kenya. District stratification was based on both geography and anticipated infection prevalence. The programme contains three tiers of monitoring: i) a national baseline survey including 200 schools in 20 districts, which aims to establish an accurate national measurement of infection levels; ii) surveys conducted pre and post intervention (pre-post surveys), which monitor 60 of the 200 schools before and immediately after the deworming activity to evaluate reductions in infections that can be directly attributed to programme implementation; and iii) high frequency surveys in 10 schools, distinct from the 60 pre-post schools, at four time points in a single year, before, during, and after treatment (Figure 1).
    Two hundred schools were examined at baseline and will be re-examined in year 3 and 5 in order to monitor long-term changes in worm infection at a national level both in terms of prevalence and intensity of infection. This sample size was chosen in order to be able to detect a five-percentage-point change in prevalence across years, assuming power β = 0.80 and test size α = 0.05, and considering the anticipated variance in prevalence. Sixty schools (a subset of the 200) will be surveyed every year for 5 years, before each treatment round to evaluate programme impact and 3–5 weeks post-treatment to evaluate treatment efficacy [2]. The same schools will be surveyed each year in the 60 pre-post survey schools, whereas in the remaining 130 schools, a different random sample of schools will be undertaken each year. In the 10 high frequency schools, a cohort of children will be followed-up longitudinally and assessed for haemoglobin concentration in addition to parasitological outcomes." Mwandawiro et al. 2013 Pgs 1-2.

• 177
  • Mwandawiro et al. 2013
  • KEMRI prevalence report - Year 2
  • Deworm the World, Kenya 2014-2015 program report, Pgs 13-14

• 178

  See Mwandawiro et al. 2013. Because this is a published paper that describes the methodologies of the prevalence surveys (sampling, timing, outcomes measured, etc.) over the course of Kenya's five-year deworming program, we consider this to effectively be a pre-registration.

• 179

  Deworm the World does not have full details about KEMRI's prevalence surveys easily available: to avoid the possibility of introducing bias, KEMRI has kept its methodologies private (so that, e.g., Deworm the World would not learn which schools KEMRI is evaluating). We intend to pursue additional details in the second half of 2016. Grace Hollister, conversation with GiveWell, June 13, 2016

• 180
  • "Based on available data and predictive maps [9,10], STH was assumed to be endemic in 66 districts. From these districts, grouped into strata, 20 districts were randomly selected for M&E in the first sampling stage, with number of districts per province proportional to population: six districts from Western Province, three from the Rift Valley, five from the Coast, and six from Nyanza (Table 1)." Mwandawiro et al. 2013 Pg 3.
  • All 66 districts in which STH was assumed to be endemic are supported by Deworm the World: "From the year 2012, the ministries of health and of education of Kenya plan to deworm all school –age children who live in 66 districts identified as having a high prevalence of soil-transmitted helminth (STH) infection and schistosomiasis in four provinces." KEMRI prevalence report - Year 2 Pg 2
  • Deworm the World told us that the KEMRI surveys are only occurring in districts funded by CIFF. CIFF funds most of the districts in the national deworming program, but the END Fund also funds several districts. Grace Hollister, conversation with GiveWell, June 13, 2016
  • Kenya's national deworming program supports more than 66 districts: Deworm the World told us that "districts" were recently re-divided into "sub-counties" (with each sub-county being approximately the same size as districts were previously). Grace Hollister, conversation with GiveWell, June 13, 2016 In Year 3 of the Kenya program, Deworm the World supported MDAs in 111 sub-counties. Deworm the World, Kenya 2014-2015 program report, Pg 10.
  • "[GiveWell]: How were the 66 districts (from which the 20 districts used in the survey were randomly selected) initially selected? [Charles Mwandawiro]: They 66 districts were picked from a map developed using historical data (research studies and MoH data). These were given by the MoH and confirmed by a team in a meeting that they were the districts which CIFF wanted to be covered (where worm infections were definitely known to occur)" Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016
  • If the 66 surveys were not selected randomly, this could cause the prevalence surveys to be somewhat biased. For example:
    • If only districts with high prevalence were chosen, it could be the case that the schools surveyed are the schools most incentivized to implement deworming effectively.
    • Or, if only districts with high prevalence were chosen, perhaps children in such districts would be more willing to take deworming pills (it is plausible that children in higher prevalence districts might feel noticeably better after deworming, and therefore be more cooperative when taking the deworming pills each year).
    • If the 66 districts were chosen based on ease of access to schools (e.g., to make surveying easier), the surveys might only be evaluating the locations in which deworming is easiest to implement.
  • "[GiveWell]: Are there ways in which these surveys might not accurately reflect the impact of the program that we might not currently understand? [Charles Mwandawiro]: Yes. For example there could be other areas (out of the 66 districts) that have infections but not covered by the programme. Such areas could be Nairobi (unprogrammed deworming) and parts of Rift Valley. Otherwise, by and large, the surveys reflect the impact of the programme as designed." Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016

• 181
  • 200 schools were randomly selected from the 20 districts for the baseline survey in early 2012, and 60 of those schools are re-surveyed before and after their MDAs each year.
    • See Figure 1 of Mwandawiro et al. 2013, Pg 3. Note the post-MDA survey is supposed to occur 3-5 weeks after the treatment. We do not know if the surveys occurred on the timeframes they were intended to, and we do not know how much time passes between the pre-MDA survey and the MDA itself.
    • "At the second sampling stage, primary schools were randomly selected from within the chosen 20 districts." Mwandawiro et al. 2013 Pg 3.
    • "The article describes the M&E design of the Kenya national SBD programme and presents results from the baseline survey conducted in early 2012." Mwandawiro et al. 2013 Pg 1.
  • Of the remaining 140 schools, 10 will be surveyed both pre- and post- MDA in Years 1, 3, and 5, while 130 are surveyed only pre-MDA in Years 1, 3, and 5.
    • See Figure 1 of Mwandawiro et al. 2013, Pg 3.
    • Note that the 130 schools surveyed pre-MDA in Years 1, 3, and 5 are not the same schools each time; they are re-selected randomly: "The same schools will be surveyed each year in the 60 pre-post survey schools, whereas in the remaining 130 schools, a different random sample of schools will be undertaken each year." Mwandawiro et al. 2013 Pgs 2-3.
  • In the 60 schools surveyed each year, the sampled children are randomly selected each year, meaning the surveys do not follow precisely the same population year-to-year.
    • "In each school, 18 children (9 girls and 9 boys) were sampled randomly from each of six classes - one Early Childhood Development (ECD) class and classes 2–6 - using computer generated random number tables, for a total of approximately 108 per school. The sampling within these specified classes aimed to target children aged 5–16 years. " Mwandawiro et al. 2013 Pg 3. Also see Figure 1 of Mwandawiro et al. 2013, Pg 3. We believe the populations will be very similar, so we are unconcerned that different children are surveyed each year.
  • The surveys measure the prevalence and intensity of both soil-transmitted helminths and schistosomes.
    • "Stool samples were obtained for each child and two slides prepared and examined for the presence and intensity of STH species and S. mansoni using the Kato Katz method, with the concentration of eggs expressed as eggs per gram (epg) of faeces. Urine samples were obtained only from children in Coast Province (where Schistosoma haematobium is widespread) and investigated for presence and intensity of S. haematobium using the urine filtration method, with the concentration of S. haematobium eggs estimated in eggs per 10 ml urine. Egg counts were performed only up to 24,000 epg and 1,000 eggs/ 10 ml urine, respectively. Infection intensities above these values were, therefore, not further quantified." Mwandawiro et al. 2013, Pg 3.
  • Additionally, in the 10 schools surveyed pre- and post-MDA every odd year, anaemia and educational outcomes are measured. We have not seen any results from the anaemia and educational outcome tests, but we only have results from Year 1 and Year 2 and we are not sure if these outcomes were measured in the first year. From Mwandawiro et al. 2013:
    • "In the 10 high frequency schools, a cohort of children will be followed-up longitudinally and assessed for haemoglobin concentration in addition to parasitological outcomes." Pg 3.
    • See Figure 1, Pg 3.
    • "In the 10 “high frequency” schools, finger-prick blood samples were obtained and analysed using a HemoCue photometer (HemoCue, Angelhom, Sweden) to estimate haemoglobin concentration." Pgs 3-4.
  • The survey protocol does not require researchers to provide treatment to students immediately after they are tested (we have seen this requirement in some of SCI's prevalence surveys): "[GiveWell]: When children are surveyed pre-MDA and found to have worms, are they then ethically required to be treated? Or does treatment occur during the MDA? [Charles Mwandawiro]: Ethically they are required to be treated but because of the number and geographical distribution they are all treated during the MDA and not separately." Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016

• 182
  • The data in the table for soil-transmitted helminths is from KEMRI prevalence report - Year 2, Pg 4 and Deworm the World, Kenya 2014-2015 program report, Pgs 12-14.
  • The data for schistosomiasis is from Table 4 of Mwandawiro et al. 2013, Pg 6 and Box 5 of KEMRI prevalence report - Year 2, Pg 15.
  • Note that the two sources we've seen describing the results from the prevalence surveys do not exactly match. In Mwandawiro et al. 2013, the prevalence at baseline was 15.6% for hookworm, 18.0% for Ascaris, and 6.6% for Trichuris, whereas the KEMRI prevalence report - Year 2 shows the baseline prevalence being 16.9% for hookworm, 19.2% for Ascaris, and 5.4% for Trichuris. (Pg 5 and Pg 4, respectively) We are not sure where this discrepancy comes from, but the numbers are close enough that we don't find it too concerning.

• 183In years 1, 3, and 5, 10 schools (randomly selected from the 200 schools at baseline but distinct from the 60 schools that will be followed each year) will be surveyed both pre-MDA and post-MDA. These schools will be surveyed intensively, with three post-MDA surveys measuring for worm prevalence and additional variables, such as anaemia and educational outcomes. The same 10 schools will be surveyed for each of Year 1, 3, and 5, and the same students from each of the 10 schools will be surveyed. Mwandawiro et al. 2013, Pgs 2-3.
• 184
  • The Year 1 baseline data in the table is from Table 4, Mwandawiro et al. 2013, Pg 6.
  • The Year 1 post-MDA data is from Table 11 of KEMRI prevalence report - Year 2, Pg 20. The report only gave prevalences by county, so we took the median of all counties and the median of counties within the Coast Province (see this Wikipedia article) for a rough comparison.
  • The data pre- and post- the Year 2 MDA is from KEMRI prevalence report - Year 2, Pgs 15 and 18.
  • The data for Year 3 is from Deworm the World, Kenya 2014-2015 program report, Pgs 12-14.
  • Note that the two sources we've seen describing the results from the prevalence surveys do not exactly match. In Mwandawiro et al. 2013, the baseline Year 1 prevalence for S. Mansoni at baseline was 0% in the Coast Province and 2.1% in all provinces, while the prevalence for S. haematobium was 14.8% in the Coast Province (Table 4, Pg 6). However, in KEMRI prevalence report - Year 2, the baseline Year 1 prevalence is reported as 0.1% for S. mansoni in the Coast Province and 1.8% in all provinces (Box 5, Pg 15), while the prevalence for S. haematobium is reported as 18.0% (Pg 18). We are not sure where these discrepancies comes from.

• 185
  • S. haematobium was only measured in the Coast Province: "Urine samples were obtained only from children in Coast Province (where Schistosoma haematobium is widespread) and investigated for presence and intensity of S. haematobium using the urine filtration method, with the concentration of S. haematobium eggs estimated in eggs per 10 ml urine." Mwandawiro et al. 2013, Pg 3.
  • "Urine samples were examined for S. haematobium infections in 9 schools in Coast Province in Kilifi and Kwale Counties." KEMRI prevalence report - Year 2, Pg 18.

• 186Table 4, Mwandawiro et al. 2013, Pg 6.
• 187
  • We were told that in the first year of Kenya's deworming program, the MDAs occurred in 12 different waves, as opposed to all on the same day. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • "[GiveWell]: How much time passes between the initial survey and the MDA? How much time passes between the MDA and the follow-up survey? [Charles Mwandawiro]: We usually do 3-5 weeks pre-MDA and 5-6 weeks post-MDA because we do both schistosomiasis and STH" Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016
  • It might be possible that the timing of the surveys could also affect observed outcomes; for example, worm burden might be especially high during some seasons in Kenya, so if pre-MDA surveys were conducted during high-burden seasons, while post-MDAs were conducted later in low-burden seasons, the effect of deworming might appear to be greater than it actually is.

• 188
  • See Figure 1 of Mwandawiro et al. 2013, Pg 3. This might be less of an issue in Year 3 and Year 5, when KEMRI plans to survey an additional 130 schools randomly selected from the same 20 districts.
  • Note that the schools are not told that they are being monitored to assess the program: "[GiveWell]: Do teachers and administrators know that they are staffing the schools that are being used to assess the program? It seems possible that in the 60 schools that are re-surveyed each year, teachers may become aware of this. [Charle Mwandawiro]: The teachers were not told so. But they possibly know that their schools are being used as examples to monitor decline of infection. This however is yet to be determined and they is no reason so far to think so since we have not seen this reflected in what we observe and get." Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016

• 189

  Deworm the World supports program costs in Cross River, Nigeria and expects to support program costs in future Nigeria states. It expects to support program costs in Pakistan, and possibly Indonesia. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 190

  We attempt to learn if there was a possibility that the program would have been funded by other donors. However, given that there is a global funding gap for deworming treatments (more), we suspect that in most cases where Deworm the World pays for a new deworming program, it is increasing the number of children dewormed.

• 191

  For example, Deworm the World is using its unrestricted funding to support its program in Nigeria and expects to financially support MDAs in Pakistan. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016

• 192

  "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2.

  "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.

• 193

  Professor Devesh Kapur Biography 2013

• 194

  Devesh Kapur conversation October 14th 2013

• 195

  "Some school health programs, such as the WIFS program in Delhi, are rolled out too quickly and their trainings are carried out poorly. Then they receive negative media attention because of students’ adverse reactions to treatments. Negative media can cause delays or cancellation of school health programs. The WIFS program has also received negative media attention in the states of Haryana and Odisha." DtWI 2013 GiveWell site visit, Pg 10.

• 196

  "There have been very few adverse reactions to deworming treatments in DtWI-supported states because of the quality of Deworm the World's trainings." DtWI 2013 GiveWell site visit, Pg 10.

• 197
  • "National funds for deworming are part of the [Weekly Iron Fortification Supplements] WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12.
  • "Wherever possible, Deworm the World works to get deworming added as a line item in the budget. Otherwise, it is difficult to ensure that long-term funding will exist for the program." DtWI 2013 GiveWell site visit, Pg 10.
  • "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.

• 198
  • Alix Zwane conversation August 30th 2013
  • "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.
  • Assam has also announced its intentions to move forward with its deworming program in March 2013: "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. As reported by NRHM, Assam any [sic] school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam midday meal report 2013, Pg 11.
  • According to Deworm the World, "No schools in Assam have been covered under the School Health Program to provide deworming in 2012/13. The programme was then re-scheduled to begin in March 2013. No evidence of this actually having taken place." DtWI Assam research 2013, Pg 1.
  • "[Deworm the World is] also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2.
  • Originally Deworm the World didn’t believe that Jharkhand conducted a school-based deworming MDA in 2013, but learned of it by early 2014.
    • Alix Zwane conversation August 30th 2013
    • GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.
  • "Note that not all of these programs would have been operating at scale (the ones we supported were)." Grace Hollister, email to GiveWell, June 9, 2016

• 199
  • Deworm the World was not involved in Punjab, Jharkhand, nor Assam launching their statewide school-based deworming programs.
  • "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.
  • "DtW has been involved in deworming programs in four different states, and is still actively involved of three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement." Grace Hollister conversation June 19th 2013, Pg 1.
  • "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1.
  • "In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J-PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there. In August a memorandum of understanding (MoU) was formalized between DtW and the relevant players in Bihar (School Health Society Bihar and Bihar Education Project Council) for program implementation." Grace Hollister conversation June 19th 2013, Pg 2.
  • "DtW leveraged its networks to engage the support of the Minister of Health, Women, and Child Welfare [in Delhi]. Education stakeholders were also brought in so that the program could reach beyond preschool to older grades as well. DtW conducted a great deal of advocacy, maintaining continuous interactions and significant support among all relevant departments to bring the program to fruition Due to the complicated agency system, there were many other government stakeholders as well, requiring a high degree of coordination by DtW to ensure program objectives could be achieved." Grace Hollister conversation June 19th 2013, Pg 3.
  • "After the deworming program launched in Delhi, Rajasthan saw the results generated by the DtW- supported program in Delhi and committed to doing a deworming program, allocating funding for it in their budget. In March 2012 they brought DtW in to help. In this case, the state already knew what it wanted and already had a school health program. They sought DtW's technical expertise, mapping ability, general program support, stakeholder coordination, etc. Deworm the World coordinated signature of a MoU between the Departments of Women and Child Development, Education, and Health, UNICEF and DtW to guide program implementation, and helped establish of a technical secretariat housed within the Education Department." Grace Hollister conversation June 19th 2013, Pg 3.
  • "DtW has worked in Delhi, Bihar, Rajasthan, and Andhra Pradesh (AP).
    • In Bihar, according to documentation provided by DtW, the program began because of previous Poverty Action Lab (J-PAL) work in Bihar. DtW also noted that Rajasthan approached them because of the success (as highlighted in the media) of the Delhi program. DtW supported a pilot program and prevalence survey in AP, and though DtW didn't continue to engage with AP, the AP government seems to have recently conducted a school-based deworming program.
    • More recently, Punjab approached DtW, but DtW didn't have the funding to move forward with Punjab, and Punjab is now implementing deworming on its own.

    It's not possible to be certain whether or not DtW has caused deworming to happen that otherwise would not have, but it's important to note that DtW took the opportunities presented to help run better programs where possible as well as improve data quality and did so on a limited budget. They are also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2.

• 200
  • GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
  • While 11 states and Delhi participated in the first NDD in 2015, all states are supposed to participate in the second NDD in 2016. "The NDD slated for February 2016 is supposed to cover all states in India, per the Ministry of Health and Family Welfare’s issuance of correspondence to state governments." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015

• 201

  "Keeping in view the problem of worm infestation of children in Assam it has been decided to conduct two rounds of De-worming every year in the month of March and September. The first round will be conducted during March’ 2010. On every Wednesday (VHND/Immunization Day) medicine for De-worming will be given to all children between the age group 1 to 5 years along with vitamin A supplementation." Assam 2010 guidelines for deworming, Pg 1.

  976,192 "children below 5 years provided vitamin A syrup" as of November/December 2010 Assam reproductive and child health 2011-2012, Pg 73.

  The National Rural Health Mission reported the progress for 2010-2011 as 92,957 "Students given IFA/ de-worming tablets". Assam state programme implementation plan 2011-2012, Pg 21.

  Assam is "supposed to provide deworming to all children, but AWC services provided in Assam are some of the worst performers for deworming coverage." DtWI Assam research 2013, Pg 1.

• 202

  "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. NB. As reported by NRHM, Assam any school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam midday meal report 2013, Pg 11.

• 203
  • Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013.
  • "Deworm the World continues to support the four states it was working in previously (Rajasthan, Delhi, Bihar, and Madhya Pradesh) and has added programs in Chhattisgarh and Uttar Pradesh. [...] Deworm the World is in discussions with Telangana [...] Deworm the World has phased out its comprehensive technical assistance to Delhi, but the national office will continue to provide minimal support, including attending meetings and doing some policy advocacy work. Deworm the World no longer has dedicated staff for Delhi." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pgs 1-2

• 204

  DtWI 2013 GiveWell government interviews.

• 205

  "If Deworm the World had not been there, how would the deworming program be different?

  • Since Deworm the World have already implemented deworming programs in Delhi and Tamil Nadu, they brought experience.
  • If a state government decides to do something, nothing is impossible. However, Deworm the World's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have Deworm the World's experience, there could have been more problems.
  • The government received important support from DtWI, particularly with distributing the deworming tablets, implementing the trainings, and developing training materials."

  "Have you ever wanted to do a program but been unable to find a technical partner?

  • Ms. Gupta is still looking for a technical partner on specific issues. For example, Rajasthan has a large population with special needs. It has generally been able to find partners for helping the visually impaired, but very few organizations work with autistic children, children with cerebral palsy, or children with mental retardation. If there were a partner to support the Rajasthan government in this area, this would be very welcome. The Rajasthan government has funds for this type of program, but are looking for a technical partner.
  • An NGO called Sight Savers works with blind children. Before Sight Savers, she was not aware of problems with low vision. Many children need magnifying glasses and large print books. Sight Savers helped them to identify doctors and hospitals to work with such children."

  DtWI 2013 GiveWell government interviews, Pg 7-9.

• 206
  • "Additionally the national government of India has expressed interest in possibly
    implementing a deworming program nationwide, rather than waiting for each state to
    launch separate programs. They have expressed interest in receiving technical assistance
    from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2
  • Grace Hollister, email to GiveWell, June 9, 2016

• 207
  • "Deworm the World Initiative at Evidence Action will be the technical assistance partner to MoHFW, Government of India. The specific responsibilities are as follows:
    Support in development of National Deworming Day implementation strategy
    Design and develop training and reference materials, community mobilization strategies for increased awareness and coverage of target beneficiaries, Monitoring and Evaluation (M&E) Systems and reporting formats" India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015, Pg 10.
  • "Deworm the World has three primary roles in National Deworming Day: • Agenda setting: Deworm the World has played a large role in getting deworming on the national government’s health agenda, which led to the creation of National Deworming Day. Before Deworm the World began working with the national government, deworming was officially a part of some health programs, but was inconsistently implemented outside of those states where Deworm the World provides technical assistance. • Developing materials and advising the national government: Deworm the World has also developed operational guidelines for program implementation, including training materials, public awareness materials, monitoring forms, guidance for teachers and health workers, and a Frequently Asked Questions guide about National Deworming Day, in partnership with the national government. The national government will distribute these materials to state governments for use on National Deworming Day; some materials will be adapted to state-specific contexts. Deworm the World’s work plan also advised the national government on appropriate timing for workshops and trainings, and plans to conduct a training for state-level functionaries once the date of deworming is announced. Deworm the World’s India Country Director Priya Jha speaks with Dr. Khera of the Child Health Division at the Indian Ministry of Health and Family Welfare on a weekly basis about plans for National Deworming Day. • Direct work with state governments: Deworm the World will work individually with the states Bihar, Rajasthan, Delhi, and Madhya Pradesh to adapt National Deworming Day guidance into those programs. Deworm the World has been working in the first three of those states for multiple rounds of deworming. Funding for working in Madhya Pradesh comes from a USAID grant, for which this will be the first round of deworming support. Deworm the World also hopes to work in Chhattisgarh with additional funds from USAID, and Odisha and Uttar Pradesh with funding from the Children's Investment Fund Foundation. Other than developing operational guidelines and materials, and providing highlevel training for key state functionaries, Deworm the World does not have the capacity to be involved in the first phase of National Deworming Day in other states in India. Deworm the World had originally planned to create a national implementation and monitoring workshop for representatives from each Indian state prior to National Deworming Day. However, it is too late to host the workshop before the first National Deworming Day in February 2015. Editor’s note (based on updates after this conversation): Deworm the World did provide some support to Chhattisgarh and supported the national government in hosting a workshop." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pgs 1-2
  • "Additionally the national government of India has expressed interest in possibly
    implementing a deworming program nationwide, rather than waiting for each state to
    launch separate programs. They have expressed interest in receiving technical assistance
    from DtWI for this project." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 2
  • "The Indian national government has recently demonstrated an increased interest in deworming programs. The Child Health Division of the Ministry of Health & Family Welfare (MoHFW) has been particularly interested in deworming. Deworm the World Initiative (DtWI)’s program in India is working with Ajay Khera, the leader of the Child Health Division, and other staff members in the division to encourage the division to continue promoting deworming. DtWI has also been meeting with the Joint Secretary of MoHFW and requesting appointments with the Minister of Health & Family Welfare. Deworming has been proposed to be included in the 100 Day Agenda of the new administration in India. DtWI is a stakeholder supporting this effort. DtWI has proposed to assist with several aspects of the program including planning, guideline development and other technical issues." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014, Pg 1
  • "States which did not receive technical support from Deworm the World may not have had adequate time or support for planning the February deworming round. This may have resulted in lower coverage and weaker monitoring. Some states would likely appreciate assistance from Deworm the World or another similar organization." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2

• 208

  India NDD documents 2015

• 209
  • "National Deworming Day was originally planned for October 2014, but was delayed after Deworm the World staff alerted the government that drug supplies were insufficient. The Indian government has publically announced that National Deworming Day will be in February 2015, but it is possible that there will be further delays." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1
  • "Deworm the World had originally planned to create a national implementation and monitoring workshop for representatives from each Indian state prior to National Deworming Day. However, it is too late to host the workshop before the first National Deworming Day in February 2015.
    Editor’s note (based on updates after this conversation): Deworm the World did provide some support to Chhattisgarh and supported the national government in hosting a workshop." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 2

• 210

  "A program of the Government of India, the February 10 event in eleven states targeted 140 million children with school-based deworming treatment." DtWI NDD blog post February 2015

• 211
  • "On National Deworming Day, 12 states in India selected for inclusion in the program’s first phase will be required to administer deworming drugs to schoolchildren, regardless of measures of worm prevalence." GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014, Pg 1
  • "India held its first National Deworming Day in February. Eleven states participated." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 2.
  • "11 dewormed in February; the 12th, Delhi, dewormed in April. This delay was due to conflicts of the February timing with state elections, which require the participation of teachers." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015

• 212
  • Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
  • Grace Hollister, email to GiveWell, June 9, 2016

• 213

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 214

  Grace Hollister, edits to GiveWell's review, November 7, 2016

• 215

  RTI is funded primarily by ENVISION (of USAID), which does not typically fund STH and schistosomiasis treatments unless they are integrated with other NTD treatments. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 216

  This understanding is from several conversation with different organizations.

• 217

  Deworm the World, SCI, Sightsavers, and END Fund have all told us that they would could additional funding to expand deworming in Nigeria.

• 218

  This understanding is from several conversation with different organizations.

• 219
  • For example, by helping governments obtain drugs in a timely manner: "in some cases we have improved the likelihood that sufficient drugs were available in a timely manner for program rollout; some of that is through support to govts in accessing the WHO donation program, other is due to pushing govts to procure needed drugs in a timely manner." Grace Hollister, edits to GiveWell's review, November 7, 2016
  • "DtWI's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have Deworm the World's experience, there could have been more problems." Veenu Gupta, Principal Secretary to School Education Department, Government of Rajasthan DtWI 2013 GiveWell government interviews, Pg 7.

• 220
  • "DtWI has helped to improve deworming trainings. Last year, Deworm the World helped to develop the content for the trainings. Deworm the World made the content more concise and easy to understand than the government would have done on its own. Deworm the World also improves trainings by tracking when people do not show up to trainings and following up with them and by determining aspects of training that could be improved in future years." Girish Bharbwag, Nodal Officer in Rajasthan DtWI 2013 GiveWell government interviews, Pg 4.
  • "And also to note that some programs administering deworming medication do not provide any training to teachers and/or health workers prior to drug distribution" Grace Hollister, edits to GiveWell's review, November 7, 2016

• 221

  The District Coordinator position, for example, is not always staffed in other school-based health programs. "The state government paid for District Coordinators for the hand washing program in 2008. However, the payment for this position was low. In the first year, there were 15-16 independent monitors. This year, there are only 7 District Coordinators. The government is finding it difficult to fill these vacancies." DtWI 2013 GiveWell government interviews, Pg 5.

  "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems.
  The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 4.

• 222

  "The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 5.

• 223
  • In Bihar, Deworm the World says that it has convinced the state to expand its program to include preschool-aged children in the next round of deworming, set for early 2015. GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
  • The MDA that Jharkhand implemented before it was in contact with Deworm the World only targeted 10-19 year olds: "Last year Jharkhand launched a deworming as part of the WIFS Program, without any direct advocacy from DtWI. The program had no formalized protocols, didn’t measure coverage, and focused on children aged 10 to 19. In 2014 Deworm the World has been meeting with relevant officials in the state to see if there is an opportunity to improve the quality of the program with technical assistance. Deworm the World is hoping to sign a Memorandum of Understanding (MOU) with the state soon, and its work there would be funded by a grant from USAID." GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014, Pg 3.

• 224

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 225
  • DtWI Monitoring Improvements 2014
  • DtWI Bihar 2014 program report
  • DtWI Delhi 2013 program report
  • DtWI Rajasthan 2013 program report
  • GiveWell DtWI 2013-2014 cost data summary

• 226

  Grace Hollister, edits to GiveWell's review, November 7, 2016

• 227

  Conversation notes here.

• 228
  • Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16
  • Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness.

    Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15

  • Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15
  • Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials.

    Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17

• 229
  • See citations in previous footnote.
  • "These calls helped to uncover problems, which are reported on the same day to the appropriate government officials to address" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • We have not seen any detailed data of the calls made or the issues uncovered or whether they are resolved.
  • We are only aware of one specific example of the type and magnitude of problems uncovered: In the Bihar 2014 program, Deworm the World-led monitoring uncovered that 18% of schools that were called had not received the drugs at the scheduled time, and it seems plausible that the issue was largely addressed before deworming day (though we have not vetted detailed data supporting this).
    "BEO [Block education officer] offices distributed the drugs among the school headmasters at block level trainings. In monitoring phone calls, 82% of the 354 schools contacted by Deworm the World tele-callers across 27 districts confirmed receiving drugs two days prior to Deworming Day. When this potentially problematic information came to light, the tele-calling team and DCs hired by Deworm the World contacted all the BEO offices to ensure delivery of drugs to all the schools before Deworming Day along with instructions issued from the BEPC nodal officer. Subsequent independent monitoring data (from visits to schools during Deworming Day and Mop-Up Day) shows that approximately 96% schools had received drugs by Deworming Day. This was a significant jump from the 82% polled only two days earlier." DtWI Bihar 2014 program report, Pg 13.

• 230

  Our observations are noted here: DtWI 2013 GiveWell site visit.

• 231

  In 2014, Deworm the World reported improvements to the training process, including focusing on areas where problems were identified in previous rounds, simplifying materials, and testing if participants are learning key messages. DtWI Monitoring Improvements 2014, Pgs 1-2.

• 232

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 233

  The prevalence surveys also examined more worm species than were necessary from a public health perspective. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 234
  • Our intervention report discusses this briefly
  • Other conversations and observations have reinforced our impression that administering deworming drugs is fairly straightforward.
  • The WHO factsheet on STH: "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet

• 235

  Deworm the World notes that: "albendazole and mebendazole are quite hardy (neither requires special storage conditions) and have a long shelf life." Grace Hollister, email to GiveWell, June 9, 2016

• 236

  For example, drug quality was tested in each program that Deworm the World supported in India in 2013-2014.

  • Bihar 2014:
    • "The drugs were safely stored in State Health Depot in Patna until November 2013. At that point, the districts began to pick up their share of the drugs from the depot. In October, Deworm the World had coordinated for lab testing of the stored drugs via ASCHO NIBULA INDUSTRIES LTD, an independent lab which approved the quality of the drugs." DtWI Bihar 2014 program report, Pg 13.
    • "The quality of drug storage was satisfactory in most schools that were monitored. 98.7% of them stored the drugs in a clean location, 91.1% of them were stored away from direct sunlight and 97.4% of them were stored away from the direct reach of children." DtWI Bihar 2014 program report annex 1, Pg 7.
  • Delhi 2013:
    • "Once the procured syrups and donated tablets were delivered to Directorate of Health Services central storage room, they were tested in a government-accredited laboratory to ensure drug quality prior to administration." DtWI Delhi 2013 program report, Pg 6.
    • From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Delhi 2013 program report, Pg 24. A table of results is also presented on the same page.
  • Rajasthan 2013:
    • "To instill confidence among the stakeholders that the drugs were of good quality, Deworm the World arranged for sample testing of the donated drugs by two independent labs7. Similarly, Rajasthan Medical Services Corporation sample tested the syrups they procured." DtWI Rajasthan 2013 program report, Pg 6.
    • From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Rajasthan 2013 program report, Pg 40. A table of results is also presented on the following page.

• 237

  "Now, state governments have responsibility for testing drugs. We have recently raised some concerns about the need to standardize the testing that is taking place, and are currently working with the MoHFW to build out the NDD operational guidelines with more detailed guidance on this point." Grace Hollister, edits to GiveWell's review, November 7, 2016

• 238

  Of classes where monitors observed deworming activities in India, there were low numbers of adverse events (see table below). We aren't sure what portion of the adverse events may be caused by incorrect dosages.

  	Bihar 2015	Rajasthan 2015	Delhi 2015	Madhya Pradesh 2015	Sample of question asked
  Classes where there were adverse events (monitors' observations)	5% (vomiting), 0% (diarrhea)	2% (vomiting), 0% (diarrhea)	5% (vomiting), 0.8% (diarrhea)	6% (vomiting), 0% (diarrhea)	"Did you see any child with adverse effects (nausea, vomiting, stomachache, etc.) after taking the medicine?"

  Sources for the information in the table:

  • Bihard 2015: Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 4
  • Rajasthan 2015: Deworming Day and Mop-up Day ( N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 4
  • Delhi 2015: Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 6
  • Madhya Pradesh 2015: Deworming Day and Mop-up Day (N = 250 schools), DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 4
  • Sample question: DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 14

• 239
  • "Note that National Deworming Day operational guidelines state that only tablets should be used. Albendazole dosage is the same for all children aged 2 and above; it is only children aged 1-2 that require a different (half) dose." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
  • From Deworm the World's report on the Delhi 2013 program: "A key discussion from round one was the administration of drug to younger children in the pre-school age for whom chewing a tablet was seen as a difficulty. Hence keeping in mind the scale of the program, with an objective of making it a safe public health initiative the committee decided that deworming would be implemented across all districts following the World Health Organization (WHO) sanction in administering albendazole 400 mg tablets and the GOI guidelines under the WIFS program for administering the syrup vial. They also decided to opt for Albendazole 200mg dose suspension for 2-6 year children as it is a single dose for this age group which leads to lesser error in administration, while Albendazole 400mg tablets for older children." DtWI Delhi 2013 program report, Pg 6.
  • The WHO factsheet on STH cites only a single recommended dosage (depending on which drug is used): "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet

• 240
  • Bihar 2015: In 8.8% of trained schools and 5.4% of untrained schools, monitors observed children given less than one tablet; in 2.6% of trained schools and 4.7% of untrained schools, monitors observed children given more than one tablet. DtWI Bihar 2015 independent monitoring tables, Pg 9 (Table 10)
  • Rajasthan 2015: In 6.0% of schools, monitors observed children given less than one tablet; in 1.6% of schools, monitors observed children given more than one tablet. DtWI Rajasthan 2015 independent monitoring tables, Pg 2 (Table 2)
  • Madhya Pradesh 2015: In 9.3% of schools, monitors observed children given less than one tablet; in 2.7% of schools, monitors observed children given more than one tablet. DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2 (Table 2)
  • Delhi 2015: In 2.5% of schools and 10.6% of anganwadis, monitors observed children given less than the prescribed dose of albendazole; in 0.8% of schools and 6.0% of anganwadis, monitors observed children given more than the prescribed dose of albendazole. DtWI Delhi 2015 independent monitoring tables, Pgs 2, 16 (Tables S1 and A1)
  • Bihar’s 2014: 8% of schools observed gave children less than one tablet and 2% gave more than one tablet. DtWI Bihar 2014 program report annex 2, Pgs 2 and 4.
  • "In 3.1% of schools and anganwadis, monitors observed children being given more than one tablet/syrup bottle. As per protocol, the children should not have been given more than one tablet. Therefore in cases such as these, the monitors were trained to intervene and prevent the administration of an additional dose." (The percentage of schools using less than one pill/bottle per child was not reported.) DtWI Rajasthan 2013 program report, Pg 38.
  • Monitors in Delhi in 2013 did not report on either of these observations. See DtWI Delhi 2013 program report, Pgs 21-25.

• 241

  "[S]ingle-dose oral therapies can kill the worms, reducing ... infections by 99 percent ... Reinfection is rapid, however, with worm burden often returning to eighty percent or more of its original level within a year ... and hence geohelminth drugs must be taken every six months and schistosomiasis drugs must be taken annually." Miguel and Kremer 2004, pg. 161.

• 242
  • "The majority of direct program costs in India are government funded. There is a cap within the National Health Mission budgets on M&E – M&E cannot exceed 10% of the overall budget. So there are specific areas, such as program monitoring, that would likely not have investment at the level we are able to provide." Grace Hollister, edits to GiveWell's review, November 7, 2016
  • "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2.
  • "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and Deworm the World's tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. Deworm the World called a random sample of 8,000 schools.
    The prevalence survey would not have happened without Deworm the World's support." DtWI 2013 GiveWell government interviews, Pg 5.

• 243

  "RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3.

• 244

  "[The Nodal Headmaster said] that most aspects of the program are excellent, but he had 2 suggestions:

  • Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so)
  • Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6.

• 245
  • "In Tanzania matters came to a head in places around Morogoro in 2008. Distribution in schools of tablets for schistosomiasis and soil-transmitted helminths provoked riots, which had to be contained by armed police. It became a significant national incident, and one of the consequences has been the delay in Tanzania adopting a fully integrated NTD programme, and the scaling back some existing drug distributions." Allen and Parker 2011, pg. 109.
  • "From these reports a number of problems with the MDA were raised which included fear of side effects from the tablets, particularly following the mass hysteria and death in Blantyre and Rumphi respectively and may explain some of the geographic heterogeneity seen. Furthermore most districts reported that MDA occurred after standard 8 students had finished exams and left school, and due to having inadequate resources for drug distribution...The side-effects incident in Blantyre and death in Rumphi had a large effect on districts and with many district reports stating that after the incidence many families refused to participate." SCI Malawi coverage survey 2012 Pgs 5, 21.

• 246
  • "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
  • Deworm the World noted that the costs in Kenya are high, partly due to higher quality M&E:
    • "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
    • "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
    • In the second round of treatment in Kenya, prevalence surveys were 11% of the total costs and other M&E was 6%. Per treatment, these costs are $0.059 ($379,523/6,405,462) and $0.035 ($222,750/6,405,462) respectively DtWI Kenya 2013-2014 cost per treatment data
  • Our understanding is that costs are higher in Kenya than India primarily due to cultural differences, including paying significantly higher allowances (per diems)to teachers for participating in the program. For more details, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell analysis of Deworm the World cost per treatment.
  • Deworm the World has also noted that treating schistosomiasis in Kenya increases the costs: "The extra costs of treating schistosomiasis in addition to STH. Schistosomiasis drugs tend to be more expensive than STH drugs and, in Kenya, schistosomiasis treatment sites are sometimes much more remote. The treatment strategy for schistosomiasis also differs from STH because schistosomiasis is more localized (e.g., it is not necessarily ideal to treat an entire sub-county). This also makes mapping schistosomiasis more expensive." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 3.

• 247

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 248

  See Evidence Action, blog post, January 16, 2015 for a description of how Deworm the World calculates the cost of deworming. See Evidence Action, blog post, July 5, 2016 for an update.

• 249

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Deworm the World thought that the costs of these activities were quite low (less than 5%). We have conservatively assumed 5% in our analysis.

• 250

  In general, cost-per-treatment figures have stayed fairly consistent over the last several years in Kenya and India:

  • In the Kenya 2013-2014 treatment round, the total cost-per-treatment was $0.56 (not including staff time). It remained at $0.56 in the 2014-2015 treatment round. GiveWell analysis of Deworm the World cost-per-treatment, October 2016
  • In the India 2012 treatment rounds for which we have information, the average cost-per-treatment was $0.38 (including staff time). In the 2013-2014 treatment rounds, the average cost-per-treatment was $0.30 (see our 2015 review). Note that we did not adjust the coverage numbers used in either of these calculations. In the 2014-2015 treatment rounds, the average cost-per-treatment was $0.32, after adjusting for the fact that Deworm the World's monitors found lower coverage than reported by the government. GiveWell analysis of Deworm the World cost-per-treatment, October 2016

• 251
  • DtWI cost per treatment summary 2013
  • GiveWell DtWI 2013-2014 cost data summary
  • GiveWell analysis of Deworm the World cost-per-treatment, October 2016
  • The rounds for which we have seen cost-per-treatment data are:
    • Bihar: 2012, 2014, and 2015
    • Delhi: 2012 and 2013
    • Rajasthan: 2012, 2013, and 2015
    • Madhya Pradesh: 2015

• 252

  GiveWell analysis of Deworm the World cost-per-treatment, October 2016 Note: We focus on these three states because Deworm the World did not send us 2015 cost-per-treatment data for any other states in India.

• 253

  For example, see our 2015 review. We estimated the staff time costs based on DtWI’s estimate of similar costs from the same states in 2012. DtWI had imputed those costs based on estimates of government employee salaries; for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50) (DtWI Rajasthan 2013 cost data).

• 254

  GiveWell analysis of Deworm the World cost-per-treatment, October 2016, Summary sheet, column H.

• 255
  • Alix Zwane conversation August 30th 2013
  • For example, for Madhya Pradesh's 2015 costs: "Drug costs are included in this model as an imputed cost to the government. Unlike other programs supported by Evidence Action in 2014/2015, Madhya Pradesh did not receive Albendazole under the WHO drug donation program. Instead, they procured the drugs locally. In order to ensure that the program had sufficent drug supplies, the government of India purchased roughly $35,000 in Albendazole and Syrups to supplement their existing stocks. Given that the number of drugs disseminated to districts for use on National Deworming Day is unavailable, and the true value of drugs used by the program far exceeds the $35,000 the government spent to supplement existing stocks, this model relies on treatment figures to estimate the value of drugs. This method of calculation is a deviation from the method used to calculate the value of drugs in models representing the cost of other geographies within India for the 2014/2015 round." Deworm the World, Madhya Pradesh cost-per-treatment - 2015

• 256
  • We have not adjusted these data for inflation or changes in exchange rates over time.
  • Deworm the World's 2013 and 2014 budgets stopped estimating the financial value of government employees’ time spent on the deworming program after the deworming rounds in 2012: "We have consulted with JPAL on the costing model approach and they have suggested, in line with papers they've done, not to quantify teachers' time nor principals' time for the actual deworming day; we still have included the cost of teachers attending the training since there is a direct cash transfer to the teachers for their coming to the training." Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014
  • Deworm the World cited JPAL CEAs in education 2011 for this approach.
  • The change is most relevant for teachers and principals, large numbers of whom attend a training and administer the pills to children. While we have seen direct costs of paying teachers and principals that attend deworming training, we are unsure whether those costs should be counted as per-diems (perhaps necessary for some trainees to pay for transport, but not accounting for the lost work time at school) or as optional extra wages (that don’t result in lost work time at school and are sufficient to incentivize the necessary labor), or something in between.
  • In order to include the value of teachers and principals’ time for the three recent rounds, we have copied the relevant expenses from the prior year’s budget of the same state (which were imputed costs) and replaced the allowances that were paid directly to teachers and principals during trainings. We believe this methodology is likely to result in some inaccuracies and we may revise it if we become confident of a more accurate method.
    • GiveWell DtWI 2013-2014 cost data summary
    • DtWI cost per treatment summary 2013, Summary.

• 257

  We estimated the staff time costs based on Deworm the World's estimate of similar costs from the same states in 2012. It had imputed those costs based on estimates of government employee salaries, for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50).

• 258

  "We include all partners’ expenditures in determining costs for the the deworming programs, but we do not consider spending that would be incurred even without deworming taking place. [...] Teachers’ and principals’ general salaries are not included because they do not spend additional time on deworming beyond what they are already compensated for by the government for regular classroom teaching." Evidence Action, blog post, July 5, 2016

• 259

  GiveWell analysis of Deworm the World cost-per-treatment, October 2016

• 260

  See GiveWell analysis of Deworm the World cost-per-treatment, October 2016.

• 261
  • GiveWell analysis of Deworm the World cost-per-treatment, October 2016
  • Note that this is only slightly lower than the cost estimate Deworm the World developed last year (see our 2015 review) because we have adjusted for the fact that Deworm the World's monitors found higher coverage than the government reported.
  • Previously, Deworm the World estimated that the deworming program in Kenya cost around $0.40 per child treated: "We note that preliminary estimates of program costs in Kenya, which we can estimate with far greater precision, suggest programming costs there of about $0.40 per child." Evidence Action cover letter 2013, Pg 2.

• 262

  Excluding the value of teachers' and principals' time, we estimate that Deworm the World's cost per treatment is $0.55 and that SCI's cost per treatment is $0.83. Supporting data and calculations are shown in GiveWell analysis of Deworm the World cost-per-treatment, October 2016 and our review of SCI. Additional notes:

  • We estimate that Deworm the World's drug costs in Kenya are $0.05 per treatment. We believe that drug costs are lower for the program in Kenya than most of SCI's programs because SCI generally treats for schistosomiasis, but a relatively small proportion of deworming in Kenya includes schistosomiasis treatment, because relatively few areas have high enough prevalence.
  • Deworm the World does more extensive prevalence surveying in Kenya than it does in other countries and than we believe SCI does; this may cause an overestimate of Deworm the World’s costs in other countries.
    • "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
    • "For the NSBDP, three prevalence and intensity surveys for STH and one for schistosomiasis are expected to be or have been completed by KEMRI. The implementation costs of these surveys were divided among the program's expected duration of five years. Therefore, this model includes 1/5 of the total survey-associated costs." DtWI Kenya 2013-2014 cost per treatment data, Introduction sheet.
  • We have not adjusted these data for inflation or changes in exchange rates over time.

• 263
  • See further discussion here.
  • Deworm the World adopted a different approach, citing JPAL CEAs in education 2011.

• 264
  • See our discussion of the rationale and limitations of this estimate here.
  • For comparison, Deworm the World previously estimated the value of in-kind support from the governments in Bihar and Rajasthan. Our understanding is that these estimates suggested in-kind cost of $0.31 per treatment in Bihar 2012 and $0.20 in Rajasthan 2012.
    • For both states, Deworm the World shared detailed data behind its cost per treatment data. This included a sheet for imputed costs. We aggregated these line items, excluding expenses from non-government partners (e.g. WHO and UNICEF), and we believe the result largely accounts for teachers' time during training and deworming day. Treatments from DtWI cost per treatment summary 2013
    • Bihar 2012: DtWI Bihar 2012 cost data details shows $5.21 million, which implies $0.31 per treatment, given 16,867,388 treatments. $5.21 million is converted from INR 276.1 million at 53 INR per USD (based on the source), and is a total of these line items:
      • Master Trainers as Trainers in Block Level Trainings
      • Master Trainers as Trainers in Sector Level Trainings
      • Block Level Training
      • Sector Level Training
      • Honorarium: Teachers and Headmasters
      • Honorarium: Health Department (Doctors and ANMs)
    • Rajasthan 2012: DtWI Rajasthan 2012 cost data details shows $2.06 million, which implies $0.20 per treatment, given 10,132,535 treatments. $2.06 million is converted from INR 109.1 million at 53 INR per USD (based on the source), and is a total of 23 line items, with these 5 largest line items accounting for over 90% of the total:
      • Teachers and Headmasters as Trainees
      • Lady Supervisors and Anganwadi Workers as Trainees
      • Block Level Training
      • Honorarium: Teachers and Headmasters
      • Honorarium: Anganwadi Workers

• 265

  See GiveWell analysis of Deworm the World cost-per-treatment, October 2016

• 266

  GiveWell analysis of Deworm the World cost-per-treatment, October 2016

• 267
  • Monitors find that the number of students dewormed according to school records was greater than the number according to those schools' classroom records for each state that Deworm the World supported in 2015; the median over-report amount was 15%.
    • Process: For each state for which we have 2015 deworming coverage validation survey data, we calculated a state-level inflation rate according to the surveyors' observations. We calculated inflation rate as the difference between reported treatments and recorded treatments, out of the total recorded treatments: (treatments reported on the school reporting form​ minus treatments recorded as tick marks in class records)/(treatments recorded as tick marks in class records). In the raw dataset, some entries are marked as missing data using error codes described in the associated code sheet; these were treated as contributing zero tickmarks or zero reported treatments (as appropriate) to the total count.
    • Bihar: 22% inflation rate: (141,377-115,889)/115,889 DtWI Bihar 2015 monitoring data for coverage validation, schools (Deworm the World also reports this figure as 22%: "The state level verification factor for Bihar was found to be 0.81972, indicating that for every 82 enrolled children who were recorded as deworming in the schools, the school reported that 100 enrolled children had been dewormed. This corresponds to an overall 22% inflation of reporting in the state, meaning that reported numbers appear to be approximately 22% higher than the numbers recorded in attendance registers." DtWI Bihar 2015 Program report, Pg 19)
    • Rajasthan: 15.0% inflation rate: (42,845-37,256)/37,256 DtWI Rajasthan 2015 monitoring data for coverage validation, schools
    • Delhi: 1.9% inflation rate: (170,060-166,956)/166,956 DtWI Delhi 2015 monitoring data for coverage validation, schools
    • Madhya Pradesh: 11.3% inflation rate: (39,983-35,912)/35,912 DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools
    • Chhattisgarh: 100.0% inflation rate: (63,162-31,582)/31,582 DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools
  • This over-reporting results from schools that reported deworming students in the school summary but did not mark any classroom records as the program protocol instructed. After excluding schools in which no classroom records were marked, the number of students dewormed according to the school records was similar to those schools' classroom records for the same states (the median over-report amount was 0.4%).
    • Process: For each state for which we have 2015 deworming coverage validation survey data, we calculated a state-level inflation rate ​excluding schools at which no tick marks were recorded​ according to the surveyors' observations. We calculated inflation rate as the difference between reported treatments and recorded treatments, out of the total recorded treatments: (treatments reported on the school reporting form, in schools with any tick marks recorded in any class records​ minus treatments recorded as tick marks in class records)/(treatments recorded as tick marks in class records). In the raw dataset, some entries are marked as missing data using error codes described in the associated code sheet; these were treated as contributing zero tickmarks or zero reported treatments (as appropriate) to the total count.
    • Bihar: -2.8% adjusted inflation rate: (112,653-115,889)/115,889. DtWI Bihar 2015 monitoring data for coverage validation, schools
    • Rajasthan: 2.9% adjusted inflation rate: (38,318-37,256)/37,256 DtWI Rajasthan 2015 monitoring data for coverage validation, schools
    • Delhi: 1.7% adjusted inflation rate: (169,748-166,956)/166,956 DtWI Delhi 2015 monitoring data for coverage validation, schools
    • Madhya Pradesh: -5.8% adjusted inflation rate: (33,844-35,912)/35,912 DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools
    • Chhattisgarh: 0.4% adjusted inflation rate: (31,694-31,582)/31,582 DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools

• 268

  This ratio is called the "verification factor." Deworm the World calculates it for each of the MDAs it supports.

  • Bihar: 76% Deworm the World, Bihar 2016 IMCV report, Pg. 21
  • Chhattisgarh: 76% Deworm the World, Chhattisgarh 2016 IMCV report, Pg. 22
  • Madhya Pradesh: 94% Deworm the World, Madhya Pradesh 2016 IMCV report, Pg. 22
  • Rajasthan: 91% Deworm the World, Rajasthan 2016 IMCV report, Pg. 25
  • Telengana: 53% Deworm the World, Telengana 2016 IMCV report, Pg. 13
  • Tripura: 88% Deworm the World, Tripura 2016 IMCV report, Pg. 27
  • Uttar Pradesh: 64% Deworm the World, Uttar Pradesh 2016 IMCV report, Pg. 24

• 269

  "We don’t have a process that is part of every round. Data quality assessments are undertaken periodically in collaboration with the government – we have completed these in a few states and in other states DQAs are in process/planned. Sampling as part of the assessment includes schools, anganwadis, blocks, and districts. One of the key issues we have found to date is the lack of available documentation at each level." Grace Hollister, edits to GiveWell's review, November 7, 2016. Note that we have not yet asked Deworm the World to share results from any of its data quality assessments with us.

• 270
  • For example, the nodal headmasters add the school totals from their respective jurisdictions and report them to the block-level officials, who use those to calculate the total children dewormed in their block and report that figure to the district, who do the same in reporting to the state level, where the grand total is calculated.
  • Bihar 2015: "With close support from our teams, the State Health Society Bihar and Bihar Education Project Council collected and compiled the coverage report for the round within the reporting timelines in the prescribed reporting format (Annexure H.1). Coverage reporting structure and timeline is shown below in Figure 4:" (see source for annexure and figure) DtWI Bihar 2015 Program report, Pg 16.
  • Delhi 2015: "In this round, each school and anganwadi was supposed to fill a one-page reporting form (annexure III). In order to improve the accuracy of coverage reporting by the schools and anganwadis, every participating school and anganwadi was instructed to follow a recording protocol for deworming. Every teacher and anganwadi worker was required to put a single tick (√) next to a child’s name in the school/anganwadi register if they were administered albendazole on deworming day and double-tick mark (√√) if dewormed on mop up day. School headmasters and anganwadi workers were responsible to compile the number of dewormed children, fill the reporting format and submit it to the next level. Reporting structure of coverage data from schools and anganwadis and timelines are given in the below flow chart:" (see source for chart) DtWI Delhi 2015 program report, Pg 15.
  • Rajasthan 2013: "School headmasters were required to hand over the completed reporting form to their respective nodal headmasters – a senior headmaster of a school looking after a cluster of schools – who in turn would submit these forms to the respective BEO office. Apart from the forms, the nodal headmasters also submitted the collated information. This information further gets consolidated first at block level, then at district level and finally at state level. Education department shared this compiled information with all the stakeholders." DtWI Rajasthan 2013 program report, Pg 14.
  • Bihar 2014 was an exception; aggregation of data for that round of treatment was done centrally, by an independent firm. "In Round 3, each school was supposed to fill a one-page, simple school summary form (Form S), capturing only the essential details on the school such as total enrollment, total number of dewormed children by date and by enrollment status, number of adults dewormed, availability of drugs, drug usage and wastage, remaining drug stocks and contact details of the headmaster. In order to ensure that coverage reporting by the schools is accurate, every participating school was instructed to follow a special recording protocol for deworming. Every teacher was required to put a single tick mark (9) next to a child’s name in the attendance register if they had consumed the tablet on Deworming Day. The teachers were instructed to put a double-tick mark (99) next to a child’s name if s/he had been administered the tablet on Mop-Up Day. These tick marks are intended to be the basis for the numbers reported by every school in the S forms. Schools were supposed to provide the number of enrolled children dewormed by counting the single and double tick marks in the attendance registers. In addition, the provision for dewormed non-enrolled children was to be maintained along with the details of adults dewormed. School headmasters were supposed to submit the filled summary form to BRP by January 30 2014. Blocks were to submit all the collected forms, without any consolidation or compilation, at DEO office by February 5 2014. Districts were instructed to submit these forms at BEPC by February 20 2014. [...] Simplification of consolidation process by replacing data consolidation at block and district level with only collection of forms and submission at higher level. [...] Further, DtWI changed its prior strategy of relying on a single and small data entry partner, and selected a reputed agency with significant experience in large scale surveys across India to do data entry. This data entry partner subsequently dedicated significant data entry resources to Bihar form entry." DtWI Bihar 2014 program report, Pgs 26-27

• 271
  • In Bihar's 2015 program, 99.8% of schools reported deworming data, so we believe minimal school-level data was missing from the aggregation process. DtWI Bihar 2015 Program report, Pg 4 (Table 1)
  • In Rajasthan's 2015 program, 94.0% of schools reported deworming coverage, so it seems possible that any deworming conducted in the remaining 6% of schools was left out of the reported coverage. @DtWI Rajasthan 2015 Program report@, Pg 4 (Table 1)
  • In Delhi's 2015 program, 100% of schools and AWCs [anganwadi child-care centres] reported deworming. @DtWI Delhi 2015 Program report@, Pg 4 (Table 1)
  • In Madhya Pradesh's 2015 program, 100% of schools and AWCs reported deworming. @DtWI Madhya Pradesh 2015 Program report@, Pg 5 (Table 1)
  • In Bihar’s 2014 program, roughly 8% of schools’ summary reports were missing ("In total, 64,724 schools out of the 70,675 targeted schools submitted their summary forms."), and it sounds as if these were simply left out of the reported coverage data ("The result of this modified data cleaning and data entry process was that the coverage data was available to share from the 64,724 schools within 3 months of the deworming date. This cleaned data indicated that 16,225,546 children were dewormed in Bihar out of which 15,489,334 were enrolled children and 736,212 were non-enrolled children.") DtWI Bihar 2014 program report, Pg. 27.
  • In Delhi’s 2013 program, Deworm the World reports that missing reports were excluded from the reported coverage figures: "The program targeted 3,032 schools and 10,500anganwadis. As on the cutoff date for report collection, 15 December 2013, data from 603 schools was pending. The above data is based on a dataset comprising 2,417 schools and 10,591 anganwadis." DtWI Delhi 2013 program report, Pg. 2.
  • We have not seen information about missing data for Rajasthan’s 2013 program.

• 272
  • Methodology:
    • Schools included: all schools monitors visited during coverage validation day, except those for which no classes records showed any dewormings (because we believe many schools just didn’t use that protocol despite deworming students) unless the principal said that no deworming had taken place in the school. We also excluded schools with missing total enrollment data.
    • Students dewormed in included schools: total ‘tick marks’ in school register for all classes in included schools. Note that we are unsure whether this measure sometimes includes unenrolled students as well.
    • Enrollment in included schools: total enrollment as reported by the monitor's check of the attendance register.
    • Total students enrolled in each state (public and private): Sum of primary, upper primary, secondary, and higher secondary enrollment figures for the 2014-2015 school year from India’s District Information System for Education (DISE). Primary and upper primary total enrollment: U-DISE Elementary Thematic Maps 2015, Pgs 60-61. Secondary and upper secondary total enrollment: 2014-15 totals U-DISE Secondary Flash Statistics 2015, Pg 34.
    • Total students enrolled in government and government-aided schools: Sum of (total enrollment * percentage of enrollment in government and government-aided schools) for primary, upper primary, secondary, and upper secondary schools. Percentage of students enrolled in government and government-aided schools by state: U-DISE Elementary Thematic Maps 2015, Pgs 62-63 and U-DISE Secondary Thematic Maps 2015, Pgs 33-34.
      • U-DISE Secondary Thematic Maps 2015, Pg 34 reports the percentage of students who are enrolled in private unaided managements. We are assuming that the remainder of students are in government or government-aided schools. U-DISE Elementary Thematic Maps 2015, Pgs 62-63 reports the percentage of students enrolled in "government management schools." We are uncertain whether "government management schools" includes government-aided private schools, but we are assuming so for these figures.
      • According to the table of contents, the map on Pg 33 of U-DISE Secondary Thematic Maps 2015 reports data on "Percentage of Secondary Enrollment by Private Unaided Management." However, the title of the map on Pg 33 is "Percentage of Professionally Qualified Teachers: Secondary Level." Based on the context in which the map appears, and because there is another map in the document with the title "Percentage of Professionally Qualified Teachers: Secondary Level" our best guess is that the table of contents is correct and the map title on Pg. 33 is an error.
    • Estimated statewide enrolled students dewormed: (Dewormed enrolled students / Enrolled students) * Statewide enrollment in government and government-aided schools.
  • Numbers
    • Bihar (DtWI Bihar 2015 monitoring data for coverage validation, schools)
      • Included schools: 557/748 schools surveyed: 234 schools without tick marks, but in 43 of those those the principal said no deworming happened; 748 - 234 + 43 = 557.
      • Enrolled students dewormed in sample: 115,815
      • Enrollment in sample: 177,464
      • Statewide enrollment in government and government-aided schools: 23,902,897
      • Estimated statewide enrolled students dewormed: 15,599,299
    • Rajasthan (DtWI Rajasthan 2015 monitoring data for coverage validation, schools)
      • Excluded schools: 46
      • Enrolled students dewormed in sample: 37,256
      • Enrollment in sample: 43,939
      • Statewide enrollment in government and government-aided schools: 7,289,229
      • Estimated statewide enrolled students dewormed: 6,180,557
    • Delhi (DtWI Delhi 2015 monitoring data for coverage validation, schools)
      • Note that Delhi didn’t deworm students in grade 11 due to examinations. We didn’t adjust any of the numbers from the raw data because it appears that grade 11 classes were still sampled by monitors, so the monitored coverage should still reflect the low participation from that grade.
        • "Due to examinations of 11 grade in April, the SHS decided to cover these children later in the year." DtWI Delhi 2015 program report, Pg 4
        • In coverage validation data, some of the reports are from class 11. DtWI Delhi 2015 monitoring data for coverage validation, schools
      • Excluded schools: 3
      • Enrolled students dewormed in sample: 166,956
      • Enrollment in sample: 218,098
      • Statewide enrollment in government and government-aided schools: 2,564,953
      • Estimated statewide enrolled students dewormed: 1,963,495
    • Madhya Pradesh (DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools)
      • Excluded schools: 178
      • Enrolled students dewormed in sample: 35,834
      • Enrollment in sample: 52,472
      • Statewide enrollment in government and government-aided schools: 11,151,527
      • Estimated statewide enrolled students dewormed: 7,615,563
    • Chhattisgarh (DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools)
      • Excluded schools: 247
      • Enrolled students dewormed in sample: 31,535
      • Enrollment in sample: 40,575
      • Statewide enrollment in government and government-aided schools: 4,644,179
      • Only students 10-19 were dewormed so we are not able to generate a similar estimate of students dewormed in this state.
  • Our calculations in this spreadsheet: GiveWell enrollment-based student coverage check 2015

• 273
  • Chhattisgarh is excluded because only some districts in the state were covered, so we are not able to apply the same methodology.
  • The calculations for this table are in this spreadsheet: GiveWell enrollment-based student coverage check 2015
  • Reported enrolled student coverage
    • Bihar: 17,600,122 ("Number of enrolled children dewormed (age 6-19 years)") DtWI Bihar 2015 Program report, Pg 4
    • Rajasthan: 6,463,898 ("Total enrolled children (6-19 years) dewormed at schools") DtWI Rajasthan 2015 program report, Pg 4
    • Delhi: "The coverage data from the schools in Delhi indicated that 1,828,562 enrolled children were dewormed in the state during deworming day and mop up day against the total target of 2,240,573 enrolled children from class 1 to 12." DtWI Delhi 2015 program report, Pg 18.
    • Madhya Pradesh: 10,073,830 ("Number of enrolled children (Class 1 to 12) dewormed at schools") DtWI Madhya Pradesh 2015 program report, Pg 5
    • Chhattisgarh: "The coverage data from the state indicated that 916,596 children in the age group 10-19 years were dewormed against the target of 978,008. These include 849,797 enrolled children at schools and 128,211 out-of-school children." DtWI Chhattisgarh 2015 coverage validation report, Pg 7

• 274

  Deworm the World, Kenya Coverage Reporting data, Year 3, "County Level-STH" sheet, cell B24.

• 275

  "Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17.

• 276

  Paul Byatta, attachments to email to GiveWell, September 23, 2016

• 277

  GiveWell analysis of Deworm the World cost-per-treatment, October 2016, "Summary" sheet, cell H36.

• 278

  See our 2014 review.

• 279
  • "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2.
  • "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.
  • "National funds for deworming are part of the [Weekly Iron Fortification Supplements] WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12.

• 280

  Excluding estimates of the costs of teacher and government staff time, Deworm the World bears approximately 90% of costs in Kenya. See GiveWell analysis of Deworm the World cost-per-treatment, October 2016, "Summary" sheet, cell Y36.

• 281
  • "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
  • Deworm the World noted that the costs in Kenya are unusually high (partly due to higher quality M&E): "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
  • Our understanding is that costs are higher in Kenya than India primarily due to cultural differences, including paying significantly higher allowances to teachers for participating in the program. For more details, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell analysis of Deworm the World cost per treatment.

• 282

  GiveWell analysis of Deworm the World financials - 2016

• 283
  • Evidence Action held about $1.1 million in unrestricted and uncommitted funding at the end of July 2016, and it expected to raise an additional $1.2 million in unrestricted funding over the remainder of 2016, although this estimate was quite rough. GiveWell analysis of Deworm the World financials - 2016
    • In 2016: Evidence Action intended to commit $163,000 of its uncommitted unrestricted funding to Deworm the World. GiveWell analysis of Deworm the World financials - 2016
    • In 2017: Evidence Action had tentative plans to commit another $600,000 to Deworm the World. GiveWell analysis of Deworm the World financials - 2016
  • Excluding GiveWell-influenced donors and Evidence Action, Deworm the World estimated that it would raise $400,000 in general support that it could use for its 2017 budget year: "It is difficult to estimate what this number would be, given that all Evidence Action financial inflows have come from the time period since GiveWell recommended Deworm the World and we do not know exactly how much in individual donations can be attributed to GiveWell (since there could be some who do not self-identify as such). We are estimating to raise approximately $400k in general support funds for the program excluding GiveWell influenced donors and Evidence Action." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
  • $163,000 + $600,000 + 400,000 = $1,163,000.
  • We expect the remainder of Evidence Action's unrestricted funding to be allocated to its other programs, organizational investments, and reserves. Evidence Action is currently working on developing a reserves policy. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 and GiveWell analysis of Deworm the World financials - 2016

• 284

  This is based on internal records of how much GiveWell-influenced donors gave to Deworm the World in the last year (as of early November 2016) when Deworm the World was on GiveWell's top charity list but was not the recommendation for marginal funding.

• 285

  GiveWell analysis of Deworm the World financials - 2016, sheet 'DtW Unres Commit.'

• 286

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 287

  GiveWell analysis of Deworm the World financials - 2016, Sheet: GiveWell's ranked funding gaps, and Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 288

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 289

  Deworm the World has significant experience in India and believes it can hire quickly for the new positions. It is currently in discussions with four states, but believes it is likely that one of the discussions will not result in Deworm the World providing additional support to that state. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 290

  Three states at $300,000 per year for 3 years is $2.7 million. Deworm the World requested an additional $1.4 million to support the expansion of its national team. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 291

  Grace Hollister, edits to GiveWell's review, November 20, 2016

• 292

  These estimates are based on the size of the target populations. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 293
  • Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
  • "Target population would still likely be in millions –smaller than originally envisioned but still a considerable number of at-risk children" Grace Hollister, edits to GiveWell's review, November 20, 2016

• 294

  Note that this is a slightly lower value than what Deworm the World estimated the province of Punjab would need in 2015, but all estimates are highly uncertain: "It is difficult to estimate how much funding will be needed, because Deworm the World has not worked in Pakistan before and it is still early in the planning stages. There is a rough funding need of $1.5-2 million per year in Punjab, which has a target population of 5.7 million children, and $500,000-800,000 per year in Sindh, which has a target population of 2.1 million children." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg. 8

• 295

  "A multi-year funding commitment will be important to encourage the government to work with Deworm the World. Ms. Hollister would like to aim for a 5-year commitment and has had a 5-year budget created. While it is unlikely that Deworm the World will get an upfront 5-year funding commitment, and it may be difficult to reach an agreement with the government without having funding for all 5 years. Deworm the World could begin with a 3-year commitment and indicate its intention to stay for 5 years and find funding to fill the gap." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 8

• 296

  GiveWell analysis of Deworm the World financials - 2016

• 297

  Grace Hollister, edits to GiveWell's review, November 20, 2016

• 298

  This is based on our internal records and Good Ventures' grants.

• 299

  When we look at what Deworm the World has spent in unrestricted funding Jan 2014 - Jun 2016 and add to that the unrestricted funding that Deworm the World currently holds, we arrive at $17,531,665. GiveWell analysis of Deworm the World financials - 2016 Sheet: "2014-2016 expenses. Subtracting GiveWell's contributions of $15.3 million, we calculate that Deworm the World has raised approximately $2.2 million in unrestricted funding from other donors.

• 300

  Our internal records show we influenced $771,159.81 to Deworm the World between 7/1/2013 and 6/30/2014, excluding Good Ventures. Good Ventures gave $1.5 million to Deworm the World in 2013 (see here). Together, that amounts to about $2.3 million.

• 301

  GiveWell analysis of Deworm the World financials - 2016 Sheet: 2014-2016 expenses

• 302

  Our internal records show GiveWell-influenced gave about $760,000 to Deworm the World between 7/1/2014 and 6/30/2015, excluding Good Ventures. Good Ventures gave $250,000 to Deworm the World in 2014 (see here). Together, that amounts to about $1 million.

• 303

  GiveWell analysis of Deworm the World financials - 2016 Sheet: 2014-2016 expenses

• 304

  See our 2015 review.

• 305
  • Spending = $336,000 (in 2014) + $743,975 (in 2015).
  • Commitments = $1.1 million
  • Total raised from GiveWell-influenced donations = $2.3 million (2013 through 2014) + $1 million (2014 through 2015). Total we estimate Deworm the World raised from other donations over this time period: $1.4 million (two thirds of the $2.2 million we know Deworm the World raised from other donors).
  • $2,179,975 / $4.7 million = .46

• 306

  Our Salesforce records show GiveWell-influenced donors gave $1.2 million to Deworm the World between 7/1/2015 and 6/30/2016, excluding Good Ventures. Good Ventures gave almost $10.8 million to Deworm the World in early 2016 (see here). Together, that amounts to about $12 million.

• 307

  January through June, Deworm the World spent $350,715. July through December, it expected to spend 841,834. GiveWell analysis of Deworm the World financials - 2016 Sheets: "2014-2016 expenses" and "2016 and 2017 budgets"

• 308

  GiveWell analysis of Deworm the World financials - 2016 Sheets: DtWI Unres Commit

• 309

  See the blog post linked above.

• 310

  See our November 2014 review:

  "The Deworm the World Initiative seeks an additional $1.3 million to support its activities in 2015 and 2016. DtWI expects to spend $377,000 of the $1.3 million (29%) it seeks on work related to expanding school-based mass deworming programs and funding related operating expenses (including impact evaluation related expenses). More specifically, these activities would be:

  • $230,000: staff to support expansion in India, new countries, and related operating and evaluation expenses.
  • $144,000: DtWI overhead. [These funds support DtWI as an organization but are not directly programmed (e.g., a portion of Alix Zwane's, the Executive Director of Evidence Action salary, Evidence Action financial staff, etc.).]
  • $500,000: evaluation of new evidence-based programs that leverage deworming. We have limited detail about what this would entail. One idea that DtWI has investigated is the possibility of distributing bednets along with deworming pills in schools as an alternative distribution mechanism to national net distributions. Another is including hand-washing educational programming alongside deworming days. This line item includes $50,000 to support DtWI's evaluation of its hygiene and deworming program funded by Dubai Cares and $50,000 to enable DtWI to hire a senior epidemiologist.
  • $230,000: staff to support evaluation of DtWI’s work in Kenya. This work is primarily funded by CIFF. DtWI believes that additional resources can improved significantly the quality of the analysis done regarding the cost effectiveness of breaking transmission.
  • $170,000: implementation support for the integrated deworming, sanitation and hygiene education program in Vietnam, in partnership with Thrive Networks."

• 311

  Also see the next footnote. Deworm the World told us that it had allocated unrestricted donations in the following way:

  • $200,000: A prevalence survey in Pakistan
  • $111,000: Support for the TUMIKIA and TakeUp studies
  • $93,000: Support for a deworming program in Vietnam
  • $12,000: A training materials project in India
  • $100,000: Salaries for staff exploring work in new countries (intended to extend to 2016)
  • $~550,000: Support for a program in Cross River, Nigeria.

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 312

  How expected spending at the end of 2014 compared to how Deworm the World actually spent and committed funding in 2015:

  • Expected staff spending: $230,000.
    • Actual spending: We are unsure how much Deworm the World actually spent and allocated to staff. It is difficult to tell what portion of Deworm the World's 2015 costs were for staff because staff costs are built into country costs.
    • Committed funding: $100,000 for salaries for staff exploring work in new countries (intended to extend to 2016).
  • Expected overhead spending: $144,000.
    • Actual spending: $239,447 on "Global" costs
    • Committed funding: None
  • Expected Vietnam spending: $170,000.
    • Actual spending: $32,613
    • Committed funding: $93,000
  • Expected evaluations spending: $500,000.
    • Actual spending: $17,059 (for the Tumikia study)
    • Committed funding: 111,000 for support for the Tumikia and TakeUp studies.
  • Expected Kenya spending: $230,000.
    • Actual spending: $130,979
    • Committed funding: None
  • Note that Deworm the World also allocated substantial funding to opportunities that we had not previously expected, such as support for a new program in Nigeria. See previous footnote.

  See the previous footnotes, Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013, and GiveWell analysis of Deworm the World financials - 2016 Sheets: "2014 - 2016 expenses" and "Expected vs. Actual spending"

• 313

  See our November 2015 review. Deworm the World intended to spend unrestricted funding on:

  • A deworming program in Pakistan (~$8.4 million for a 3-year program: $2 million per year in Punjab and $0.8 million per year in Sindh)
  • A deworming program in a new country (~$6 million for a 3-year program)
  • A deworming program in Vietnam (~$2.6 million, unclear how many years)
  • Reserves ($2.8 million)

• 314
  • GiveWell analysis of Deworm the World financials - 2016 Sheets: "DtW Unres Commit" and "Expected vs. Actual spending"
  • Another donor funded more of the Vietnam program than Deworm the World expected and the prevalence surveys in Vietnam revealed a need for a smaller program than expected. Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 315

  Deworm the World staff, conversations with GiveWell, October 3-4, 2016

• 316

  "Evidence Action launched formally in 2013 to scale programs with sustainable business models that have been proven to be effective so that they benefit millions of people.

  Evidence Action leads and manages two programs incubated by Innovations for Poverty Action: Dispensers for Safe Water and the Deworm the World Initiative. We also run Evidence Action Beta where we are currently testing a number of other rigorously-evaluated interventions for scale-up." Evidence Action website, Who we are (November 2016)

• 317

  "At the time of your visit we were and have remained actively engaged in correcting and converting our financial information for 2013, 2014, 2015 and 2016. As Jeff told Elie in August, in the last several months we embarked on a massive effort to upgrade our financial systems and practices. These include the implementation of a new global accounting system (Intacct), conversion of our books from cash to accrual, and the resolution of many outstanding financial issues from the IPA spinoff. Thus, we have been slow in responding to your questions.

  As always, we appreciate GiveWell’s in-depth questions and the value you place on transparency. The continued inquiry reinforces our commitment to improve our finance and administrative capacity, and will strengthen our organizational ability to operate efficiently at scale. The combination of our effort to improve our financial data and management, and your queries have helped us discover past coding errors, and to refine our planning around internal funds transfers. Although we continue to review, examine and strengthen our records and systems, we are more confident in the accuracy of the information provided here in response to your questions." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016

• 318
  • One of the documents had an incorrect formula (adding in expenses rather than subtracting them), while another double counted about $750,000: "The version shared during the [meeting] reflected $750K for core support allocated to programs, as well as $1.3M in unrestricted fund transfers to programs. After realizing that we had double counted the $750K in core support, we eliminated it in the version sent on Oct 11." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
  • Deworm the World's restricted funding on hand was about $2.5 million lower after corrections had been made. Our understanding is that about $1.7 million of that was funding that had been mislabelled as restricted, but was actually unrestricted, and the rest of the decrease was due to the fact that Deworm the World had actually already spent a portion of its restricted funding in India (this change increased Deworm the World's expenses figure by $500,000 and Evidence Action's starting revenue by approximately $200,000).
    • "$1.7M in Deworming restricted funding was booked into our financial system as revenue against a current commitment instead of as new revenue. The coding has been corrected and this amount is now reflected in an increase in revenue on the report (cell D5) [...] $730K in expenses incurred in India between March and July 2016 were not included in the previous report - increasing total expenses by a like amount. Note that $187K in organization development costs allocated to the program in the form of indirect costs in past tables was backed-out (see second bullet under Evidence Action updates below) resulting in a net increase in expenses of $540K (cell D6)." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
  • Deworm the World's revenue increased by $2.7 million after corrections. We believe this is from the $1.7 million that was initially mislabelled as restricted funding and an additional $1 million that was incorrectly allocated internally. Correcting the misallocation also caused Evidence Action's revenue to decrease by $1 million.
    • "$1.7M in Deworming restricted funding was booked into our financial system as revenue against a current commitment instead of as new revenue. The coding has been corrected and this amount is now reflected in an increase in revenue on the report (cell D5). [...] $1.0M in Deworming program general support funds (what we had been calling "unrestricted" for deworming) received in 2015 and 2016 were coded inconsistently. These costs were correctly coded to the Deworming program but were assigned the wrong funder code. The coding has been correctly aligned, and this amount is now reflected in an increase in revenue on the report (also cell D5)." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
  • Evidence Action's expenses increased $750,000 after corrections, to reflect that the funding had been used on organizational expenses (this was previously unreflected in the original documents): "The $750K was spent on one-time operational costs such as temporary accountants to convert our books from cash to accrual, transition to a new accounting system, separation costs from the incubator established in India, and relocation costs for the Nairobi office after our landlord ended the lease." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
  • Once these changes had been made, Evidence Action revised the amount of unrestricted funding it intended to allocate to Deworm the World this year, decreasing it from $350,000 originally to $163,000.

• 319

  Originally, Deworm the World's financial documents indicated that Deworm the World had $6,363,718 in unrestricted funding available; after corrections, this figure was $10,969,610.

• 320

  "Note that this is specific to CIFF; the END Fund grant is slated to end in 2018" Grace Hollister, edits to GiveWell's review, November 20, 2016

• 321

  Grace Hollister, edits to GiveWell's review, November 20, 2016

• 322

  GiveWell's non-verbatim summary of a conversation with END Fund staff, October 17, 2016

• 323

  Grace Hollister, email to GiveWell, June 9, 2016

• 324
  • "[In Nigeria] Deworm the World would ideally like to make a 3-year commitment in Cross River to increase the chance of government approval, increase stability, enable Deworm the World to establish a partnership with the government, and take steps toward institutionalization of deworming programs.
    There is some inherent risk in multi-year commitments, but one year is not always enough time to build a new program that runs effectively, and governments would be reluctant to work with Deworm the World if they were limited to one year. [...] One-year commitments can be costly for Deworm the World because governments typically expect that programs will continue past the first year, and it can be difficult to find funding." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7
  • "[In Pakistan] A multi-year funding commitment will be important to encourage the government to work with Deworm the World. Ms. Hollister would like to aim for a 5-year commitment and has had a 5-year budget created. While it is unlikely that Deworm the World will get an up-front 5-year funding commitment, it may be difficult to reach an agreement with the government without having funding for all 5 years. Deworm the World could begin with a 3-year commitment and indicate its intention to stay for 5 years and find funding to fill the gap." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 8

• 325
  • "Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage.
  • GiveWell conducted a site visit to DSW in November 2012 and published notes. DSW 2012 GiveWell site visit

• 326

  GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and Evidence Action 2015 funding gap analysis

• 327

  Evidence Action, Projected allocation of unrestricted funds, 2016.

• 328

  Evidence Action, Projected allocation of unrestricted funds, 2016

• 329

  GiveWell analysis of Deworm the World financials - 2016, Sheet: "EA Unres Commit - guess"

• 330

  Deworm the World's early 2016 plan indicated spending $100,000 on Deworm the World if it raised $1.9 million. Evidence Action, Projected allocation of unrestricted funds, 2016

• 331

  Originally $300,000 was expected to be used for Evidence Action Beta. Evidence Action, Projected allocation of unrestricted funds, 2016

• 332

  It's possible that support for Evidence Action Beta is included under the organizational development category. It is also possible that Evidence Action did not need to support Evidence Action Beta after Good Ventures provided Evidence Action a grant at the recommendation of GiveWell as part of GiveWell's experimental work.

• 333

  The rest was allocated to organizational development. Evidence Action, Projected allocation of unrestricted funds, 2016 and GiveWell analysis of Deworm the World financials - 2016, Sheet: "EA Unres Commit - guess"

• 334

  GiveWell analysis of Deworm the World financials - 2016, Sheet: "EA Unres Commit - guess"

• 335

  "...many of the investments of unrestricted have substantial positive benefit for DtW, even as they are not specifically programmatic in nature. These include investment in financial capacity, and the transition of the Indian entity (which at the moment is exclusively working on deworming, though this may not be the case in the future)." Grace Hollister, edits to GiveWell's review, November 20, 2016

• 336

  WHO, Summary of global update on preventive chemotherapy implementation in 2015, Pg 456-457, Table 1.

• 337
  • WHO Weekly epidemiological record, 18 December 2015, Pg 707 (Table 1)
  • In 2013 about 34% of all children in need of STH deworming received treatment. WHO Weekly epidemiological record, 6 March 2015, Pg 91 (Table 1)
  • Older data: "In 2009, a total of 314.6 million children were reportedly dewormed, of whom 274 million required preventive chemotherapy. This corresponds to approximately 31% of all children in need of treatment." WHO STH treatment report, Pg. 34.