Evidence Action's Deworm the World Initiative - November 2017 Version

We have published a more recent review of this organization. See our most recent report on Deworm the World.

Published: November 2017

Summary

What do they do? Evidence Action's Deworm the World Initiative (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates government-run school-based deworming programs. (More)

Does it work? We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost. Deworm the World sends monitors to schools during and after deworming to determine whether the programs it supports have reached a large proportion of children targeted. We have reviewed data from each of its major programs, which overall indicate strong results. (More)

What do you get for your dollar? Our best guess is that deworming is generally highly cost-effective. We estimate that, in Kenya, the cost per child dewormed is about $0.71 per child, or $0.50 per child excluding in-kind contributions from governments. We estimate that the cost per child treated in India is less than half of that. We expect the cost per treatment in other countries to be closer to that in Kenya than India, and may be more expensive in the early stages of a program. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)

Is there room for more funding? We believe that Deworm the World is very likely to be constrained by funding. In the next three years, we expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive in that time. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities. (More) December 2017 update: In November 2017, we recommended that Good Ventures give $15.2 million to Evidence Action’s Deworm the World Initiative (including $5.5 million to build its reserves). We also recommended that GiveWell’s Board of Directors grant $0.7 million in discretionary funds. After accounting for these grants, we estimate that Deworm the World has a remaining funding gap of about $8.5 million for its deworming programs.

Evidence Action's Deworm the World Initiative is recommended because of its:

• Focus on a program with a strong track record and excellent cost-effectiveness. (More)
• Strong process for assessing whether the deworming programs it supports are successfully deworming children. (More)
• Standout transparency – it has shared significant, detailed information about its programs with us.
• Room for more funding – we believe Deworm the World will be able to use additional funds to start or maintain deworming programs.

Major open questions include:

• A major portion of the past impact that we attribute to Deworm the World is based on its work in India. In India, implementation of the deworming program is funded by the government and Deworm the World provides technical assistance. In estimating Deworm the World's cost-effectiveness, we have assumed that, because Deworm the World's expenditures account for 45% of the total costs of the India program, it should receive credit for causing 45% of the treatments to occur. However, it seems plausible to us that Deworm the World should receive significantly less or more credit than we have given it, depending on how crucial its technical assistance has been in causing the distributions to take place and in raising coverage rates.

Our review process

Our review process has consisted of:

• Extensive conversations with Deworm the World Director Grace Hollister and other Deworm the World and Evidence Action staff since 2012.1
• Reviewing documents Deworm the World sent in response to our queries.
• Site visits:
  • In November 2012, we visited Deworm the World's office in Nairobi, Kenya and met its staff there. (Notes from our visit)
  • In October 2013, we visited Deworm the World's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with Deworm the World. (Notes from our visit)
• In 2015, we retained two journalists to visit areas served by Deworm the World in Kenya. We published their report on our blog.
• Conversations with the Children's Investment Fund Foundation (CIFF), a funder of Deworm the World.2

All content on Deworm the World, including past reviews, updates, blog posts, and conversation notes, is available here. We have also published a page with additional, detailed information on the program to supplement some of the sections below.

What do they do?

The Deworm the World Initiative is a program led by Evidence Action, an organization that focuses on scaling up interventions that it believes are cost-effective and evidence-based. Deworm the World advocates for and supports the implementation of government-run deworming programs for preschool- and school-age children.3 The support that Deworm the World provides is of two types: 1) offering technical assistance to governments implementing deworming, and 2) funding components of deworming programs.4

The deworming programs that Deworm the World supports are focused on executing school-based mass drug administrations (MDAs), in which the aim is to treat the entire population of children within a geographic area by distributing deworming pills.5 Deworm the World focuses on MDAs that treat children infected with soil-transmitted helminthiasis (STH).6 Where needed, these programs also include treatment for schistosomiasis.7

In the countries it works in, Deworm the World works primarily with government staff to implement deworming programs; as it has expanded to new countries, it has started to also collaborate with non-governmental partners to support governments.8 Deworm the World has also funded or is considering working on a few projects that fall outside of its purview of supporting the direct implementation of deworming programs, specifically related to monitoring and research (see footnote).9

Deworm the World was founded in 2007,10 and as of 2017 had supported deworming treatments in India, Kenya, Ethiopia, Nigeria, and Vietnam and had started preliminary support for a deworming program in Pakistan.11 Many of these programs are recent and represent progress on Evidence Action's efforts to scale up Deworm the World; as of early 2015, Deworm the World had only supported treatments in India and Kenya. On a separate page with additional information about Deworm the World, we discuss more details of its work by country.

Below, we discuss:

• Deworm the World's role in government-led deworming programs
• A breakdown of Deworm the World’s spending
• Deworm the World's relationship to Evidence Action

Deworm the World’s role in government-led deworming programs

The deworming programs that Deworm the World supports are implemented by the governments it works with.12 Below, we expand on Deworm the World's role in the programs it supports. Note that we use "Deworming Day" to mean the day on which the MDA takes place. Similarly, we use "Mop-Up Day" to refer to the day that occurs several days after Deworming Day and is when students who were absent or sick on Deworming Day are given their deworming pills (although note that not all countries have just one Deworming Day or include a Mop-Up Day in their program).13

The assistance that Deworm the World provides in each country varies based on what each partnering government needs.14 Historically, Deworm the World's role has included the following:15

1. Advocacy. Deworm the World actively encourages national and large sub-national governments to implement mass school-based deworming programs.16 Our impression is that Deworm the World's advocacy consists of meeting with health and education officials in a government to discuss the benefits of deworming and how a deworming program might be implemented.17 Deworm the World has told us that it will not work with a government on a national deworming program until it has built a strong working relationship with that government via its advocacy.18 Deworm the World also participates in the broader "STH community"; that is, it works with other organizations advocating for and implementing activities that aim to further reduce or eliminate STH globally.19
2. Prevalence surveys. Before Deworm the World helps launch a deworming program in a new area, it evaluates whether the prevalence of worm infections is sufficient to justify an MDA for the school-age population.20 If no prevalence surveys have been conducted recently, it generally commissions one.21 The results of prevalence surveys are used to determine the appropriate treatment strategy (in particular, MDA frequency) for a given location.22 It also plans to conduct follow-up prevalence surveys periodically, so that it can track the impact of the MDAs and refine treatment strategies as needed, in accordance with WHO guidelines.23 Deworm the World generally contracts out work on prevalence surveys.24
3. High-level program planning. Deworm the World has told us that it often assists governments with high-level operational decisions, such as developing the country's treatment strategy and operational guidelines25 and creating a budget for the program.26
   
4. Drug procurement and protocols. Deworm the World assists governments in obtaining drugs, designing drug distribution and tracking processes, and developing adverse event protocols for cases where children react poorly to treatment.27 For example, Deworm the World has helped governments submit requests for deworming drugs (albendazole or praziquantel) to the World Health Organization (WHO) global drug donation program.28
5. Program preparation: trainings and distribution of materials. Deworm the World has helped governments design and organize what it calls a "training cascade" (more detail in the footnote).29 Through the training cascade, teachers and other government staff learn how to implement a Deworming Day and receive materials necessary for implementation (such as reporting forms and drugs).30 In the past, Deworm the World has hired or trained staff to lead the trainings and developed materials for the trainings.31 In India, Deworm the World has also arranged tele-callers to reach out to schools to assess their preparedness and notify government officials of any problems before Deworming Day.32
6. Community sensitization. Deworm the World supports community sensitization efforts, which aim to make local communities aware of Deworming Day and the benefits of deworming children.33 For example, via the training cascade, teachers are instructed to spread the word about Deworming Day to their communities.34 Deworm the World has also developed text message campaigns, organized public announcement events, and edited mass media materials to be more appropriate for local contexts.35
7. Monitoring and evaluation. Deworm the World told us that it helps governments design or improve reporting and monitoring systems. It also collects monitoring data independently.36 Deworm the World focuses on assisting with the collection of three main types of monitoring data:37
   • Monitoring before and during deworming: Monitors hired by Deworm the World visit schools before and during Deworming Day and Mop-Up Day. They are meant to assess both a) how prepared schools and health systems are to implement deworming and b) the extent to which proper procedures are followed.38 Monitoring visits may include assessments of the quality of trainings, community sensitization efforts, and Deworming Day activities, depending on what Deworm the World and the government agree to monitor.39
   • Coverage reporting: On Deworming Day and Mop-Up Day, teachers are asked to mark the number of children that they deworm and schools complete specially designed reporting forms to tally the number of children treated. This data is then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (e.g., in India: school, node, block, district, and state) to create a reported coverage estimate for a region.40 Deworm the World notes that, more recently, block officials in India have submitted coverage data online to the national government.41
   • Coverage validation: Approximately one week after Mop-Up Day, Deworm the World sends independent monitors back to schools to check the coverage data and attendance records recorded at schools against the data submitted and ask students about whether or not they were dewormed.42 This data can then be compared to the coverage data reported by the government.

   Typically, Deworm the World hires and trains third-party monitors to collect process monitoring and coverage validation data; the following footnote includes Deworm the World’s descriptions of the monitor selection process used in four states in India in 2015.43 In Kenya, Evidence Action (Deworm the World's parent organization) maintains a monitoring team year-round that Deworm the World makes use of.44

On a separate page, we detail Deworm the World's work by country.

Breakdown of Deworm the World’s spending

We summarized Deworm the World's spending for 2016 and the first half of 2017 in this spreadsheet.

In short, in this period:

• Deworm the World spent a total of $8.0 million in 2016 and $5.2 million in the first half of 2017.
• About two-thirds of spending was from funding sources that are restricted to a particular project, primarily programs in India and Kenya which are funded by CIFF and the END Fund. The other third of funding was from funds that Deworm the World can allocate at its discretion (this includes funds directed by GiveWell).
• Deworm the World's biggest programs by far in this period (and historically) were in Kenya (41% of program spending) and India (46% of program spending). Nigeria accounted for 8% of spending, Pakistan 3%, and Ethiopia and Vietnam 1% each.45

For information on spending in previous periods, see the previous versions of our review of Deworm the World.

Deworm the World and Evidence Action

In early 2013, Innovations for Poverty Action (IPA) announced the formation of Evidence Action to scale cost-effective and evidence-based programs. Two IPA initiatives, Deworm the World and Dispensers for Safe Water, were spun off from IPA to be managed by Evidence Action. Evidence Action has since built a department for investigating, testing, and considering new programs for scaling up called Evidence Action Beta; one program in the Beta portfolio is No Lean Season, which GiveWell also recommends as a top charity.46 We focus this review on Deworm the World and discuss the room for more funding implications of Deworm the World being a program of a larger organization below.

Does it work?

We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost. Evidence from Deworm the World's monitoring makes a relatively strong case that the programs Deworm the World has supported have successfully dewormed children.

While Deworm the World's track record in Kenya and India is strong, it has recently expanded to several new countries. As with any major scale-up, there is a risk that it may not achieve as strong results as it has in the past or may not produce similarly strong evidence of its impact. We have seen monitoring from two of its new countries of operation, Vietnam and Nigeria. These results indicate that Deworm the World is using similar monitoring processes in new countries as it has in Kenya and India and the results have been reasonably strong.

In the sections below, we focus on the following questions to understand whether Deworm the World’s activities are having the intended impact.

• Are mass school-based deworming programs effective when implemented well?
• Are Deworm the World's programs targeted at areas of need?
• Are deworming pills delivered to and ingested by recipients?
• How does Deworm the World affect program outcomes?
• Are there any negative or offsetting impacts?

Are mass school-based deworming programs effective when implemented well?

Deworm the World supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.

There are some important differences between the type and severity of worm infections in the places Deworm the World works and the places where the key studies on improved life outcomes from deworming took place (which we discuss below). In particular, Deworm the World primarily provides support to mass drug administrations (MDAs) that treat populations where fewer children are infected with soil-transmitted helminths and where the severity of infections tends to be lower. In addition, several of the programs Deworm the World supports do not treat schistosomiasis because it is not endemic in the areas the programs support.47

Are Deworm the World's programs targeted at areas of need?

What is the likely impact per treatment in Deworm the World's programs compared with the independent studies on the impact of deworming? In general, mass deworming programs treat everyone in a targeted demographic, regardless of whether each individual is infected (more). Because of this, the benefits (and therefore the cost-effectiveness) of a program are highly dependent on the baseline prevalence of worm infections. In this section, we discuss how the disease burden in the areas Deworm the World works in compares to the places where the independent studies that form the evidence base for the impact of deworming were conducted. While it is our understanding that Deworm the World programs generally target areas that require mass treatment according to WHO guidelines,48 the disease burden in Deworm the World areas is on average lower than in the study areas, so our expectation is that the impact per child treated is lower in Deworm the World areas. We adjust our cost-effectiveness estimate accordingly (more below). In this spreadsheet, we compare the worm prevalence in places where Deworm the World currently supports a program to the prevalence from the studies providing the best evidence for the benefits of deworming. The prevalences in the table for Madhya Pradesh and Chhattisgarh were measured after multiple rounds of treatment, while Bihar, Rajasthan, and Uttar Pradesh's prevalences were measured after one round of treatment. This makes it difficult to compare the prevalences in a meaningful way. It is likely that prevalence was higher at baseline for these regions. Deworm the World notes that there are relevant methodological differences between the prevalence surveys, which makes them difficult to compare;49 we agree that this data is not ideal for our purposes but believe that it provides the best estimate we have and adjusting for baseline infection rates is an important part of our cost-effectiveness model. Treatment for lymphatic filariasis In some of the countries where Deworm the World works, there are existing programs to treat lymphatic filariasis (LF).50 Albendazole, the same drug used to treat STH, is usually used in combination with one additional drug to treat LF (and the same dosage is used for both treatments).51 For areas that have existing LF treatment programs, the effect of Deworm the World’s support may be to transition an area from once-per-year deworming treatment (for STH) to twice-per-year treatment.52 We detail what we know about the status of LF programs in the areas in which Deworm the World works on a separate page with additional information about Deworm the World. Are deworming pills delivered to and ingested by recipients? Administration of deworming drugs is a relatively straightforward program, though any MDA could encounter challenges when operating at a large scale. The information we have seen from monitors hired and trained by Deworm the World in India, Kenya, Vietnam, Ethiopia, and Cross River, Nigeria suggests that the programs successfully deliver pills to children, who then swallow them. Additionally, prevalence surveys in Kenya, and to a lesser degree Bihar, India, show that the prevalence rates of STH and schistosomiasis have declined substantially since Deworm the World started supporting MDAs in those areas, providing additional evidence that the treatments are reaching recipients.53 Evidence from monitoring For each of its programs, Deworm the World hires monitors (who are not associated with the government implementing the program) to:54 [Sometimes] Observe trainings. For some of its programs, Deworm the World sends monitors to visit a random selection of trainings.55 The visits are unannounced, and the monitors use a standardized checklist to track if the training covers all required topics.56 Monitors also test how the training increases the knowledge of training attendees by selecting several attendees before the training and administering a short quiz to them, then selecting several of the attendees to re-take the quiz after the training.57 [Sometimes] Visits or calls to communities and schools before Deworming Day. For some of its programs, Deworm the World asks monitors to visit or call, unannounced, a random selection of schools and/or communities before Deworming Day.58 At the schools, monitors will interview teachers and/or headmasters, asking a variety of questions to assess preparedness, such as whether or not the school has enough treatments for Deworming Day and if a representative from the school attended training.59 In communities, monitors select a sample of adults to interview, asking questions to assess their awareness of Deworming Day, which helps Deworm the World determine how successful its community sensitization efforts were.60 Observe activities on Deworming Day and Mop-Up Day. In all of its programs, Deworm the World sends its monitors, unannounced, to observe a random sample of schools on Deworming Day and Mop-Up Day.61 At the schools, monitors interview teachers and school administrators to assess how prepared the school was for Deworming Day. For example, monitors ask if the school has sufficient drugs for Deworming Day, whether or not a school representative attended training, and a variety of questions to test teachers' knowledge about the proper procedures for the MDA campaign, like what the teacher should do if a child is feeling sick.62 Then, the monitors observe randomly selected classes, recording details about the Deworming Day activities, such as whether deworming is in progress, teachers are documenting who is dewormed, and teachers are watching to make sure that students swallow the pills.63 Conduct a coverage validation exercise. In most of its programs, Deworm the World sends out monitors within 1-2 weeks (though sometimes longer) of Deworming Day and Mop-Up Day to conduct a coverage validation exercise at a randomly selected sample of schools.64 At the schools, monitors ask teachers questions about Deworming Day, such as if the school had sufficient tablets.65 They also check class registers and record the number of students that were dewormed according to the school's records.66 Finally, they randomly select a small sample of students to interview, asking the children several questions about their experience on Deworming Day.67 For example, they might ask if the child received a pill and if the child swallowed the pill.68 We believe that the last two types of monitoring are especially valuable. (We note that Deworm the World is not supporting coverage validation in schools in Ethiopia, because household coverage surveys conducted after the Deworming Days are occurring there instead. Deworm the World also did not support coverage validation monitoring for the first MDA it supported in Cross River, Nigeria, although it hopes to in subsequent rounds.69 ) While we believe that Deworm the World's monitoring is fairly high quality, we have a few concerns about its methods: Deworm the World does follow-up calls to audit monitors' work in Kenya. It also has some checks on monitors' work in India. We are not aware of audits of monitors' work in other locations.70 Deworm the World's program and monitoring are done through schools, which makes it difficult to know how many non-enrolled children are potentially being missed by the program. Schools are supposed to target and record the treatments they distribute to non-enrolled children, but most locations do not have reliable data on how many non-enrolled children there are.71 Deworm the World notes that it has piloted community-based coverage evaluation surveys, in addition to school-based monitoring, in a few states in India and will share results when they are available.72 Even though school staff are not aware ahead of time that a monitor is coming to visit, once a monitor shows up, they may be motivated to execute the program in a more rigorous fashion than they would have otherwise. It is possible that children feel pressure to say that they took the deworming pill on coverage validation day. Deworm the World tries to ask students away from their teachers, to reduce pressure, and also has some checks on students' answers.73 Results from monitoring We find the evidence produced by Deworm the World from observations on Deworming Day and during the coverage validation exercise to be compelling support for the claim that children receive and ingest pills during Deworm the World-supported MDAs; we have laid out this evidence in this spreadsheet. We also see some reason to interpret these data cautiously, as some of it was self-reported by people who could be biased to want favorable monitoring results. We note that some of the results from Kenya seem quite high to us when compared to the results we have seen from India. We are not sure if this is due to some bias in the way that the Kenya data is collected, or if Deworm the World's program in Kenya is simply higher-quality.74 See this spreadsheet for a sample of methods used in and results from recent monitoring in Kenya, India, Vietnam, Ethiopia, and Cross River, Nigeria. The spreadsheet does not include all data the monitors collected. We have selected questions that we found particularly relevant to assessing the quality of the programs and easy to interpret. (Note: Previously, we summarized monitoring from the 2013-2014 programs and the 2012 programs). We have not prioritized reviewing the data from the monitoring Deworm the World conducts before Deworming Day, although we think it could also provide insight as to the quality of Deworm the World's program. However, we have included some metrics in our table related to the preparedness of schools and their adherence to proper protocols (from questions that are asked on Deworming Day). Based on the data and reports we have seen, we feel fairly confident that most schools in India, Kenya, Nigeria, and Vietnam are prepared to implement Deworming Days: most schools seem to have a sufficient number of deworming pills available for children; fewer appear to have adequately trained teachers (the fact that the surveys identified low training rates in several cases increases our confidence in the reliability of the surveys). We feel less confident that schools are following proper procedures, especially with regards to accurate reporting. Ethiopia coverage survey Although we have not yet seen results from Deworm the World's standard monitoring for the MDAs conducted in Ethiopia in mid- and late 2015, we have seen a coverage survey conducted after the April 2015 MDA.75 It is our impression that the coverage survey was primarily supported by Deworm the World's partner in Ethiopia, the Schistosomiasis Control Initiative (SCI). We have seen headline results from a 2016 coverage survey from SCI as well. We write about the methods used for SCI's coverage surveys here and lay out the results and methods used for the Ethiopia coverage survey in this spreadsheet. We believe the coverage survey provides relatively strong evidence that deworming pills were distributed and ingested by children in Ethiopia. Evidence from prevalence surveys over time Deworm the World or its partners have conducted surveys in Kenya to track changes in schistosomiasis and STH prevalence and intensity rates following Deworm the World-supported treatment programs. In general, prevalence and intensity of the parasites decreased over time. This increases our confidence that Deworm the World's program is effectively delivering treatments to children. Deworm the World has also conducted two surveys of prevalence in Bihar, India, before and after treatment; the results from the surveys are not directly comparable, but point in the direction of the program having an effect on prevalence levels. We discuss methodology, results, and limitations to using these surveys as evidence of Deworm the World's impact on another page. How does Deworm the World affect program outcomes? Deworm the World may be having an impact in the following ways: It may increase the likelihood that a government implements a deworming program, by advocating for deworming programs, by offering to provide technical assistance, and/or by funding implementation. Deworm the World expects to pay for the majority of financial program costs in many of the new countries to which it has expanded or intends to expand.76 In situations where Deworm the World is funding a deworming program, we believe it is likely that Deworm the World plays an instrumental role in causing the program to happen.77 Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World. It may improve the quality of a deworming program that would have been implemented without Deworm the World (leading to more children dewormed effectively or improved cost-effectiveness). Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are uncertain about this. Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World. Are there any negative or offsetting impacts? We discuss several possible considerations but do not see significant concerns. Administering deworming drugs seems to be a relatively straightforward program.78 However, there are potential issues that could reduce the effectiveness of some treatments, such as: Drug quality: For example, if drugs are not stored properly they may lose effectiveness or expire.79 Our understanding is that Deworm the World periodically tests the quality of drugs and has monitored storage conditions in each of its recent programs, and this information suggests there have been minimal issues.80 In India, state governments are responsible for procuring the drugs from local suppliers and quality testing them.81 Deworm the World told us that it has conducted additional testing of drugs in India and found that they meet national standards but may not fully meet international standards. It is working with the government to determine next steps.82 Dosage: If the incorrect dosage is given, the drugs may not have the intended effect and/or children may experience additional side effects.83 It appears that for STH treatment, all children of a given age group are given the same dose of albendazole and that the dose is generally a single tablet for children 2 years old and above, and half a tablet for those between the ages of 1 and 2.84 Deworm the World monitors reported that, in recent programs in India, up to 9% of schools observed gave children less than the prescribed dose of albendazole and up to 5% gave more than the prescribed dose of albendazole.85 Replacement of government funding: We have limited information about whether governments would pay for the parts of the program paid for by Deworm the World in its absence, though our impression is that they would not.86 We also have little information about what governments would use deworming funds for if they did not choose to implement deworming programs. Diversion of skilled labor: Drug distribution occurs only once or twice per year and is conducted by teachers in schools. Based on our site visit in Rajasthan, our impression is that the Nodal Officer (the state official who manages all state school-based programs), the Nodal Officer’s staff, and the people that the Nodal Officer manages throughout the state (Resource Persons and Community Development Project Officers) have significant capacity to take on additional programs, so their taking on this program doesn't impose a significant burden on their time.87 On the other hand, a principal we spoke with commented that he would prefer fewer school-based health programs because they take focus away from the school day.88 Adverse effects and unintended consequences of taking deworming drugs: Our understanding is that expected side effects are minimal and there is little reason to be concerned that drug resistance is currently a major issue (more information from our report on deworming). Popular discontent: We have heard a couple of accounts of discontent in response to mass drug administration campaigns, including one case that led to riots.89 While the accounts we have heard are from programs supported by the Schistosomiasis Control Initiative, we think it is possible that other deworming programs could cause similar effects. What do you get for your dollar? This section examines the data that we have to inform our estimate of the expected cost-effectiveness of additional donations to Deworm the World. Note that the number of lives significantly improved is a function of a number of difficult-to-estimate factors. We incorporate these into a cost-effectiveness model which is available here. In this section, we focus on the cost per child treated, which is an important input in our cost-effectiveness model. On a separate page, we discuss: How accurate are Deworm the World's reported coverage figures? Deworm the World uses government-reported figures on the number of treatments delivered when calculating its cost per treatment. We remain uncertain of the accuracy of these figures. More here. Does Deworm the World "leverage" government funds, such that its activities mobilize resources from other actors? We could imagine that Deworm the World's funds have substantial leverage but could also imagine that other actors’ involvement is causing Deworm the World to pay for things for which other actors would otherwise have paid. Deworm the World may have less leverage in its future programs than it has had in past programs. More here. What is the cost per treatment? We estimate that in India children are dewormed for a total of about $0.34 per child, or $0.09 per child excluding the value of teachers’ and principals’ time spent on the program. In Kenya, we estimate the total cost per treatment at about $0.71 per treatment or $0.50 excluding the value of teachers’ and principals’ time spent on the program. We expect the cost per treatment in the countries Deworm the World has expanded to recently and may expand to in the future to be closer to Kenya's costs, but higher at first than Kenya due to additional costs associated with starting a new program. Deworm the World told us that the cost per treatment in India is unusually low.90 We note that Deworm the World's cost per child analysis from Cross River, Nigeria for 2016 was about 50% higher than its estimate for Kenya in 2015-2016.91 Full details in this spreadsheet. Our approach Our cost per treatment analyses rely heavily on analyses Deworm the World has done to allocate its own costs across programs, collect data on costs paid by other partners and governments, and estimate costs when data is not available. Deworm the World has told us that it tries to capture all costs of its program, regardless of who pays for the cost. We have made several adjustments to Deworm the World's estimates: For India, about two thirds of the total cost comes from contributions of time from government employees—which have been monetized according to salary levels—rather than financial costs. Appropriately valuing that time spent, and estimating how much time is spent by teachers and others due to the deworming program, is likely the largest source of uncertainty in the cost-per-child-treated calculation for India.92 In its own calculations, Deworm the World excludes the value of government employees' time because the government would have incurred these costs in the absence of the program.93 There are several high-level costs not directly attributable to programs that Deworm the World does not include (such as exploratory work in new geographies that does not lead to a new program).94 We have included an estimate of these costs in our analyses. For Kenya and Nigeria, we have included an estimate of in-kind contributions from governments. This estimate is very rough and uses the same approach that we use in estimating the total cost per treatment for SCI. In short, we assume that government contributions account for 30% of the cost of the program, based on a 2011 paper that analyzed the costs of an SCI program in Niger.95 We adjust the number of treatments reported by Deworm the World-supported programs by the attendance and coverage rates found in Deworm the World's monitoring. Details of this adjustment are in this spreadsheet, sheet "No. of children dewormed." We start with this total cost figure and apply adjustments in our cost-effectiveness analysis to account for cases where we believe the charity's funds have caused other actors to shift funds from a less cost-effective use to a more cost-effective use ("leverage") or from a more cost-effective use to a less cost-effective use ("funging"). We note that Deworm the World shared cost per treatment analyses for 2016 MDAs in Cross River, Nigeria and Vietnam, for which we have not yet made the adjustments needed to make the figures comparable to our cost per treatment estimates for Kenya and India. This is in part because we do not have all of the same monitoring information for Cross River and Vietnam. Vietnam accounted for only 1% of Deworm the World's recent spending (see above) and no scale-up of work is planned there.96 Deworm the World expects to spend a significant portion of its budget in Nigeria in the future97 and completing a cost per treatment analysis for Deworm the World's prior work in Nigeria will be a priority for us in 2018. Is there room for more funding? We believe that Deworm the World could productively use more funding than it expects to receive and that it is very likely to be constrained by funding. In short: Total opportunities to spend funds productively: Deworm the World has identified opportunities to maintain and expand its work over the next three years (2018-2020) totaling $23.5 million. It may be able to use funding beyond this amount to (a) make longer-term commitments (four or five years instead of three); (b) take advantage of opportunities that have not been identified yet; (c) protect against loss of funding for programs currently supported by restricted grants (and not included in the $23.5 million figure) and cost overruns. Cash on hand: As of the end of June 2017, Deworm the World held $0.4 million in uncommitted funds. It also holds $17.5 million that has been committed to projects and $2.7 million in reserves, i.e. funds that are set aside only for use in unexpected circumstances. Expected additional funding: We estimate that Deworm the World will receive an additional $2.8 million in unrestricted funding for its work in 2018, and about $0.6 million in each of the following two years. Additional considerations: Prior to 2017, we found errors in Deworm the World's financial statements that reduced our confidence that we had a complete, accurate understanding of Deworm the World's financial situation. Evidence Action has taken steps to improve its financial data, including that for Deworm the World, and we did not find any cause for concern in its 2017 financial reports. Below, we also discuss two issues that are material to a determination of Deworm the World’s room for more funding: the possibility of alternative funders of its work in Kenya and its relationship with Evidence Action. In sum, we expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive over the next three years. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities. December 2017 update: In November 2017, we recommended that Good Ventures give $15.2 million to Evidence Action’s Deworm the World Initiative (including $5.5 million to build its reserves). We also recommended that GiveWell’s Board of Directors grant $0.7 million in discretionary funds. After accounting for these grants, we estimate that Deworm the World has a remaining funding gap of about $8.5 million for its deworming programs. Available and expected funds At the end of June 2017, Deworm the World held $20.6 million, of which:98 $2.1 million was restricted by donors to specific programs $15.3 million was unrestricted and committed to projects $2.7 million was set aside for reserves $0.4 million was uncommitted We expect that Deworm the World will receive additional donations over the next three years from the following sources. Note that we have excluded funding that is restricted to specific programs because this amount is excluded from both the available funds calculation and the list of spending opportunities. Donors who are not influenced by GiveWell's research: Based on what it has received in the past year from donors we do not attribute to GiveWell's recommendation, we project that Deworm the World will receive roughly $0.6 million in unrestricted funding per year that is independent of GiveWell's recommendation.99 Donors who give based on GiveWell's top charity list, but do not follow our recommendation for marginal funding: GiveWell maintains both a list of all top charities that meet our criteria and a recommendation for which charity or charities to give to in order to maximize the impact of additional donations, given the cost-effectiveness of remaining funding gaps. We estimate that Deworm the World will receive about $1.7 million over the next year from donors who use our top charity list but don't follow our recommendation for marginal donations.100 We include only one year of funding of this type in order not to make the assumption, for the purpose of the room for more funding analysis, that charities will continue to be recommended and receive this funding. This is because we aim to structure GiveWell-directed funding such that, if we were to stop recommending the organization next year, it would have one to two years of funding available to allow it to adjust to the loss in funding. Donors who follow GiveWell's recommendation for marginal donations: Our estimate of room for more funding is used to make a recommendation to these donors. In sum, we estimate that, including uncommitted funds on hand and funds held by GiveWell for supporting Deworm the World, Deworm the World will have about $4.5 million in unrestricted funding for 2018-2020.101 Uses of additional funding Deworm the World is seeking funding to continue programs in Kenya and India and believes it may have opportunities to expand its work in India, Nigeria, and Pakistan. We ask top charities to consider GiveWell-directed funds to be multi-year grants. The amount of GiveWell-directed funding that a top charity receives can vary greatly from year to year, and spending the funds over two to three years can help smooth these fluctuations. Details of how Deworm the World expects to spend additional funding are in this spreadsheet, sheet "Spending opportunities." In short: Nigeria: Deworm the World has previously made three-year commitments to fund deworming in Cross River, Oyo, and Ogun states and has set aside funding for three years of work in Rivers state. It is seeking additional funding for three types of work in Nigeria, totaling $8.5 million: (a) expansion to two additional states; (b) conducting MDAs in local government areas that are missed by other programs in other states; and (c) technical support for the Federal Ministry of Health. Pakistan: Deworm the World has been exploring the possibility of funding deworming in Pakistan for several years. It is seeking $8.4 million to start and scale up this program in 2018-2020 and would prefer to secure funding now for 2021-2022 to ensure the stability of the program.102 India: Deworm the World is seeking $3.4 million for expanding to an additional state, covering costs in existing states that are outside of the grant agreements that fund other costs in those states, and conducting monitoring in states it does not provide technical assistance to in order to better understand program performance and improve its work at the national level. Kenya: Deworm the World has worked in Kenya for over 5 years, primarily with funding from the Children's Investment Fund Foundation (CIFF), which has not been renewed. Last year, GiveWell-directed funding provided Deworm the World with half of the funding it needed to fund the program for a further four years (with the expectation that the END Fund would fund the other half, $1.25 million per year). In mid-2017, GiveWell granted $2.25 million to Deworm the World, of which it allocated $2 million to Kenya.103 Deworm the World raised $0.5 million from the END Fund and the END Fund will consider renewing this funding annually. In 2017, Deworm the World signed an agreement with the government of Kenya to fund the program through 2022.104 Taking into account the possibility of funding from the END Fund, we estimate that Deworm the World needs an additional $1.7 million to fund the program through 2020.105 It is also seeking funds to cover the remaining funding gap for 2021-2022. Global team costs: $1.5 million for personnel, travel, office costs, and other miscellaneous costs in 2019 and 2020. The total of the above is $23.5 million. In addition, Deworm the World would like to increase its reserves. It has drawn down some of its reserves in the past year and, as it grows, aims to hold more funding in reserve to protect against underestimation of costs and overestimation of revenue from other sources. In particular, there are some spending opportunities that we have not included above because they have, to date, been funded by restricted grants. Continuation of these grants is not confirmed. Deworm the World would like to have reserves available to ensure that it can continue this work if restricted funding is not renewed.106 We believe that there is some risk that Deworm the World will encounter limitations to its ability to scale, other than funding. These risks are lowest in cases where it is continuing existing programs and greatest in cases where it would be starting new programs, particularly in Pakistan, where it has spent little funding to date. For example, Deworm the World could find that funding needs in some locations have been met by other funders,107 that it is difficult to hire qualified staff or find a strong implementing partner, or that governments are not interested in Deworm the World's support. Considerations relevant to assessing Deworm the World’s room for more funding Quality of financial information In 2016, we spent more time than we had previously on trying to understand Evidence Action's financials, and we found several errors (details in footnote).108 Once these mistakes had been corrected, Deworm the World's records indicated that it held approximately $4.6 million more in available unrestricted funding than the original financial documents had shown.109 In 2017, Evidence Action requested $2.6 million through GiveWell Incubation Grants to strengthen its operations, including its accounting team and systems. Good Ventures, on GiveWell's recommendation, made this grant in April 2017. In June 2017, Evidence Action hired a CFO. Our impression from speaking with the CFO is that he is continuing the cleanup of past accounting records and instituting new systems going forward to improve the quality of Evidence Action's financial information.110 Note that we discussed with Evidence Action whether changes identified as part of this process would affect Deworm the World’s cost per treatment analysis; with one small exception, Evidence Action told us that the cost per treatment would not be affected as the total amount spent on deworming did not change, only the allocation to specific deworming grants.111 We did not find errors in the financial information Evidence Action provided in 2017. Alternative funders for Kenya Deworm the World is seeking funding to support the next four years (mid-2018 through mid-2022) of a national deworming program in Kenya. Deworm the World previously had a five-year grant from CIFF (and a smaller amount from the END Fund) to support the program; the CIFF grant ended in 2017.112 CIFF has not renewed funding to Deworm the World, but remains interested in the possibility of funding deworming in Kenya in the future. It is unclear to us whether CIFF, in the absence of Deworm the World providing funding to the Kenya school-based deworming program, would fund this work or similar work. Therefore, there is some uncertainty around how much GiveWell-directed funding is needed to support the Kenya program. Deworm the World notes that the government of Kenya has requested its support in fundraising for the next four years of the program.113 In 2016, we projected that the END Fund would fund half of the Kenya program ($1.25 million per year). In 2017, the END Fund indicated its intent to provide $0.5 million for 2018 and expressed its interest in providing $0.5 million per year for 2019 and 2020, if it has sufficient funding to do so.114 We have accounted for this funding (with a discount for uncertainty) in the list of spending opportunities discussed above. Implications of Deworm the World Initiative being part of a larger organization Deworm the World Initiative is a program of Evidence Action. Evidence Action supports other programs in addition to Deworm the World, such as Dispensers for Safe Water, a GiveWell standout charity, and No Lean Season, a GiveWell top charity.115 This has some implications relevant to Deworm the World’s room for more funding: donations to Evidence Action, even if restricted to Deworm the World, might change the actions that staff take to fundraise (i.e., which grants they pursue, what type of funding they ask for). We've seen some evidence that this has been the case in the past and provide details on our page with additional information about Deworm the World. Deworm the World also notes that many of Evidence Action's investments in general organizational development have benefited Deworm the World, as well as Evidence Action's other programs.116 In our 2016 review of Deworm the World, we noted that we expected Evidence Action to allocate some unrestricted funding to Deworm the World in 2017. In September 2017, Evidence Action told us that it did not allocate any unrestricted funding to Deworm the World in 2017 and did not expect to allocate any of its unrestricted funding to Deworm the World in the near future. Evidence Action noted:117 At the time of submitting the 2016 report [to GiveWell], it was anticipated that Evidence Action would allocate unrestricted funds to Deworm the World Initiative in both 2016 and 2017. As greater visibility has been obtained into the organizational financial position [as part of the accounting clean up process], it became evident that the level of available and uncommitted unrestricted funding is very low—at the end of June 2017, unrestricted funds covered approximately 4 weeks of the organization's annual expenses. [...] For 2016 and 2017, given the low unrestricted funding levels, the organization sought to preserve these funds to serve as a minimal organizational backstop, and use these funds only for programs with significant and acute gaps. Global need for treatment There appears to be a substantial unmet need for STH and schistosomiasis treatment globally. In 2017, the World Health Organization (WHO) released a report on 2016 treatments stating that:118 69% of school-age children in need of treatment were treated for STH in 2016, up from 63% in 2015 and 45% in 2014. Coverage was 65% in African countries in 2016. 52% of school-age children in need of treatment were treated for schistosomiasis in 2016, up from 42% in 2015. We have not vetted this data. Evidence Action as an organization We believe that Evidence Action is a strong organization: Track record: Evidence Action's Deworm the World Initiative has a track record of assisting governments with deworming programs. Self-evaluation: Deworm the World collects a large amount of relevant data about its programs, demonstrating a commitment to self-evaluation. Communication: Evidence Action's Deworm the World staff have generally communicated clearly and directly with us, given thoughtful answers to our critical questions, and shared significant, substantive information. Transparency: Evidence Action has consistently been strong in its commitment to transparency. It has provided the information we’ve asked for and has not hesitated to share it publicly (unless it had what we felt was a good reason). We wrote more on how we think about evaluating organizations in our 2012 blog post. Sources Document Source Alderman et al. 2006 Source (archive) Alix Zwane conversation August 30th 2013 Unpublished Alix Zwane conversation June 4th 2013 Source Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013 Unpublished Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 Unpublished Allen and Parker 2011 Source (archive) Assam 2010 guidelines for deworming Source Assam midday meal report 2013 Source (archive) Assam reproductive and child health 2011-2012 Source (archive) Assam state programme implementation plan 2011-2012 Source (archive) Baird et al 2012 Source Bleakley 2007 Source (archive) CIFF conversation September 10th 2013 Source Croke 2014 Source (archive) Devesh Kapur conversation October 14th 2013 Source Deworm the World 2015 Uttar Pradesh prevalence survey report Source Deworm the World and SCI, Ethiopia coverage survey Source Deworm the World Chhattisgarh Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World External Bihar 2016 Costing Model Source Deworm the World External Chhattisgarh 2016 Costing Model Source Deworm the World External Cross River Nigeria 2016 Costing Model Source Deworm the World External Kenya 2016 Costing Model Source Deworm the World External Madhya Pradesh 2016 Costing Model Source Deworm the World External Rajasthan 2016 Costing Model Source Deworm the World External Telangana 2016 Costing Model Source Deworm the World External Tripura 2016 Costing Model Source Deworm the World External Uttar Pradesh 2016 Costing Model Source Deworm the World Financials 2017 Source Deworm the World Jharkhand Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Kenya Year 4 Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Nigeria Cross River Independent Process Monitoring Report, November 2016 Source Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Unpublished Deworm the World Telangana Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Tripura Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Uttar Pradesh Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Vietnam Independent Monitoring Report, November-December 2016 Source Deworm the World, 2015 expense summary Unpublished Deworm the World, 2015 expense summary - by funder Unpublished Deworm the World, Bihar 2016 IMCV report Source Deworm the World, Chhattisgarh 2016 IMCV report Source Deworm the World, Chhattisgarh prevalence survey report, August 2016 Source Deworm the World, Ethiopia independent monitoring report, Year 1 Source Deworm the World, Ethiopia prevalence survey report Source Deworm the World, Ethiopia Workplan Unpublished Deworm the World, Jharkhand Independent Monitoring Report 2016 Unpublished Deworm the World, Kenya 2014-2015 program report Source Deworm the World, Kenya Coverage Reporting data, Year 3 Source Deworm the World, Kenya Deworming Day data, Year 3 Source Deworm the World, Kenya Narrative Report - Year 1 Source Deworm the World, Kenya Narrative Report - Year 2, Quarter 4 Source Deworm the World, Kenya Narrative Report - Year 3, Quarter 3 Source Deworm the World, Kenya process monitoring report, Year 4 Source Deworm the World, Kenya Year 2, DD - Main instrument Source Deworm the World, Kenya Year 2, Pre DD - School instrument Source Deworm the World, Kenya Year 3, DD - Main instrument Source Deworm the World, Kenya Year 3, Post DD - Coverage instrument Source Deworm the World, Kenya Year 3, Pre TT form Source Deworm the World, Madhya Pradesh 2015 prevalence survey report Source Deworm the World, Madhya Pradesh 2016 IMCV report Source Deworm the World, Madhya Pradesh cost-per-treatment - 2015 Unpublished Deworm the World, National Deworming Day states, August 2016 Source Deworm the World, Rajasthan 2016 IMCV report Source Deworm the World, target populations and parasitology data Source Deworm the World, Telangana 2016 IMCV report Source Deworm the World, Tripura 2016 IMCV report Source Deworm the World, Uttar Pradesh 2016 IMCV report Source Deworm the World, Uttar Pradesh prevalence survey report, December 2015 Source Deworm the World, Vietnam 2016 monitoring survey form for coverage validation Source Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day Source Deworm the World, Vietnam baseline prevalence survey - 4 provinces Source Deworm the World, Vietnam final report for STH survey in 21 provinces Source Deworm the World, Vietnam Independent Monitoring Report, 2016 Source DSW 2012 GiveWell site visit Source DtWI 2013 GiveWell government interviews Source DtWI 2013 GiveWell site visit Source DtWI Assam research 2013 Source DtWI Bihar 2011 cost data Source DtWI Bihar 2011 coverage data Source DtWI Bihar 2011 monitoring data for deworming day Source DtWI Bihar 2011 monitoring data for mop-up day Source DtWI Bihar 2011 Monitoring Form for Deworming Day Source DtWI Bihar 2011 monitoring form for mop-up day Source DtWI Bihar 2011 prevalence survey report Source DtWI Bihar 2011 program report Source DtWI Bihar 2012 cost data Source DtWI Bihar 2012 cost data details Unpublished DtWI Bihar 2012 coverage data Source DtWI Bihar 2012 monitoring data for coverage validation Source DtWI Bihar 2012 monitoring report Source DtWI Bihar 2014 cost data Unpublished DtWI Bihar 2014 program report Source DtWI Bihar 2014 program report annex 1 Source DtWI Bihar 2014 program report annex 2 Source DtWI Bihar 2015 independent monitoring tables Source DtWI Bihar 2015 monitoring data for coverage validation, schools Source DtWI Bihar 2015 monitoring data from deworming day, schools Source DtWI Bihar 2015 monitoring data from mopup day, schools Source DtWI Bihar 2015 monitoring survey for coverage validation, schools Source DtWI Bihar 2015 monitoring survey from deworming day, schools Source DtWI Bihar 2015 monitoring survey from mopup day, schools Source DtWI Bihar 2015 Prevalence Survey report Source DtWI Bihar 2015 Program report Source DtWI Bihar Feb 2016 Program report Source DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015 Unpublished DtWI Chhattisgarh 2015 coverage validation report Source DtWI Chhattisgarh 2015 coverage validation tables Unpublished DtWI Chhattisgarh 2015 independent monitoring tables Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools Source DtWI Chhattisgarh Feb 2016 Program report Source DtWI class register audits 2013 Source DtWI cost narrative 2013 Source DtWI Cost per treatment blog post January 2015 Source (archive) DtWI cost per treatment summary 2013 Source DtWI coverage data 2013 - 2014 Source DtWI Delhi 2012 cost data Source DtWI Delhi 2012 coverage data by anganwadi Source DtWI Delhi 2012 coverage data by school Source DtWI Delhi 2012 coverage report Source DtWI Delhi 2012 monitoring data Source DtWI Delhi 2012 monitoring form deworming day Source DtWI Delhi 2012 prevalence survey design Source DtWI Delhi 2012 prevalence survey report Source DtWI Delhi 2012 program report Source DtWI Delhi 2013 cost data Unpublished DtWI Delhi 2013 program report Source DtWI Delhi 2015 independent monitoring tables Source DtWI Delhi 2015 monitoring data for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring data for coverage validation, schools Source DtWI Delhi 2015 monitoring data from deworming day, anganwadis Source DtWI Delhi 2015 monitoring data from deworming day, schools Source DtWI Delhi 2015 monitoring data from mopup day, anganwadis Source DtWI Delhi 2015 monitoring data from mopup day, schools Source DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring survey for coverage validation, schools Source DtWI Delhi 2015 monitoring survey from deworming day, anganwadis Source DtWI Delhi 2015 monitoring survey from deworming day, schools Source DtWI Delhi 2015 monitoring survey from mopup day, anganwadis Source DtWI Delhi 2015 monitoring survey from mopup day, schools Source DtWI Delhi 2015 program report Source DtWI External Vietnam Costing Model Source DtWI Kenya 2013-2014 cost per treatment data Source DtWI Kenya 2013-2014 program report Source DtWI Kenya 2015-16 program report Source DtWI Madhya Pradesh 2015 coverage validation form Unpublished DtWI Madhya Pradesh 2015 deworming day monitoring form Unpublished DtWI Madhya Pradesh 2015 independent monitoring tables Source DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools Source DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools Source DtWI Madhya Pradesh 2015 program report Source DtWI Madhya Pradesh Feb 2016 program report Source DtWI Monitoring Improvements 2014 Source DtWI NDD blog post February 2015 Source (archive) DtWI NDD Year 1 M&E review July 2015 Source DtWI Rajasthan 2012 cost data Source DtWI Rajasthan 2012 cost data details Unpublished DtWI Rajasthan 2012 coverage data for anganwadi Source DtWI Rajasthan 2012 coverage data for schools Source DtWI Rajasthan 2012 monitoring data for coverage validation in anganwadis Source DtWI Rajasthan 2012 monitoring data for coverage validation in schools Source DtWI Rajasthan 2012 monitoring form coverage day Source DtWI Rajasthan 2012 monitoring form pre-deworming day Source DtWI Rajasthan 2012 monitoring report Source DtWI Rajasthan 2012 prevalence survey report Source DtWI Rajasthan 2013 cost data Unpublished DtWI Rajasthan 2013 prevalence survey report Source DtWI Rajasthan 2013 program report Source DtWI Rajasthan 2015 independent monitoring tables Source DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring data for coverage validation, schools Source DtWI Rajasthan 2015 monitoring data from deworming day, schools Source DtWI Rajasthan 2015 monitoring data from mopup day, schools Source DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring survey for coverage validation, schools Source DtWI Rajasthan 2015 monitoring survey from deworming day, schools Source DtWI Rajasthan 2015 monitoring survey from mopup day, schools Source DtWI Rajasthan 2015 program report Source DtWI Rajasthan Feb 2016 Program report Source DtWI Telangana Feb 2016 Program report Source DtWI Uttar Pradesh Feb 2016 Program report Source Evidence Action 2014 budget Unpublished Evidence Action 2015 draft budget Unpublished Evidence Action 2015 funding gap analysis Source Evidence Action 2016-2017 Revenue Report Source Evidence Action 2017 financials, GiveWell submission Source Evidence Action cover letter 2013 Source Evidence Action launch announcement 2013 Source (archive) Evidence Action Q1 financials, 2016 Unpublished Evidence Action website 2013 Source (archive) Evidence Action website announcement April 2014 Source (archive) Evidence Action website, Deworm the World Initiative (December 2016) Source (archive) Evidence Action website, Deworm the World Initiative (March 2016) Source (archive) Evidence Action website, Deworm the World Initiative (October 2015) Source (archive) Evidence Action website, Evidence Action Beta (October 2015) Source (archive) Evidence Action website, Who we are (November 2016) Source (archive) Evidence Action, 2015 financials by program Unpublished Evidence Action, Baseline Survey Report of Soil-Transmitted Helminths Prevalence in Pakistan Source Evidence Action, blog post, April 27, 2016 Source (archive) Evidence Action, blog post, August 1, 2016 Source Evidence Action, blog post, December 21, 2015 Source (archive) Evidence Action, blog post, January 16, 2015 Source (archive) Evidence Action, blog post, January 8, 2015 Source (archive) Evidence Action, blog post, July 5, 2016 Source (archive) Evidence Action, blog post, June 12, 2015 Source (archive) Evidence Action, blog post, June 30, 2016 Source (archive) Evidence Action, Projected allocation of unrestricted funds, 2016 Source GiveWell analysis of Deworm the World 2014 Financial summary Source GiveWell analysis of Deworm the World cost per treatment Source GiveWell analysis of Deworm the World cost-per-treatment, 2016 Source GiveWell analysis of Deworm the World cost-per-treatment, October 2016 Source GiveWell analysis of Deworm the World financials - 2016 Source GiveWell DtWI 2013-2014 cost data summary Source GiveWell enrollment-based student coverage check 2015 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013 Source GiveWell's non-verbatim summary of a conversation with END Fund staff, October 17, 2016 Source GiveWell's notes from site visit to India, October 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014 Source GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014 Source GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 Source Global Alliance to Eliminate Lymphatic Filariasis - Prevention Source (archive) Grace Hollister conversation June 19th 2013 Source Grace Hollister, conversation with GiveWell, April 20, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 11, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 24, 2016 Unpublished Grace Hollister, conversation with GiveWell, July 25, 2016 Unpublished Grace Hollister, conversation with GiveWell, June 13, 2016 Unpublished Grace Hollister, conversation with GiveWell, May 5, 2016 Unpublished Grace Hollister, conversation with GiveWell, September 1, 2016 Unpublished Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Unpublished Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, March 23, 2015 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 Source Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 Unpublished Grace Hollister, edits to GiveWell's review, November 20, 2016 Unpublished Grace Hollister, edits to GiveWell's review, November 7, 2016 Unpublished Grace Hollister, email to GiveWell, June 9, 2016 Unpublished Grace Hollister, email to GiveWell, March 6, 2016 Unpublished Grace Hollister, email to GiveWell, September 13, 2016 Unpublished Harvard Business School Kenya Case Study A 2010 Unpublished India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015 Source India NDD documents 2015 Source (archive) Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014 Unpublished JPAL CEAs in education 2011 Source Kabatereine et al. 2001 Source (archive) KEMRI prevalence report - Year 2 Unpublished Kenya National School Based Deworming Programme Year 4 2016 Impact Analysis Source LF treatment coverage (2004-2014) Source (archive) Miguel and Kremer 2004 Source Mwandawiro et al. 2013 Source (archive) National Vector Borne Disease Control Programme, All About Lymphatic Filariasis Source (archive) Neetu Chandra Sharma, Daily Mail - India article, August 8, 2016 Source (archive) Nigerian Tribune, "Panic in Ogun schools over deworm exercise," December 2017 Source (archive) Paul Byatta, attachments to email to GiveWell, September 23, 2016 Source Paul Byatta, conversation with GiveWell, September 20, 2016 Unpublished Paul Monaghan, conversation with GiveWell, September 8, 2016 Unpublished Preventive chemotherapy in human helminthiasis 2006 Source (archive) Professor Devesh Kapur Biography 2013 Source (archive) Reserve Bank of India, GDP per capita, Table 10, September 16, 2015 Source (archive) SCI Malawi coverage survey 2012 Source STH coalition framework for action November 2014 Source (archive) U-DISE Elementary Thematic Maps 2015 Source (archive) U-DISE Secondary Flash Statistics 2015 Source (archive) U-DISE Secondary Thematic Maps 2015 Source (archive) WHO soil-transmitted helminthiases 2012 Source (archive) WHO STH factsheet Source (archive) WHO STH treatment report Source (archive) WHO Weekly epidemiological record, 18 December 2015 Source WHO Weekly epidemiological record, 6 March 2015 Source (archive) WHO, Helminth control in school-age children Source WHO, Helminth control in school-age children second edition Source WHO, Summary of global update on preventive chemotherapy implementation in 2015 Source WHO, Summary of global update on preventive chemotherapy implementation in 2016 Source World Schistosomiasis Risk Chart 2012 Source 1 GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013 Alix Zwane conversation June 4th 2013 Grace Hollister conversation June 19th 2013 Alix Zwane conversation August 30th 2013 Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014 GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014 GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015 GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 In 2016 and through most of 2017, we deprioritized publishing notes from our conversations with Deworm the World. GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017 2 CIFF conversation September 10th 2013. We have also had several additional conversations with CIFF about its work with Deworm the World. 3 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. 4 "More specifically, Evidence Action advocates for school-based deworming to policymakers and provides technical assistance to launch, strengthen and sustain school-based deworming programs." Evidence Action website, Deworm the World Initiative (March 2016) An example of Deworm the World supporting a program with funding: "Deworm the World is working in partnership with Thrive Networks in Vietnam on an integrated program of both deworming and water, sanitation, and hygiene (WASH) education, and this includes an RCT to explore the impact of hygiene education in combination with deworming. This program is funded primarily by Dubai Cares, with Deworm the World slated to provide unrestricted funding for certain program components." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 5. 5 "Is mass treatment justified? On cost-effectiveness grounds we believe that it clearly is, as the cost of treatment is cheaper than individual screening. The WHO states that the cost of screening is four to ten times that of the treatment itself. Because the drugs are very safe and has [sic] no side effects for the uninfected, the WHO does not recommend individual screening. The WHO instead recommends mass drug administration in areas where more than 20% of children are infected." Evidence Action website, Deworm the World Initiative (March 2016) Deworm the World focuses on school-based programs because the highest burdens for STH and schistosomiasis (the two diseases that Deworm the World targets) tend to be observed in children. For this reason, a significant decrease in the worm burden in children for these diseases translates to a significant decrease in the burden across an entire community. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 6 "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis. Deworm the World is one of the only organizations focused on STH (while there are other programs that focus on schistosomiasis), and there is significant room to scale-up [sic] STH programs. In some ways, schistosomiasis has an even larger gap to fill than STH. If Deworm the World shifted its focus to include schistosomiasis, it might widen the existing STH gap. Additionally, Deworm the World specializes in school-based deworming, which is not the ideal approach in every situation." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5. This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. 7 "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5. 8 "We work with governments around the world to eliminate the public health threat of worms through scaling up school-based mass deworming programs." Evidence Action website, Deworm the World Initiative (December 2016) For example, Deworm the World is partnering, or planning to partner, with local organizations in Pakistan, Vietnam, and Nigeria: Nigeria: "Deworm the World is in discussions with a potential partner which plans to work in Cross River on other integrated NTD treatment. The need to scale up treatment for schistosomiasis and STH among school-age children has not yet been addressed, and Deworm the World has been in discussions with this partner, the state NTD coordinator, and other state officials about creating a school-based deworming program to treat both STH and schistosomiasis beginning in 2016." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, pg. 6. Vietnam: "[In Vietnam] Deworm the World’s partnership with both the government and Thrive Networks is a new working model for Deworm the World; elsewhere, it has supported government implementation or run its program independently. Deworm the World has only one staff member in the country. Dubai Cares provides most of the program funding. All funding goes to Thrive Networks, which provides money to the government for implementation expenses." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 3. Pakistan: "[In Pakistan, Deworm the World] plans to contract with a local organization to do these [prevalence] surveys, but does not yet have a signed agreement...Deworm the World plans to work in partnership with the same local organization to provide technical support, likely beginning in Punjab and later expanding to Sindh." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. 9 For example, Deworm the World has considered supporting evaluations or monitoring of different deworming-related programs: "If funding permits, the Kenyan government may begin a lymphatic filariasis treatment program along its coast. If it does so, Deworm the World will provide process monitoring and coverage validation for the treatments." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 3. We learned in later conversations that Deworm the World received a grant from the END Fund to work on lymphatic filariasis treatment in the coastal region. "We are contributing approx $111k to the TUMIKIA and TakeUp studies, complementary studies leveraging the Kenya program to look at the potential for breaking STH transmission." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Deworm the World has told us that it is also interested in opportunistically evaluating new evidence-based programs that may efficiently complement deworming. GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014. We discussed this possibility again with Deworm the World in 2017. In 2017, Deworm the World sought funding "to be able to conduct independent monitoring and community evaluation surveys in states [in India] where we do not provide technical assistance, to better understand processes and program quality in those areas and enable improved support from the central level. This funding would enable those activities in three states." See this spreadsheet, sheet "Spending opportunities." 10 "In January 2007, the [Young Global Leaders] launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs." Harvard Business School Kenya Case Study A 2010, Pg 7. 11 Deworm the World has supported deworming activities in India since 2009, in Kenya since 2012, and in Ethiopia since 2014: "[Where We Work, Bihar State, India]: 1st deworming round in 2011 reached 17 million children." Evidence Action website, Deworm the World Initiative (March 2016) "[Where We Work, Kenya] With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively." Evidence Action website, Deworm the World Initiative (March 2016) Deworm the World supported a pilot deworming program in Ethiopia in April 2015 and another deworming program in October and November of 2015. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Deworm the World supported treatments in Cross River, Nigeria in mid 2016: "This week, the Cross River State Ministry of Health’s Neglected Tropical Diseases (NTD) unit launched its inaugural statewide school-based deworming exercise that will treat against two neglected tropical diseases that are particularly common in children: schistosomiasis and soil-transmitted helminthiasis (STH). The school-based deworming exercise will cover 11 of the 18 local government areas in Cross River for the first time, and is targeting 600,000 at-risk school-aged children in primary and junior secondary public and private schools. Other NTDs endemic to the state (lymphatic filariasis and onchocerciasis) will be treated through a community-based approach, according to standard practice." Evidence Action, blog post, June 30, 2016 Deworm the World supported treatments in Vietnam in April and November 2016. DtWI External Vietnam Costing Model Deworm the World supported a prevalence survey in Pakistan between August and December of 2016 (see Evidence Action, Baseline Survey Report of Soil-Transmitted Helminths Prevalence in Pakistan). In 2017, Deworm the World told us it was seeking funding to implement deworming in Pakistan starting in 2018. See this spreadsheet, sheet "Spending opportunities." 12 "The government is fully responsible for program implementation, and … these programs leverage thousands of govt personnel from health and education to be able to run." Grace Hollister, email to GiveWell, June 9, 2016 13 "Note that there are not single deworming days in all countries, nor do all countries have a mop-up day. In all cases there are mass campaigns, but the structure varies by country." Grace Hollister, email to GiveWell, June 9, 2016 14 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. For example, Deworm the World has assisted in a number of areas in India: "Andhra Pradesh... Deworm the World’s contributions Prevalence survey... Operational support Helped government develop operational plans and budgets Coordinated cross-sectoral partners through the establishment of a State School Health Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS) Coordinated drug donation made by Feed the Children Designed a monitoring and evaluation (M&E) system Created government tableau for community awareness Trainings Conducted a master training session for program Designed training cascade for the master trainees to train the rest of the implementers Designed training materials Developed materials and campaigns for community sensitization Bihar… Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andhra Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO… Delhi… In addition to the standard contributions (see Andhra Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children. Rajasthan… DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO." @Grace Hollister Conversation June 19th 2013@, Pg 1-4. 15 We have matched our descriptions to Deworm the World's standard categorization, albeit in a slightly different order. An archived version of the link is here. 16 "As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program." @Alix Zwane Conversation June 4th 2013@, Pg 2. "Deworm the World does not yet have an agreement with the government in Pakistan to conduct the surveys, but hopes to accomplish this in the next month, and anticipates that the prevalence surveys will be conducted beginning in January or February of 2016. It is expected that a clear articulation of need will be an important factor in building a strong case to the government in favor of deworming programs, and it may be best to wait until the results of the surveys are available in the second quarter of 2016 before beginning discussions with the government on a scaled school-based program. Treatment may not begin until 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. Deworm the World's advocacy often occurs side-by-side with Deworm the World's technical assistance; once Deworm the World proves that a deworming program can be well-executed, it is easier to interest national governments in funding deworming programs. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 17 For example: "DtW has been involved in deworming programs in four different states, and is still actively involved of [sic] three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement...In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. [Note: these notes contain additional examples of Deworm the World's advocacy for other states in India] "Together, we suggested to the Federal Ministry of Health that they expand this initial work. What would it take to treat at least 75% of all at-risk school-age children in the country and to launch a truly national program? SCI helped Oumer Shafi, the committed and action-oriented Coordinator for Neglected Tropical Diseases in the Federal Ministry of Health, develop a detailed action plan. This entailed sophisticated statistical analysis to determine how many deworming sites would be required to reach at least 80% of kids at risk. Meanwhile, I worked closely with Birhan Mengistu, an up-and-coming leader seconded from the World Health Organization, and with other Ministry of Health staff. We sat for hours hunched over laptop screens to develop detailed five-year budgets, talking through row after row of spreadsheets and reviewing everything from the cost of fuel for drug transport to the needs of teachers. Together with the Federal Ministry of Health, we were able to think and act boldly. We are excited to continue to partner with SCI and are seeking other partners who also share common goals and values to rapidly scale school-based deworming in endemic countries. ...When we floated the idea of vastly increasing the scope of the originally proposed deworming rounds to be a truly national plan treating upwards of 75% of all children at-risk, Shafi didn’t flinch." Evidence Action, blog post, June 12, 2015 Other advocacy activities can include discussing: "how deworming can fit into the current policy environment and policy priorities of a government, how such a program can/should be financed, the robust evidence of impact, how a country can best take advantage of WHO drug donations, encourage program champions within government, help establish program governance structures. Once a program is established, advocacy doesn’t end – we work with govts to ensure the continuation of the above. Typically we refer to this group of activities as policy and advocacy, because there is a heavy emphasis on the former." Grace Hollister, email to GiveWell, June 9, 2016 These discussions also provide opportunities for Deworm the World to assess how well a deworming program with the government might run. If Deworm the World discovered from its advocacy discussions that there were high rates of teacher or student absenteeism, then it might conclude that a school-based deworming program may not work in the country. Deworm the World assesses risks like this through a diagnostic survey of the country’s capacity, including school attendance rates, which must be sufficiently high if a school-based deworming program is to succeed. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 18 Deworm the World builds strong working relationships with governments to try to ensure that its programs will be effective, and it will not commit to a program if it does not foresee success in that country. It can decide to abandon plans for a program before a memorandum of understanding (MoU) is signed. In one state in India, Jharkhand, Deworm the World explored a program, but did not build a strong working relationship with the government, so Deworm the World pulled out of discussions before discussing an MoU or investing much money. The discussion stage with governments is important for helping Deworm the World assess the government's position and viability as a partner. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 19 "The global STH community has changed significantly in the last couple of years, especially due to the formation of the STH Coalition. The community is now prioritizing STH (in a way similar to how LF became prioritized with the formation of the Global Alliance to Eliminate Lymphatic Filariasis, which has seen significant success). The STH community is developing plans to scale-up [sic] treatment, especially in high burden countries such as: Ethiopia Nigeria India Pakistan The Democratic Republic of the Congo Indonesia The Philippines Tanzania As part of the STH Coalition, Evidence Action is chairing a working group on school-age children. It has used some of its unrestricted funding to hire consultants to create ‘snapshots’ of each country, including obstacles, gaps, potential strategies, and financial needs. Deworm the World expects to see an increase in partnerships between the various groups in the STH community. Deworm the World hopes to leverage partnerships with existing organizations in, e.g., Nigeria, Pakistan, Ethiopia, etc., to provide catalytic support (rather than opening its own offices in those places)." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 6. 20 "We work with epidemiologists and local partners to assess worm prevalence and intensity, obtaining data to develop a targeted treatment strategy and to determine parasitological impact once programs are in place." Evidence Action website, Deworm the World Initiative (March 2016) A few examples of Deworm the World assisting with prevalence surveys include: "Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children." Grace Hollister conversation June 19th 2013, Pg 1. "DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged [sic] children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment." Grace Hollister conversation June 19th 2013, Pg 2. "In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city." Grace Hollister conversation June 19th 2013, Pg 3. "DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year." Grace Hollister conversation June 19th 2013, Pg 4. 21 For example, Deworm the World is currently supporting prevalence surveys in Pakistan because Pakistan has not yet been "mapped" (i.e., prevalence surveys have not yet been conducted in Pakistan), so nobody knows how heavy the worm burden is in Pakistan or where deworming efforts should be focused. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Deworm the World has estimated very roughly that its three-year program in Nepal would cost about $6 million ($2 million per year). This is based on a cost per child of no more than $0.50 and a target population of about 6 million children. However, the latter estimate is based on the outdated prevalence survey data mentioned above. Deworm the World will need to conduct a new survey to determine an exact target population and a more accurate budget." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pgs 1-2. "Deworm the World will likely wait to expand its Vietnam activities until further mapping and impact evaluation have been completed." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pg 7. 22 Comments from Deworm the World in response to a draft of this page in October 2017. 23 Deworm the World originally planned to do prevalence surveys every few years but may do them less frequently going forward. "DtWI would like to do prevalence surveys after every 3 years or so. Ideally, prevalence surveys would be carried out after every third round of treatment immediately prior to the following round." GiveWell's notes from site visit to India, October 2013 "Note: this [follow-up prevalence survey] strategy is evolving; WHO recommendations are to conduct sentinel site surveys after 5-6 rounds of treatment, and we are moving in that direction. Key is how a new survey would impact the treatment strategy" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Key M&E activities in India include "post-round 3 impact measurement" prevalence surveys. DtWI NDD Year 1 M&E review July 2015, Pg 3. Deworm the World has said that a Rajasthan follow-up prevalence survey is tentatively planned for late 2017. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Grace Hollister, edits to GiveWell's review, November 7, 2016 24 Deworm the World told us that it works "with partners with expertise in STH parasitology and epidemiology." Grace Hollister, edits to GiveWell's review, November 7, 2016. Examples of Deworm the World working with partners on prevalence surveys: "The WHO reports that Pakistan is endemic for STH, but there is not yet sufficient evidence of prevalence and intensity to develop an evidence-based treatment strategy. Deworm the World has committed unrestricted funding to fund prevalence surveys in two large provinces, Punjab and Sindh. It is targeting these provinces because their school enrollment rates are high, the areas are fairly secure, and they contain a significant percentage of the population of Pakistan. Deworm the World plans to contract with a local organization to do these surveys, but does not yet have a signed agreement." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. "Following three rounds of school-based deworming, Evidence Action - Deworm the World Initiative recommended a second prevalence survey to the Bihar government, in order to understand the effect of deworming in Bihar on STH infection levels. With approvals from the State Government, in January and February 2015, Evidence Action - Deworm the World Initiative, conducted an STH prevalence survey among school-age children in government primary schools in Bihar. The survey took place in 65 schools in 14 districts, covering all three agro-climatic zones in the state. The National Institute of Epidemiology – Chennai (NIE) designed the survey, and analyzed the dataset to produce epidemiological findings. Field teams hired through GfK Mode (an agency with prior experience in sample collection for STH prevalence surveys), visited the households of children in the selected schools to collect stool samples and information related to school, household, deworming, and sanitation, to better understand infection patterns and allow for sample weighting. The 2 Post Graduate Institute of Medical Education and Research – Chandigarh (PGIMER) analyzed stool samples in field laboratories, which were set up in district and block health facilities, using the WHO recommended Kato-Katz method." DtWI Bihar 2015 Prevalence Survey report, Pgs 1-2. 25 Comments from Deworm the World in response to a draft of this page in October 2017. 26 "We work closely with the Ministries of Education and Health to design a program with joint ownership, develop operational plans and budgets, coordinate logistics, and provide on-the-ground support to ensure a high quality outcome." Evidence Action website, Deworm the World Initiative (March 2016) Note that we do not feel like we have a strong understanding of Deworm the World's activities in this area; for example, we have not asked Deworm the World what it has brought to the planning, budgeting, or logistics processes that would not have otherwise been included. We do not have a strong sense from Deworm the World's website about what these activities involve (e.g., we do not know what it means for Deworm the World to have "coordinated logistics"). 27 "We help governments evaluate appropriate drug treatment strategies and dosage, support drug procurement including through global pharmaceutical donation programs, and design robust serious adverse event protocols and drug tracking systems." Evidence Action website, Deworm the World Initiative (December 2016) Grace Hollister, edits to GiveWell's review, November 7, 2016 28 "DtWI provided support to the state government in submitting the drug requisition to WHO in March 2013, as well as in shipping, custom clearances and transportation upon arrival in India." DtWI Rajasthan 2013 program report, Pg 6. Deworm the World told us that before it started conversations with the Indian government, the government was not aware that it could obtain albendazole for free from the World Health Organization. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 29 "We consistently design and support training through an efficient multi-tier cascade approach that is tailored to the local context, ensuring knowledge reaches from the national level all the way to the teachers responsible for administering deworming medication." Evidence Action website, Deworm the World Initiative (March 2016) GiveWell's understanding is that training cascades involve a series of trainings that start at high levels in the government, and proceed in a step-wise fashion down to local levels, where teachers are eventually trained. At each training, materials (such as deworming drugs and posters that notify the community about deworming day) are passed down from the staff member leading the training to the staff members attending the training, until materials eventually reach teachers. A description of a training and distribution cascade: "Transportation of tablets to all districts was managed and supported by DtWI, in coordination with SHS and DHFW to the district level dispensary, from where they were collected by the respective teachers for their school. For anganwadis, the syrups were sent to the nodal officer who arranged further transportation to the supervisors who in turn handed over to the anganwadis. Training about health education on types of STH, need for deworming, transmission of worms through open defecation and other practices and how to safely administer deworming drug was conducted through a cascade model. In the first step of the cascade, training of trainers was conducted at the state level where a total of 1040 participants attended (390 WCD + 650 education department) in 60 sessions. These trainers further trained 3032 headmasters and 3032 teachers and 10,500anganwadi workers in groups of 30 participants. The training on deworming was integrated with training for the WIFS program to effectively utilize time of participants and trainers and reduce training costs. A simplified training manual was developed that included content on deworming and WIFS into a single document. At the state level training sessions, training videos on three types of soil transmitted [sic] helminths (ascaris, trichuris and hookworm) and worm infestation cycle were also used. DtWI hired district coordinators (DCs) to provide short term support to the deworming program. They played a key role in ensuring that drugs in adequate quantities were available with the district-level dispensaries and nodal offices for further distribution. The DCs collated information on shortfall or surplus of drugs at district level and shared information with the DtWI state team, who coordinated with SHS to ensured-distribution [sic] or fresh supplies to districts facing drug deficits." DtWI Delhi 2013 program report, Pg 7. See Pgs. 12-13 of DtWI Delhi 2015 program report for a visual representation and description of a recent training and distribution cascade. 30 For example, a description of the training cascade in Kenya: "The National School-Based Deworming Programme uses a cascade implementation model that efficiently and cost-effectively delivers training materials, deworming tablets, monitoring forms, funds, trainings other programme materials and resources from the national level to schools. At the national level, the Programme trains a team of MoEST and MoH officials as master trainers, requisitions deworming tablets through the MoH, and develops treatment and implementation strategies, training materials and monitoring tools. Thereafter, an initial planning meeting is held with county and sub-county leadership. This meeting is followed by two levels of trainings on how to successfully implement the Deworming Programme: Sub-County Training and Teacher Trainings. These trainings prepare sub-county and division officials to plan subsequent programme activities within the cascade, distribution of materials, planning of deworming and community mobilization and sensitization. After these trainings and community mobilization, the critical day of implementation occurs – Deworming Day – where teachers administer deworming tablets to millions of children in over 11,000 schools across Kenya and fill in monitoring forms to capture treatment data. These forms and any unused deworming tablets are moved up through a “Reverse Cascade” as described below. The cascade model helps to manage the national scale of the NSBDP, and therefore, builds capacity for successful implementation at various levels. Additionally, the cascade brings together MoEST and MoH personnel through collaborative leadership responsibilities for the planning, implementation and monitoring of programme activities at all levels. The cascade is outlined in the infographic below." DtWI Kenya 2013-2014 program report, Pgs 4-5. "Drug distribution: As per NDD operational guidelines, and established best practice, drug distribution was integrated with the training cascade (as detailed in the training section below), whereby NDD kits were provided to health functionaries at the district level trainings for onward distribution. The kits included drugs, IEC materials, and reporting forms." DtWI Madhya Pradesh 2015 program report, Pg 12. 31 For example: "DtWI provided technical expertise across all program components, and served as the primary coordinating body among implementing agencies. DtWI facilitated drug donations from WHO, provided professional master trainers for training of trainers, developed training materials such as flipcharts, modified training booklet and reporting forms, designed and carried out independent monitoring, and developed adverse event protocols." DtWI Rajasthan 2013 program report, Pg 5. "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." DtWI Madhya Pradesh 2015 program report, Pg 11. "We hire master trainers, or train govt staff to be master trainers." Grace Hollister, email to GiveWell, June 9, 2016 32 Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16. Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15. Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15. Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17. 33 "We work with governments and communications experts to design locally appropriate awareness campaigns to communicate messages through a wide variety of channels to increase public acceptance and effectiveness of deworming programs." Evidence Action website, Deworm the World Initiative (March 2016) 34 For example: "As part of their training, school headmasters/teachers were instructed to share information on the deworming program in the morning prayer sessions at their respective schools on a daily basis from October 6, 2013 onwards. They were also advised to convene school management committee meetings to communicate about the benefits of deworming and the schedule of deworming program. School headmasters were also advised to carry out student rallies / processions (prabhat pheri) to create awareness in the communities." DtWI Rajasthan 2013 program report, Pgs 8-9. 35 "One other key strategy adopted by DtWI to spread awareness was through text (SMS) reminders over mobile phones to school teachers, headmasters, Child Development Project Officers (CDPOs) and lady supervisors as a reminder about deworming day. SMSs were also used to reinforce precautions on drug administration, such as not giving drugs on an empty stomach, but only after midday meals and not giving drugs to sick children. In all, about 80,000 text messages were sent to school teachers and headmasters three times – a total of 2,40,000 [sic] messages. These messages were sent a day before deworming day, on mop-up day and after mop-up day. About 1400 such messages were sent twice to lady supervisors and CDPOs on a day before deworming day and on mop-up day. Similarly, five rounds of around 2400 text messages were sent to block level officials to expedite coverage reporting. This was an example of ensuring last-mile communication at low cost of about 12 paisa per message (or roughly 1/5th of a cent)." DtWI Rajasthan 2013 program report, Pg 9. "Additionally, mike announcements were made at public places in blocks and district headquarters by Evidence Action for 5 days, closer to deworming day (Annexure E.4)." DtWI Bihar 2015 Program report, Pg 13. "The State Health Society Bihar and Evidence Action rolled out a media mix to generate community awareness and increase program visibility to improve coverage in the state (Annexure E.1). We supported the adaptation and contextualization of prototypes from the National Deworming Day IEC resource toolkit. At the state level, State Health Society Bihar, in coordination with the Department of Public Relation, Government of Bihar, published newspaper advertisement in four dailies one day prior to deworming and mop up [sic] day, i.e., on 20 and 25 February (Annexure E.2). Radio jingles, customized into three local dialects, were aired from 15 to 26 February on the All India Radio to maximize outreach to the community. For additional visibility of the program at the community level, State Health Society Bihar printed 513,625 posters (7 for each school, including distribution in the local community), 1068 banners for Primary Health Centers, hoardings at 38 district headquarter [sic]. All of these were adapted and contextualized by Evidence Action." DtWI Bihar 2015 Program report, Pg 12. Other community awareness activities include creating posters to display at schools or advertising the deworming day in the newspaper. "Activities designed to enhance community awareness on deworming were rolled out to improve overall program coverage. The awareness activities included newspaper advertisements a day prior to the deworming day; a 60-second radio jingle aired on 3 FM channels from April 7 to 15 by School Health Scheme, and banners displayed at schools. Evidence Action was part of the committee formed by the state government for contextualization of the radio jingle. Evidence Action extended support to the state in contextualizing IEC materials from the National Deworming Day guidelines. The Directorate of Family Welfare also independently developed and printed handbills for the distribution at anganwadis to mobilize people on deworming day. The School Health Scheme provided banners to the schools, the distribution of which was integrated in trainings for teachers. The Delhi state government also used an e-portal to disseminate key information, including dates for deworming and mop up [sic] days, benefits of deworming, and details of the launch event." DtWI Delhi 2015 program report, Pgs 11-12. 36 "We help governments design monitoring systems to measure effectiveness in achieving intended program results. We also conduct independent monitoring to validate program results, and evaluate the impact of programs in reducing worm prevalence and intensity." Evidence Action website, Deworm the World Initiative (March 2016) 37 Note that Deworm the World hires monitors for the first and third type of monitoring data collected, but that the second is collected entirely by government staff: "Coverage reporting is done by the government- we sometimes assist in the data analysis, designing reporting forms, and ensuring that the 'reverse cascade' is appropriately designed." Grace Hollister, email to GiveWell, June 9, 2016 38 "Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program." DtWI Madhya Pradesh 2015 program report, Pg 16. 39 In India, on Deworming Day and Mop-Up Day, Deworm the World commissions independent monitors who go to schools to gather data on whether principals and teachers are prepared for Deworming Day, the availability of drugs and supplementary materials, whether students are being dewormed, whether proper procedures are being followed, and more. For example, see Deworming Day monitoring data from Rajasthan in 2015: DtWI Rajasthan 2015 monitoring survey from deworming day, schools (shows which questions were asked) and DtWI Rajasthan 2015 monitoring data from deworming day, schools (shows the survey responses). Kenya: Trainings monitoring: "PMCV [Process Monitoring and Coverage Validation] officers observed 36 CHEW [Community Health Extension Worker] Forums aimed at introducing the deworming sensitization message and materials/methods as well as asigning [sic] CHEWs to schools for monitoring. A successful community health extension worker forum is one that starts on time and where all the materials were present. Overall, 63% of participants arrived before training, whereas 22% arrived 1hr after the forum had begun and 15% of participants arrived more than 1hr after the forum’s commencement. Lateness appears to be a commonality to all training sessions. Materials required for CHEW training include a powerpoint printout, CHEW checklist and Severe Adverse Event (SAE; side-effects of the drugs) protocol. In 51% of forums, ALL of the Materials Pack was distributed at the start of the forum. In 13% of forums, SOME of the Materials Pack was distributed at the start of the forum. In 36% of forums, NONE of the Materials Pack was distributed at the start of the forum." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 9. Community sensitization monitoring: "A number of parents were also interview [sic] at schools on Deworming Day regarding their knowledge of deworming and the source of that knowledge. The intention behind this exercise was to compare the information source to those interviewed prior to deworming as a measure of consistency. In Figure 4, the results of the interviews pre-deworming day are compared with those parents interviewed on deworming day. The results remain largely similar, however more parents reported getting their information from ‘other’ sources (51%) when interviewed on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 14. School preparedness monitoring: "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15. Deworming Day and Mop-Up Day monitoring: "PMCV field officers visit schools on Deworming Day to observe procedure and interview teachers/head teachers regarding deworming. The number of schools observed on Deworming Day treating for STH in Year 3 was 247. The combined population of registered children at the observed schools was 88,820 children. It is estimated that 7,485 children were directly observed being treated for STH. Seven schools treating for both STH and SCH were observed. The total registered population of children in these schools was 3,198 children and 352 children were directly observed by field officers participating in Deworming Day. A quality Deworming Day is regarded to be one where: Deworming occurs within 1 week before [sic] teacher training The school would have the correct materials (including sufficient drugs) in place before commencement Children of the appropriate ages are treated (ages 2-14 years) Non-enrolled and ECD aged children are prioritized for treatment within the schools The correct dosage of drugs is given to all children" Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16. 40 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. For example, see DtWI Delhi 2012 coverage data by school. 41 Comments from Deworm the World in response to a draft of this page in October 2017. 42 Grace Hollister, email to GiveWell, March 6, 2016 43 "A competitive RFP [request for proposal] process is now used to identify a professional survey organization to provide independent monitors. There are requirements placed on the experience of these monitors." DtWI Monitoring Improvements 2014, Pg 1. Bihar: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state… Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively." DtWI Bihar 2015 Program report, Pg. 15. Rajasthan: "Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state... Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively." DtWI Rajasthan 2015 program report, Pg 14. Madhya Pradesh: "Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively." DtWI Madhya Pradesh 2015 program report, Pg 15. Delhi: "[Evidence Action] hired an independent research agency, Sigma Research and Consulting Private Limited [sic] that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively." DtWI Delhi 2015 program report, Pg 14. Deworm the World was also involved in Chhattisgarh, but as it was engaged relatively late in the process, it did not conduct all of its standard monitoring activities in the state: "Although we place great emphasis on understanding the extent to which the school and health systems are ready to implement deworming, the extent to which deworming processes are being followed, and the extent to which coverage has occurred as planned, in Chhattisgarh we supported only with the coverage validation activity at schools due to time constraints." DtWI Chhattisgarh 2015 coverage validation report, Pgs 2-3. 44 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Evidence Action has a permanent Monitoring, Learning, and Information Systems team. Deworm the World leverages this team for M&E." Grace Hollister, email to GiveWell, June 9, 2016 "The field officers that collect the data in the field are short term hires who come from the counties in which the program is implemented" Grace Hollister, edits to GiveWell's review, November 7, 2016 45 Note that these figures exclude Deworm the World's global (non-program-specific) spending and its indirect costs. 46 "Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success. "Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action." Evidence Action launch announcement 2013 "Evidence Action Beta investigates what interventions might be suitable for massive scale up [sic] – finding the next thing that works." Evidence Action website, Evidence Action Beta (October 2015) 47 India: "…absent from most of the country, [schistosomiasis] risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1. "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3. See section on worm prevalence and intensity in India and Kenya below. 48 WHO, Helminth control in school-age children, Pg 74 "Based on the findings of the prevalence survey and WHO guidelines, Evidence Action recommends an annual school based deworming program for school-age children in the state. [...] Given the pre-existing deworming treatments described above, this prevalence survey cannot be considered a baseline survey of an untreated population, but is rather a survey to assess STH infection rates in a treated population, to determine an optimal treatment strategy." DtWI Madhya Pradesh 2015 program report, Pg 38. "Our recommendation is explained in the prevalence survey report. The prevalence and intensity rates from the survey are not "baseline" data, given that there has been relatively regular administration of albendazole in MP through the BSM program that treated PSAC since 2005, and the LF program which provided community-wide treatment of 11 districts of MP (the number of endemic districts had fallen to 8 by 2014). As a result, these deworming efforts have likely had an impact on STH prevalence and MP could not be considered an untreated baseline population. We therefore did not apply the WHO guidelines for baseline STH prevalence. "Annexure 10 of the WHO guidelines suggest continuing annual treatment for populations which have received deworming for several years, and prevalence is still greater than 10%. In addition, the high rate of open defecation in the state, and the planned ending of the LF program in MP, increased the risk of infection and a potential resurgence in prevalence." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Grace Hollister, email to GiveWell, June 9, 2016 49 Deworm the World noted that the KEMRI surveys in Kenya are designed for impact assessment, while the surveys in India are designed for mapping. This means that the schools selected in KEMRI's surveys are all from places where treatment is required or taking place. In mapping surveys, schools are selected to be representative of a larger geographic area (e.g., they may be selected in part based on which agro-climatic region they are in). Additionally, Deworm the World noted that the surveys in India are looking at a much larger population than the Miguel and Kremer 2004 and Croke 2014 studies examined and there may be substantial variation in prevalence across a given area. Grace Hollister, conversation with GiveWell, June 13, 2016 and Grace Hollister, edits to GiveWell's review, November 7, 2016 50 For example, India has such a program: The National Vector Borne Disease Control Programme @LF treatment coverage 2015@ In Kenya, the LF program is housed within the country’s neglected tropical disease (NTD) unit, which has asked Deworm the World if it might support its process monitoring and coverage validation (PMCV) operations for LF. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Deworm the World has told us that in both countries (Kenya and India), LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Additionally, our understanding from a number of conversations with Deworm the World and others is that many countries are beginning to shut down their LF programs as they eliminate the disease. 51 "DEC [Diethylcarbamazine] + Albendazole in selected distt & DEC in other distt" @LF treatment drugs 2012@. "Albendazole, the same drug used to treat STH, is usually used to treat LF." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Assam 2010 guidelines for deworming recommends 400mg of albendazole for children being dewormed, and Global Alliance to Eliminate Lymphatic Filariasis - Prevention also recommends 400mg doses. 52 Or from twice-per-year to thrice-per-year. Note that community-based treatment, such as is typically used for LF, involves enlisting several people to travel from house to house to administer treatment, making it much more time-consuming and costly than school-based programs. Because its goal is to treat every person in a community, multiple trips to a single area may be required to ensure total coverage (e.g., if a household member is not at home during the first visit). GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Deworm the World also claims that it generally knows where there is overlap between areas that are endemic for STH or schistosomiasis and areas that are endemic for LF. For example, in Kenya, only the coastal area is LF-endemic, but worm infections are more widespread, so LF treatment efforts involve a smaller number of subcounties. Community-based treatment programs might be cost-effective in places endemic with many NTDs but might not be in places only endemic for STH or schistosomiasis. Deworm the World has told us that many places do not have much overlap between different NTDs so school-based deworming programs can provide a cost-effective alternative to community-based treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 53 More detail on this page. 54 We have gained this understanding through many conversations and across multiple years of reviewing Deworm the World. See our review process. Note that the methods Deworm the World uses in each country, for each round of MDA, might not be the same as those outlined here; Deworm the World adjusts its monitoring based on past learning or new contextual constraints. On a separate page, we lay out some broad descriptions of the monitoring processes we have seen in Deworm the World's monitoring reports for states in India. This is to provide an example for the reader of the type of monitoring that Deworm the World conducts and to support the subsequent claims in our report. We have not included excerpts from Deworm the World's 2016 reports from India; we skimmed the reports to see if they appeared to be substantially different from the 2015 reports and felt that they were of similar quality. 55 For example, as part of its monitoring in Kenya, monitors visit sub-county trainings and teacher trainings: "The intent of Sub-Country Training (SCT) sessions is to ensure that Sub-county and division-level trainers understand the purpose and procedure of deworming. The successful completion of this activity allows the division trainers to then conduct the same activity with teachers in their sub-counties. "PMCV teams attended 38 out of the total 111 SCTs conducted in the third year of the program. Field officers interview participants before and after the training and completed observations during the course of the activity." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 5. "Teacher Trainings (TT) are conducted by the division trainers using a “Teacher Training Booklet” as reference material. A quality TT session is considered to be one where the necessary content is covered and retained by participants. TTs also serve the function of distribution of drugs and materials (monitoring forms, posters) to teachers. It is the aim of the program to provide all schools with their required drugs and materials at teacher training sessions. Teachers are expected to use the “Deworming Day Checklist” to conduct operations on the day. They are also expected to sensitize other teachers at their schools who did not attend the training on deworming day procedures (see the section on deworming day contained in this report). "A total of 76 TT sessions were observed by PMCV field officers in Year 3. Of those training sessions, 13 were specifically SCH trainings." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 7. The trainings that monitors are sent to are randomly selected by a computer program. Paul Byatta, conversation with GiveWell, September 20, 2016 56 Paul Byatta, conversation with GiveWell, September 20, 2016 57 We believe that monitors are supposed to select every third teacher that arrives at the training until they have interviewed four teachers: "SURVEY INSTRUCTIONS [...] Select every third participant that arrives before the start of the training. Interview at least four participants." Deworm the World, Kenya Year 3, Pre TT form, Pg. 1 58 For example: Kenya. In Kenya, Deworm the World's monitors visit schools and communities before Deworming Day to assess their level of preparedness for the upcoming MDA. Schools are selected randomly, and then monitors visit the communities near the schools that are selected. Paul Byatta, conversation with GiveWell, September 20, 2016 "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15. "Use the PRE-DD-SCHOOL instrument to conduct interviews and note observations at the selected primary school before deworming day. In case selected school is closed, does not exist or has already dewormed move to the next nearby primary school. Allow Head teacher or Rep to consult with Head teacher/ REP when necessary. PRE-DD-SCHOOL should be completed along with PRE-DD-ECD or PRE-DD-COMMUNITY." Deworm the World, Kenya Year 2, Pre DD - School instrument, Pg 1. "PMCV Field officers interviewed a total of 716 parents with children. Of these, 379 were parents of enrolled children across 130 different schools and 337 were parents of non-enrolled children. The number of parents with at least one child enrolled in early childhood development (ECD) was 283 of the sample population, or 65%. Just over one third or 35% of parents had no child enrolled in ECD. The average age of those children reported to be enrolled (by their parents) was 7.7 years, whereas the average age of non-enrolled children was 3.7 years. "PMCV field officers observed parents’ level of awareness of Deworming Day, their intentions regarding taking children to be dewormed and documented the primary source by which parents were receiving such information." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 11. The next section is titled "4.1. Pre-Deworming Interviews - Parents." India: In India, telecallers phone schools ahead of Deworming Day to assess the school's preparedness. We have not yet requested data from these calls. For example: "Our tele-callers place phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17. 59 For example, a monitoring report we have from Kenya (for the 2014-15 round, or Year 3) includes some sample results from the pre-Deworming Day visits: The report notes that 97% of schools visited pre-Deworming Day had a teacher who had attended a training in the last fifteen days: "According to interviews with head teachers, 97% of these schools had a teacher who had attended training in the past 15 days. A further 86% of trained teachers had trained or sensitized other teachers on how to administer drugs and conduct deworming day. Almost all teachers (99%) found the Teacher Training Booklet to be ‘very’ or ‘somewhat’ useful in this process and 95% reported to use it often." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15. While 95% of schools had received some deworming tablets, only 87% of schools believed that they had received enough to cover the children at their school on Deworming Day: "At the time of PMCV visits, 95% of schools reported having received deworming tablets, with 97% of those schools having received them at the time of TT. This percentage is higher than those originally observed by PMCV officers with only 92% of schools reported to receive their drugs during TT. Upon further investigation, 87% of schools considered that they had received a sufficient supply of drugs for their current enrolled and ECD populations. It is likely that these schools requested additional drugs, because 98% of schools were observed to have sufficient drugs in place on deworming day. Only 5% of schools did not have all monitoring forms present prior to deworming day. Such schools have always sought support from the sub-county offices that organize additional prints or photocopying to ensure they have the forms on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16. Note that Deworm the World shared the pre-Deworming Day data that informs the monitoring report and we have not yet analyzed it. There appears to be a large number of questions in the survey that were not included in the report. The survey instruments that Deworm the World's monitors use are long (e.g., see Deworm the World, Kenya Year 2, Pre DD - School instrument, in which both teachers and randomly selected students are interviewed). The monitoring reports only include a few summary metrics, and we are unsure how the metrics included in the report are chosen. It is possible that the report's metrics are those that reflect most favorably on Deworm the World and that vetting the full data could reveal that schools are less adequately prepared. 60 When selecting parents to interview in the community, monitors ask the school headmaster to point to a student's house near the school. The monitor begins by interviewing that student's family, then walks to subsequent households, skipping every other household until the monitor has interviewed at least 6 families with an enrolled school-age child and 6 families without. Paul Byatta, conversation with GiveWell, September 20, 2016 In Kenya, monitors are also asked to interview the Community Health Extension Worker (CHEW) in the community, since they assist with Deworming Day: "On selecting the CHEW to interview, there is only one CHEW per community. A few communities do not have CHEWs at all." Paul Byatta, attachments to email to GiveWell, September 23, 2016 For example, from Kenya: "Of those parents aware of deworming, only 41% knew the correct Deworming Day date, 81% knew the correct target population, and 48% knew the correct age group. These results indicate that although parents report being aware of deworming, almost half do not have the information required to attend (date). There is the scope to find a more robust method of ensuring information retention in awareness of deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 12. 61 See this spreadsheet, sheet "Monitoring methods and results." Note: We have not seen this type of monitoring from Jharkhand 2016, and are uncertain whether Deworm the World did monitoring on Deworming Day or just coverage validation. Deworm the World, Jharkhand Independent Monitoring Report 2016. Note that if the chosen school is closed on the day of the deworming, they are instructed to go to an assigned backup school instead. India: For example, in Delhi, monitors are instructed in the survey instrument, "Is the school open? [If not,] Did you go to the buffer school?" DtWI Delhi 2015 monitoring survey from deworming day, schools Pg 3. It is our impression that most schools were open when monitors visited. We spot-checked several states' data in 2016: DtWI Bihar 2015 monitoring data from deworming day, schools, Column L shows that all schools monitors visited were open. DtWI Rajasthan 2015 monitoring data from deworming day, schools, Column M shows that 124 of the 125 schools monitors visited were open. Kenya: In Kenya, it is rare that monitors find schools to be closed or missing because Deworm the World has operated in Kenya for several years. However, monitors will try to go to another school if their school is closed. Paul Byatta, conversation with GiveWell, September 20, 2016 "When the monitor is unable to monitor the assigned school (because of closure, etc) we ask the monitor to i) to communicate the same with his supervisor and, ii) to visit the nearest school if the nearest school was not part of the randomly picked school." Paul Byatta, attachments to email to GiveWell, September 23, 2016 Vietnam: The backup schools are also randomly selected. Paul Monaghan, conversation with GiveWell, September 8, 2016 62 See, for example, DtWI Delhi 2015 monitoring survey from deworming day, schools, Deworm the World, Kenya Year 3, DD - Main instrument, and Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day Note that it is not actually dangerous to give the deworming pills to sick children; Deworm the World simply prefers not to deworm sick children to avoid causing people to associate the deworming pills with illness. "Deworming pills should not be distributed to sick children...This is not because deworming pills could harm sick children. It is because DtWI wants to avoid people (and potentially the media) blaming the deworming pill for a child’s illness." DtWI 2013 GiveWell site visit 63 For example, in Kenya: "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment. Deworming was reported to occur inside classes in 47% of observed schools and outside in 53% of schools. Deworming was considered to be ‘systematic’ in 98% of schools. The correct dosage for albendazole is one tablet per child and the correct age is 2-14 years. These procedures were observed to be followed correctly by 86% of teachers observed by field officers. Coverage: Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools. Teachers were reported to correctly observe children swallowing PZQ in 99% of schools and ALB in 96% of schools. Observing children swallowing is most important when treating for SCH as the tablet does not taste pleasant and there are high chances of children spitting if not observed." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17. For example, in India: When monitors visit schools on Deworming Day, they make observations and ask questions related to the process of deworming, to check whether or not it is being implemented correctly. For example, monitors observe whether deworming is in progress and whether teachers are using proper recording protocol, with results given for: "Deworming activity was taking place" "Followed any recording protocol" "Protocol followed: Putting single/double tick" Deworm the World, Telangana 2016 IMCV report, Pg 13. 64 Deworm the World is not supporting this monitoring in Ethiopia, because SCI and Deworm the World are supporting coverage surveys to be conducted after the Deworming Days there instead. Deworm the World also did not support this coverage validation monitoring for the first MDA it supported in Cross River, Nigeria, although it hopes to in subsequent rounds. Grace Hollister, edits to GiveWell's review, November 7, 2016 For example, this is what we heard about the monitoring conducted in Vietnam. Paul Monaghan, conversation with GiveWell, September 8, 2016. Mr. Monaghan noted that he had based the monitoring methods in Vietnam off of Deworm the World's methods used in its other countries. In India: We believe coverage validation was a few days or weeks after deworming days, varying by state and year. For example: Madhya Pradesh 2015: "The monitors visited 125 randomly selected schools on NDD, and an additional 125 schools on mop-up day (February 14) to check for adequacy of drug supplies and awareness materials, and assess whether teachers had received training, and had knowledge of adverse event management protocols and reporting processes. Monitors gathered data by observing deworming and by interviewing headmasters, teachers, and randomly selected students. An additional 750 randomly sampled schools were surveyed from February 18-26 to check whether deworming occurred and reporting protocols were followed, and to validate the coverage reporting." DtWI Madhya Pradesh 2015 program report, Pg 16 Delhi 2015: "These monitors were to visit total of 400 randomly selected schools and 400 randomly selected anganwadis; 80 schools and 80 anganwadis on deworming day and mop up day each (April 16 & April 20); and 240 schools and 240 anganwadis during coverage validation (April 23-27, 2015)." DtWI Delhi 2015 program report, Pg 14 Jharkhand 2016: "Jharkhand observed the third round of NDD in 19 of 24 districts on August 10, 2016 followed by Mop-Up Day on August 17, 2016.... Coverage validation was undertaken from September 14-20, 2016 during which 100 monitors were targeted to visit 500 randomly selected government/government-aided schools and 500 anganwadis to verify the reported coverage numbers." Deworm the World, Jharkhand Independent Monitoring Report 2016, Pgs 1-2 65 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools 66 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools. 67 For example, in Kenya, it is our understanding that monitors interview three randomly selected children from three separate classes, for a total of nine students, at each school they visit. See Deworm the World, Kenya Year 3, Post DD - Coverage instrument. On Pgs 1-3 there are three spaces for randomly selected classes. The instructions read: "Thank the Head Teacher or designate and request to speak to pupils of the randomized class...CHOOSE CHILD 5, 10 AND 15TH ON FORM E. IF LESS THAN 15 CHILDREN, SELECT THE LAST CHILD. ENSURE TO INTERVIEW AT LEAST THREE CHILDREN. ASK THE TEACHER FOR PERMISSION TO SPEAK TO THEM ONE AT A TIME Ask questions in multiple ways for interviews with students, use local language if possible. Don’t rush responses. Try to make them feel at ease. Speak to one child at a time at a place where they are comfortable.... [interview questions]...END, MOVE TO THE NEXT SAMPLED CLASS" 68 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools 69 Grace Hollister, edits to GiveWell's review, November 7, 2016 70 Vietnam: There were no checks on the monitors' work for the first round of treatment. Paul Monaghan, conversation with GiveWell, September 8, 2016 Kenya: In Kenya, Deworm the World does have some checks on monitors' work. A randomly selected 10% of the schools that are being visited by monitors are called the same day that the monitors visit, to ensure that the monitor is actually at the school. Paul Byatta, conversation with GiveWell, September 20, 2016 "We do back-checks for the pre-deworming, de-worming and post-deworming day… Generally, we seek to establish that the monitor did actually visit the school assigned to him/her, interviewed at least the headteacher of the school and that there is broad consistency on some of the data they collect by the monitor and the back-checker. For instance, the back-checker also asks about the availability of forms and drugs at schools, which is also information that is asked by the monitor. We share back-checks results with the short term monitors before we make the last payment to them. The other purpose is to improve our data collection training for the subsequent waves." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pgs 2-3 India: "In India, the following steps are implemented to serve as check on the IMCV work. Some of these are detailed in the contracts with IM firms – let me know if you’d like to see those: 1. We collect photographs of all the schools and anganwadis (with the names) visited during PMCV in the states. 2. We also collect signatures and mobile numbers of headmasters/school teachers and anganwadi workers, which is the part of PMCV format itself. 3. Together with signature of the teachers/head masters, school stamp is also taken on signature sheet which is part of PMCV tools. Since we collect CAPI based data, monitors carry hard copy of signature sheet to get the stamp of the schools. 4. Additionally, random calls are also made by state team/tele-callers to confirm that monitors visited to designated school and anganwadis. 5. Evidence Action staff (including RCs & DCs) visits in selected schools and anganwadis on NDD and mop-up day to check if monitors visited the selected sites." Grace Hollister, edits to GiveWell's review, November 7, 2016 71 Grace Hollister, conversation with GiveWell, September 1, 2016 For example, in Kenya: "Does the monitoring team have an estimate of how many non-enrolled children the deworming program reaches? We get this from treatment forms that schools submit back via “the reverse cascade”. For PMCV, we monitor that teachers are aware they should be treating non-enrolled, and they are aware of age categories targeted for non-enrolled. When we do data audit on treatment forms, we also check that data for on non-enrolled is entered accurately." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pg 3 72 Comments from Deworm the World in response to a draft of this page in October 2017. 73 Kenya: "Show student 3 tablets, ask: Which one of these three tablets did you take? Circle the indicated tablet." Deworm the World, Kenya Year 3, Post DD - Coverage instrument Vietnam: "What type of medicine was given out? (prompt the child with options)" and "Do you remember the color of the tablet? (do not prompt)" are questions on the coverage validation survey form for student interviews. Deworm the World, Vietnam 2016 monitoring survey form for coverage validation, Pg 7. India: In India, monitors show children the deworming tablet and ask if it's the tablet that they took, which we do not believe is a rigorous method of verifying student answers. For example, see Pgs 20-23 of DtWI Bihar 2015 monitoring survey for coverage validation, schools 74 Deworm the World notes that an important difference between the two programs is scale: The Kenya program is significantly smaller than the India program, thus it may be easier to execute the program more effectively. Grace Hollister, email to GiveWell, June 9, 2016 75 Deworm the World and SCI, Ethiopia coverage survey 76 Deworm the World supports program costs in Cross River, Nigeria and expects to support program costs in future Nigeria states. It expects to support program costs in Pakistan, and possibly Indonesia. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 77 We attempt to learn if there was a possibility that the program would have been funded by other donors. However, given that there is a global funding gap for deworming treatments (more), we suspect that in most cases where Deworm the World pays for a new deworming program, it is increasing the number of children dewormed. 78 Our intervention report discusses this briefly. Other conversations and observations have reinforced our impression that administering deworming drugs is fairly straightforward. The WHO factsheet on STH: "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet 79 Deworm the World notes that: "albendazole and mebendazole are quite hardy (neither requires special storage conditions) and have a long shelf life." Grace Hollister, email to GiveWell, June 9, 2016 80 For example, drug quality was tested in each program that Deworm the World supported in India in 2013-2014. Bihar 2014: "The drugs were safely stored in State Health Depot in Patna until November 2013. At that point, the districts began to pick up their share of the drugs from the depot. In October, Deworm the World had coordinated for lab testing of the stored drugs via ASCHO NIBULA INDUSTRIES LTD, an independent lab which approved the quality of the drugs." DtWI Bihar 2014 program report, Pg 13. "The quality of drug storage was satisfactory in most schools that were monitored. 98.7% of them stored the drugs in a clean location, 91.1% of them were stored away from direct sunlight and 97.4% of them were stored away from the direct reach of children." DtWI Bihar 2014 program report annex 1, Pg 7. Delhi 2013: "Once the procured syrups and donated tablets were delivered to Directorate of Health Services central storage room, they were tested in a government-accredited laboratory to ensure drug quality prior to administration." DtWI Delhi 2013 program report, Pg 6. From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Delhi 2013 program report, Pg 24. A table of results is also presented on the same page. Rajasthan 2013: "To instill confidence among the stakeholders that the drugs were of good quality, Deworm the World arranged for sample testing of the donated drugs by two independent labs7. Similarly, Rajasthan Medical Services Corporation sample tested the syrups they procured." DtWI Rajasthan 2013 program report, Pg 6. From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Rajasthan 2013 program report, Pg 40. A table of results is also presented on the following page. 81 "Now, state governments have responsibility for testing drugs. We have recently raised some concerns about the need to standardize the testing that is taking place, and are currently working with the MoHFW to build out the NDD operational guidelines with more detailed guidance on this point." Grace Hollister, edits to GiveWell's review, November 7, 2016 82 Comments from Deworm the World in response to a draft of this page in October 2017. 83 Of classes where monitors observed deworming activities in India, there were low numbers of adverse events (see table below). We don't know what portion of the adverse events may be caused by incorrect dosages. Bihar 2015 Rajasthan 2015 Delhi 2015 Madhya Pradesh 2015 Sample of question asked Classes where there were adverse events (monitors' observations) 5% (vomiting), 0% (diarrhea) 2% (vomiting), 0% (diarrhea) 5% (vomiting), 0.8% (diarrhea) 6% (vomiting), 0% (diarrhea) "Did you see any child with adverse effects (nausea, vomiting, stomachache, etc.) after taking the medicine?" [Table last updated in 2016] Sources for the information in the table: Bihar 2015: Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 4 Rajasthan 2015: Deworming Day and Mop-up Day ( N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 4 Delhi 2015: Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 6 Madhya Pradesh 2015: Deworming Day and Mop-up Day (N = 250 schools), DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 4 Sample question: DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 14 84 "Note that National Deworming Day operational guidelines state that only tablets should be used. Albendazole dosage is the same for all children aged 2 and above; it is only children aged 1-2 that require a different (half) dose." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 From Deworm the World's report on the Delhi 2013 program: "A key discussion from round one was the administration of drug to younger children in the pre-school age for whom chewing a tablet was seen as a difficulty. Hence keeping in mind the scale of the program, with an objective of making it a safe public health initiative the committee decided that deworming would be implemented across all districts following the World Health Organization (WHO) sanction in administering albendazole 400 mg tablets and the GOI guidelines under the WIFS program for administering the syrup vial. They also decided to opt for Albendazole 200mg dose suspension for 2-6 year children as it is a single dose for this age group which leads to lesser error in administration, while Albendazole 400mg tablets for older children." DtWI Delhi 2013 program report, Pg 6. The WHO factsheet on STH cites only a single recommended dosage (depending on which drug is used): "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet 85 Bihar 2015: In 8.8% of trained schools and 5.4% of untrained schools, monitors observed children given less than one tablet; in 2.6% of trained schools and 4.7% of untrained schools, monitors observed children given more than one tablet. DtWI Bihar 2015 independent monitoring tables, Pg 9 (Table 10) Rajasthan 2015: In 6.0% of schools, monitors observed children given less than one tablet; in 1.6% of schools, monitors observed children given more than one tablet. DtWI Rajasthan 2015 independent monitoring tables, Pg 2 (Table 2) Madhya Pradesh 2015: In 9.3% of schools, monitors observed children given less than one tablet; in 2.7% of schools, monitors observed children given more than one tablet. DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2 (Table 2) Delhi 2015: In 2.5% of schools and 10.6% of anganwadis, monitors observed children given less than the prescribed dose of albendazole; in 0.8% of schools and 6.0% of anganwadis, monitors observed children given more than the prescribed dose of albendazole. DtWI Delhi 2015 independent monitoring tables, Pgs 2, 16 (Tables S1 and A1) Bihar’s 2014: 8% of schools observed gave children less than one tablet and 2% gave more than one tablet. DtWI Bihar 2014 program report annex 2, Pgs 2 and 4. "In 3.1% of schools and anganwadis, monitors observed children being given more than one tablet/syrup bottle. As per protocol, the children should not have been given more than one tablet. Therefore in cases such as these, the monitors were trained to intervene and prevent the administration of an additional dose." (The percentage of schools using less than one pill/bottle per child was not reported.) DtWI Rajasthan 2013 program report, Pg 38. Monitors in Delhi in 2013 did not report on either of these observations. See DtWI Delhi 2013 program report, Pgs 21-25. 86 "The majority of direct program costs in India are government funded. There is a cap within the National Health Mission budgets on M&E – M&E cannot exceed 10% of the overall budget. So there are specific areas, such as program monitoring, that would likely not have investment at the level we are able to provide." Grace Hollister, edits to GiveWell's review, November 7, 2016 "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2. "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and Deworm the World's tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. Deworm the World called a random sample of 8,000 schools. The prevalence survey would not have happened without Deworm the World's support." DtWI 2013 GiveWell government interviews, Pg 5. 87 "RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3. 88 "[The Nodal Headmaster said] that most aspects of the program are excellent, but he had 2 suggestions: Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so) Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6. 89 "In Tanzania matters came to a head in places around Morogoro in 2008. Distribution in schools of tablets for schistosomiasis and soil-transmitted helminths provoked riots, which had to be contained by armed police. It became a significant national incident, and one of the consequences has been the delay in Tanzania adopting a fully integrated NTD programme, and the scaling back some existing drug distributions." Allen and Parker 2011, pg. 109. "From these reports a number of problems with the MDA were raised which included fear of side effects from the tablets, particularly following the mass hysteria and death in Blantyre and Rumphi respectively and may explain some of the geographic heterogeneity seen. Furthermore most districts reported that MDA occurred after standard 8 students had finished exams and left school, and due to having inadequate resources for drug distribution...The side-effects incident in Blantyre and death in Rumphi had a large effect on districts and with many district reports stating that after the incidence many families refused to participate." SCI Malawi coverage survey 2012 Pgs 5, 21. 90 "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 Deworm the World noted that the costs in Kenya are high, partly due to higher quality M&E: "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 In the second round of treatment in Kenya, prevalence surveys were 11% of the total costs and other M&E was 6%. Per treatment, these costs are $0.059 ($379,523/6,405,462) and $0.035 ($222,750/6,405,462) respectively DtWI Kenya 2013-2014 cost per treatment data Our understanding is that costs are higher in Kenya than India primarily due to cultural differences, including paying significantly higher allowances (per diems)to teachers for participating in the program. For more details, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell analysis of Deworm the World cost per treatment. Deworm the World has also noted that treating schistosomiasis in Kenya increases the costs: "The extra costs of treating schistosomiasis in addition to STH. Schistosomiasis drugs tend to be more expensive than STH drugs and, in Kenya, schistosomiasis treatment sites are sometimes much more remote. The treatment strategy for schistosomiasis also differs from STH because schistosomiasis is more localized (e.g., it is not necessarily ideal to treat an entire sub-county). This also makes mapping schistosomiasis more expensive." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 3. 91 Deworm the World estimates that the Cross River program in 2016 cost a total of $0.69 per child and the Kenya program cost $0.45 per child in 2015-2016. See this spreadsheet, sheets "Cross River 2016" and "Kenya 2015-16." Note that we make several adjustments to these figures to get our estimate of total cost per treatment (see sheet "Summary). 92 We estimated the staff time costs based on Deworm the World's estimate of similar costs from the same states in 2012. It had imputed those costs based on estimates of government employee salaries, for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50). 93 "We include all partners’ expenditures in determining costs for the the deworming programs, but we do not consider spending that would be incurred even without deworming taking place. [...] Teachers’ and principals’ general salaries are not included because they do not spend additional time on deworming beyond what they are already compensated for by the government for regular classroom teaching." Evidence Action, blog post, July 5, 2016 94 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Deworm the World thought that the costs of these activities were quite low (less than 5%). We have assumed 3.5% in our analysis. 95 See our discussion of the rationale and limitations of this estimate here. 96 See this spreadsheet, sheets "Funding commitments" and "Spending opportunities." 97 See this spreadsheet, sheets "Funding commitments" and "Spending opportunities." 98 See this spreadsheet, sheet 'Funding commitments.' 99 See this spreadsheet, sheet 'Available and expected funding.' 100 This is based on internal records of how much GiveWell-influenced donors gave to Deworm the World in the last year when Deworm the World was on GiveWell's top charity list but was not the recommendation for marginal funding. See this spreadsheet, sheet 'Available and expected funding.' 101 See this spreadsheet, sheet 'Available and expected funding.' 102 GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017. 103 Discussed in this blog post. 104 GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017. 105 See this spreadsheet, sheet "Spending opportunities," row 13. 106 See this spreadsheet, sheet "Spending opportunities," rows 20-24. 107 For example, the END Fund told us it was interested in funding deworming in additional states in Nigeria. See this spreadsheet. 108 One of the documents had an incorrect formula (adding in expenses rather than subtracting them), while another double counted about $750,000: "The version shared during the [meeting] reflected $750K for core support allocated to programs, as well as $1.3M in unrestricted fund transfers to programs. After realizing that we had double counted the $750K in core support, we eliminated it in the version sent on Oct 11." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Deworm the World's restricted funding on hand was about $2.5 million lower after corrections had been made. Our understanding is that about $1.7 million of that was funding that had been mislabelled as restricted, but was actually unrestricted, and the rest of the decrease was due to the fact that Deworm the World had actually already spent a portion of its restricted funding in India (this change increased Deworm the World's expenses figure by $500,000 and Evidence Action's starting revenue by approximately $200,000). "$1.7M in Deworming restricted funding was booked into our financial system as revenue against a current commitment instead of as new revenue. The coding has been corrected and this amount is now reflected in an increase in revenue on the report (cell D5) [...] $730K in expenses incurred in India between March and July 2016 were not included in the previous report - increasing total expenses by a like amount. Note that $187K in organization development costs allocated to the program in the form of indirect costs in past tables was backed-out (see second bullet under Evidence Action updates below) resulting in a net increase in expenses of $540K (cell D6)." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Deworm the World's revenue increased by $2.7 million after corrections. We believe this is from the $1.7 million that was initially mislabelled as restricted funding and an additional $1 million that was incorrectly allocated internally. Correcting the misallocation also caused Evidence Action's revenue to decrease by $1 million. "$1.7M in Deworming restricted funding was booked into our financial system as revenue against a current commitment instead of as new revenue. The coding has been corrected and this amount is now reflected in an increase in revenue on the report (cell D5). [...] $1.0M in Deworming program general support funds (what we had been calling "unrestricted" for deworming) received in 2015 and 2016 were coded inconsistently. These costs were correctly coded to the Deworming program but were assigned the wrong funder code. The coding has been correctly aligned, and this amount is now reflected in an increase in revenue on the report (also cell D5)." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Evidence Action's expenses increased $750,000 after corrections, to reflect that the funding had been used on organizational expenses (this was previously unreflected in the original documents): "The $750K was spent on one-time operational costs such as temporary accountants to convert our books from cash to accrual, transition to a new accounting system, separation costs from the incubator established in India, and relocation costs for the Nairobi office after our landlord ended the lease." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Once these changes had been made, Evidence Action revised the amount of unrestricted funding it intended to allocate to Deworm the World this year, decreasing it from $350,000 originally to $163,000. 109 Originally, Deworm the World's financial documents indicated that Deworm the World had $6,363,718 in unrestricted funding available; after corrections, this figure was $10,969,610. GiveWell analysis of Deworm the World financials - 2016, tab "DtW Unres Commit" 110 John de Wet, Evidence Action CFO, conversation with GiveWell, September 29, 2017. 111 GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017 112 "Note that this is specific to CIFF; the END Fund grant is slated to end in 2018" Grace Hollister, edits to GiveWell's review, November 20, 2016 113 Grace Hollister, edits to GiveWell's review, November 20, 2016 114 Ellen Agler, END Fund CEO, conversation with GiveWell, October 9, 2017 115 "Evidence Action leads and manages two programs incubated by Innovations for Poverty Action: Dispensers for Safe Water and the Deworm the World Initiative. We also run Evidence Action Beta where we are currently testing a number of other rigorously-evaluated interventions for scale-up." Evidence Action website, Who we are (November 2016). No Lean Season is a project of Evidence Action Beta. 116 "...many of the investments of unrestricted have substantial positive benefit for DtW, even as they are not specifically programmatic in nature. These include investment in financial capacity, and the transition of the Indian entity (which at the moment is exclusively working on deworming, though this may not be the case in the future)." Grace Hollister, edits to GiveWell's review, November 20, 2016 117 Evidence Action 2017 financials, GiveWell submission, "Unrestricted commitments" sheet 118 WHO, Summary of global update on preventive chemotherapy implementation in 2016, Pg 590, Table 1. WHO, Summary of global update on preventive chemotherapy implementation in 2015, Pg 456, Table 1. WHO Weekly epidemiological record, 18 December 2015, Pg. 707, Table 1.