Evidence Action's Deworm the World Initiative - November 2019 Version

We have published a more recent review of this organization. See our most recent report on Deworm the World.

Published: November 2019

Summary

What do they do? Evidence Action's Deworm the World Initiative (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates government-run school-based deworming programs. (More)

Does it work? We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost. Deworm the World sends monitors to schools during and, for most distributions, after deworming to determine whether the programs it supports have reached a large proportion of children targeted. We have reviewed data from each of its major programs, which overall indicate strong results. (More)

What do you get for your dollar? Our best guess is that deworming is generally highly cost-effective. We estimate that, in Kenya, the cost per child dewormed is about $0.66 per child, or $0.46 per child excluding in-kind contributions from governments. We estimate that the cost per child treated in India is roughly half of that. We expect the cost per treatment in other countries to be closer to that in Kenya than India, and may be more expensive in the early stages of a program. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)

Is there room for more funding? In the next three years, we estimate that Deworm the World will have opportunities to spend about $3.5 million more than we expect it to receive in that time. Deworm the World is also seeking to increase its reserves by about $7.5 million. (More)

Evidence Action's Deworm the World Initiative is recommended because of its:

• Focus on a program with a strong track record and excellent cost-effectiveness. (More)
• Strong process for assessing whether the deworming programs it supports are successfully deworming children. (More)
• Standout transparency – it has shared significant, detailed information about its programs with us.
• Room for more funding – we believe Deworm the World will be able to use additional funds to start or maintain deworming programs.

Our review process

Our review process has consisted of:

• Extensive conversations with Deworm the World Director Grace Hollister and other Deworm the World and Evidence Action staff since 2012.1
• Reviewing documents Deworm the World sent in response to our queries.
• Site visits:
  • In November 2012, we visited Deworm the World's office in Nairobi, Kenya and met its staff there. (Notes from our visit)
  • In October 2013, we visited Deworm the World's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with Deworm the World. (Notes from our visit)
• In 2015, we retained two journalists to visit areas served by Deworm the World in Kenya. We published their report on our blog.
• Conversations with the Children's Investment Fund Foundation (CIFF), a funder of Deworm the World.2

All content on Deworm the World, including past reviews, updates, blog posts, and conversation notes, is available here. We have also published a page with additional, detailed information on the program to supplement some of the sections below.

What do they do?

The Deworm the World Initiative is a program led by Evidence Action, an organization that focuses on scaling up interventions that it believes are cost-effective and evidence-based. Deworm the World advocates for and supports the implementation of government-run deworming programs for preschool- and school-age children.3 The support that Deworm the World provides is of two types: 1) offering technical assistance to governments implementing deworming, and 2) funding components of deworming programs.4

The deworming programs that Deworm the World supports are focused on executing school-based mass drug administrations (MDAs), in which the aim is to treat the entire population of children within a geographic area by distributing deworming pills.5 Deworm the World focuses on MDAs that treat children infected with soil-transmitted helminthiasis (STH).6 Where needed, these programs also include treatment for schistosomiasis.7

In the countries it works in, Deworm the World works primarily with government staff to implement deworming programs; as it has expanded to new countries, it has started to also collaborate with non-governmental partners to support governments.8 Deworm the World has also funded or is considering working on a few projects that fall outside of its purview of supporting the direct implementation of deworming programs, specifically related to monitoring and research (see footnote).9

Deworm the World was founded in 2007,10 and as of 2019 had supported deworming treatments in India, Kenya, Ethiopia, Nigeria, Vietnam and Pakistan.11 Many of these programs are recent and represent progress on Evidence Action's efforts to scale up Deworm the World; as of early 2015, Deworm the World had only supported treatments in India and Kenya. On a separate page with additional information about Deworm the World, we discuss more details of its work by country.

Below, we discuss:

• Deworm the World's role in government-led deworming programs
• A breakdown of Deworm the World’s spending
• Deworm the World's relationship to Evidence Action

Deworm the World’s role in government-led deworming programs

The deworming programs that Deworm the World supports are implemented by the governments it works with.12 Below, we expand on Deworm the World's role in the programs it supports. Note that we use "Deworming Day" to mean the day on which the MDA takes place. Similarly, we use "Mop-Up Day" to refer to the day that occurs several days after Deworming Day and is when students who were absent or sick on Deworming Day are given their deworming pills (although note that not all countries have just one Deworming Day or include a Mop-Up Day in their program).13

The assistance that Deworm the World provides in each country varies based on what each partnering government needs.14 Historically, Deworm the World's role has included the following:15

1. Advocacy. Deworm the World actively encourages national and large sub-national governments to implement mass school-based deworming programs.16 Our impression is that Deworm the World's advocacy consists of meeting with health and education officials in a government to discuss the benefits of deworming and how a deworming program might be implemented.17 Deworm the World has told us that it will not work with a government on a national deworming program until it has built a strong working relationship with that government via its advocacy.18 Deworm the World also participates in the broader "STH community"; that is, it works with other organizations advocating for and implementing activities that aim to further reduce or eliminate STH globally.19
2. Prevalence surveys. Before Deworm the World helps launch a deworming program in a new area, it evaluates whether the prevalence of worm infections is sufficient to justify an MDA for the school-age population.20 If no prevalence surveys have been conducted recently, it generally commissions one.21 The results of prevalence surveys are used to determine the appropriate treatment strategy (in particular, MDA frequency) for a given location.22 It also plans to conduct follow-up prevalence surveys periodically, so that it can track the impact of the MDAs and refine treatment strategies as needed, in accordance with WHO guidelines.23 Deworm the World generally contracts out work on prevalence surveys.24
3. High-level program planning. Deworm the World has told us that it often assists governments with high-level operational decisions, such as developing the country's treatment strategy and operational guidelines25 and creating a budget for the program.26
4. Drug procurement and protocols. Deworm the World assists governments in obtaining drugs, designing drug distribution and tracking processes, and developing adverse event protocols for cases where children react poorly to treatment.27 For example, Deworm the World has helped governments submit requests for deworming drugs (albendazole or praziquantel) to the World Health Organization (WHO) global drug donation program.28
5. Program preparation: trainings and distribution of materials. Deworm the World has helped governments design and organize what it calls a "training cascade" (more detail in the footnote).29 Through the training cascade, teachers and other government staff learn how to implement a Deworming Day and receive materials necessary for implementation (such as reporting forms and drugs).30 In the past, Deworm the World has hired or trained staff to lead the trainings and developed materials for the trainings.31 In India, Deworm the World has also arranged tele-callers to reach out to schools to assess their preparedness and notify government officials of any problems before Deworming Day.32
6. Community sensitization. Deworm the World supports community sensitization efforts, which aim to make local communities aware of Deworming Day and the benefits of deworming children.33 For example, via the training cascade, teachers are instructed to spread the word about Deworming Day to their communities.34 Deworm the World has also developed text message campaigns, organized public announcement events, and edited mass media materials to be more appropriate for local contexts.35
7. Monitoring and evaluation. Deworm the World told us that it helps governments design or improve reporting and monitoring systems. It also collects monitoring data independently.36 Deworm the World focuses on assisting with the collection of three main types of monitoring data:37
   • Monitoring before and during deworming: Monitors hired by Deworm the World visit schools before and during Deworming Day and Mop-Up Day. They are meant to assess both a) how prepared schools and health systems are to implement deworming and b) the extent to which proper procedures are followed.38 Monitoring visits may include assessments of the quality of trainings, community sensitization efforts, and Deworming Day activities, depending on what Deworm the World and the government agree to monitor.39
   • Coverage reporting: On Deworming Day and Mop-Up Day, teachers are asked to mark the number of children that they deworm and schools complete specially designed reporting forms to tally the number of children treated. This data is then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (e.g., in India: school, node, block, district, and state) to create a reported coverage estimate for a region.40 Deworm the World notes that, more recently, block officials in India have submitted coverage data online to the national government.41
   • Coverage validation: Approximately one week after Mop-Up Day, Deworm the World sends independent monitors back to schools to check the coverage data and attendance records recorded at schools against the data submitted and ask students about whether or not they were dewormed.42 This data can then be compared to the coverage data reported by the government.

   Typically, Deworm the World hires and trains third-party monitors to collect process monitoring and coverage validation data; the following footnote includes Deworm the World’s descriptions of the monitor selection process used in four states in India in 2015.43 In Kenya, Evidence Action (Deworm the World's parent organization) maintains a monitoring team year-round that Deworm the World makes use of.44

On a separate page, we detail Deworm the World's work by country.

Breakdown of Deworm the World’s spending

We summarize Deworm the World's spending for 2017 and 2018 in this spreadsheet.

In short:

• Deworm the World spent a total of $12.4 million in 2018, up from $10.2 million in 2017.
• About half of spending was from funding sources that are restricted to a particular project, primarily programs in India and Kenya which are funded by CIFF, the END Fund, EPIC Foundation UK and EPIC Foundation France, and Dubai Cares.45 The other half of funding was from funds that Deworm the World can allocate at its discretion (this includes funds directed by GiveWell).
• Deworm the World's biggest programs in 2018 (and historically) were in Kenya (31% of program spending) and India (44% of program spending). Nigeria accounted for 17% of spending, Pakistan for 7%, and, Ethiopia, and Vietnam for less than 1% each.46

For information on spending in previous periods, see the previous versions of our review of Deworm the World.

Deworm the World and Evidence Action

In early 2013, Innovations for Poverty Action (IPA) announced the formation of Evidence Action to scale cost-effective and evidence-based programs. Two IPA initiatives, Deworm the World and Dispensers for Safe Water, were spun off from IPA to be managed by Evidence Action. Evidence Action subsequently built a department for investigating, testing, and considering new programs for scaling up called Evidence Action Beta; one program in the Beta portfolio was No Lean Season, which GiveWell also recommended as a top charity in 2017.47 We removed No Lean Season from our list of top charities in 2018; see this blog post for more detail. In 2019, Evidence Action replaced Evidence Action Beta with Evidence Action Accelerator, which will continue the work of developing new programs.

We focus this review on Deworm the World and discuss the room for more funding implications of Deworm the World being a program of a larger organization below.

Does it work?

We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost. Evidence from Deworm the World's monitoring makes a relatively strong case that the programs Deworm the World has supported have successfully dewormed children.

Here we focus on Deworm the World's monitoring from Kenya, India, and Nigeria, because those are by far its largest programs (with the exception of Pakistan, where the deworming began too recently for us to have received monitoring results; see above). Deworm the World's track record in Kenya and India is strong. Deworm the World's track record in Nigeria is more limited, but what we have seen seems fairly strong.

In the sections below, we focus on the following questions to understand whether Deworm the World’s activities are having the intended impact.

• Are mass school-based deworming programs effective when implemented well?
• Are Deworm the World's programs targeted at areas of need?
• Are deworming pills delivered to and ingested by recipients?
• How does Deworm the World affect program outcomes?
• Are there any negative or offsetting impacts?

Are mass school-based deworming programs effective when implemented well?

Deworm the World supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.

There are some important differences between the type and severity of worm infections in the places Deworm the World works and the places where the key studies on improved life outcomes from deworming took place (which we discuss below). In particular, Deworm the World primarily provides support to mass drug administrations (MDAs) that treat populations in which fewer children are infected with soil-transmitted helminths and where the severity of infections tends to be lower (as compared to populations in the key studies we refer to above). In addition, several of the programs Deworm the World supports do not treat schistosomiasis because it is not endemic in the areas the programs support.48

Are Deworm the World's programs targeted at areas of need?

What is the likely impact per treatment in Deworm the World's programs compared with the independent studies on the impact of deworming? In general, mass deworming programs treat everyone in a targeted demographic, regardless of whether each individual is infected (more). Because of this, the benefits (and therefore the cost-effectiveness) of a program are highly dependent on the baseline prevalence of worm infections. In this section, we discuss how the disease burden in the areas Deworm the World works in compares to the places where the independent studies that form the evidence base for the impact of deworming were conducted. While it is our understanding that Deworm the World programs generally target areas that require mass treatment according to WHO guidelines,49 the disease burden in Deworm the World areas is on average lower than in the study areas, so our expectation is that the impact per child treated is lower in Deworm the World areas. We adjust our cost-effectiveness estimate accordingly (more below). In this spreadsheet, we compare the worm prevalence in places where Deworm the World currently supports a program to the prevalence from the studies providing the best evidence for the benefits of deworming. The prevalences in the table may not be directly comparable to one another. Prevalence surveys were conducted in Madhya Pradesh and Chhattisgarh after multiple rounds of treatment, in Bihar and Rajasthan after one round of treatment, and in Uttar Pradesh after multiple rounds of treatment in some districts and one round in other districts.50 It is likely that prevalence was higher at baseline for these regions. Deworm the World notes that with a few exceptions, it conducts prevalence surveys prior to supporting treatment.51 Deworm the World notes that there are relevant methodological differences between the prevalence surveys, which makes them difficult to compare;52 we agree that this data is not ideal for our purposes but believe that it provides the best estimate we have and adjusting for baseline infection rates is an important part of our cost-effectiveness model. Treatment for lymphatic filariasis In some of the countries where Deworm the World works, there are existing programs to treat lymphatic filariasis (LF).53 Albendazole, the same drug used to treat STH, is usually used in combination with one additional drug to treat LF (and the same dosage is used for both treatments).54 For areas that have existing LF treatment programs, the effect of Deworm the World’s support may be to transition an area from once-per-year deworming treatment (for STH) to twice-per-year treatment.55 We detail what we know about the status of LF programs in the areas in which Deworm the World works on a separate page with additional information about Deworm the World. Are deworming pills delivered to and ingested by recipients? Administration of deworming drugs is a relatively straightforward program, though any MDA could encounter challenges, particularly when operating at a large scale. The information we have seen from monitors hired and trained by Deworm the World in India, Kenya, Nigeria, Vietnam, and Ethiopia suggests that the programs successfully deliver pills to children, who then swallow them. Additionally, prevalence surveys in Kenya, and to a lesser degree Bihar, India, show that the prevalence rates of STH and schistosomiasis have declined substantially since Deworm the World started supporting MDAs in those areas, providing additional evidence that the treatments are reaching recipients.56 Evidence from monitoring For each of its programs, Deworm the World hires monitors (who are not associated with the government implementing the program) to:57 [Sometimes] Observe trainings. For most of its programs (those in Kenya, Nigeria, India, and Vietnam), Deworm the World sends monitors to visit a random selection of trainings.58 In Ethiopia, regions are purposively selected for the monitoring of trainings.59 The visits are unannounced, and the monitors use a standardized checklist to track if the training covers all required topics.60 Monitors also test how the training increases the knowledge of training attendees by selecting several attendees before the training and administering a short quiz to them, then selecting several of the attendees to re-take the quiz after the training.61 [Sometimes] Visits or calls to communities and schools before Deworming Day. For some of its programs, Deworm the World asks monitors to visit or call, unannounced, a random selection of schools and/or communities before Deworming Day.62 At the schools, monitors will interview teachers and/or headmasters, asking a variety of questions to assess preparedness, such as whether or not the school has enough treatments for Deworming Day and if a representative from the school attended training.63 In communities, monitors select a sample of adults to interview, asking questions to assess their awareness of Deworming Day, which helps Deworm the World determine how successful its community sensitization efforts were.64 Observe activities on Deworming Day and Mop-Up Day. In all of its programs, Deworm the World sends its monitors, unannounced, to observe a random sample of schools on Deworming Day and Mop-Up Day.65 At the schools, monitors interview teachers and school administrators to assess how prepared the school was for Deworming Day. For example, monitors often ask if the school has sufficient drugs for Deworming Day, whether or not a school representative attended training, and a variety of questions to test teachers' knowledge about the proper procedures for the MDA campaign, such as what the teacher should do if a child is feeling sick.66 Then, the monitors observe randomly selected classes, recording details about the Deworming Day activities, such as whether deworming is in progress, teachers are documenting who is dewormed, and teachers are watching to make sure that students swallow the pills.67 Conduct a coverage validation exercise. In two of its three largest programs, Deworm the World has typically sent out monitors within 1 to 2 weeks (though sometimes longer) of Deworming Day and Mop-Up Day to conduct a coverage validation exercise; it has more recently begun doing so in the third program as well.68 In India, coverage validation takes place at a randomly selected sample of schools. In Kenya and Nigeria, coverage validation takes place at households as well as schools. We have not yet seen monitoring results from Pakistan because the program began recently; Deworm the World tells us Pakistan uses the same methodology as Kenya and Nigeria.69 At the schools in India, monitors ask teachers questions about Deworming Day, such as if the school had sufficient tablets.70 During school-based coverage validation in all countries, monitors generally check class registers and record the number of students that were dewormed according to the school's records.71 (They don't appear to have done so in Nigeria as part of coverage validation.) Finally, they randomly select a small sample of students to interview, asking the children several questions about their experience on Deworming Day.72 For example, they ask if the child received a pill and if the child swallowed the pill.73 In India, Deworm the World interviews only children who were present on either National Deworming Day or Mop-up Day in schools where deworming happened on at least one of those days.74 Attendance data is collected separately in India to assess the rate of absenteeism on Deworming Day and Mop-up Day.75 In Kenya, Nigeria, and Vietnam, Deworm the World interviews a sample of all children, regardless of whether they were present on Deworming Day or Mop-up Day.76 The last two types of monitoring are especially valuable for our understanding of how well the programs are operating. We note that Deworm the World did not support coverage validation monitoring for the first three MDAs it supported in Cross River State or for the first MDAs it supported in Ogun, Oyo, and Rivers States in Nigeria.77 Deworm the World does follow-up calls and in-person backchecks to audit monitors' work in Kenya and India. In Nigeria, it conducts in-person backchecks.78 While we believe that Deworm the World's monitoring is overall of high quality, we note a few ways in which its choice of methods leave us with questions about how to interpret the results: Deworm the World's program and monitoring are generally done through schools, which makes it difficult to know how many non-enrolled children are potentially being missed by the program. Schools are supposed to target and record the treatments they distribute to non-enrolled children, but most locations do not have reliable data on how many non-enrolled children there are.79 In Nigeria and Kenya, Deworm the World's coverage validation has taken place in schools as well as in the community, in order to reach non-enrolled children.80 Deworm the World notes that it has piloted community-based coverage evaluation surveys, in addition to school-based monitoring, in a few states in India and will share results when they are available.81 Even though school staff are not aware ahead of time that a monitor is coming to visit, once a monitor shows up, they may be motivated to execute the program in a more rigorous fashion than they would have otherwise. This potential bias is one of the reasons that we also rely on coverage validation that takes place in a different sample of schools after Deworming Day when assessing the quality of Deworm the World's programs. Reports we have seen indicate that Deworm the World successfully observes the deworming process in the vast majority of schools it visits in Nigeria and Vietnam, but only observes deworming in approximately half of the schools it visits in India due to deworming not being in progress at the time the monitors conduct their observations, either because it has already been completed (which Deworm the World notes may often be the case), because it has not started yet, or because it is not taking place that day.82 We have seen some data on this from Kenya but have not reviewed it in depth.83 It is possible that children feel pressure to say that they took the deworming pill on coverage validation day. Coverage validation has generally found very high proportions of students reporting being dewormed, with a coverage rate in one survey of 100%.84 Very high coverage numbers may be partially explained by the fact that in India, Deworm the World interviews only children who were present on either National Deworming Day or Mop-up Day in schools where deworming occurred on at least one of those days; see footnote for clarification from Deworm the World.85 In Kenya and Vietnam, Deworm the World interviews children regardless of whether or not they were present on Deworming Day or Mop-up Day.86 Coverage rates in Nigeria have been lower. One possible explanation for the difference in coverage rates is that in Nigeria, Deworm the World interviews children at home as well as at school, so it may reach children who were not present in school when deworming happened.87 Deworm the World tries to ask students away from their teachers, to reduce pressure, and also has some checks on students' answers.88 In Nigeria when children are interviewed at home rather than at school, their parents are usually present during the interview.89 Results from monitoring We find the evidence produced by Deworm the World from observations on Deworming Day and during the coverage validation exercise to be compelling support for the claim that children receive and ingest pills during Deworm the World-supported MDAs; we have laid out this evidence in this spreadsheet. We also see some reason to interpret these data cautiously, as some of it was self-reported by people who could be biased to want favorable monitoring results. We note that some of the results from Kenya and Vietnam seem quite high to us when compared to the results we have seen from India and Nigeria. We do not know if this is due to some bias in the way that the Kenya and Vietnam data are collected, or if Deworm the World's programs in Kenya and Vietnam are simply higher-quality.90 See this spreadsheet for a sample of methods used in and results from recent monitoring in Kenya, India, Nigeria, Vietnam, and Ethiopia. The spreadsheet does not include all data the monitors collected. We have selected questions that we found particularly relevant to assessing the quality of the programs. (Note: Previously, we summarized monitoring from the 2013-2014 programs and the 2012 programs). We have not prioritized reviewing the data from the monitoring Deworm the World conducts before Deworming Day, although we think it could also provide insight as to the quality of Deworm the World's program. However, we have included some metrics in our table related to the preparedness of schools and their adherence to proper protocols (from questions that are asked on Deworming Day). Based on the data and reports we have seen, we feel fairly confident that most schools in India, Kenya, Nigeria, and Vietnam are prepared to implement Deworming Days: most schools seem to have a sufficient number of deworming pills available for children; fewer appear to have adequately trained teachers (the fact that the surveys identified low training rates in several cases increases our confidence in the reliability of the surveys). We feel less confident that schools are following proper procedures, especially with regards to accurate reporting. Evidence from prevalence surveys over time Deworm the World or its partners have conducted surveys in Kenya to track changes in schistosomiasis and STH prevalence and intensity rates following Deworm the World-supported treatment programs. In general, prevalence and intensity of the parasites decreased over time that the national deworming program was active. Our understanding is that the most likely cause of this decrease was the national deworming program, which Deworm the World supports; alternative explanations for the decrease, such as the existence of other large-scale deworming programs or widespread changes in water and sanitation facilities or practices, seem to us to be unlikely. Deworm the World has also conducted two surveys of prevalence in Bihar, India, before and after treatment; the results from the surveys are not directly comparable, but point in the direction of the program having an effect on prevalence levels. We discuss methodology, results, and limitations to using these surveys as evidence of Deworm the World's impact on another page. How does Deworm the World affect program outcomes? Deworm the World may be having an impact in the following ways: It may increase the likelihood that a government implements a deworming program, by advocating for deworming programs, by offering to provide technical assistance, and/or by funding implementation. Deworm the World expects to pay for the majority of financial program costs in many of the new countries to which it has expanded or intends to expand.91 In situations where Deworm the World is funding a deworming program, we believe it is likely that Deworm the World plays an instrumental role in causing the program to happen.92 Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World. It may improve the quality of a deworming program that would have been implemented without Deworm the World (leading to more children dewormed effectively or improved cost-effectiveness). Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are uncertain about this. Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World. Are there any negative or offsetting impacts? We discuss several possible considerations but do not see significant concerns. Administering deworming drugs seems to be a relatively straightforward program.93 However, there are potential issues that could reduce the effectiveness of some treatments, such as: Drug quality: For example, if drugs are not stored properly they may lose effectiveness or expire.94 Our understanding is that Deworm the World periodically tests the quality of drugs and has monitored storage conditions in each of its recent programs, and this information suggests there have been minimal issues.95 In India, state governments are responsible for procuring the drugs from local suppliers and quality testing them.96 Deworm the World told us that it has conducted additional testing of drugs in India and found that they meet national standards but may not fully meet international standards. It is working with the government to determine next steps.97 Dosage: If the incorrect dosage is given, the drugs may not have the intended effect and/or children may experience additional side effects.98 It appears that for STH treatment, all children of a given age group are given the same dose of albendazole and that the dose is generally a single tablet for children 2 years old and above, and half a tablet for those between the ages of 1 and 2.99 Deworm the World monitors reported in 2015 (we haven't seen more recent data) that in programs in India, up to 9% of schools observed gave children less than the prescribed dose of albendazole and up to 5% gave more than the prescribed dose of albendazole.100 Replacement of government funding: We have limited information about whether governments would pay for the parts of the program paid for by Deworm the World in its absence, though our impression is that they would not.101 We also have little information about what governments would use deworming funds for if they did not choose to implement deworming programs. Diversion of skilled labor: Drug distribution occurs only once or twice per year and is conducted by teachers in schools. Based on our site visit in Rajasthan, our impression is that the Nodal Officer (the state official who manages all state school-based programs), the Nodal Officer’s staff, and the people that the Nodal Officer manages throughout the state (Resource Persons and Community Development Project Officers) have significant capacity to take on additional programs, so their taking on this program doesn't impose a significant burden on their time.102 On the other hand, a principal we spoke with commented that he would prefer fewer school-based health programs because they take focus away from the school day.103 Adverse effects and unintended consequences of taking deworming drugs: Our understanding is that expected side effects are minimal and there is little reason to be concerned that drug resistance is currently a major issue. We note that fainting was common in at least one MDA supported by Deworm the World; this MDA included praziquantel for schistosomiasis, and Deworm the World attributes the fainting cases to students taking the drug with empty stomachs.104 More information in our report on deworming. Popular discontent: We have heard a couple of accounts of discontent in response to mass drug administration campaigns supported by the Schistosomiasis Control Initiative, including one case that led to riots.105 Additionally, during deworming activities supported by Evidence Action's Deworm the World Initiative in Ogun State, Nigeria in December 2017, rumors of students collapsing reportedly generated panic that led some parents to take their children out of school; the Ogun State government denied that any students collapsed.106 During deworming activities supported by Deworm the World in Oyo State, Nigeria in February 2018, rumors of students reacting badly or dying after receiving deworming drugs spread, and several schools canceled Deworming Day; an investigation found that the children likely died of food poisoning unrelated to deworming.107 What do you get for your dollar? This section examines the data that we have to inform our estimate of the expected cost-effectiveness of additional donations to Deworm the World. Note that the number of lives significantly improved is a function of a number of difficult-to-estimate factors. We incorporate these into a cost-effectiveness model which is available here. In this section, we focus on the cost per child treated, which is an important input in our cost-effectiveness model. On a separate page, we discuss: How accurate are Deworm the World's reported coverage figures? Deworm the World uses government-reported figures on the number of treatments delivered when calculating its cost per treatment. We remain uncertain of the accuracy of these figures. More here. Does Deworm the World "leverage" government funds, such that its activities mobilize resources from other actors? We could imagine that Deworm the World's funds have substantial leverage but could also imagine that other actors’ involvement is causing Deworm the World to pay for things for which other actors would otherwise have paid. Deworm the World may have less leverage in its future programs than it has had in past programs. More here. What is the cost per treatment? We estimate that in India children are dewormed for a total of about $0.35 per child, or $0.10 per child excluding the value of teachers’ and principals’ time spent on the program. In Kenya, we estimate the total cost per treatment at about $0.66 per treatment or $0.46 excluding the value of teachers’ and principals’ time spent on the program. We expect the cost per treatment in the countries Deworm the World has expanded to recently and may expand to in the future to be closer to Kenya's costs, but higher at first than Kenya due to additional costs associated with starting a new program. Deworm the World told us that the cost per treatment in India is unusually low.108 We note that Deworm the World's cost per child analysis from Cross River, Nigeria for 2016 was about 50% higher than its estimate for Kenya in 2015-2016.109 Full details in this spreadsheet. Our approach Our cost per treatment analyses rely heavily on analyses Deworm the World has done to allocate its own costs across programs, collect data on costs paid by other partners and governments, and estimate costs when data is not available. Deworm the World has told us that it tries to capture all costs of its program, regardless of who pays for the cost (including donated drugs). We have made several adjustments to Deworm the World's estimates: For India, about two thirds of the total cost comes from contributions of time from government employees—which have been monetized according to salary levels—rather than financial costs. Appropriately valuing that time spent, and estimating how much time is spent by teachers and others due to the deworming program, is likely the largest source of uncertainty in the cost-per-child-treated calculation for India.110 In its own calculations, Deworm the World excludes the value of government employees' time because the government would have incurred these costs in the absence of the program.111 There are a few high-level costs not directly attributable to programs that Deworm the World does not include (such as exploratory work in new geographies that does not lead to a new program).112 We have included an estimate of these costs in our analyses. For Kenya, Nigeria, and Vietnam we have included an estimate of in-kind contributions from governments. This estimate is very rough and uses the same approach that we use in estimating the total cost per treatment for SCI. In short, we assume that government contributions account for 30% of the cost of the program, based on a 2011 paper that analyzed the costs of an SCI program in Niger.113 We adjust the number of treatments reported by Deworm the World-supported programs by the attendance and coverage rates found in Deworm the World's monitoring. Details of this adjustment are in this spreadsheet, "No. of children dewormed" sheets. We start with this total cost figure and apply adjustments in our cost-effectiveness analysis to account for cases where we believe the charity's funds have caused other actors to shift funds from a less cost-effective use to a more cost-effective use ("leverage") or from a more cost-effective use to a less cost-effective use ("funging"). Is there room for more funding? We believe that Deworm the World could productively use more funding than it expects to receive. We estimate that its funding gap for the next three years, which we estimate at $3.5 million, is small relative to the size of its budget if fully funded (about $54 million over three years). In short: Total opportunities to spend funds productively: Deworm the World has identified as-yet-unfunded opportunities to maintain and expand its work over the next three years (2019-2021) totaling $16.0 million. Cash on hand: As of the end of June 2019, Deworm the World held no uncommitted funds. It holds $43.5 million that has been committed to projects or reserves ($10.5 million is held in reserve). Expected additional funding: Evidence Action will be transferring about $6.8 million to Deworm the World as a result of the closure of its No Lean Season program. We estimate that Deworm the World will receive an additional $5.7 million in funding for the work included in this analysis in the next three years. In sum, we expect Deworm the World to have opportunities to spend $3.5 million more than we expect it to receive over the next three years.114 Funding beyond this level would allow Deworm the World to build its reserves. It currently holds $10.5 million in reserve and is seeking to increase its reserves by $7.5 million. Details below and in this spreadsheet. Uncommitted and expected funds At the end of June 2019, Deworm the World held $43.5 million, all of which was committed to programs for 2019-2022 or its reserves.115 We expect that Deworm the World will receive additional donations over the next three years from the following sources. Note that we have excluded funding that is restricted to specific programs because this amount is excluded from both the expected funds calculation and the list of spending opportunities. Donors who are not influenced by GiveWell's research: Based on what it has received in the past year from donors we do not attribute to GiveWell's recommendation and earned in interest on funds that it holds, we project that Deworm the World will receive roughly $1.1 million per year, for a total of $3.3 million over three years, in flexible funding that is independent of GiveWell's recommendation.116 Donors who give based on GiveWell's top charity list, but do not follow our recommendation for marginal funding: GiveWell maintains both a list of all top charities that meet our criteria and a recommendation for which charity or charities to give to in order to maximize the impact of additional donations, given the cost-effectiveness of remaining funding gaps. We estimate that Deworm the World will receive about $1.5 million over the next year from donors who use our top charity list but don't follow our recommendation for marginal donations.117 We include only one year of funding of this type in order not to make the assumption, for the purpose of the room for more funding analysis, that charities will continue to be recommended and receive this funding. This is because we aim to structure GiveWell-directed funding such that, if we were to stop recommending the organization next year, it would have one to two years of funding available to allow it to adjust to the loss in funding. In addition, in June 2019, Evidence Action announced that it would be closing its No Lean Season program (more in this blog post). We have asked Evidence Action to redirect the remaining funding it received from GiveWell, including funding from Good Ventures, for No Lean Season to Deworm the World. Evidence Action expects to transfer $6.8 million from No Lean Season to Deworm the World. In sum, we estimate that, including $0.9 million that was held by GiveWell for supporting Deworm the World as of this analysis, Deworm the World will have about $12.5 million in flexible funding for 2020-2022.118 Our understanding is that Evidence Action has limited unrestricted funding, has not allocated unrestricted funding to Deworm the World in recent years, and is unlikely to do so in the next few years.119 Additional spending opportunities Deworm the World is seeking funding to continue programs in Kenya, India, Pakistan, and Nigeria and believes it may have opportunities to expand its work in Nigeria. It is also seeking funding to expand its reserves. We ask top charities to consider GiveWell-directed funds to be multi-year grants. The amount of GiveWell-directed funding that a top charity receives can vary greatly from year to year, and spending the funds over two to three years can help smooth these fluctuations. Details of Deworm the World's additional spending opportunities are in this spreadsheet, sheet "Source: DtWI RFMF." In short: Nigeria: Deworm the World has funding to support deworming in Ogun State through 2021, Oyo and Rivers States through 2020 and in Cross River State through 2019. It is seeking funding for three types of work in Nigeria, totaling $5.6 million: (a) extending funding available for Oyo, Ogun, and Rivers States to 2022 ($2.8 million),120 (c) expanding to one additional state, Lagos State ($1.5 million) and (d) national-level costs in 2020-2022, including providing technical support for the Federal Ministry of Health and centralized costs for supporting state programs ($1.3 million). Deworm the World is not continuing its work on Cross River State after 2019.121 India: Deworm the World is seeking $4.4 million for national-level costs, primarily in 2021-2022 ($1.9 million); supplementing funding for states that may continue to receive restricted support from other funders in 2021-2022 ($1.4 million); continuing to fund work in Haryana, Rajasthan, Karnataka, Madhya Pradesh, and Uttarakhand through 2022 ($0.6 million); and funding prevalence surveys in these five states ($0.5 million).122 Kenya: In 2017, Deworm the World signed an agreement with the government of Kenya to continue funding deworming in Kenya through 2022.123 Deworm the World has received funding for its work in Kenya through 2021, primarily from GiveWell and the END Fund. Taking into account the possibility of continued funding from the END Fund, Deworm the World needs $2.3 million to fund the program through 2022.124 Pakistan: Deworm the World has allocated some flexible funding to fund deworming in Pakistan in 2020-2021 and, as of August 2019, was in the late stages of discussion with Dubai Cares for a grant of $3 million for Pakistan. It is seeking $1.5 million to fill the remaining funding gap for the program through 2022. Global team costs: Deworm the World is seeking $2.1 million for extending funding available for its global team (staff who work across multiple country programs) costs to 2022. The total of the above is $16.0 million. In addition, Deworm the World would like to substantially increase its reserves. Its work is highly dependent on GiveWell-directed funding. If GiveWell were to stop recommending funding to Deworm the World in any given year, Deworm the World would generally have one to two years of funding remaining for its programs. Given the size of its programs, Deworm the World believes it would be unlikely to be able to replace much of this funding via another source and it is concerned that scaling down programs in that time would carry major costs for Deworm the World's impact and Evidence Action's reputation.125 It is seeking $7.5 million to increase its reserves to 100% of its annual budget.126 If it is not fully funded, Deworm the World expects it will prioritize in the following order:127 First-tier priorities, totaling $11.3 million, which we expect will be fully funded with available and expected funding: Extending the funding runway in all locations where it currently operates to 2021 and to 2022 in Nigeria and Kenya, and starting a new program in Lagos State, Nigeria. Deworm the World told us that it views expansion to Lagos as a relatively high priority because it believes it is well-positioned to work in Lagos based on its experience in neighboring Ogun State and because achieving greater overall scale in Nigeria will decrease its cost per child treated in the country.128 Second-tier priority, totaling $2.9 million, of which we expect there to be a funding gap of $1.7 million: Extending the funding runway for India to 2022. Third-tier priority, totaling $1.5 million excluding reserves and $9.3 million including reserves, which we expect to be a funding gap based on projections of available and expected funding: Extending the funding runway for Pakistan to 2022. Deworm the World told us that extending the funding runway in Pakistan is currently a lower priority than the first and second tier activities due to a) some uncertainty as to the level of additional funding that will be needed, which is dependent on how quickly it scales up in the country, and b) its agreements with its partners not being due for renewal in the next year.129 Deworm the World also includes increasing its reserves as a third-tier priority. A note about reserve funding We have not yet applied a consistent approach to how we account for organizations holding and building up funding reserves in our analyses of their room for more funding. We expect to improve on our approach in the future. We consider room for more funding on a three-year basis in order to mitigate the risk to the organization of fluctuations in GiveWell-directed funding. Some organizations have a policy of holding reserves in addition to this three-year funding. Possibilities of approaches we could take in considering reserves in the context of our room for more funding analyses include: Refer to the organization's reserve policy to determine how much funding it should set aside as reserves. Evidence Action adopted a policy in 2018 to build reserves for each program, over the course of five years, up to the level at which reserves would be sufficient to cover one year of expenses.130 Consider reserve funding to be available for supporting program costs in the third year of our room for more funding analysis. Vary the time period included in our room for more funding analysis based on organization-specific factors such as what portion of an organization's spending are costs that take time to wind down (for example, core staff salaries) and costs that can be scaled down quickly (for example, drug purchases), and/or the percent of the organization's budget that is supported by GiveWell-directed funds. An approach like this, if designed well, would better achieve the goal of mitigating risk to the organization, but would likely add significant complexity to the analyses. Global need for treatment There appears to be a substantial unmet need for STH and schistosomiasis treatment globally. In 2017, the World Health Organization (WHO) released a report on 2016 treatments stating that:131 69% of school-age children in need of treatment were treated for STH in 2016, up from 63% in 2015 and 45% in 2014. Coverage was 65% in African countries in 2016. 52% of school-age children in need of treatment were treated for schistosomiasis in 2016, up from 42% in 2015. We have not vetted this data. Evidence Action as an organization We use qualitative assessments of our top charities to inform our funding recommendations. See this page for more information about this process and for our qualitative assessment of Evidence Action as an organization. Sources Document Source Alderman et al. 2006 Source (archive) Alix Zwane conversation August 30th 2013 Unpublished Alix Zwane conversation June 4th 2013 Source Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013 Unpublished Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 Unpublished Allen and Parker 2011 Source (archive) Assam 2010 guidelines for deworming Source Assam midday meal report 2013 Source (archive) Assam reproductive and child health 2011-2012 Source (archive) Assam state programme implementation plan 2011-2012 Source (archive) Baird et al 2012 Source Bleakley 2007 Source (archive) CIFF conversation September 10th 2013 Source Croke 2014 Source (archive) Devesh Kapur conversation October 14th 2013 Source Deworm the World 2015 Uttar Pradesh prevalence survey report Source Deworm the World and SCI, Ethiopia coverage survey 2015 Source Deworm the World Chhattisgarh Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World External Bihar 2016 Costing Model Source Deworm the World External Chhattisgarh 2016 Costing Model Source Deworm the World External Cross River Nigeria 2016 Costing Model Source Deworm the World External Kenya 2016 Costing Model Source Deworm the World External Madhya Pradesh 2016 Costing Model Source Deworm the World External Rajasthan 2016 Costing Model Source Deworm the World External Telangana 2016 Costing Model Source Deworm the World External Tripura 2016 Costing Model Source Deworm the World External Uttar Pradesh 2016 Costing Model Source Deworm the World Financials 2017 Source Deworm the World Jharkhand Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Kenya Year 4 Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Nigeria Cross River Independent Process Monitoring Report, November 2016 Source Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Unpublished Deworm the World target and treatment figures 2018 Source Deworm the World Telangana Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Tripura Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Uttar Pradesh Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Vietnam Independent Monitoring Report, November-December 2016 Source Deworm the World, 2015 expense summary Unpublished Deworm the World, 2015 expense summary - by funder Unpublished Deworm the World, Bihar 2016 IMCV report Source Deworm the World, Bihar cost per child model 2017 Source Deworm the World, Bihar cost per child model 2018 Source Deworm the World, Bihar NDD PMCV Report Appendix, August 2017 Source Deworm the World, Bihar NDD PMCV report, August 2018 Source Deworm the World, Bihar NDD PMCV Report, February 2017 Source Deworm the World, Bihar NDD PMCV report, February 2018 Source Deworm the World, Bihar NDD Program Report, August 2017 Source Deworm the World, Bihar NDD Program Report, February 2017 Source Deworm the World, Bihar PMCV report, August 2017 Source Deworm the World, Chhattisgarh 2016 IMCV report Source Deworm the World, Chhattisgarh cost per child model 2017 Source Deworm the World, Chhattisgarh cost per child model 2018 Source Deworm the World, Chhattisgarh NDD PMCV Report Appendix, August 2017 Source Deworm the World, Chhattisgarh NDD PMCV report, August 2018 Source Deworm the World, Chhattisgarh NDD PMCV Report, February 2017 Source Deworm the World, Chhattisgarh NDD PMCV report, February 2018 Source Deworm the World, Chhattisgarh NDD Program Report, August 2017 Source Deworm the World, Chhattisgarh NDD Program Report, February 2017 Source Deworm the World, Chhattisgarh PMCV report, August 2017 Source Deworm the World, Chhattisgarh prevalence survey report, August 2016 Source Deworm the World, Chhattisgarh Program Report, August 2016 Source Deworm the World, Cross River State PM report, May 2018 Source Deworm the World, Cross River State PMCV report, October 2018 Source Deworm the World, Cross River State, Nigeria Coverage Evaluation Survey 2017 Unpublished Deworm the World, Cross River State, Nigeria, 2016-2017 Treatment Data Source Deworm the World, Cross River State, Nigeria, Year 2 Round 1 PMCV report, June 2017 Source Deworm the World, Ethiopia independent monitoring report, Year 1 Source Deworm the World, Ethiopia prevalence survey report Source Deworm the World, Ethiopia Workplan Unpublished Deworm the World, Haryana NDD CV report, February 2018 Source Deworm the World, Haryana NDD PMCV report, August 2018 Source Deworm the World, Jharkhand cost per child model 2017 Source Deworm the World, Jharkhand cost per child model 2018 Source Deworm the World, Jharkhand Independent Monitoring Report 2016 Unpublished Deworm the World, Jharkhand NDD Brief Report, August 2016 Source Deworm the World, Jharkhand NDD PMCV Report Appendix, August 2017 Source Deworm the World, Jharkhand NDD PMCV report, August 2018 Source Deworm the World, Jharkhand NDD PMCV Report, February 2017 Source Deworm the World, Jharkhand NDD PMCV report, February 2018 Source Deworm the World, Jharkhand NDD Program Report, August 2017 Source Deworm the World, Jharkhand NDD Program Report, February 2017 Source Deworm the World, Jharkhand PMCV report, August 2017 Source Deworm the World, Karnataka cost per child model 2018 Source Deworm the World, Karnataka NDD PMCV Report Appendix, August 2017 Source Deworm the World, Karnataka NDD PMCV report, August 2018 Source Deworm the World, Karnataka NDD PMCV report, February 2018 Source Deworm the World, Karnataka PMCV report, August 2017 Source Deworm the World, Kenya 2014-2015 program report Source Deworm the World, Kenya cost per child model 2017 Source Deworm the World, Kenya cost per child model 2018 Source Deworm the World, Kenya Coverage Reporting data, Year 3 Source Deworm the World, Kenya Deworming Day data, Year 3 Source Deworm the World, Kenya Narrative Report - Year 1 Source Deworm the World, Kenya Narrative Report - Year 2, Quarter 4 Source Deworm the World, Kenya Narrative Report - Year 3, Quarter 3 Source Deworm the World, Kenya NSBD PMCV Report Year 5 Source Deworm the World, Kenya NSBD PMCV report, Y6 2018 Source Deworm the World, Kenya NSBD Program Report Year 5 Source Deworm the World, Kenya NSBDP Coverage Validation Report Year 4 Source Deworm the World, Kenya NSBDP PMCV Report Year 5, CIFF Source Deworm the World, Kenya NSBDP PMCV Report, END Fund, July 2017, Year 5 Wave 1 Source Deworm the World, Kenya NSBDP PMCV Trends, Years 2-5, CIFF Source Deworm the World, Kenya PMCV Report Year 4 2016, Wave 1 Source Deworm the World, Kenya PMCV Report Year 4 2016, Wave 2 Source Deworm the World, Kenya process monitoring report, Year 4 Source Deworm the World, Kenya STH and Schistosomiasis Treatment Report Year 3 2015 Source Deworm the World, Kenya STH and Schistosomiasis Treatment Report Year 4 2016 Source Deworm the World, Kenya Year 2, DD - Main instrument Source Deworm the World, Kenya Year 2, Pre DD - School instrument Source Deworm the World, Kenya Year 3, DD - Main instrument Source Deworm the World, Kenya Year 3, Post DD - Coverage instrument Source Deworm the World, Kenya Year 3, Pre TT form Source Deworm the World, Madhya Pradesh 2015 prevalence survey report Source Deworm the World, Madhya Pradesh 2016 IMCV report Source Deworm the World, Madhya Pradesh cost per child model 2017 Source Deworm the World, Madhya Pradesh cost per child model 2018 Source Deworm the World, Madhya Pradesh cost-per-treatment - 2015 Unpublished Deworm the World, Madhya Pradesh NDD PMCV Report, February 2017 Source Deworm the World, Madhya Pradesh NDD PMCV report, February 2018 Source Deworm the World, Madhya Pradesh NDD Program Report, February 2017 Source Deworm the World, National Deworming Day states, August 2016 Source Deworm the World, Nigeria cost per child model 2017 Source Deworm the World, Nigeria cost per child model 2018 Source Deworm the World, Ogun State PM report, June 2018 Source Deworm the World, Oyo State PM report, February 2018 Source Deworm the World, Oyo State PMCV report, October 2018 Source Deworm the World, Rajasthan 2016 IMCV report Source Deworm the World, Rajasthan cost per child model 2017 Source Deworm the World, Rajasthan cost per child model 2018 Source Deworm the World, Rajasthan NDD PMCV Report, February 2017 Source Deworm the World, Rajasthan NDD PMCV report, February 2018 Source Deworm the World, Rajasthan NDD Program Report, February 2017 Source Deworm the World, responses to GiveWell questions, May 16, 2019 Unpublished Deworm the World, Rivers State PM report, June 2018 Source Deworm the World, Rivers State PMCV report, November 2018 Source Deworm the World, target populations and parasitology data Source Deworm the World, Telangana 2016 IMCV report Source Deworm the World, Telangana cost per child model 2017 Source Deworm the World, Telangana cost per child model 2018 Source Deworm the World, Telangana NDD PMCV Report Appendix, August 2017 Source Deworm the World, Telangana NDD PMCV Report, February 2017 Source Deworm the World, Telangana NDD PMCV report, February 2018 Source Deworm the World, Telangana NDD Program Report, August 2017 Source Deworm the World, Telangana NDD Program Report, February 2017 Source Deworm the World, Telangana PMCV report, August 2017 Source Deworm the World, Telangana Program Report, August 2016 Source Deworm the World, Tripura 2016 IMCV report Source Deworm the World, Tripura cost per child model 2017 Source Deworm the World, Tripura cost per child model 2018 Source Deworm the World, Tripura NDD PMCV Report Appendix, August 2017 Source Deworm the World, Tripura NDD PMCV report, August 2018 Source Deworm the World, Tripura NDD PMCV Report, February 2017 Source Deworm the World, Tripura NDD PMCV report, January 2018 Source Deworm the World, Tripura NDD Program Report, August 2017 Source Deworm the World, Tripura NDD Program Report, February 2017 Source Deworm the World, Tripura PMCV report, August 2017 Source Deworm the World, Tripura PMCV Report, February 2016 Source Deworm the World, Tripura Program Report, August 2016 Source Deworm the World, Uttar Pradesh 2016 IMCV report Source Deworm the World, Uttar Pradesh cost per child model 2017 Source Deworm the World, Uttar Pradesh cost per child model 2018 Source Deworm the World, Uttar Pradesh NDD PMCV Report Appendix, August 2017 Source Deworm the World, Uttar Pradesh NDD PMCV report, August 2018 Source Deworm the World, Uttar Pradesh NDD PMCV Report, February 2017 Source Deworm the World, Uttar Pradesh NDD PMCV report, February 2018 Source Deworm the World, Uttar Pradesh NDD Program Report, August 2017 Source Deworm the World, Uttar Pradesh NDD Program Report, February 2017 Source Deworm the World, Uttar Pradesh NDD Program Report, September 2016 Source Deworm the World, Uttar Pradesh PMCV report, August 2017 Source Deworm the World, Uttar Pradesh prevalence survey report, December 2015 Source Deworm the World, Uttarakhand NDD CV report, February 2018 Source Deworm the World, Uttarakhand NDD PMCV report, August 2018 Source Deworm the World, Vietnam 2016 monitoring survey form for coverage validation Source Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day Source Deworm the World, Vietnam baseline prevalence survey - 4 provinces Source Deworm the World, Vietnam cost per child model 2017 Source Deworm the World, Vietnam cost per child model 2018 Source Deworm the World, Vietnam final report for STH survey in 21 provinces Source Deworm the World, Vietnam IM Report on MDA 1, April 2016 Source Deworm the World, Vietnam IM Report on MDA 2, November 2016 Source Deworm the World, Vietnam Independent Monitoring Report on MDA 3, November 2017 Source Deworm the World, Vietnam Independent Monitoring Report, 2016 Source Deworm the World, Vietnam NIMPE Report on MDA 1, April 2016 Source Deworm the World, Vietnam NIMPE Report on MDA 2, November 2016 Source Deworm the World, Vietnam NIMPE Report on MDA 3, November 2017 Source DSW 2012 GiveWell site visit Source DtWI 2013 GiveWell government interviews Source DtWI 2013 GiveWell site visit Source DtWI Assam research 2013 Source DtWI Bihar 2011 cost data Source DtWI Bihar 2011 coverage data Source DtWI Bihar 2011 monitoring data for deworming day Source DtWI Bihar 2011 monitoring data for mop-up day Source DtWI Bihar 2011 Monitoring Form for Deworming Day Source DtWI Bihar 2011 monitoring form for mop-up day Source DtWI Bihar 2011 prevalence survey report Source DtWI Bihar 2011 program report Source DtWI Bihar 2012 cost data Source DtWI Bihar 2012 cost data details Unpublished DtWI Bihar 2012 coverage data Source DtWI Bihar 2012 monitoring data for coverage validation Source DtWI Bihar 2012 monitoring report Source DtWI Bihar 2014 cost data Unpublished DtWI Bihar 2014 program report Source DtWI Bihar 2014 program report annex 1 Source DtWI Bihar 2014 program report annex 2 Source DtWI Bihar 2015 independent monitoring tables Source DtWI Bihar 2015 monitoring data for coverage validation, schools Source DtWI Bihar 2015 monitoring data from deworming day, schools Source DtWI Bihar 2015 monitoring data from mopup day, schools Source DtWI Bihar 2015 monitoring survey for coverage validation, schools Source DtWI Bihar 2015 monitoring survey from deworming day, schools Source DtWI Bihar 2015 monitoring survey from mopup day, schools Source DtWI Bihar 2015 Prevalence Survey report Source DtWI Bihar 2015 Program report Source DtWI Bihar Feb 2016 Program report Source DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015 Unpublished DtWI Chhattisgarh 2015 coverage validation report Source DtWI Chhattisgarh 2015 coverage validation tables Unpublished DtWI Chhattisgarh 2015 independent monitoring tables Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools Source DtWI Chhattisgarh Feb 2016 Program report Source DtWI class register audits 2013 Source DtWI cost narrative 2013 Source DtWI Cost per treatment blog post January 2015 Source (archive) DtWI cost per treatment summary 2013 Source DtWI coverage data 2013 - 2014 Source DtWI Delhi 2012 cost data Source DtWI Delhi 2012 coverage data by anganwadi Source DtWI Delhi 2012 coverage data by school Source DtWI Delhi 2012 coverage report Source DtWI Delhi 2012 monitoring data Source DtWI Delhi 2012 monitoring form deworming day Source DtWI Delhi 2012 prevalence survey design Source DtWI Delhi 2012 prevalence survey report Source DtWI Delhi 2012 program report Source DtWI Delhi 2013 cost data Unpublished DtWI Delhi 2013 program report Source DtWI Delhi 2015 independent monitoring tables Source DtWI Delhi 2015 monitoring data for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring data for coverage validation, schools Source DtWI Delhi 2015 monitoring data from deworming day, anganwadis Source DtWI Delhi 2015 monitoring data from deworming day, schools Source DtWI Delhi 2015 monitoring data from mopup day, anganwadis Source DtWI Delhi 2015 monitoring data from mopup day, schools Source DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring survey for coverage validation, schools Source DtWI Delhi 2015 monitoring survey from deworming day, anganwadis Source DtWI Delhi 2015 monitoring survey from deworming day, schools Source DtWI Delhi 2015 monitoring survey from mopup day, anganwadis Source DtWI Delhi 2015 monitoring survey from mopup day, schools Source DtWI Delhi 2015 program report Source DtWI External Vietnam Costing Model Source DtWI Kenya 2013-2014 cost per treatment data Source DtWI Kenya 2013-2014 program report Source DtWI Kenya 2015-16 program report Source DtWI Madhya Pradesh 2015 coverage validation form Unpublished DtWI Madhya Pradesh 2015 deworming day monitoring form Unpublished DtWI Madhya Pradesh 2015 independent monitoring tables Source DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools Source DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools Source DtWI Madhya Pradesh 2015 program report Source DtWI Madhya Pradesh Feb 2016 program report Source DtWI Monitoring Improvements 2014 Source DtWI NDD blog post February 2015 Source (archive) DtWI NDD Year 1 M&E review July 2015 Source DtWI Rajasthan 2012 cost data Source DtWI Rajasthan 2012 cost data details Unpublished DtWI Rajasthan 2012 coverage data for anganwadi Source DtWI Rajasthan 2012 coverage data for schools Source DtWI Rajasthan 2012 monitoring data for coverage validation in anganwadis Source DtWI Rajasthan 2012 monitoring data for coverage validation in schools Source DtWI Rajasthan 2012 monitoring form coverage day Source DtWI Rajasthan 2012 monitoring form pre-deworming day Source DtWI Rajasthan 2012 monitoring report Source DtWI Rajasthan 2012 prevalence survey report Source DtWI Rajasthan 2013 cost data Unpublished DtWI Rajasthan 2013 prevalence survey report Source DtWI Rajasthan 2013 program report Source DtWI Rajasthan 2015 independent monitoring tables Source DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring data for coverage validation, schools Source DtWI Rajasthan 2015 monitoring data from deworming day, schools Source DtWI Rajasthan 2015 monitoring data from mopup day, schools Source DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring survey for coverage validation, schools Source DtWI Rajasthan 2015 monitoring survey from deworming day, schools Source DtWI Rajasthan 2015 monitoring survey from mopup day, schools Source DtWI Rajasthan 2015 program report Source DtWI Rajasthan Feb 2016 Program report Source DtWI Telangana Feb 2016 Program report Source DtWI Uttar Pradesh Feb 2016 Program report Source Evidence Action 2014 budget Unpublished Evidence Action 2015 draft budget Unpublished Evidence Action 2015 funding gap analysis Source Evidence Action 2016-2017 Revenue Report Source Evidence Action 2017 financials, GiveWell submission Source Evidence Action cover letter 2013 Source Evidence Action launch announcement 2013 Source (archive) Evidence Action Q1 financials, 2016 Unpublished Evidence Action website 2013 Source (archive) Evidence Action website announcement April 2014 Source (archive) Evidence Action website, Deworm the World Initiative (December 2016) Source (archive) Evidence Action website, Deworm the World Initiative (March 2016) Source (archive) Evidence Action website, Deworm the World Initiative (October 2015) Source (archive) Evidence Action website, Evidence Action Beta (October 2015) Source (archive) Evidence Action website, Who we are (November 2016) Source (archive) Evidence Action, 2015 financials by program Unpublished Evidence Action, Baseline Survey Report of Soil-Transmitted Helminths Prevalence in Pakistan Source Evidence Action, blog post, April 27, 2016 Source (archive) Evidence Action, blog post, August 1, 2016 Source Evidence Action, blog post, December 21, 2015 Source (archive) Evidence Action, blog post, January 16, 2015 Source (archive) Evidence Action, blog post, January 8, 2015 Source (archive) Evidence Action, blog post, July 5, 2016 Source (archive) Evidence Action, blog post, June 12, 2015 Source (archive) Evidence Action, blog post, June 30, 2016 Source (archive) Evidence Action, Organizational Report 2017-2018, Unaudited Source Evidence Action, Projected allocation of unrestricted funds, 2016 Source Evidence Action, Reserves Policy, 2018 Source Evidence Action, Revenue Report 2017-2018, Unaudited Source GiveWell analysis of Deworm the World 2014 Financial summary Source GiveWell analysis of Deworm the World cost per treatment Source GiveWell analysis of Deworm the World cost-per-treatment, 2016 Source GiveWell analysis of Deworm the World cost-per-treatment, October 2016 Source GiveWell analysis of Deworm the World financials - 2016 Source GiveWell DtWI 2013-2014 cost data summary Source GiveWell enrollment-based student coverage check 2015 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013 Source GiveWell's non-verbatim summary of a conversation with END Fund staff, October 17, 2016 Source GiveWell's notes from site visit to India, October 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014 Source GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014 Source GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 Source Global Alliance to Eliminate Lymphatic Filariasis - Prevention Source (archive) Grace Hollister conversation June 19th 2013 Source Grace Hollister, conversation with GiveWell, April 20, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 11, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 24, 2016 Unpublished Grace Hollister, conversation with GiveWell, July 25, 2016 Unpublished Grace Hollister, conversation with GiveWell, June 13, 2016 Unpublished Grace Hollister, conversation with GiveWell, May 5, 2016 Unpublished Grace Hollister, conversation with GiveWell, September 1, 2016 Unpublished Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Unpublished Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, March 23, 2015 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 Source Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 Unpublished Grace Hollister, edits to GiveWell's review, November 20, 2016 Unpublished Grace Hollister, edits to GiveWell's review, November 7, 2016 Unpublished Grace Hollister, email to GiveWell, June 9, 2016 Unpublished Grace Hollister, email to GiveWell, March 6, 2016 Unpublished Grace Hollister, email to GiveWell, September 13, 2016 Unpublished Harvard Business School Kenya Case Study A 2010 Unpublished India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015 Source India NDD documents 2015 Source (archive) Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014 Unpublished JPAL CEAs in education 2011 Source Kabatereine et al. 2001 Source (archive) KEMRI prevalence report - Year 2 Unpublished Kenya National School Based Deworming Programme Year 4 2016 Impact Analysis Source LF treatment coverage (2004-2014) Source (archive) LF treatment coverage 2015 Unpublished LF treatment drugs 2012 Unpublished Miguel and Kremer 2004 Source Mwandawiro et al. 2013 Source (archive) National Vector Borne Disease Control Programme, All About Lymphatic Filariasis Source (archive) Neetu Chandra Sharma, Daily Mail - India article, August 8, 2016 Source (archive) Nigerian Tribune, "Panic in Ogun schools over deworm exercise," December 2017 Source (archive) Paul Byatta, attachments to email to GiveWell, September 23, 2016 Source Paul Byatta, conversation with GiveWell, September 20, 2016 Unpublished Paul Monaghan, conversation with GiveWell, September 8, 2016 Unpublished Preventive chemotherapy in human helminthiasis 2006 Source (archive) Professor Devesh Kapur Biography 2013 Source (archive) Reserve Bank of India, GDP per capita, Table 10, September 16, 2015 Source (archive) SCI Malawi coverage survey 2012 Source SCI, Ethiopia coverage survey dashboard 2016 Source SCI, Ethiopia coverage survey protocol 2017 Source STH coalition framework for action November 2014 Source (archive) U-DISE Elementary Thematic Maps 2015 Source (archive) U-DISE Secondary Flash Statistics 2015 Source (archive) U-DISE Secondary Thematic Maps 2015 Source (archive) WHO soil-transmitted helminthiases 2012 Source (archive) WHO STH factsheet Source (archive) WHO STH treatment report Source (archive) WHO Weekly epidemiological record, 18 December 2015 Source WHO Weekly epidemiological record, 6 March 2015 Source (archive) WHO, Helminth control in school-age children Source WHO, Helminth control in school-age children second edition Source WHO, Summary of global update on preventive chemotherapy implementation in 2015 Source WHO, Summary of global update on preventive chemotherapy implementation in 2016 Source World Schistosomiasis Risk Chart 2012 Source 1 GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013 Alix Zwane conversation June 4th 2013 Grace Hollister conversation June 19th 2013 Alix Zwane conversation August 30th 2013 Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014 GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014 GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015 GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 Starting in 2016, we deprioritized publishing notes from our conversations with Deworm the World. We continued speaking regularly with Deworm the World staff in 2016-2018. GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017 2 CIFF conversation September 10th 2013. We have also had several additional conversations with CIFF about its work with Deworm the World. 3 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. 4 "More specifically, Evidence Action advocates for school-based deworming to policymakers and provides technical assistance to launch, strengthen and sustain school-based deworming programs." Evidence Action website, Deworm the World Initiative (March 2016) An example of Deworm the World supporting a program with funding: "Deworm the World is working in partnership with Thrive Networks in Vietnam on an integrated program of both deworming and water, sanitation, and hygiene (WASH) education, and this includes an RCT to explore the impact of hygiene education in combination with deworming. This program is funded primarily by Dubai Cares, with Deworm the World slated to provide unrestricted funding for certain program components." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 5. 5 "Is mass treatment justified? On cost-effectiveness grounds we believe that it clearly is, as the cost of treatment is cheaper than individual screening. The WHO states that the cost of screening is four to ten times that of the treatment itself. Because the drugs are very safe and has [sic] no side effects for the uninfected, the WHO does not recommend individual screening. The WHO instead recommends mass drug administration in areas where more than 20% of children are infected." Evidence Action website, Deworm the World Initiative (March 2016) Deworm the World focuses on school-based programs because the highest burdens for STH and schistosomiasis (the two diseases that Deworm the World targets) tend to be observed in children. For this reason, a significant decrease in the worm burden in children for these diseases translates to a significant decrease in the burden across an entire community. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 6 "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis. Deworm the World is one of the only organizations focused on STH (while there are other programs that focus on schistosomiasis), and there is significant room to scale-up [sic] STH programs. In some ways, schistosomiasis has an even larger gap to fill than STH. If Deworm the World shifted its focus to include schistosomiasis, it might widen the existing STH gap. Additionally, Deworm the World specializes in school-based deworming, which is not the ideal approach in every situation." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5. This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. 7 "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5. 8 "We work with governments around the world to eliminate the public health threat of worms through scaling up school-based mass deworming programs." Evidence Action website, Deworm the World Initiative (December 2016) For example, Deworm the World is partnering, or planning to partner, with local organizations in Pakistan, Vietnam, and Nigeria: Nigeria: "Deworm the World is in discussions with a potential partner which plans to work in Cross River on other integrated NTD treatment. The need to scale up treatment for schistosomiasis and STH among school-age children has not yet been addressed, and Deworm the World has been in discussions with this partner, the state NTD coordinator, and other state officials about creating a school-based deworming program to treat both STH and schistosomiasis beginning in 2016." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, pg. 6. Vietnam: "[In Vietnam] Deworm the World’s partnership with both the government and Thrive Networks is a new working model for Deworm the World; elsewhere, it has supported government implementation or run its program independently. Deworm the World has only one staff member in the country. Dubai Cares provides most of the program funding. All funding goes to Thrive Networks, which provides money to the government for implementation expenses." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 3. Pakistan: "[In Pakistan, Deworm the World] plans to contract with a local organization to do these [prevalence] surveys, but does not yet have a signed agreement...Deworm the World plans to work in partnership with the same local organization to provide technical support, likely beginning in Punjab and later expanding to Sindh." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. 9 For example, Deworm the World has considered supporting evaluations or monitoring of different deworming-related programs: "If funding permits, the Kenyan government may begin a lymphatic filariasis treatment program along its coast. If it does so, Deworm the World will provide process monitoring and coverage validation for the treatments." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 3. We learned in later conversations that Deworm the World received a grant from the END Fund to work on lymphatic filariasis treatment in the coastal region. "We are contributing approx $111k to the TUMIKIA and TakeUp studies, complementary studies leveraging the Kenya program to look at the potential for breaking STH transmission." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Deworm the World has told us that it is also interested in opportunistically evaluating new evidence-based programs that may efficiently complement deworming. GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014. We discussed this possibility again with Deworm the World in 2017. In 2017, Deworm the World sought funding "to be able to conduct independent monitoring and community evaluation surveys in states [in India] where we do not provide technical assistance, to better understand processes and program quality in those areas and enable improved support from the central level. This funding would enable those activities in three states." See this spreadsheet, sheet "Spending opportunities." 10 "In January 2007, the [Young Global Leaders] launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs." Harvard Business School Kenya Case Study A 2010, Pg 7. 11 Deworm the World has supported deworming activities in India since 2009, in Kenya since 2012, and in Ethiopia since 2014: "[Where We Work, Bihar State, India]: 1st deworming round in 2011 reached 17 million children." Evidence Action website, Deworm the World Initiative (March 2016) "[Where We Work, Kenya] With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively." Evidence Action website, Deworm the World Initiative (March 2016) Deworm the World supported a pilot deworming program in Ethiopia in April 2015 and another deworming program in October and November of 2015. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Deworm the World supported treatments in Cross River, Nigeria in mid 2016: "This week, the Cross River State Ministry of Health’s Neglected Tropical Diseases (NTD) unit launched its inaugural statewide school-based deworming exercise that will treat against two neglected tropical diseases that are particularly common in children: schistosomiasis and soil-transmitted helminthiasis (STH). The school-based deworming exercise will cover 11 of the 18 local government areas in Cross River for the first time, and is targeting 600,000 at-risk school-aged children in primary and junior secondary public and private schools. Other NTDs endemic to the state (lymphatic filariasis and onchocerciasis) will be treated through a community-based approach, according to standard practice." Evidence Action, blog post, June 30, 2016 Deworm the World supported treatments in Vietnam in April and November 2016. DtWI External Vietnam Costing Model Deworm the World supported a prevalence survey in Pakistan between August and December of 2016 (Conversations with Grace Hollister, August 11, 2016 and January 4, 2017; see Evidence Action, Baseline Survey Report of Soil-Transmitted Helminths Prevalence in Pakistan). It first supported treatments in Pakistan in 2019. See this Evidence Action post. 12 "The government is fully responsible for program implementation, and … these programs leverage thousands of govt personnel from health and education to be able to run." Grace Hollister, email to GiveWell, June 9, 2016 13 "Note that there are not single deworming days in all countries, nor do all countries have a mop-up day. In all cases there are mass campaigns, but the structure varies by country." Grace Hollister, email to GiveWell, June 9, 2016 14 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. For example, Deworm the World has assisted in a number of areas in India: "Andhra Pradesh... Deworm the World’s contributions Prevalence survey... Operational support Helped government develop operational plans and budgets Coordinated cross-sectoral partners through the establishment of a State School Health Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS) Coordinated drug donation made by Feed the Children Designed a monitoring and evaluation (M&E) system Created government tableau for community awareness Trainings Conducted a master training session for program Designed training cascade for the master trainees to train the rest of the implementers Designed training materials Developed materials and campaigns for community sensitization Bihar… Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andhra Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO… Delhi… In addition to the standard contributions (see Andhra Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children. Rajasthan… DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO." @Grace Hollister Conversation June 19th 2013@, Pg 1-4. 15 We have matched our descriptions to Deworm the World's standard categorization, albeit in a slightly different order. An archived version of the link is here. 16 "As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program." @Alix Zwane Conversation June 4th 2013@, Pg 2. "Deworm the World does not yet have an agreement with the government in Pakistan to conduct the surveys, but hopes to accomplish this in the next month, and anticipates that the prevalence surveys will be conducted beginning in January or February of 2016. It is expected that a clear articulation of need will be an important factor in building a strong case to the government in favor of deworming programs, and it may be best to wait until the results of the surveys are available in the second quarter of 2016 before beginning discussions with the government on a scaled school-based program. Treatment may not begin until 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. Deworm the World's advocacy often occurs side-by-side with Deworm the World's technical assistance; once Deworm the World proves that a deworming program can be well-executed, it is easier to interest national governments in funding deworming programs. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 17 For example: "DtW has been involved in deworming programs in four different states, and is still actively involved of [sic] three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement...In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. [Note: these notes contain additional examples of Deworm the World's advocacy for other states in India] "Together, we suggested to the Federal Ministry of Health that they expand this initial work. What would it take to treat at least 75% of all at-risk school-age children in the country and to launch a truly national program? SCI helped Oumer Shafi, the committed and action-oriented Coordinator for Neglected Tropical Diseases in the Federal Ministry of Health, develop a detailed action plan. This entailed sophisticated statistical analysis to determine how many deworming sites would be required to reach at least 80% of kids at risk. Meanwhile, I worked closely with Birhan Mengistu, an up-and-coming leader seconded from the World Health Organization, and with other Ministry of Health staff. We sat for hours hunched over laptop screens to develop detailed five-year budgets, talking through row after row of spreadsheets and reviewing everything from the cost of fuel for drug transport to the needs of teachers. Together with the Federal Ministry of Health, we were able to think and act boldly. We are excited to continue to partner with SCI and are seeking other partners who also share common goals and values to rapidly scale school-based deworming in endemic countries. ...When we floated the idea of vastly increasing the scope of the originally proposed deworming rounds to be a truly national plan treating upwards of 75% of all children at-risk, Shafi didn’t flinch." Evidence Action, blog post, June 12, 2015 Other advocacy activities can include discussing: "how deworming can fit into the current policy environment and policy priorities of a government, how such a program can/should be financed, the robust evidence of impact, how a country can best take advantage of WHO drug donations, encourage program champions within government, help establish program governance structures. Once a program is established, advocacy doesn’t end – we work with govts to ensure the continuation of the above. Typically we refer to this group of activities as policy and advocacy, because there is a heavy emphasis on the former." Grace Hollister, email to GiveWell, June 9, 2016 These discussions also provide opportunities for Deworm the World to assess how well a deworming program with the government might run. If Deworm the World discovered from its advocacy discussions that there were high rates of teacher or student absenteeism, then it might conclude that a school-based deworming program may not work in the country. Deworm the World assesses risks like this through a diagnostic survey of the country’s capacity, including school attendance rates, which must be sufficiently high if a school-based deworming program is to succeed. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 18 Deworm the World builds strong working relationships with governments to try to ensure that its programs will be effective, and it will not commit to a program if it does not foresee success in that country. It can decide to abandon plans for a program before a memorandum of understanding (MoU) is signed. In one state in India, Jharkhand, Deworm the World explored a program, but did not build a strong working relationship with the government, so Deworm the World pulled out of discussions before discussing an MoU or investing much money. The discussion stage with governments is important for helping Deworm the World assess the government's position and viability as a partner. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 19 "The global STH community has changed significantly in the last couple of years, especially due to the formation of the STH Coalition. The community is now prioritizing STH (in a way similar to how LF became prioritized with the formation of the Global Alliance to Eliminate Lymphatic Filariasis, which has seen significant success). The STH community is developing plans to scale-up [sic] treatment, especially in high burden countries such as: Ethiopia Nigeria India Pakistan The Democratic Republic of the Congo Indonesia The Philippines Tanzania As part of the STH Coalition, Evidence Action is chairing a working group on school-age children. It has used some of its unrestricted funding to hire consultants to create ‘snapshots’ of each country, including obstacles, gaps, potential strategies, and financial needs. Deworm the World expects to see an increase in partnerships between the various groups in the STH community. Deworm the World hopes to leverage partnerships with existing organizations in, e.g., Nigeria, Pakistan, Ethiopia, etc., to provide catalytic support (rather than opening its own offices in those places)." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 6. 20 "We work with epidemiologists and local partners to assess worm prevalence and intensity, obtaining data to develop a targeted treatment strategy and to determine parasitological impact once programs are in place." Evidence Action website, Deworm the World Initiative (March 2016) A few examples of Deworm the World assisting with prevalence surveys include: "Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children." Grace Hollister conversation June 19th 2013, Pg 1. "DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged [sic] children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment." Grace Hollister conversation June 19th 2013, Pg 2. "In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city." Grace Hollister conversation June 19th 2013, Pg 3. "DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year." Grace Hollister conversation June 19th 2013, Pg 4. 21 For example, Deworm the World is currently supporting prevalence surveys in Pakistan because Pakistan has not yet been "mapped" (i.e., prevalence surveys have not yet been conducted in Pakistan), so nobody knows how heavy the worm burden is in Pakistan or where deworming efforts should be focused. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Deworm the World has estimated very roughly that its three-year program in Nepal would cost about $6 million ($2 million per year). This is based on a cost per child of no more than $0.50 and a target population of about 6 million children. However, the latter estimate is based on the outdated prevalence survey data mentioned above. Deworm the World will need to conduct a new survey to determine an exact target population and a more accurate budget." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pgs 1-2. "Deworm the World will likely wait to expand its Vietnam activities until further mapping and impact evaluation have been completed." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pg 7. 22 Comments from Deworm the World in response to a draft of this page in October 2017. 23 Deworm the World originally planned to do prevalence surveys every few years but may do them less frequently going forward. "DtWI would like to do prevalence surveys after every 3 years or so. Ideally, prevalence surveys would be carried out after every third round of treatment immediately prior to the following round." GiveWell's notes from site visit to India, October 2013 "Note: this [follow-up prevalence survey] strategy is evolving; WHO recommendations are to conduct sentinel site surveys after 5-6 rounds of treatment, and we are moving in that direction. Key is how a new survey would impact the treatment strategy" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Key M&E activities in India include "post-round 3 impact measurement" prevalence surveys. DtWI NDD Year 1 M&E review July 2015, Pg 3. Deworm the World has said that a Rajasthan follow-up prevalence survey is tentatively planned for late 2017. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Grace Hollister, edits to GiveWell's review, November 7, 2016 24 Deworm the World told us that it works "with partners with expertise in STH parasitology and epidemiology." Grace Hollister, edits to GiveWell's review, November 7, 2016. Examples of Deworm the World working with partners on prevalence surveys: "The WHO reports that Pakistan is endemic for STH, but there is not yet sufficient evidence of prevalence and intensity to develop an evidence-based treatment strategy. Deworm the World has committed unrestricted funding to fund prevalence surveys in two large provinces, Punjab and Sindh. It is targeting these provinces because their school enrollment rates are high, the areas are fairly secure, and they contain a significant percentage of the population of Pakistan. Deworm the World plans to contract with a local organization to do these surveys, but does not yet have a signed agreement." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. "Following three rounds of school-based deworming, Evidence Action - Deworm the World Initiative recommended a second prevalence survey to the Bihar government, in order to understand the effect of deworming in Bihar on STH infection levels. With approvals from the State Government, in January and February 2015, Evidence Action - Deworm the World Initiative, conducted an STH prevalence survey among school-age children in government primary schools in Bihar. The survey took place in 65 schools in 14 districts, covering all three agro-climatic zones in the state. The National Institute of Epidemiology – Chennai (NIE) designed the survey, and analyzed the dataset to produce epidemiological findings. Field teams hired through GfK Mode (an agency with prior experience in sample collection for STH prevalence surveys), visited the households of children in the selected schools to collect stool samples and information related to school, household, deworming, and sanitation, to better understand infection patterns and allow for sample weighting. The 2 Post Graduate Institute of Medical Education and Research – Chandigarh (PGIMER) analyzed stool samples in field laboratories, which were set up in district and block health facilities, using the WHO recommended Kato-Katz method." DtWI Bihar 2015 Prevalence Survey report, Pgs 1-2. 25 Comments from Deworm the World in response to a draft of this page in October 2017. 26 "We work closely with the Ministries of Education and Health to design a program with joint ownership, develop operational plans and budgets, coordinate logistics, and provide on-the-ground support to ensure a high quality outcome." Evidence Action website, Deworm the World Initiative (March 2016) Note that we do not feel like we have a strong understanding of Deworm the World's activities in this area; for example, we have not asked Deworm the World what it has brought to the planning, budgeting, or logistics processes that would not have otherwise been included. We do not have a strong sense from Deworm the World's website about what these activities involve (e.g., we do not know what it means for Deworm the World to have "coordinated logistics"). 27 "We help governments evaluate appropriate drug treatment strategies and dosage, support drug procurement including through global pharmaceutical donation programs, and design robust serious adverse event protocols and drug tracking systems." Evidence Action website, Deworm the World Initiative (December 2016) Grace Hollister, edits to GiveWell's review, November 7, 2016 28 "DtWI provided support to the state government in submitting the drug requisition to WHO in March 2013, as well as in shipping, custom clearances and transportation upon arrival in India." DtWI Rajasthan 2013 program report, Pg 6. Deworm the World told us that before it started conversations with the Indian government, the government was not aware that it could obtain albendazole for free from the World Health Organization. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 29 "We consistently design and support training through an efficient multi-tier cascade approach that is tailored to the local context, ensuring knowledge reaches from the national level all the way to the teachers responsible for administering deworming medication." Evidence Action website, Deworm the World Initiative (March 2016) GiveWell's understanding is that training cascades involve a series of trainings that start at high levels in the government, and proceed in a step-wise fashion down to local levels, where teachers are eventually trained. At each training, materials (such as deworming drugs and posters that notify the community about deworming day) are passed down from the staff member leading the training to the staff members attending the training, until materials eventually reach teachers. A description of a training and distribution cascade: "Transportation of tablets to all districts was managed and supported by DtWI, in coordination with SHS and DHFW to the district level dispensary, from where they were collected by the respective teachers for their school. For anganwadis, the syrups were sent to the nodal officer who arranged further transportation to the supervisors who in turn handed over to the anganwadis. Training about health education on types of STH, need for deworming, transmission of worms through open defecation and other practices and how to safely administer deworming drug was conducted through a cascade model. In the first step of the cascade, training of trainers was conducted at the state level where a total of 1040 participants attended (390 WCD + 650 education department) in 60 sessions. These trainers further trained 3032 headmasters and 3032 teachers and 10,500anganwadi workers in groups of 30 participants. The training on deworming was integrated with training for the WIFS program to effectively utilize time of participants and trainers and reduce training costs. A simplified training manual was developed that included content on deworming and WIFS into a single document. At the state level training sessions, training videos on three types of soil transmitted [sic] helminths (ascaris, trichuris and hookworm) and worm infestation cycle were also used. DtWI hired district coordinators (DCs) to provide short term support to the deworming program. They played a key role in ensuring that drugs in adequate quantities were available with the district-level dispensaries and nodal offices for further distribution. The DCs collated information on shortfall or surplus of drugs at district level and shared information with the DtWI state team, who coordinated with SHS to ensured-distribution [sic] or fresh supplies to districts facing drug deficits." DtWI Delhi 2013 program report, Pg 7. See Pgs. 12-13 of DtWI Delhi 2015 program report for a visual representation and description of a recent training and distribution cascade. 30 For example, a description of the training cascade in Kenya: "The National School-Based Deworming Programme uses a cascade implementation model that efficiently and cost-effectively delivers training materials, deworming tablets, monitoring forms, funds, trainings other programme materials and resources from the national level to schools. At the national level, the Programme trains a team of MoEST and MoH officials as master trainers, requisitions deworming tablets through the MoH, and develops treatment and implementation strategies, training materials and monitoring tools. Thereafter, an initial planning meeting is held with county and sub-county leadership. This meeting is followed by two levels of trainings on how to successfully implement the Deworming Programme: Sub-County Training and Teacher Trainings. These trainings prepare sub-county and division officials to plan subsequent programme activities within the cascade, distribution of materials, planning of deworming and community mobilization and sensitization. After these trainings and community mobilization, the critical day of implementation occurs – Deworming Day – where teachers administer deworming tablets to millions of children in over 11,000 schools across Kenya and fill in monitoring forms to capture treatment data. These forms and any unused deworming tablets are moved up through a “Reverse Cascade” as described below. The cascade model helps to manage the national scale of the NSBDP, and therefore, builds capacity for successful implementation at various levels. Additionally, the cascade brings together MoEST and MoH personnel through collaborative leadership responsibilities for the planning, implementation and monitoring of programme activities at all levels. The cascade is outlined in the infographic below." DtWI Kenya 2013-2014 program report, Pgs 4-5. "Drug distribution: As per NDD operational guidelines, and established best practice, drug distribution was integrated with the training cascade (as detailed in the training section below), whereby NDD kits were provided to health functionaries at the district level trainings for onward distribution. The kits included drugs, IEC materials, and reporting forms." DtWI Madhya Pradesh 2015 program report, Pg 12. 31 For example: "DtWI provided technical expertise across all program components, and served as the primary coordinating body among implementing agencies. DtWI facilitated drug donations from WHO, provided professional master trainers for training of trainers, developed training materials such as flipcharts, modified training booklet and reporting forms, designed and carried out independent monitoring, and developed adverse event protocols." DtWI Rajasthan 2013 program report, Pg 5. "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." DtWI Madhya Pradesh 2015 program report, Pg 11. "We hire master trainers, or train govt staff to be master trainers." Grace Hollister, email to GiveWell, June 9, 2016 32 Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16. Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15. Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15. Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17. 33 "We work with governments and communications experts to design locally appropriate awareness campaigns to communicate messages through a wide variety of channels to increase public acceptance and effectiveness of deworming programs." Evidence Action website, Deworm the World Initiative (March 2016) 34 For example: "As part of their training, school headmasters/teachers were instructed to share information on the deworming program in the morning prayer sessions at their respective schools on a daily basis from October 6, 2013 onwards. They were also advised to convene school management committee meetings to communicate about the benefits of deworming and the schedule of deworming program. School headmasters were also advised to carry out student rallies / processions (prabhat pheri) to create awareness in the communities." DtWI Rajasthan 2013 program report, Pgs 8-9. 35 "One other key strategy adopted by DtWI to spread awareness was through text (SMS) reminders over mobile phones to school teachers, headmasters, Child Development Project Officers (CDPOs) and lady supervisors as a reminder about deworming day. SMSs were also used to reinforce precautions on drug administration, such as not giving drugs on an empty stomach, but only after midday meals and not giving drugs to sick children. In all, about 80,000 text messages were sent to school teachers and headmasters three times – a total of 2,40,000 [sic] messages. These messages were sent a day before deworming day, on mop-up day and after mop-up day. About 1400 such messages were sent twice to lady supervisors and CDPOs on a day before deworming day and on mop-up day. Similarly, five rounds of around 2400 text messages were sent to block level officials to expedite coverage reporting. This was an example of ensuring last-mile communication at low cost of about 12 paisa per message (or roughly 1/5th of a cent)." DtWI Rajasthan 2013 program report, Pg 9. "Additionally, mike announcements were made at public places in blocks and district headquarters by Evidence Action for 5 days, closer to deworming day (Annexure E.4)." DtWI Bihar 2015 Program report, Pg 13. "The State Health Society Bihar and Evidence Action rolled out a media mix to generate community awareness and increase program visibility to improve coverage in the state (Annexure E.1). We supported the adaptation and contextualization of prototypes from the National Deworming Day IEC resource toolkit. At the state level, State Health Society Bihar, in coordination with the Department of Public Relation, Government of Bihar, published newspaper advertisement in four dailies one day prior to deworming and mop up [sic] day, i.e., on 20 and 25 February (Annexure E.2). Radio jingles, customized into three local dialects, were aired from 15 to 26 February on the All India Radio to maximize outreach to the community. For additional visibility of the program at the community level, State Health Society Bihar printed 513,625 posters (7 for each school, including distribution in the local community), 1068 banners for Primary Health Centers, hoardings at 38 district headquarter [sic]. All of these were adapted and contextualized by Evidence Action." DtWI Bihar 2015 Program report, Pg 12. Other community awareness activities include creating posters to display at schools or advertising the deworming day in the newspaper. "Activities designed to enhance community awareness on deworming were rolled out to improve overall program coverage. The awareness activities included newspaper advertisements a day prior to the deworming day; a 60-second radio jingle aired on 3 FM channels from April 7 to 15 by School Health Scheme, and banners displayed at schools. Evidence Action was part of the committee formed by the state government for contextualization of the radio jingle. Evidence Action extended support to the state in contextualizing IEC materials from the National Deworming Day guidelines. The Directorate of Family Welfare also independently developed and printed handbills for the distribution at anganwadis to mobilize people on deworming day. The School Health Scheme provided banners to the schools, the distribution of which was integrated in trainings for teachers. The Delhi state government also used an e-portal to disseminate key information, including dates for deworming and mop up [sic] days, benefits of deworming, and details of the launch event." DtWI Delhi 2015 program report, Pgs 11-12. 36 "We help governments design monitoring systems to measure effectiveness in achieving intended program results. We also conduct independent monitoring to validate program results, and evaluate the impact of programs in reducing worm prevalence and intensity." Evidence Action website, Deworm the World Initiative (March 2016) 37 Note that Deworm the World hires monitors for the first and third type of monitoring data collected, but that the second is collected entirely by government staff: "Coverage reporting is done by the government- we sometimes assist in the data analysis, designing reporting forms, and ensuring that the 'reverse cascade' is appropriately designed." Grace Hollister, email to GiveWell, June 9, 2016 38 "Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program." DtWI Madhya Pradesh 2015 program report, Pg 16. 39 In India, on Deworming Day and Mop-Up Day, Deworm the World commissions independent monitors who go to schools to gather data on whether principals and teachers are prepared for Deworming Day, the availability of drugs and supplementary materials, whether students are being dewormed, whether proper procedures are being followed, and more. For example, see Deworming Day monitoring data from Rajasthan in 2015: DtWI Rajasthan 2015 monitoring survey from deworming day, schools (shows which questions were asked) and DtWI Rajasthan 2015 monitoring data from deworming day, schools (shows the survey responses). Kenya: Trainings monitoring: "PMCV [Process Monitoring and Coverage Validation] officers observed 36 CHEW [Community Health Extension Worker] Forums aimed at introducing the deworming sensitization message and materials/methods as well as asigning [sic] CHEWs to schools for monitoring. A successful community health extension worker forum is one that starts on time and where all the materials were present. Overall, 63% of participants arrived before training, whereas 22% arrived 1hr after the forum had begun and 15% of participants arrived more than 1hr after the forum’s commencement. Lateness appears to be a commonality to all training sessions. Materials required for CHEW training include a powerpoint printout, CHEW checklist and Severe Adverse Event (SAE; side-effects of the drugs) protocol. In 51% of forums, ALL of the Materials Pack was distributed at the start of the forum. In 13% of forums, SOME of the Materials Pack was distributed at the start of the forum. In 36% of forums, NONE of the Materials Pack was distributed at the start of the forum." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 9. Community sensitization monitoring: "A number of parents were also interview [sic] at schools on Deworming Day regarding their knowledge of deworming and the source of that knowledge. The intention behind this exercise was to compare the information source to those interviewed prior to deworming as a measure of consistency. In Figure 4, the results of the interviews pre-deworming day are compared with those parents interviewed on deworming day. The results remain largely similar, however more parents reported getting their information from ‘other’ sources (51%) when interviewed on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 14. School preparedness monitoring: "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15. Deworming Day and Mop-Up Day monitoring: "PMCV field officers visit schools on Deworming Day to observe procedure and interview teachers/head teachers regarding deworming. The number of schools observed on Deworming Day treating for STH in Year 3 was 247. The combined population of registered children at the observed schools was 88,820 children. It is estimated that 7,485 children were directly observed being treated for STH. Seven schools treating for both STH and SCH were observed. The total registered population of children in these schools was 3,198 children and 352 children were directly observed by field officers participating in Deworming Day. A quality Deworming Day is regarded to be one where: Deworming occurs within 1 week before [sic] teacher training The school would have the correct materials (including sufficient drugs) in place before commencement Children of the appropriate ages are treated (ages 2-14 years) Non-enrolled and ECD aged children are prioritized for treatment within the schools The correct dosage of drugs is given to all children" Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16. 40 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. For example, see DtWI Delhi 2012 coverage data by school. 41 Comments from Deworm the World in response to a draft of this page in October 2017. 42 Grace Hollister, email to GiveWell, March 6, 2016 43 "A competitive RFP [request for proposal] process is now used to identify a professional survey organization to provide independent monitors. There are requirements placed on the experience of these monitors." DtWI Monitoring Improvements 2014, Pg 1. Bihar: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state… Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively." DtWI Bihar 2015 Program report, Pg. 15. Rajasthan: "Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state... Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively." DtWI Rajasthan 2015 program report, Pg 14. Madhya Pradesh: "Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively." DtWI Madhya Pradesh 2015 program report, Pg 15. Delhi: "[Evidence Action] hired an independent research agency, Sigma Research and Consulting Private Limited [sic] that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively." DtWI Delhi 2015 program report, Pg 14. Deworm the World was also involved in Chhattisgarh, but as it was engaged relatively late in the process, it did not conduct all of its standard monitoring activities in the state: "Although we place great emphasis on understanding the extent to which the school and health systems are ready to implement deworming, the extent to which deworming processes are being followed, and the extent to which coverage has occurred as planned, in Chhattisgarh we supported only with the coverage validation activity at schools due to time constraints." DtWI Chhattisgarh 2015 coverage validation report, Pgs 2-3. 44 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Evidence Action has a permanent Monitoring, Learning, and Information Systems team. Deworm the World leverages this team for M&E." Grace Hollister, email to GiveWell, June 9, 2016 "The field officers that collect the data in the field are short term hires who come from the counties in which the program is implemented" Grace Hollister, edits to GiveWell's review, November 7, 2016 45 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. 46 Note that these figures exclude Deworm the World's global (non-program-specific) spending and its indirect costs. 47 "Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success. "Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action." Evidence Action launch announcement 2013 "Evidence Action Beta investigates what interventions might be suitable for massive scale up [sic] – finding the next thing that works." Evidence Action website, Evidence Action Beta (October 2015) 48 India: "…absent from most of the country, [schistosomiasis] risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1. "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3. See section on worm prevalence and intensity in India and Kenya below. 49 WHO, Helminth control in school-age children, Pg 74 "Based on the findings of the prevalence survey and WHO guidelines, Evidence Action recommends an annual school based deworming program for school-age children in the state. [...] Given the pre-existing deworming treatments described above, this prevalence survey cannot be considered a baseline survey of an untreated population, but is rather a survey to assess STH infection rates in a treated population, to determine an optimal treatment strategy." DtWI Madhya Pradesh 2015 program report, Pg 38. "Our recommendation is explained in the prevalence survey report. The prevalence and intensity rates from the survey are not "baseline" data, given that there has been relatively regular administration of albendazole in MP through the BSM program that treated PSAC since 2005, and the LF program which provided community-wide treatment of 11 districts of MP (the number of endemic districts had fallen to 8 by 2014). As a result, these deworming efforts have likely had an impact on STH prevalence and MP could not be considered an untreated baseline population. We therefore did not apply the WHO guidelines for baseline STH prevalence. "Annexure 10 of the WHO guidelines suggest continuing annual treatment for populations which have received deworming for several years, and prevalence is still greater than 10%. In addition, the high rate of open defecation in the state, and the planned ending of the LF program in MP, increased the risk of infection and a potential resurgence in prevalence." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Grace Hollister, email to GiveWell, June 9, 2016 50 DtWI Madhya Pradesh 2015 program report, Pg 7. Deworm the World, Chhattisgarh prevalence survey report, August 2016, Pgs 4-5. DtWI Bihar 2011 prevalence survey report DtWI Rajasthan 2013 prevalence survey report Deworm the World 2015 Uttar Pradesh prevalence survey report, Pg 9. Deworm the World notes that in Madhya Pradesh, Chhattisgarh, and Uttar Pradesh, prior treatment was either through lymphatic filariasis programs or non-standardized deworming, and that in Bihar, a baseline survey was done before school-based deworming took place. Grace Hollister, comments on a draft of this review, September 6, 2018. 51 Grace Hollister, comments on a draft of this review, September 6, 2018. 52 Deworm the World noted that the KEMRI surveys in Kenya are designed for impact assessment, while the surveys in India are designed for mapping. This means that the schools selected in KEMRI's surveys are all from places where treatment is required or taking place. In mapping surveys, schools are selected to be representative of a larger geographic area (e.g., they may be selected in part based on which agro-climatic region they are in). Additionally, Deworm the World noted that the surveys in India are looking at a much larger population than the Miguel and Kremer 2004 and Croke 2014 studies examined and there may be substantial variation in prevalence across a given area. Grace Hollister, conversation with GiveWell, June 13, 2016 and Grace Hollister, edits to GiveWell's review, November 7, 2016 53 For example, India has such a program: The National Vector Borne Disease Control Programme LF treatment coverage 2015 In Kenya, the LF program is housed within the country’s neglected tropical disease (NTD) unit, which has asked Deworm the World if it might support its process monitoring and coverage validation (PMCV) operations for LF. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Deworm the World has told us that in both countries (Kenya and India), LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Additionally, our understanding from a number of conversations with Deworm the World and others is that many countries are beginning to shut down their LF programs as they eliminate the disease. 54 "DEC [Diethylcarbamazine] + Albendazole in selected distt & DEC in other distt" LF treatment drugs 2012. "Albendazole, the same drug used to treat STH, is usually used to treat LF." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Assam 2010 guidelines for deworming recommends 400mg of albendazole for children being dewormed, and Global Alliance to Eliminate Lymphatic Filariasis - Prevention also recommends 400mg doses. 55 Or from twice-per-year to thrice-per-year. Note that community-based treatment, such as is typically used for LF, involves enlisting several people to travel from house to house to administer treatment, making it much more time-consuming and costly than school-based programs. Because its goal is to treat every person in a community, multiple trips to a single area may be required to ensure total coverage (e.g., if a household member is not at home during the first visit). GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Deworm the World also claims that it generally knows where there is overlap between areas that are endemic for STH or schistosomiasis and areas that are endemic for LF. For example, in Kenya, only the coastal area is LF-endemic, but worm infections are more widespread, so LF treatment efforts involve a smaller number of subcounties. Community-based treatment programs might be cost-effective in places endemic with many NTDs but might not be in places only endemic for STH or schistosomiasis. Deworm the World has told us that many places do not have much overlap between different NTDs so school-based deworming programs can provide a cost-effective alternative to community-based treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 56 More detail on this page. 57 We have gained this understanding through many conversations and across multiple years of reviewing Deworm the World. See our review process. Note that the methods Deworm the World uses in each country, for each round of MDA, might not be the same as those outlined here; Deworm the World adjusts its monitoring based on past learning or new contextual constraints. In the cell notes in this spreadsheet, we include descriptions of the monitoring processes we have seen in Deworm the World's monitoring reports. 58 For example, as part of its monitoring in Kenya, monitors visit sub-county trainings and teacher trainings: "The intent of Sub-Country Training (SCT) sessions is to ensure that Sub-county and division-level trainers understand the purpose and procedure of deworming. The successful completion of this activity allows the division trainers to then conduct the same activity with teachers in their sub-counties. "PMCV teams attended 38 out of the total 111 SCTs conducted in the third year of the program. Field officers interview participants before and after the training and completed observations during the course of the activity." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 5. "Teacher Trainings (TT) are conducted by the division trainers using a “Teacher Training Booklet” as reference material. A quality TT session is considered to be one where the necessary content is covered and retained by participants. TTs also serve the function of distribution of drugs and materials (monitoring forms, posters) to teachers. It is the aim of the program to provide all schools with their required drugs and materials at teacher training sessions. Teachers are expected to use the “Deworming Day Checklist” to conduct operations on the day. They are also expected to sensitize other teachers at their schools who did not attend the training on deworming day procedures (see the section on deworming day contained in this report). "A total of 76 TT sessions were observed by PMCV field officers in Year 3. Of those training sessions, 13 were specifically SCH trainings." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 7. The trainings that monitors are sent to are randomly selected by a computer program. Paul Byatta, conversation with GiveWell, September 20, 2016 The list of countries in which this takes place is from Grace Hollister, comments on a draft of this review, September 11, 2018. 59 Grace Hollister, comments on a draft of this review, September 11, 2018. 60 Paul Byatta, conversation with GiveWell, September 20, 2016 61 We believe that monitors are supposed to select every third teacher that arrives at the training until they have interviewed four teachers: "SURVEY INSTRUCTIONS [...] Select every third participant that arrives before the start of the training. Interview at least four participants." Deworm the World, Kenya Year 3, Pre TT form, Pg. 1 62 For example: Kenya. In Kenya, Deworm the World's monitors visit schools and communities before Deworming Day to assess their level of preparedness for the upcoming MDA. Schools are selected randomly, and then monitors visit the communities near the schools that are selected. Paul Byatta, conversation with GiveWell, September 20, 2016 "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15. "Use the PRE-DD-SCHOOL instrument to conduct interviews and note observations at the selected primary school before deworming day. In case selected school is closed, does not exist or has already dewormed move to the next nearby primary school. Allow Head teacher or Rep to consult with Head teacher/ REP when necessary. PRE-DD-SCHOOL should be completed along with PRE-DD-ECD or PRE-DD-COMMUNITY." Deworm the World, Kenya Year 2, Pre DD - School instrument, Pg 1. "PMCV Field officers interviewed a total of 716 parents with children. Of these, 379 were parents of enrolled children across 130 different schools and 337 were parents of non-enrolled children. The number of parents with at least one child enrolled in early childhood development (ECD) was 283 of the sample population, or 65%. Just over one third or 35% of parents had no child enrolled in ECD. The average age of those children reported to be enrolled (by their parents) was 7.7 years, whereas the average age of non-enrolled children was 3.7 years. "PMCV field officers observed parents’ level of awareness of Deworming Day, their intentions regarding taking children to be dewormed and documented the primary source by which parents were receiving such information." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 11. The next section is titled "4.1. Pre-Deworming Interviews - Parents." India: In India, telecallers phone schools ahead of Deworming Day to assess the school's preparedness. We have not yet requested data from these calls. For example: "Our tele-callers place phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17. 63 For example, a monitoring report we have from Kenya (for the 2014-15 round, or Year 3) includes some sample results from the pre-Deworming Day visits: The report notes that 97% of schools visited pre-Deworming Day had a teacher who had attended a training in the last fifteen days: "According to interviews with head teachers, 97% of these schools had a teacher who had attended training in the past 15 days. A further 86% of trained teachers had trained or sensitized other teachers on how to administer drugs and conduct deworming day. Almost all teachers (99%) found the Teacher Training Booklet to be ‘very’ or ‘somewhat’ useful in this process and 95% reported to use it often." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15. While 95% of schools had received some deworming tablets, only 87% of schools believed that they had received enough to cover the children at their school on Deworming Day: "At the time of PMCV visits, 95% of schools reported having received deworming tablets, with 97% of those schools having received them at the time of TT. This percentage is higher than those originally observed by PMCV officers with only 92% of schools reported to receive their drugs during TT. Upon further investigation, 87% of schools considered that they had received a sufficient supply of drugs for their current enrolled and ECD populations. It is likely that these schools requested additional drugs, because 98% of schools were observed to have sufficient drugs in place on deworming day. Only 5% of schools did not have all monitoring forms present prior to deworming day. Such schools have always sought support from the sub-county offices that organize additional prints or photocopying to ensure they have the forms on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16. Note that Deworm the World shared the pre-Deworming Day data that informs the monitoring report and we have not yet analyzed it. There appears to be a large number of questions in the survey that were not included in the report. The survey instruments that Deworm the World's monitors use are long (e.g., see Deworm the World, Kenya Year 2, Pre DD - School instrument, in which both teachers and randomly selected students are interviewed). The monitoring reports only include a few summary metrics, and we are unsure how the metrics included in the report are chosen. It is possible that the report's metrics are those that reflect most favorably on Deworm the World and that vetting the full data could reveal that schools are less adequately prepared. 64 When selecting parents to interview in the community, monitors ask the school headmaster to point to a student's house near the school. The monitor begins by interviewing that student's family, then walks to subsequent households, skipping every other household until the monitor has interviewed at least 6 families with an enrolled school-age child and 6 families without. Paul Byatta, conversation with GiveWell, September 20, 2016 In Kenya, monitors are also asked to interview the Community Health Extension Worker (CHEW) in the community, since they assist with Deworming Day: "On selecting the CHEW to interview, there is only one CHEW per community. A few communities do not have CHEWs at all." Paul Byatta, attachments to email to GiveWell, September 23, 2016 For example, from Kenya: "Of those parents aware of deworming, only 41% knew the correct Deworming Day date, 81% knew the correct target population, and 48% knew the correct age group. These results indicate that although parents report being aware of deworming, almost half do not have the information required to attend (date). There is the scope to find a more robust method of ensuring information retention in awareness of deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 12. 65 See this spreadsheet, sheet "Monitoring methods and results." Note: We have not seen this type of monitoring from Jharkhand 2016, and are uncertain whether Deworm the World did monitoring on Deworming Day or just coverage validation (described below). Deworm the World, Jharkhand Independent Monitoring Report 2016. In 2017, Deworm the World conducted both observations of the deworming process and coverage validation in Jharkhand. Deworm the World, Jharkhand NDD PMCV Report, February 2017 and Deworm the World, Jharkhand PMCV report, August 2017. Note that if the chosen school is closed on the day of the deworming, they are generally instructed to go to an assigned backup school instead. India: For example, in Delhi, monitors are instructed in the survey instrument, "Is the school open? [If not,] Did you go to the buffer school?" DtWI Delhi 2015 monitoring survey from deworming day, schools, Pg 3. According to Deworm the World, Madhya Pradesh NDD PMCV Report, February 2017, "If a school or anganwadi was found to be closed or non-traceable during process monitoring, it was replaced by another nearby site," Pg 5. It is our impression that most schools were open when monitors visited. We spot-checked several states' data in 2016: DtWI Bihar 2015 monitoring data from deworming day, schools, Column L shows that all schools monitors visited were open. DtWI Rajasthan 2015 monitoring data from deworming day, schools, Column M shows that 124 of the 125 schools monitors visited were open. Kenya: In Kenya, it is rare that monitors find schools to be closed or missing because Deworm the World has operated in Kenya for several years. However, monitors will try to go to another school if their school is closed. Paul Byatta, conversation with GiveWell, September 20, 2016 "When the monitor is unable to monitor the assigned school (because of closure, etc) we ask the monitor to i) to communicate the same with his supervisor and, ii) to visit the nearest school if the nearest school was not part of the randomly picked school." Paul Byatta, attachments to email to GiveWell, September 23, 2016 Vietnam: The backup schools are also randomly selected. Paul Monaghan, conversation with GiveWell, September 8, 2016 Nigeria: "Initially, replacement schools were also randomized and shared alongside sampled schools. However the issue of timing was an issue, as monitors who needed to replace schools were often too far away from the randomly selected schools and had to travel too far to get to a replacement school. Because deworming mostly happens in the morning, by the time monitors arrived at the replacement school, they may not be able to observe/conduct most of the deworming activities. This sometimes meant we were unable to meet our monitoring targets or monitors arrive way after deworming has started and we only get partial information from the exercise. Subsequently, we started to select replacement schools based on proximity to the sampled school that couldn't be monitored to reduce travel time. We would usually do this with the help of the LGA NTD team as they are more informed about schools within a particular location. In 2019 we decided to make it more systematic and to include randomization into the process by identifying the three schools closest to the initially sampled school that needs to be replaced, and then randomly select one of them." Brett Sedgewick, comments on a draft of this review, October 31, 2019. 66 See, for example, DtWI Delhi 2015 monitoring survey from deworming day, schools, Deworm the World, Kenya Year 3, DD - Main instrument, and Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day Note that it is not actually dangerous to give the deworming pills to sick children; Deworm the World simply prefers not to deworm sick children to avoid causing people to associate the deworming pills with illness: "Deworming pills should not be distributed to sick children...This is not because deworming pills could harm sick children. It is because DtWI wants to avoid people (and potentially the media) blaming the deworming pill for a child’s illness." DtWI 2013 GiveWell site visit 67 For example, in Kenya: "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment. Deworming was reported to occur inside classes in 47% of observed schools and outside in 53% of schools. Deworming was considered to be ‘systematic’ in 98% of schools. The correct dosage for albendazole is one tablet per child and the correct age is 2-14 years. These procedures were observed to be followed correctly by 86% of teachers observed by field officers. Coverage: Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools. Teachers were reported to correctly observe children swallowing PZQ in 99% of schools and ALB in 96% of schools. Observing children swallowing is most important when treating for SCH as the tablet does not taste pleasant and there are high chances of children spitting if not observed." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17. For example, in India: When monitors visit schools on Deworming Day, they make observations and ask questions related to the process of deworming, to check whether or not it is being implemented correctly. For example, monitors observe whether deworming is in progress and whether teachers are using proper recording protocol, with results given for: "Deworming activity is taking place in the class/Anganwadi" "Teachers/ Anganwadi worker following the protocol of putting single tick (deworming day) or double tick (mop-up day) on each child's name/roll no. in the attendance register after giving them the deworming tablet" "Practice followed by teacher, if the ticking/double ticking protocol did not followed" Deworm the World, Telangana 2016 IMCV report, Pg 19, Table 3. 68 See this spreadsheet for more details. 69 "India only conducts CV at schools, while Kenya, Nigeria, and Pakistan all utilize the same combined household and school-based CV methodology now." Brett Sedgewick, comments on a draft of this review, October 31, 2019. 70 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools "This is the case in India, however in [Pakistan]/[Nigeria]/[Kenya] such questions are asked during process monitoring on Deworming Day, not during CV." Brett Sedgewick, comments on a draft of this review, October 31, 2019. 71 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools. 72 For example, in Kenya, it is our understanding that monitors interview three randomly selected children from three separate classes, for a total of nine students, at each school they visit. See Deworm the World, Kenya Year 3, Post DD - Coverage instrument. On Pgs 1-3 there are three spaces for randomly selected classes. The instructions read: "Thank the Head Teacher or designate and request to speak to pupils of the randomized class...CHOOSE CHILD 5, 10 AND 15TH ON FORM E. IF LESS THAN 15 CHILDREN, SELECT THE LAST CHILD. ENSURE TO INTERVIEW AT LEAST THREE CHILDREN. ASK THE TEACHER FOR PERMISSION TO SPEAK TO THEM ONE AT A TIME Ask questions in multiple ways for interviews with students, use local language if possible. Don’t rush responses. Try to make them feel at ease. Speak to one child at a time at a place where they are comfortable.... [interview questions]...END, MOVE TO THE NEXT SAMPLED CLASS" "Note that this occurs at households as well for CV in Kenya, Nigeria (and Pakistan)." Brett Sedgewick, comments on a draft of this review, October 31, 2019. 73 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools "This is asked in all geographies. In addition, children are asked about unprogrammed deworming during CV in [Kenya], [Nigeria], [Pakistan]." Brett Sedgewick, comments on a draft of this review, October 31, 2019. 74 "Children were interviewed only from those schools which conducted deworming on either day i.e. NDD or mop-up day. Three classes were selected randomly from each school by using random number tables. Further, one student was selected from each class randomly considering their presence either on the NDD or mop-up day (attendance is verified through attendance register) using random table." Deworm the World, email to GiveWell, May 30, 2018. 75 Grace Hollister, comments on a draft of this review, August 20, 2018. 76 Grace Hollister, comments on a draft of this review, August 20, 2018. 77 Grace Hollister, edits to GiveWell's review, November 7, 2016 Grace Hollister, Global Deworming Lead, Evidence Action, conversation with GiveWell, May 16, 2018 Deworm the World, responses to GiveWell questions, May 16, 2019 78 Kenya: In Kenya, Deworm the World does have some checks on monitors' work. A randomly selected 10% of the schools that are being visited by monitors are called the same day that the monitors visit, to ensure that the monitor is actually at the school. Paul Byatta, conversation with GiveWell, September 20, 2016 "We do back-checks for the pre-deworming, de-worming and post-deworming day… Generally, we seek to establish that the monitor did actually visit the school assigned to him/her, interviewed at least the headteacher of the school and that there is broad consistency on some of the data they collect by the monitor and the back-checker. For instance, the back-checker also asks about the availability of forms and drugs at schools, which is also information that is asked by the monitor. We share back-checks results with the short term monitors before we make the last payment to them. The other purpose is to improve our data collection training for the subsequent waves." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pgs 2-3 India: "In India, the following steps are implemented to serve as check on the IMCV work. Some of these are detailed in the contracts with IM firms – let me know if you’d like to see those: 1. We collect photographs of all the schools and anganwadis (with the names) visited during PMCV in the states. 2. We also collect signatures and mobile numbers of headmasters/school teachers and anganwadi workers, which is the part of PMCV format itself. 3. Together with signature of the teachers/head masters, school stamp is also taken on signature sheet which is part of PMCV tools. Since we collect CAPI based data, monitors carry hard copy of signature sheet to get the stamp of the schools. 4. Additionally, random calls are also made by state team/tele-callers to confirm that monitors visited to designated school and anganwadis. 5. Evidence Action staff (including RCs & DCs) visits in selected schools and anganwadis on NDD and mop-up day to check if monitors visited the selected sites." Grace Hollister, edits to GiveWell's review, November 7, 2016 Additional information and clarification provided in Deworm the World's comments on a draft of this review, September 12, 2018. 79 Grace Hollister, conversation with GiveWell, September 1, 2016 For example, in Kenya: "Does the monitoring team have an estimate of how many non-enrolled children the deworming program reaches? We get this from treatment forms that schools submit back via “the reverse cascade”. For PMCV, we monitor that teachers are aware they should be treating non-enrolled, and they are aware of age categories targeted for non-enrolled. When we do data audit on treatment forms, we also check that data for on non-enrolled is entered accurately." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pg 3 In comments on a draft of this review on September 12, 2018, Deworm the World noted: "We have begun to address this issue through methodology changes in our coverage validation surveys. In Kenya we have also begun to use the enrollment rates obtained from the Ministry of Education and the number enrolled in schools to come up with the estimate for the non-enrolled population within the county." 80 See this spreadsheet for details. As one example: "In each of the selected segments, a household survey was administered to all at-risk persons within the visited households. It was expected that the household survey would give a representative proportion of the non-enrolled population. After administering the household survey, the monitor would liaise with a village leader to request information about the schools (primary and junior secondary) where most children in the selected segment attend. The proportion of household to school surveys was determined using state enrollment rates. Cross River State data shows an enrollment rate of 62%, and therefore 62% of the sample was distributed among the 60 schools (1 primary, 1 junior secondary in each of the 30 segments) interviewed, with the rest of the surveys administered in the community." Deworm the World, Cross River State PMCV report, October 2018, Pg 13. "Coverage validation aimed to estimate the program reach and surveyed coverage. This was done using WHO guidelines for coverage evaluation surveys in schools and communities post-deworming. For this exercise, six randomly selected counties, 3,937 households, and 312 schools were sampled." Deworm the World, Kenya NSBD PMCV report, Y6 2018, Pg 3. "India only conducts CV at schools, while Kenya, Nigeria, and Pakistan all utilize the same combined household and school-based CV methodology now." Brett Sedgewick, comments on a draft of this review, October 31, 2019. As noted in the preceding bullet point, Deworm the World has told us it also uses this methodology in Pakistan; we have not yet seen monitoring from Pakistan as the program began recently. 81 Comments from Deworm the World in response to a draft of this page in October 2017 and in September 2018. 82 See this spreadsheet for details. "Surveyors in India are assigned to multiple schools on NDD and Mop-Up Day, so it is often the case that a school may have already completed deworming by the time the surveyor finishes one site and moves to the next." Deworm the World, comments on a draft of this review, September 12, 2018. 83 "In 96% of observed schools, Deworming Day happened systematically." Deworm the World, Kenya NSBD PMCV report, Y6 2018, Pg. 13. "96% of monitored schools were implementing DD according to protocol when visited on DD." Brett Sedgewick, comments on a draft of this review, October 31, 2019. We are uncertain whether these comments mean that deworming was observed in 96% of schools or that 96% of schools in which deworming was observed performed deworming in a systematic manner. 84 See this spreadsheet for more information. 85 "Children were interviewed only from those schools which conducted deworming on either day i.e. NDD or mop-up day. Three classes were selected randomly from each school by using random number tables. Further, one student was selected from each class randomly considering their presence either on the NDD or mop-up day (attendance is verified through attendance register) using random table." Deworm the World, email to GiveWell, May 30, 2018. "Even though we only interview children who were present on deworming day in India, we still collect data on (and adjust for) the percentage of children who were absent on DD and the percentage of schools that did not implement deworming on DD. It is correct that in Jharkhand Aug. ‘17 it was reported that among children who were present on DD, 100% received deworming tablets. However, this statistic alone is not accounting for the fact that only 86% of children were in attendance on DD or MUD. We still collect this data on attendance; it’s just presented separately than the statistic on the percentage of children present who received a tablet. In Kenya, our coverage validation surveys are structured differently, such that we are interviewing children despite their attendance/enrollment status." Deworm the World, comments on a draft of this review, September 12, 2018. 86 Deworm the World, comments on a draft of this review, September 12, 2018. 87 "There isn't sufficient comparator data to know whether the coverage differences are a result of methodological differences, programmatic quality, or a combination of the two." Brett Sedgewick, comments on a draft of this review, October 31, 2019. See this spreadsheet. 88 Kenya: "Show student 3 tablets, ask: Which one of these three tablets did you take? Circle the indicated tablet." Deworm the World, Kenya Year 3, Post DD - Coverage instrument Vietnam: "What type of medicine was given out? (prompt the child with options)" and "Do you remember the color of the tablet? (do not prompt)" are questions on the coverage validation survey form for student interviews. Deworm the World, Vietnam 2016 monitoring survey form for coverage validation, Pg 7. India: In India, monitors show children the deworming tablet and ask if it is the tablet that they took, which we do not believe is a rigorous method of verifying student answers. For example, see Pgs 20-23 of DtWI Bihar 2015 monitoring survey for coverage validation, schools Nigeria: In Nigeria, independent monitors have pictures of the pills that they show children so that children can visually identify the pill and respond accordingly if they took a deworming pill. Deworm the World, responses to GiveWell questions, May 16, 2019. 89 "We train monitors to interview children in private by taking them aside for interviews. In practice this is often not in a separate room, but away from the class. Private rooms are difficult to secure and parents do not trust a stranger to be alone with their child. In most cases, an adult is present during the interview because the children are minors." Deworm the World, responses to GiveWell questions, May 16, 2019. 90 Deworm the World notes that an important difference between the Kenya and India programs is scale. The Kenya program is significantly smaller than the India program, thus it may be easier to execute the program more effectively. Grace Hollister, email to GiveWell, June 9, 2016 In Nigeria, Deworm the World does both community-based and school-based coverage validation, which may capture more unenrolled children who may be less likely to receive treatment. 91 Deworm the World supports program costs in Cross River, Nigeria and expects to support program costs in future Nigeria states. It expects to support program costs in Pakistan, and possibly Indonesia. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 92 We attempt to learn if there was a possibility that the program would have been funded by other donors. However, given that there is a global funding gap for deworming treatments (more), we suspect that in most cases where Deworm the World pays for a new deworming program, it is increasing the number of children dewormed. 93 Our intervention report discusses this briefly. Other conversations and observations have reinforced our impression that administering deworming drugs is fairly straightforward. The WHO factsheet on STH: "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet 94 Deworm the World notes that: "albendazole and mebendazole are quite hardy (neither requires special storage conditions) and have a long shelf life." Grace Hollister, email to GiveWell, June 9, 2016 95 For example, drug quality was tested in each program that Deworm the World supported in India in 2013-2014. Bihar 2014: "The drugs were safely stored in State Health Depot in Patna until November 2013. At that point, the districts began to pick up their share of the drugs from the depot. In October, Deworm the World had coordinated for lab testing of the stored drugs via ASCHO NIBULA INDUSTRIES LTD, an independent lab which approved the quality of the drugs." DtWI Bihar 2014 program report, Pg 13. "The quality of drug storage was satisfactory in most schools that were monitored. 98.7% of them stored the drugs in a clean location, 91.1% of them were stored away from direct sunlight and 97.4% of them were stored away from the direct reach of children." DtWI Bihar 2014 program report annex 1, Pg 7. Delhi 2013: "Once the procured syrups and donated tablets were delivered to Directorate of Health Services central storage room, they were tested in a government-accredited laboratory to ensure drug quality prior to administration." DtWI Delhi 2013 program report, Pg 6. From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Delhi 2013 program report, Pg 24. A table of results is also presented on the same page. Rajasthan 2013: "To instill confidence among the stakeholders that the drugs were of good quality, Deworm the World arranged for sample testing of the donated drugs by two independent labs7. Similarly, Rajasthan Medical Services Corporation sample tested the syrups they procured." DtWI Rajasthan 2013 program report, Pg 6. From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Rajasthan 2013 program report, Pg 40. A table of results is also presented on the following page. 96 "Now, state governments have responsibility for testing drugs. We have recently raised some concerns about the need to standardize the testing that is taking place, and are currently working with the MoHFW to build out the NDD operational guidelines with more detailed guidance on this point." Grace Hollister, edits to GiveWell's review, November 7, 2016 97 Comments from Deworm the World in response to a draft of this page in October 2017. 98 Of classes where monitors observed deworming activities in India, there were low numbers of adverse events (see table below). We don't know what portion of the adverse events may be caused by incorrect dosages. Bihar 2015 Rajasthan 2015 Delhi 2015 Madhya Pradesh 2015 Sample of question asked Classes where there were adverse events (monitors' observations) 5% (vomiting), 0% (diarrhea) 2% (vomiting), 0% (diarrhea) 5% (vomiting), 0.8% (diarrhea) 6% (vomiting), 0% (diarrhea) "Did you see any child with adverse effects (nausea, vomiting, stomachache, etc.) after taking the medicine?" [Table last updated in 2016] Sources for the information in the table: Bihar 2015: Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 4 Rajasthan 2015: Deworming Day and Mop-up Day ( N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 4 Delhi 2015: Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 6 Madhya Pradesh 2015: Deworming Day and Mop-up Day (N = 250 schools), DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 4 Sample question: DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 14 99 "Note that National Deworming Day operational guidelines state that only tablets should be used. Albendazole dosage is the same for all children aged 2 and above; it is only children aged 1-2 that require a different (half) dose." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 From Deworm the World's report on the Delhi 2013 program: "A key discussion from round one was the administration of drug to younger children in the pre-school age for whom chewing a tablet was seen as a difficulty. Hence keeping in mind the scale of the program, with an objective of making it a safe public health initiative the committee decided that deworming would be implemented across all districts following the World Health Organization (WHO) sanction in administering albendazole 400 mg tablets and the GOI guidelines under the WIFS program for administering the syrup vial. They also decided to opt for Albendazole 200mg dose suspension for 2-6 year children as it is a single dose for this age group which leads to lesser error in administration, while Albendazole 400mg tablets for older children." DtWI Delhi 2013 program report, Pg 6. The WHO factsheet on STH cites only a single recommended dosage (depending on which drug is used): "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet 100 Bihar 2015: In 8.8% of trained schools and 5.4% of untrained schools, monitors observed children given less than one tablet; in 2.6% of trained schools and 4.7% of untrained schools, monitors observed children given more than one tablet. DtWI Bihar 2015 independent monitoring tables, Pg 9 (Table 10) Rajasthan 2015: In 6.0% of schools, monitors observed children given less than one tablet; in 1.6% of schools, monitors observed children given more than one tablet. DtWI Rajasthan 2015 independent monitoring tables, Pg 2 (Table 2) Madhya Pradesh 2015: In 9.3% of schools, monitors observed children given less than one tablet; in 2.7% of schools, monitors observed children given more than one tablet. DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2 (Table 2) Delhi 2015: In 2.5% of schools and 6.0% of anganwadis, monitors observed children given less than the prescribed dose of albendazole; in 0.8% of schools and 10.6% of anganwadis, monitors observed children given more than the prescribed dose of albendazole. DtWI Delhi 2015 independent monitoring tables, Pgs 2, 16 (Tables S1 and A1) Bihar’s 2014: 8% of schools observed gave children less than one tablet and 2% gave more than one tablet. DtWI Bihar 2014 program report annex 2, Pgs 2 and 4. "In 3.1% of schools and anganwadis, monitors observed children being given more than one tablet/syrup bottle. As per protocol, the children should not have been given more than one tablet. Therefore in cases such as these, the monitors were trained to intervene and prevent the administration of an additional dose." (The percentage of schools using less than one pill/bottle per child was not reported.) DtWI Rajasthan 2013 program report, Pg 38. Monitors in Delhi in 2013 did not report on either of these observations. See DtWI Delhi 2013 program report, Pgs 21-25. 101 "The majority of direct program costs in India are government funded. There is a cap within the National Health Mission budgets on M&E – M&E cannot exceed 10% of the overall budget. So there are specific areas, such as program monitoring, that would likely not have investment at the level we are able to provide." Grace Hollister, edits to GiveWell's review, November 7, 2016 "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2. "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and Deworm the World's tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. Deworm the World called a random sample of 8,000 schools. The prevalence survey would not have happened without Deworm the World's support." DtWI 2013 GiveWell government interviews, Pg 5. 102 "RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3. 103 "[The Nodal Headmaster said] that most aspects of the program are excellent, but he had 2 suggestions: Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so) Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6. 104 According to Deworm the World, Cross River State, Nigeria, Year 2 Round 1 PMCV report, June 2017, Figure 14, children were observed fainting in approximately 70% of schools. Deworm the World attributes fainting to taking praziquantel on an empty stomach: "We had an instance where as high as 5 pupils from one school were reported to have fainted after taking PZQ in Ogoja LGA. A team of health workers and the LGA coordinator rushed to the school and discovered that the pupils fainted because they were dewormed too late (1:00 pm), six hours after their meal in the morning 8:00am) before coming to school. Henceforth, we advised the teachers and LGA teams not to deworm children with PZQ later than 11 am. This is because almost all children eat at home before coming to school and anything after long break, they won't have enough food left in their bellies." Deworm the World, email to GiveWell, May 30, 2018. 105 "In Tanzania matters came to a head in places around Morogoro in 2008. Distribution in schools of tablets for schistosomiasis and soil-transmitted helminths provoked riots, which had to be contained by armed police. It became a significant national incident, and one of the consequences has been the delay in Tanzania adopting a fully integrated NTD programme, and the scaling back some existing drug distributions." Allen and Parker 2011, pg. 109. "From these reports a number of problems with the MDA were raised which included fear of side effects from the tablets, particularly following the mass hysteria and death in Blantyre and Rumphi respectively and may explain some of the geographic heterogeneity seen. Furthermore most districts reported that MDA occurred after standard 8 students had finished exams and left school, and due to having inadequate resources for drug distribution...The side-effects incident in Blantyre and death in Rumphi had a large effect on districts and with many district reports stating that after the incidence many families refused to participate." SCI Malawi coverage survey 2012 Pgs 5, 21. 106 "THERE was confusion on Wednesday in some public primary and secondary schools in Ogun State, over the administration of anti-worm tablets. Nigerian Tribune gathered that some students reportedly collapsed in the cause of administering the tablets on them. This resulted into rumour that spread like wildfire across the length and breadth of the state, as parents stormed various school to withdraw their wards. When the Nigerian Tribune visited Egba High School, Asero and Asero High School both in Abeokuta South Local Government Area of the state, some parents were sighted at the school gate, who had come to confirm the incident and probably withdraw their wards. There was calmness in both schools as students in the Senior Secondary Classes were said to be preparing for their examinations. Meanwhile, the Ogun State Government through the State Commissioner for Health, Dr Babatunde Ipaye, has denied any case as a result of the anti-worm drug. Ipaye in a statement made available to the Nigerian Tribune in Abeokuta, said that no pupil or student to the best of his knowledge had reacted to the drug in the state. He explained that the exercise was done by his Ministry in collaboration with Evidence Action." Nigerian Tribune, "Panic in Ogun schools over deworm exercise," December 2017 107 "There were statewide rumors about children reacting badly to the medicines and other rumors about children dying after administration of medicine; this may have had an impact in that 90 (94%) of the 95 monitored schools administering the medicines on deworming day... On the deworming day, rumors broke out state-wide, originating from Saki LGA about children reacting to PZQ, unfortunately, in Surulere LGA, 2 out of 3 children from the same family who were all dewormed died on the deworming day, the death of these kids further worsened the situation as the news spread to neighboring LGAs. The LDA NTD coordinator, PHC coordinator, teachers from the children’s school and the LIE were promptly arrested by the police. Investigation surrounding the likely cause of their death was conducted at a teaching hospital in a nearby LGA, and it was confirmed that the kids died of food poisoning resulting from consumption of fruits sprayed with herbicides. This has been established as the cause of their death and the case has since been closed. Unfortunately, the damage to the SBD which coincided with the period had been done." Deworm the World, Oyo State PM report, February 2018, Pg 1. 108 "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 Deworm the World noted that the costs in Kenya are high, partly due to higher quality M&E: "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 In the second round of treatment in Kenya, prevalence surveys were 11% of the total costs and other M&E was 6%. Per treatment, these costs are $0.059 ($379,523/6,405,462) and $0.035 ($222,750/6,405,462) respectively DtWI Kenya 2013-2014 cost per treatment data "The cost per treatment in Kenya is higher than in India. This is in part because in Kenya Deworm the World is treating for schistosomiasis in addition to STH, and schistosomiasis treatments incur greater costs, including a higher drug cost and more extensive mapping. Other factors include the cost of additional supplies, higher operational costs, teacher allowances, and working on a smaller scale, which is less cost-effective. There are 6 million children in need of treatment in Kenya and the WHO estimates 241 million children at risk in India." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015. Deworm the World has also noted that treating schistosomiasis in Kenya increases the costs: "The extra costs of treating schistosomiasis in addition to STH. Schistosomiasis drugs tend to be more expensive than STH drugs and, in Kenya, schistosomiasis treatment sites are sometimes much more remote. The treatment strategy for schistosomiasis also differs from STH because schistosomiasis is more localized (e.g., it is not necessarily ideal to treat an entire sub-county). This also makes mapping schistosomiasis more expensive." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 3. 109 Deworm the World estimates that the Cross River program in 2016 cost a total of $0.69 per child and the Kenya program cost $0.45 per child in 2015-2016. See this spreadsheet, sheet "Cross River, Nigeria 2016," cell C11, and this spreadsheet, sheet "Kenya 2015-16," cell G11. Note that we make several adjustments to these figures to get our estimate of total cost per treatment (see sheet "Summary" in this spreadsheet). 110 We estimated the staff time costs based on Deworm the World's estimate of similar costs from the same states in 2012. It had imputed those costs based on estimates of government employee salaries, for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50). 111 "We include all partners’ expenditures in determining costs for the the deworming programs, but we do not consider spending that would be incurred even without deworming taking place. [...] Teachers’ and principals’ general salaries are not included because they do not spend additional time on deworming beyond what they are already compensated for by the government for regular classroom teaching." Evidence Action, blog post, July 5, 2016 112 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Deworm the World thought that the costs of these activities were quite low (less than 5%). We have assumed 3.5% in our analysis. 113 See our discussion of the rationale and limitations of this estimate here. 114 $16.0 million in spending opportunities less $12.5 million in expected funding. 115 See this spreadsheet, "Available and expected funding" sheet, cell B9. 116 See this spreadsheet, "Available and expected funding" sheet, row 17. 117 This is based on internal records of how much GiveWell-influenced donors gave to Deworm the World in 2018 when Deworm the World was on GiveWell's top charity list but was not the recommendation for marginal funding. See this spreadsheet, "Available and expected funding" sheet, row 22. 118 See this spreadsheet, "Available and expected funding" sheet, rows 21 and 32. 119 As of June 30, 2019, Evidence Action held $5.6 million in unrestricted funding (see this spreadsheet, sheet "Source: Unrestricted", cell B5). For 2020, it expected to have designated $2.3 million as operating reserves, allocated $1.0 million to the Dispensers for Safe Water program and $0.5 million to other uses (not including Deworm the World), and raised an additional $0.8 million, leaving it with an uncommitted unrestricted balance of $2.6 million (see this spreadsheet, sheet "Source: Unrestricted", cells E3:F8). Evidence Action told us it did not expect to allocate a portion of these funds to Deworm the World (John de Wet, Chief Financial & Administrative Officer, Evidence Action, conversation with GiveWell, August 22, 2019). As of June 30, 2018, Evidence Action held $2.2 million in unrestricted funding and projected that it would raise $0.6 million in unrestricted funding in the second half of 2018. It expected to allocate $0.5 million to Dispensers for Safe Water, $0.3 million to Evidence Action Beta, and no funding to Deworm the World. Evidence Action, Organizational Report 2017-2018, Unaudited, “Unrestricted forecasts” sheet. "At the time of submitting the 2016 report [to GiveWell], it was anticipated that Evidence Action would allocate unrestricted funds to Deworm the World Initiative in both 2016 and 2017. As greater visibility has been obtained into the organizational financial position [as part of the accounting clean up process], it became evident that the level of available and uncommitted unrestricted funding is very low—at the end of June 2017, unrestricted funds covered approximately 4 weeks of the organization's annual expenses. [...] For 2016 and 2017, given the low unrestricted funding levels, the organization sought to preserve these funds to serve as a minimal organizational backstop, and use these funds only for programs with significant and acute gaps." Evidence Action 2017 financials, GiveWell submission, "Unrestricted commitments" sheet. 120 See this spreadsheet, sheet "Source: DtWI RFMF", cells G15:G17. 523,677 (Oyo State) + 1,352,370 (Ogun State) + 916,328 (Rivers State) = 2,792,375. 121 Grace Hollister, Global Deworming Lead, Evidence Action, and Brett Sedgwick, Director, Deworm the World, conversation with GiveWell, May 9, 2019 122 Additional calculations in this spreadsheet, sheet "Spending opportunities," cells S17:S23. 123 “Deworm the World signed a memorandum of understanding (MOU) with the government of Kenya a few days ago confirming a second 5-year commitment.” GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017, Pg 2. 124 See this spreadsheet, sheet "Source: DtWI RFMF," row 13. 125 This summary is based on several conversations with Kanika Bahl, Evidence Action CEO, in 2018. 126 See this spreadsheet, sheet "Source: DtWI RFMF," rows 25-26: From cell H25: "Given that we… plan to allocate the additional ~$840k GiveWell is holding on our behalf to reserves, we require an additional $2.2M to reach our reserves target at the start of 2020. [In] 2021… we estimate requiring an additional $4.5M to comply with the organization's reserves policy. " $840,000 + $2.2 million + $4.5 million = $7.54 million. 127 See this spreadsheet, sheet "Source: DtWI RFMF," columns J:L. For calculations of the totals in each category see sheet "Spending opportunities." We have calculated this slightly differently than Deworm the World, based on how we allocated global team costs. 128 Grace Hollister, Global Deworming Lead, Evidence Action, and Brett Sedgwick, Director, Deworm the World, conversation with GiveWell, August 22, 2019 129 Grace Hollister, Global Deworming Lead, Evidence Action, and Brett Sedgwick, Director, Deworm the World, conversation with GiveWell, August 22, 2019 130 Evidence Action, Reserves Policy, 2018, Pg 5. 131 WHO, Summary of global update on preventive chemotherapy implementation in 2016, Pg 590, Table 1. WHO, Summary of global update on preventive chemotherapy implementation in 2015, Pg 456, Table 1. WHO Weekly epidemiological record, 18 December 2015, Pg. 707, Table 1.