We have published a more recent review of this organization. See our most recent report on Deworm the World.
Evidence Action's Deworm the World Initiative is one of our top-rated charities and we believe that it offers donors an outstanding opportunity to accomplish good with their donations.
More information: What is our evaluation process?
Published: November 2021
Summary
What do they do? Evidence Action's Deworm the World Initiative (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates government-run school-based deworming programs. (More)
Does it work? We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost. Deworm the World sends monitors to schools during and, for most distributions, after deworming to determine whether the programs it supports have reached a large proportion of children targeted. We have reviewed data from each of its major programs, which overall indicate strong results. (More)
What do you get for your dollar? Our best guess is that deworming is generally highly cost-effective. We estimate that, in Kenya, the cost per child dewormed is about $0.66 per child, or $0.46 per child excluding in-kind contributions from governments. We estimate that the cost per child treated in India is roughly half of that. We expect the cost per treatment in other countries to be closer to that in Kenya than India, and may be more expensive in the early stages of a program. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)
Is there room for more funding? Our most recent analysis finds that Deworm the World currently has $6.4 million in room for more funding—prior to receiving any grants that GiveWell specifically makes or recommends—to support its deworming portfolio in 2022-24. (More)
Evidence Action's Deworm the World Initiative is recommended because of its:
- Focus on a program with a strong track record and excellent cost-effectiveness. (More)
- Strong process for assessing whether the deworming programs it supports are successfully deworming children. (More)
- Standout transparency – it has shared significant, detailed information about its programs with us.
- Room for more funding – we believe Deworm the World will be able to use additional funds to start or maintain deworming programs.
Table of Contents
- Summary
- Our review process
- What do they do?
-
Does it work?
- Are mass school-based deworming programs effective when implemented well?
-
Are Deworm the World's programs targeted at areas of need?
- What is the likely impact per treatment in Deworm the World's programs compared with the independent studies on the impact of deworming? In general, mass deworming programs treat everyone in a targeted demographic, regardless of whether each individual is infected (more). Because of this, the benefits (and therefore the cost-effectiveness) of a program are highly dependent on the baseline prevalence of worm infections. In this section, we discuss how the disease burden in the areas Deworm the World works in compares to the places where the independent studies that form the evidence base for the impact of deworming were conducted. While it is our understanding that Deworm the World programs generally target areas that require mass treatment according to WHO guidelines,56 the disease burden in Deworm the World areas is on average lower than in the study areas, so our expectation is that the impact per child treated is lower in Deworm the World areas. We adjust our cost-effectiveness estimate accordingly (more below). In this spreadsheet, we compare the worm prevalence in places where Deworm the World currently supports a program to the prevalence from the studies providing the best evidence for the benefits of deworming. The prevalences in the table may not be directly comparable to one another. Prevalence surveys were conducted in Madhya Pradesh and Chhattisgarh after multiple rounds of treatment, in Bihar and Rajasthan after one round of treatment, and in Uttar Pradesh after multiple rounds of treatment in some districts and one round in other districts.57 It is likely that prevalence was higher at baseline for these regions. Deworm the World notes that with a few exceptions, it conducts prevalence surveys prior to supporting treatment.58 Deworm the World notes that there are relevant methodological differences between the prevalence surveys, which makes them difficult to compare;59 we agree that this data is not ideal for our purposes but believe that it provides the best estimate we have and adjusting for baseline infection rates is an important part of our cost-effectiveness model. Treatment for lymphatic filariasis In some of the countries where Deworm the World works, there are existing programs to treat lymphatic filariasis (LF).60 Albendazole, the same drug used to treat STH, is usually used in combination with one additional drug to treat LF (and the same dosage is used for both treatments).61 For areas that have existing LF treatment programs, the effect of Deworm the World’s support may be to transition an area from once-per-year deworming treatment (for STH) to twice-per-year treatment.62 We detail what we know about the status of LF programs in the areas in which Deworm the World works on a separate page with additional information about Deworm the World. Are deworming pills delivered to and ingested by recipients? The information we have seen from monitors hired and trained by Deworm the World in India, Kenya, Nigeria, and Pakistan suggests that the programs successfully deliver pills to children, who then swallow them. Additionally, prevalence surveys in Kenya, and to a lesser degree Bihar, India, show that the prevalence rates of STH and schistosomiasis have declined substantially since Deworm the World started supporting MDAs in those areas, providing additional evidence that the treatments are reaching recipients.63 Evidence from monitoring Deworm the World conducts monitoring to assess the quality of program implementation ("process monitoring") and to evaluate government reports of what proportion of the target population was reached with deworming treatment in the previous MDA ("coverage validation"). We use results from past MDAs to understand the impact we should expect future MDAs to have. Specifically, we use coverage validation results about the proportion of targeted children reached, along with data on program spending, to estimate the cost of reaching a child with deworming. Our interpretation of these coverage validation results is informed by their comprehensiveness and the methodology used to collect them. Comprehensiveness See this spreadsheet for all process monitoring and coverage validation results we have seen from Deworm the World's programs in India, Kenya, Nigeria, Pakistan, Ethiopia, and Vietnam. We focus our review on coverage validation results, which we believe provide the best indication of program impact. Specifically, we focus on results from India, Kenya, Nigeria, and Pakistan because those are Deworm the World's largest programs.64 We have seen coverage validation results from 2014-19 in Kenya and from 2015-19 in India; over the years, we have reviewed these results to varying degrees of depth due to their high degree of consistency across years. We have also reviewed coverage validation results from 2018-19 in Nigeria and 2019 in Pakistan, which was the first year that Deworm the World supported a program in that location. In general, in locations in Nigeria and Pakistan where Deworm the World has supported two deworming rounds per year,65 it has only conducted coverage validation for one of the two rounds, but conducted process monitoring for all rounds.66 We thus believe that we have seen a relatively thorough picture of the impact of Deworm the World's programs; we incorporate this assessment into our cost-effectiveness model.67 Methodology For each of its programs, Deworm the World hires monitors (who are not associated with the government implementing the program) to:68 [Sometimes] Make calls to communities and schools before Deworming Day. In India, Deworm the World asks monitors to call, unannounced, a random selection of schools and/or communities before Deworming Day. During the calls, monitors interview teachers, headmasters, and other functionaries, asking a variety of questions to assess preparedness, such as whether or not the school has enough treatments for Deworming Day and if a representative from the school attended training. 69 Observe activities on Deworming Day and Mop-Up Day. In all of its programs, Deworm the World sends its monitors, unannounced, to observe a random sample of schools on Deworming Day and Mop-Up Day.70 At the schools, monitors interview teachers and school administrators to assess how prepared the school was for Deworming Day. For example, monitors often ask if the school has sufficient drugs for Deworming Day, whether or not a school representative attended training, and a variety of questions to test teachers' knowledge about the proper procedures for the MDA campaign, such as what the teacher should do if a child is feeling sick.71 Then, the monitors observe randomly selected classes, recording details about the Deworming Day activities, such as whether deworming is in progress, teachers are documenting who is dewormed, and teachers are watching to make sure that students swallow the pills.72 In Kenya, Nigeria, and Pakistan, monitors visit communities surrounding schools, select a sample of adults to interview, and ask questions to assess their awareness of Deworming Day. This helps Deworm the World determine how successful its community sensitization efforts were.73 Conduct a coverage validation exercise. In India and Kenya, Deworm the World has typically sent out monitors within 1 to 2 weeks (though sometimes longer) of Deworming Day and Mop-Up Day to conduct a coverage validation exercise. It has also done so in the two most recent program years in Nigeria and in its first program year in Pakistan.74 In India, coverage validation takes place at a randomly selected sample of schools. In Kenya, Nigeria, and Pakistan, coverage validation takes place at households as well as schools in order to evaluate coverage for both enrolled and non-enrolled children.75 During household-based coverage validation, households are randomly selected, and then eligible children within households are interviewed about their experience on Deworming Day.76 During school-based coverage validation, monitors randomly select a small sample of students to interview, asking the children questions about their experience on Deworming Day, which are largely the same questions as those asked during household interviews.77 For example, they ask if the child received a pill and if the child swallowed the pill under supervision.78 In India, monitors also check class registers and record the number of students that were dewormed according to the school or anganwadi center's records.79 In India, Deworm the World interviews only enrolled children who were present on either Deworming Day or Mop-up Day in schools where deworming happened on at least one of those days;80 attendance data is collected separately in India to assess the rate of absenteeism on Deworming Day and Mop-up Day.81 In Kenya, Nigeria, and Vietnam, Deworm the World interviews a sample of all children, regardless of whether they were present on Deworming Day or Mop-up Day or whether they are enrolled or non-enrolled.82 While we believe that Deworm the World's monitoring is overall of high quality, we note a few ways in which its methodology may produce bias in the results: Process monitoring: Even though school staff are not aware ahead of time that a monitor is coming to visit, once a monitor arrives, staff may be motivated to execute the program in a more rigorous fashion than they would have otherwise. This potential bias is one of the reasons that we also rely on coverage validation that takes place in a different sample of schools after Deworming Day when assessing the quality of Deworm the World's programs. Reports we have seen indicate that, in general, monitors observe the deworming process in the vast majority (90% or above) of schools they visit in Kenya, Nigeria, and Pakistan, though there have been some exceptions (detailed in the footnote).83 This rate has been more variable in India, ranging from 24% to 87%.84 Low observation rates could be due to deworming not being in progress at the time the monitors conduct their observations, either because it has already been completed (which Deworm the World notes may often be the case), because it has not started yet, or because it is not taking place that day.85 Coverage validation: Deworm the World's program and monitoring are generally done through schools, which may lead non-enrolled children to be missed by the program. Schools are supposed to target and record the treatments they distribute to non-enrolled children.86 Most locations do not have reliable data on how many non-enrolled children there are.87 In Nigeria, Kenya, and Pakistan, Deworm the World's coverage validation has taken place in communities in addition to schools, in order to estimate coverage among all children in the targeted age range, including non-enrolled children.88 For the 2019 coverage validation results we have reviewed from Nigeria and Pakistan, only small samples of non-enrolled children were interviewed in households,89 suggesting that enrollment rates in the surveyed areas were high.90 A potential source of bias in Deworm the World's coverage validation surveys is the surveys' heavy reliance on self-reported responses from children. These responses are at risk of recall bias, though this concern is reduced by the fact that coverage validation has typically been conducted within a few weeks of MDA. They may also be at risk of social desirability bias: it is possible that children may overreport coverage if they feel pressure from teachers to report positively on their administration of deworming treatment, or if children believe that this is the preferred response of data collectors. Coverage validation has generally found very high proportions of students reporting being dewormed, with a coverage rate in one survey of 100%.91 Very high coverage numbers in India may be partially explained by the fact, discussed above, that Deworm the World interviews only children who were present on either Deworming Day or Mop-up Day in schools where deworming occurred on at least one of those days; see footnote for clarification from Deworm the World.92 Deworm the World tries to interview students away from their teachers, to reduce pressure, and also has some checks on students' answers.93 When children are interviewed at home rather than at school, their parents are usually present during the interview.94 Results We believe that results from Deworm the World's coverage validation, and to a lesser extent its process monitoring, provide relatively strong evidence that a high proportion of the target population has been reached with deworming treatment in past MDAs. We use coverage validation results about the proportion of targeted children reached, along with data on program spending, to estimate the cost of reaching a child with deworming. See this spreadsheet for all results we have reviewed from Deworm the World's programs. In short:95 In the majority of surveys we have reviewed, coverage of school-aged children was above 75% (the WHO-recommended minimum threshold). Across years, median coverage has been 97% in India (2015-19), 93% in Kenya (2014-19), 80% in Nigeria (2018-19), and 79% in Pakistan (2019). In the spreadsheet, we also report results we have reviewed from Deworm the World's process monitoring on Deworming Day. We have not prioritized reviewing results from the process monitoring Deworm the World conducts before Deworming Day, although we think it could also provide insight as to the quality of Deworm the World's program. Evidence from prevalence surveys over time Deworm the World or its partners have conducted surveys in India and Kenya to track changes in schistosomiasis and STH prevalence and intensity rates following Deworm the World-supported treatment programs. We have reviewed prevalence surveys conducted in Kenya from 2012 to 2016. In general, prevalence and intensity of the parasites decreased over this time period. Our understanding is that the most likely cause of this decrease was the national deworming program, which Deworm the World supports; alternative explanations for the decrease, such as the existence of other large-scale deworming programs or widespread changes in water and sanitation facilities or practices, seem to us to be unlikely. We have also reviewed two surveys of prevalence conducted in Bihar, India, at baseline and after several years of treatment; the results from the surveys are not directly comparable, but point in the direction of the program having an effect on prevalence levels. We discuss methodology, results, and limitations to using these surveys as evidence of Deworm the World's impact on another page. Deworm the World has also shared prevalence surveys conducted in Kenya and in two states in India in 2018. We have not yet reviewed their results or methodology in depth. How does Deworm the World affect program outcomes? Deworm the World may be having an impact in the following ways: It may increase the likelihood that a government implements a deworming program, by advocating for deworming programs, by offering to provide technical assistance, and/or by funding implementation. Deworm the World expects to pay for the majority of financial program costs in many of the new countries to which it has expanded or intends to expand.96 In situations where Deworm the World is funding a deworming program, we believe it is likely that Deworm the World plays an instrumental role in causing the program to happen.97 Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World. It may improve the quality of a deworming program that would have been implemented without Deworm the World (leading to more children dewormed effectively or improved cost-effectiveness). Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are uncertain about this. Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World. Are there any negative or offsetting impacts? In this section, we consider factors that are not accounted for in the above evidence that could offset the impact of Deworm the World's programs, either through reducing their effectiveness or contributing to negative outcomes. Drug quality: If drugs are not stored properly they may lose effectiveness or expire.98 Our understanding is that Deworm the World periodically tests the quality of drugs and has monitored storage conditions in each of its recent programs, and this information suggests there have been minimal issues.99 In India, state governments are responsible for procuring the drugs from local suppliers and quality testing them.100 Dosage: If the incorrect dosage is given, the drugs may not have the intended effect and/or children may experience additional side effects.101 It appears that for STH treatment, all children of a given age group are given the same dose of albendazole and that the dose is generally a single tablet for children 2 years old and above, and half a tablet for those between the ages of 1 and 2.102 Deworm the World monitors reported in 2015 (we haven't seen more recent data) that in programs in India, up to 9% of schools observed gave children less than the prescribed dose of albendazole and up to 5% gave more than the prescribed dose of albendazole.103 Replacement of government funding: We have limited information about whether governments would pay for the parts of the program paid for by Deworm the World in its absence, though our impression is that they would not.104 We also have little information about what governments would use deworming funds for if they did not choose to implement deworming programs. Diversion of skilled labor: Drug distribution occurs only once or twice per year and is conducted by teachers in schools. Based on our site visit in Rajasthan, our impression is that the Nodal Officer (the state official who manages all state school-based programs), the Nodal Officer’s staff, and the people that the Nodal Officer manages throughout the state (Resource Persons and Community Development Project Officers) have significant capacity to take on additional programs, so their taking on this program doesn't impose a significant burden on their time.105 On the other hand, a principal we spoke with commented that he would prefer fewer school-based health programs because they take focus away from the school day.106 Adverse effects and unintended consequences of taking deworming drugs: Our understanding is that expected side effects are minimal and there is little reason to be concerned that drug resistance is currently a major issue. We note that fainting was common in at least one MDA supported by Deworm the World; this MDA included praziquantel for schistosomiasis, and Deworm the World attributes the fainting cases to students taking the drug with empty stomachs.107 More information in our report on deworming. Popular discontent: We have heard a couple of accounts of discontent in response to mass drug administration campaigns supported by the SCI Foundation, including one case that led to riots.108 Additionally, during deworming activities supported by Evidence Action's Deworm the World Initiative in Ogun State, Nigeria in December 2017, rumors of students collapsing reportedly generated panic that led some parents to take their children out of school; the Ogun State government denied that any students collapsed.109 During deworming activities supported by Deworm the World in Oyo State, Nigeria in February 2018, rumors of students reacting badly or dying after receiving deworming drugs spread, and several schools canceled Deworming Day; an investigation found that the children likely died of food poisoning unrelated to deworming.110 What do you get for your dollar? This section was last updated in November 2020. The information that Deworm the World has provided since that date is not yet reflected in this section. This section examines the data that we have to inform our estimate of the expected cost-effectiveness of additional donations to Deworm the World. Note that the number of lives significantly improved is a function of a number of difficult-to-estimate factors. We incorporate these into a cost-effectiveness model which is available here. In this section, we focus on the cost per child treated, which is an important input in our cost-effectiveness model. On a separate page, we discuss: How accurate are Deworm the World's reported coverage figures? Deworm the World uses government-reported figures on the number of treatments delivered when calculating its cost per treatment. We remain uncertain of the accuracy of these figures. More here. Does Deworm the World "leverage" government funds, such that its activities mobilize resources from other actors? We could imagine that Deworm the World's funds have substantial leverage but could also imagine that other actors’ involvement is causing Deworm the World to pay for things for which other actors would otherwise have paid. Deworm the World may have less leverage in its future programs than it has had in past programs. More here. What is the cost per treatment? We estimate that in India children are dewormed for a total of about $0.35 per child, or $0.10 per child excluding the value of teachers’ and principals’ time spent on the program. In Kenya, we estimate the total cost per treatment at about $0.66 per treatment or $0.46 excluding the value of teachers’ and principals’ time spent on the program. We expect the cost per treatment in the countries Deworm the World has expanded to recently and may expand to in the future to be closer to its costs in Kenya in the long run, but higher than its costs in Kenya at first due to additional costs associated with starting a new program. Deworm the World told us that the cost per treatment in India is unusually low.111 We note that Deworm the World's cost per child analysis from Cross River, Nigeria for 2016 was about 50% higher than its estimate for Kenya in 2015-2016.112 Full details in this spreadsheet. Our approach Our cost per treatment analyses rely heavily on analyses Deworm the World has done to allocate its own costs across programs, collect data on costs paid by other partners and governments, and estimate costs when data is not available. Deworm the World has told us that it tries to capture all costs of its program, regardless of who pays for the cost (including donated drugs). We have made several adjustments to Deworm the World's estimates: For India, about two thirds of the total cost comes from contributions of time from government employees—which have been monetized according to salary levels—rather than financial costs. Appropriately valuing that time spent, and estimating how much time is spent by teachers and others due to the deworming program, is likely the largest source of uncertainty in the cost-per-child-treated calculation for India.113 In its own calculations, Deworm the World excludes the value of government employees' time because the government would have incurred these costs in the absence of the program.114 There are a few high-level costs not directly attributable to programs that Deworm the World does not include (such as exploratory work in new geographies that does not lead to a new program).115 We have included an estimate of these costs in our analyses. For Kenya, Nigeria, Pakistan, and Vietnam we have included an estimate of in-kind contributions from governments. This estimate is very rough and uses the same approach that we use in estimating the total cost per treatment for SCI. In short, we assume that government contributions account for 30% of the cost of the program, based on a 2011 paper that analyzed the costs of an SCI program in Niger.116 We adjust the number of treatments reported by Deworm the World-supported programs by the attendance and coverage rates found in Deworm the World's monitoring. Details of this adjustment are in this spreadsheet, "No. of children dewormed" sheets. We start with this total cost figure and apply adjustments in our cost-effectiveness analysis to account for cases where we believe the charity's funds have caused other actors to shift funds from a less cost-effective use to a more cost-effective use ("leverage") or from a more cost-effective use to a less cost-effective use ("funging"). Is there room for more funding? We conduct "room for more funding" analysis to understand what portion of Deworm the World's ideal future budget it will be unable to support with the funding it has or should expect to have available. We may then choose to either make or recommend grants to support those unfunded activities. Our most recent analysis finds that Deworm the World currently has $6.4 million in room for more funding—prior to receiving any grants that GiveWell specifically makes or recommends—to support its deworming portfolio in 2022-24. Room for more funding analysis In general, we assess top charities' funding needs over a three-year period.117 We ask top charities to report their ideal budgets over the next three years, along with information about their current available funding and funding pipeline. The difference between a charity's three-year budget and the funding we project that it will have available to support that budget is the charity's "room for more funding." For this analysis, we focus on the portion of Deworm the World's portfolio supported by funding that can be used flexibly across locations. We exclude the portion of Deworm the World's portfolio supported by funding restricted to specific locations. The main components of our room for more funding analyses are: Available funding. We ask top charities to report how much funding they currently hold in the bank, including in reserves,118 and how much of this funding is committed or expected to be spent on specific future activities. The difference between these figures is the amount available to allocate to the charity's unfunded spending opportunities. Expected funding. We project the amount of additional funding that top charities will receive to support their work over the next three years. These projections represent our best guesses based on top charities' past revenue and our understanding of their funding pipelines. They typically include funding currently held by GiveWell to be granted to the top charity, projected funding due to being a GiveWell top charity,119 and, if the top charity is part of a larger organization, projected unrestricted funding from that parent organization. They exclude any funding we may specifically recommend to the top charity subsequent to the analysis. We add this projected funding to the amount available to allocate to the charity's unfunded spending opportunities. Spending opportunities. We ask top charities to report their ideal budgets in each of the next three years and to provide details on the specific spending opportunities included in these budgets. These opportunities are typically presented as one program year in a specific implementation geography (for example, deworming in Kenya in 2024), and they can represent either an extension of the top charity's previous support to a geography or an expansion of support to a new geography. We ask top charities to report the order in which they would prioritize funding these opportunities, which helps us to understand how available and expected funding will be allocated and what the marginal impact of additional funding beyond that amount would be. A charity's room for more funding represents the total budget for the charity's spending opportunities, less its available and expected funding. For example, if a charity proposes spending $50 million over the next three years and holds $10 million in uncommitted funding, and we project that it will receive an additional $15 million in revenue over the next three years, that charity's room for more funding is $25 million. (Note that a charity's total room for more funding figure includes funding gaps at all levels of cost-effectiveness—see below.) Our most recent analysis of Deworm the World's room for more funding can be found in this spreadsheet. Grant investigation process Room for more funding analysis is a key part of our grant investigation process. We periodically request the information described above from top charities and update our room for more funding analyses. Our default is to update each top charity's room for more funding analysis annually, though we may choose to do so more or less frequently. The cadence on which we conduct updates depends largely on how often we grant funding to a top charity120 and how much we expect that charity's funding and budgets to have changed since our most recent funding decision.121 We have typically updated our analysis of Deworm the World's room for more funding on an annual basis. Our most recent analysis of Deworm the World's room for more funding can be found in this spreadsheet. After completing such an update, we may then choose to investigate potential grants to support the spending opportunities that we do not expect to be funded with the charity's available and expected funding, which we refer to as "funding gaps." The principles we follow in deciding whether or not to fill a funding gap are described on this page. The first of those principles is to put significant weight on our cost-effectiveness estimates. We use GiveDirectly's unconditional cash transfers as a benchmark for comparing the cost-effectiveness of different funding gaps, which we describe in multiples of "cash." Thus, if we estimate that a funding gap is "10x cash," this means we estimate it to be ten times as cost-effective as unconditional cash transfers. As of this writing, we have typically funded opportunities that meet or exceed a relatively high bar: 8x cash, or eight (or more) times as cost-effective as GiveDirectly's unconditional cash transfers. (Note that a charity's total room for more funding figure includes funding gaps at all levels of cost-effectiveness.) If we decide to fill a funding gap, we either make a grant from our Maximum Impact Fund122 or recommend that another funder—typically Open Philanthropy123 —makes a grant. [Note added October 2022: We updated the name of this fund in September 2022; more information here.] This page lists all grants made or recommended by GiveWell. Typically, when GiveWell donors make a donation to a top charity,124 we don't expect that donation to be directed to a specific funding gap, but rather to contribute to supporting the overall portfolio of opportunities included within a charity's room for more funding. Deworm the World's room for more funding Our most recent analysis of Deworm the World's room for more funding can be found in this spreadsheet. In that analysis, we estimate that Deworm the World has $6.4 million in room for more funding—prior to receiving any grants that GiveWell specifically makes or recommends—to support its deworming portfolio in 2022-24. In sheet "RFMF projections," we present this figure broken down by funding gap. We will consider making or recommending grants to fill each of these funding gaps. As of this writing, we expect to have sufficient funding at our discretion to fill all funding gaps we identify among our top charities that meet our current cost-effectiveness bar of 8x cash or better; we also expect to fill some funding gaps in the 5-8x cash range (for more details, see this blog post). Our top recommendation to GiveWell donors seeking to do the most good possible is to donate to the Maximum Impact Fund, which we allocate quarterly to the charity with the highest-priority funding gap(s). [Note added October 2022: We updated the name of this fund in September 2022; more information here.] Evidence Action as an organization We use qualitative assessments of our top charities to inform our funding recommendations. See this page for more information about this process and for our qualitative assessment of Evidence Action as an organization. Sources Document Source Alderman et al. 2006 Source (archive) Alix Zwane conversation August 30th 2013 Unpublished Alix Zwane conversation June 4th 2013 Source Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013 Unpublished Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 Unpublished Allen and Parker 2011 Source (archive) Assam 2010 guidelines for deworming Source Assam midday meal report 2013 Source (archive) Assam reproductive and child health 2011-2012 Source (archive) Assam state programme implementation plan 2011-2012 Source (archive) Baird et al 2012 Source Bleakley 2007 Source (archive) CIFF conversation September 10th 2013 Source Croke 2014 Source (archive) Devesh Kapur conversation October 14th 2013 Source Deworm the World 2015 Uttar Pradesh prevalence survey report Source Deworm the World and SCI, Ethiopia coverage survey 2015 Source Deworm the World Chhattisgarh Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World External Bihar 2016 Costing Model Source Deworm the World External Chhattisgarh 2016 Costing Model Source Deworm the World External Cross River Nigeria 2016 Costing Model Source Deworm the World External Kenya 2016 Costing Model Source Deworm the World External Madhya Pradesh 2016 Costing Model Source Deworm the World External Rajasthan 2016 Costing Model Source Deworm the World External Telangana 2016 Costing Model Source Deworm the World External Tripura 2016 Costing Model Source Deworm the World External Uttar Pradesh 2016 Costing Model Source Deworm the World Financials 2017 Source Deworm the World Jharkhand Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Kenya Year 4 Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Nigeria Cross River Independent Process Monitoring Report, November 2016 Source Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Unpublished Deworm the World target and treatment figures 2018 Source Deworm the World Telangana Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Tripura Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Uttar Pradesh Process Monitoring and Coverage Validation Report, August 2016 Source Deworm the World Vietnam Independent Monitoring Report, November-December 2016 Source Deworm the World, 2015 expense summary Unpublished Deworm the World, 2015 expense summary - by funder Unpublished Deworm the World, 2019 cost per child analysis guide Source Deworm the World, 2019 expenses Source Deworm the World, 2019 target and treatment figures Source Deworm the World, 2020 expense report Source Deworm the World, Bihar 2016 IMCV report Source Deworm the World, Bihar 2019 cost per child model Source Deworm the World, Bihar cost per child model 2017 Source Deworm the World, Bihar cost per child model 2018 Source Deworm the World, Bihar NDD PMCV Report Appendix, August 2017 Source Deworm the World, Bihar NDD PMCV report, August 2018 Source Deworm the World, Bihar NDD PMCV Report, February 2017 Source Deworm the World, Bihar NDD PMCV report, February 2018 Source Deworm the World, Bihar NDD Program Report, August 2017 Source Deworm the World, Bihar NDD Program Report, February 2017 Source Deworm the World, Bihar PM report, September 2020 Unpublished Deworm the World, Bihar PMCV report, August 2017 Source Deworm the World, Chhattisgarh 2016 IMCV report Source Deworm the World, Chhattisgarh 2019 cost per child model Source Deworm the World, Chhattisgarh cost per child model 2017 Source Deworm the World, Chhattisgarh cost per child model 2018 Source Deworm the World, Chhattisgarh NDD PMCV Report Appendix, August 2017 Source Deworm the World, Chhattisgarh NDD PMCV report, August 2018 Source Deworm the World, Chhattisgarh NDD PMCV Report, February 2017 Source Deworm the World, Chhattisgarh NDD PMCV report, February 2018 Source Deworm the World, Chhattisgarh NDD Program Report, August 2017 Source Deworm the World, Chhattisgarh NDD Program Report, February 2017 Source Deworm the World, Chhattisgarh PM report, February 2020 Unpublished Deworm the World, Chhattisgarh PM report, September-October 2020 Unpublished Deworm the World, Chhattisgarh PMCV report, August 2017 Source Deworm the World, Chhattisgarh prevalence survey report, August 2016 Source Deworm the World, Chhattisgarh Program Report, August 2016 Source Deworm the World, Cross River State PM report, May 2018 Source Deworm the World, Cross River State PMCV report, October 2018 Source Deworm the World, Cross River State process monitoring and coverage validation report, November 2019 Source Deworm the World, Cross River State, Nigeria Coverage Evaluation Survey 2017 Unpublished Deworm the World, Cross River State, Nigeria, 2016-2017 Treatment Data Source Deworm the World, Cross River State, Nigeria, Year 2 Round 1 PMCV report, June 2017 Source Deworm the World, Ethiopia independent monitoring report, Year 1 Source Deworm the World, Ethiopia prevalence survey report Source Deworm the World, Ethiopia Workplan Unpublished Deworm the World, Haryana 2019 cost per child model Source Deworm the World, Haryana NDD CV report, February 2018 Source Deworm the World, Haryana NDD PMCV report, August 2018 Source Deworm the World, Haryana PM report, February 2020 Unpublished Deworm the World, Haryana PM report, October 2020 Unpublished Deworm the World, ICT PMCV report, January 2020 Source Deworm the World, Islamabad Capital Territory process monitoring and coverage validation report, April 2019 Source Deworm the World, Islamabad Capital Territory process monitoring and coverage validation report, February 2019 Source Deworm the World, Jharkhand 2019 cost per child model Source Deworm the World, Jharkhand cost per child model 2017 Source Deworm the World, Jharkhand cost per child model 2018 Source Deworm the World, Jharkhand Independent Monitoring Report 2016 Unpublished Deworm the World, Jharkhand NDD Brief Report, August 2016 Source Deworm the World, Jharkhand NDD PMCV Report Appendix, August 2017 Source Deworm the World, Jharkhand NDD PMCV report, August 2018 Source Deworm the World, Jharkhand NDD PMCV Report, February 2017 Source Deworm the World, Jharkhand NDD PMCV report, February 2018 Source Deworm the World, Jharkhand NDD Program Report, August 2017 Source Deworm the World, Jharkhand NDD Program Report, February 2017 Source Deworm the World, Jharkhand PM report, September 2020 Unpublished Deworm the World, Jharkhand PMCV report, August 2017 Source Deworm the World, Jharkhand PMCV report, February 2020 Unpublished Deworm the World, Karnataka 2019 cost per child model Source Deworm the World, Karnataka cost per child model 2018 Source Deworm the World, Karnataka NDD PMCV Report Appendix, August 2017 Source Deworm the World, Karnataka NDD PMCV report, August 2018 Source Deworm the World, Karnataka NDD PMCV report, February 2018 Source Deworm the World, Karnataka PM report, September 2020 Unpublished Deworm the World, Karnataka PMCV report, August 2017 Source Deworm the World, Karnataka PMCV report, February 2020 Unpublished Deworm the World, Kenya 2014-2015 program report Source Deworm the World, Kenya 2019 cost per child model Source Deworm the World, Kenya cost per child model 2017 Source Deworm the World, Kenya cost per child model 2018 Source Deworm the World, Kenya Coverage Reporting data, Year 3 Source Deworm the World, Kenya Deworming Day data, Year 3 Source Deworm the World, Kenya Narrative Report - Year 1 Source Deworm the World, Kenya Narrative Report - Year 2, Quarter 4 Source Deworm the World, Kenya Narrative Report - Year 3, Quarter 3 Source Deworm the World, Kenya NSBD PMCV Report Year 5 Source Deworm the World, Kenya NSBD PMCV report, Y6 2018 Source Deworm the World, Kenya NSBD Program Report Year 5 Source Deworm the World, Kenya NSBDP Coverage Validation Report Year 4 Source Deworm the World, Kenya NSBDP PMCV Report Year 5, CIFF Source Deworm the World, Kenya NSBDP PMCV Report, END Fund, July 2017, Year 5 Wave 1 Source Deworm the World, Kenya NSBDP PMCV Trends, Years 2-5, CIFF Source Deworm the World, Kenya PMCV Report Year 4 2016, Wave 1 Source Deworm the World, Kenya PMCV Report Year 4 2016, Wave 2 Source Deworm the World, Kenya PMCV report, 2020 Source Deworm the World, Kenya process monitoring report, Year 4 Source Deworm the World, Kenya remote coverage validation methods report, August 2020 Source Deworm the World, Kenya STH and Schistosomiasis Treatment Report Year 3 2015 Source Deworm the World, Kenya STH and Schistosomiasis Treatment Report Year 4 2016 Source Deworm the World, Kenya Year 2, DD - Main instrument Source Deworm the World, Kenya Year 2, Pre DD - School instrument Source Deworm the World, Kenya Year 3, DD - Main instrument Source Deworm the World, Kenya Year 3, Post DD - Coverage instrument Source Deworm the World, Kenya Year 3, Pre TT form Source Deworm the World, Khyber Pakhtunkhwa process monitoring and coverage validation report, October 2019 Source Deworm the World, Madhya Pradesh 2015 prevalence survey report Source Deworm the World, Madhya Pradesh 2016 IMCV report Source Deworm the World, Madhya Pradesh 2019 cost per child model Source Deworm the World, Madhya Pradesh cost per child model 2017 Source Deworm the World, Madhya Pradesh cost per child model 2018 Source Deworm the World, Madhya Pradesh cost-per-treatment - 2015 Unpublished Deworm the World, Madhya Pradesh NDD PMCV Report, February 2017 Source Deworm the World, Madhya Pradesh NDD PMCV report, February 2018 Source Deworm the World, Madhya Pradesh NDD Program Report, February 2017 Source Deworm the World, Madhya Pradesh PM report, September-October 2020 Unpublished Deworm the World, National Deworming Day states, August 2016 Source Deworm the World, Nigeria 2019 cost per child model Source Deworm the World, Nigeria cost per child model 2017 Source Deworm the World, Nigeria cost per child model 2018 Source Deworm the World, Ogun State PM report, June 2018 Source Deworm the World, Ogun State PMCV report, November 2020 Source Deworm the World, Ogun State process monitoring and coverage validation report, June 2019 Source Deworm the World, Ogun State process monitoring and coverage validation report, November 2019 Source Deworm the World, Oyo State PM report, February 2018 Source Deworm the World, Oyo State PMCV report, November 2020 Source Deworm the World, Oyo State PMCV report, October 2018 Source Deworm the World, Oyo State process monitoring and coverage validation report, July 2019 Source Deworm the World, Oyo State process monitoring and coverage validation report, November 2019 Source Deworm the World, Pakistan 2019 cost per child model Source Deworm the World, Rajasthan 2016 IMCV report Source Deworm the World, Rajasthan 2019 cost per child model Source Deworm the World, Rajasthan cost per child model 2017 Source Deworm the World, Rajasthan cost per child model 2018 Source Deworm the World, Rajasthan NDD PMCV Report, February 2017 Source Deworm the World, Rajasthan NDD PMCV report, February 2018 Source Deworm the World, Rajasthan NDD Program Report, February 2017 Source Deworm the World, Rajasthan PM report, October 2020 Unpublished Deworm the World, responses to GiveWell questions, May 16, 2019 Unpublished Deworm the World, Rivers State PM report, June 2018 Source Deworm the World, Rivers State PMCV report, January 2021 Source Deworm the World, Rivers State PMCV report, November 2018 Source Deworm the World, Rivers State process monitoring and coverage validation report, July 2019 Source Deworm the World, Rivers State process monitoring and coverage validation report, November 2019 Source Deworm the World, Sindh PM report, January 2020 Source Deworm the World, Strategic refresh, 2020 Unpublished Deworm the World, target populations and parasitology data Source Deworm the World, Telangana 2016 IMCV report Source Deworm the World, Telangana 2019 cost per child model Source Deworm the World, Telangana cost per child model 2017 Source Deworm the World, Telangana cost per child model 2018 Source Deworm the World, Telangana NDD PMCV Report Appendix, August 2017 Source Deworm the World, Telangana NDD PMCV Report, February 2017 Source Deworm the World, Telangana NDD PMCV report, February 2018 Source Deworm the World, Telangana NDD Program Report, August 2017 Source Deworm the World, Telangana NDD Program Report, February 2017 Source Deworm the World, Telangana PM report, October 2020 Unpublished Deworm the World, Telangana PMCV report, August 2017 Source Deworm the World, Telangana Program Report, August 2016 Source Deworm the World, Tripura 2016 IMCV report Source Deworm the World, Tripura 2019 cost per child model Source Deworm the World, Tripura cost per child model 2017 Source Deworm the World, Tripura cost per child model 2018 Source Deworm the World, Tripura NDD PMCV Report Appendix, August 2017 Source Deworm the World, Tripura NDD PMCV report, August 2018 Source Deworm the World, Tripura NDD PMCV Report, February 2017 Source Deworm the World, Tripura NDD PMCV report, January 2018 Source Deworm the World, Tripura NDD Program Report, August 2017 Source Deworm the World, Tripura NDD Program Report, February 2017 Source Deworm the World, Tripura PM report, September-October 2020 Unpublished Deworm the World, Tripura PMCV report, August 2017 Source Deworm the World, Tripura PMCV Report, February 2016 Source Deworm the World, Tripura Program Report, August 2016 Source Deworm the World, Uttar Pradesh 2016 IMCV report Source Deworm the World, Uttar Pradesh 2019 cost per child model Source Deworm the World, Uttar Pradesh cost per child model 2017 Source Deworm the World, Uttar Pradesh cost per child model 2018 Source Deworm the World, Uttar Pradesh NDD PMCV Report Appendix, August 2017 Source Deworm the World, Uttar Pradesh NDD PMCV report, August 2018 Source Deworm the World, Uttar Pradesh NDD PMCV Report, February 2017 Source Deworm the World, Uttar Pradesh NDD PMCV report, February 2018 Source Deworm the World, Uttar Pradesh NDD Program Report, August 2017 Source Deworm the World, Uttar Pradesh NDD Program Report, February 2017 Source Deworm the World, Uttar Pradesh NDD Program Report, September 2016 Source Deworm the World, Uttar Pradesh PM report, August-October 2020 Unpublished Deworm the World, Uttar Pradesh PMCV report, August 2017 Source Deworm the World, Uttar Pradesh prevalence survey report, December 2015 Source Deworm the World, Uttarakhand 2019 cost per child model Source Deworm the World, Uttarakhand NDD CV report, February 2018 Source Deworm the World, Uttarakhand NDD PMCV report, August 2018 Source Deworm the World, Uttarakhand PM report, September-October 2020 Unpublished Deworm the World, Uttarakhand PMCV report, February 2020 Unpublished Deworm the World, Vietnam 2016 monitoring survey form for coverage validation Source Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day Source Deworm the World, Vietnam baseline prevalence survey - 4 provinces Source Deworm the World, Vietnam cost per child model 2017 Source Deworm the World, Vietnam cost per child model 2018 Source Deworm the World, Vietnam final report for STH survey in 21 provinces Source Deworm the World, Vietnam IM Report on MDA 1, April 2016 Source Deworm the World, Vietnam IM Report on MDA 2, November 2016 Source Deworm the World, Vietnam Independent Monitoring Report on MDA 3, November 2017 Source Deworm the World, Vietnam Independent Monitoring Report, 2016 Source Deworm the World, Vietnam NIMPE Report on MDA 1, April 2016 Source Deworm the World, Vietnam NIMPE Report on MDA 2, November 2016 Source Deworm the World, Vietnam NIMPE Report on MDA 3, November 2017 Source DSW 2012 GiveWell site visit Source DtWI 2013 GiveWell government interviews Source DtWI 2013 GiveWell site visit Source DtWI Assam research 2013 Source DtWI Bihar 2011 cost data Source DtWI Bihar 2011 coverage data Source DtWI Bihar 2011 monitoring data for deworming day Source DtWI Bihar 2011 monitoring data for mop-up day Source DtWI Bihar 2011 Monitoring Form for Deworming Day Source DtWI Bihar 2011 monitoring form for mop-up day Source DtWI Bihar 2011 prevalence survey report Source DtWI Bihar 2011 program report Source DtWI Bihar 2012 cost data Source DtWI Bihar 2012 cost data details Unpublished DtWI Bihar 2012 coverage data Source DtWI Bihar 2012 monitoring data for coverage validation Source DtWI Bihar 2012 monitoring report Source DtWI Bihar 2014 cost data Unpublished DtWI Bihar 2014 program report Source DtWI Bihar 2014 program report annex 1 Source DtWI Bihar 2014 program report annex 2 Source DtWI Bihar 2015 independent monitoring tables Source DtWI Bihar 2015 monitoring data for coverage validation, schools Source DtWI Bihar 2015 monitoring data from deworming day, schools Source DtWI Bihar 2015 monitoring data from mopup day, schools Source DtWI Bihar 2015 monitoring survey for coverage validation, schools Source DtWI Bihar 2015 monitoring survey from deworming day, schools Source DtWI Bihar 2015 monitoring survey from mopup day, schools Source DtWI Bihar 2015 Prevalence Survey report Source DtWI Bihar 2015 Program report Source DtWI Bihar Feb 2016 Program report Source DtWI budget vs actual spending of Good Ventures 2013 grant, October 2015 Unpublished DtWI Chhattisgarh 2015 coverage validation report Source DtWI Chhattisgarh 2015 coverage validation tables Unpublished DtWI Chhattisgarh 2015 independent monitoring tables Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring data for coverage validation, schools Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, anganwadis Source DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools Source DtWI Chhattisgarh Feb 2016 Program report Source DtWI class register audits 2013 Source DtWI cost narrative 2013 Source DtWI Cost per treatment blog post January 2015 Source (archive) DtWI cost per treatment summary 2013 Source DtWI coverage data 2013 - 2014 Source DtWI Delhi 2012 cost data Source DtWI Delhi 2012 coverage data by anganwadi Source DtWI Delhi 2012 coverage data by school Source DtWI Delhi 2012 coverage report Source DtWI Delhi 2012 monitoring data Source DtWI Delhi 2012 monitoring form deworming day Source DtWI Delhi 2012 prevalence survey design Source DtWI Delhi 2012 prevalence survey report Source DtWI Delhi 2012 program report Source DtWI Delhi 2013 cost data Unpublished DtWI Delhi 2013 program report Source DtWI Delhi 2015 independent monitoring tables Source DtWI Delhi 2015 monitoring data for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring data for coverage validation, schools Source DtWI Delhi 2015 monitoring data from deworming day, anganwadis Source DtWI Delhi 2015 monitoring data from deworming day, schools Source DtWI Delhi 2015 monitoring data from mopup day, anganwadis Source DtWI Delhi 2015 monitoring data from mopup day, schools Source DtWI Delhi 2015 monitoring survey for coverage validation, anganwadis Source DtWI Delhi 2015 monitoring survey for coverage validation, schools Source DtWI Delhi 2015 monitoring survey from deworming day, anganwadis Source DtWI Delhi 2015 monitoring survey from deworming day, schools Source DtWI Delhi 2015 monitoring survey from mopup day, anganwadis Source DtWI Delhi 2015 monitoring survey from mopup day, schools Source DtWI Delhi 2015 program report Source DtWI External Vietnam Costing Model Source DtWI Kenya 2013-2014 cost per treatment data Source DtWI Kenya 2013-2014 program report Source DtWI Kenya 2015-16 program report Source DtWI Madhya Pradesh 2015 coverage validation form Unpublished DtWI Madhya Pradesh 2015 deworming day monitoring form Unpublished DtWI Madhya Pradesh 2015 independent monitoring tables Source DtWI Madhya Pradesh 2015 monitoring data for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring data from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring data from mopup day, schools Source DtWI Madhya Pradesh 2015 monitoring survey for coverage validation, schools Source DtWI Madhya Pradesh 2015 monitoring survey from deworming day, schools Source DtWI Madhya Pradesh 2015 monitoring survey from mopup day, schools Source DtWI Madhya Pradesh 2015 program report Source DtWI Madhya Pradesh Feb 2016 program report Source DtWI Monitoring Improvements 2014 Source DtWI NDD blog post February 2015 Source (archive) DtWI NDD Year 1 M&E review July 2015 Source DtWI Rajasthan 2012 cost data Source DtWI Rajasthan 2012 cost data details Unpublished DtWI Rajasthan 2012 coverage data for anganwadi Source DtWI Rajasthan 2012 coverage data for schools Source DtWI Rajasthan 2012 monitoring data for coverage validation in anganwadis Source DtWI Rajasthan 2012 monitoring data for coverage validation in schools Source DtWI Rajasthan 2012 monitoring form coverage day Source DtWI Rajasthan 2012 monitoring form pre-deworming day Source DtWI Rajasthan 2012 monitoring report Source DtWI Rajasthan 2012 prevalence survey report Source DtWI Rajasthan 2013 cost data Unpublished DtWI Rajasthan 2013 prevalence survey report Source DtWI Rajasthan 2013 program report Source DtWI Rajasthan 2015 independent monitoring tables Source DtWI Rajasthan 2015 monitoring data for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring data for coverage validation, schools Source DtWI Rajasthan 2015 monitoring data from deworming day, schools Source DtWI Rajasthan 2015 monitoring data from mopup day, schools Source DtWI Rajasthan 2015 monitoring survey for coverage validation, anganwadis Source DtWI Rajasthan 2015 monitoring survey for coverage validation, schools Source DtWI Rajasthan 2015 monitoring survey from deworming day, schools Source DtWI Rajasthan 2015 monitoring survey from mopup day, schools Source DtWI Rajasthan 2015 program report Source DtWI Rajasthan Feb 2016 Program report Source DtWI Telangana Feb 2016 Program report Source DtWI Uttar Pradesh Feb 2016 Program report Source Evidence Action 2014 budget Unpublished Evidence Action 2015 draft budget Unpublished Evidence Action 2015 funding gap analysis Source Evidence Action 2016-2017 Revenue Report Source Evidence Action 2017 financials, GiveWell submission Source Evidence Action cover letter 2013 Source Evidence Action launch announcement 2013 Source (archive) Evidence Action Q1 financials, 2016 Unpublished Evidence Action website 2013 Source (archive) Evidence Action website announcement April 2014 Source (archive) Evidence Action website, Deworm the World Initiative (December 2016) Source Evidence Action website, Deworm the World Initiative (March 2016) Source (archive) Evidence Action website, Deworm the World Initiative (October 2015) Source (archive) Evidence Action website, Evidence Action Beta (October 2015) Source (archive) Evidence Action website, Who we are (November 2016) Source (archive) Evidence Action, 2015 financials by program Unpublished Evidence Action, Baseline Survey Report of Soil-Transmitted Helminths Prevalence in Pakistan Source Evidence Action, blog post, April 27, 2016 Source (archive) Evidence Action, blog post, August 1, 2016 Source Evidence Action, blog post, December 21, 2015 Source (archive) Evidence Action, blog post, January 16, 2015 Source (archive) Evidence Action, blog post, January 8, 2015 Source (archive) Evidence Action, blog post, July 5, 2016 Source (archive) Evidence Action, blog post, June 12, 2015 Source (archive) Evidence Action, blog post, June 30, 2016 Source (archive) Evidence Action, Organizational Report 2017-2018, Unaudited Source Evidence Action, Projected allocation of unrestricted funds, 2016 Source Evidence Action, Reserves Policy, 2018 Source Evidence Action, Revenue Report 2017-2018, Unaudited Source GiveWell analysis of Deworm the World 2014 Financial summary Source GiveWell analysis of Deworm the World cost per treatment Source GiveWell analysis of Deworm the World cost-per-treatment, 2016 Source GiveWell analysis of Deworm the World cost-per-treatment, October 2016 Source GiveWell analysis of Deworm the World financials - 2016 Source GiveWell DtWI 2013-2014 cost data summary Source GiveWell enrollment-based student coverage check 2015 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 Source GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013 Source GiveWell's non-verbatim summary of a conversation with END Fund staff, October 17, 2016 Source GiveWell's notes from site visit to India, October 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014 Source GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015 Source GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015 Unpublished GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014 Source GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014 Source GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 Source Global Alliance to Eliminate Lymphatic Filariasis - Prevention Source (archive) Grace Hollister conversation June 19th 2013 Source Grace Hollister, conversation with GiveWell, April 20, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 11, 2016 Unpublished Grace Hollister, conversation with GiveWell, August 24, 2016 Unpublished Grace Hollister, conversation with GiveWell, July 25, 2016 Unpublished Grace Hollister, conversation with GiveWell, June 13, 2016 Unpublished Grace Hollister, conversation with GiveWell, May 5, 2016 Unpublished Grace Hollister, conversation with GiveWell, September 1, 2016 Unpublished Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 Unpublished Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, March 23, 2015 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016 Unpublished Grace Hollister, Deworm the World Director, email to GiveWell, October 4, 2016 Source Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 Unpublished Grace Hollister, edits to GiveWell's review, November 20, 2016 Unpublished Grace Hollister, edits to GiveWell's review, November 7, 2016 Unpublished Grace Hollister, email to GiveWell, June 9, 2016 Unpublished Grace Hollister, email to GiveWell, March 6, 2016 Unpublished Grace Hollister, email to GiveWell, September 13, 2016 Unpublished Harvard Business School Kenya Case Study A 2010 Unpublished India Ministry of Health and Family Welfare Deworming Guidelines Draft 2015 Source India NDD documents 2015 Source (archive) Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014 Unpublished JPAL CEAs in education 2011 Source Kabatereine et al. 2001 Source (archive) KEMRI prevalence report - Year 2 Unpublished Kenya National School Based Deworming Programme Year 4 2016 Impact Analysis Source LF treatment coverage (2004-2014) Source (archive) LF treatment coverage 2015 Unpublished LF treatment drugs 2012 Unpublished Miguel and Kremer 2004 Source Mwandawiro et al. 2013 Source (archive) National Vector Borne Disease Control Programme, All About Lymphatic Filariasis Source (archive) Neetu Chandra Sharma, Daily Mail - India article, August 8, 2016 Source (archive) Nigerian Tribune, "Panic in Ogun schools over deworm exercise," December 2017 Source (archive) Paul Byatta, attachments to email to GiveWell, September 23, 2016 Source Paul Byatta, conversation with GiveWell, September 20, 2016 Unpublished Paul Monaghan, conversation with GiveWell, September 8, 2016 Unpublished Preventive chemotherapy in human helminthiasis 2006 Source (archive) Professor Devesh Kapur Biography 2013 Source (archive) Reserve Bank of India, GDP per capita, Table 10, September 16, 2015 Source (archive) SCI Malawi coverage survey 2012 Source SCI, Ethiopia coverage survey dashboard 2016 Source SCI, Ethiopia coverage survey protocol 2017 Source Sedgewick and Wang’ombe 2020 Unpublished STH coalition framework for action November 2014 Source (archive) U-DISE Elementary Thematic Maps 2015 Source (archive) U-DISE Secondary Flash Statistics 2015 Source (archive) U-DISE Secondary Thematic Maps 2015 Source (archive) WHO soil-transmitted helminthiases 2012 Source (archive) WHO STH factsheet Source (archive) WHO STH treatment report Source (archive) WHO Weekly epidemiological record, 18 December 2015 Source WHO Weekly epidemiological record, 6 March 2015 Source (archive) WHO, Helminth control in school-age children Source WHO, Helminth control in school-age children second edition Source WHO, Preventive chemotherapy: tools for improving the quality of reported data and information, 2019 Source WHO, Summary of global update on preventive chemotherapy implementation in 2015 Source WHO, Summary of global update on preventive chemotherapy implementation in 2016 Source World Schistosomiasis Risk Chart 2012 Source 1 GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013 Alix Zwane conversation June 4th 2013 Grace Hollister conversation June 19th 2013 Alix Zwane conversation August 30th 2013 Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014 GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014 GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014 GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015 GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 Starting in 2016, we deprioritized publishing notes from our conversations with Deworm the World. We continued speaking regularly with Deworm the World staff in 2016-2018. GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Kanika Bahl on October 16, 2017 2 CIFF conversation September 10th 2013. We have also had several additional conversations with CIFF about its work with Deworm the World. 3 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. 4 "More specifically, Evidence Action advocates for school-based deworming to policymakers and provides technical assistance to launch, strengthen and sustain school-based deworming programs." Evidence Action website, Deworm the World Initiative (March 2016) An example of Deworm the World supporting a program with funding: "Deworm the World is working in partnership with Thrive Networks in Vietnam on an integrated program of both deworming and water, sanitation, and hygiene (WASH) education, and this includes an RCT to explore the impact of hygiene education in combination with deworming. This program is funded primarily by Dubai Cares, with Deworm the World slated to provide unrestricted funding for certain program components." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 5. 5 "Is mass treatment justified? On cost-effectiveness grounds we believe that it clearly is, as the cost of treatment is cheaper than individual screening. The WHO states that the cost of screening is four to ten times that of the treatment itself. Because the drugs are very safe and has [sic] no side effects for the uninfected, the WHO does not recommend individual screening. The WHO instead recommends mass drug administration in areas where more than 20% of children are infected." Evidence Action website, Deworm the World Initiative (March 2016) Deworm the World focuses on school-based programs because the highest burdens for STH and schistosomiasis (the two diseases that Deworm the World targets) tend to be observed in children. For this reason, a significant decrease in the worm burden in children for these diseases translates to a significant decrease in the burden across an entire community. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 6 "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis. Deworm the World is one of the only organizations focused on STH (while there are other programs that focus on schistosomiasis), and there is significant room to scale-up [sic] STH programs. In some ways, schistosomiasis has an even larger gap to fill than STH. If Deworm the World shifted its focus to include schistosomiasis, it might widen the existing STH gap. Additionally, Deworm the World specializes in school-based deworming, which is not the ideal approach in every situation." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5. This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. 7 "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5. 8 "We work with governments around the world to eliminate the public health threat of worms through scaling up school-based mass deworming programs." Evidence Action website, Deworm the World Initiative (December 2016) For example, Deworm the World partners with a local organization in Pakistan: "The majority of Evidence Action's work in Pakistan is supported through a grant Evidence Action has provided to our Pakistan based implementing partner, Interactive Research & Development (IRD)." Deworm the World, 2020 expense report 9 "In January 2007, the [Young Global Leaders] launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs." Harvard Business School Kenya Case Study A 2010, Pg 7. 10 See this spreadsheet, sheet "[2020] Summary by country." Prior to 2020, Deworm the World supported deworming programs in Ethiopia and Vietnam: Deworm the World's support of Vietnam's deworming program ceased in early 2019. "Supported ~3M treatments over 4 rounds (2016 - 2018). Over 95% coverage in each round. Successfully exited in 2019 with government-owned revised treatment strategies." Deworm the World, Strategic refresh, 2020. See more details of Deworm the World's support to Vietnam in this blog post. As of 2019, Deworm the World no longer formally provides technical assistance to Ethiopia's deworming program. Through 2020, it continued to be part of a technical advisory group for the country's deworming program, along with the government and other donors. Deworm the World, conversation with GiveWell, April 6, 2020 (unpublished). "As of 2019 DtWI no longer has a formal technical assistance role in Ethiopia. We continued serving Ethiopia's TAG into 2020." Deworm the World, comments on a draft of this page, October 2020. 11 "The government is fully responsible for program implementation, and … these programs leverage thousands of govt personnel from health and education to be able to run." Grace Hollister, email to GiveWell, June 9, 2016 12 "Note that there are not single deworming days in all countries, nor do all countries have a mop-up day. In all cases there are mass campaigns, but the structure varies by country." Grace Hollister, email to GiveWell, June 9, 2016 13 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process. For example, Deworm the World has assisted in a number of areas in India: "Andhra Pradesh... Deworm the World’s contributions Prevalence survey... Operational support Helped government develop operational plans and budgets Coordinated cross-sectoral partners through the establishment of a State School Health Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS) Coordinated drug donation made by Feed the Children Designed a monitoring and evaluation (M&E) system Created government tableau for community awareness Trainings Conducted a master training session for program Designed training cascade for the master trainees to train the rest of the implementers Designed training materials Developed materials and campaigns for community sensitization Bihar… Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andhra Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO… Delhi… In addition to the standard contributions (see Andhra Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children. Rajasthan… DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO." @Grace Hollister Conversation June 19th 2013@, Pg 1-4. 14 We have matched our descriptions to Deworm the World's standard categorization, albeit in a slightly different order. An archived version of the link is here. 15 "As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program." @Alix Zwane Conversation June 4th 2013@, Pg 2. "Deworm the World does not yet have an agreement with the government in Pakistan to conduct the surveys, but hopes to accomplish this in the next month, and anticipates that the prevalence surveys will be conducted beginning in January or February of 2016. It is expected that a clear articulation of need will be an important factor in building a strong case to the government in favor of deworming programs, and it may be best to wait until the results of the surveys are available in the second quarter of 2016 before beginning discussions with the government on a scaled school-based program. Treatment may not begin until 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. Deworm the World's advocacy often occurs side-by-side with Deworm the World's technical assistance; once Deworm the World proves that a deworming program can be well-executed, it is easier to interest national governments in funding deworming programs. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 16 For example: "DtW has been involved in deworming programs in four different states, and is still actively involved of [sic] three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement...In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. [Note: these notes contain additional examples of Deworm the World's advocacy for other states in India] "Together, we suggested to the Federal Ministry of Health that they expand this initial work. What would it take to treat at least 75% of all at-risk school-age children in the country and to launch a truly national program? SCI helped Oumer Shafi, the committed and action-oriented Coordinator for Neglected Tropical Diseases in the Federal Ministry of Health, develop a detailed action plan. This entailed sophisticated statistical analysis to determine how many deworming sites would be required to reach at least 80% of kids at risk. Meanwhile, I worked closely with Birhan Mengistu, an up-and-coming leader seconded from the World Health Organization, and with other Ministry of Health staff. We sat for hours hunched over laptop screens to develop detailed five-year budgets, talking through row after row of spreadsheets and reviewing everything from the cost of fuel for drug transport to the needs of teachers. Together with the Federal Ministry of Health, we were able to think and act boldly. We are excited to continue to partner with SCI and are seeking other partners who also share common goals and values to rapidly scale school-based deworming in endemic countries. ...When we floated the idea of vastly increasing the scope of the originally proposed deworming rounds to be a truly national plan treating upwards of 75% of all children at-risk, Shafi didn’t flinch." Evidence Action, blog post, June 12, 2015 Other advocacy activities can include discussing: "how deworming can fit into the current policy environment and policy priorities of a government, how such a program can/should be financed, the robust evidence of impact, how a country can best take advantage of WHO drug donations, encourage program champions within government, help establish program governance structures. Once a program is established, advocacy doesn’t end – we work with govts to ensure the continuation of the above. Typically we refer to this group of activities as policy and advocacy, because there is a heavy emphasis on the former." Grace Hollister, email to GiveWell, June 9, 2016 These discussions also provide opportunities for Deworm the World to assess how well a deworming program with the government might run. If Deworm the World discovered from its advocacy discussions that there were high rates of teacher or student absenteeism, then it might conclude that a school-based deworming program may not work in the country. Deworm the World assesses risks like this through a diagnostic survey of the country’s capacity, including school attendance rates, which must be sufficiently high if a school-based deworming program is to succeed. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 17 Deworm the World builds strong working relationships with governments to try to ensure that its programs will be effective, and it will not commit to a program if it does not foresee success in that country. It can decide to abandon plans for a program before a memorandum of understanding (MoU) is signed. In one state in India, Jharkhand, Deworm the World explored a program, but did not build a strong working relationship with the government, so Deworm the World pulled out of discussions before discussing an MoU or investing much money. The discussion stage with governments is important for helping Deworm the World assess the government's position and viability as a partner. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 18 "The global STH community has changed significantly in the last couple of years, especially due to the formation of the STH Coalition. The community is now prioritizing STH (in a way similar to how LF became prioritized with the formation of the Global Alliance to Eliminate Lymphatic Filariasis, which has seen significant success). The STH community is developing plans to scale-up [sic] treatment, especially in high burden countries such as: Ethiopia Nigeria India Pakistan The Democratic Republic of the Congo Indonesia The Philippines Tanzania As part of the STH Coalition, Evidence Action is chairing a working group on school-age children. It has used some of its unrestricted funding to hire consultants to create ‘snapshots’ of each country, including obstacles, gaps, potential strategies, and financial needs. Deworm the World expects to see an increase in partnerships between the various groups in the STH community. Deworm the World hopes to leverage partnerships with existing organizations in, e.g., Nigeria, Pakistan, Ethiopia, etc., to provide catalytic support (rather than opening its own offices in those places)." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 6. 19 "We work with epidemiologists and local partners to assess worm prevalence and intensity, obtaining data to develop a targeted treatment strategy and to determine parasitological impact once programs are in place." Evidence Action website, Deworm the World Initiative (March 2016) A few examples of Deworm the World assisting with prevalence surveys include: "Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children." Grace Hollister conversation June 19th 2013, Pg 1. "DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged [sic] children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment." Grace Hollister conversation June 19th 2013, Pg 2. "In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city." Grace Hollister conversation June 19th 2013, Pg 3. "DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year." Grace Hollister conversation June 19th 2013, Pg 4. 20 For example, in 2016, Deworm the World supported prevalence surveys in Pakistan because Pakistan had not yet been "mapped" (i.e., prevalence surveys had not yet been conducted in Pakistan), so it was unknown how heavy the worm burden was in Pakistan or where deworming efforts should be focused. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Deworm the World will likely wait to expand its Vietnam activities until further mapping and impact evaluation have been completed." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pg 7. 21 Comments from Deworm the World in response to a draft of this page in October 2017. 22 "DtWI would like to do prevalence surveys after every 3 years or so. Ideally, prevalence surveys would be carried out after every third round of treatment immediately prior to the following round." GiveWell's notes from site visit to India, October 2013 "Note: this [follow-up prevalence survey] strategy is evolving; WHO recommendations are to conduct sentinel site surveys after 5-6 rounds of treatment, and we are moving in that direction. Key is how a new survey would impact the treatment strategy" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Key M&E activities in India include "post-round 3 impact measurement" prevalence surveys. DtWI NDD Year 1 M&E review July 2015, Pg 3. 23 Deworm the World told us that it works "with partners with expertise in STH parasitology and epidemiology." Grace Hollister, edits to GiveWell's review, November 7, 2016. Examples of Deworm the World working with partners on prevalence surveys: "The WHO reports that Pakistan is endemic for STH, but there is not yet sufficient evidence of prevalence and intensity to develop an evidence-based treatment strategy. Deworm the World has committed unrestricted funding to fund prevalence surveys in two large provinces, Punjab and Sindh. It is targeting these provinces because their school enrollment rates are high, the areas are fairly secure, and they contain a significant percentage of the population of Pakistan. Deworm the World plans to contract with a local organization to do these surveys, but does not yet have a signed agreement." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7. "Following three rounds of school-based deworming, Evidence Action - Deworm the World Initiative recommended a second prevalence survey to the Bihar government, in order to understand the effect of deworming in Bihar on STH infection levels. With approvals from the State Government, in January and February 2015, Evidence Action - Deworm the World Initiative, conducted an STH prevalence survey among school-age children in government primary schools in Bihar. The survey took place in 65 schools in 14 districts, covering all three agro-climatic zones in the state. The National Institute of Epidemiology – Chennai (NIE) designed the survey, and analyzed the dataset to produce epidemiological findings. Field teams hired through GfK Mode (an agency with prior experience in sample collection for STH prevalence surveys), visited the households of children in the selected schools to collect stool samples and information related to school, household, deworming, and sanitation, to better understand infection patterns and allow for sample weighting. The 2 Post Graduate Institute of Medical Education and Research – Chandigarh (PGIMER) analyzed stool samples in field laboratories, which were set up in district and block health facilities, using the WHO recommended Kato-Katz method." DtWI Bihar 2015 Prevalence Survey report, Pgs 1-2. 24 Comments from Deworm the World in response to a draft of this page in October 2017. 25 "We work closely with the Ministries of Education and Health to design a program with joint ownership, develop operational plans and budgets, coordinate logistics, and provide on-the-ground support to ensure a high quality outcome." Evidence Action website, Deworm the World Initiative (March 2016) Note that we do not feel like we have a strong understanding of Deworm the World's activities in this area; for example, we have not asked Deworm the World what it has brought to the planning, budgeting, or logistics processes that would not have otherwise been included. We do not have a strong sense from Deworm the World's website about what these activities involve (e.g., we do not know what it means for Deworm the World to have "coordinated logistics"). 26 "We help governments evaluate appropriate drug treatment strategies and dosage, support drug procurement including through global pharmaceutical donation programs, and design robust serious adverse event protocols and drug tracking systems." Evidence Action website, Deworm the World Initiative (December 2016) Grace Hollister, edits to GiveWell's review, November 7, 2016 27 "DtWI provided support to the state government in submitting the drug requisition to WHO in March 2013, as well as in shipping, custom clearances and transportation upon arrival in India." DtWI Rajasthan 2013 program report, Pg 6. Deworm the World told us that before it started conversations with the Indian government, the government was not aware that it could obtain albendazole for free from the World Health Organization. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 28 "We consistently design and support training through an efficient multi-tier cascade approach that is tailored to the local context, ensuring knowledge reaches from the national level all the way to the teachers responsible for administering deworming medication." Evidence Action website, Deworm the World Initiative (March 2016) GiveWell's understanding is that training cascades involve a series of trainings that start at high levels in the government, and proceed in a step-wise fashion down to local levels, where teachers are eventually trained. At each training, materials (such as deworming drugs and posters that notify the community about deworming day) are passed down from the staff member leading the training to the staff members attending the training, until materials eventually reach teachers. A description of a training and distribution cascade: "Transportation of tablets to all districts was managed and supported by DtWI, in coordination with SHS and DHFW to the district level dispensary, from where they were collected by the respective teachers for their school. For anganwadis, the syrups were sent to the nodal officer who arranged further transportation to the supervisors who in turn handed over to the anganwadis. Training about health education on types of STH, need for deworming, transmission of worms through open defecation and other practices and how to safely administer deworming drug was conducted through a cascade model. In the first step of the cascade, training of trainers was conducted at the state level where a total of 1040 participants attended (390 WCD + 650 education department) in 60 sessions. These trainers further trained 3032 headmasters and 3032 teachers and 10,500anganwadi workers in groups of 30 participants. The training on deworming was integrated with training for the WIFS program to effectively utilize time of participants and trainers and reduce training costs. A simplified training manual was developed that included content on deworming and WIFS into a single document. At the state level training sessions, training videos on three types of soil transmitted [sic] helminths (ascaris, trichuris and hookworm) and worm infestation cycle were also used. DtWI hired district coordinators (DCs) to provide short term support to the deworming program. They played a key role in ensuring that drugs in adequate quantities were available with the district-level dispensaries and nodal offices for further distribution. The DCs collated information on shortfall or surplus of drugs at district level and shared information with the DtWI state team, who coordinated with SHS to ensured-distribution [sic] or fresh supplies to districts facing drug deficits." DtWI Delhi 2013 program report, Pg 7. See Pgs. 12-13 of DtWI Delhi 2015 program report for a visual representation and description of a recent training and distribution cascade. 29 For example, a description of the training cascade in Kenya: "The National School-Based Deworming Programme uses a cascade implementation model that efficiently and cost-effectively delivers training materials, deworming tablets, monitoring forms, funds, trainings other programme materials and resources from the national level to schools. At the national level, the Programme trains a team of MoEST and MoH officials as master trainers, requisitions deworming tablets through the MoH, and develops treatment and implementation strategies, training materials and monitoring tools. Thereafter, an initial planning meeting is held with county and sub-county leadership. This meeting is followed by two levels of trainings on how to successfully implement the Deworming Programme: Sub-County Training and Teacher Trainings. These trainings prepare sub-county and division officials to plan subsequent programme activities within the cascade, distribution of materials, planning of deworming and community mobilization and sensitization. After these trainings and community mobilization, the critical day of implementation occurs – Deworming Day – where teachers administer deworming tablets to millions of children in over 11,000 schools across Kenya and fill in monitoring forms to capture treatment data. These forms and any unused deworming tablets are moved up through a “Reverse Cascade” as described below. The cascade model helps to manage the national scale of the NSBDP, and therefore, builds capacity for successful implementation at various levels. Additionally, the cascade brings together MoEST and MoH personnel through collaborative leadership responsibilities for the planning, implementation and monitoring of programme activities at all levels. The cascade is outlined in the infographic below." DtWI Kenya 2013-2014 program report, Pgs 4-5. "Drug distribution: As per NDD operational guidelines, and established best practice, drug distribution was integrated with the training cascade (as detailed in the training section below), whereby NDD kits were provided to health functionaries at the district level trainings for onward distribution. The kits included drugs, IEC materials, and reporting forms." DtWI Madhya Pradesh 2015 program report, Pg 12. 30 For example: "DtWI provided technical expertise across all program components, and served as the primary coordinating body among implementing agencies. DtWI facilitated drug donations from WHO, provided professional master trainers for training of trainers, developed training materials such as flipcharts, modified training booklet and reporting forms, designed and carried out independent monitoring, and developed adverse event protocols." DtWI Rajasthan 2013 program report, Pg 5. "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." DtWI Madhya Pradesh 2015 program report, Pg 11. "We hire master trainers, or train govt staff to be master trainers." Grace Hollister, email to GiveWell, June 9, 2016 31 Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16. Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15. Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15. Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17. 32 "We work with governments and communications experts to design locally appropriate awareness campaigns to communicate messages through a wide variety of channels to increase public acceptance and effectiveness of deworming programs." Evidence Action website, Deworm the World Initiative (March 2016) 33 For example: "As part of their training, school headmasters/teachers were instructed to share information on the deworming program in the morning prayer sessions at their respective schools on a daily basis from October 6, 2013 onwards. They were also advised to convene school management committee meetings to communicate about the benefits of deworming and the schedule of deworming program. School headmasters were also advised to carry out student rallies / processions (prabhat pheri) to create awareness in the communities." DtWI Rajasthan 2013 program report, Pgs 8-9. 34 "One other key strategy adopted by DtWI to spread awareness was through text (SMS) reminders over mobile phones to school teachers, headmasters, Child Development Project Officers (CDPOs) and lady supervisors as a reminder about deworming day. SMSs were also used to reinforce precautions on drug administration, such as not giving drugs on an empty stomach, but only after midday meals and not giving drugs to sick children. In all, about 80,000 text messages were sent to school teachers and headmasters three times – a total of 2,40,000 [sic] messages. These messages were sent a day before deworming day, on mop-up day and after mop-up day. About 1400 such messages were sent twice to lady supervisors and CDPOs on a day before deworming day and on mop-up day. Similarly, five rounds of around 2400 text messages were sent to block level officials to expedite coverage reporting. This was an example of ensuring last-mile communication at low cost of about 12 paisa per message (or roughly 1/5th of a cent)." DtWI Rajasthan 2013 program report, Pg 9. "Additionally, mike announcements were made at public places in blocks and district headquarters by Evidence Action for 5 days, closer to deworming day (Annexure E.4)." DtWI Bihar 2015 Program report, Pg 13. "The State Health Society Bihar and Evidence Action rolled out a media mix to generate community awareness and increase program visibility to improve coverage in the state (Annexure E.1). We supported the adaptation and contextualization of prototypes from the National Deworming Day IEC resource toolkit. At the state level, State Health Society Bihar, in coordination with the Department of Public Relation, Government of Bihar, published newspaper advertisement in four dailies one day prior to deworming and mop up [sic] day, i.e., on 20 and 25 February (Annexure E.2). Radio jingles, customized into three local dialects, were aired from 15 to 26 February on the All India Radio to maximize outreach to the community. For additional visibility of the program at the community level, State Health Society Bihar printed 513,625 posters (7 for each school, including distribution in the local community), 1068 banners for Primary Health Centers, hoardings at 38 district headquarter [sic]. All of these were adapted and contextualized by Evidence Action." DtWI Bihar 2015 Program report, Pg 12. Other community awareness activities include creating posters to display at schools or advertising the deworming day in the newspaper. "Activities designed to enhance community awareness on deworming were rolled out to improve overall program coverage. The awareness activities included newspaper advertisements a day prior to the deworming day; a 60-second radio jingle aired on 3 FM channels from April 7 to 15 by School Health Scheme, and banners displayed at schools. Evidence Action was part of the committee formed by the state government for contextualization of the radio jingle. Evidence Action extended support to the state in contextualizing IEC materials from the National Deworming Day guidelines. The Directorate of Family Welfare also independently developed and printed handbills for the distribution at anganwadis to mobilize people on deworming day. The School Health Scheme provided banners to the schools, the distribution of which was integrated in trainings for teachers. The Delhi state government also used an e-portal to disseminate key information, including dates for deworming and mop up [sic] days, benefits of deworming, and details of the launch event." DtWI Delhi 2015 program report, Pgs 11-12. 35 "We help governments design monitoring systems to measure effectiveness in achieving intended program results. We also conduct independent monitoring to validate program results, and evaluate the impact of programs in reducing worm prevalence and intensity." Evidence Action website, Deworm the World Initiative (March 2016) 36 Note that Deworm the World hires monitors for the first and third type of monitoring data collected, but that the second is collected entirely by government staff: "Coverage reporting is done by the government- we sometimes assist in the data analysis, designing reporting forms, and ensuring that the 'reverse cascade' is appropriately designed." Grace Hollister, email to GiveWell, June 9, 2016 37 "Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program." DtWI Madhya Pradesh 2015 program report, Pg 16. 38 In India, on Deworming Day and Mop-Up Day, Deworm the World commissions independent monitors who go to schools to gather data on whether principals and teachers are prepared for Deworming Day, the availability of drugs and supplementary materials, whether students are being dewormed, whether proper procedures are being followed, and more. For example, see Deworming Day monitoring data from Rajasthan in 2015: DtWI Rajasthan 2015 monitoring survey from deworming day, schools (shows which questions were asked) and DtWI Rajasthan 2015 monitoring data from deworming day, schools (shows the survey responses). Kenya: Trainings monitoring: "PMCV [Process Monitoring and Coverage Validation] officers observed 36 CHEW [Community Health Extension Worker] Forums aimed at introducing the deworming sensitization message and materials/methods as well as asigning [sic] CHEWs to schools for monitoring. A successful community health extension worker forum is one that starts on time and where all the materials were present. Overall, 63% of participants arrived before training, whereas 22% arrived 1hr after the forum had begun and 15% of participants arrived more than 1hr after the forum’s commencement. Lateness appears to be a commonality to all training sessions. Materials required for CHEW training include a powerpoint printout, CHEW checklist and Severe Adverse Event (SAE; side-effects of the drugs) protocol. In 51% of forums, ALL of the Materials Pack was distributed at the start of the forum. In 13% of forums, SOME of the Materials Pack was distributed at the start of the forum. In 36% of forums, NONE of the Materials Pack was distributed at the start of the forum." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 9. Community sensitization monitoring: "A number of parents were also interview [sic] at schools on Deworming Day regarding their knowledge of deworming and the source of that knowledge. The intention behind this exercise was to compare the information source to those interviewed prior to deworming as a measure of consistency. In Figure 4, the results of the interviews pre-deworming day are compared with those parents interviewed on deworming day. The results remain largely similar, however more parents reported getting their information from ‘other’ sources (51%) when interviewed on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 14. School preparedness monitoring: "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15. Deworming Day and Mop-Up Day monitoring: "PMCV field officers visit schools on Deworming Day to observe procedure and interview teachers/head teachers regarding deworming. The number of schools observed on Deworming Day treating for STH in Year 3 was 247. The combined population of registered children at the observed schools was 88,820 children. It is estimated that 7,485 children were directly observed being treated for STH. Seven schools treating for both STH and SCH were observed. The total registered population of children in these schools was 3,198 children and 352 children were directly observed by field officers participating in Deworming Day. A quality Deworming Day is regarded to be one where: Deworming occurs within 1 week before [sic] teacher training The school would have the correct materials (including sufficient drugs) in place before commencement Children of the appropriate ages are treated (ages 2-14 years) Non-enrolled and ECD aged children are prioritized for treatment within the schools The correct dosage of drugs is given to all children" Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16. 39 This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. For example, see DtWI Delhi 2012 coverage data by school. 40 Comments from Deworm the World in response to a draft of this page in October 2017. 41 Grace Hollister, email to GiveWell, March 6, 2016 42 "A competitive RFP [request for proposal] process is now used to identify a professional survey organization to provide independent monitors. There are requirements placed on the experience of these monitors." DtWI Monitoring Improvements 2014, Pg 1. Bihar: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state… Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively." DtWI Bihar 2015 Program report, Pg. 15. Rajasthan: "Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state... Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively." DtWI Rajasthan 2015 program report, Pg 14. Madhya Pradesh: "Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively." DtWI Madhya Pradesh 2015 program report, Pg 15. Delhi: "[Evidence Action] hired an independent research agency, Sigma Research and Consulting Private Limited [sic] that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively." DtWI Delhi 2015 program report, Pg 14. Deworm the World was also involved in Chhattisgarh, but as it was engaged relatively late in the process, it did not conduct all of its standard monitoring activities in the state: "Although we place great emphasis on understanding the extent to which the school and health systems are ready to implement deworming, the extent to which deworming processes are being followed, and the extent to which coverage has occurred as planned, in Chhattisgarh we supported only with the coverage validation activity at schools due to time constraints." DtWI Chhattisgarh 2015 coverage validation report, Pgs 2-3. 43 Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016 "Evidence Action has a permanent Monitoring, Learning, and Information Systems team. Deworm the World leverages this team for M&E." Grace Hollister, email to GiveWell, June 9, 2016 "The field officers that collect the data in the field are short term hires who come from the counties in which the program is implemented" Grace Hollister, edits to GiveWell's review, November 7, 2016 Deworm the World, comments on a draft of this review, November 2021. 44 Deworm the World, comments on a draft of this review, November 2021. 45 These totals exclude spending on Kenya's lymphatic filariasis program, which Deworm the World has supported with restricted funding from the END Fund since 2017. See cell notes in this spreadsheet, cell A6 on sheet “[2020] Summary by country” and cell A7 on sheets "[2019] Summary by country," "[2018] Summary by country," and "[2017] Summary by country." 46 For example, in Kenya and Nigeria: "Kenya’s National School-Based Deworming (NSBD) program carried out only one wave of treatment in its eighth year of deworming (2019-2020) due to the Ministry of Education’s decision to close schools as a result of COVID-19." Pg. 5, Deworm the World, Kenya PMCV report, 2020 "Due to the COVID-19 pandemic, only one round of deworming took place in 2020, rather than the two that were initially planned…in Ogun." Pg. 4, Deworm the World, Ogun State PMCV report, November 2020 "Due to the COVID-19 pandemic, only one round of deworming took place in 2020, rather than the two that were initially planned…in Oyo." Pg. 4, Deworm the World, Oyo State PMCV report, November 2020 47 For example, in Jharkhand, India: "Amid the COVID-19 pandemic the August round of the NDD was revised from a fixed day approach to an extended program. States/UTs implemented the program with a modified strategy through a community- based door-to-door approach as per the guidelines from the MoHFW. In this community- based approach, front line functionaries (Sahiyas, AWWs, and ANMs) administered albendazole to the targeted children in the community through door-to-door visits, ensuring COVID-19 precautionary measures." Pg. 1, Deworm the World, Jharkhand PM report, September 2020 48 For example, in Jharkhand, India: "Since in the August 2020 NDD round program implementation approach was revised amid the COVID-19 pandemic and physical monitoring of the program was not feasible due to requirement of maintaining social distancing, Evidence Action, adopted phone-based surveys for independent process monitoring (IPM) of the community-based National Deworming Program in the state...In this round coverage validation survey was not conducted as verification of program coverage related documents and records was not feasible through phone-based survey." Pg. 1, Deworm the World, Jharkhand PM report, September 2020 49 In 2020, CIFF and Dubai Cares provided restricted funding for Deworm the World's support of India's deworming program. Conversation with Deworm the World, January 7, 2020 (unpublished). "Kenya's National School Based Deworming program is supported by GiveWell directed/general program restricted funds and program restricted grants (The END Fund and Epic)." Deworm the World, 2020 expense report "As of September 2019, Pakistan's deworming program is supported by GiveWell directed/ general restricted funds and program restricted funds from Dubai Cares." Deworm the World, 2020 expense report 50 Note that these figures exclude Deworm the World's global (non-country-specific) spending and its indirect costs. The proportion of Deworm the World's spending that has gone to its newest program, in Pakistan, has increased in the last three years from just 1% in 2017. Prior to 2020, Deworm the World supported deworming programs in Ethiopia and Vietnam: Deworm the World's support of Vietnam's deworming program ceased in early 2019. "Supported ~3M treatments over 4 rounds (2016 - 2018). Over 95% coverage in each round. Successfully exited in 2019 with government-owned revised treatment strategies." Deworm the World, Strategic refresh, 2020. See more details of Deworm the World's support to Vietnam in this blog post. As of 2019, Deworm the World no longer formally provides technical assistance to Ethiopia's deworming program. Through 2020, it continued to be part of a technical advisory group for the country's deworming program, along with the government and other donors. Deworm the World, conversation with GiveWell, April 6, 2020 (unpublished). "As of 2019 DtWI no longer has a formal technical assistance role in Ethiopia. We continued serving Ethiopia's TAG into 2020." Deworm the World, comments on a draft of this page, October 2020. 51 See this spreadsheet, sheet "[2019] Summary by country," column F. 52 "Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success. "Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action." Evidence Action launch announcement 2013 53 "Evidence Action Beta investigates what interventions might be suitable for massive scale up [sic] – finding the next thing that works." Evidence Action website, Evidence Action Beta (October 2015) 54 See this spreadsheet, sheet "[2019] Summary by country," column "% of total spending by country." 55 India: "…absent from most of the country, [schistosomiasis] risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1. "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3. See section on worm prevalence and intensity in India and Kenya below. 56 WHO, Helminth control in school-age children, Pg 74 "Based on the findings of the prevalence survey and WHO guidelines, Evidence Action recommends an annual school based deworming program for school-age children in the state. [...] Given the pre-existing deworming treatments described above, this prevalence survey cannot be considered a baseline survey of an untreated population, but is rather a survey to assess STH infection rates in a treated population, to determine an optimal treatment strategy." DtWI Madhya Pradesh 2015 program report, Pg 38. "Our recommendation is explained in the prevalence survey report. The prevalence and intensity rates from the survey are not "baseline" data, given that there has been relatively regular administration of albendazole in MP through the BSM program that treated PSAC since 2005, and the LF program which provided community-wide treatment of 11 districts of MP (the number of endemic districts had fallen to 8 by 2014). As a result, these deworming efforts have likely had an impact on STH prevalence and MP could not be considered an untreated baseline population. We therefore did not apply the WHO guidelines for baseline STH prevalence. "Annexure 10 of the WHO guidelines suggest continuing annual treatment for populations which have received deworming for several years, and prevalence is still greater than 10%. In addition, the high rate of open defecation in the state, and the planned ending of the LF program in MP, increased the risk of infection and a potential resurgence in prevalence." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Grace Hollister, email to GiveWell, June 9, 2016 57 DtWI Madhya Pradesh 2015 program report, Pg 7. Deworm the World, Chhattisgarh prevalence survey report, August 2016, Pgs 4-5. DtWI Bihar 2011 prevalence survey report DtWI Rajasthan 2013 prevalence survey report Deworm the World 2015 Uttar Pradesh prevalence survey report, Pg 9. Deworm the World notes that in Madhya Pradesh, Chhattisgarh, and Uttar Pradesh, prior treatment was either through lymphatic filariasis programs or non-standardized deworming, and that in Bihar, a baseline survey was done before school-based deworming took place. Grace Hollister, comments on a draft of this review, September 6, 2018. 58 Grace Hollister, comments on a draft of this review, September 6, 2018. 59 Deworm the World noted that the KEMRI surveys in Kenya are designed for impact assessment, while the surveys in India are designed for mapping. This means that the schools selected in KEMRI's surveys are all from places where treatment is required or taking place. In mapping surveys, schools are selected to be representative of a larger geographic area (e.g., they may be selected in part based on which agro-climatic region they are in). Additionally, Deworm the World noted that the surveys in India are looking at a much larger population than the Miguel and Kremer 2004 and Croke 2014 studies examined and there may be substantial variation in prevalence across a given area. Grace Hollister, conversation with GiveWell, June 13, 2016 and Grace Hollister, edits to GiveWell's review, November 7, 2016 60 For example, India has such a program: The National Vector Borne Disease Control Programme LF treatment coverage 2015 In Kenya, the LF program is housed within the country’s neglected tropical disease (NTD) unit, which has asked Deworm the World if it might support its process monitoring and coverage validation (PMCV) operations for LF. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Deworm the World has told us that in both countries (Kenya and India), LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Additionally, our understanding from a number of conversations with Deworm the World and others is that many countries are beginning to shut down their LF programs as they eliminate the disease. 61 "DEC [Diethylcarbamazine] + Albendazole in selected distt & DEC in other distt" LF treatment drugs 2012. "Albendazole, the same drug used to treat STH, is usually used to treat LF." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 Assam 2010 guidelines for deworming recommends 400mg of albendazole for children being dewormed, and Global Alliance to Eliminate Lymphatic Filariasis - Prevention also recommends 400mg doses. 62 Or from twice-per-year to thrice-per-year. Note that community-based treatment, such as is typically used for LF, involves enlisting several people to travel from house to house to administer treatment, making it much more time-consuming and costly than school-based programs. Because its goal is to treat every person in a community, multiple trips to a single area may be required to ensure total coverage (e.g., if a household member is not at home during the first visit). GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 Deworm the World also claims that it generally knows where there is overlap between areas that are endemic for STH or schistosomiasis and areas that are endemic for LF. For example, in Kenya, only the coastal area is LF-endemic, but worm infections are more widespread, so LF treatment efforts involve a smaller number of subcounties. Community-based treatment programs might be cost-effective in places endemic with many NTDs but might not be in places only endemic for STH or schistosomiasis. Deworm the World has told us that many places do not have much overlap between different NTDs so school-based deworming programs can provide a cost-effective alternative to community-based treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015 63 More detail on this page. 64 See this spreadsheet, sheet "[2019] Summary by country," column "% of total spending by country." 65 Nigeria conducts biannual treatment. While Pakistan does not, Deworm the World supported two rounds of treatment in 2019 in order to reach more children. "In Nigeria there are two rounds of treatment based on LGA prevalence and intensity. Pakistan on the other hand does not have biannual treatment; there were two waves of treatment in 2019 in order to try and achieve higher coverage as a new program." Deworm the World, comments on a draft of this page, October 2020. 66 See this spreadsheet, sheets "Monitoring methods and results" and "Results (Nigeria and Pakistan 2019 CV)." 67 See the most recent version of the model here. For our estimate of the degree to which Deworm the World's monitoring and evaluation data reflect the true impact of its programs, see section "Quality of monitoring and evaluation," line "Misappropriation without monitoring results." 68 We have gained this understanding through many conversations and across multiple years of reviewing Deworm the World. See our review process. Deworm the World notes, "In Kenya, some monitoring is conducted by Evidence Action staff." Deworm the World, comments on a draft of this page, October 2020. Note that the methods Deworm the World uses in each country, for each round of MDA, might not be the same as those outlined here; Deworm the World adjusts its monitoring based on past learning or new contextual constraints. See this spreadsheet for descriptions of the monitoring processes we have seen in Deworm the World's monitoring reports. 69 Deworm the World, comments on a draft of this page, October 2020. 70 See this spreadsheet, sheet "Monitoring methods and results." Note: We have not seen this type of monitoring from Jharkhand for 2016, and are uncertain whether Deworm the World did monitoring on Deworming Day or just coverage validation (described below). Deworm the World, Jharkhand Independent Monitoring Report 2016. In 2017, Deworm the World conducted both observations of the deworming process and coverage validation in Jharkhand. Deworm the World, Jharkhand NDD PMCV Report, February 2017 and Deworm the World, Jharkhand PMCV report, August 2017. Note that if the chosen school is closed on the day of the deworming, they are generally instructed to go to an assigned backup school instead. India: For example, in Delhi, monitors are instructed in the survey instrument, "Is the school open? [If not,] Did you go to the buffer school?" DtWI Delhi 2015 monitoring survey from deworming day, schools, Pg 3. According to Deworm the World, Madhya Pradesh NDD PMCV Report, February 2017, "If a school or anganwadi was found to be closed or non-traceable during process monitoring, it was replaced by another nearby site," Pg 5. It is our impression that most schools were open when monitors visited. We spot-checked several states' data in 2016: DtWI Bihar 2015 monitoring data from deworming day, schools, Column L shows that all schools monitors visited were open. DtWI Rajasthan 2015 monitoring data from deworming day, schools, Column M shows that 124 of the 125 schools monitors visited were open. Kenya: In Kenya, it is rare that monitors find schools to be closed or missing because Deworm the World has operated in Kenya for several years. However, monitors will try to go to another school if their school is closed. Paul Byatta, conversation with GiveWell, September 20, 2016 "When the monitor is unable to monitor the assigned school (because of closure, etc) we ask the monitor to i) to communicate the same with his supervisor and, ii) to visit the nearest school if the nearest school was not part of the randomly picked school." Paul Byatta, attachments to email to GiveWell, September 23, 2016 Vietnam: The backup schools are also randomly selected. Paul Monaghan, conversation with GiveWell, September 8, 2016 Nigeria: "Initially, replacement schools were also randomized and shared alongside sampled schools. However the issue of timing was an issue, as monitors who needed to replace schools were often too far away from the randomly selected schools and had to travel too far to get to a replacement school. Because deworming mostly happens in the morning, by the time monitors arrived at the replacement school, they may not be able to observe/conduct most of the deworming activities. This sometimes meant we were unable to meet our monitoring targets or monitors arrive way after deworming has started and we only get partial information from the exercise. Subsequently, we started to select replacement schools based on proximity to the sampled school that couldn't be monitored to reduce travel time. We would usually do this with the help of the LGA NTD team as they are more informed about schools within a particular location. In 2019 we decided to make it more systematic and to include randomization into the process by identifying the three schools closest to the initially sampled school that needs to be replaced, and then randomly select one of them." Brett Sedgewick, comments on a draft of this review, October 31, 2019. 71 See, for example, DtWI Delhi 2015 monitoring survey from deworming day, schools, Deworm the World, Kenya Year 3, DD - Main instrument, and Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day Note that it is not actually dangerous to give the deworming pills to sick children; Deworm the World simply prefers not to deworm sick children to avoid causing people to associate the deworming pills with illness: "Deworming pills should not be distributed to sick children...This is not because deworming pills could harm sick children. It is because DtWI wants to avoid people (and potentially the media) blaming the deworming pill for a child’s illness." DtWI 2013 GiveWell site visit 72 For example, in Kenya: "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment. Deworming was reported to occur inside classes in 47% of observed schools and outside in 53% of schools. Deworming was considered to be ‘systematic’ in 98% of schools. The correct dosage for albendazole is one tablet per child and the correct age is 2-14 years. These procedures were observed to be followed correctly by 86% of teachers observed by field officers. Coverage: Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools. Teachers were reported to correctly observe children swallowing PZQ in 99% of schools and ALB in 96% of schools. Observing children swallowing is most important when treating for SCH as the tablet does not taste pleasant and there are high chances of children spitting if not observed." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, p. 17 For example, in India: When monitors visit schools on Deworming Day, they make observations and ask questions related to the process of deworming, to check whether or not it is being implemented correctly. For example, monitors observe whether deworming is in progress and whether teachers are using proper recording protocol, with results given for: "Deworming activity is taking place in the class/Anganwadi" "Teachers/ Anganwadi worker following the protocol of putting single tick (deworming day) or double tick (mop-up day) on each child's name/roll no. in the attendance register after giving them the deworming tablet" "Practice followed by teacher, if the ticking/double ticking protocol did not followed" Deworm the World, Telangana 2016 IMCV report, p. 19, Table 3 73 Deworm the World, comments on a draft of this page, October 2020. For example, in Pakistan: "Community sensitization prior to conducting the Deworming Day is an evidenced key ingredient for MDA success. On Deworming Day, monitors held interviews with 114 and 57 parents of enrolled and non-enrolled children. Key to this interview was to gauge awareness of the upcoming MDA, as well as their sources of information for the MDA. At the end of the interviews, monitors also sought to determine what proportion of parents would be sending their children for deworming as a proxy for the effectiveness of the sensitization efforts." Deworm the World, Islamabad Capital Territory process monitoring and coverage validation report, February 2019, p. 17 74 See this spreadsheet for more details. 75 "India only conducts CV at schools, while Kenya, Nigeria, and Pakistan all utilize the same combined household and school-based CV methodology now." Brett Sedgewick, comments on a draft of this review, October 31, 2019. See this spreadsheet for more details. 76 For example, see this spreadsheet, sheets "Methods (Nigeria 2019 CV)" and "Methods (Pakistan 2019 CV)." 77 For example, in Kenya, it is our understanding that monitors interview three randomly selected children from three separate classes, for a total of nine students, at each school they visit. See Deworm the World, Kenya Year 3, Post DD - Coverage instrument. On pages 1-3 there are three spaces for randomly selected classes. The instructions read: "Thank the Head Teacher or designate and request to speak to pupils of the randomized class...CHOOSE CHILD 5, 10 AND 15TH ON FORM E. IF LESS THAN 15 CHILDREN, SELECT THE LAST CHILD. ENSURE TO INTERVIEW AT LEAST THREE CHILDREN. ASK THE TEACHER FOR PERMISSION TO SPEAK TO THEM ONE AT A TIME Ask questions in multiple ways for interviews with students, use local language if possible. Don’t rush responses. Try to make them feel at ease. Speak to one child at a time at a place where they are comfortable.... [interview questions]...END, MOVE TO THE NEXT SAMPLED CLASS" 78 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools "This is asked in all geographies. In addition, children are asked about unprogrammed deworming during CV in [Kenya], [Nigeria], [Pakistan]." Brett Sedgewick, comments on a draft of this review, October 31, 2019. 79 For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools. 80 "Children were interviewed only from those schools which conducted deworming on either day i.e. NDD or mop-up day. Three classes were selected randomly from each school by using random number tables. Further, one student was selected from each class randomly considering their presence either on the NDD or mop-up day (attendance is verified through attendance register) using random table." Deworm the World, email to GiveWell, May 30, 2018 (unpublished). 81 Grace Hollister, comments on a draft of this review, August 20, 2018. 82 Grace Hollister, comments on a draft of this review, August 20, 2018. Deworm the World, comments on a draft of this page, October 2020. 83 See this spreadsheet, sheet "Monitoring methods and results," column "% of schools where monitors observed Deworming Day activities." In 2019, monitors observed the deworming process in a lower proportion of schools in the first and second rounds of deworming in Ogun State, Nigeria (83% and 70%, respectively), the second round in Cross River State, Nigeria (77%), and the second round in Islamabad Capital Territory, Pakistan (63%). 84 See this spreadsheet, sheet "Monitoring methods and results," column "% of schools where monitors observed Deworming Day activities." 85 See this spreadsheet for details. "Surveyors in India are assigned to multiple schools on NDD and Mop-Up Day, so it is often the case that a school may have already completed deworming by the time the surveyor finishes one site and moves to the next." Deworm the World, comments on a draft of this review, September 12, 2018. 86 For example, in Kenya: "Does the monitoring team have an estimate of how many non-enrolled children the deworming program reaches? We get this from treatment forms that schools submit back via “the reverse cascade”. For PMCV, we monitor that teachers are aware they should be treating non-enrolled, and they are aware of age categories targeted for non-enrolled. When we do data audit on treatment forms, we also check that data for on non-enrolled is entered accurately." Paul Byatta, attachments to email to GiveWell, September 23, 2016, p. 3 87 In comments on a draft of this review on September 12, 2018, Deworm the World noted: "We have begun to address this issue through methodology changes in our coverage validation surveys. In Kenya we have also begun to use the enrollment rates obtained from the Ministry of Education and the number enrolled in schools to come up with the estimate for the non-enrolled population within the county." 88 For examples, see this spreadsheet, sheets "Methods (Nigeria 2019 CV)" and "Methods (Pakistan 2019 CV)." 89 See cell notes in this spreadsheet, sheet "Results (Nigeria and Pakistan 2019 CV)," column "Enrollment status." In these surveys, coverage of non-enrolled children ranged from 12%-65%, compared to 65%-95% of enrolled children. See this spreadsheet, sheet "Results (Nigeria and Pakistan 2019 CV)," columns "Enrollment status." 90 For example, Table 2 of Deworm the World, Ogun State process monitoring and coverage validation report, June 2019 shows that, of 1,664 children targeted for interview through community surveys, only 217 were ultimately interviewed. Of this discrepancy, the report states: "The true net enrollment rate may have been much higher than the quoted 54% used to generate the sample for community interviews. This could have led to a smaller proportion of children being found in the community than anticipated at the planning stage. Consequently the small number of non-enrolled children interviewed makes it difficult to make any deductions about this population type." p. 11 91 See this spreadsheet for more information. 92 "Children were interviewed only from those schools which conducted deworming on either day i.e. NDD or mop-up day. Three classes were selected randomly from each school by using random number tables. Further, one student was selected from each class randomly considering their presence either on the NDD or mop-up day (attendance is verified through attendance register) using random table." Deworm the World, email to GiveWell, May 30, 2018. "Even though we only interview children who were present on deworming day in India, we still collect data on (and adjust for) the percentage of children who were absent on DD and the percentage of schools that did not implement deworming on DD. It is correct that in Jharkhand Aug. ‘17 it was reported that among children who were present on DD, 100% received deworming tablets. However, this statistic alone is not accounting for the fact that only 86% of children were in attendance on DD or MUD. We still collect this data on attendance; it’s just presented separately than the statistic on the percentage of children present who received a tablet. In Kenya, our coverage validation surveys are structured differently, such that we are interviewing children despite their attendance/enrollment status." Deworm the World, comments on a draft of this review, September 12, 2018. 93 Kenya: "Show student 3 tablets, ask: Which one of these three tablets did you take? Circle the indicated tablet." Deworm the World, Kenya Year 3, Post DD - Coverage instrument India: In India, monitors show children the deworming tablet and ask if it is the tablet that they took, which we do not believe is a rigorous method of verifying student answers. For example, see pages 20-23 of DtWI Bihar 2015 monitoring survey for coverage validation, schools. Nigeria: In Nigeria, independent monitors have pictures of the pills that they show children so that children can visually identify the pill and respond accordingly if they took a deworming pill. Deworm the World, responses to GiveWell questions, May 16, 2019. 94 "We train monitors to interview children in private by taking them aside for interviews. In practice this is often not in a separate room, but away from the class. Private rooms are difficult to secure and parents do not trust a stranger to be alone with their child. In most cases, an adult is present during the interview because the children are minors." Deworm the World, responses to GiveWell questions, May 16, 2019. 95 See this spreadsheet, sheets "Results (summary)" and "Monitoring methods and results." 96 Deworm the World supports program costs in Cross River, Nigeria and expects to support program costs in future Nigeria states. It expects to support program costs in Pakistan, and possibly Indonesia. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 97 We attempt to learn if there was a possibility that the program would have been funded by other donors. However, given that there is a global funding gap for deworming treatments (more), we suspect that in most cases where Deworm the World pays for a new deworming program, it is increasing the number of children dewormed. 98 Deworm the World notes that: "albendazole and mebendazole are quite hardy (neither requires special storage conditions) and have a long shelf life." Grace Hollister, email to GiveWell, June 9, 2016 99 For example, drug quality was tested in each program that Deworm the World supported in India in 2013-2014. Bihar 2014: "The drugs were safely stored in State Health Depot in Patna until November 2013. At that point, the districts began to pick up their share of the drugs from the depot. In October, Deworm the World had coordinated for lab testing of the stored drugs via ASCHO NIBULA INDUSTRIES LTD, an independent lab which approved the quality of the drugs." DtWI Bihar 2014 program report, Pg 13. "The quality of drug storage was satisfactory in most schools that were monitored. 98.7% of them stored the drugs in a clean location, 91.1% of them were stored away from direct sunlight and 97.4% of them were stored away from the direct reach of children." DtWI Bihar 2014 program report annex 1, Pg 7. Delhi 2013: "Once the procured syrups and donated tablets were delivered to Directorate of Health Services central storage room, they were tested in a government-accredited laboratory to ensure drug quality prior to administration." DtWI Delhi 2013 program report, Pg 6. From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Delhi 2013 program report, Pg 24. A table of results is also presented on the same page. Rajasthan 2013: "To instill confidence among the stakeholders that the drugs were of good quality, Deworm the World arranged for sample testing of the donated drugs by two independent labs7. Similarly, Rajasthan Medical Services Corporation sample tested the syrups they procured." DtWI Rajasthan 2013 program report, Pg 6. From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Rajasthan 2013 program report, Pg 40. A table of results is also presented on the following page. 100 "Now, state governments have responsibility for testing drugs. We have recently raised some concerns about the need to standardize the testing that is taking place, and are currently working with the MoHFW to build out the NDD operational guidelines with more detailed guidance on this point." Grace Hollister, edits to GiveWell's review, November 7, 2016 101 Of classes where monitors observed deworming activities in India, there were low numbers of adverse events (see table below). We don't know what portion of the adverse events may be caused by incorrect dosages. Bihar 2015 Rajasthan 2015 Delhi 2015 Madhya Pradesh 2015 Sample of question asked Classes where there were adverse events (monitors' observations) 5% (vomiting), 0% (diarrhea) 2% (vomiting), 0% (diarrhea) 5% (vomiting), 0.8% (diarrhea) 6% (vomiting), 0% (diarrhea) "Did you see any child with adverse effects (nausea, vomiting, stomachache, etc.) after taking the medicine?" [Table last updated in 2016] Sources for the information in the table: Bihar 2015: Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 4 Rajasthan 2015: Deworming Day and Mop-up Day ( N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 4 Delhi 2015: Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 6 Madhya Pradesh 2015: Deworming Day and Mop-up Day (N = 250 schools), DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 4 Sample question: DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 14 102 "Note that National Deworming Day operational guidelines state that only tablets should be used. Albendazole dosage is the same for all children aged 2 and above; it is only children aged 1-2 that require a different (half) dose." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 From Deworm the World's report on the Delhi 2013 program: "A key discussion from round one was the administration of drug to younger children in the pre-school age for whom chewing a tablet was seen as a difficulty. Hence keeping in mind the scale of the program, with an objective of making it a safe public health initiative the committee decided that deworming would be implemented across all districts following the World Health Organization (WHO) sanction in administering albendazole 400 mg tablets and the GOI guidelines under the WIFS program for administering the syrup vial. They also decided to opt for Albendazole 200mg dose suspension for 2-6 year children as it is a single dose for this age group which leads to lesser error in administration, while Albendazole 400mg tablets for older children." DtWI Delhi 2013 program report, Pg 6. The WHO factsheet on STH cites only a single recommended dosage (depending on which drug is used): "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet 103 Bihar 2015: In 8.8% of trained schools and 5.4% of untrained schools, monitors observed children given less than one tablet; in 2.6% of trained schools and 4.7% of untrained schools, monitors observed children given more than one tablet. DtWI Bihar 2015 independent monitoring tables, Pg 9 (Table 10) Rajasthan 2015: In 6.0% of schools, monitors observed children given less than one tablet; in 1.6% of schools, monitors observed children given more than one tablet. DtWI Rajasthan 2015 independent monitoring tables, Pg 2 (Table 2) Madhya Pradesh 2015: In 9.3% of schools, monitors observed children given less than one tablet; in 2.7% of schools, monitors observed children given more than one tablet. DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2 (Table 2) Delhi 2015: In 2.5% of schools and 6.0% of anganwadis, monitors observed children given less than the prescribed dose of albendazole; in 0.8% of schools and 10.6% of anganwadis, monitors observed children given more than the prescribed dose of albendazole. DtWI Delhi 2015 independent monitoring tables, Pgs 2, 16 (Tables S1 and A1) Bihar’s 2014: 8% of schools observed gave children less than one tablet and 2% gave more than one tablet. DtWI Bihar 2014 program report annex 2, Pgs 2 and 4. "In 3.1% of schools and anganwadis, monitors observed children being given more than one tablet/syrup bottle. As per protocol, the children should not have been given more than one tablet. Therefore in cases such as these, the monitors were trained to intervene and prevent the administration of an additional dose." (The percentage of schools using less than one pill/bottle per child was not reported.) DtWI Rajasthan 2013 program report, Pg 38. Monitors in Delhi in 2013 did not report on either of these observations. See DtWI Delhi 2013 program report, Pgs 21-25. 104 "The majority of direct program costs in India are government funded. There is a cap within the National Health Mission budgets on M&E – M&E cannot exceed 10% of the overall budget. So there are specific areas, such as program monitoring, that would likely not have investment at the level we are able to provide." Grace Hollister, edits to GiveWell's review, November 7, 2016 "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2. "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and Deworm the World's tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. Deworm the World called a random sample of 8,000 schools. The prevalence survey would not have happened without Deworm the World's support." DtWI 2013 GiveWell government interviews, Pg 5. 105 "RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3. 106 "[The Nodal Headmaster said] that most aspects of the program are excellent, but he had 2 suggestions: Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so) Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6. 107 According to Deworm the World, Cross River State, Nigeria, Year 2 Round 1 PMCV report, June 2017, Figure 14, children were observed fainting in approximately 70% of schools. Deworm the World attributes fainting to taking praziquantel on an empty stomach: "We had an instance where as high as 5 pupils from one school were reported to have fainted after taking PZQ in Ogoja LGA. A team of health workers and the LGA coordinator rushed to the school and discovered that the pupils fainted because they were dewormed too late (1:00 pm), six hours after their meal in the morning 8:00am) before coming to school. Henceforth, we advised the teachers and LGA teams not to deworm children with PZQ later than 11 am. This is because almost all children eat at home before coming to school and anything after long break, they won't have enough food left in their bellies." Deworm the World, email to GiveWell, May 30, 2018. 108 "In Tanzania matters came to a head in places around Morogoro in 2008. Distribution in schools of tablets for schistosomiasis and soil-transmitted helminths provoked riots, which had to be contained by armed police. It became a significant national incident, and one of the consequences has been the delay in Tanzania adopting a fully integrated NTD programme, and the scaling back some existing drug distributions." Allen and Parker 2011, pg. 109. "From these reports a number of problems with the MDA were raised which included fear of side effects from the tablets, particularly following the mass hysteria and death in Blantyre and Rumphi respectively and may explain some of the geographic heterogeneity seen. Furthermore most districts reported that MDA occurred after standard 8 students had finished exams and left school, and due to having inadequate resources for drug distribution...The side-effects incident in Blantyre and death in Rumphi had a large effect on districts and with many district reports stating that after the incidence many families refused to participate." SCI Malawi coverage survey 2012 Pgs 5, 21. 109 "THERE was confusion on Wednesday in some public primary and secondary schools in Ogun State, over the administration of anti-worm tablets. Nigerian Tribune gathered that some students reportedly collapsed in the cause of administering the tablets on them. This resulted into rumour that spread like wildfire across the length and breadth of the state, as parents stormed various school to withdraw their wards. When the Nigerian Tribune visited Egba High School, Asero and Asero High School both in Abeokuta South Local Government Area of the state, some parents were sighted at the school gate, who had come to confirm the incident and probably withdraw their wards. There was calmness in both schools as students in the Senior Secondary Classes were said to be preparing for their examinations. Meanwhile, the Ogun State Government through the State Commissioner for Health, Dr Babatunde Ipaye, has denied any case as a result of the anti-worm drug. Ipaye in a statement made available to the Nigerian Tribune in Abeokuta, said that no pupil or student to the best of his knowledge had reacted to the drug in the state. He explained that the exercise was done by his Ministry in collaboration with Evidence Action." Nigerian Tribune, "Panic in Ogun schools over deworm exercise," December 2017 110 "There were statewide rumors about children reacting badly to the medicines and other rumors about children dying after administration of medicine; this may have had an impact in that 90 (94%) of the 95 monitored schools administering the medicines on deworming day... On the deworming day, rumors broke out state-wide, originating from Saki LGA about children reacting to PZQ, unfortunately, in Surulere LGA, 2 out of 3 children from the same family who were all dewormed died on the deworming day, the death of these kids further worsened the situation as the news spread to neighboring LGAs. The LDA NTD coordinator, PHC coordinator, teachers from the children’s school and the LIE were promptly arrested by the police. Investigation surrounding the likely cause of their death was conducted at a teaching hospital in a nearby LGA, and it was confirmed that the kids died of food poisoning resulting from consumption of fruits sprayed with herbicides. This has been established as the cause of their death and the case has since been closed. Unfortunately, the damage to the SBD which coincided with the period had been done." Deworm the World, Oyo State PM report, February 2018, Pg 1. 111 "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 Deworm the World noted that the costs in Kenya are high, partly due to higher quality M&E: "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5 "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 In the second round of treatment in Kenya, prevalence surveys were 11% of the total costs and other M&E was 6%. Per treatment, these costs are $0.059 ($379,523/6,405,462) and $0.035 ($222,750/6,405,462) respectively DtWI Kenya 2013-2014 cost per treatment data "The cost per treatment in Kenya is higher than in India. This is in part because in Kenya Deworm the World is treating for schistosomiasis in addition to STH, and schistosomiasis treatments incur greater costs, including a higher drug cost and more extensive mapping. Other factors include the cost of additional supplies, higher operational costs, teacher allowances, and working on a smaller scale, which is less cost-effective. There are 6 million children in need of treatment in Kenya and the WHO estimates 241 million children at risk in India." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015. Deworm the World has also noted that treating schistosomiasis in Kenya increases the costs: "The extra costs of treating schistosomiasis in addition to STH. Schistosomiasis drugs tend to be more expensive than STH drugs and, in Kenya, schistosomiasis treatment sites are sometimes much more remote. The treatment strategy for schistosomiasis also differs from STH because schistosomiasis is more localized (e.g., it is not necessarily ideal to treat an entire sub-county). This also makes mapping schistosomiasis more expensive." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 3. 112 Deworm the World estimates that the Cross River program in 2016 cost a total of $0.69 per child and the Kenya program cost $0.45 per child in 2015-2016. See this spreadsheet, sheet "Cross River, Nigeria 2016," cell C11, and Deworm the World External Kenya 2016 Costing Model, sheet "Costing model," cell E9. Note that we make several adjustments to these figures to get our estimate of total cost per treatment (see sheet "Summary" in this spreadsheet). 113 We estimated the staff time costs based on Deworm the World's estimate of similar costs from the same states in 2012. It had imputed those costs based on estimates of government employee salaries, for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50). 114 "We include all partners’ expenditures in determining costs for the the deworming programs, but we do not consider spending that would be incurred even without deworming taking place. [...] Teachers’ and principals’ general salaries are not included because they do not spend additional time on deworming beyond what they are already compensated for by the government for regular classroom teaching." Evidence Action, blog post, July 5, 2016 115 Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Deworm the World thought that the costs of these activities were quite low (less than 5%). We have assumed 3.5% in our analysis. 116 See our discussion of the rationale and limitations of this estimate here. 117 For a discussion of why we consider funding a charity's work up to three years in the future, see this blog post. 118 Some of our top charities have a policy of holding funding reserves. In our room for more funding analyses, we typically include reserved funding as funding available to support program activities. We do this both to ensure consistency across top charities (as not all top charities hold reserves) and to understand the true effect of granting additional funding (i.e. whether additional funding would support undertaking additional program activities versus building or maintaining reserves). 119 GiveWell maintains a list of all charities that meet our criteria, along with a recommendation to donate to our Maximum Impact Fund. [Note added October 2022: We updated the name of this fund in September 2022; more information here.] Some donors give based on our top charity list but do not follow our donation recommendation. In our projections of future funding, we typically count only one year of funding that an organization receives as a result of being on our list of top charities in order to retain the flexibility to change our recommendations in future years. 120 We update a top charity's room for more funding analysis more frequently if we grant funding to it more frequently. 121 We update a top charity's room for more funding analysis more frequently if we have reason to believe that its funding and budgets have changed substantially. 122 For a list of grants we have made from our Maximum Impact Fund, see this page, section "Past recipients of the Maximum Impact Fund." [Note added October 2022: We updated the name of this fund in September 2022; more information here.] 123 Open Philanthropy, a philanthropic organization with which we work closely, is the largest single funder of our top charities. The vast majority of Open Philanthropy's current giving comes from Good Ventures. We make recommendations to Open Philanthropy each year for how much funding to provide to our top charities and how to allocate that funding among them. An example of these recommendations from November 2020 can be found on this page. 124 This includes donations made to charities on our checkout page, donations made directly to the organizations, and donations through other third-party organizations that share GiveWell’s recommendations (e.g., One for the World).
- Treatment for lymphatic filariasis
- Are deworming pills delivered to and ingested by recipients?
- How does Deworm the World affect program outcomes?
- Are there any negative or offsetting impacts?
- What do you get for your dollar?
- Is there room for more funding?
- Evidence Action as an organization
- Sources
Our review process
Our review process has consisted of:
- Extensive conversations with Deworm the World Director Grace Hollister and other Deworm the World and Evidence Action staff since 2012.1
- Reviewing documents Deworm the World sent in response to our queries.
- Site visits:
- In November 2012, we visited Deworm the World's office in Nairobi, Kenya and met its staff there. (Notes from our visit)
- In October 2013, we visited Deworm the World's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with Deworm the World. (Notes from our visit)
- In 2015, we retained two journalists to visit areas served by Deworm the World in Kenya. We published their report on our blog.
- Conversations with the Children's Investment Fund Foundation (CIFF), a funder of Deworm the World.2
All content on Deworm the World, including past reviews, updates, blog posts, and conversation notes, is available here. We have also published a page with additional, detailed information on the program to supplement some of the sections below.
What do they do?
The Deworm the World Initiative is a program led by Evidence Action, an organization that focuses on scaling up interventions that it believes are cost-effective and evidence-based. Deworm the World advocates for and supports the implementation of government-run deworming programs for preschool- and school-age children.3 The support that Deworm the World provides is of two types: 1) offering technical assistance to governments implementing deworming, and 2) funding components of deworming programs.4
The deworming programs that Deworm the World supports are focused on executing school-based mass drug administrations (MDAs), in which the aim is to treat the entire population of children within a geographic area by distributing deworming pills.5 Deworm the World focuses on MDAs that treat children infected with soil-transmitted helminthiasis (STH).6 Where needed, these programs also include treatment for schistosomiasis.7 In the countries it works in, Deworm the World works primarily with government staff to implement deworming programs; as it has expanded to new countries, it has started to also collaborate with non-governmental partners to support governments.8
Deworm the World was founded in 2007.9 Since then, it has supported deworming programs in India, Kenya, Ethiopia, Nigeria, Vietnam, and Pakistan. As of 2020, it was active in India, Kenya, Nigeria, and Pakistan.10 On a separate page with additional information about Deworm the World, we discuss more details of its work by country.
Below, we discuss:
- Deworm the World's role in government-led deworming programs
- A breakdown of Deworm the World’s spending
- Deworm the World's relationship to Evidence Action
Deworm the World’s role in government-led deworming programs
The deworming programs that Deworm the World supports are implemented by the governments with which it works.11 Below, we expand on Deworm the World's role in the programs it supports. Note that we use "Deworming Day" to mean the day on which the MDA takes place. Similarly, we use "Mop-Up Day" to refer to the day that occurs several days after Deworming Day and is when students who were absent or sick on Deworming Day are given their deworming pills (although note that not all countries have just one Deworming Day or include a Mop-Up Day in their program).12
The assistance that Deworm the World provides in each country varies based on what each partnering government needs.13 Historically, Deworm the World's role has included the following:14
- Advocacy. Deworm the World actively encourages national and large sub-national governments to implement mass school-based deworming programs.15 Our impression is that Deworm the World's advocacy consists of meeting with health and education officials in a government to discuss the benefits of deworming and how a deworming program might be implemented.16 Deworm the World has told us that it will not work with a government on a national deworming program until it has built a strong working relationship with that government via its advocacy.17 Deworm the World also participates in the broader "STH community"; that is, it works with other organizations advocating for and implementing activities that aim to further reduce or eliminate STH globally.18
- Prevalence surveys. Before Deworm the World helps launch a deworming program in a new area, it evaluates whether the prevalence of worm infections is sufficient to justify an MDA for the school-age population.19 If no prevalence surveys have been conducted recently, it generally commissions one.20 The results of prevalence surveys are used to determine the appropriate treatment strategy (in particular, MDA frequency) for a given location.21 It also conducts follow-up prevalence surveys periodically, so that it can track the impact of the MDAs and refine treatment strategies as needed, in accordance with WHO guidelines.22 Deworm the World generally contracts out work on prevalence surveys.23
- High-level program planning. Deworm the World has told us that it often assists governments with high-level operational decisions, such as developing the country's treatment strategy and operational guidelines24 and creating a budget for the program.25
- Drug procurement and protocols. Deworm the World assists governments in obtaining drugs, designing drug distribution and tracking processes, and developing adverse event protocols for cases where children react poorly to treatment.26 For example, Deworm the World has helped governments submit requests for deworming drugs (albendazole or praziquantel) to the World Health Organization (WHO) global drug donation program.27
- Program preparation: trainings and distribution of materials. Deworm the World has helped governments design and organize what it calls a "training cascade" (more detail in the footnote).28 Through the training cascade, teachers and other government staff learn how to implement a Deworming Day and receive materials necessary for implementation (such as reporting forms and drugs).29 In the past, Deworm the World has hired or trained staff to lead the trainings and developed materials for the trainings.30 In India, Deworm the World has also arranged tele-callers to reach out to schools to assess their preparedness and notify government officials of any problems before Deworming Day.31
- Community sensitization. Deworm the World supports community sensitization efforts, which aim to make local communities aware of Deworming Day and the benefits of deworming children.32 For example, via the training cascade, teachers are instructed to spread the word about Deworming Day to their communities.33 Deworm the World has also developed text message campaigns, organized public announcement events, and edited mass media materials to be more appropriate for local contexts.34
- Monitoring and evaluation. Deworm the World told us that it helps governments design or improve reporting and monitoring systems. It also collects monitoring data independently.35
Deworm the World focuses on assisting with the collection of three main types of monitoring data:36
- Monitoring before and during deworming: Monitors hired by Deworm the World visit schools before and during Deworming Day and Mop-Up Day. They are meant to assess both a) how prepared schools and health systems are to implement deworming and b) the extent to which proper procedures are followed.37 Monitoring visits may include assessments of the quality of trainings, community sensitization efforts, and Deworming Day activities, depending on what Deworm the World and the government agree to monitor.38
- Coverage reporting: On Deworming Day and Mop-Up Day, teachers are asked to mark the number of children that they deworm and schools complete specially designed reporting forms to tally the number of children treated. This data is then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (e.g., in India: school, node, block, district, and state) to create a reported coverage estimate for a region.39 Deworm the World notes that, as of at least 2017, block officials in India have submitted coverage data online to the national government.40
- Coverage validation: Approximately one to two weeks after deworming day (or Mop-Up Day) in India, Deworm the World sends independent monitors back to schools to check the coverage data and attendance records recorded at schools against the data submitted and ask students about whether or not they were dewormed. In Kenya, Nigeria, and Pakistan, monitors also visit households during coverage validation to interview children who are not enrolled in school or are not at school at the time of the school visit.41 This data can then be compared to the coverage data reported by the government.
Typically, Deworm the World hires and trains third-party monitors to collect process monitoring and coverage validation data; the following footnote includes Deworm the World’s descriptions of the monitor selection process used in four states in India in 2015.42 In Kenya, Evidence Action (Deworm the World's parent organization) maintains a monitoring team year-round that Deworm the World makes use of; this team conducts some data collection, and trains and manages third-party firms that conduct the majority of the data collection.43 In Nigeria, all data collection is conducted by third-party firms that are trained and managed by Evidence Action staff. In Pakistan, all data is collected by third-party firms that are hired and trained by Evidence Action’s in-country partner.44
On a separate page, we detail Deworm the World's work by country.
Breakdown of Deworm the World’s spending
We summarize Deworm the World's spending from 2017 through 2020 in this spreadsheet.
In short:
- Deworm the World spent a total of $8.7 million on deworming programs in 2020, down from $13.9 million in 2019.45 Our understanding is that this reduction in spending is largely explained by program disruptions caused by the COVID-19 pandemic, which included some rounds of deworming being missed,46 other rounds being modified,47 and monitoring activities being constrained.48 We have not asked Deworm the World for details on how modifications made to program delivery and monitoring affected program costs in 2020.
- In 2020, about 35% of its spending was from funding sources that are restricted to a particular project. This restricted funding supported programs in India, Kenya, and Pakistan, which were funded by CIFF, the END Fund, EPIC Foundation UK and EPIC Foundation France, and Dubai Cares.49 The other 65% of funding was from funds that Deworm the World can allocate at its discretion (this includes GiveWell-directed funds). Deworm the World used this unrestricted funding to fully support its program in Nigeria and to support a portion of program costs in India, Kenya, and Pakistan.
- Deworm the World's biggest program in 2020 (and historically) was in India (accounting for 45% of its program spending). Kenya, Nigeria, and Pakistan accounted for around 20% of Deworm the World's program spending each.50 The breakdown of total spending by country was similar in 2019.51
For information on spending in previous periods, see the previous versions of our review of Deworm the World.
Deworm the World and Evidence Action
In early 2013, Innovations for Poverty Action (IPA) announced the formation of Evidence Action to scale cost-effective and evidence-based programs. Two IPA initiatives, Deworm the World and Dispensers for Safe Water, were spun off from IPA to be managed by Evidence Action.52 Evidence Action subsequently built a department called Evidence Action Beta for investigating, testing, and considering new programs for scaling up;53 one program in the Beta portfolio was No Lean Season, which GiveWell also recommended as a top charity in 2017. We removed No Lean Season from our list of top charities in 2018; see this blog post for more detail. In 2019, Evidence Action replaced Evidence Action Beta with Evidence Action Accelerator, which will continue the work of developing new programs. We focus this review specifically on the Deworm the World program.
Does it work?
This section was last updated in November 2020. The information that Deworm the World has provided since that date is not yet reflected in this section.
We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.
We believe the evidence from Deworm the World's monitoring makes a relatively strong case that the programs it has supported have successfully dewormed children. Here we focus on Deworm the World's monitoring from Kenya, India, Nigeria, and Pakistan because those are its largest programs.54 Deworm the World's track record in Kenya and India is strong. Deworm the World's track record in Nigeria and Pakistan is more limited, but what we have seen seems fairly strong.
In the sections below, we focus on the following questions to understand whether Deworm the World’s activities are having the intended impact.
- Are mass school-based deworming programs effective when implemented well?
- Are Deworm the World's programs targeted at areas of need?
- Are deworming pills delivered to and ingested by recipients?
- How does Deworm the World affect program outcomes?
- Are there any negative or offsetting impacts?
Are mass school-based deworming programs effective when implemented well?
Deworm the World supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.
There are some important differences between the type and severity of worm infections in the places Deworm the World works and the places where the key studies on improved life outcomes from deworming took place (which we discuss below). In particular, Deworm the World primarily provides support to mass drug administrations (MDAs) that treat populations in which fewer children are infected with soil-transmitted helminths and where the severity of infections tends to be lower (as compared to populations in the key studies we refer to above). In addition, some of the programs Deworm the World supports do not treat schistosomiasis because it is not endemic in the areas the programs support.55