We have published a more recent review of this organization. See our most recent report on Evidence Action's Deworm the World Initiative.
The Deworm the World Initiative, led by Evidence Action, is one of our top-rated charities and we feel that it offers donors an outstanding opportunity to accomplish good with their donations.
More information: What is our evaluation process?
Published: November 2016
Summary
What do they do? The Deworm the World Initiative (evidenceaction.org/#deworm-the-world), led by Evidence Action, advocates for, supports, and evaluates government-run school-based deworming programs. (More)
Does it work? We believe there is relatively strong evidence for the positive impact of deworming. Deworm the World sends monitors to schools during and after deworming to determine whether the programs it supports have reached a large proportion of children targeted. We have reviewed data from its two largest programs and one new program, which overall indicate strong results. (More)
What do you get for your dollar? We estimate that, in most of Deworm the World's programs, children will be dewormed for a total of about $0.79 per child, or $0.55 per child excluding the value of teachers’ and principals’ time spent on the program. The cost per child treated in India is less than half of that. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)
Is there room for more funding? We believe that Deworm the World is likely to be slightly constrained by funding in the next year and that additional funds would increase the chances that it is able to take advantage of any high-value opportunities it encounters. We expect Deworm the World to have $11 million to spend or commit in the next year (it has about $8.9 million on hand and we expect it will receive about $2.2 in additional funding). We estimate that if it received an additional $13.4 million (allowing it to spend or commit a total of $24.4 million) its chances of being constrained by funding would be reduced to 5%. (More)
Deworm the World is recommended because of its:
- Focus on a program with a strong track record and excellent cost-effectiveness. (More)
- Strong process for assessing whether the deworming programs it supports are successfully deworming children. (More)
- Standout transparency – it has shared significant, detailed information about its programs with us.
- Room for more funding – we believe Deworm the World will be able to use additional funds to start or maintain deworming programs.
Major open questions include:
- Deworm the World's spending has lagged behind revenues as it has worked to start programs in new locations. It is unclear whether it will be able to scale up as quickly as it intends to.
- We are unsure how additional donations to Deworm the World will impact how Evidence Action, Deworm the World’s parent organization, chooses to allocate its unrestricted funding.
- We are not sure how Deworm the World’s success and cost-effectiveness in new countries will compare to its track record in India and Kenya.
Table of Contents
- Summary
- Our review process
- What do they do?
- Does it work?
- What do you get for your dollar?
- Is there room for more funding?
- Deworm the World as an organization
- Sources
Our review process
Our review process has consisted of:
- Conversations with Deworm the World Director Grace Hollister and other Deworm the World and Evidence Action staff since 2012.1
- A conversation with the Children's Investment Fund Foundation (CIFF), a funder of Deworm the World.2
- Reviewing documents Deworm the World sent in response to our queries.
- Following Deworm the World's progress and plans for funds raised as a result of GiveWell's recommendation (as well as Evidence Action's overall progress and plans).
- Site visits:
- In November 2012, we visited Deworm the World's office in Nairobi, Kenya and met its staff there. (Notes from our visit)
- In October 2013, we visited Deworm the World's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with Deworm the World. (Notes from our visit)
- In October 2016, we visited Evidence Action's offices in Washington, D.C. We met with five Deworm the World staff members, including leadership and program managers. We also met with several members of the Evidence Action leadership and finance teams.
- In 2015, we retained two journalists to visit areas served by Deworm the World in Kenya. We published their report on our blog.
All content on Deworm the World, including past reviews, updates, blog posts and conversation notes, is available here. We have also published a page with additional, detailed information on Deworm the World Initiative to supplement some of the sections below.
What do they do?
The Deworm the World Initiative, led by Evidence Action, advocates for and supports the implementation of government-run deworming programs for preschool- and school-age children.3 The support that Deworm the World provides is of two types: 1) Deworm the World offers technical assistance to governments implementing deworming, and 2) Deworm the World may fund components of deworming programs.4
The deworming programs that Deworm the World supports are focused on executing school-based mass drug administrations (MDAs), in which the aim is to treat the entire population of children within a geographic area by distributing deworming pills.5 Deworm the World focuses on MDAs that treat children infected with soil-transmitted helminthiasis (STH).6 Where needed, these programs may also include treatment for schistosomiasis.7
In the countries it works in, Deworm the World works primarily with government staff to implement deworming programs; as it has expanded to new countries, it has started to also collaborate with NGO partners to support governments.8 Additionally, Deworm the World has also funded or is considering working on a few projects that fall outside of its purview of supporting the direct implementation of deworming programs, specifically around monitoring and research (see footnote).9
Deworm the World was founded in 2007,10 and as of September 2016 had supported deworming treatments in India, Kenya, Ethiopia, Nigeria, and Vietnam and had started preliminary support for a deworming program in Pakistan.11 Many of these programs are recent and represent progress on Deworm the World's efforts to scale; as of early 2015, Deworm the World had only supported treatments in India and Kenya.12
Below, we discuss:
- Deworm the World's role in government-led deworming programs
- A breakdown of Deworm the World’s spending
- Deworm the World's relationship to Evidence Action
On a separate page with additional information about Deworm the World, we discuss the status of its work by country.
Deworm the World’s role in government-led deworming programs
The deworming programs that Deworm the World supports are implemented by the governments it works with.13 Below, we expand on Deworm the World's role in the programs it supports. Note that we use "Deworming Day" to mean the day on which the MDA takes place. Similarly, we use "Mop-Up Day" to refer to the day that occurs several days after Deworming Day and is when students who were absent or sick on Deworming Day are given their deworming pills (although note that not all countries have just one Deworming Day or include a Mop-Up Day in their program).14
The assistance that Deworm the World provides in each country varies based on what each partnering government needs.15 Historically, Deworm the World's role has included the following:16
- Advocacy. Deworm the World actively encourages national and large sub-national governments to implement mass school-based deworming programs.17 Our impression is that Deworm the World's advocacy consists of meeting with health and education officials in a government to discuss the benefits of deworming and how a deworming program might be implemented.18 Deworm the World has told us that it will not work with a government on a national deworming program until it has built a strong working relationship with that government via its advocacy.19 Deworm the World also participates in the broader "STH community"; that is, it works with other organizations advocating for and implementing activities that will further reduce or eliminate STH globally.20
- Prevalence surveys. Before Deworm the World helps launch a deworming program in a new area, it evaluates whether the prevalence of worm infections is sufficient to justify an MDA for the school-age population.21 If no prevalence surveys have been conducted recently, it generally commissions one.22 It also plans to conduct follow-up prevalence surveys periodically, so that it can track the impact of the MDAs and refine treatment strategies as needed, in accordance with WHO guidelines.23 Deworm the World generally contracts out work on prevalence surveys.24
- High-level program planning. Deworm the World has told us that it often assists governments with high-level operational decisions, such as developing the country's treatment strategy and how to budget for the program.25
- Drug procurement and protocols. Deworm the World assists governments in obtaining drugs, designing drug distribution and tracking processes, and developing adverse event protocols for cases where children react poorly to treatment.26 For example, Deworm the World has helped governments submit requests for deworming drugs (albendazole or praziquantel) to the World Health Organization (WHO) global drug donation program.27
- Program preparation: trainings and distribution of materials. Deworm the World has helped governments design and organize what it calls a "training cascade" (more detail in the footnote).28 Through the training cascade, teachers and other government staff learn how to implement a Deworming Day and receive materials necessary for implementation (such as reporting forms and drugs).29 In the past, Deworm the World has hired or trained staff to lead the trainings and developed materials for the trainings.30 In India, Deworm the World has also arranged tele-callers to reach out to schools to assess their preparedness and notify government officials of any problems before Deworming Day.31
- Community sensitization. Deworm the World supports community sensitization efforts, which aim to make local communities aware of Deworming Day and the benefits of deworming children.32 For example, via the training cascade, teachers are instructed to spread the word about Deworming Day to their communities.33 Deworm the World has also developed text message campaigns, organized public announcement events, and edited mass media materials to be more appropriate for local contexts.34
- Monitoring and evaluation. Deworm the World told us that it helps governments design or improve reporting and monitoring systems. It also collects monitoring data independently.35
Deworm the World focuses on assisting with the collection of three main types of monitoring data:36
- Monitoring before and during deworming: Monitors hired by Deworm the World visit schools before and during Deworming Day and Mop-Up Day. They are meant to assess both a) how prepared schools and health systems are to implement deworming and b) the extent to which proper procedures are followed.37 Monitoring visits may include assessments of the quality of trainings, community sensitization efforts, and Deworming Day activities, depending on what Deworm the World and the government agree to monitor.38
- Coverage reporting: On Deworming Day and Mop-Up Day, teachers are asked to mark the number of children that they deworm and schools complete specially designed reporting forms to tally the number of children treated. This data is then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (e.g., in India: school, node, block, district, and state) to create a reported coverage estimate for a region.39
- Coverage validation: Approximately one week after Mop-Up Day, Deworm the World sends independent monitors back to schools to check the coverage data recorded at schools against the data submitted and ask students about whether or not they were dewormed.40 This data can then be compared to the coverage data reported by the government.
Typically, Deworm the World hires and trains third-party monitors to collect process monitoring and coverage validation data; this footnote includes Deworm the World’s descriptions of the monitor selection process used in four states in India in 2015.41 In Kenya, Evidence Action (Deworm the World's parent organization) maintains a monitoring team year-round that Deworm the World makes use of.42
On a separate page, we detail Deworm the World's work by country.
Breakdown of Deworm the World’s spending
We have seen detailed breakdowns of Deworm the World's spending for 2014, 2015, and the first half of 2016.43 The vast majority of this spending (about 89%) was funded by restricted funding.44 Restricted funding from the Children's Investment Fund Foundation (CIFF) and the END Fund fully fund Deworm the World’s work in Kenya, where it has supported a nationwide deworming program since 2012.45 In 2014 and 2012 respectively, Deworm the World received funding commitments from CIFF/Dubai Cares and USAID to fully support its work in India; in 2015 a grant from the END Fund was received to support work in Rajasthan.46
Deworm the World’s spending on deworming programs is supplemented by the spending of partner organizations and the implementing governments. Below is a breakdown of how Deworm the World spent its funding from January 2015 through June 2016 globally as well as in Kenya and India.47 In 2015, Deworm the World spent about $5.2 million on all expenses.48 Deworm the World has told us that the majority of its personnel costs fall within the "program management" category.49
| India | Kenya | All locations | |
|---|---|---|---|
| Program management | 39% | 32% | 45% |
| Training | 7% | 45% | 21% |
| Monitoring and evaluation | 18% | 10% | 11% |
| Prevalence surveys | 17% | 0% | 9% |
| Policy | 6% | 5% | 6% |
| Awareness | 10% | 6% | 6% |
| Drugs | 3% | 1% | 2% |
| Total | 100% | 100% | 100% |
Deworm the World and Evidence Action
In early 2013, Innovations for Poverty Action (IPA) announced the formation of Evidence Action to scale cost-effective and evidence-based programs. Two IPA initiatives, Deworm the World and Dispensers for Safe Water, were spun off from IPA to be managed by Evidence Action. It has since built a department for investigating, testing, and considering new programs for scaling up called Evidence Action Beta; one program in the Beta portfolio is No Lean Season, which GiveWell has recommended grants to.51 We focus this review on Deworm the World and discuss the room for more funding implications of Deworm the World being a program of a larger organization below.
Does it work?
Deworm the World-assisted mass drug administration programs are focused on delivering treatments that have been independently studied in rigorous trials and found to be effective. Evidence from Deworm the World's monitoring makes a relatively strong case that the programs Deworm the World has supported have successfully dewormed children.
While Deworm the World's track record in Kenya and India is strong, it has recently expanded to several new countries. As with any major scale-up, there is a risk that it may not achieve as strong results as it has in the past or may not produce similarly strong evidence of its impact. We have seen preliminary monitoring from one of its new countries of operation (Vietnam); these results indicate that Deworm the World is using similar monitoring processes in new countries as it has in Kenya and India and that results in Vietnam have been reasonably strong.
Here we focus on the following questions to understand whether Deworm the World’s activities are having the intended impact (details in the sections that follow).
- Are mass school-based deworming programs effective when implemented well?
- Are Deworm the World's programs targeted at areas of need?
- Are deworming pills delivered to and ingested by recipients?
- How does Deworm the World affect program outcomes?
- Are there any negative or offsetting impacts?
Are mass school-based deworming programs effective when implemented well?
Deworm the World supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.
There are some important differences between the type and severity of worm infections in the places Deworm the World works and the places where the key studies on improved life outcomes from deworming took place (which we discuss below). In particular, Deworm the World primarily provides support to mass drug administrations (MDAs) that treat populations where fewer children are infected with soil-transmitted helminths and where the severity of infections tends to be lower. In addition, several of the programs Deworm the World supports do not treat schistosomiasis because it is not endemic in the areas the programs support.52
Is the program targeted at areas of need?
Deworm the World has told us that it advocates for governments to follow WHO guidelines for treatment of STH (the guidelines are based on baseline prevalence):53
- If prevalence is below 20%, MDA is not needed;
- If prevalence is between 20% and 50%, MDAs should occur once per year;
- If prevalence is greater than 50%, MDAs should occur twice per year.
- After multiple (5-6) years of treatments, WHO recommends further annual MDAs for areas with at least 10% STH prevalence and MDAs every two years for areas with 1-10% prevalence.54
Deworm the World also advocates for governments to follow WHO guidelines for treatment of schistosomiasis, in the areas it works where schistosomiasis is endemic.55 We have seen evidence indicating that Deworm the World primarily supports annual MDAs in areas with prevalence greater than 20% and biannual MDAs in areas with prevalence greater than 50%; we have not seen any indication that Deworm the World systematically treats populations with low infection rates.56
Baseline infection status: Deworm the World programs vs. rigorously studied programs
Although Deworm the World only supports MDAs in areas where the prevalence is high enough to warrant treatment according to WHO, most of the populations that Deworm the World's program targets have a lower prevalence and intensity of STH and schistosomiasis than the populations treated by the deworming programs discussed in our intervention report on deworming. Given this, we assume that the programs Deworm the World supports have less impact per child treated than the deworming programs that have been rigorously studied.
Key pieces of evidence that we discuss in our report on deworming (Miguel and Kremer 2004, Baird et al 2012, and Croke 2014) are from deworming experiments conducted in Kenya and Uganda in the late 1990s and early 2000s. Prior to receiving deworming treatment, the participants in those studies had relatively high rates of moderate-to-heavy infections of schistosomes or hookworm.57
In this spreadsheet, we compare the prevalence of the places in which Deworm the World currently supports a program to the prevalence from the Croke study and the Miguel and Kremer study. The prevalences in the table for Madhya Pradesh and Chhattisgarh were measured after multiple rounds of treatment, while Bihar's, Rajasthan's, and Uttar Pradesh's prevalences were measured after one round of treatment. This makes it difficult to compare the prevalences in a meaningful way. It is likely that prevalence was higher at baseline for these regions.
Deworm the World notes that there are relevant methodological differences between the prevalence surveys, which makes them difficult to compare (see our previous footnote for more detail on methodology); we do not fully understand which methodological differences Deworm the World is referring to and we believe the surveys are similar enough that comparison remains a useful exercise.58
We would guess that future programs Deworm the World supports will also treat in areas with different worm types and significantly lower prevalence than those in the key deworming studies, largely because prevalence in those studies was unusually high.59
Treatment for lymphatic filariasis
In some of the countries where Deworm the World works, there are existing programs to treat lymphatic filariasis (LF).60 Albendazole, the same drug used to treat STH, is usually used in combination with one additional drug to treat LF (and the same dosage is used for both treatments).61 For areas that have existing LF treatment programs, the effect of Deworm the World’s support may be to transition an area from once-per-year deworming treatment to twice-per-year treatment.62
We detail what we know about the status of LF programs in the areas in which Deworm the World works on a separate page with additional information about Deworm the World.
Are deworming pills delivered to and ingested by recipients?
Administration of deworming drugs is a relatively straightforward program, though any MDA could encounter many challenges when operating at a large scale. The information we have seen from monitors hired and trained by Deworm the World in India, Kenya, Vietnam, and Ethiopia suggests that the programs successfully deliver pills to children, who then swallow them. We have also seen one coverage survey from Ethiopia's national deworming program pilot in April 2015, which was supported by Deworm the World and SCI and indicates that pills were delivered and ingested. Additionally, prevalence surveys in Kenya, and to a lesser degree Bihar, India, show that the prevalence rates of STH and schistosomiasis have declined substantially since Deworm the World started supporting MDAs in those areas, providing additional evidence that the treatments are reaching recipients.
Evidence from monitoring
For each of its programs, Deworm the World hires monitors (who are not associated with the government implementing the program) to:63
- [Sometimes] Observe trainings. For some of its programs, Deworm the World sends monitors to visit a random selection of trainings.64 The visits are unannounced, and the monitors use a standardized checklist to track if the training covers all required topics.65 Monitors also test how the training increases the knowledge of training attendees by selecting several attendees before the training and administering a short quiz to them, then selecting several attendees after the training to again take the quiz.66
- [Sometimes] Visits or calls to communities and schools before Deworming Day. For some of its programs, Deworm the World asks monitors to visit or call, unannounced, a random selection of schools and/or communities before Deworming Day.67 At the schools, monitors will interview teachers and/or headmasters, asking a variety of questions to assess preparedness, such as whether or not the school has enough treatments for Deworming Day and if a representative from the school attended training.68 In communities, monitors select a sample of adults to interview, asking questions to assess their awareness of Deworming Day, which helps Deworm the World determine how successful its community sensitization efforts were.69
- Observe activities on Deworming Day and Mop-Up Day. In all of its programs, Deworm the World sends its monitors, unannounced, to observe a random sample of schools on Deworming Day and Mop-Up Day.70 If the chosen school is closed on the day of the deworming, they are instructed to go to an assigned backup school instead.71 At the schools, monitors interview teachers and school administrators to assess how prepared the school was for Deworming Day. For example, monitors ask if the school has sufficient drugs for Deworming Day, whether or not a school representative attended training, and a variety of questions to test teachers' knowledge about the proper procedures for the MDA campaign, like what the teacher should do if a child is feeling sick.72 Then, the monitors observe randomly selected classes, recording details about the Deworming Day activities, such as whether deworming is in progress, teachers are documenting who is dewormed, and teachers are watching to make sure that students swallow the pills.73
- Conduct a coverage validation exercise. In most of its programs, Deworm the World sends out monitors within 1-2 weeks of Deworming Day and Mop-Up Day to conduct a coverage validation exercise at a randomly selected sample of schools.74 At the schools, monitors ask teachers questions about Deworming Day, such as if the school had sufficient tablets.75 They also check class registers and record the number of students that were dewormed according to the school's records.76 Finally, they randomly select a small sample of students to interview, asking the children several questions about their experience on Deworming Day.77 For example, they might ask if the child received a pill, if the child swallowed the pill, and how the child was feeling on Deworming Day.78
We believe that the last two types of monitoring are especially valuable. However, Deworm the World is not supporting an coverage validation exercise in schools in Ethiopia, because household coverage surveys conducted after the Deworming Days are occurring there instead. Deworm the World also did not support coverage validation monitoring for the first MDA it supported in Cross River, Nigeria, although it hopes to in subsequent rounds.79
While we believe that Deworm the World's monitoring is fairly high-quality, we have a few concerns about its methods:
- Deworm the World does follow-up calls to audit monitors' work in Kenya. It also has some checks on monitors' work in India. We are not aware of audits of monitors' work in other locations.80
- Deworm the World's program and monitoring are done through schools, which makes it difficult to know how many non-enrolled children are potentially being missed by the program. Schools are supposed to target and record the treatments they distribute to non-enrolled children, but most locations do not have reliable data on how many non-enrolled children there are.81
- Even though school staff are not aware ahead of time that a monitor is coming to visit, once a monitor shows up, they may be motivated to execute the program in a more rigorous fashion than they would have otherwise.
- It is possible that children feel pressure to say that they took the deworming pill on coverage validation day. Deworm the World tries to ask students away from their teachers, to reduce pressure, and also has some checks on students' answers.82
Results from monitoring
We find the evidence produced by Deworm the World from observations on Deworming Day and during the coverage validation exercise to be compelling support for the claim that children receive and ingest pills during Deworm the World-supported MDAs; we have laid out this evidence in this spreadsheet.83 We also see some reason to interpret these data cautiously, as some of it was self-reported by people who could be biased to want favorable monitoring results. We note that some of the results from Kenya seem quite high to us when compared to the results we have seen from India. We are not sure if this is due to some bias in the way that the Kenya data is collected, or if Deworm the World's program in Kenya is simply higher-quality.84
See this spreadsheet for a sample of methods used in and results from recent monitoring in Kenya, India, and Vietnam (the three countries from which we've seen this data). We have not yet seen Deworm the World's standard monitoring data from Nigeria or Ethiopia, but Deworm the World told us that it should be similar to the type of monitoring that we've seen in the past.85
The spreadsheet does not include all data the monitors collected. We have selected questions that we found particularly relevant to assessing the quality of the programs and easy to interpret. (Note: Previously, we summarized monitoring from the 2013-2014 programs and the 2012 programs).
We have not prioritized reviewing the data from the monitoring Deworm the World conducts before Deworming Day, although we think it could also provide insight as to the quality of Deworm the World's program. However, we have included some metrics in our table related to the preparedness of schools and their adherence to proper protocols (from questions that are asked on Deworming Day). Based on the data and reports we have seen, we feel fairly confident that most schools in India and Kenya are prepared to implement Deworming Days: most schools seem to have sufficient number of deworming pills available for children; fewer appear to have adequately trained teachers (the fact that the surveys identified low training rates in several cases increases our confidence in the reliability of the surveys). We feel less confident that schools are following proper procedures, especially with regards to accurate reporting.
Ethiopia coverage survey
Although we have not yet seen results from Deworm the World's standard monitoring for the MDAs conducted in Ethiopia in mid and late 2015, we have seen a coverage survey conducted after the April 2015 MDA.86 It is our impression that the coverage survey was primarily supported by Deworm the World's partner in Ethiopia, SCI. We write about the methods used for SCI's coverage surveys here and lay out the results and methods used for the Ethiopia coverage survey in this spreadsheet. We believe the coverage survey provides relatively strong evidence that deworming pills were distributed and ingested by children in Ethiopia.
Evidence from prevalence surveys over time
Deworm the World or its partners have conducted several surveys to track changes in schistosomiasis and STH prevalence and intensity rates following Deworm the World-supported treatment programs. In general, prevalence and intensity of the parasites decreased over time in each of the countries studied. This increases our confidence that Deworm the World's program is effectively delivering treatments to children. We discuss methodology, results, and limitations to using these surveys as evidence of Deworm the World's impact on another page.
How does Deworm the World affect program outcomes?
Deworm the World may be having an impact in the following ways:
- It may increase the likelihood that a government implements a deworming program, by advocating for deworming programs, by offering to provide technical assistance, and/or by funding implementation. Deworm the World expects to pay for the majority of financial program costs in many of the new countries to which it has expanded or intends to expand.87 In situations where Deworm the World is funding a deworming program, we believe it's likely that Deworm the World plays an instrumental role in causing the program to happen.88 Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World.
- It may improve the quality of a deworming program that would have been implemented without Deworm the World (leading to more children dewormed effectively or improved cost-effectiveness). Our intuition is that Deworm the World’s activities increase the quality of the programs it supports, but we are uncertain about this. Full discussion of the evidence we have reviewed on this question is on a separate page with additional information on Deworm the World.
Are there any negative or offsetting impacts?
We discuss several possible considerations but do not see significant concerns.
Administering deworming drugs seems to be a relatively straightforward program.89 However, there are potential issues that could reduce the effectiveness of some treatments, such as:
- Drug quality: For example, if drugs are not stored properly they may lose effectiveness or expire.90 Our understanding is that Deworm the World periodically tests the quality of drugs and has monitored storage conditions in each of its recent programs, and this information suggests there have been minimal issues.91 In India, state governments are responsible for testing the drugs they use.92
- Dosage: If the incorrect dosage is given, the drugs may not have the intended effect and/or children may experience additional side effects.93 It appears that for STH treatment, all children of a given age group are given the same dose of albendazole and that the dose is generally a single tablet for children 2 years old and above, and half a tablet for those between the ages of 1 and 2.94 Deworm the World monitors reported that, in recent programs in India, up to 9% of schools observed gave children less than the prescribed dose of albendazole and up to 5% gave more than the prescribed dose of albendazole.95
- Concerns over whether treatment is sustained: We believe it is important that deworming programs are sustained over time, as re-infection is rapid and a one-time treatment may have little long-term effect.96
- Replacement of government funding: We have limited information about whether governments would pay for the parts of the program paid for by Deworm the World in its absence, though our impression is that they would not.97 We also have little information about what governments would use deworming funds for if they did not choose to implement deworming programs.
- Diversion of skilled labor: Drug distribution occurs only once or twice per year and is conducted by teachers in schools. Based on our site visit in Rajasthan, our impression is that the Nodal Officer (the state official who manages all state school-based programs), the Nodal Officer’s staff, and the people that the Nodal Officer manages throughout the state (Resource Persons and Community Development Project Officers) have significant capacity to take on additional programs, so their taking on this program doesn't impose a significant burden on their time.98 On the other hand, a principal we spoke with commented that he would prefer fewer school-based health programs because they take focus away from the school day.99
- Adverse effects and unintended consequences of taking deworming drugs: Our understanding is that expected side effects are minimal and there is little reason to be concerned that drug resistance is currently a major issue (more information from our report on deworming).
- Popular discontent: We have heard a couple of accounts of discontent in response to mass drug administration campaigns, including one case that led to riots.100 While the accounts we have heard are from programs supported by the Schistosomiasis Control Initiative, we think it is possible that other deworming programs could cause similar discontent.
What do you get for your dollar?
This section examines the data that we have to inform our estimate of the expected cost-effectiveness of additional donations to Deworm the World. Note that the number of lives significantly improved is a function of a number of difficult-to-estimate factors. We incorporate these into a cost-effectiveness model which is available here. In this section, we focus on the cost per child treated, which is an important input in our cost-effectiveness model.
On a separate page, we discuss:
- How accurate are Deworm the World's reported coverage figures? Deworm the World uses government-reported figures on the number of treatments delivered when calculating its cost-per-treatment. We remain uncertain of the accuracy of these figures. More here.
- Does Deworm the World "leverage" government funds, such that its activities mobilize resources from other actors? We could imagine that Deworm the World's funds have substantial leverage but could also imagine that other actors’ involvement is causing Deworm the World to pay for things for which other actors would otherwise have paid. Deworm the World may have less leverage in its future programs than it has had in past programs. More here.
What is the cost per treatment?
We estimate that in India children are dewormed for a total of about $0.32 per child, or $0.09 per child excluding the value of teachers’ and principals’ time spent on the program. In Kenya, we estimate the total cost per treatment at about $0.79 per treatment or $0.55 excluding the value of teachers’ and principals’ time spent on the program. We expect the cost per treatment in Deworm the World’s potential new countries to be closer to Kenya's costs; however, we are highly uncertain about this.
When considering the cost-effectiveness of additional donations, we consider the cost per treatment for the Deworm the World programs for which we have data: India and Kenya. Deworm the World told us that the cost per treatment in India is unusually low; it expects other programs' cost per treatment to be more similar to the program that it supports in Kenya.101 Deworm the World told us that its initial, rough estimates of the cost per treatment in Vietnam and Nigeria look very similar to the costs in Kenya, although these estimates could change significantly as more data comes in.102
Note that in our cost-per-treatment analyses below, we use data that Deworm the World has sent us for its most recent rounds of treatment in Kenya and India for which it has cost data. It is possible that this makes Deworm the World's program look more cost-effective than it actually is; for example, this may exclude start-up costs from our analysis. Deworm the World has told us that it tries to capture all costs of its program, regardless of who pays for the cost. However, we know that it does not include the value of teacher time during trainings and on Deworming Day103 and that there are several high-level costs not directly attributable to programs that Deworm the World does not include (such as exploratory work in new geographies that does not lead to a new program).104 We have included these costs in our analyses.
To see cost-per-treatment figures across multiple years, see our previous reviews or GiveWell analysis of Deworm the World cost-per-treatment, October 2016.105
India
As of September 2016, Deworm the World had estimated the total cost of nine deworming rounds in India.106 These estimates include the costs listed in the following tables. Each of the costs was paid by a combination of government, Deworm the World, and other partners. The following table shows the percentage breakdown for the most recent round of deworming in Bihar, Rajasthan, and Madhya Pradesh.107 Note that it does not include the value of school and government staff time during training, deworming day, and mop-up day.108
| Cost category | Percentage of total costs | % paid by DtWI |
|---|---|---|
| Policy & advocacy | 1% | 100% |
| Prevalence surveys | 6% | 100% |
| Drug procurement & management | 37% | 0% |
| Training & distribution costs | 10% | 36% |
| Public mobilization & community sensitization | 16% | 22% |
| Monitoring & evaluation | 7% | 86% |
| Program management | 20% | 100% | Additional high-level costs (GiveWell assumption) | 4% | 100% |
| Total | 100% | 44% |
Deworm the World has estimated some of the above costs because it did not have access to full cost data for governments and other partners, and boundaries between the cost categories are not always clear.110
The table below shows the costs to Deworm the World, its partners, and the governments, as well as estimates of the cost per child treated, for the most recent rounds of treatment in Bihar, Rajasthan, and Madhya Pradesh.111 About two thirds of the total cost comes from contributions of time from government employees—which have been monetized according to salary levels—rather than financial costs. We would guess that appropriately valuing that time spent, and estimating how much time is spent by teachers and others due to the deworming program, is the largest source of uncertainty in the cost-per-child-treated calculation.112 In its own calculations, Deworm the World excludes the value of government employees' time because the government would have incurred these costs in the absence of the program.113
| Expense category | Recent deworming rounds | Cost per child |
|---|---|---|
| Deworm the World | $1,602,778 | $0.04 |
| Partners | $1,214,409 | $0.03 |
| Government financial costs | $864,870 | $0.02 |
| Government staff time value | $9,791,115 | $0.23 |
| Subtotal without staff time | $3,682,057 | $0.09 |
| Total costs | $13,473,172 | $0.32 |
Kenya
We estimate the total cost per treatment in Kenya to be about $0.79 (details below).115
In the third round of treatment in Kenya, we estimate that the total cost per treatment was $0.55 (not including the value of teachers' time).116 Deworm the World's cost per treatment in Kenya is in the same range as our estimate of SCI's cost per treatment, which averages data across nine programs in Africa and three years.117 Our estimate of Deworm the World’s total cost per treatment in Kenya attempts to include all partners’ costs so that it represents everything required to deliver the treatments.118 Our estimate includes the value of teachers' time, even though this does not represent an additional financial cost to the program (because the government pays the teachers with or without the program). In our analysis of SCI, we estimate that 30% of the total program costs are in-kind support from the government.119 Applying the same approach to Deworm the World (because we do not have comparable data for Deworm the World), we estimate that the total cost per treatment in Kenya is $0.79.120
| Expense category | Recent deworming rounds | Cost per child |
|---|---|---|
| Deworm the World | $3,157,187 | $0.50 |
| Partners | $329,688 | $0.05 |
| Government financial costs | $14,918 | $0.00 |
| Government staff time value | $1,485,851 | $0.23 |
| Subtotal without staff time | $3,501,793 | $0.55 |
| Total costs | $4,987,644 | $0.79 |
Is there room for more funding?
We believe that there is a 50% chance that Deworm the World will be slightly constrained by funding in the next year and that additional funds would increase the chances that it is able to take advantage of any high-value opportunities it encounters. We estimate that if it received an additional $13.4 million its chances of being constrained by funding would be reduced to 5%.
In short, we calculate this from (more details in the sections below):
- Total opportunities to spend funds productively: We estimate that Deworm the World could productively use or commit between $11.4 million (50% confidence) and $24.4 million (5% confidence) in unrestricted funding in its next budget year. This excludes $6.6 million that is restricted or already allocated to its programs.
- Cash on hand: As of the end of July 2016, Deworm the World held $8.9 million that will be available in 2017: $17.6 million in cash on hand, of which $6.6 million was committed and $2.1 million that it expects to set aside for reserves.
- Expected additional funding: We estimate that Deworm the World will receive an additional $2.2 million in unrestricted funding for its work in 2017.
Below, we also discuss:
- Past spending: To date, Deworm the World's spending has lagged behind revenues as it has worked to start programs in new locations. We see this as an ongoing risk, but not a major reason to reduce support. While it has often deviated from previous plans, we believe its spending decisions have been reasonable and loosely in accordance with our expectations.
- Additional considerations: We have found errors in Deworm the World's financial statements that reduce our confidence that we have a complete, accurate understanding of Deworm the World's financial situation. We also discuss four issues that are material to a determination of Deworm the World’s room for more funding: (a) the possibility of alternative funders of its work in Kenya, (b) its preference for multi-year commitments from donors, (c) its expectation that it will utilize a new operating model when expanding into new countries, and (d) its relationship with Evidence Action, which also has unrestricted funding available.
Available and expected funds
At the end of July 2016, Deworm the World held approximately $17.6 million, of which:122
- $1.5 million was restricted by donors to specific programs
- $5.1 million was unrestricted and committed to projects or set aside for reserves ($1.3 million)
- $11 million was unrestricted and uncommitted, of which Deworm the World expects to set aside $2.1 million for reserves, leaving $8.9 million available
We expect that Deworm the World will receive additional donations over the remainder of 2016 and in 2017 from:
- Donors who are not influenced by GiveWell's research: Deworm the World expects to receive roughly $1.2 million in unrestricted funding.123
- Donors who give based on GiveWell's top charity list, but do not follow our recommendation for marginal funding: GiveWell maintains both a list of all top charities that meet our criteria and a recommendation for which charity or charities to give to to maximize the impact of additional donations, given cost-effectiveness of remaining funding gaps. We estimate that Deworm the World will receive about $1 million from donors who use our top charity list but don't follow our recommendation for marginal donations.124
- Donors who follow GiveWell's recommendation for marginal donations: Our estimate of room for more funding is used to make a recommendation to these donors.
With $8.9 million in available funds on hand and $2.2 million expected in additional funding ($1.2 and $1 million from the first two groups, respectively), we estimate that Deworm the World will have about $11 million available in 2017.125
Deworm the World expects to receive additional restricted funding during the remainder of 2016 and in 2017 for some of its programs; we have not asked for additional detail about this restricted funding because Deworm the World does not expect the funding gaps we are considering to be filled by restricted funding.126
Uses of additional funding
In the table below, we've briefly summarized the details of Deworm the World's funding gaps; further detail follows the table. We discuss our prioritization of these funding gaps more below.
| Opportunity | Total additional cost (millions USD) | Cumulative funding need (millions USD) | GiveWell's prioritization |
|---|---|---|---|
| Supporting 50% of the costs for 2 years of (a) continuing the national deworming program in Kenya and (b) additional prevalence surveys in Kenya | 2.4 | Deworm the World will have sufficient funds | Execution level 1 |
| Expansion to 2 new states in India for 3 years | 2.9 | Deworm the World will have sufficient funds | Execution level 1 |
| Expansion to 3 new states in Nigeria for 3 years | 6.1 | 0.4 | Execution level 1 |
| Supporting 50% of the Kenya program and additional surveys for an additional 2 years (4 years total) | 2.4 | 2.8 | Execution level 2 |
| Expansion to 1 additional state in India for 3 years | 1.2 | 4.0 | Execution level 2 |
| Expansion to 1 province in Pakistan for 2 years | 0.5 | 4.5 | Execution level 2 |
| Expansion to Indonesia | 4.0 | 8.5 | Execution level 3 |
| Supporting 50% of the Kenya program and additional surveys for an additional year (5 years total) | 1.2 | 9.7 | Execution level 3 |
| Expansion to 1 additional province in Pakistan | 3.7 | 13.4 | Execution level 3 |
| Total | 24.4 | 13.4 |
More detail:128
- Kenya: Deworm the World is seeking funding to support an additional five years of the national school-based deworming program in Kenya. The first five years of the program were supported primarily by CIFF, and we discuss the possibility of CIFF continuing funding for the program below. The total cost of the five-year program is projected to be $10.5 million, and Deworm the World would like an additional $1.5 million (total over 5 years) to support prevalence surveys to track the program's impact. It expects other funders to support approximately half of these costs and is looking to fill the remaining half of this gap.
- India: Currently, all states and territories in India participate in National Deworming Day. However, Deworm the World believes that some states could benefit from additional technical assistance to achieve greater coverage and better monitor programs. It believes it can scale up its program to support three additional states in 2017.129 Each state would require approximately $300,000 per year, and Deworm the World would like to be able to commit to three years of support for each state. Additionally, Deworm the World would need funding to support the scale up of its national team to support the state teams.130
- Nigeria: Deworm the World would like to expand its support in Nigeria to three additional states. As of late 2016, it received permission from the government of Nigeria to move forward with its support for three states.131 We estimate that it will need approximately $6.1 million to support all three states for three years.132 These estimates include some national-level costs to support the state programs.
- Pakistan: Deworm the World would like to fund a deworming program in Pakistan, but is waiting on the results of the prevalence survey it recently supported to determine what size of a program Pakistan needs. Preliminary results indicate that Pakistan may have a lower worm prevalence than initially thought, which could lead to a smaller program than expected in Pakistan.133 Conservatively, Deworm the World believes it may only commit to supporting one province in Pakistan in 2017, which would require about $500,000 for two years.134 In the past, Deworm the World told us that being able to commit to multiple years of funding would make negotiations with provinces in Pakistan easier, and that it might be difficult to reach an agreement without having five full years of funding available.135 If Deworm the World were to commit to two provinces in Pakistan, it might need up to $4.2 million in 2017.136
- Indonesia: Deworm the World has started preliminary conversations with Indonesia's government about supporting a deworming program there. It believes that conversations might move forward quickly in 2017 because, unlike in Pakistan, the Indonesian government has an NTD department to coordinate with. Deworm the World is very unsure how large of a program it might support in Indonesia; it estimates between $1.6 and $4 million, but these figures could easily change.
- Reserves: In its room for more funding analysis, Deworm the World noted that it expected to set aside an additional $2.1 million, to add to the $1.32 million it currently holds in reserve. This amount may vary with its total 2017 budget.137 We are unsure about the circumstances under which Deworm the World would make use of those reserves, given that its budget is largely supported by multi-year restricted grants or multi-year allocations from unrestricted funds. We do not believe Deworm the World needs reserves in cases where it already has the funding on hand for a multi-year program, but we believe reserves might be needed for (a) global costs and (b) cases where Deworm the World receives its funding for a program in installments, because historically Deworm the World has experienced cases of installments not being sent in a timely manner.
GiveWell's prioritization of Deworm the World's funding gaps
We have broken down our our top charities' funding gaps and ranked them based on:
- Capacity relevance: how important the funding is for the charity's development and future success.
- Execution relevance: how likely the charity's activities will be constrained if it does not receive the funding.
We believe that "capacity-relevant" gaps are the most important to fill, and "execution"-related gaps vary in importance. More explanation of this model is in this blog post.
In the table above, we have not ranked any of the funding gaps "capacity relevant" because we do not see a strong case that filling those gaps would increase our confidence in Deworm the World's performance (we already believe Deworm the World has a strong track record) or would have an outsized impact on unlocking additional funding opportunities in the near future (in part because Deworm the World makes multi-year agreements).
We consider the funding gaps to be "execution" gaps and assign them a level (1, 2 or 3) by how likely we believe it is that Deworm the World would be constrained by funding (rather than other factors, such as an inability to grow staff capacity quickly enough) if it is unable to fill the funding gap. Level 1 is 50% chance of funding being the constraint, level 2 is 20% chance, and level 3 is 5% chance. These judgements are rough and largely based on a) what Deworm the World has told us about the progress on the various opportunities it is pursuing and b) intuitions formed from following Deworm the World's progress over several years.
Deworm the World has worked in Kenya and India for over 5 years, and we feel confident that Deworm the World will be able to scale up or continue its support in those countries with additional funding. We consider a large portion of the India gap and the first two years of the Kenya gap to be "Execution Level 1" gaps. We also feel moderately confident in Deworm the World making significant progress in Nigeria in 2017; Deworm the World has already successfully supported one state there and has progressed to a late stage in discussions with another three states. So, we consider the funding required to scale up to three additional states in Nigeria to also be "Execution Level 1."
We are less confident in Deworm the World's scale-up in Indonesia and Pakistan. We believe it is somewhat more likely Deworm the World will start to support a program in Pakistan because it has already completed a prevalence survey there, so we have labelled a small portion of the Pakistan gap "Execution Level 2" (i.e., there is only a 20% chance that Deworm the World will be constrained if it does not receive funding for this opportunity).
We think Deworm the World's ambitious growth opportunities are the most uncertain, so we have labelled the full amount of these opportunities (scaling expansively in Pakistan and Indonesia, and funding Kenya for a full 5 additional years) to be "Execution Level 3" gaps.
Past uses of unrestricted funds
In the past, Deworm the World has spent and committed unrestricted funding more slowly than we expected and on different activities than it predicted it would. However, we believe that the alternative uses of unrestricted funding have been reasonable.
Speed of spending
GiveWell started recommending Deworm the World in November 2013; since then (through July 2016), we have influenced about $15.3 million in unrestricted funding to Deworm the World, about $12 million of which was in the last year.138 We estimate that, in the same time period, Deworm the World has raised $2.2 million from other sources.139 So far, Deworm the World has spent $2.3 million of this funding (13%) and committed an additional $5.1 million to future projects (total 42% spent or committed).
More details below (we've also laid out the information in this spreadsheet):
- 2013 - 2014: GiveWell first recommended Deworm the World in fall 2013. Over the 2013 giving season and through June 2014, GiveWell-influenced donors gave about $2.3 million to Deworm the World.140 In 2014, Deworm the World only spent $336,000 in unrestricted funding, although in late 2014 it told us it had plans for how it would spend the remainder in the coming years.141
- 2014 - 2015: Between July 2014 and June 2015, GiveWell-influenced donors gave slightly over $1 million to Deworm the World.142 In 2015, Deworm the World spent $743,975 in unrestricted funding.143 In late 2015, it told us that $1.1 million of its unrestricted funding on hand was committed to future projects.144 We estimate that this means that by the end of 2015, Deworm the World had spent or committed approximately 46% of the total unrestricted funding it had raised from late 2013 through June 2015.145
- 2015 - 2016: Between July 2015 and June 2016, GiveWell-influenced donors gave approximately $12 million to Deworm the World.146 As of June 2016, Deworm the World expected that by the end of the year it would have spent about $1.2 million of unrestricted funding in 2016.147 As of July 2016, Deworm the World had committed $5.1 million of its unrestricted funding on hand for future projects.148
Activities funded vs. expectations
Because Deworm the World has spent its funding significantly more slowly than expected, it can be difficult to match up how its spending relates to its past plans. In general, we do not believe Deworm the World has closely followed the plans for spending it has shared with us; however, we believe the nature of Deworm the World's work can require significant shifts in planned spending. We believe that Deworm the World's spending choices have been reasonable.
- 2013 expectations vs. 2014 actual: We detailed Deworm the World's use (and planned use) of the funding it raised in 2013-2014 in this November 2014 blog post. Overall, Deworm the World's funding decisions were roughly in line with our previous expectations.149
- 2014 expectations vs. 2015 actual: At the end of 2014, Deworm the World told us that it would spend additional unrestricted funding on new staff, central costs, expansion to Vietnam, and evaluation of new evidence-based programs related to deworming (details in footnote).150 Including commitments, our impression is that Deworm the World roughly followed this plan.151 It spent less on Vietnam, Kenya, and evaluations, and more on central costs, than expected (details in footnote).152
- 2015 expectations vs. 2016 budget: At the end of 2015, Deworm the World told us that it intended to spend additional unrestricted funding primarily on multi-year deworming programs in Pakistan, Vietnam, and one other new country and on reserves.153 As of late 2016, Deworm the World had committed more than expected to a multi-year program in Nigeria and less than expected to Vietnam, and it had not yet committed to any multi-year programs in Pakistan or another new country.154 We believe this is primarily because Deworm the World's progress has been somewhat slower than expected, not because Deworm the World has significantly changed its plans.155
Considerations relevant to assessing Deworm the World’s room for more funding
Uncertainty on accuracy of financial information
Deworm the World's parent organization, Evidence Action, was created in 2013 to take over two programs incubated at Innovations for Poverty Action (IPA): Deworm the World and Dispensers for Safe Water.156 Our understanding is that there were and still are challenges coordinating grant management across the two organizations. As of October 2016, Evidence Action was in the process of overhauling its financial system.157
In 2016, we spent more time than we had previously on trying to understand Evidence Action's financials, and we found several errors (details in footnote).158 Once these mistakes had been corrected, Deworm the World's records indicated that it held approximately $4.6 million more in available unrestricted funding than the original financial documents had shown.159
While we are more confident in the updated numbers, we are not fully confident that all errors have been corrected.
Alternative funders for Kenya
Deworm the World is seeking funding to support the next five years of a national deworming program in Kenya. Deworm the World currently has a five-year grant from CIFF (and a smaller amount from the END Fund) to support the program; the CIFF grant is ending in June 2017.160 It is unclear to us whether CIFF will continue providing funding for the program and, if so, for how long. Due to this uncertainty, we are not sure if GiveWell-influenced funding is needed to support the Kenya program. However, Deworm the World has told us that the government of Kenya has requested its support in fundraising for the next five years of the program.161
Deworm the World has also told us that the END Fund and other donors are interested in funding a portion of the program. Deworm the World expects these funders could provide about half of the funding for the program. We have some uncertainty about whether 50% is the right division because (a) the END Fund told us that it was interested in providing $1.5 million for the first year of the program (out of $2.4 million), but it might not have enough funding to do so,162 and (b) based on private conversations with donors, we think that the support of some of the "other donors" may hinge on GiveWell's recommendation and therefore should possibly be included in our estimate of room for more funding.
Due to the possibility that Deworm the World's unrestricted funding may displace funding from CIFF, and, to a lesser extent, the END Fund and other donors, we consider the opportunity to fund the Kenya program to be less cost-effective in expectation than it would be if we were confident in the size of the gap. We have included the first two years of the program as an execution level 1 funding gap because our understanding is that Deworm the World will prioritize funding this program over most other opportunities.
Multi-year commitments
Deworm the World has told us that it prefers to have enough funding to make 3-5 year commitments when attempting to launch a new program. It told us that governments typically ask for multi-year commitments because a) deworming programs must be sustained over time to cause the desired impacts and b) governments want assurance that support will be sustained so that they can better plan how to use their funding.163 As a result, Deworm the World expects that having multiple years of allocated funding makes partnerships with governments (formalized by signed memoranda of understanding (MOUs)) more likely and reduces the amount of time before programs are launched.164
Additionally, Deworm the World has told us that having multiple years of funding for its programs allows staff to spend less of their time fundraising and more time on other aspects of their work.
We are not sure that 3-5 year commitments are necessary, but we do not feel that Deworm the World’s preference is unreasonable.
Challenges of expanding to new countries
Launching programs in new countries may introduce challenges that are hard to predict, such as differences in cultural and bureaucratic expectations, increased difficulty of predicting program costs and success, and political and economic instability. So far, we believe that Deworm the World has done reasonably well supporting programs in new geographies, although, because these programs are new, we have only seen limited data from them. We believe Pakistan may be a more challenging and costly environment to work in than the other countries Deworm the World has previously entered.
Deworm the World Initiative and Evidence Action
Deworm the World Initiative is led by Evidence Action. Evidence Action supports other programs in addition to Deworm the World.165
This has some implications relevant to Deworm the World’s room for more funding: donations to Evidence Action, even if restricted to Deworm the World, might change the actions that staff take to fundraise (i.e., which grants they pursue, what type of funding they ask for). We've seen some evidence that this has been the case in the past and provide details on our page with additional information about Deworm the World..
Evidence Action also shared with us a rough estimate of how it would allocate $1.8 million in unrestricted funding that it expects to have in 2017: $600,000 is expected to be allocated to Deworm the World while $500,000 is expected to go to Dispensers for Safe Water and $100,00 to Evidence Action Beta (the rest is for organizational development).166 Deworm the World also notes that many of Evidence Action's investments in general organizational development have benefited Deworm the World, as well as Evidence Action's other programs, substantially.167
Global need for treatment
There appears to be a substantial unmet need for STH and schistosomiasis treatment globally.
In 2016, the WHO released a report on 2015 treatments stating that:168
- 63% of school-age children in need of treatment were treated for STH in 2015. This is a large increase over WHO's report for 2014, which reported 45% coverage.169
- 42% of school-age children in need of treatment were treated for schistosomiasis in 2015.
We have not vetted this data.
Deworm the World as an organization
We believe that the Deworm the World Initiative, led by Evidence Action, is a strong organization:
- Track record: Deworm the World has a track record of assisting governments with deworming programs.
- Self-evaluation: Deworm the World collects a large amount of relevant data about its programs, demonstrating a commitment to self-evaluation.
- Communication: Deworm the World has generally communicated clearly and directly with us, given thoughtful answers to our critical questions, and shared significant, substantive information.
- Transparency: Deworm the World is very transparent.
More on how we think about evaluating organizations at our 2012 blog post.
Sources
- 1
- GiveWell's non-verbatim summary of a conversation with Alix Zwane and Karen Levy on May 14, 2013
- Alix Zwane conversation June 4th 2013
- Grace Hollister conversation June 19th 2013
- Alix Zwane conversation August 30th 2013
- Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013
- GiveWell’s non-verbatim summary of a conversation with Alix Zwane on December 20th, 2013
- GiveWell’s non-verbatim summary of a conversation with Alix Zwane on February 18th, 2014
- GiveWell’s non-verbatim summary of conversations with Alix Zwane and Grace Hollister on February 26 and March 17, 2014
- GiveWell’s non-verbatim summary of a conversation with Grace Hollister on June 24th, 2014
- GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014
- GiveWell's non-verbatim summary of a conversation with Alix Zwane and Jessica Harrison on November 4th, 2014
- GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015
- GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015
- GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015
- GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015
- In 2016, we deprioritized publishing notes from our conversations with Deworm the World.
- 2
- 3
This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.
- 4
- "More specifically, Evidence Action advocates for school-based deworming to policymakers and provides technical assistance to launch, strengthen and sustain school-based deworming programs." Evidence Action website, Deworm the World Initiative (March 2016)
- An example of Deworm the World supporting a program with funding: "Deworm the World is working in partnership with Thrive Networks in Vietnam on an integrated program of both deworming and water, sanitation, and hygiene (WASH) education, and this includes an RCT to explore the impact of hygiene education in combination with deworming. This program is funded primarily by Dubai Cares, with Deworm the World slated to provide unrestricted funding for certain program components." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 5.
- 5
- "Is mass treatment justified? On cost-effectiveness grounds we believe that it clearly is, as the cost of treatment is cheaper than individual screening. The WHO states that the cost of screening is four to ten times that of the treatment itself. Because the drugs are very safe and has [sic] no side effects for the uninfected, the WHO does not recommend individual screening. The WHO instead recommends mass drug administration in areas where more than 20% of children are infected." Evidence Action website, Deworm the World Initiative (March 2016)
- Deworm the World focuses on school-based programs because the highest burdens for STH and schistosomiasis (the two diseases that Deworm the World targets) tend to be observed in children. For this reason, a significant decrease in the worm burden in children for these diseases translates to a significant decrease in the burden across an entire community. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- 6
- "Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis. Deworm the World is one of the only organizations focused on STH (while there are other programs that focus on schistosomiasis), and there is significant room to scale-up [sic] STH programs. In some ways, schistosomiasis has an even larger gap to fill than STH. If Deworm the World shifted its focus to include schistosomiasis, it might widen the existing STH gap. Additionally, Deworm the World specializes in school-based deworming, which is not the ideal approach in every situation." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5.
- This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.
- 7
"Overall, Deworm the World plans to continue to focus on STH rather than, e.g., schistosomiasis, though it will continue to support schistosomiasis treatment in those places where it overlaps with STH, and to coordinate with the Global Schistosomiasis Alliance to adopt complementary strategies. There are many places that need treatment for STH but not schistosomiasis." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5.
- 8
- "We work with governments around the world to eliminate the public health threat of worms through scaling up school-based mass deworming programs." Evidence Action website, Deworm the World Initiative (December 2016)
- For example, Deworm the World is partnering, or planning to partner, with local organizations in Pakistan, Vietnam, and Nigeria:
- Nigeria: "Deworm the World is in discussions with a potential partner which plans to work in Cross River on other integrated NTD treatment. The need to scale up treatment for schistosomiasis and STH among school-age children has not yet been addressed, and Deworm the World has been in discussions with this partner, the state NTD coordinator, and other state officials about creating a school-based deworming program to treat both STH and schistosomiasis beginning in 2016." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, pg. 6.
- Vietnam: "[In Vietnam] Deworm the World’s partnership with both the government and Thrive Networks is a new working model for Deworm the World; elsewhere, it has supported government implementation or run its program independently. Deworm the World has only one staff member in the country. Dubai Cares provides most of the program funding. All funding goes to Thrive Networks, which provides money to the government for implementation expenses." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015, Pg 3.
- Pakistan: "[In Pakistan, Deworm the World] plans to contract with a local organization to do these [prevalence] surveys, but does not yet have a signed agreement...Deworm the World plans to work in partnership with the same local organization to provide technical support, likely beginning in Punjab and later expanding to Sindh." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.
- 9
For example, Deworm the World has considered supporting evaluations or monitoring of different deworming-related programs:
- "If funding permits, the Kenyan government may begin a lymphatic filariasis treatment program along its coast. If it does so, Deworm the World will provide process monitoring and coverage validation for the treatments." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 3.
- "We are contributing approx $111k to the TUMIKIA and TakeUp studies, complementary studies leveraging the Kenya program to look at the potential for breaking STH transmission." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- Deworm the World has told us that it is also interested in opportunistically evaluating new evidence-based programs that may efficiently complement deworming. GiveWell’s non-verbatim summary of a conversation with Alix Zwane on October 23rd, 2014
- 10
"In January 2007, the [Young Global Leaders] launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs." Harvard Business School Kenya Case Study A 2010, Pg 7.
- 11
- Deworm the World has supported deworming activities in India since 2009, in Kenya since 2012, and in Ethiopia since 2014:
- "[Where We Work, Bihar State, India]: 1st deworming round in 2011 reached 17 million children." Evidence Action website, Deworm the World Initiative (March 2016)
- "[Where We Work, Kenya] With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively." Evidence Action website, Deworm the World Initiative (March 2016)
- Deworm the World supported a pilot deworming program in Ethiopia in April 2015 and another deworming program in October and November of 2015. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
- Deworm the World supported treatments in Cross River, Nigeria in mid 2016: "This week, the Cross River State Ministry of Health’s Neglected Tropical Diseases (NTD) unit launched its inaugural statewide school-based deworming exercise that will treat against two neglected tropical diseases that are particularly common in children: schistosomiasis and soil-transmitted helminthiasis (STH). The school-based deworming exercise will cover 11 of the 18 local government areas in Cross River for the first time, and is targeting 600,000 at-risk school-aged children in primary and junior secondary public and private schools. Other NTDs endemic to the state (lymphatic filariasis and onchocerciasis) will be treated through a community-based approach, according to standard practice." Evidence Action, blog post, June 30, 2016
- Deworm the World supported treatments in Vietnam in mid 2016: "On April 28 and 29, more than 700,000 primary school children across four provinces in northern Vietnam will line up in their classrooms to receive a deworming tablet. 8.5 million children in Vietnam are at risk of parasitic worm infections that can harm their health, development, and school participation. Evidence Action’s Deworm the World Initiative supports the Government of Vietnam as it strengthens and improves school-based deworming to keep children healthy and in school. Between now and 2018, the program will distribute more than four million treatments to combat worms in the four provinces." Evidence Action, blog post, April 27, 2016
- Updates on activities in India, Kenya, and Ethiopia were discussed in various conversations; for example, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell’s non-verbatim summary of a conversation with Grace Hollister on February 24, 2015
- Plans for Nigeria, Pakistan, and Vietnam were discussed here: GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015
- Progress in Pakistan was delayed after Kato Katz kits for the planned prevalence survey did not arrive in time. The prevalence survey was rescheduled to begin in August 2016. Deworm the World is waiting on the results of the upcoming prevalence survey to move forward with a treatment strategy. Grace Hollister, conversation with GiveWell, August 11, 2016
- Deworm the World has supported deworming activities in India since 2009, in Kenya since 2012, and in Ethiopia since 2014:
- 12
See the descriptions of programs by country below.
- 13
"The government is fully responsible for program implementation, and … these programs leverage thousands of govt personnel from health and education to be able to run." Grace Hollister, email to GiveWell, June 9, 2016
- 14
"Note that there are not single deworming days in all countries, nor do all countries have a mop-up day. In all cases there are mass campaigns, but the structure varies by country." Grace Hollister, email to GiveWell, June 9, 2016
- 15
- This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. See our review process.
- For example, Deworm the World has assisted in a number of areas in India:
- "Andhra Pradesh...
Deworm the World’s contributions- Prevalence survey...
- Operational support
- Helped government develop operational plans and budgets
- Coordinated cross-sectoral partners through the establishment of a State School Health Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS)
- Coordinated drug donation made by Feed the Children
- Designed a monitoring and evaluation (M&E) system
- Created government tableau for community awareness
- Trainings
- Conducted a master training session for program
- Designed training cascade for the master trainees to train the rest of the implementers
- Designed training materials
- Developed materials and campaigns for community sensitization
- Bihar…
Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andhra Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO… - Delhi…
In addition to the standard contributions (see Andhra Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children. - Rajasthan…
DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO."
@Grace Hollister Conversation June 19th 2013@, Pg 1-4.
- "Andhra Pradesh...
- 16
We have matched our descriptions to Deworm the World's standard categorization, albeit in a slightly different order.
- 17
- "As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program."
@Alix Zwane Conversation June 4th 2013@, Pg 2. - "Deworm the World does not yet have an agreement with the government in Pakistan to conduct the surveys, but hopes to accomplish this in the next month, and anticipates that the prevalence surveys will be conducted beginning in January or February of 2016. It is expected that a clear articulation of need will be an important factor in building a strong case to the government in favor of deworming programs, and it may be best to wait until the results of the surveys are available in the second quarter of 2016 before beginning discussions with the government on a scaled school-based program. Treatment may not begin until 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.
- Deworm the World's advocacy often occurs side-by-side with Deworm the World's technical assistance; once Deworm the World proves that a deworming program can be well-executed, it is easier to interest national governments in funding deworming programs. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
- "As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program."
- 18
For example:
- "DtW has been involved in deworming programs in four different states, and is still actively involved of [sic] three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement...In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. [Note: these notes contain additional examples of Deworm the World's advocacy for other states in India]
- "Together, we suggested to the Federal Ministry of Health that they expand this initial work. What would it take to treat at least 75% of all at-risk school-age children in the country and to launch a truly national program?
SCI helped Oumer Shafi, the committed and action-oriented Coordinator for Neglected Tropical Diseases in the Federal Ministry of Health, develop a detailed action plan. This entailed sophisticated statistical analysis to determine how many deworming sites would be required to reach at least 80% of kids at risk.
Meanwhile, I worked closely with Birhan Mengistu, an up-and-coming leader seconded from the World Health Organization, and with other Ministry of Health staff. We sat for hours hunched over laptop screens to develop detailed five-year budgets, talking through row after row of spreadsheets and reviewing everything from the cost of fuel for drug transport to the needs of teachers.
Together with the Federal Ministry of Health, we were able to think and act boldly. We are excited to continue to partner with SCI and are seeking other partners who also share common goals and values to rapidly scale school-based deworming in endemic countries.
...When we floated the idea of vastly increasing the scope of the originally proposed deworming rounds to be a truly national plan treating upwards of 75% of all children at-risk, Shafi didn’t flinch."
Evidence Action, blog post, June 12, 2015 - Other advocacy activities can include discussing: "how deworming can fit into the current policy environment and policy priorities of a government, how such a program can/should be financed, the robust evidence of impact, how a country can best take advantage of WHO drug donations, encourage program champions within government, help establish program governance structures. Once a program is established, advocacy doesn’t end – we work with govts to ensure the continuation of the above. Typically we refer to this group of activities as policy and advocacy, because there is a heavy emphasis on the former." Grace Hollister, email to GiveWell, June 9, 2016
- These discussions also provide opportunities for Deworm the World to assess how well a deworming program with the government might run. If Deworm the World discovered from its advocacy discussions that there were high rates of teacher or student absenteeism, then it might conclude that a school-based deworming program may not work in the country. Deworm the World assesses risks like this through a diagnostic survey of the country’s capacity, including school attendance rates, which must be sufficiently high if a school-based deworming program is to succeed. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- 19
Deworm the World builds strong working relationships with governments to try to ensure that its programs will be effective, and it will not commit to a program if it does not foresee success in that country. It can decide to abandon plans for a program before a memorandum of understanding (MoU) is signed. In one state in India, Jharkhand, Deworm the World explored a program, but did not build a strong working relationship with the government, so Deworm the World pulled out of discussions before discussing an MoU or investing much money. The discussion stage with governments is important for helping Deworm the World assess the government's position and viability as a partner. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- 20
"The global STH community has changed significantly in the last couple of years, especially due to the formation of the STH Coalition. The community is now prioritizing STH (in a way similar to how LF became prioritized with the formation of the Global Alliance to Eliminate Lymphatic Filariasis, which has seen significant success).
The STH community is developing plans to scale-up [sic] treatment, especially in high burden countries such as:- Ethiopia
- Nigeria
- India
- Pakistan
- The Democratic Republic of the Congo
- Indonesia
- The Philippines
- Tanzania
As part of the STH Coalition, Evidence Action is chairing a working group on school-age children. It has used some of its unrestricted funding to hire consultants to create ‘snapshots’ of each country, including obstacles, gaps, potential strategies, and financial needs.
Deworm the World expects to see an increase in partnerships between the various groups in the STH community. Deworm the World hopes to leverage partnerships with existing organizations in, e.g., Nigeria, Pakistan, Ethiopia, etc., to provide catalytic support (rather than opening its own offices in those places)." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 6.
- 21
- "We work with epidemiologists and local partners to assess worm prevalence and intensity, obtaining data to develop a targeted treatment strategy and to determine parasitological impact once programs are in place." Evidence Action website, Deworm the World Initiative (March 2016)
- A few examples of Deworm the World assisting with prevalence surveys include:
- "Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children." Grace Hollister conversation June 19th 2013, Pg 1.
- "DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged [sic] children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment." Grace Hollister conversation June 19th 2013, Pg 2.
- "In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city." Grace Hollister conversation June 19th 2013, Pg 3.
- "DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year." Grace Hollister conversation June 19th 2013, Pg 4.
- 22
- For example, Deworm the World is currently supporting prevalence surveys in Pakistan because Pakistan has not yet been "mapped" (i.e., prevalence surveys have not yet been conducted in Pakistan), so nobody knows how heavy the worm burden is in Pakistan or where deworming efforts should be focused. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
- "Deworm the World has estimated very roughly that its three-year program in Nepal would cost about $6 million ($2 million per year). This is based on a cost per child of no more than $0.50 and a target population of about 6 million children. However, the latter estimate is based on the outdated prevalence survey data mentioned above. Deworm the World will need to conduct a new survey to determine an exact target population and a more accurate budget." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pgs 1-2.
- "Deworm the World will likely wait to expand its Vietnam activities until further mapping and impact evaluation have been completed." GiveWell’s non-verbatim summary of a conversation with Grace Hollister and Alix Zwane on March 30, 2015, Pg 7.
- 23
- Deworm the World originally planned to do prevalence surveys every few years but may do them less frequently going forward.
- "DtWI would like to do prevalence surveys after every 3 years or so. Ideally, prevalence surveys would be carried out after every third round of treatment immediately prior to the following round." GiveWell's notes from site visit to India, October 2013
- "Note: this [follow-up prevalence survey] strategy is evolving; WHO recommendations are to conduct sentinel site surveys after 5-6 rounds of treatment, and we are moving in that direction. Key is how a new survey would impact the treatment strategy" Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- Key M&E activities in India include "post-round 3 impact measurement" prevalence surveys. DtWI NDD Year 1 M&E review July 2015, Pg 3.
- Deworm the World has said that a Rajasthan follow-up prevalence survey is tentatively planned for late 2017. Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- Grace Hollister, edits to GiveWell's review, November 7, 2016
- 24
Deworm the World told us that it works "with partners with expertise in STH parasitiology and epidemiology." Grace Hollister, edits to GiveWell's review, November 7, 2016. Examples of Deworm the World working with partners on prevalence surveys:
- "The WHO reports that Pakistan is endemic for STH, but there is not yet sufficient evidence of prevalence and intensity to develop an evidence-based treatment strategy. Deworm the World has committed unrestricted funding to fund prevalence surveys in two large provinces, Punjab and Sindh. It is targeting these provinces because their school enrollment rates are high, the areas are fairly secure, and they contain a significant percentage of the population of Pakistan. Deworm the World plans to contract with a local organization to do these surveys, but does not yet have a signed agreement." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7.
- "Following three rounds of school-based deworming, Evidence Action - Deworm the World Initiative recommended a second prevalence survey to the Bihar government, in order to understand the effect of deworming in Bihar on STH infection levels. With approvals from the State Government, in January and February 2015, Evidence Action - Deworm the World Initiative, conducted an STH prevalence survey among school-age children in government primary schools in Bihar. The survey took place in 65 schools in 14 districts, covering all three agro-climatic zones in the state. The National Institute of Epidemiology – Chennai (NIE) designed the survey, and analyzed the dataset to produce epidemiological findings. Field teams hired through GfK Mode (an agency with prior experience in sample collection for STH prevalence surveys), visited the households of children in the selected schools to collect stool samples and information related to school, household, deworming, and sanitation, to better understand infection patterns and allow for sample weighting. The 2 Post Graduate Institute of Medical Education and Research – Chandigarh (PGIMER) analyzed stool samples in field laboratories, which were set up in district and block health facilities, using the WHO recommended Kato-Katz method." DtWI Bihar 2015 Prevalence Survey report, Pgs 1-2.
- 25
- "We work closely with the Ministries of Education and Health to design a program with joint ownership, develop operational plans and budgets, coordinate logistics, and provide on-the-ground support to ensure a high quality outcome." Evidence Action website, Deworm the World Initiative (March 2016)
- Note that we do not feel like we have a strong understanding of Deworm the World's activities in this area; for example, we have not asked Deworm the World what it has brought to the planning, budgeting, or logistics processes that would not have otherwise been included. We do not have a strong sense from Deworm the World's website about what these activities involve (e.g., we do not know what it means for Deworm the World to have "coordinated logistics").
- 26
- "We help governments evaluate appropriate drug treatment strategies and dosage, support drug procurement including through global pharmaceutical donation programs, and design robust serious adverse event protocols and drug tracking systems." Evidence Action website, Deworm the World Initiative (December 2016)
- Grace Hollister, edits to GiveWell's review, November 7, 2016
- 27
- "DtWI provided support to the state government in submitting the drug requisition to WHO in March 2013, as well as in shipping, custom clearances and transportation upon arrival in India." DtWI Rajasthan 2013 program report, Pg 6.
- Deworm the World told us that before it started conversations with the Indian government, the government was not aware that it could obtain albendazole for free from the World Health Organization. Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
- 28
- "We consistently design and support training through an efficient multi-tier cascade approach that is tailored to the local context, ensuring knowledge reaches from the national level all the way to the teachers responsible for administering deworming medication." Evidence Action website, Deworm the World Initiative (March 2016)
- GiveWell's understanding is that training cascades involve a series of trainings that start at high levels in the government, and proceed in a step-wise fashion down to local levels, where teachers are eventually trained. At each training, materials (such as deworming drugs and posters that notify the community about deworming day) are passed down from the staff member leading the training to the staff members attending the training, until materials eventually reach teachers.
- A description of a training and distribution cascade: "Transportation of tablets to all districts was managed and supported by DtWI, in coordination with SHS and DHFW to the district level dispensary, from where they were collected by the respective teachers for their school. For anganwadis, the syrups were sent to the nodal officer who arranged further transportation to the supervisors who in turn handed over to the anganwadis. Training about health education on types of STH, need for deworming, transmission of worms through open defecation and other practices and how to safely administer deworming drug was conducted through a cascade model. In the first step of the cascade, training of trainers was conducted at the state level where a total of 1040 participants attended (390 WCD + 650 education department) in 60 sessions. These trainers further trained 3032 headmasters and 3032 teachers and 10,500anganwadi workers in groups of 30 participants. The training on deworming was integrated with training for the WIFS program to effectively utilize time of participants and trainers and reduce training costs. A simplified training manual was developed that included content on deworming and WIFS into a single document. At the state level training sessions, training videos on three types of soil transmitted [sic] helminths (ascaris, trichuris and hookworm) and worm infestation cycle were also used. DtWI hired district coordinators (DCs) to provide short term support to the deworming program. They played a key role in ensuring that drugs in adequate quantities were available with the district-level dispensaries and nodal offices for further distribution. The DCs collated information on shortfall or surplus of drugs at district level and shared information with the DtWI state team, who coordinated with SHS to ensured-distribution [sic] or fresh supplies to districts facing drug deficits." DtWI Delhi 2013 program report, Pg 7.
- See Pgs. 12-13 of DtWI Delhi 2015 program report for a visual representation and description of a recent training and distribution cascade.
- "We consistently design and support training through an efficient multi-tier cascade approach that is tailored to the local context, ensuring knowledge reaches from the national level all the way to the teachers responsible for administering deworming medication." Evidence Action website, Deworm the World Initiative (March 2016)
- 29
- For example, a description of the training cascade in Kenya: "The National School-Based Deworming Programme uses a cascade implementation model that efficiently and cost-effectively delivers training materials, deworming tablets, monitoring forms, funds, trainings other programme materials and resources from the national level to schools. At the national level, the Programme trains a team of MoEST and MoH officials as master trainers, requisitions deworming tablets through the MoH, and develops treatment and implementation strategies, training materials and monitoring tools. Thereafter, an initial planning meeting is held with county and sub-county leadership. This meeting is followed by two levels of trainings on how to successfully implement the Deworming Programme: Sub-County Training and Teacher Trainings. These trainings prepare sub-county and division officials to plan subsequent programme activities within the cascade, distribution of materials, planning of deworming and community mobilization and sensitization. After these trainings and community mobilization, the critical day of implementation occurs – Deworming Day – where teachers administer deworming tablets to millions of children in over 11,000 schools across Kenya and fill in monitoring forms to capture treatment data. These forms and any unused deworming tablets are moved up through a “Reverse Cascade” as described below. The cascade model helps to manage the national scale of the NSBDP, and therefore, builds capacity for successful implementation at various levels. Additionally, the cascade brings together MoEST and MoH personnel through collaborative leadership responsibilities for the planning, implementation and monitoring of programme activities at all levels. The cascade is outlined in the infographic below." DtWI Kenya 2013-2014 program report, Pgs 4-5.
- "Drug distribution: As per NDD operational guidelines, and established best practice, drug distribution was integrated with the training cascade (as detailed in the training section below), whereby NDD kits were provided to health functionaries at the district level trainings for onward distribution. The kits included drugs, IEC materials, and reporting forms." DtWI Madhya Pradesh 2015 program report, Pg 12.
- 30
For example:
- "DtWI provided technical expertise across all program components, and served as the primary coordinating body among implementing agencies. DtWI facilitated drug donations from WHO, provided professional master trainers for training of trainers, developed training materials such as flipcharts, modified training booklet and reporting forms, designed and carried out independent monitoring, and developed adverse event protocols." DtWI Rajasthan 2013 program report, Pg 5.
- "51 district coordinators were hired to support on-the-ground program coordination for a three month period around the Deworming round. District coordinators were instrumental in ensuring that IEC and training materials printed by Evidence Action were handed over to district medical officers one week prior to NDD. This was a time-bound activity with tight timelines, but was critical to the program implementation. District coordinators ensured timely delivery of training materials, and further distribution of NDD kits at the trainings for all functionaries at school and anganwadi levels. They participated in trainings at district and block levels and escalated any observed gaps to regional coordinators and the state team for appropriate follow-up at the state level." DtWI Madhya Pradesh 2015 program report, Pg 11.
- "We hire master trainers, or train govt staff to be master trainers." Grace Hollister, email to GiveWell, June 9, 2016
- 31
- Bihar 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers made approximately 19,567 successful calls[20] made during the period of January to March 2015. These calls were made to 534 blocks across 38 districts to assess preparedness on all program areas. Daily tracking sheets outlining issues arising at districts, blocks, and schools were identified during the process and were shared with the state to assist the government to take real-time corrective action." @DtWI Bihar 2015 program report@, Pg 16.
- Rajasthan 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Evidence Action’s tele-callers tracked the status of training sessions and availability of drugs and IEC materials at the district, block, and school/anganwadi levels through approximately 14,485 successful[19] calls. Tele-callers made 258 calls to the Department of Health and 7,717 calls to ICDS at district, project, and sector level. Another 4,598 calls were made to block and district-level education officials to track various program components. In total 734 calls were made to schools covering 249 blocks across the 33 districts to assess preparedness. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the districts, blocks, and schools/anganwadi levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Rajasthan 2015 program report, Pg 15.
- Delhi 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers tracked the status of training, drugs, and IEC material availability at the district, and school/anganwadi through phone calls. Approximately 8,504 successful[12] calls were made to the education, health, and WCD departments during this period." DtWI Delhi 2015 program report, Pg 15.
- Madhya Pradesh 2015: "Telephone Monitoring and Cross Verification for Process Monitoring: Our tele-callers placed phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful[13] calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17.
- 32
"We work with governments and communications experts to design locally appropriate awareness campaigns to communicate messages through a wide variety of channels to increase public acceptance and effectiveness of deworming programs." Evidence Action website, Deworm the World Initiative (March 2016)
- 33
For example: "As part of their training, school headmasters/teachers were instructed to share information on the deworming program in the morning prayer sessions at their respective schools on a daily basis from October 6, 2013 onwards. They were also advised to convene school management committee meetings to communicate about the benefits of deworming and the schedule of deworming program. School headmasters were also advised to carry out student rallies / processions (prabhat pheri) to create awareness in the communities." DtWI Rajasthan 2013 program report, Pgs 8-9.
- 34
- "One other key strategy adopted by DtWI to spread awareness was through text (SMS) reminders over mobile phones to school teachers, headmasters, Child Development Project Officers (CDPOs) and lady supervisors as a reminder about deworming day. SMSs were also used to reinforce precautions on drug administration, such as not giving drugs on an empty stomach, but only after midday meals and not giving drugs to sick children. In all, about 80,000 text messages were sent to school teachers and headmasters three times – a total of 2,40,000 [sic] messages. These messages were sent a day before deworming day, on mop-up day and after mop-up day. About 1400 such messages were sent twice to lady supervisors and CDPOs on a day before deworming day and on mop-up day. Similarly, five rounds of around 2400 text messages were sent to block level officials to expedite coverage reporting. This was an example of ensuring last-mile communication at low cost of about 12 paisa per message (or roughly 1/5th of a cent)." DtWI Rajasthan 2013 program report, Pg 9.
- "Additionally, mike announcements were made at public places in blocks and district headquarters by Evidence Action for 5 days, closer to deworming day (Annexure E.4)." DtWI Bihar 2015 Program report, Pg 13.
- "The State Health Society Bihar and Evidence Action rolled out a media mix to generate community awareness and increase program visibility to improve coverage in the state (Annexure E.1). We supported the adaptation and contextualization of prototypes from the National Deworming Day IEC resource toolkit. At the state level, State Health Society Bihar, in coordination with the Department of Public Relation, Government of Bihar, published newspaper advertisement in four dailies one day prior to deworming and mop up [sic] day, i.e., on 20 and 25 February (Annexure E.2). Radio jingles, customized into three local dialects, were aired from 15 to 26 February on the All India Radio to maximize outreach to the community. For additional visibility of the program at the community level, State Health Society Bihar printed 513,625 posters (7 for each school, including distribution in the local community), 1068 banners for Primary Health Centers, hoardings at 38 district headquarter [sic]. All of these were adapted and contextualized by Evidence Action." DtWI Bihar 2015 Program report, Pg 12.
- Other community awareness activities include creating posters to display at schools or advertising the deworming day in the newspaper.
- "Activities designed to enhance community awareness on deworming were rolled out to improve overall program coverage. The awareness activities included newspaper advertisements a day prior to the deworming day; a 60-second radio jingle aired on 3 FM channels from April 7 to 15 by School Health Scheme, and banners displayed at schools. Evidence Action was part of the committee formed by the state government for contextualization of the radio jingle. Evidence Action extended support to the state in contextualizing IEC materials from the National Deworming Day guidelines. The Directorate of Family Welfare also independently developed and printed handbills for the distribution at anganwadis to mobilize people on deworming day. The School Health Scheme provided banners to the schools, the distribution of which was integrated in trainings for teachers. The Delhi state government also used an e-portal to disseminate key information, including dates for deworming and mop up [sic] days, benefits of deworming, and details of the launch event." DtWI Delhi 2015 program report, Pgs 11-12.
- 35
"We help governments design monitoring systems to measure effectiveness in achieving intended program results. We also conduct independent monitoring to validate program results, and evaluate the impact of programs in reducing worm prevalence and intensity." Evidence Action website, Deworm the World Initiative (March 2016)
- 36
Note that Deworm the World hires monitors for the first and third type of monitoring data collected, but that the second is collected entirely by government staff: "Coverage reporting is done by the government- we sometimes assist in the data analysis, designing reporting forms, and ensuring that the 'reverse cascade' is appropriately designed." Grace Hollister, email to GiveWell, June 9, 2016
- 37
"Process monitoring assesses the preparedness of the schools, anganwadis, and health systems to implement mass deworming and the extent to which they have followed correct processes to ensure a high quality deworming program." DtWI Madhya Pradesh 2015 program report, Pg 16.
- 38
- In India, on Deworming Day and Mop-Up Day, Deworm the World commissions independent monitors who go to schools to gather data on whether principals and teachers are prepared for Deworming Day, the availability of drugs and supplementary materials, whether students are being dewormed, whether proper procedures are being followed, and more. For example, see Deworming Day monitoring data from Rajasthan in 2015: DtWI Rajasthan 2015 monitoring survey from deworming day, schools (shows which questions were asked) and DtWI Rajasthan 2015 monitoring data from deworming day, schools (shows the survey responses).
- Kenya:
- Trainings monitoring: "PMCV [Process Monitoring and Coverage Validation] officers observed 36 CHEW [Community Health Extension Worker] Forums aimed at introducing the deworming sensitization message and materials/methods as well as asigning [sic] CHEWs to schools for monitoring. A successful community health extension worker forum is one that starts on time and where all the materials were present. Overall, 63% of participants arrived before training, whereas 22% arrived 1hr after the forum had begun and 15% of participants arrived more than 1hr after the forum’s commencement. Lateness appears to be a commonality to all training sessions. Materials required for CHEW training include a powerpoint printout, CHEW checklist and Severe Adverse Event (SAE; side-effects of the drugs) protocol.
In 51% of forums, ALL of the Materials Pack was distributed at the start of the forum. In 13% of forums, SOME of the Materials Pack was distributed at the start of the forum. In 36% of forums, NONE of the Materials Pack was distributed at the start of the forum." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 9. - Community sensitization monitoring: "A number of parents were also interview [sic] at schools on Deworming Day regarding their knowledge of deworming and the source of that knowledge. The intention behind this exercise was to compare the information source to those interviewed prior to deworming as a measure of consistency. In Figure 4, the results of the interviews pre-deworming day are compared with those parents interviewed on deworming day. The results remain largely similar, however more parents reported getting their information from ‘other’ sources (51%) when interviewed on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 14.
- School preparedness monitoring: "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.
- Deworming Day and Mop-Up Day monitoring: "PMCV field officers visit schools on Deworming Day to observe procedure and interview teachers/head teachers regarding deworming. The number of schools observed on Deworming Day treating for STH in Year 3 was 247. The combined population of registered children at the observed schools was 88,820 children. It is estimated that 7,485 children were directly observed being treated for STH. Seven schools treating for both STH and SCH were observed. The total registered population of children in these schools was 3,198 children and 352 children were directly observed by field officers participating in Deworming Day. A quality Deworming Day is regarded to be one where:
- Deworming occurs within 1 week before [sic] teacher training
- The school would have the correct materials (including sufficient drugs) in place before commencement
- Children of the appropriate ages are treated (ages 2-14 years)
- Non-enrolled and ECD aged children are prioritized for treatment within the schools
- The correct dosage of drugs is given to all children"
Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16.
- Trainings monitoring: "PMCV [Process Monitoring and Coverage Validation] officers observed 36 CHEW [Community Health Extension Worker] Forums aimed at introducing the deworming sensitization message and materials/methods as well as asigning [sic] CHEWs to schools for monitoring. A successful community health extension worker forum is one that starts on time and where all the materials were present. Overall, 63% of participants arrived before training, whereas 22% arrived 1hr after the forum had begun and 15% of participants arrived more than 1hr after the forum’s commencement. Lateness appears to be a commonality to all training sessions. Materials required for CHEW training include a powerpoint printout, CHEW checklist and Severe Adverse Event (SAE; side-effects of the drugs) protocol.
- 39
This understanding is from many conversations with Deworm the World and following Deworm the World's progress over time. For example, see DtWI Delhi 2012 coverage data by school.
- 40
Grace Hollister, email to GiveWell, March 6, 2016
- 41
- "A competitive RFP [request for proposal] process is now used to identify a professional survey organization to provide independent monitors. There are requirements placed on the experience of these monitors." DtWI Monitoring Improvements 2014, Pg 1.
- Bihar: "Through a competitive selection process, Evidence Action hired GfK Mode Private Limited as the independent monitoring agency that provided 125 monitors, who conducted monitoring activities of the deworming program across the state… Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively." DtWI Bihar 2015 Program report, Pg. 15.
- Rajasthan: "Through a competitive selection process, Evidence Action hired the State Institute of Health and Family Welfare (SIHFW), Jaipur as the independent monitoring agency. SIHFW provided 125 monitors who conducted monitoring activities of the deworming program across the state... Evidence Action held a detailed two-day training at the SIHFW campus in Jaipur to ensure the monitors were equipped with the necessary program knowledge to conduct monitoring effectively." DtWI Rajasthan 2015 program report, Pg 14.
- Madhya Pradesh: "Evidence Action hired an experienced independent research agency, SPECTRA Research and Development Private Limited, to conduct field-level process monitoring and coverage validation across 125 blocks in 50 districts of the state. A two-day training was held with 125 independent monitors and supervisors to equip them with knowledge to monitor the deworming program effectively." DtWI Madhya Pradesh 2015 program report, Pg 15.
- Delhi: "[Evidence Action] hired an independent research agency, Sigma Research and Consulting Private Limited [sic] that has experience in implementing field-based surveys, to conduct process monitoring and coverage validation in schools and anganwadis in Delhi. A two-day training was held with 80 independent monitors and supervisors to equip them with the knowledge to undertake the deworming program and undertake monitoring effectively." DtWI Delhi 2015 program report, Pg 14.
- Deworm the World was also involved in Chhattisgarh, but as it was engaged relatively late in the process, it did not conduct all of its standard monitoring activities in the state: "Although we place great emphasis on understanding the extent to which the school and health systems are ready to implement deworming, the extent to which deworming processes are being followed, and the extent to which coverage has occurred as planned, in Chhattisgarh we supported only with the coverage validation activity at schools due to time constraints." DtWI Chhattisgarh 2015 coverage validation report, Pgs 2-3.
- 42
- Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
- "Evidence Action has a permanent Monitoring, Learning, and Information Systems team. Deworm the World leverages this team for M&E." Grace Hollister, email to GiveWell, June 9, 2016
- "The field officers that collect the data in the field are short term hires who come from the counties in which the program is implemented" Grace Hollister, edits to GiveWell's review, November 7, 2016
- 43
See GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses. Note that these figures include central costs, so this reflects Deworm the World’s full budget.
- 44
See GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses.
- 45
- "Kenya’s National School-Based Deworming Program started in 2009. In 2012, it expanded to a nationwide program aimed at treating all at-risk Kenyan children each year for at least five years. Implemented by the Ministry of Education, Science, and Technology and the Ministry of Health. With support of Evidence Action’s Deworm the World Initiative, the Government of Kenya successfully reached 5.9 million preschool and school-age children in 2012/13 and 6.4 million children in 2013/14, surpassing targets by 18% and 12% respectively. School year 2014/15 results will be released in October. Technical and operational assistance to the program will continue through 2017." Evidence Action website, Deworm the World Initiative (October 2015)
- "Deworm the World is planning a strategy to sustain and institutionalize the program after its current grants expire in June 2017." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 4.
- 46
- "DtWI’s funding gap for 2015 will be small because DtWI recently received a large grant from CIFF. The CIFF grant will cover DtWI’s expenses in India, even with the large growth that DtWI is planning." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 2.
- Grace Hollister, email to GiveWell, June 9, 2016
- 47
Some of the differences between spending patterns in Kenya and India are due to the facts that a) Deworm the World pays for direct implementation costs in Kenya that it doesn't pay in India, b) Deworm the World does not pay for prevalence surveys in Kenya, and c) in Kenya, teachers are paid significantly higher allowances (per diems) for participating in the program:
- "It is important to note that there are fundamental differences between the Kenya and India programs, as funding to Evidence Acton supports direct implementation costs in Kenya (where the government does not pay those costs, but rather provides in-kind support), and India, where the government pays the lion’s share of implementation costs." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015
- Grace Hollister, conversations with GiveWell, February 25, 2016 and March 10, 2016
- 48
GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses
- 49
Grace Hollister, email to GiveWell, June 9, 2016
- 50
- GiveWell analysis of Deworm the World financials - 2016, Sheet: 2014-2016 expenses
- See our 2014 review for a summary of Deworm the World's costs pre-2014.
- 51
- "Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success.
"Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action." Evidence Action launch announcement 2013
- "Evidence Action Beta investigates what interventions might be suitable for massive scale up [sic] – finding the next thing that works." Evidence Action website, Evidence Action Beta (October 2015)
- "Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success.
- 52
- India: "…absent from most of the country, [schistosomiasis] risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1.
- "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3.
- See section on worm prevalence and intensity in India and Kenya below.
- 53
See the tables at WHO, Helminth control in school-age children second edition, Pg 18. "When the prevalence of any STH infection is under 20%, large-scale preventive chemotherapy interventions are not recommended. Affected individuals should be treated on a case-by-case basis."
- 54
- WHO, Helminth control in school-age children, Pg 74
- "Based on the findings of the prevalence survey and WHO guidelines, Evidence Action recommends an annual school based deworming program for school-age children in the state. [...] Given the pre-existing deworming treatments described above, this prevalence survey cannot be considered a baseline survey of an untreated population, but is rather a survey to assess STH infection rates in a treated population, to determine an optimal treatment strategy." DtWI Madhya Pradesh 2015 program report, Pg 38.
- "Our recommendation is explained in the prevalence survey report. The prevalence and intensity rates from the survey are not "baseline" data, given that there has been relatively regular administration of albendazole in MP through the BSM program that treated PSAC since 2005, and the LF program which provided community-wide treatment of 11 districts of MP (the number of endemic districts had fallen to 8 by 2014). As a result, these deworming efforts have likely had an impact on STH prevalence and MP could not be considered an untreated baseline population. We therefore did not apply the WHO guidelines for baseline STH prevalence.
"Annexure 10 of the WHO guidelines suggest continuing annual treatment for populations which have received deworming for several years, and prevalence is still greater than 10%. In addition, the high rate of open defecation in the state, and the planned ending of the LF program in MP, increased the risk of infection and a potential resurgence in prevalence." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015 - Grace Hollister, email to GiveWell, June 9, 2016
- 55
See the tables at WHO, Helminth control in school-age children second edition, Pg 18, for the WHO guidelines for treatment of schistosomiasis.
- 56
- See Deworm the World, target populations and parasitology data, which shows the prevalence rates for STH and schistosomiasis in Kenya, India, Vietnam, Ethiopia, and Nigeria, as well as the treatment frequency in each of those countries for Deworm the World supported programs. Note that several of the countries in which Deworm the World works have programs to treat lymphatic filariasis, which we discuss more below.
- Note that in 2016 Deworm the World might support biannual treatments for the four provinces in Vietnam in which it works, even though recent prevalence surveys revealed that the provinces only needed once per year or once per two years treatments. This is because Deworm the World and its partners had originally developed a biannual treatment strategy and now need to adjust the strategy according to the recent prevalence survey results. Deworm the World does not intend to support biannual treatments in Vietnam post-2016. Paul Monaghan, conversation with GiveWell, September 8, 2016
- 57
- Details of the Kenya studies (Miguel and Kremer 2004, Baird et al 2012) in our Reanalysis of the Miguel and Kremer deworming experiment page.
- Details of the Uganda study (Croke 2014) here.
- 58
Deworm the World noted that the KEMRI surveys in Kenya are designed for impact assessment, while the surveys in India are designed for mapping. This means that the schools selected in KEMRI's surveys are all from places where treatment is required or taking place. In mapping surveys, schools are selected to be representative of a larger geographic area (e.g., they may be selected in part based on which agro-climatic region they are in). Additionally, Deworm the World noted that the surveys in India are looking at a much larger population than the Miguel and Kremer 2004 and Croke 2014 studies examined and there may be substantial variation in prevalence across a given area. However, we still believe that making the comparisons we do is somewhat useful. Grace Hollister, conversation with GiveWell, June 13, 2016 and Grace Hollister, edits to GiveWell's review, November 7, 2016
- 59
- Deworm the World tends to focus on treating STH, but the deworming programs we reviewed focused primarily on schistosomiasis.
- For example, our understanding is that, as in India, there is no schistosomiasis in Pakistan (one of the countries Deworm the World is starting to work in).
- 60
- For example, India has such a program: The National Vector Borne Disease Control Programme LF treatment coverage 2015
- In Kenya, the LF program is housed within the country’s neglected tropical disease (NTD) unit, which has asked Deworm the World if it might support its process monitoring and coverage validation (PMCV) operations for LF. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- Deworm the World has told us that in both countries (Kenya and India), LF programs have generally been either unfunded or underfunded, resulting in sporadic treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- Additionally, our understanding from a number of conversations with Deworm the World and others is that many countries are beginning to shut down their LF programs as they eliminate the disease.
- 61
- "DEC [Diethylcarbamazine] + Albendazole in selected distt & DEC in other distt" LF treatment drugs 2012.
- "Albendazole, the same drug used to treat STH, is usually used to treat LF." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- Assam 2010 guidelines for deworming recommends 400mg of albendazole for children being dewormed, and Global Alliance to Eliminate Lymphatic Filariasis - Prevention also recommends 400mg doses.
- 62
- Or from twice-per-year to thrice-per-year.
- Note that community-based treatment, such as is typically used for LF, involves enlisting several people to travel from house to house to administer treatment, making it much more time-consuming and costly than school-based programs. Because its goal is to treat every person in a community, multiple trips to a single area may be required to ensure total coverage (e.g., if a household member is not at home during the first visit). GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- Deworm the World also claims that it generally knows where there is overlap between areas that are endemic for STH or schistosomiasis and areas that are endemic for LF. For example, in Kenya, only the coastal area is LF-endemic, but worm infections are more widespread, so LF treatment efforts involve a smaller number of subcounties. Community-based treatment programs might be cost-effective in places endemic with many NTDs but might not be in places only endemic for STH or schistosomiasis. Deworm the World has told us that many places do not have much overlap between different NTDs so school-based deworming programs can provide a cost-effective alternative to community-based treatment. GiveWell’s non-verbatim summary of a conversation with Grace Hollister on April 8, 2015
- 63
We have gained this understanding through many conversations and across multiple years of reviewing Deworm the World. See our review process.
Note that the methods Deworm the World uses in each country, for each round of MDA, might not be the same as those outlined here; Deworm the World adjusts its monitoring based on past learning or new contextual constraints.
On a separate page, we lay out some broad descriptions of the monitoring processes we have seen in Deworm the World's monitoring reports for states in India. This is to provide an example for the reader of the type of monitoring that Deworm the World conducts and to support the subsequent claims in our report. We have not included excerpts from Deworm the World's 2016 reports from India; we skimmed the reports to see if they appeared to be substantially different from the 2015 reports and felt that they were of similar quality.
- 64
For example, as part of its monitoring in Kenya, monitors visit sub-county trainings and teacher trainings:
- "The intent of Sub-Country Training (SCT) sessions is to ensure that Sub-county and division-level trainers understand the purpose and procedure of deworming. The successful completion of this activity allows the division trainers to then conduct the same activity with teachers in their sub-counties.
"PMCV teams attended 38 out of the total 111 SCTs conducted in the third year of the program. Field officers interview participants before and after the training and completed observations during the course of the activity."
Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 5. - "Teacher Trainings (TT) are conducted by the division trainers using a “Teacher Training Booklet” as reference material. A quality TT session is considered to be one where the necessary content is covered and retained by participants. TTs also serve the function of distribution of drugs and materials (monitoring forms, posters) to teachers. It is the aim of the program to provide all schools with their required drugs and materials at teacher training sessions. Teachers are expected to use the “Deworming Day Checklist” to conduct operations on the day. They are also expected to sensitize other teachers at their schools who did not attend the training on deworming day procedures (see the section on deworming day contained in this report).
"A total of 76 TT sessions were observed by PMCV field officers in Year 3. Of those training sessions, 13 were specifically SCH trainings."
Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 7. - The trainings that monitors are sent to are randomly selected by a computer program. Paul Byatta, conversation with GiveWell, September 20, 2016
- "The intent of Sub-Country Training (SCT) sessions is to ensure that Sub-county and division-level trainers understand the purpose and procedure of deworming. The successful completion of this activity allows the division trainers to then conduct the same activity with teachers in their sub-counties.
- 65
Paul Byatta, conversation with GiveWell, September 20, 2016
- 66
We believe that monitors are supposed to select every third teacher that arrives at the training until they have interviewed four teachers:
"SURVEY INSTRUCTIONS [...]- Select every third participant that arrives before the start of the training.
- Interview at least four participants."
- 67
For example:- Kenya. In Kenya, Deworm the World's monitors visit schools and communities before Deworming Day to assess their level of preparedness for the upcoming MDA. Schools are selected randomly, and then monitors visit the communities near the schools that are selected. Paul Byatta, conversation with GiveWell, September 20, 2016
- "PMCV field officers visited 256 schools prior to Deworming Day in order to assess preparedness for deworming activities and to review the effect of teacher trainings. A total of 244 of the schools were planning on participating in deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.
- "Use the PRE-DD-SCHOOL instrument to conduct interviews and note observations at the selected primary school before deworming day. In case selected school is closed, does not exist or has already dewormed move to the next nearby primary school. Allow Head teacher or Rep to consult with Head teacher/ REP when necessary. PRE-DD-SCHOOL should be completed along with PRE-DD-ECD or PRE-DD-COMMUNITY." Deworm the World, Kenya Year 2, Pre DD - School instrument, Pg 1.
- "PMCV Field officers interviewed a total of 716 parents with children. Of these, 379 were parents of enrolled children across 130 different schools and 337 were parents of non-enrolled children. The number of parents with at least one child enrolled in early childhood development (ECD) was 283 of the sample population, or 65%. Just over one third or 35% of parents had no child enrolled in ECD. The average age of those children reported to be enrolled (by their parents) was 7.7 years, whereas the average age of non-enrolled children was 3.7 years.
"PMCV field officers observed parents’ level of awareness of Deworming Day, their intentions regarding taking children to be dewormed and documented the primary source by which parents were receiving such information." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 11. The next section is titled "4.1. Pre-Deworming Interviews - Parents."
- India: In India, telecallers phone schools ahead of Deworming Day to assess the school's preparedness. We have not yet requested data from these calls.
- For example: "Our tele-callers place phone calls to track the delivery and availability of training, drug, and IEC materials at the district, block, and school/anganwadi levels as Deworming Day approached. Approximately 4,840 successful calls were made from February 1 to 14, including 1,097 calls to schools across 313 blocks and 51 districts, and another 3,586 calls to block and district officials. Tele-callers created tracking sheets to outline issues identified during calls and monitoring visits. Issues at the district, block, and school levels were shared with the state government to ensure that the government was able to take corrective action to address the gaps in real time as necessary." DtWI Madhya Pradesh 2015 program report, Pgs 16-17.
- Kenya. In Kenya, Deworm the World's monitors visit schools and communities before Deworming Day to assess their level of preparedness for the upcoming MDA. Schools are selected randomly, and then monitors visit the communities near the schools that are selected. Paul Byatta, conversation with GiveWell, September 20, 2016
- 68
For example, the most recent monitoring report we have from Kenya (for the 2014-15 round, or Year 3) includes some sample results from the pre-Deworming Day visits:
- The report notes that 97% of schools visited pre-Deworming Day had a teacher who had attended a training in the last fifteen days: "According to interviews with head teachers, 97% of these schools had a teacher who had attended training in the past 15 days. A further 86% of trained teachers had trained or sensitized other teachers on how to administer drugs and conduct deworming day. Almost all teachers (99%) found the Teacher Training Booklet to be ‘very’ or ‘somewhat’ useful in this process and 95% reported to use it often." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 15.
- While 95% of schools had received some deworming tablets, only 87% of schools believed that they had received enough to cover the children at their school on Deworming Day: "At the time of PMCV visits, 95% of schools reported having received deworming tablets, with 97% of those schools having received them at the time of TT. This percentage is higher than those originally observed by PMCV officers with only 92% of schools reported to receive their drugs during TT. Upon further investigation, 87% of schools considered that they had received a sufficient supply of drugs for their current enrolled and ECD populations. It is likely that these schools requested additional drugs, because 98% of schools were observed to have sufficient drugs in place on deworming day. Only 5% of schools did not have all monitoring forms present prior to deworming day. Such schools have always sought support from the sub-county offices that organize additional prints or photocopying to ensure they have the forms on deworming day." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 16.
- Note that Deworm the World shared the pre-Deworming Day data that informs the monitoring report and we have not yet analyzed it. There appears to be a large number of questions in the survey that were not included in the report. The survey instruments that Deworm the World's monitors use are long (e.g., see Deworm the World, Kenya Year 2, Pre DD - School instrument, in which both teachers and randomly selected students are interviewed). The monitoring reports only include a few summary metrics, and we are unsure how the metrics included in the report are chosen. It is possible that the report's metrics are those that reflect most favorably on Deworm the World and that vetting the full data could reveal that schools are less adequately prepared.
- 69
- When selecting parents to interview in the community, monitors ask the school headmaster to point to a student's house near the school. The monitor begins by interviewing that student's family, then walks to subsequent households, skipping every other household until the monitor has interviewed at least 6 families with an enrolled school-age child and 6 families without. Paul Byatta, conversation with GiveWell, September 20, 2016
- In Kenya, monitors are also asked to interview the Community Health Extension Worker (CHEW) in the community, since they assist with Deworming Day: "On selecting the CHEW to interview, there is only one CHEW per community. A few communities do not have CHEWs at all." Paul Byatta, attachments to email to GiveWell, September 23, 2016
- For example, from Kenya: "Of those parents aware of deworming, only 41% knew the correct Deworming Day date, 81% knew the correct target population, and 48% knew the correct age group. These results indicate that although parents report being aware of deworming, almost half do not have the information required to attend (date). There is the scope to find a more robust method of ensuring information retention in awareness of deworming." Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 12.
- 70
- India: In India, Deworm the World's monitors survey randomly selected schools (intended to be representative at the state level) on Deworming Day and Mop-Up Day to check whether deworming is occurring. They also visit another set of randomly sampled schools within a few weeks of Deworming Day, to check schools' records for evidence of deworming and interview children and teachers about the Deworming Day. Examples from several Deworm the World-supported states:
- Bihar: "Two-stage probability sampling was used to select schools for coverage validation on deworming day and mop-up day. First, 125 blocks were selected from all 38 districts by probability proportional to size sampling (Probability proportional to size sampling (PPS) selected blocks in Bihar, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. Schools were then randomly selected from the selected blocks.), followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students.
During coverage validation an additional 748 randomly sampled schools were surveyed after deworming days to check whether deworming occurred, if reporting protocols were followed, and to validate the coverage reporting. Coverage validation data was gathered through interviews with headmasters and 3 students (in 3 different randomly selected classes in each school), and by checking of all class registers and reporting forms." DtWI Bihar 2015 Program report, Pg. 15. - Rajasthan: "Two-stage probability sampling was used to select schools for independent monitoring on Deworming Day and mop-up day. First, 125 blocks were selected from all 33 districts by probability proportional to size (PPS) sampling (Blocks were selected by Probability proportional to size sampling (PPS) in Rajasthan, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. Schools were then randomly selected from the selected blocks.), followed by random sampling of schools to provide state-wide estimates of indicators." @DtWI Rajasthan 2015 Program report@, Pg. 14.
- Madhya Pradesh: "A two-stage probability sampling process (Blocks were selected by Probability Proportional to Size (PPS) sampling, followed by random sampling of schools to provide state-wide estimates of indicators. We used PPS sampling to select blocks in Madhya Pradesh, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. After selecting blocks, we randomly selected schools from within these blocks.) was followed to select schools for NDD, mop-up day and coverage validation." @DtWI Madhya Pradesh 2015 Program report@, Pg. 16.
- Delhi: "The method of stratified random sampling using proportional allocation approach (A random sample of population in which the population is first divided into distinct subgroups or strata, and random samples are then taken separately for each stratum proportional to the size of the subgroup.) was followed for selection of schools and anganwadis for deworming day, mop-up day, and coverage validation monitoring to provide state-wide estimates of indicators." @DtWI Delhi 2015 Program report@, Pg. 14.
- Bihar: "Two-stage probability sampling was used to select schools for coverage validation on deworming day and mop-up day. First, 125 blocks were selected from all 38 districts by probability proportional to size sampling (Probability proportional to size sampling (PPS) selected blocks in Bihar, according to the number of schools in that block. PPS corrects for unequal selection probabilities in random sampling of unequally sized blocks. Schools were then randomly selected from the selected blocks.), followed by random sampling of schools to provide state-wide estimates of indicators. Evidence Action held a detailed training on February 15 and 16 to ensure the monitors were equipped with the necessary knowledge on the deworming program to conduct monitoring effectively. The monitors visited the 123 randomly selected schools on deworming day and an additional 124 schools on mop up day to check for adequacy of drug supplies, awareness materials, whether teachers had received training, knowledge of adverse event management protocols, and reporting processes. The monitors gathered data through observation of deworming and interviews of headmasters, teachers, and randomly selected students.
- Kenya: The schools observed on Deworming Day are randomly selected. Paul Byatta, conversation with GiveWell, September 20, 2016
- "There is no direct sampling link between specific schools monitored on the pre-deworming day and those monitored on the deworming day. However, 20% of schools that are monitored for Pre-deworming, Deworming and Post-Deworming are randomly selected within the same sub-counties in which we monitored sub-county and teacher training events, while 80% of the schools that are monitored are randomly selected from the full pool of targeted schools, but stratified by county." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pg. 2
- Vietnam: For the first round of monitoring, main schools were randomly selected, but satellite schools were not, because Deworm the World did not have a full list of satellite schools when the monitoring began. Going forward, all schools should be randomly selected. Additionally, during the first round, monitors called ahead to districts several days before Deworming Day to confirm that Deworming Day would be occurring in the schools they were assigned to monitor. It is possible that this notified schools ahead of time that a monitor would be coming, although Deworm the World's Paul Monaghan doubted it. Paul Monaghan, conversation with GiveWell, September 8, 2016
- India: In India, Deworm the World's monitors survey randomly selected schools (intended to be representative at the state level) on Deworming Day and Mop-Up Day to check whether deworming is occurring. They also visit another set of randomly sampled schools within a few weeks of Deworming Day, to check schools' records for evidence of deworming and interview children and teachers about the Deworming Day. Examples from several Deworm the World-supported states:
- 71
- India: For example, in Delhi, monitors are instructed in the survey instrument, "Is the school open? [If not,] Did you go to the buffer school?" DtWI Delhi 2015 monitoring survey from deworming day, schools Pg 3. It is our impression that most schools were open when monitors visited. We spot-checked several states' data:
- DtWI Bihar 2015 monitoring data from deworming day, schools, Column L shows that all schools monitors visited were open.
- DtWI Rajasthan 2015 monitoring data from deworming day, schools, Column M shows that 124 of the 125 schools monitors visited were open.
- Kenya: In Kenya, it is rare that monitors find schools to be closed or missing because Deworm the World has operated in Kenya for several years. However, monitors will try to go to another school if their school is closed. Paul Byatta, conversation with GiveWell, September 20, 2016
- "When the monitor is unable to monitor the assigned school (because of closure, etc) we ask the monitor to i) to communicate the same with his supervisor and, ii) to visit the nearest school if the nearest school was not part of the randomly picked school." Paul Byatta, attachments to email to GiveWell, September 23, 2016
- Vietnam: The backup schools are also randomly selected. Paul Monaghan, conversation with GiveWell, September 8, 2016
- India: For example, in Delhi, monitors are instructed in the survey instrument, "Is the school open? [If not,] Did you go to the buffer school?" DtWI Delhi 2015 monitoring survey from deworming day, schools Pg 3. It is our impression that most schools were open when monitors visited. We spot-checked several states' data:
- 72
- See, for example, DtWI Delhi 2015 monitoring survey from deworming day, schools, Deworm the World, Kenya Year 3, DD - Main instrument, and Deworm the World, Vietnam 2016 monitoring survey form for Deworming Day
- Note that it is not actually dangerous to give the deworming pills to sick children; Deworm the World simply prefers not to deworm sick children to avoid causing people to associate the deworming pills with illness.
- "Deworming pills should not be distributed to sick children...This is not because deworming pills could harm sick children. It is because DtWI wants to avoid people (and potentially the media) blaming the deworming pill for a child’s illness." DtWI 2013 GiveWell site visit
- Note that it is not actually dangerous to give the deworming pills to sick children; Deworm the World simply prefers not to deworm sick children to avoid causing people to associate the deworming pills with illness.
- 73
- For example, in Kenya: "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment.
Deworming was reported to occur inside classes in 47% of observed schools and outside in 53% of schools. Deworming was considered to be ‘systematic’ in 98% of schools. The correct dosage for albendazole is one tablet per child and the correct age is 2-14 years. These procedures were observed to be followed correctly by 86% of teachers observed by field officers.
Coverage: Coverage is defined as the number of children dewormed according to the school/class register. SCH tablet (PZQ) coverage was 99% across schools treating for SCH. Also executed was the use of ‘tablet poles’ for the treatment of SCH in 74% of schools. STH tablet (ALB) coverage was 99% across observed schools. Teachers were reported to correctly observe children swallowing PZQ in 99% of schools and ALB in 96% of schools. Observing children swallowing is most important when treating for SCH as the tablet does not taste pleasant and there are high chances of children spitting if not observed."
Deworm the World, Kenya Narrative Report - Year 3, Quarter 3, Pg 17. - For example, in India: When monitors visit schools on Deworming Day, they make observations and ask questions related to the process of deworming, to check whether or not it is being implemented correctly. For example, monitors observe whether or not teachers ask their students if they are sick before giving them their pills and how many pills teachers give students. Monitors are prompted to answer:
- "Did the teacher ask the children if they are sick/under medication before giving the medicine?"
- "Did you see any child being given less than one deworming tablet?"
- "Did you see any child being given more than one deworming tablet?"
DtWI Rajasthan 2015 monitoring survey from mopup day, schools, Pgs 13-14.
- For example, in Kenya: "Briefly, systematic and successful deworming days are such that classes are arranged in lines, children wash their hands before deworming, teachers are clearly documenting the names of those dewormed, and there are stations for children who experience any side effects after treatment.
- 74
- Deworm the World is not supporting this monitoring in Ethiopia, because SCI and Deworm the World are supporting coverage surveys to be conducted after the Deworming Days there instead. Deworm the World also did not support this coverage validation monitoring for the first MDA it supported in Cross River, Nigeria, although it hopes to in subsequent rounds. Grace Hollister, edits to GiveWell's review, November 7, 2016
- For example, this is what we heard about the monitoring conducted in Vietnam. Paul Monaghan, conversation with GiveWell, September 8, 2016. Mr. Monaghan noted that he had based the monitoring methods in Vietnam off of Deworm the World's methods used in its other countries.
- In India: We believe coverage validation was a few days or up to a couple weeks after deworming days. For example:
- Madhya Pradesh 2015: "The monitors visited 125 randomly selected schools on NDD, and an additional 125 schools on mop-up day (February 14) to check for adequacy of drug supplies and awareness materials, and assess whether teachers had received training, and had knowledge of adverse event management protocols and reporting processes. Monitors gathered data by observing deworming and by interviewing headmasters, teachers, and randomly selected students. An additional 750 randomly sampled schools were surveyed from February 18-26 to check whether deworming occurred and reporting protocols were followed, and to validate the coverage reporting." DtWI Madhya Pradesh 2015 program report, Pg 16
- Delhi 2015: "These monitors were to visit total of 400 randomly selected schools and 400 randomly selected anganwadis; 80 schools and 80 anganwadis on deworming day and mop up day each (April 16 & April 20); and 240 schools and 240 anganwadis during coverage validation (April 23-27, 2015)." DtWI Delhi 2015 program report, Pg 14
- 75
For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools
- 76
For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools
- 77
For example, in Kenya, it is our understanding that monitors interview three randomly selected children from three separate classes, for a total of nine students, at each school they visit. See Deworm the World, Kenya Year 3, Post DD - Coverage instrument. On Pgs 1-3 there are three spaces for randomly selected classes. The instructions read: "Thank the Head Teacher or designate and request to speak to pupils of the randomized class...CHOOSE CHILD 5, 10 AND 15TH ON FORM E. IF LESS THAN 15 CHILDREN, SELECT THE LAST CHILD. ENSURE TO INTERVIEW AT LEAST THREE CHILDREN. ASK THE TEACHER FOR PERMISSION TO SPEAK TO THEM ONE AT A TIME Ask questions in multiple ways for interviews with students, use local language if possible. Don’t rush responses. Try to make them feel at ease. Speak to one child at a time at a place where they are comfortable.... [interview questions]...END, MOVE TO THE NEXT SAMPLED CLASS"
- 78
For example, see DtWI Chhattisgarh 2015 monitoring survey for coverage validation, schools
- 79
Grace Hollister, edits to GiveWell's review, November 7, 2016
- 80
- Vietnam: There were no checks on the monitors' work for the first round of treatment. Paul Monaghan, conversation with GiveWell, September 8, 2016
- Kenya: In Kenya, Deworm the World does have some checks on monitors' work. A randomly selected 10% of the schools that are being visited by monitors are called the same day that the monitors visit, to ensure that the monitor is actually at the school. Paul Byatta, conversation with GiveWell, September 20, 2016
- "We do back-checks for the pre-deworming, de-worming and post-deworming day… Generally, we seek to establish that the monitor did actually visit the school assigned to him/her, interviewed at least the headteacher of the school and that there is broad consistency on some of the data they collect by the monitor and the back-checker. For instance, the back-checker also asks about the availability of forms and drugs at schools, which is also information that is asked by the monitor. We share back-checks results with the short term monitors before we make the last payment to them. The other purpose is to improve our data collection training for the subsequent waves." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pgs 2-3
- India: "In India, the following steps are implemented to serve as check on the IMCV work. Some of these are detailed in the contracts with IM firms – let me know if you’d like to see those:
1. We collect photographs of all the schools and anganwadis (with the names) visited during PMCV in the states.
2. We also collect signatures and mobile numbers of headmasters/school teachers and anganwadi workers, which is the part of PMCV format itself.
3. Together with signature of the teachers/head masters, school stamp is also taken on signature sheet which is part of PMCV tools. Since we collect CAPI based data, monitors carry hard copy of signature sheet to get the stamp of the schools.
4. Additionally, random calls are also made by state team/tele-callers to confirm that monitors visited to designated school and anganwadis.
5. Evidence Action staff (including RCs & DCs) visits in selected schools and anganwadis on NDD and mop-up day to check if monitors visited the selected sites."
Grace Hollister, edits to GiveWell's review, November 7, 2016
- 81
- Grace Hollister, conversation with GiveWell, September 1, 2016
- For example, in Kenya: "Does the monitoring team have an estimate of how many non-enrolled children the deworming program reaches? We get this from treatment forms that schools submit back via “the reverse cascade”. For PMCV, we monitor that teachers are aware they should be treating non-enrolled, and they are aware of age categories targeted for non-enrolled. When we do data audit on treatment forms, we also check that data for on non-enrolled is entered accurately." Paul Byatta, attachments to email to GiveWell, September 23, 2016, Pg 3
- 82
- Kenya: "Show student 3 tablets, ask: Which one of these three tablets did you take? Circle the indicated tablet." Deworm the World, Kenya Year 3, Post DD - Coverage instrument
- Vietnam: "What type of medicine was given out? (prompt the child with options)" and "Do you remember the color of the tablet? (do not prompt)" are questions on the coverage validation survey form for student interviews. Deworm the World, Vietnam 2016 monitoring survey form for coverage validation, Pg 7.
- India: In India, monitors show children the deworming tablet and ask if it's the tablet that they took, which we do not believe is a rigorous method of verifying student answers. For example, see Pgs 20-23 of DtWI Bihar 2015 monitoring survey for coverage validation, schools
- 83
Note: We have not included examples from Deworm the World's 2016 reports from India; we skimmed the reports to see if they appeared to be substantially different from the 2015 reports and felt that they were of similar quality. We have also not included results from Year 4 of Kenya's program; we have only seen a report on monitoring done before and during Deworming Day for Year 4 (Deworm the World, Kenya process monitoring report, Year 4)
- 84
Deworm the World notes that an important difference between the two programs is scale: The Kenya program is significantly smaller than the India program, thus it may be easier to execute the program more effectively. Grace Hollister, email to GiveWell, June 9, 2016
- 85
"[GiveWell]:I forgot to ask on the call if there's anything significantly different we should expect for the monitoring from Deworm the World in the future. For example, should we expect the Ethiopia and Nigeria monitoring to look similar to the type of monitoring Deworm the World has conducted in other places?
[Deworm the World]: Yes, the monitoring will be relatively similar in other locations but does get customized based on the needs/interest of the government and time/budget availability. For instance, in Vietnam we were not able to conduct training monitoring during the MDA earlier this year." Grace Hollister, email to GiveWell, September 13, 2016 - 86
- 87
Deworm the World supports program costs in Cross River, Nigeria and expects to support program costs in future Nigeria states. It expects to support program costs in Pakistan, and possibly Indonesia. Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 88
We attempt to learn if there was a possibility that the program would have been funded by other donors. However, given that there is a global funding gap for deworming treatments (more), we suspect that in most cases where Deworm the World pays for a new deworming program, it is increasing the number of children dewormed.
- 89
- Our intervention report discusses this briefly
- Other conversations and observations have reinforced our impression that administering deworming drugs is fairly straightforward.
- The WHO factsheet on STH: "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet
- 90
Deworm the World notes that: "albendazole and mebendazole are quite hardy (neither requires special storage conditions) and have a long shelf life." Grace Hollister, email to GiveWell, June 9, 2016
- 91
For example, drug quality was tested in each program that Deworm the World supported in India in 2013-2014.
- Bihar 2014:
- "The drugs were safely stored in State Health Depot in Patna until November 2013. At that point, the districts began to pick up their share of the drugs from the depot. In October, Deworm the World had coordinated for lab testing of the stored drugs via ASCHO NIBULA INDUSTRIES LTD, an independent lab which approved the quality of the drugs." DtWI Bihar 2014 program report, Pg 13.
- "The quality of drug storage was satisfactory in most schools that were monitored. 98.7% of them stored the drugs in a clean location, 91.1% of them were stored away from direct sunlight and 97.4% of them were stored away from the direct reach of children." DtWI Bihar 2014 program report annex 1, Pg 7.
- Delhi 2013:
- "Once the procured syrups and donated tablets were delivered to Directorate of Health Services central storage room, they were tested in a government-accredited laboratory to ensure drug quality prior to administration." DtWI Delhi 2013 program report, Pg 6.
- From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Delhi 2013 program report, Pg 24. A table of results is also presented on the same page.
- Rajasthan 2013:
- "To instill confidence among the stakeholders that the drugs were of good quality, Deworm the World arranged for sample testing of the donated drugs by two independent labs7. Similarly, Rajasthan Medical Services Corporation sample tested the syrups they procured." DtWI Rajasthan 2013 program report, Pg 6.
- From its monitoring results: "Drug storage conditions were satisfactory in almost all schools and anganwadis." DtWI Rajasthan 2013 program report, Pg 40. A table of results is also presented on the following page.
- Bihar 2014:
- 92
"Now, state governments have responsibility for testing drugs. We have recently raised some concerns about the need to standardize the testing that is taking place, and are currently working with the MoHFW to build out the NDD operational guidelines with more detailed guidance on this point." Grace Hollister, edits to GiveWell's review, November 7, 2016
- 93
Of classes where monitors observed deworming activities in India, there were low numbers of adverse events (see table below). We aren't sure what portion of the adverse events may be caused by incorrect dosages.
Bihar 2015 Rajasthan 2015 Delhi 2015 Madhya Pradesh 2015 Sample of question asked Classes where there were adverse events (monitors' observations) 5% (vomiting), 0% (diarrhea) 2% (vomiting), 0% (diarrhea) 5% (vomiting), 0.8% (diarrhea) 6% (vomiting), 0% (diarrhea) "Did you see any child with adverse effects (nausea, vomiting, stomachache, etc.) after taking the medicine?" Sources for the information in the table:
- Bihard 2015: Deworming Day and Mop-up Day (N = 247 schools), DtWI Bihar 2015 independent monitoring tables, Pg 4
- Rajasthan 2015: Deworming Day and Mop-up Day ( N = 250), DtWI Rajasthan 2015 independent monitoring tables, Pg 4
- Delhi 2015: Deworming Day and Mop-up Day (N = 147 schools), DtWI Delhi 2015 independent monitoring tables, Pg 6
- Madhya Pradesh 2015: Deworming Day and Mop-up Day (N = 250 schools), DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 4
- Sample question: DtWI Madhya Pradesh 2015 deworming day monitoring form, Pg 14
- 94
- "Note that National Deworming Day operational guidelines state that only tablets should be used. Albendazole dosage is the same for all children aged 2 and above; it is only children aged 1-2 that require a different (half) dose." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- From Deworm the World's report on the Delhi 2013 program: "A key discussion from round one was the administration of drug to younger children in the pre-school age for whom chewing a tablet was seen as a difficulty. Hence keeping in mind the scale of the program, with an objective of making it a safe public health initiative the committee decided that deworming would be implemented across all districts following the World Health Organization (WHO) sanction in administering albendazole 400 mg tablets and the GOI guidelines under the WIFS program for administering the syrup vial. They also decided to opt for Albendazole 200mg dose suspension for 2-6 year children as it is a single dose for this age group which leads to lesser error in administration, while Albendazole 400mg tablets for older children." DtWI Delhi 2013 program report, Pg 6.
- The WHO factsheet on STH cites only a single recommended dosage (depending on which drug is used): "The recommended medicines – albendazole (400 mg) and mebendazole (500 mg) – are effective, inexpensive and easy to administer by non-medical personnel (e.g. teachers)." WHO STH factsheet
- 95
- Bihar 2015: In 8.8% of trained schools and 5.4% of untrained schools, monitors observed children given less than one tablet; in 2.6% of trained schools and 4.7% of untrained schools, monitors observed children given more than one tablet. DtWI Bihar 2015 independent monitoring tables, Pg 9 (Table 10)
- Rajasthan 2015: In 6.0% of schools, monitors observed children given less than one tablet; in 1.6% of schools, monitors observed children given more than one tablet. DtWI Rajasthan 2015 independent monitoring tables, Pg 2 (Table 2)
- Madhya Pradesh 2015: In 9.3% of schools, monitors observed children given less than one tablet; in 2.7% of schools, monitors observed children given more than one tablet. DtWI Madhya Pradesh 2015 independent monitoring tables, Pg 2 (Table 2)
- Delhi 2015: In 2.5% of schools and 10.6% of anganwadis, monitors observed children given less than the prescribed dose of albendazole; in 0.8% of schools and 6.0% of anganwadis, monitors observed children given more than the prescribed dose of albendazole. DtWI Delhi 2015 independent monitoring tables, Pgs 2, 16 (Tables S1 and A1)
- Bihar’s 2014: 8% of schools observed gave children less than one tablet and 2% gave more than one tablet. DtWI Bihar 2014 program report annex 2, Pgs 2 and 4.
- "In 3.1% of schools and anganwadis, monitors observed children being given more than one tablet/syrup bottle. As per protocol, the children should not have been given more than one tablet. Therefore in cases such as these, the monitors were trained to intervene and prevent the administration of an additional dose." (The percentage of schools using less than one pill/bottle per child was not reported.) DtWI Rajasthan 2013 program report, Pg 38.
- Monitors in Delhi in 2013 did not report on either of these observations. See DtWI Delhi 2013 program report, Pgs 21-25.
- 96
"[S]ingle-dose oral therapies can kill the worms, reducing ... infections by 99 percent ... Reinfection is rapid, however, with worm burden often returning to eighty percent or more of its original level within a year ... and hence geohelminth drugs must be taken every six months and schistosomiasis drugs must be taken annually." Miguel and Kremer 2004, pg. 161.
- 97
- "The majority of direct program costs in India are government funded. There is a cap within the National Health Mission budgets on M&E – M&E cannot exceed 10% of the overall budget. So there are specific areas, such as program monitoring, that would likely not have investment at the level we are able to provide." Grace Hollister, edits to GiveWell's review, November 7, 2016
- "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2.
- "District Coordinators (temporary Deworm the World employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by Deworm the World that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and Deworm the World's tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. Deworm the World called a random sample of 8,000 schools.
The prevalence survey would not have happened without Deworm the World's support." DtWI 2013 GiveWell government interviews, Pg 5.
- 98
"RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3.
- 99
"[The Nodal Headmaster said] that most aspects of the program are excellent, but he had 2 suggestions:
- Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so)
- Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6.
- 100
- "In Tanzania matters came to a head in places around Morogoro in 2008. Distribution in schools of tablets for schistosomiasis and soil-transmitted helminths provoked riots, which had to be contained by armed police. It became a significant national incident, and one of the consequences has been the delay in Tanzania adopting a fully integrated NTD programme, and the scaling back some existing drug distributions." Allen and Parker 2011, pg. 109.
- "From these reports a number of problems with the MDA were raised which included fear of side effects from the tablets, particularly following the mass hysteria and death in Blantyre and Rumphi respectively and may explain some of the geographic heterogeneity seen. Furthermore most districts reported that MDA occurred after standard 8 students had finished exams and left school, and due to having inadequate resources for drug distribution...The side-effects incident in Blantyre and death in Rumphi had a large effect on districts and with many district reports stating that after the incidence many families refused to participate." SCI Malawi coverage survey 2012 Pgs 5, 21.
- 101
- "Deworm the World’s cost per treatment in Kenya is likely more reflective of the costs of future programs (e.g., in Nigeria and Ethiopia) than its cost per treatment in India." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
- Deworm the World noted that the costs in Kenya are high, partly due to higher quality M&E:
- "In some ways, Deworm the World’s program in Kenya has served as a proof of principle for the effectiveness of school-based deworming and is a "gold standard" that is unlikely to be exactly replicated elsewhere." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 5
- "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- In the second round of treatment in Kenya, prevalence surveys were 11% of the total costs and other M&E was 6%. Per treatment, these costs are $0.059 ($379,523/6,405,462) and $0.035 ($222,750/6,405,462) respectively DtWI Kenya 2013-2014 cost per treatment data
- Our understanding is that costs are higher in Kenya than India primarily due to cultural differences, including paying significantly higher allowances (per diems)to teachers for participating in the program. For more details, see GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 and GiveWell analysis of Deworm the World cost per treatment.
- Deworm the World has also noted that treating schistosomiasis in Kenya increases the costs: "The extra costs of treating schistosomiasis in addition to STH. Schistosomiasis drugs tend to be more expensive than STH drugs and, in Kenya, schistosomiasis treatment sites are sometimes much more remote. The treatment strategy for schistosomiasis also differs from STH because schistosomiasis is more localized (e.g., it is not necessarily ideal to treat an entire sub-county). This also makes mapping schistosomiasis more expensive." GiveWell’s non-verbatim summary of a conversation with Grace Hollister on July 22, 2015, Pg 3.
- 102
Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 103
See Evidence Action, blog post, January 16, 2015 for a description of how Deworm the World calculates the cost of deworming. See Evidence Action, blog post, July 5, 2016 for an update.
- 104
Deworm the World staff, conversations with GiveWell, October 3-4, 2016 Deworm the World thought that the costs of these activities were quite low (less than 5%). We have conservatively assumed 5% in our analysis.
- 105
In general, cost-per-treatment figures have stayed fairly consistent over the last several years in Kenya and India:
- In the Kenya 2013-2014 treatment round, the total cost-per-treatment was $0.56 (not including staff time). It remained at $0.56 in the 2014-2015 treatment round. GiveWell analysis of Deworm the World cost-per-treatment, October 2016
- In the India 2012 treatment rounds for which we have information, the average cost-per-treatment was $0.38 (including staff time). In the 2013-2014 treatment rounds, the average cost-per-treatment was $0.30 (see our 2015 review). Note that we did not adjust the coverage numbers used in either of these calculations. In the 2014-2015 treatment rounds, the average cost-per-treatment was $0.32, after adjusting for the fact that Deworm the World's monitors found lower coverage than reported by the government. GiveWell analysis of Deworm the World cost-per-treatment, October 2016
- 106
- DtWI cost per treatment summary 2013
- GiveWell DtWI 2013-2014 cost data summary
- GiveWell analysis of Deworm the World cost-per-treatment, October 2016
- The rounds for which we have seen cost-per-treatment data are:
- Bihar: 2012, 2014, and 2015
- Delhi: 2012 and 2013
- Rajasthan: 2012, 2013, and 2015
- Madhya Pradesh: 2015
- 107
GiveWell analysis of Deworm the World cost-per-treatment, October 2016 Note: We focus on these three states because Deworm the World did not send us 2015 cost-per-treatment data for any other states in India.
- 108
For example, see our 2015 review. We estimated the staff time costs based on DtWI’s estimate of similar costs from the same states in 2012. DtWI had imputed those costs based on estimates of government employee salaries; for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50) (DtWI Rajasthan 2013 cost data).
- 109
GiveWell analysis of Deworm the World cost-per-treatment, October 2016, Summary sheet, column H.
- 110
- Alix Zwane conversation August 30th 2013
- For example, for Madhya Pradesh's 2015 costs: "Drug costs are included in this model as an imputed cost to the government. Unlike other programs supported by Evidence Action in 2014/2015, Madhya Pradesh did not receive Albendazole under the WHO drug donation program. Instead, they procured the drugs locally. In order to ensure that the program had sufficent drug supplies, the government of India purchased roughly $35,000 in Albendazole and Syrups to supplement their existing stocks. Given that the number of drugs disseminated to districts for use on National Deworming Day is unavailable, and the true value of drugs used by the program far exceeds the $35,000 the government spent to supplement existing stocks, this model relies on treatment figures to estimate the value of drugs. This method of calculation is a deviation from the method used to calculate the value of drugs in models representing the cost of other geographies within India for the 2014/2015 round." Deworm the World, Madhya Pradesh cost-per-treatment - 2015
- 111
- We have not adjusted these data for inflation or changes in exchange rates over time.
- Deworm the World's 2013 and 2014 budgets stopped estimating the financial value of government employees’ time spent on the deworming program after the deworming rounds in 2012: "We have consulted with JPAL on the costing model approach and they have suggested, in line with papers they've done, not to quantify teachers' time nor principals' time for the actual deworming day; we still have included the cost of teachers attending the training since there is a direct cash transfer to the teachers for their coming to the training." Jessica Harrison, DtWI Associate Director, email exchange with GiveWell, November 2014
- Deworm the World cited JPAL CEAs in education 2011 for this approach.
- The change is most relevant for teachers and principals, large numbers of whom attend a training and administer the pills to children. While we have seen direct costs of paying teachers and principals that attend deworming training, we are unsure whether those costs should be counted as per-diems (perhaps necessary for some trainees to pay for transport, but not accounting for the lost work time at school) or as optional extra wages (that don’t result in lost work time at school and are sufficient to incentivize the necessary labor), or something in between.
- In order to include the value of teachers and principals’ time for the three recent rounds, we have copied the relevant expenses from the prior year’s budget of the same state (which were imputed costs) and replaced the allowances that were paid directly to teachers and principals during trainings. We believe this methodology is likely to result in some inaccuracies and we may revise it if we become confident of a more accurate method.
- 112
We estimated the staff time costs based on Deworm the World's estimate of similar costs from the same states in 2012. It had imputed those costs based on estimates of government employee salaries, for example, in Rajasthan it estimated 300,000 teacher- and principal-days were used in deworming day and mop-up day, and valued that time at 150 rupees per day (about $2.50).
- 113
"We include all partners’ expenditures in determining costs for the the deworming programs, but we do not consider spending that would be incurred even without deworming taking place. [...] Teachers’ and principals’ general salaries are not included because they do not spend additional time on deworming beyond what they are already compensated for by the government for regular classroom teaching." Evidence Action, blog post, July 5, 2016
- 114
GiveWell analysis of Deworm the World cost-per-treatment, October 2016
- 115
See GiveWell analysis of Deworm the World cost-per-treatment, October 2016.
- 116
- GiveWell analysis of Deworm the World cost-per-treatment, October 2016
- Note that this is only slightly lower than the cost estimate Deworm the World developed last year (see our 2015 review) because we have adjusted for the fact that Deworm the World's monitors found higher coverage than the government reported.
- Previously, Deworm the World estimated that the deworming program in Kenya cost around $0.40 per child treated: "We note that preliminary estimates of program costs in Kenya, which we can estimate with far greater precision, suggest programming costs there of about $0.40 per child." Evidence Action cover letter 2013, Pg 2.
- 117
Excluding the value of teachers' and principals' time, we estimate that Deworm the World's cost per treatment is $0.55 and that SCI's cost per treatment is $0.83. Supporting data and calculations are shown in GiveWell analysis of Deworm the World cost-per-treatment, October 2016 and our review of SCI. Additional notes:
- We estimate that Deworm the World's drug costs in Kenya are $0.05 per treatment. We believe that drug costs are lower for the program in Kenya than most of SCI's programs because SCI generally treats for schistosomiasis, but a relatively small proportion of deworming in Kenya includes schistosomiasis treatment, because relatively few areas have high enough prevalence.
- Deworm the World does more extensive prevalence surveying in Kenya than it does in other countries and than we believe SCI does; this may cause an overestimate of Deworm the World’s costs in other countries.
- "Please also note that our cost per child estimates include the costs of prevalence surveying (in Kenya and in India). The Kenya program has more surveying (including pre and post MDA testing annually) than does the India program (where we undertake baseline surveys and follow-up surveys). Where there are not prevalence survey costs each year, we amortize the costs over rounds." Grace Hollister, Deworm the World Director, email exchange with GiveWell, October 2015
- "For the NSBDP, three prevalence and intensity surveys for STH and one for schistosomiasis are expected to be or have been completed by KEMRI. The implementation costs of these surveys were divided among the program's expected duration of five years. Therefore, this model includes 1/5 of the total survey-associated costs." DtWI Kenya 2013-2014 cost per treatment data, Introduction sheet.
- We have not adjusted these data for inflation or changes in exchange rates over time.
- 118
- See further discussion here.
- Deworm the World adopted a different approach, citing JPAL CEAs in education 2011.
- 119
- See our discussion of the rationale and limitations of this estimate here.
- For comparison, Deworm the World previously estimated the value of in-kind support from the governments in Bihar and Rajasthan. Our understanding is that these estimates suggested in-kind cost of $0.31 per treatment in Bihar 2012 and $0.20 in Rajasthan 2012.
- For both states, Deworm the World shared detailed data behind its cost per treatment data. This included a sheet for imputed costs. We aggregated these line items, excluding expenses from non-government partners (e.g. WHO and UNICEF), and we believe the result largely accounts for teachers' time during training and deworming day. Treatments from DtWI cost per treatment summary 2013
- Bihar 2012: DtWI Bihar 2012 cost data details shows $5.21 million, which implies $0.31 per treatment, given 16,867,388 treatments. $5.21 million is converted from INR 276.1 million at 53 INR per USD (based on the source), and is a total of these line items:
- Master Trainers as Trainers in Block Level Trainings
- Master Trainers as Trainers in Sector Level Trainings
- Block Level Training
- Sector Level Training
- Honorarium: Teachers and Headmasters
- Honorarium: Health Department (Doctors and ANMs)
- Rajasthan 2012: DtWI Rajasthan 2012 cost data details shows $2.06 million, which implies $0.20 per treatment, given 10,132,535 treatments. $2.06 million is converted from INR 109.1 million at 53 INR per USD (based on the source), and is a total of 23 line items, with these 5 largest line items accounting for over 90% of the total:
- Teachers and Headmasters as Trainees
- Lady Supervisors and Anganwadi Workers as Trainees
- Block Level Training
- Honorarium: Teachers and Headmasters
- Honorarium: Anganwadi Workers
- 120
See GiveWell analysis of Deworm the World cost-per-treatment, October 2016
- 121
GiveWell analysis of Deworm the World cost-per-treatment, October 2016
- 122
- 123
- Evidence Action held about $1.1 million in unrestricted and uncommitted funding at the end of July 2016, and it expected to raise an additional $1.2 million in unrestricted funding over the remainder of 2016, although this estimate was quite rough. GiveWell analysis of Deworm the World financials - 2016
- In 2016: Evidence Action intended to commit $163,000 of its uncommitted unrestricted funding to Deworm the World. GiveWell analysis of Deworm the World financials - 2016
- In 2017: Evidence Action had tentative plans to commit another $600,000 to Deworm the World. GiveWell analysis of Deworm the World financials - 2016
- Excluding GiveWell-influenced donors and Evidence Action, Deworm the World estimated that it would raise $400,000 in general support that it could use for its 2017 budget year: "It is difficult to estimate what this number would be, given that all Evidence Action financial inflows have come from the time period since GiveWell recommended Deworm the World and we do not know exactly how much in individual donations can be attributed to GiveWell (since there could be some who do not self-identify as such). We are estimating to raise approximately $400k in general support funds for the program excluding GiveWell influenced donors and Evidence Action." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
- $163,000 + $600,000 + 400,000 = $1,163,000.
- We expect the remainder of Evidence Action's unrestricted funding to be allocated to its other programs, organizational investments, and reserves. Evidence Action is currently working on developing a reserves policy. Deworm the World staff, conversations with GiveWell, October 3-4, 2016 and GiveWell analysis of Deworm the World financials - 2016
- Evidence Action held about $1.1 million in unrestricted and uncommitted funding at the end of July 2016, and it expected to raise an additional $1.2 million in unrestricted funding over the remainder of 2016, although this estimate was quite rough. GiveWell analysis of Deworm the World financials - 2016
- 124
This is based on internal records of how much GiveWell-influenced donors gave to Deworm the World in the last year (as of early November 2016) when Deworm the World was on GiveWell's top charity list but was not the recommendation for marginal funding.
- 125
GiveWell analysis of Deworm the World financials - 2016, sheet 'DtW Unres Commit.'
- 126
Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 127
GiveWell analysis of Deworm the World financials - 2016, Sheet: GiveWell's ranked funding gaps, and Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 128
Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 129
Deworm the World has significant experience in India and believes it can hire quickly for the new positions. It is currently in discussions with four states, but believes it is likely that one of the discussions will not result in Deworm the World providing additional support to that state. Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 130
Three states at $300,000 per year for 3 years is $2.7 million. Deworm the World requested an additional $1.4 million to support the expansion of its national team. Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 131
Grace Hollister, edits to GiveWell's review, November 20, 2016
- 132
These estimates are based on the size of the target populations. Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 133
- Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
- "Target population would still likely be in millions –smaller than originally envisioned but still a considerable number of at-risk children" Grace Hollister, edits to GiveWell's review, November 20, 2016
- 134
Note that this is a slightly lower value than what Deworm the World estimated the province of Punjab would need in 2015, but all estimates are highly uncertain: "It is difficult to estimate how much funding will be needed, because Deworm the World has not worked in Pakistan before and it is still early in the planning stages. There is a rough funding need of $1.5-2 million per year in Punjab, which has a target population of 5.7 million children, and $500,000-800,000 per year in Sindh, which has a target population of 2.1 million children." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg. 8
- 135
"A multi-year funding commitment will be important to encourage the government to work with Deworm the World. Ms. Hollister would like to aim for a 5-year commitment and has had a 5-year budget created. While it is unlikely that Deworm the World will get an upfront 5-year funding commitment, and it may be difficult to reach an agreement with the government without having funding for all 5 years. Deworm the World could begin with a 3-year commitment and indicate its intention to stay for 5 years and find funding to fill the gap." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 8
- 136
- 137
Grace Hollister, edits to GiveWell's review, November 20, 2016
- 138
This is based on our internal records and Good Ventures' grants.
- 139
When we look at what Deworm the World has spent in unrestricted funding Jan 2014 - Jun 2016 and add to that the unrestricted funding that Deworm the World currently holds, we arrive at $17,531,665. GiveWell analysis of Deworm the World financials - 2016 Sheet: "2014-2016 expenses. Subtracting GiveWell's contributions of $15.3 million, we calculate that Deworm the World has raised approximately $2.2 million in unrestricted funding from other donors.
- 140
Our internal records show we influenced $771,159.81 to Deworm the World between 7/1/2013 and 6/30/2014, excluding Good Ventures. Good Ventures gave $1.5 million to Deworm the World in 2013 (see here). Together, that amounts to about $2.3 million.
- 141
GiveWell analysis of Deworm the World financials - 2016 Sheet: 2014-2016 expenses
- 142
Our internal records show GiveWell-influenced gave about $760,000 to Deworm the World between 7/1/2014 and 6/30/2015, excluding Good Ventures. Good Ventures gave $250,000 to Deworm the World in 2014 (see here). Together, that amounts to about $1 million.
- 143
GiveWell analysis of Deworm the World financials - 2016 Sheet: 2014-2016 expenses
- 144
See our 2015 review.
- 145
- Spending = $336,000 (in 2014) + $743,975 (in 2015).
- Commitments = $1.1 million
- Total raised from GiveWell-influenced donations = $2.3 million (2013 through 2014) + $1 million (2014 through 2015). Total we estimate Deworm the World raised from other donations over this time period: $1.4 million (two thirds of the $2.2 million we know Deworm the World raised from other donors).
- $2,179,975 / $4.7 million = .46
- 146
Our Salesforce records show GiveWell-influenced donors gave $1.2 million to Deworm the World between 7/1/2015 and 6/30/2016, excluding Good Ventures. Good Ventures gave almost $10.8 million to Deworm the World in early 2016 (see here). Together, that amounts to about $12 million.
- 147
January through June, Deworm the World spent $350,715. July through December, it expected to spend 841,834. GiveWell analysis of Deworm the World financials - 2016 Sheets: "2014-2016 expenses" and "2016 and 2017 budgets"
- 148
GiveWell analysis of Deworm the World financials - 2016 Sheets: DtWI Unres Commit
- 149
See the blog post linked above.
- 150
See our November 2014 review:
"The Deworm the World Initiative seeks an additional $1.3 million to support its activities in 2015 and 2016. DtWI expects to spend $377,000 of the $1.3 million (29%) it seeks on work related to expanding school-based mass deworming programs and funding related operating expenses (including impact evaluation related expenses). More specifically, these activities would be:
- $230,000: staff to support expansion in India, new countries, and related operating and evaluation expenses.
- $144,000: DtWI overhead. [These funds support DtWI as an organization but are not directly programmed (e.g., a portion of Alix Zwane's, the Executive Director of Evidence Action salary, Evidence Action financial staff, etc.).]
- $500,000: evaluation of new evidence-based programs that leverage deworming. We have limited detail about what this would entail. One idea that DtWI has investigated is the possibility of distributing bednets along with deworming pills in schools as an alternative distribution mechanism to national net distributions. Another is including hand-washing educational programming alongside deworming days. This line item includes $50,000 to support DtWI's evaluation of its hygiene and deworming program funded by Dubai Cares and $50,000 to enable DtWI to hire a senior epidemiologist.
- $230,000: staff to support evaluation of DtWI’s work in Kenya. This work is primarily funded by CIFF. DtWI believes that additional resources can improved significantly the quality of the analysis done regarding the cost effectiveness of breaking transmission.
- $170,000: implementation support for the integrated deworming, sanitation and hygiene education program in Vietnam, in partnership with Thrive Networks."
- 151
Also see the next footnote. Deworm the World told us that it had allocated unrestricted donations in the following way:
- $200,000: A prevalence survey in Pakistan
- $111,000: Support for the TUMIKIA and TakeUp studies
- $93,000: Support for a deworming program in Vietnam
- $12,000: A training materials project in India
- $100,000: Salaries for staff exploring work in new countries (intended to extend to 2016)
- $~550,000: Support for a program in Cross River, Nigeria.
Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013
- 152
How expected spending at the end of 2014 compared to how Deworm the World actually spent and committed funding in 2015:
- Expected staff spending: $230,000.
- Actual spending: We are unsure how much Deworm the World actually spent and allocated to staff. It is difficult to tell what portion of Deworm the World's 2015 costs were for staff because staff costs are built into country costs.
- Committed funding: $100,000 for salaries for staff exploring work in new countries (intended to extend to 2016).
- Expected overhead spending: $144,000.
- Actual spending: $239,447 on "Global" costs
- Committed funding: None
- Expected Vietnam spending: $170,000.
- Actual spending: $32,613
- Committed funding: $93,000
- Expected evaluations spending: $500,000.
- Actual spending: $17,059 (for the Tumikia study)
- Committed funding: 111,000 for support for the Tumikia and TakeUp studies.
- Expected Kenya spending: $230,000.
- Actual spending: $130,979
- Committed funding: None
- Note that Deworm the World also allocated substantial funding to opportunities that we had not previously expected, such as support for a new program in Nigeria. See previous footnote.
See the previous footnotes, Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013, and GiveWell analysis of Deworm the World financials - 2016 Sheets: "2014 - 2016 expenses" and "Expected vs. Actual spending"
- Expected staff spending: $230,000.
- 153
See our November 2015 review. Deworm the World intended to spend unrestricted funding on:
- A deworming program in Pakistan (~$8.4 million for a 3-year program: $2 million per year in Punjab and $0.8 million per year in Sindh)
- A deworming program in a new country (~$6 million for a 3-year program)
- A deworming program in Vietnam (~$2.6 million, unclear how many years)
- Reserves ($2.8 million)
- 154
- GiveWell analysis of Deworm the World financials - 2016 Sheets: "DtW Unres Commit" and "Expected vs. Actual spending"
- Another donor funded more of the Vietnam program than Deworm the World expected and the prevalence surveys in Vietnam revealed a need for a smaller program than expected. Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 155
Deworm the World staff, conversations with GiveWell, October 3-4, 2016
- 156
"Evidence Action launched formally in 2013 to scale programs with sustainable business models that have been proven to be effective so that they benefit millions of people.
Evidence Action leads and manages two programs incubated by Innovations for Poverty Action: Dispensers for Safe Water and the Deworm the World Initiative. We also run Evidence Action Beta where we are currently testing a number of other rigorously-evaluated interventions for scale-up." Evidence Action website, Who we are (November 2016)
- 157
"At the time of your visit we were and have remained actively engaged in correcting and converting our financial information for 2013, 2014, 2015 and 2016. As Jeff told Elie in August, in the last several months we embarked on a massive effort to upgrade our financial systems and practices. These include the implementation of a new global accounting system (Intacct), conversion of our books from cash to accrual, and the resolution of many outstanding financial issues from the IPA spinoff. Thus, we have been slow in responding to your questions.
As always, we appreciate GiveWell’s in-depth questions and the value you place on transparency. The continued inquiry reinforces our commitment to improve our finance and administrative capacity, and will strengthen our organizational ability to operate efficiently at scale. The combination of our effort to improve our financial data and management, and your queries have helped us discover past coding errors, and to refine our planning around internal funds transfers. Although we continue to review, examine and strengthen our records and systems, we are more confident in the accuracy of the information provided here in response to your questions." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
- 158
- One of the documents had an incorrect formula (adding in expenses rather than subtracting them), while another double counted about $750,000: "The version shared during the [meeting] reflected $750K for core support allocated to programs, as well as $1.3M in unrestricted fund transfers to programs. After realizing that we had double counted the $750K in core support, we eliminated it in the version sent on Oct 11." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
- Deworm the World's restricted funding on hand was about $2.5 million lower after corrections had been made. Our understanding is that about $1.7 million of that was funding that had been mislabelled as restricted, but was actually unrestricted, and the rest of the decrease was due to the fact that Deworm the World had actually already spent a portion of its restricted funding in India (this change increased Deworm the World's expenses figure by $500,000 and Evidence Action's starting revenue by approximately $200,000).
- "$1.7M in Deworming restricted funding was booked into our financial system as revenue against a current commitment instead of as new revenue. The coding has been corrected and this amount is now reflected in an increase in revenue on the report (cell D5) [...] $730K in expenses incurred in India between March and July 2016 were not included in the previous report - increasing total expenses by a like amount. Note that $187K in organization development costs allocated to the program in the form of indirect costs in past tables was backed-out (see second bullet under Evidence Action updates below) resulting in a net increase in expenses of $540K (cell D6)." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
- Deworm the World's revenue increased by $2.7 million after corrections. We believe this is from the $1.7 million that was initially mislabelled as restricted funding and an additional $1 million that was incorrectly allocated internally. Correcting the misallocation also caused Evidence Action's revenue to decrease by $1 million.
- "$1.7M in Deworming restricted funding was booked into our financial system as revenue against a current commitment instead of as new revenue. The coding has been corrected and this amount is now reflected in an increase in revenue on the report (cell D5). [...] $1.0M in Deworming program general support funds (what we had been calling "unrestricted" for deworming) received in 2015 and 2016 were coded inconsistently. These costs were correctly coded to the Deworming program but were assigned the wrong funder code. The coding has been correctly aligned, and this amount is now reflected in an increase in revenue on the report (also cell D5)." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
- Evidence Action's expenses increased $750,000 after corrections, to reflect that the funding had been used on organizational expenses (this was previously unreflected in the original documents): "The $750K was spent on one-time operational costs such as temporary accountants to convert our books from cash to accrual, transition to a new accounting system, separation costs from the incubator established in India, and relocation costs for the Nairobi office after our landlord ended the lease." Grace Hollister, Deworm the World Director, email to GiveWell, October 27, 2016
- Once these changes had been made, Evidence Action revised the amount of unrestricted funding it intended to allocate to Deworm the World this year, decreasing it from $350,000 originally to $163,000.
- 159
Originally, Deworm the World's financial documents indicated that Deworm the World had $6,363,718 in unrestricted funding available; after corrections, this figure was $10,969,610.
- 160
"Note that this is specific to CIFF; the END Fund grant is slated to end in 2018" Grace Hollister, edits to GiveWell's review, November 20, 2016
- 161
Grace Hollister, edits to GiveWell's review, November 20, 2016
- 162
GiveWell's non-verbatim summary of a conversation with END Fund staff, October 17, 2016
- 163
Grace Hollister, email to GiveWell, June 9, 2016
- 164
- "[In Nigeria] Deworm the World would ideally like to make a 3-year commitment in Cross River to increase the chance of government approval, increase stability, enable Deworm the World to establish a partnership with the government, and take steps toward institutionalization of deworming programs.
There is some inherent risk in multi-year commitments, but one year is not always enough time to build a new program that runs effectively, and governments would be reluctant to work with Deworm the World if they were limited to one year. [...] One-year commitments can be costly for Deworm the World because governments typically expect that programs will continue past the first year, and it can be difficult to find funding." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 7 - "[In Pakistan] A multi-year funding commitment will be important to encourage the government to work with Deworm the World. Ms. Hollister would like to aim for a 5-year commitment and has had a 5-year budget created. While it is unlikely that Deworm the World will get an up-front 5-year funding commitment, it may be difficult to reach an agreement with the government without having funding for all 5 years. Deworm the World could begin with a 3-year commitment and indicate its intention to stay for 5 years and find funding to fill the gap." GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015, Pg 8
- "[In Nigeria] Deworm the World would ideally like to make a 3-year commitment in Cross River to increase the chance of government approval, increase stability, enable Deworm the World to establish a partnership with the government, and take steps toward institutionalization of deworming programs.
- 165
- "Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage.
- GiveWell conducted a site visit to DSW in November 2012 and published notes. DSW 2012 GiveWell site visit
- 166
GiveWell analysis of Deworm the World financials - 2016, Sheet: "EA Unres Commit - guess"
- 167
"...many of the investments of unrestricted have substantial positive benefit for DtW, even as they are not specifically programmatic in nature. These include investment in financial capacity, and the transition of the Indian entity (which at the moment is exclusively working on deworming, though this may not be the case in the future)." Grace Hollister, edits to GiveWell's review, November 20, 2016
- 168
WHO, Summary of global update on preventive chemotherapy implementation in 2015, Pg 456-457, Table 1.
- 169
- WHO Weekly epidemiological record, 18 December 2015, Pg 707 (Table 1)
- In 2013 about 34% of all children in need of STH deworming received treatment. WHO Weekly epidemiological record, 6 March 2015, Pg 91 (Table 1)
- Older data: "In 2009, a total of 314.6 million children were reportedly dewormed, of whom 274 million required preventive chemotherapy. This corresponds to approximately 31% of all children in need of treatment." WHO STH treatment report, Pg. 34.