Deworm the World Initiative, Led By Evidence Action - 2013 Review

Deworming in India vs. Sub-Saharan Africa

There are two differences we discuss below: (a) the prevalence of particular worms and the prevalence of high intensity infections and (b) the existence of other neglected tropical disease programs that may treat the same condition.

Difference in worm prevalence and intensity

Key pieces of evidence that we discuss in our report on deworming (Miguel and Kremer 2004 and Baird et al 2012) are from a deworming experiment conducted in Kenya in the late 1990s. The participants in that study had relatively high rates of moderate-to-heavy infections of schistosomiasis and all three soil-transmitted helminths (STH).95

The situation in India is different. Schistosomiasis is not present in the vast majority of India,96 and those DtWI has helped treat (thus far) have significantly lower intensity infections than those treated in the Miguel and Kremer experiment in Kenya. In the table below, we compare the prevalence, moderate-or-heavy prevalence (using definitions from Miguel and Kremer 2004), and average eggs per gram of Bihar, Rajasthan, and the population in the Miguel and Kremer experiment.

Difference in pre-existing programs

India has an existing program to treat lymphatic filariasis (LF).103 Albendazole, the same drug used to treat STH, is sometimes used to treat LF.104 The treatment for LF in India is annual administration of DEC or DEC and Albendazole105 and the WHO recommends annual albendazole treatment for areas with 20-50% prevalence of STH.106 According to the Indian government's program website, the LF treatment program has high coverage across many states (e.g., 80-90% in states carrying out the program), including one of the three states in which DtWI is currently active.107 Therefore, it appears that in some cases the effect of DtWI's work may be to transition from annual to biannual treatment in appropriate areas, and we are unsure of how the impact of a such a transition would compare to the impact of transitioning from no treatment to some treatment.

How leveraged are DtWI contributions?

We have written before about the complexity involved in trying to understand leverage in charity, and below we lay out the relevant issues worth considering in the case of DtWI.

Donors to DtWI support only the costs paid by DtWI, which are less than 10% of the overall cost of the program. The role these funds play in the program is an important consideration in determining the cost effectiveness of donations.

The role DtWI funds play, and the resulting cost effectiveness, could range between:

• High leverage, high cost effectiveness. If DtWI leverages government funds that otherwise would not have been spent or would have been spent on a less valuable program by causing the government to run deworming programs or by making those programs significantly better than they otherwise would be, then a small contribution from DtWI has a large impact on the number of children receiving treatment and DtWI is highly cost effective.
• Small marginal contribution to the quality of the program. If DtWI's involvement doesn't cause more infected children to be reached, then DtWI’s contribution is more appropriately considered a small marginal contribution to the program that would have happened without its help. It is possible in that case that DtWI contributes little to increasing the number of children who receive treatment, but may improve the functioning of the program in less obvious ways.

We would guess that DtWI has played a role in increasing the likelihood that governments conduct deworming programs, while also contributing to the quality of programs. We also believe that it is very unlikely that DtWI was causally necessary for each of the programs it has assisted.

We believe that DtWI's leverage could reasonably range from less than 1x (i.e., programs would take place without DtWI's involvement) to 4-5x. We do not think we have a reasonable way of refining this estimate further, but this footnote provides individual GiveWell staff estimates of the leverage they would guess DtWI has.108 We plan to revisit this calculation annually.

Room for more funds

DtWI has told us it could effectively use $2-3 million to support its activities.109 DtWI's main need is for unrestricted funds to respond to unanticipated situations, including requests from governments for technical support. DtWI would also like the opportunity to explore how their programs might contribute to STH elimination, not only control, by expanding coverage and reaching some adults. Quantifying the marginal impact of these funds is extremely difficult.

DtWI aims to raise the following types of funding:

• Restricted funding to expand DtWI-led deworming programs into additional states in India
• Unrestricted funding to give it more flexibility in responding to situational needs and working capital to improve the organization’s stability and efficiency

Restricted funds for state-specific scale-up

DtWI has recently received substantial financial support to enable it to significantly expand its activities in India.

• In 2012, DtWI received a 5-year, $5.23 million grant from the United States Agency for International Development (USAID) to support its work in India. This grant is specifically earmarked to support DtWI expansion into new states.110
• DtWI is also in conversations with another funder with hopes of receiving $1 million in funding for state-specific expansion.

In sum, DtWI hopes to have $2.67 million available for state-specific scale-up in 2014. Based on past expenditures of approximately $220,000 to $483,000 per state,111 these funds should enable DtWI to expand into 4-6 more states.112

Unrestricted funding

Alix Zwane, Evidence Action’s Executive Director, told us that DtWI has a pressing need for unrestricted funds.113

These funds would support two goals:

• Give DtWI the flexibility to pursue new models of providing technical assistance to Indian states. For example, in 2013, Punjab approached DtWI for assistance in implementing a deworming program. According to DtWI, Punjab did not require DtWI’s standard, full engagement and DtWI considered providing less intensive consulting to Punjab. However, DtWI did not have the funds available to do so, and Punjab went ahead with deworming without DtWI’s assistance. DtWI has told us that it would like to explore this "consulting model" and hopes to allocate $200,000 in 2014 to explore this.114 Our understanding is that this consulting model is one example of a model DtWI could pursue with unrestricted funds, and with funds available it would be in a position to consider other opportunities as they arose.
• Provide DtWI with "working capital" enabling it to be a more robust organization. DtWI has told us that it believes working capital of approximately 10% of its overall budget would provide it with more stability and flexibility. DtWI was explicit that this 10% figure was not chosen with substantial analysis but is just intended to be a rough guide of what it believes would help. DtWI has not given specific examples of how it would use "working capital" funds but believes that some working capital would enable it to reduce the need to periodically focus on short-term, pressing fundraising and instead devote more effort to its core work. For example, DtWI told us that in 2012, its senior staff focused primarily on fundraising for a substantial period of time to raise the funds needed to continue its work in Bihar.115

Quantifying the marginal impact of such funds would largely be an exercise in guesswork. We could see a case that most of DtWI’s impact has come from sparking interest in deworming in a few initial states, which will lead to more interest over time (in Rajasthan, our impression is that the largest factor in the deworming program’s being started was media coverage of the program in Delhi116 ), and that further funds will be far less impactful. We could also see a case that by allowing flexibility, unrestricted funds allow substantial upside for DtWI and could be more impactful than the bulk of DtWI’s funds, which are restricted.

Deworm the World Initiative and Evidence Action

The Deworm the World Initiative is an initiative led by Evidence Action,117 run by Executive Director, Alix Zwane. Evidence Action includes a second program: Dispensers for Safe Water (DSW).118

Donations to DtWI, even if restricted to DtWI, would likely change the actions staff take to fundraise (i.e., which grants they pursue, what type of funding they ask for). Therefore, we do not believe donors can realistically donate to DtWI and support only DtWI and not DSW or Evidence Action more broadly. Dr. Zwane has told us that DSW has a funding gap of $2 million for 2014 and $5 million for 2015.119 Dr. Zwane has told us that she is currently in conversations with donors to close this gap, but none of these donations are assured.120

We recommend that donors recognize this fact and support Evidence Action with unrestricted gifts (to reduce the administrative burden placed on Evidence Action); Dr. Zwane has told us that Evidence Action recognizes that funds Evidence Action receives via GiveWell are due to DtWI and are aimed at supporting it to the extent possible.

As with all our recommendations, we will follow Evidence Action’s progress in the future and report publicly on how it spends all funds it receives.

The organization

We have limited observations on which to base an assessment of the organization:

• Evidence Action hired a new Executive Director, Alix Zwane, in 2013. Dr. Zwane has communicated extremely clearly and directly with us and given thoughtful answers to our critical questions, and has shared significant, substantive information. This represents a change from previous interactions with Deworm the World, in which we’ve struggled to get clear answers to our questions and to receive clearance to share information.
• We believe DtWI’s self-evaluation to have been less rigorous than that of our other top charities to date, though we are hopeful that it will improve going forward.

More on how we think about evaluating organizations at our 2012 blog post.

Sources

Uncited supplementary materials

• 1

  “In January 2007, the YGL launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs.” Harvard Business School Kenya Case Study A 2010 Pg 7.

• 2

  “Deworm the World (DtW) is funded through 2015-2016 in Kenya, thanks to the Children’s Investment Fund Foundation (CIFF), and the Ending Neglected Diseases (END) Fund.”
  Alix Zwane conversation June 4th 2013, Pg 1.

• 3

  “As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program.”
  @Alix Zwane Conversation June 4th 2013@, Pg 2.

• 4
  • “Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children.” Grace Hollister conversation June 19th 2013, Pg 1.
  • “DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment.” Grace Hollister conversation June 19th 2013, Pg 2.
  • “In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city.” Grace Hollister conversation June 19th 2013, Pg 3.
  • “DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year.” Grace Hollister conversation June 19th 2013, Pg 4.

• 5

  “Andhra Pradesh...
  Deworm the World [Initiative]’s contributions

  • Prevalence survey...
  • Operational support
    • Helped government develop operational plans and budgets
    • Coordinated cross-sectoral partners through the establishment of a State School Healthe Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS)
    • Coordinated drug donation made by Feed the Children
    • Designed a monitoring and evaluation (M&E) system
    • Created government tableau for community awareness
  • Trainings
    • Conducted a master training session for program
    • Designed training cascade for the master trainees to train the rest of the implementors
    • Designed training materials
    • Developed materials and campaigns for community sensitization

  …

  Bihar…
  Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andrha Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO…

  Delhi…
  In addition to the standard contributions (see Andrha Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children.

  Rajasthan…
  DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO.” @Grace Hollister Conversation June 19th 2013@, Pg 1-4.

• 6

  Data summary and primary sources in DtWI cost per treatment summary 2013, Sheet DtWI v partners. Includes costs from three rounds of deworming: Bihar 2012, Delhi 2012, and Rajasthan 2012. Excludes data from Bihar 2011 because we do not have a budget breakdown for DtWI for that deworming round.

  Descriptions of the categories are available in DtWI cost narrative 2013.

  DtWI notes that these campaign-oriented costs don't include any costs of running DtWI as a whole where they don't pertain to a particular campaign.

• 7

  Details on DtWI’s timeline in the states in which it has worked are included in the notes from our conversation with DtWI Director Grace Hollister.

  Since the conversation with Ms. Hollister, we have learned of the following recent and upcoming deworming rounds that DtWI is supporting:

  • Bihar: The third round is now scheduled for January 2014.
  • Delhi: conducted a second round of deworming with DtWI’s assistance on October 3rd, 2013. Results are pending.
  • Rajasthan: the October 2013 round of treatment was underway as of this writing.

  "Bihar Round 3 is scheduled for January 16, 2014." Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013.
  "Delhi round 2 occurred Thursday October 3rd, 2013, and Rajasthan round 2 occurred October 15th, 2013." DtWI 2013 GiveWell site visit, Pg 12.

• 8

  “Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success.

  Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action.” Evidence Action launch announcement 2013

• 9

  Evidence Action 2014 budget

• 10

  Evidence Action 2014 budget

• 11

  “INDIA: Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore).” World Schistosomiasis Risk Chart 2012, Pg 3.

• 12

  "Prevalence and intensity mapping to target treatment is what is recommended by the WHO and what is encouraged by DtW." Alix Zwane conversation June 4th 2013, Pg 3.

• 13

  Delhi:
  DtWI Delhi 2012 prevalence survey report

  • School and participant selection: "Pots were distributed to 5240 children aged 1-20. The sample was stratified into three groups MCD schools (1600), Government Schools (1600) and Slums (2040) and weighted in its distribution according to the populations and numbers of schools and slums in each district (and zone where appropriate). At all sites 40 children were sampled (with a target of 30 samples returned from each site and a total sample size of 3870)." Pg 1. We are unsure of whether the 40 students in each school were selected randomly in Delhi or by some other process.
  • Participation rate: Table One, on Pg 1, shows that 3668 out of 5240 pots were returned, a participation rate of 70%. In addition, "Of the 3668 children remaining, 441 were dropped from the data set due to missing parasitological data and a further 3 who lacked age data," leaving a sample of 3224 out of 5240 (62%). Pg 2.
  • The report did not provide details of the technique used to analyze the samples.

  
  
  Bihar
  DtWI Bihar 2011 prevalence survey report

  • Selection of districts and schools: "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected from the state schools database." Pg 9. For more details see DtWI Delhi 2012 prevalence survey design
  • Selection of students: "Within each school 65 children aged 6 and above representing both sexes equally from class 1 to 6 were randomly selected, class by class, using random number tables." Pg 9.
  • Technique: "Screening of infection for STH was based on a double Kato-Katz smear of 41.7 mg prepared from fresh stool samples." Pg 11.
  • Participation rate: "From a total of 1,281 school children registered in the survey and provided with pots, 1,159 returned samples." (90.5%) Pg 11.

  
  
  Rajasthan:
  DtWI Rajasthan 2012 prevalence survey report

  • District selection: "Thirteen districts were randomly selected and compared with secondary data collected for all districts. This information was circulated to the task force constituted to oversee the prevalence survey, who noted a good spread of key factors expected to influence worm prevalence: elevation, population density, and climate.” Pg 6.
  • School selection: “Four schools were randomly selected per district from a complete list provided by the RCEE, excluding those with enrolment lower than 30 students, for a total of 52 schools… The targeted anganwadis were drawn from the one or two anganwadis nearest to the randomly selected school, as a comprehensive database of anganwadis was unavailable.” Pg 6.
  • Participant selection: “35 children (aged 6-15) were selected per school. Selection was undertaken by having children from each grade line up, separated by gender, and selecting students randomly from each queue while attempting to have a 50-50 male-female split. From each anganwadi, 24 children (aged 2-5) were randomly selected and invited to participate.” Pg 7.
  • Technique: “Samples were analyzed by Kato Katz technique using a World Health Organization (WHO) standard template.” Pg 9.
  • Participant selection:The report did not provide details on participation rate, but DtWI reports that it was approximately 56% and is verifying that number. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 14using WHO definitions of moderate intensity:
  • "Ascarsis [sic] lumbricoides: 1 - 4,999 epg: light-intensity infections; 5,000 - 49,999: moderate-intensity infections; >50,000 epg: heavy intensity infections
  • Trichuris trichiura: 1 - 999 epg: light-intensity infections; 1,000 - 9,999: moderate-intensity infections; >10,000: heavy-intensity infections
  • Hookworms: 1 - 1,999: light-intensity infections; 2,000 - 3,999: moderate-intensity infections; >4,000: heavy-intensity infections"

  WHO soil-transmitted helminthiases 2012, Pg viii.

• 15
  • There is a pre-existing LF program that provides albendazole to a large portion of the population in Bihar annually.
  • Based on timing of the decision to deworm in Bihar, there was only time to collect prevalence survey from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet DtWI's typical standard of quality.
  • Before the DtWI-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report
  • Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness.

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 16
  • "A clustered, stratified and randomized sample is suggested for the Delhi prevalence study." DtWI Delhi 2012 prevalence survey design, Pg 1.
  • DtWI chose to encourage and support deworming in Delhi despite low prevalence since it plays a leadership role in India and might encourage other states to start deworming programs.

• 17
  • The prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. DtWI Rajasthan 2012 prevalence survey report.
  • "Results also reflect underlying uncertainty in the data, which one might expect to be very high as 20% of the schools sampled examined less than 10 children. This poor response introduces a strong possibility of a bias, which based on the common assumption that risk is higher in non-enrolled children would likely result in an underestimate of prevalence.” DtWI Rajasthan 2012 prevalence survey report.
  • "Areas in western Rajasthan are predicted to have a higher risk of Ascaris infection compared to eastern parts of the state. Figure 5 shows the probability that the predicted prevalence of Ascaris exceeds 20%, based on a model including EVI and altitude. While there is a high probability that areas in western Rajasthan will require at least annual MDA, there is likely to be a lower prevalence of Ascaris throughout the east of Rajasthan." DtWI Rajasthan 2012 prevalence survey report, Pg 14.
  • "The climatic thresholds for hookworm transmission limits in Asia have been previously modelled using an aridity index, mean and maximum land surface temperature (LST). The map of these limits is shown in Figure 6, which categorises areas as unstable transmission (Mean LSTH 5-10°C, Aridity Index <0.2) or beyond transmission (Max LST >42°C, Mean LSTH <5°C, Aridity Index <0.03). These limits are based on experimental data showing that the development of infectious stages of hookworm ceases at 40° Celsius, and observed relationships between prevalence and environmental covariates using data from the Global Atlas of Helminth Infections. Based on these limits, the prevalence of hookworm in Rajasthan would be expected to oppose that of Ascaris and be more prevalent in the east of the state. All hookworm positive cases identified from this survey originated from areas within transmission limits." DtWI Rajasthan 2012 prevalence survey report, Pg 15-16.
  • DtWI concluded, "Based on Ascaris prevalence levels in the surveyed districts, observed associations with environmental data, hookworm transmission limits and the possibility that data are an underestimate of the true prevalence, annual deworming throughout the state is recommended." DtWI Rajasthan 2012 prevalence survey report, Pg 17.

• 18As discussed in a footnote above, the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence.
• 19This figure represents just Ascaris and Trichuris prevalence, not hookworm, as the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence.
• 20As discussed in a footnote above, the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence.
• 21This figure represents just Ascaris and Trichuris prevalence, not hookworm, as the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence.
• 22

  Preventive chemotherapy in human helminthiasis 2006, Pg 41.

• 23

  With respect to prevalence greater than 50%, The WHO adds that "If resources are available, a third drug distribution intervention might be added. In this case the appropriate frequency of treatment would be every 4 months" Preventive chemotherapy in human helminthiasis 2006, Pg 41.

• 24
  See:
  • DtWI Bihar 2011 coverage data
  • DtWI Bihar 2012 coverage data
  • DtWI Delhi 2012 coverage report
  • DtWI Delhi 2012 coverage data by school
  • DtWI Delhi 2012 coverage data by anganwadi
  • DtWI Rajasthan 2012 coverage data for schools
  • DtWI Rajasthan 2012 coverage data for anganwadi

• 25DtWI Bihar 2011 coverage data
• 26DtWI Bihar 2012 coverage data
• 27DtWI Delhi 2012 coverage report
• 28DtWI Rajasthan 2012 coverage data for schools DtWI Rajasthan 2012 coverage data for anganwadi
• 29

  In Rajasthan, for example, there are 257 blocks for 80,000 schools: "There are 257 blocks in Rajasthan." DtWI 2013 GiveWell site visit, Pg 3. "There are 80,000 public schools in Rajasthan." DtWI 2013 GiveWell site visit, Pg 4.

• 30
  • In Bihar in 2011, monitors visited a randomly selected set of schools on deworming day and mop up day. In one of the three phases of the program, monitors were asked to record the number of children dewormed on that day by asking the teacher conducting the deworming and checking the school’s forms. They were not asked to compare two sources of data. See @DtWI Bihar 2011 monitoring form for deworming day@, DtWI Bihar 2011 monitoring form for mop-up day, DtWI Bihar 2011 monitoring data for deworming day, and DtWI Bihar 2011 monitoring data for mop-up day. DtWI shared the data from this monitoring process with us, but there appears to be a problem with the data we received (the data in the column for “How many children were dewormed today” is yes/no rather than numerical). DtWI Bihar 2011 monitoring data for deworming day, worksheet "Phase 3".
  • In Bihar in 2012, monitors visiting randomly selected schools after the deworming days were asked to record the number of children dewormed according to the class registers and according to the school summary report. The class registers were available for examination in “almost all schools,” while the summary form was available in only 69% of schools. DtWI Bihar 2012 monitoring report, Pg 11.
  • In Delhi in 2012, monitors visiting randomly selected schools after the deworming days were asked to record the number of children dewormed according to the class registers and according to the school summary report. Data was available for comparison for 65% of classes visited. DtWI Delhi 2012 monitoring data
  • In Rajasthan in 2012, monitors visiting randomly selected "independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation." DtWI Rajasthan 2012 monitoring report, Pg 1.

• 31
  • In the monitoring report on Bihar’s 2012 deworming day, the median number of children dewormed per class according to the class registers was 35, while the median number according to the school summary forms was 38. DtWI Bihar 2012 monitoring report. Our own analysis finds a median of 38 for both, an average of 49 according to the class registers and 50 according to the summary reports, and an inflation rate of 1.5%. (Data missing for 31% of classes visited.)
  • In Delhi in 2012, the median number of children dewormed per class according to the class registers was 88, while the median number according to the school summary forms was 89.5. The average number from the registers was very slightly higher than from the summary forms (128 versus 127). The inflation rate on the summary forms was 1%.(Data missing for 38% of classes visited.)
  • In Rajasthan in 2012, the median number of children dewormed per class according to the class registers was 17, while the median number according to the school summary forms was 18. The average number from the registers was 25 and from the summary forms was 28. The inflation rate on the summary forms was 11.3%. The vast majority of the inflation comes from schools in which the inflation rate is several hundred percent. (Data missing for 20% of classes visited.)
  • Data summarized in DtWI class register audits 2013.
  • DtWI explains, "In those small percentage of cases where [class summaries report significantly higher coverage numbers than school attendance registers], we believe it is not monitor error, but a failure by the school teachers to tick the attendance register. On deworming day Round 2 in Rajasthan, DtWI staff observed that a teacher had directly filled out the school summary form with the number of children dewormed rather than entering it on the register. We speculate, based on our own experiences monitoring the process across all the states, that the main reason for this difference is the teachers either partially fill out the register and then go directly to the summary form, or they simply do not fill out the register on one of the two days. This is a common problem that we try to rectify through the training. In Bihar Round 2, similar to Rajasthan Round 1, there were some schools where the summary form totals were greater than the attendance registrar ticking. Given that the percentage of these errors is not large, and our own staff’s experiences of visiting schools where attendance registers were not ticked (rather the school form was filled out that day itself), we do not find it surprising that this discrepancy exists. We do not believe this discrepancy indicates a systematic bias in the reported deworming numbers." Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 32
  • Bihar 2011:
    • Process: “Moreover, independent monitors were mobilized in each block (560 total) to visit 5% of all government schools on deworming and mop-up days to ensure that an adequate quantity of drugs were available, trained teachers were administrating drugs according to protocol, and community sensitization initiatives were successful in mobilizing non-enrolled children to attend. One auditor per district was in charge of supervising monitors to ensure quality monitoring of all schools.” DtWI Bihar 2011 program report, Pg 9.
    • What we’ve seen: Monitoring forms used for data collection and full monitoring data from deworming day and mop-up day. We have not seen a summary of this data and have not analyzed it ourselves.
  • Bihar 2012:
    • Process: “Independent monitors visited a randomly selected sample of schools over four days – Deworming Day, Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 1216 schools (1.75% of 69,299 schools in Bihar) targeted for monitoring. From each of the 38 districts in Bihar, 2 blocks were selected by simple random sampling, for a total of 76 blocks. 2 clusters were randomly selected from each of the 76 blocks, excluding clusters with fewer than 8 schools. In each of these 152 clusters, 8 schools were randomly selected for a total of 1216 schools. Each monitor was assigned a block comprising 16 schools to be monitored. From this list of 16 schools, the monitor could visit any 4 schools on Deworming Day, another 4 on Mop-Up Day, and 8 more schools over the two Coverage Validation days. Out of the total sample of 1216 schools, monitors were able to survey 1196 schools comprising 302 schools on Deworming Day, 296 on Mop-Up Day, and 598 on Coverage Validation days.” DtWI Bihar 2012 monitoring report, Pg 1.
    • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Bihar 2012 monitoring report).
  • Delhi 2012:
    • Process: “Monitoring visits were conducted by senior government officials. Random site visits by independent auditors occurred at a subset of training sessions and participating institutions. Coverage validation engaged independent auditors who conducted random site visits at a representative sample of schools and anganwadis to validate coverage statistics. This source of information was carefully compared with programme reports collated from each school and anganwadi to arrive at an accurate assessment of programme coverage.” DtWI Delhi 2012 program report, Pgs 7-8.
    • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Delhi 2012 program report, Pgs 17-22).
  • Rajasthan 2012:
    • Process: “In order to evaluate the efficacy of the deworming protocol and process, independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 990 schools (1.1% of 90,488 schools in Rajasthan) targeted for monitoring. From each of the 33 districts in Rajasthan, 2 blocks were selected by simple random sampling. In each of these 66 blocks, 15 schools were randomly selected for a total of 990 schools. One monitor was assigned to each of these blocks. From the list of 15 schools in a block, the monitor could visit any 3 schools before Deworming Day for Preparation Monitoring, another 2 schools on Deworming Day, 2 more schools on Mop-Up Day, and 4 more schools over the two Coverage Validation days. The remaining 4 schools served as a buffer in case a particular school could not be visited. Hence, the actual sample size was 726 schools (0.8% of all schools).” DtWI Rajasthan 2012 monitoring report, Pg 1.
    • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Rajasthan 2012 monitoring report).

• 33

  DtWI Rajasthan 2012 monitoring report

• 34

  DtWI Bihar 2011 program report, Pg 9.

• 35

  “A multi-stage sampling strategy was used to select the 1216 schools (1.75% of 69,299 schools in Bihar) targeted for monitoring.” DtWI Bihar 2012 monitoring report, Pg 1.

• 36

  Total schools: “The programme targeted 3,032 schools and 9,934 anganwadis [pre-schools].“ Pg 11.
  Deworming day: “This data analysis is based on 45 schools and 15 anganwadis." DtWI Delhi 2012 program report, Pg 17.

  Mop-up day: “This data was collected from 49 schools and 14 anganwadis." DtWI Delhi 2012 program report, Pg 19.
  Coverage validation: “This data was collected from 80 schools and 87 anganwadis.” DtWI Delhi 2012 program report, Pg 20.

• 37

  “A multi-stage sampling strategy was used to select the 990 schools (1.1% of 90,488 schools in Rajasthan) targeted for monitoring.” DtWI Rajasthan 2012 monitoring report, Pg 1.

• 38

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

  “On deworming days, district-level officials from both the Health and Education Departments visited randomly selected schools to confirm that children were being dewormed in accordance with protocols. Both government monitors and the existing cadre of external monitors of the WHO’s polio programme were trained to monitor the school-based deworming programme in Bihar.” DtWI Bihar 2011 program report, Pg 9.

• 39

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 40

  "Delhi: For Round 1, we used government officials and monitors from the Health Department who had worked as monitors in the polio program (mostly medical doctors)." Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

  “Monitoring visits were conducted by senior government officials. Random site visits by independent auditors occurred at a subset of training sessions and participating institutions. Coverage validation engaged independent auditors who conducted random site visits at a representative sample of schools and anganwadis to validate coverage statistics.” DtWI Delhi 2012 program report, Pg 7.

• 41

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 42

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

  "In order to evaluate the efficacy of the deworming protocol and process, independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation." DtWI Rajasthan 2012 monitoring report, Pg 1.

• 43DtWI Bihar 2012 monitoring report
• 44DtWI Delhi 2012 monitoring data
• 45DtWI Rajasthan 2012 monitoring report
• 46Some of the monitoring forms we received are not translated into English; we have not checked whether the questions were phrased the same in each round of treatment, or whether the translations to English are accurate. Sample questions here are taken from one of the monitoring forms with English translations.
• 47Monitors asked the principal of each school whether they had received the deworming tablets or not.
• 48DtWI Rajasthan 2012 monitoring form pre-deworming day, Pg 2.
• 49DtWI Rajasthan 2012 monitoring form pre-deworming day, Pg 4.
• 50DtWI Delhi 2012 monitoring form deworming day, Pg 1.
• 51DtWI Delhi 2012 monitoring form deworming day, Pg 4.
• 52Monitors were instructed to visit one or more classrooms while they were administering the deworming tablets and answer a handful of questions about the process. The monitors were not required to watch the entire process for each class visited, or every class in each school, so when the monitors report that they did not observe any sick children receiving the tablet, they are not reporting that no sick children received the tablet in that class or school.
• 53DtWI Delhi 2012 monitoring form deworming day, Pg 5.
• 54DtWI Rajasthan 2012 monitoring form coverage day, Pg 5.
• 55DtWI Rajasthan 2012 monitoring form coverage day, Pg 5.
• 56

  DtWI Bihar 2011 monitoring data for deworming day

• 57

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 58

  "National funds for deworming are part of the WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12.

  "Wherever possible, DtWI works to get deworming added as a line item in the budget. Otherwise, it is difficult to ensure that long-term funding will exist for the program." DtWI 2013 GiveWell site visit, Pg 10.

  "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.

• 59

  Alix Zwane conversation August 30th 2013

  Assam has also announced its intentions to move forward with its deworming program in March 2013: "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. As reported by NRHM, Assam any [sic] school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam mid day meal report 2013, Pg 11.

  According to DtWI, "No schools in Assam have been covered under the School Health Program to provide deworming in 2012/13. The programme was then re-scheduled to begin in March 2013. No evidence of this actually having taken place." DtWI Assam research 2013, Pg 1.

  "[DtWI] are also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2.

• 60

  "DtW has been involved in deworming programs in four different states, and is still actively involved of three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement." Grace Hollister conversation June 19th 2013, Pg 1.

  "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1.

  "In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J-PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there. In August a memorandum of understanding (MoU) was formalized between DtW and the relevant players in Bihar (School Health Society Bihar and Bihar Education Project Council) for program implementation." Grace Hollister conversation June 19th 2013, Pg 2.

  "DtW leveraged its networks to engage the support of the Minister of Health, Women, and Child Welfare [in Delhi]. Education stakeholders were also brought in so that the program could reach beyond preschool to older grades as well. DtW conducted a great deal of advocacy, maintaining continuous interactions and significant support among all relevant departments to bring the program to fruition Due to the complicated agency system, there were many other government stakeholders as well, requiring a high degree of coordination by DtW to ensure program objectives could be achieved." Grace Hollister conversation June 19th 2013, Pg 3.

  "After the deworming program launched in Delhi, Rajasthan saw the results generated by the DtW- supported program in Delhi and committed to doing a deworming program, allocating funding for it in their budget. In March 2012 they brought DtW in to help. In this case, the state already knew what it wanted and already had a school health program. They sought DtW's technical expertise, mapping ability, general program support, stakeholder coordination, etc. DtWI coordinated signature of a MoU between the Departments of Women and Child Development, Education, and Health, UNICEF and DtW to guide program implementation, and helped establish of a technical secretariat housed within the Education Department." Grace Hollister conversation June 19th 2013, Pg 3.

  "DtW has worked in Delhi, Bihar, Rajasthan, and Andhra Pradesh (AP).
  • In Bihar, according to documentation provided by DtW, the program began because of previous Poverty Action Lab (J-PAL) work in Bihar. DtW also noted that Rajasthan approached them because of the success (as highlighted in the media) of the Delhi program. DtW supported a pilot program and prevalence survey in AP, and though DtW didn't continue to engage with AP, the AP government seems to have recently conducted a school-based deworming program.
  • More recently, Punjab approached DtW, but DtW didn't have the funding to move forward with Punjab, and Punjab is now implementing deworming on its own.

  It's not possible to be certain whether or not DtW has caused deworming to happen that otherwise would not have, but it's important to note that DtW took the opportunities presented to help run better programs where possible as well as improve data quality and did so on a limited budget. They are also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2.

• 61

  "Keeping in view the problem of worm infestation of children in Assam it has been decided to conduct two rounds of De-worming every year in the month of March and September. The first round will be conducted during March’ 2010. On every Wednesday (VHND/Immunization Day) medicine for De-worming will be given to all children between the age group 1 to 5 years along with vitamin A supplementation." Assam 2010 guidelines for deworming, Pg 1.

  976,192 "children below 5 years provided vitamin A syrup" as of November/December 2010 Assam reproductive and child health 2011-2012, Pg 73.

  The National Rural Health Mission reported the progress for 2010-2011 as 92,957 "Students given IFA/ de-worming tablets". Assam state programme implementation plan 2011-2012, Pg 21.

  Assam is "supposed to provide deworming to all children, but AWC services provided in Assam are some of the worst performers for deworming coverage."
  DtWI Assam research 2013, Pg 1.

• 62

  "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. NB. As reported by NRHM, Assam any school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam mid day meal report 2013, Pg 11.

• 63

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 64DtWI 2013 GiveWell government interviews
• 65

  "If DtWI had not been there, how would the deworming program be different?

  • Since DtWI have already implemented deworming programs in Delhi and Tamil Nadu, they brought experience.
  • If a state government decides to do something, nothing is impossible. However, DtWI's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have DtWI’s experience, there could have been more problems.
  • The government received important support from DtWI, particularly with distributing the deworming tablets, implementing the trainings, and developing training materials."

  "Have you ever wanted to do a program but been unable to find a technical partner?

  • Ms. Gupta is still looking for a technical partner on specific issues. For example, Rajasthan has a large population with special needs. It has generally been able to find partners for helping the visually impaired, but very few organizations work with autistic children, children with cerebral palsy, or children with mental retardation. If there were a partner to support the Rajasthan government in this area, this would be very welcome. The Rajasthan government has funds for this type of program, but are looking for a technical partner.
  • An NGO called Sight Savers works with blind children. Before Sight Savers, she was not aware of problems with low vision. Many children need magnifying glasses and large print books. Sight Savers helped them to identify doctors and hospitals to work with such children."

  DtWI 2013 GiveWell government interviews, Pg 7-9.

• 66

  "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2.

  "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2.

• 67

  "Wherever possible, DtWI works to get deworming added as a line item in the budget. Otherwise, it is difficult to ensure that long-term funding will exist for the program." DtWI 2013 GiveWell site visit, Pg 10.

  "Recently, [Bihar] added the deworming program as a line item in the state budget, which makes the funding for deworming programs more secure." DtWI 2013 GiveWell site visit, Pg 2.

• 68

  Professor Devesh Kapur Biography 2013

• 69

  Devesh Kapur conversation October 14th 2013

• 70

  "Some school health programs, such as the WIFS program in Delhi, are rolled out too quickly and their trainings are carried out poorly. Then they receive negative media attention because of students’ adverse reactions to treatments. Negative media can cause delays or cancellation of school health programs. The WIFS program has also received negative media attention in the states of Haryana and Odisha." DtWI 2013 GiveWell site visit, Pg 10.

• 71

  "There have been very few adverse reactions to deworming treatments in DtWI-supported states because of the quality of DtWI’s trainings." DtWI 2013 GiveWell site visit, Pg 10.

• 72

  "DtWI's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have DtWI’s experience, there could have been more problems." Veenu Gupta, Principal Secretary to School Education Department, Government of Rajasthan DtWI 2013 GiveWell government interviews, Pg 7.

• 73

  "DtWI has helped to improve deworming trainings. Last year, DtWI helped to develop the content for the trainings. DtWI made the content more concise and easy to understand than the government would have done on its own. DtWI also improves trainings by tracking when people do not show up to trainings and following up with them and by determining aspects of training that could be improved in future years." Girish Bharbwag, Nodal Officer in Rajasthan DtWI 2013 GiveWell government interviews, Pg 4.

• 74

  The District Coordinator position, for example, is not always staffed in other school based health programs. "The state government paid for District Coordinators for the hand washing program in 2008. However, the payment for this position was low. In the first year, there were 15-16 independent monitors. This year, there are only 7 District Coordinators. The government is finding it difficult to fill these vacancies." DtWI 2013 GiveWell government interviews, Pg 5.

  "District Coordinators (temporary DtWI employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems.
  The presence of District Coordinators, combined with the independent monitors hired by DtWI that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 4.

• 75

  "The presence of District Coordinators, combined with the independent monitors hired by DtWI that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 5.

• 76

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 77

  "Single-dose oral therapies can kill the worms, reducing ... infections by 99 percent ... Reinfection is rapid, however, with worm burden often returning to eighty percent or more of its original level within a year ... and hence geohelminth drugs must be taken every six months and schistosomiasis drugs must be taken annually." Miguel and Kremer 2004, Pg 161.

• 78

  Devesh Kapur conversation October 14th 2013

• 79

  Total government costs from Bihar 2011, Bihar 2012, Delhi 2012, Rajasthan 2012 are $14,253,193.
  Total all-inclusive costs from the same programs are $15,935,050.
  See DtWI Bihar 2011 cost data, DtWI Bihar 2012 cost data, DtWI Delhi 2012 cost data, DtWI Rajasthan 2012 cost data
  See What do you get for your dollar? Section for more details.

• 80
  • "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2.
  • "District Coordinators (temporary DtWI employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by DtWI that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and DtWI’s tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. DtWI called a random sample of 8,000 schools.
    The prevalence survey would not have happened without DtWI’s support." DtWI 2013 GiveWell government interviews, Pg 5.

• 81

  "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively. It is possible that DtW brings its expertise to the table and leverages these available resources." CIFF conversation September 10th 2013, Pg 2.

• 82

  "RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3.

• 83

  "[The Nodal Head Master said] that most aspects of the program are excellent, but he had 2 suggestions:
  - Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so)
  Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6.

• 84

  DtWI provided breakdowns by cost category for round 2 in Bihar and the first rounds in Delhi and Rajasthan. For Bihar round 1, it provided total costs covered by each DtWI and the government, but did not provide a breakdown by cost category. For that reason, we include Bihar round 1 in the overall cost analysis, but exclude it from the breakdown by cost category.
  Numbers in the below table are calculated in DtWI cost per treatment summary 2013, Sheet DtWI v partners.

• 85"Deworming Day Support: School-based deworming programs require the time of teachers, principals and government officials. The time involved for each type of personnel to prepare for deworming, conduct training and/or sensitization, and record and report results is estimated. This time, along with estimates of the number of government officials and teachers involved and their respective average salaries, is used to estimate this cost component of the cost. These costs reflect the large number of schools involved in this program." DtWI cost narrative 2013, Pg 2.
• 86"Training: Training costs that were not incurred directly by the Deworm the World Initiative were imputed from market rates. Training material costs are based on estimates from experiences with printing in the state. Honoraria and travel allowances for training participants are based on estimates of daily stipends for trainers and attendees, and associated travel allowances. Venue related costs are estimated based on costs of venues of similar size across the state." DtWI cost narrative 2013, Pg 1.
• 87"Drugs: a. Procurement: In cases where donated drugs were provided to the b. Drug packaging, testing, and distribution: Direct costs incurred for programs at no cost, the value of those tablets is imputed based on market rates for those tablets in India at the time. When the Government directly procured the drugs, those costs were calculated on the basis of discussion with government officials and estimates based on market rates. These activities are included, along with in-kind support such as personnel for drug repackaging, for which market based estimates of those costs are used." DtWI cost narrative 2013, Pg 1.
• 88Program Planning and Management: The costs of overall programmatic support provided by governments are estimated based on market estimates of the salaries and time of the personnel involved." DtWI cost narrative 2013, Pg 2.
• 89"Public Awareness and Mobilization: Media campaign costs incurred by the government and partners are estimated based on market rates for TV, radio, electronic, and print media advertising across the state. Information on the number of the different types and modes of advertising was collected from government sources for these calculations." DtWI cost narrative 2013, Pg 1.
• 90"Prevalence Survey: These include all direct costs involved in the execution of worm prevalence and intensity surveys conducted in the field, as well as imputed costs for services provided in kind by the government. The latter may include the provision of laboratories, manpower in the form of technicians, and training support in terms of venue and manpower." DtWI cost narrative 2013, Pg 1.
• 91"Monitoring and Evaluation: a. Training of monitors: Where the government provided trainers, training materials and/or training facilities, these costs are estimated based on market rates. b. Monitoring personnel: Where monitors were provided by the government, the cost of their time and travel allowances are estimated. Costs associated with time taken by government officials for reporting purposes or collecting forms, is imputed." DtWI cost narrative 2013, Pg 1-2.
• 92

  For example, in order to account for the cost of centralized organizational costs such as fundraising expenses, DtWI increases its cost estimate for each campaign by a flat percentage. DtWI Rajasthan 2012 cost data, DtWI Delhi 2012 cost data, DtWI Bihar 2012 cost data.

• 93

  DtWI cost per treatment summary 2013, Summary.

• 94

  "We note that preliminary estimates of program costs in Kenya, which we can estimate with far greater precision, suggest programming costs there of about $0.40 per child." Evidence Action cover letter 2013, Pg 2.

• 95

  Details on our Reanalysis of the Miguel and Kremer deworming experiment page.

• 96

  "…absent from most of the country, risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1.

  "INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3.

• 97using Miguel and Kremer 2004 definitions of moderate intensity, which are not the same epg levels as WHO definitions for all worms
  • "Following Brooker, Miguel, et al. (2000), thresholds for moderate infection are 250 epg for Schistosomiasis. mansoni and 5,000 epg for Roundworm, the WHO standards, and 750 epg for Hookworm and 400 epg for Whipworm, both somewhat lower than the WHO standard." Miguel and Kremer 2004, Pg. 167.

• 98
  • There is a pre-existing LF program that provides albendazole to a large portion of the population in Bihar annually.
  • Based on timing of the decision to deworm in Bihar, there was only time to collect prevalence survey from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet DtWI's typical standard of quality.
  • Before the DtWI-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report, Pg 9.
  • Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness.

  Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 99
  • The prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. DtWI Rajasthan 2012 prevalence survey report.
  • "Results also reflect underlying uncertainty in the data, which one might expect to be very high as 20% of the schools sampled examined less than 10 children. This poor response introduces a strong possibility of a bias, which based on the common assumption that risk is higher in non-enrolled children would likely result in an underestimate of prevalence.” DtWI Rajasthan 2012 prevalence survey report.
  • "Areas in western Rajasthan are predicted to have a higher risk of Ascaris infection compared to eastern parts of the state. Figure 5 shows the probability that the predicted prevalence of Ascaris exceeds 20%, based on a model including EVI and altitude. While there is a high probability that areas in western Rajasthan will require at least annual MDA, there is likely to be a lower prevalence of Ascaris throughout the east of Rajasthan." DtWI Rajasthan 2012 prevalence survey report, Pg 14.
  • "The climatic thresholds for hookworm transmission limits in Asia have been previously modelled using an aridity index, mean and maximum land surface temperature (LST). The map of these limits is shown in Figure 6, which categorises areas as unstable transmission (Mean LSTH 5-10°C, Aridity Index <0.2) or beyond transmission (Max LST >42°C, Mean LSTH <5°C, Aridity Index <0.03). These limits are based on experimental data showing that the development of infectious stages of hookworm ceases at 40° Celsius, and observed relationships between prevalence and environmental covariates using data from the Global Atlas of Helminth Infections. Based on these limits, the prevalence of hookworm in Rajasthan would be expected to oppose that of Ascaris and be more prevalent in the east of the state. All hookworm positive cases identified from this survey originated from areas within transmission limits." DtWI Rajasthan 2012 prevalence survey report, Pg 15-16.
  • DtWI concluded, "Based on Ascaris prevalence levels in the surveyed districts, observed associations with environmental data, hookworm transmission limits and the possibility that data are an underestimate of the true prevalence, annual deworming throughout the state is recommended." DtWI Rajasthan 2012 prevalence survey report, Pg 17.

• 100Data from Miguel and Kremer 2004 (Table II, Pg 168.) is provided in this table for comparison, as this paper represents what we believe to be the strongest piece of evidence of long term damages from intestinal parasites (see discussion in this section). We provide data from year one of the study, which is the time period before el nino weather conditions set in and increased the prevalence and intensity of local infections.
• 101Hookworm infections were not measured in DtWI's 2012 Rajasthan survey. DtWI Rajasthan 2012 prevalence survey report
• 102Hookworm infections were not measured in DtWI's 2012 Rajasthan survey. DtWI Rajasthan 2012 prevalence survey report
• 103

  The National Vector Borne Disease Control Programme LF treatment coverage 2012

• 104

  "DEC + Albendazole in selected distt & DEC in other distt" LF treatment drugs 2012.

• 105

  "Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
  Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except:
  - children below 2 years
  - pregnant women
  - seriously ill persons
  (DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012.

• 106

  Preventive chemotherapy in human helminthiasis 2006, Pg 41.

• 107

  The LF program has recently been active in Bihar, but not Rajasthan or Delhi. LF treatment coverage 2012

• 108
  • Timothy Telleen-Lawton: 3x. I expect that DtWI's leverage value for some states will be about 1x, and for some states will be close to 10x (when they are a critical factor in the program happening at all in a given year). With large uncertainties about how much leverage they deserve credit for even within the states they operate currently, I would guess a geometrically central estimate of 3x.
  • Elie Hassenfeld: 3x. Very few deworming programs exist in India without DtWI's involvement, and I suspect we'll see the pattern continue in the future. I recognize that this may only be a reflection of the fact that states that decide to deworm seek assistance from DtWI even though they might have chosen to deworm without DtWI's help.
  • Holden Karnofsky: 3x. I feel roughly the same way as Elie and Tim.
  • Alexander Berger: 2x. I agree with Tim's general reasoning but my guess is that in most cases, the impact of DtWI funding is less than its proportion (i.e. a multiplier below 1x), but that in some states it garners considerable leverage. I also think that the degree of leverage is likely to decline relatively quickly as the organization grows, and may be less than one at the current margin (i.e. DtWI may be "being leveraged" at this point instead of "leveraging additional funding").

• 109

  Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013

• 110

  "The current funding comes from USAID ($5 million over 5 years), the Michael & Susan Dell Foundation, and The Douglas B. Marshall Jr. Family Foundation, as well as donations made by smaller, individual donors." Alix Zwane conversation June 4th 2013, Pg 1.

• 111

  $483,000 is reported as the estimated costs to DtW in Bihar's first round. DtWI Bihar 2011 cost data, "Summary of prog costs" worksheet.

  $220,345 is reported as the estimated costs to DtWI in Bihar's second round. DtWI Bihar 2012 cost data, "DtWI expense breakdown" worksheet.

• 112

  According to DtWI, outreach to additional states has been conducted only in one state so far, and consisted of several phone calls and meetings. The initial meeting focused on providing information about DtWI’s technical support and expertise, including track record of success in other states. The government indicated that it had initiated deworming but suspended it a couple of years ago. DtWI's understanding is that preschool children are currently treated through an integrated vitamin distribution program. Subsequent communication in May 2013 indicated that the Government needed time to consider the various facets of a school-based deworming program internally among various stakeholders. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 113

  Alix Zwane conversation August 30th 2013

• 114

  Evidence Action 2014 budget, Pg 2.
  Alix Zwane conversation August 30th 2013

• 115

  Alix Zwane conversation August 30th 2013

• 116

  "How the deworming program in Rajasthan began:
  · Ms. Gupta was a Commissioner Associate when she first heard about deworming, via media coverage of the deworming program in Delhi.
  · After she heard about deworming, she spoke with other staff members about the possibility of doing deworming in Rajasthan. Then they got in touch with DtWI, and DtWI was very responsive." DtWI 2013 GiveWell government interviews, Pg 6.

• 117

  "Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage.

• 118

  "Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage.

  GiveWell conducted a site visit to DSW in November 2012 and published notes. DSW 2012 GiveWell site visit

  See GiveWell's review on Water quality interventions.

• 119

  Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013

• 120

  Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013