Schistosomiasis Control Initiative (SCI) – June 2016 version

We have published a more recent review of this organization. See our most recent report on the SCI Foundation, formerly known as the Schistosomiasis Control Initiative.

Published: June 2016

Summary

What do they do? SCI works with governments in sub-Saharan Africa to create or scale up programs that treat schistosomiasis and soil-transmitted helminthiasis (STH) ("deworming"). SCI's role has primarily been to identify country recipients, provide funding to governments for government-implemented programs, provide advisory support, and conduct monitoring and evaluation on the process and outcomes of the programs. (More)

Does it work? We believe that there is relatively strong evidence for the positive impact of deworming. SCI has conducted studies in about half of the countries it works in (including the countries with the largest programs) to determine whether its programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but have some methodological limitations. Note that we have not asked SCI for monitoring and evaluation results since September 2015 because we have put other questions on hold while we focused our questions on SCI's finances. (More)

What do you get for your dollar? Our most recent estimate is from late 2015. We estimate that children are dewormed for a total of around $1.26 per child. This figure relies on several difficult-to-estimate inputs including how to account for (a) donated drugs and (b) in-kind contributions from governments with which SCI works. Excluding drugs and government contributions, we estimate that SCI's cost per treatment is $0.53. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)

Is there room for more funding? It is our understanding that donations SCI receives now will be allocated to SCI's April 2017 to March 2018 budget year. We have not attempted to understand SCI's room for more funding for its April 2017 to March 2018 budget year in detail, but we think it is highly unlikely that SCI has raised enough funding to cover its planned activities. (More)

SCI is recommended because of its:

• Focus on a program with a strong track record and excellent cost-effectiveness. (More)
• Track record – SCI has repeatedly demonstrated success at starting and expanding national deworming programs.
• Room for more funding – we believe SCI will be able to use additional funds to deliver additional treatments in 2017-18. (More)

Major unresolved issues include:

• Although our understanding of SCI's financial position and spending has improved, we continue to have some concerns about SCI's financial reporting and financial management. We have remaining concerns about SCI's use of an accounting system ill-suited to its needs, and also recently learned that SCI made two substantial financial errors that impacted funding from GiveWell-influenced donors and our room for more funding analysis last year. (More)
• The evidence we have seen on SCI's track record of reaching those it has targeted for treatment is fairly limited. We have seen results from about half of the countries SCI has worked in and, for those countries from which we have seen results, we have generally seen one year of results, though SCI has worked in the country for several years. We have some questions about the results we have seen because of methodological limitations of the studies and past challenges communicating with SCI about its monitoring (see blog posts from 2013 and 2014).

Our review process

We began reviewing SCI in 2009. Our review has consisted of:

• Reviewing published studies on SCI's programs.
• Extensive communications with SCI Director Alan Fenwick and Deputy Director Wendy Harrison to discuss SCI's methods and funding needs.
• Requesting and reviewing SCI internal financial, organizational, and monitoring and evaluation documents.
• Visiting a national schistosomiasis control program meeting and demonstration mass drug administration in Malawi in October 2011 (notes and photos from this visit).
• Following SCI's progress and plans for funds raised as a result of GiveWell's recommendation (see our updates on SCI's progress).
• Meetings with eight SCI staff members at SCI's London headquarters in October 2014, including leadership, program managers, and monitoring and evaluation and finance staff.
• In the first half of 2016, we focused on learning whether SCI would produce higher quality financial data (more in our 2016 SCI research plan). We believed higher quality financial data was necessary for us to continue recommending SCI and put other questions on hold while we determined whether SCI would be able to provide this data.

All content on the Schistosomiasis Control Initiative, including past reviews, updates, blog posts and conversation notes, is available here.

What do they do?

SCI works with governments in sub-Saharan Africa to create or scale up mass drug administration programs (MDAs) for neglected tropical diseases (NTDs), particularly schistosomiasis and soil-transmitted helminthiasis (STH), in school-aged children and other groups determined to be at high risk.1 SCI's role has primarily been to solicit grants from large funders, identify country recipients, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI's model involves both (a) employing staff for program management and technical assistance (capacity building) and (b) funding governments to carry out mapping and treatment programs.

In our November 2015 review, we noted that we found SCI's reporting on its spending and financial position opaque; we were not able to determine how much funding SCI held, and our understanding of how SCI and the governments it supports had spent funding in the past was limited.2 Documents SCI sent us in 2016 have substantially improved our understanding of SCI's financial position and its recent spending in some of its country programs.

Major funding sources for SCI's work

Large grants

SCI's work has been driven by a number of large grants, each with somewhat different program designs and geographic coverage. SCI's major grants:

• Initial Gates Foundation grant: SCI was founded in 2002 through a $32 million grant from the Bill and Melinda Gates Foundation.3 This grant was used to create national treatment programs for schistosomiasis and soil-transmitted helminthiasis (STH) in six countries.4
• Grants for integrated NTD control: In 2006, SCI received large grants from USAID and the Gates Foundation to support integrated NTD programs in eight countries for five years to treat lymphatic filariasis, onchocerciasis, and trachoma, in addition to schistosomiasis and STH.5 It received a grant in 2007 to expand its work to Rwanda and Burundi.6 All of these grants were completed in 2011.7
• DFID grants: In 2010, SCI received £10.5 million8 (plus separate funding for drugs) from the UK's Department for International Development (DFID) for treating schistosomiasis and STH9 in eight countries over five years.10 Other NTDs are not covered by the grant, though DFID also provided funding to the Filarial Programmes Support Unit (FPSU; formerly the Centre for Neglected Tropical Diseases) to integrate treatment for lymphatic filariasis with SCI-funded schistosomiasis and STH programs in six countries.11 In 2014, DFID awarded SCI an additional £16.6 million over four and a half years (June 2014 to December 2018) to extend the program and expand it to an additional two countries.12

Unrestricted funding

Prior to 2011, unrestricted funds accounted for a very small portion of SCI's total funding.13 SCI told us that this funding was primarily used to fund treatments in regions of Côte d'Ivoire and Mozambique.14

In part due to GiveWell's recommendation, since November 2011, SCI has received significantly more unrestricted funds: GiveWell has tracked about $14.2 million in donations to SCI due to our research.15 We estimate that at least 70% of the unrestricted funds that SCI raised in April 2013 to March 2015 (the period for which we've seen data) were due to GiveWell's recommendation.16

Unrestricted funds now make up a large portion of SCI's revenue. For April 2015 to March 2016, SCI reports that 48% of its revenue was unrestricted.17

SCI’s role in mass drug administration programs

SCI's role in mass drug administrations (MDAs) in general is to:18

• Advocate for the benefits of mass drug distributions to government officials.
• Assist with planning and budgeting.
• Deliver funding and drugs to governments.
• Provide financial management and technical support.
• Develop procedures for monitoring and evaluation, analyze data, and write reports.

In 2014, we spoke with four of SCI's program managers to better understand SCI's role in four countries. These conversations were mostly consistent with our general understanding of SCI's work. We selected Côte d'Ivoire, Ethiopia, and Mozambique because SCI has spent significant unrestricted funds, the type of funding GiveWell has recommended, in these countries (more below). SCI selected Uganda as the fourth case study. Summaries follow.

In addition, in 2015, SCI shared some details about its work in Sudan, where it recently began supporting a program, and in Nigeria, where it is considering supporting a program in the future. We have not yet written up summaries of this work (documents SCI shared are listed in footnote).19

So far in 2016, we have not spent much research time attempting to update our understanding of SCI's role in MDA programs. We have instead focused our research on the quality of SCI's financial information (more below).

Côte d'Ivoire

• History: There was no large-scale schistosomiasis treatment program in the country prior to SCI's involvement.20 SCI began working in Côte d'Ivoire in 2010, but mapping and treatments did not begin until 2012 due to political turmoil. It has funded mapping and treatment for all qualifying districts.21 SCI expected to fund about 4.9 million treatments during its 2014-15 budget year;22 it reported delivering 3.1 million treatments in that period.23 During its 2015-16 budget year, SCI intended to deliver around 0.6 million treatments; we have not yet seen data on how many treatments were actually delivered.24 SCI previously told us that it spent a small amount of unrestricted funding in the country prior to 2011; we do not know what this funding was for.25
• Current plans (as of April 2016): SCI plans to deliver 2.4 million treatments in its April 2016 to March 2017 budget year.26
• Funding: SCI has used DFID funding, a grant from the company Vitol, and unrestricted funding in Côte d'Ivoire.27 In the 2014-15 budget year, SCI spent about $250,000 in restricted funding and $820,000 in unrestricted funding in the country, a significant increase over 2013-14, when SCI spent $580,000 in restricted funding and $20,000 in unrestricted funding.28 In 2015-16, SCI spent around $130,000 in unrestricted funding and around $70,000 in restricted funding in Côte d'Ivoire.29
• Impact of unrestricted funds: SCI told us in 2014 that it believed that availability of unrestricted funds had allowed the program to scale up faster than it otherwise could have.30
• Partners: In the early stages of the program, SCI primarily worked with the Ministry of Health and, for mapping, the Université Félix Houphouët-Boigny. Starting in 2014, SCI began working with other partners, including Sightsavers, the END Fund, and Helen Keller International on a more integrated NTD program.31
• Role of SCI staff: SCI's Program Manager for Côte d'Ivoire described her role (as of October 2014) as providing technical expertise, helping create budgets, monitoring training and MDAs, responding to problems (for example, surveying health workers to understand low coverage of non-enrolled children, and advising on how to proceed with the program during a security threat), refining the protocol for the coverage survey, and coordinating with partners for integrated MDAs.32 From November 2013 through November 2014, she traveled to Côte d'Ivoire five times and spent almost four months there in total to assist with and monitor mapping, sentinel site and coverage survey data collection, three MDAs, and budgeting.33 She also noted the role of other SCI staff in the program: the finance team checks receipts against expense reports,34 and the biostatistician analyzes monitoring data and advises government staff on data issues.35

Ethiopia

• History: There was no sustained, large-scale schistosomiasis treatment program in the country prior to SCI's involvement, only a one-off, sub-national treatment.36 SCI began conversations with the government of Ethiopia about starting a national schistosomiasis treatment program in 2012. In 2013, it funded planning for the program. It funded mapping between November 2013 and April 2014, and a first round of treatment in 2013.37 In April 2015, Ethiopia treated a reported 2.9 million children for schistosomiasis and STH. SCI planned to conduct a coverage survey following this round of treatment and collected baseline data on infection rates from 22 sentinel schools prior to the treatment round (more on these data collection methods below).38 We have not yet seen results. During the remainder of SCI's 2015-16 budget year, SCI reports that an additional 5 million children received treatments.39
• Current plans (as of April 2016): SCI plans to deliver schistosomiasis treatments to 6.5 million children (4.5 million of which will also receive STH treatments) and 6.7 million STH-only treatments in its 2016-17 budget year in Ethiopia.40
• Funding: SCI spent around $2.1 million in unrestricted funding and $0.8 million in restricted funding in Ethiopia between April 2015 and March 2016..41 The END Fund and DFID have now allocated restricted funds for treatment in the country.42
• Impact of unrestricted funds: SCI believes that availability of mapping data significantly improved the chances of securing funding from the END Fund and DFID, and that, at a minimum, the availability of unrestricted funds accelerated the program by a few years.43 Ethiopia's Neglected Tropical Diseases Program Manager told us that the government had not dedicated funding to schistosomiasis and that without SCI's involvement the program would likely not have gone forward. According to him, Ethiopia was not aware of any other potential partners for this work when it began working with SCI.44
• Partners: SCI is funding the Federal Ministry of Health (FMOH) to carry out treatments. It partnered with the Ethiopian Public Health Institute, the technical arm of the FMOH, and the Partnership for Child Development (PCD) on mapping. PCD funded mapping in part of the country and delivered treatments in a pilot program. Deworm the World Initiative is providing technical assistance to the government.45
• Role of SCI staff: SCI's Program Manager for Ethiopia described his role (as of October 2014) as assisting with initial program set up and funder agreements, drafting detailed plans for the next 12 months and general plans for the next 5 years, and providing technical assistance for mapping and some guidance on MDA implementation. He lives in the country46 and works closely with the government NTD team.47

Mozambique

• History: There was no large-scale schistosomiasis treatment program in the country prior to SCI/FPSU's involvement. SCI had previously funded a small-scale treatment program in Mozambique with unrestricted funding.48 SCI's grant from DFID, which started in 2010, includes funding for a sub-grant to FPSU to run schistosomiasis treatment programs in three countries, one of which is Mozambique.49 SCI and FPSU decided on this arrangement because SCI believed it would increase the chances of DFID awarding the grant.50 From April 2015 to March 2016, SCI reports delivering 4.4 million treatments, out of a targeted 5.8 million.51
• Current plans (as of April 2016): SCI reports that it plans to deliver 5.8 million treatments in Mozambique between April 2016 and March 2017.52 SCI told us that one of the rounds of MDA originally scheduled for 2016-17 is now scheduled for 2017-18.53
• Funding: SCI has spent both restricted funding from DFID and unrestricted funding in Mozambique. The data for before the 2015-16 budget year we have seen on SCI's spending in Mozambique is incomplete (see below), but our impression is that Mozambique has been one of SCI's largest programs.54 It has funding from the SCORE project for research on "gaining and sustaining control of schistosomiasis" in one province.55
• Impact of unrestricted funds: SCI told us that unrestricted funds have helped the program scale, but perhaps not as much as it might appear, since SCI has shifted DFID funding to other countries with the expectation of using unrestricted funds in Mozambique.56
• Partners: The Ministry of Health implements the program. SCI/FPSU also collaborate with RTI International, which is working on trachoma in Mozambique and has staff in-country. RTI helps to keep SCI/FPSU informed about program progress.57
• Role of SCI staff: FPSU manages the program day-to-day. SCI has provided some assistance with monitoring and evaluation. We do not have more detail on SCI/FPSU role in the country.58

Uganda

• History: SCI has worked in Uganda since 2003 and has received funding for this work from each of its large grants: Gates Foundation, USAID, and DFID. There was a gap in SCI's work between the USAID and DFID funding.59 SCI reported that it delivered 23,000 treatments in April 2014 to March 2015.60 It had planned to fund 400,000 treatments this period;61 we haven't discussed with SCI why significantly fewer treatments were delivered than planned. Between April 2015 and March 2016, SCI reports that it delivered 890,000 treatments in Uganda; 1.2 million treatments were planned.62
• Current plans (as of April 2016): SCI plans to deliver 1.1 million treatments in Uganda between April 2016 and March 2017.63
• Funding: In recent years, SCI has primarily used DFID funding in Uganda (about $176,000 in the fiscal year covering 2013-14, $30,000 in 2014-15, and $378,000 in 2015-16).64 In 2014, SCI allocated a small amount of unrestricted funding to Uganda to increase the number of sentinel sites for operational research purposes.65 SCI also spent about $30,000 in unrestricted funds in Uganda in 2014-15;66 we don't know what these funds were used for. SCI spent around $176,000 in unrestricted funding in Uganda in its 2015-16 budget year, substantially more than previous years.67
• Partners: RTI International, funded by USAID, works in Uganda on an integrated mass NTD treatment program. SCI provides schistosomiasis treatment in those areas where RTI does not.68
• Role of SCI staff: SCI told us that, due to strong in-country capacity, SCI provides only limited technical assistance for MDA. Instead, SCI's technical assistance in the country is focused on operational research on how best to move toward elimination of schistosomiasis.69 SCI's program manager for Uganda describes her role as (as of October 2014) assisting with budgets and plans, talking to the government regularly (largely through email) leading up to an MDA to make sure all the components are prepared, traveling to the country to oversee collection of prevalence and intensity data prior to an MDA, following up with the government after an MDA to get the data needed for reporting to DFID, and participating in activity reviews after each stage of the program (for example, after an MDA and after monitoring and evaluation activities).70

Breakdown of SCI’s spending

Our research on SCI in 2016 so far has focused on the quality of SCI's financial data. We include our updated views on the quality of SCI's financial documents and spending breakdowns by country and within countries below.

Financial update

We wrote in our November 2015 review that SCI's accounting system seemed ill-suited to its needs and that SCI's financial information was limited in scope and prone to containing errors. In December 2015, we spoke to SCI's former Finance and Operations Manager, who confirmed that the financial systems SCI was using were not well suited for NGO grant management, and that SCI had not allocated sufficient staff capacity to managing its finances.71 Due to these concerns, we decided to focus our research in 2016 on the quality of SCI's financial documents.72 We felt that seeing significant improvements in the quality of SCI's financial documents was necessary for us to continue recommending SCI, and we put other research questions on hold for the first half of 2016.73

The financial documents SCI has provided us with in 2016 so far allow us to answer some questions that we were not able to answer in previous years:

• At the beginning of its 2016-17 budget year, SCI had $15.8 million ($8.6 million from restricted sources and $7.2 million from unrestricted sources) available to allocate to its April 2016 to March 2017 programs, research, central costs, and funding reserve.74 SCI planned to allocate nearly all of its available funding to programs and central costs in its 2016-17 budget year.75 For comparison, in several of our past reviews and updates on SCI, we have noted that we were uncertain about how much funding SCI held.76
• A breakdown of SCI's spending within six country programs during its 2015-16 budget year is below. We wrote in our November 2015 review that we had not seen recent information on how SCI spent funding within country programs.

Despite these improvements, we have some remaining concerns about SCI's financial systems and reporting:

• SCI is housed within Imperial College London. Because it is not a standalone charity, it does not publish annual financial statements or undergo annual organization-wide audits (as U.S. charities are required to do).77 SCI notes that it has been in discussion with KPMG, Imperial College’s auditors, about certifying SCI’s accounts as a separate organization.78
• It continues to use an accounting system created by Imperial College which breaks up its finances into many different "accounts" rather than giving an overall view of the organization's financial position.79 In February 2016, SCI told us that it would start to use a new accounting software package beginning in April.80 We have not yet seen financial data produced using the new software.
• Based on our experiences with SCI's financial reports, we believe that they are prone to containing errors. We detail the errors we have learned about in this footnote (including some errors we found in 2016).81 Of particular note: (1) a July 2015 grant from GiveWell for about $333,000 was misallocated within Imperial College until we noticed it was missing from SCI's revenue in March 2016; and (2) in 2015, SCI provided inaccurate information about how much funding it would have from other sources in 2016, leading us to overestimate its room for more funding by $1.5 million.82

Spending breakdown by country

April 2015 to March 2016 expenditures by country (in millions USD)83

Notes about this data:

• In our November 2015 review, we noted that some expenses recorded as central expenditures were actually used for the Zambia, Liberia, and Mozambique country programs.84 It is our understanding that central expenditures in the table above do not include any such expenses.85
• SCI notes that an error appears to have caused unrestricted funding to be recorded as restricted funding for Malawi.86 We have not corrected the error in the table above.

For breakdowns of SCI's spending in 2013-14 and 2014-15, see our November 2015 review.

Spending breakdown within country programs

In 2015, SCI began to use a system of country cashbooks, which compare monthly in-country actual spending to budgets.87 SCI sent us country cashbooks covering April 2015 to March 2016 for Niger, Côte d'Ivoire, Democratic Republic of the Congo, Malawi, Tanzania, and Uganda.88 We have only seen country cashbooks for six country programs, but it is our understanding that SCI plans to eventually collect country cashbooks from all country programs.89 Notes about this data:

• Activity categories in the country cashbooks sometimes appear to be loosely defined and overlapping. For example, expenditure on drug distribution materials is sometimes classified as "drug distribution" and sometimes as "drug logistics."90
• We do not have a clear understanding of the relationship between expenses recorded in country cashbooks and expenses recorded in SCI budget vs. actuals 2015-16 Redacted. For example, for Malawi, the total expenditure recorded for the 2015-16 budget year in both the country cashbook and SCI budget vs. actuals 2015-16 Redacted was $0.8 million, but total expenditure for Tanzania differed substantially in the two sources.91 SCI notes that the discrepancy for Tanzania is due to SCI recording the expenditure when the funds were sent to the country, while the cashbooks record when the funds are spent in-country.92 It is our understanding that SCI is in the process of changing this system.
• The total amount of spending recorded in the six country cashbooks we have seen is $2.1 million, a relatively small proportion of the $9.2 million in total spending within country programs in its 2015-16 budget year.93

In-country SCI spending in six countries, April 2015 – March 201694

For breakdowns of spending within countries supported by DFID grants between 2011 and 2013, see our November 2015 review of SCI.

Other projects

In addition, SCI has received some smaller grants for a variety of projects, including:

• Research. SCI has received a number of smaller grants to carry out research related to NTD control.95
• Other NTD-related activities. SCI has also used funding from individuals for surgeries for hydrocele (a symptom of lymphatic filariasis) in Niger, and health education and water and sanitation programs in Burundi.96

Does it work?

SCI's mass drug administration programs are focused on delivering treatments that have been independently studied in rigorous trials and found to be effective.

SCI has conducted surveys in some of the countries it has worked in to determine whether its programs have reached a large proportion of children targeted. We have now seen some recent monitoring results from about half of the countries in which SCI works, including the five countries where SCI has delivered the largest number of treatments. For those countries from which we have seen monitoring results, we have generally seen one year of results, though SCI has worked in the country for several years. We have some questions about the results we have seen because of methodological limitations of the studies. In 2016, we have focused our research on the quality of SCI's financial documents (more above), and have not attempted to update our understanding of its monitoring data.

To determine SCI's track record at executing programs, we have considered:

1. Surveys of changes in prevalence and intensity of infection over time in three of the countries SCI has worked in. The surveys show substantial improvements following SCI treatment programs. These surveys have a number of limitations and represent a small portion of SCI's past work.
2. Treatment coverage surveys from six of the countries SCI has worked in, including many of the countries where SCI's work has been focused in the past five years. These surveys track what percentage of individuals who were targeted for treatment actually received treatment. Overall, the surveys found moderate rates of coverage. We note some limitations of these surveys below.
3. Other published papers that might reflect the treatment coverage achieved by SCI's programs by directly measuring deworming drug uptake or by measuring worm prevalence in countries where SCI has worked. The evidence is mixed, but makes a weak case for low coverage in the areas studied.

Details follow.

Are mass school-based deworming programs effective?

SCI supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.

There are some important differences between the type and severity of worm infections in the places SCI works and the places where the key studies on improved life outcomes from deworming took place, which we discuss below.

Are SCI's monitoring results representative of its work overall?

We have now seen recent monitoring results from about half of the countries in which SCI works, including the five countries were SCI has delivered the largest number of treatments. For those countries from which we have seen monitoring results, we have generally seen one year of results, though SCI has worked in the country for several years.

Details in this spreadsheet.

For countries or years for which we have not seen monitoring data, it is generally unclear to us whether SCI collected additional monitoring data that it has not shared with us or whether it did not collect monitoring data. SCI told us that it is sometimes unable to share results because third parties (e.g. governments, WHO, funders) often need to give permission before data can be shared and because it can take some time for data, once collected, to reach SCI because in some countries it is cleaned and analyzed by country program staff before being shared with SCI.97

For Mozambique, one of SCI's largest recipients of both restricted and unrestricted funds, SCI shared a report from a consultant who visited the country in May 2015 to assist with data cleaning and analysis for prevalence data from 2012, 2013, and 2014. The report notes major problems with this data and the refusal of the government to allow SCI and other international partners to have access to the data outside of Mozambique.98

Are targeted children being reached?

Coverage surveys

SCI has conducted, or worked with partners to conduct, surveys in Côte d'Ivoire (in 2014), Malawi (2012 and 2014), Uganda (2014), Mozambique (2015), Zanzibar (2015), and Zambia (2015) to determine what proportion of people targeted for mass drug administration received treatment.

In each of the surveys, surveyors visited a sample of households and asked children, or their parents on their behalf, whether they received treatment in the most recent MDA. Other survey questions, such as age, gender, where the respondent received the treatment, and why they did not take the drug(s), were often included as well. The methodology used differed somewhat across countries. We have summarized the details of the methodologies used in the surveys in this spreadsheet ("Methods" sheet). Key differences in the methods used across countries include:99

• Selection of geographic target area: All of the surveys were limited to specific geographic areas (such as districts). In Uganda, Zambia, and Côte d'Ivoire, these were selected randomly or nearly randomly. In Malawi (both 2012 and 2014 surveys), the districts were purposefully selected and not intended to be nationally representative. The selection procedure for Zanzibar was not given in the survey report.100 We are aware of one case, in Mozambique, where a selected village was replaced by another because the surveyors could not locate it; this issue was not mentioned in the survey report.101
• Independence from the government: In Mozambique, the survey was carried out by government health staff, who may have had an incentive to bias the results. SCI told us, "[M]ost of the interviews in one district were done by the other district officers with no connection with the district."102 The reports on the Malawi 2014 and Côte d'Ivoire surveys note that the surveyors were independent of the government. SCI told us that university students or staff conducted the other surveys (Malawi 2012, Uganda, Zanzibar, and Zambia).103
• Length of time between MDA and survey: This varied between one and six months. Intuitively speaking, the more time that passes, the less likely children are to remember accurately and the more likely they are to confuse past MDAs (we discuss one case below, from Mozambique, where there may have been confusion about MDAs). Mozambique had the shortest interval at 1-2 months and Zambia had the longest at 5-6 months. Other surveys were generally carried out 2-4 months after the MDA.
• Whether parents or children were interviewed: In Mozambique, parents were interviewed about whether their children took the drugs. In both Côte d'Ivoire and Malawi (both 2012 and 2014 surveys), if children in a household were not available then their parents were interviewed about whether the children had received deworming drugs. SCI made different choices about whether to include these responses in the results, which slightly inflated the results overall.104 SCI told us that parents were not asked to answer on behalf of their children in Uganda or Zanzibar.105 The report on the Zambia survey does not mention adults answering for children.

In addition to limitations in specific surveys, we note some cross-cutting limitations to the methods used:

• Use of verification methods: In Côte d'Ivoire, Malawi (2014), Uganda, Zanzibar, and Zambia participants were asked a number of questions about the treatment program, such as whether they recognized pills or dose poles presented by the interviewers, what they thought of the pills (praziquantel is very large and tastes bitter) and how many pills they took.106 Answers to these questions were not recorded in Côte d'Ivoire,107 and we do not know if they were recorded in the other surveys. We believe that the answers to questions such as these would provide additional evidence about the quality of the coverage results and it is unclear to us why SCI has not recorded, or if it has recorded it, why it has not shared this information. In Mozambique, respondents were asked whether they recognized the dose pole used in schistosomiasis MDAs. However, in that survey parents were surveyed on their children's behalf108 and most children (79%) received drugs at school,109 presumably when parents were not present. We don't know how to interpret the result that a very high percentage of parents (median 90%, ranging 61-94% across provinces) reported recognizing the dose pole.110 SCI hypothesized that parents may either recognize the dose pole from publicity efforts prior to the MDA or remember a similar dose pole from previous MDAs for lymphatic filariasis.111 In either case, this may indicate that the coverage survey is not measuring actual delivery of drugs to children in the most recent MDA.
• Accuracy of responses from young children: SCI told us, "It can be difficult to get clear, accurate answers from young children (5-6 years old). Children, especially younger ones, may be influenced by others who are around during the survey. This is especially so because surveyors often interview older children first, in front of younger siblings."112
• Supervision and auditing of surveys: There is no mention in the coverage survey results of any re-surveying of households to check the accuracy of the data collected and, although "supervisors" are mentioned in several reports, it is not clear what role they played.113 SCI described to us the supervision used in the Mozambique survey (details in footnote); audits were not mentioned.114

The fact that the surveys identified low coverage in several cases increases our confidence in their reliability. Given the smaller sample size, government involvement in the survey, and question about parents recognizing the dose pole noted above, we are more skeptical about the results from Mozambique than those from other countries.

Mass drug administration coverage among school-aged children

Notes about these results:

• Results broken down by district (or similar geographic region) are in this spreadsheet.
• PZQ is the drug used to treat schistosomiasis. ALB is the drug used to treat STH.
• Ranges and medians are calculated from district-level results.
• The results are for school-aged children only. Some of the surveys also measured coverage rates in adults, who are targeted for treatment in some SCI-supported programs.
• The results exclude districts in Uganda that were included in the survey but which do not receive support from SCI.115
• For context, the World Health Organization recommends that treatment programs aim for coverage rates above 75%.116

Have infection rates decreased in targeted populations?

SCI has conducted surveys to track changes in schistosomiasis and STH prevalence and intensity rates following SCI-supported treatment programs. In each of these studies, SCI tracked infection rates at the same schools ("sentinel sites") each year. In general, prevalence and intensity of the parasites decreased over time in each of the countries studied. We note several methodological limitations of these surveys below.

Which prevalence and intensity studies provide evidence of SCI's impact?

Below, we discuss results from studies of schistosomiasis and STH prevalence and intensity from three countries: Niger (2004-2006),117 Burundi (2007-2010),118 and Malawi (2012-2015).119

SCI also shared studies from Uganda120 and Burkina Faso,121 and we included results from these studies in our previous reviews of SCI. We learned in 2013 (and in follow up work in 2014) that participants in the studies in Uganda and Burkina Faso received separate, more intensive treatment than other children in those countries (discussed in blog posts in 2013 and 2014). Therefore, we believe that the results from Uganda and Burkina Faso do not reflect the quality of the national programs which were supported by SCI.

It is our understanding that, in the Niger, Burundi, and Malawi studies, study participants received treatment in the same manner as other children in the country, and thus that those studies reflect the performance of the national MDAs. However, we are not highly confident in this conclusion because we had difficulties communicating clearly with SCI about the methodology of these studies (discussed in more detail in the blog posts linked above).

SCI told us that it is currently conducting similar studies in Mozambique, Ethiopia, DRC, Liberia, Côte d'Ivoire, Zambia, Tanzania, Uganda, and Niger (for more recent years). We have not yet seen results from any of these studies.122

Niger, Burundi, and Malawi prevalence and intensity studies

As discussed above, SCI has conducted studies to track changes in schistosomiasis and STH prevalence and intensity rates following SCI-supported treatment programs in Niger, Burundi, and Malawi. In each study, with one exception, the same individuals were examined before the initial round of treatment and before each subsequent round of treatment.123 In the 2015 Malawi study, SCI switched to a cross-sectional sample, where random children from the same schools were surveyed, rather than the same individuals.124 To partially account for this change, the data from Malawi presented below is for the 6-8 year old age group only. There is no control group for these studies.125

In general, prevalence and intensity for the two main types of parasites that cause schistosomiasis, S. haemotobium and S. mansoni, and for hookworm (more on the other two STHs below), decreased over time in each of the countries studied. Though it is possible that other factors besides the treatment program caused these changes (such as improved sanitation infrastructure), the pattern of decline in a short period following treatment strongly suggests that treatment caused or contributed to the declines.

Changes in worm prevalence and intensity126

For the other two prominent soil-transmitted helminths, ascaris and trichuris, infection prevalence was low in the Niger and Malawi studies.148 In Burundi, prevalence of ascaris and trichuris decreased somewhat (though in a few cases there were temporary increases). Data from Burundi are given in the footnote.149

Some of the studies also report results for other indicators of disease such as anemia. We omit discussion of these other indicators because they are more likely to be influenced by external factors than are prevalence and intensity (see our previous review of SCI for discussion of these indicators).

Limitations of the prevalence and intensity study data include:

• Monitoring of selected locations. It appears that, in the Niger and Burundi pilot studies, locations included in the study were selectively chosen rather than selected to be a representative sample of treated areas.150
• Low follow-up rates. Follow up rates were low in two of the three countries (at the first year follow-up, 89% in Niger, 33%-50% in the pilot survey and 53%-80% in the other schools survey in Burundi, and 52% in the first follow up in Malawi).151 To be included in follow up surveys, children must be present in school when the surveys are done.152 If those who are present in school are less likely to be infected than those who are not present, this could lead to overstating the impact of the program. The connection between infection status and absenteeism could be a direct relationship (infection could cause absenteeism) or an indirect one (a third factor, such as poverty, could cause both higher levels of infection – perhaps through poor sanitation infrastructure – and absenteeism). Because the second follow up in Malawi was done as a cross-sectional study among children present in school on the day of the study and we present results for children ages 6-8 only, absenteeism and dropout are less likely to bias the results than in the panel studies.

Results from Yemen

In addition, we have two types of results from Yemen:

• Partial sentinel site data: A report SCI shared with us mentions that initial analysis of sentinel site data from July 2014 in Yemen found that prevalence of schistosomiasis decreased substantially.153 These results only include 2,000 of the 8,000 individuals who were surveyed at baseline and the result noted that "a full round of impact evaluation" would be completed in September 2014.154 We have not seen more details about these initial results or any results from the full round.
• Remapping survey: This study compared the number of districts at high-risk, moderate-risk, low-risk for, and uninfected with schistosomiasis at "baseline" (data collected between 2004 and 2010) and in 2014.155 It found large improvements after 2-3 rounds of treatment.156 It is not clear to us whether baseline and follow up results are directly comparable. Baseline data was collected over several years and details of the methodology used at baseline are not given.

Additional academic evidence

To provide additional information on SCI's track record, in 2014, we conducted a search for published papers on treatment coverage rates and schistosomiasis and STH prevalence and intensity in countries where SCI has worked.

We focused on papers by Melissa Parker and Tim Allen, who were funded by the Bill and Melinda Gates Foundation to provide an anthropological perspective on SCI's work,157 papers cited in papers by Melissa Parker and Tim Allen, and other papers we identified on this topic from a Google Scholar search.158

The papers we identified were all from Tanzania, Uganda, and Zanzibar, perhaps because these are places that Melissa Parker and Tim Allen's work has focused on.

This spreadsheet summarizes the papers we considered. We have not fully vetted these studies. The studies were generally designed for purposes other than to evaluate SCI's programs so in many cases there is uncertainty about SCI's role in the areas studied. In the discussion below, we have excluded studies that we do not believe were conducted soon after treatment programs in areas targeted by SCI-funded programs.

SCI staff and/or SCI funding were involved in many of the studies.159

It is difficult to draw any conclusions from these studies because of small sample sizes and lack of clarity on SCI's role in each location. In summary:160

• Tanzania (excluding Zanzibar): Two studies at the district level, Stothard et al. 2013 and Chaula and Tarimo 2014, show relatively low schistosomiasis prevalence after treatment (4% and 15% respectively; the latter claims that prevalence pre-MDA was 30%). SCI was working in Tanzania at the time of both studies, but it is not clear if SCI was working in the specific areas studied. Chaula and Tarimo 2014 also shows low treatment coverage (around 40%), though the researchers asked in 2013 about treatment provided in 2011 and 2012.
• Zanzibar: All studies were conducted on Unguja, "the largest and most populated island of Zanzibar."161 Two studies, Stothard et al. 2009 and Rudge et al. 2008, each in a single school, found high prevalence of schistosomiasis in areas that had received treatment (50% and 65% prevalence, respectively). The sample sizes were small, it was not clear why these particular schools were selected, and, while SCI was working in Zanzibar at the time of both studies, but it is not clear if SCI was working in the specific areas studied. Another study, Knopp et al. 2009, in two schools in an area that had received treatment (but perhaps not from SCI) found moderate prevalence of various STHs (21% prevalence for hookworm and low infection intensity). The schools were selected because they had been surveyed in 1994, when prevalence was found to be much higher.
• Uganda: We reviewed three studies of schistosomiasis prevalence: Muhumuza et al. 2013 was conducted in part of a district that seems to have received several rounds of treatment (but perhaps not from SCI). Brooker et al. 2005 was conducted across a district that had its first MDA the year before, which was funded by SCI. Standley et al. 2009 was conducted across six districts, one district that had never been treated (but from which only one school, which had low prevalence, was included in the analysis), two districts that seem to have been treated only once, and three districts that seem to have been treated for many years by SCI. Standley et al. 2009 explicitly aimed to "ambush" schools that might not have been reached by treatment programs.162 Each study found moderate to high prevalence of schistosomiasis (Muhumuza et al. 2013: 35%, Brooker et al. 2005: 28%, and Standley et al. 2009: 42%). Muhumuza et al. 2013 and Standley et al. 2009 found fairly high infection intensity (116 and 634 average eggs per gram respectively among positive cases; the WHO threshold for "high intensity" is 400 eggs per gram163 ). Muhumuza et al. 2013 found 28% coverage in the most recent round of treatment. The coverage survey in Muhumuza et al. 2013 seems to have been six months after treatment.

Are there any negative or offsetting impacts?

• Concerns over whether treatment was sustained: We believe it is important that deworming programs are sustained over time, as re-infection is rapid and a one-time treatment may have little long-term effect.164 It is not clear to us the extent to which SCI-funded programs have succeeded in treating the same children multiple times, as opposed to simply treating the same areas multiple times (and thus treating different children once each).

  
  We remain unsure about how many treatments are needed to impact health. SCI told us that its views on what groups should be treated and how often "is largely based [on] WHO recommendations, but also on government wishes and on intuition and common sense, though it usually works and SCI collects sufficient data to know when it isn't working. In general, in high endemicity areas re-infection is a major issue; in lower endemicity areas, a single treatment can be sufficient."165 One example of the variation in treatment patterns is what SCI told us about its program in Yemen:
  

  SCI distinguishes between high, medium and low prevalence areas. In high prevalence areas, SCI treats the whole population once, and children for five years. In medium prevalence areas, SCI treats the whole population once, and children every other year. In low prevalence areas, SCI treats children every other year.166

• Displacement of government funding for deworming: This could be a concern if, due to SCI's spending on national deworming programs, government funds that otherwise would have been spent on deworming are spent on other, less worthwhile budget items. In the past, SCI has largely supported programs that did not exist before its support.167 We have not seen data on government spending on NTDs before and after receiving SCI support. Ethiopia's Neglected Tropical Diseases Program Manager told us that the government had not dedicated funding to schistosomiasis and that without SCI's involvement the program would likely not have gone forward.168
• Diversion of skilled labor: Drug distribution occurs only once or twice per year and appears to be conducted by teachers, community drug distributors (who receive minimal training to fulfill this role), and health center staff.169 Given the limited time and skill demands of mass drug distribution, we are not highly concerned about distorted incentives for skilled professionals.
• Popular discontent: We have heard a couple accounts of discontent in response to SCI's mass drug administration campaigns, including one case that led to riots.170 SCI notes that following episodes of popular discontent, it has worked with governments to improve public education about the programs.171

What do you get for your dollar?

This section examines the data that we have to inform our estimate of the expected cost-effectiveness of additional donations to SCI.

Note that the number of lives significantly improved is a function of a number of difficult-to-estimate factors, many of which we discuss below. We incorporate these into a cost-effectiveness model which is available here.

We focus on the following questions:

• What is the cost per schistosomiasis treatment? We estimate that on average the total cost of a schistosomiasis treatment delivered in SCI's programs is $1.26. Excluding the cost of drugs (which are often donated) and in-kind government contributions to the programs, we estimate that SCI's cost per treatment is $0.53. These estimates rely on a number of uncertain assumptions.
• How much impact do SCI's programs have (per child treated) compared with the ones on which the evidence for deworming's positive impact on life outcomes is based? Because the key deworming studies provided treatments in areas of unusually high prevalence, we believe that SCI's programs are likely to have less impact on a per-person basis.

We have not seen complete treatment data on SCI's treatments in its 2015-16 budget year, so our cost per treatment analysis is based on earlier data. As discussed above, the information we have seen on SCI’s expenses prior to its 2015-16 budget year is limited and possibly unreliable. Similarly, we are not confident in the accuracy of the data we have seen on number treatments delivered. Given this, we make a number of assumptions and judgments in interpreting the data that we have seen and this could introduce errors (which could potentially overstate or understate the actual cost, though we generally try to conservatively err on the side of counting fewer treatments and more costs).

In 2014, SCI estimated its cost per treatment at $0.80.172 In October 2015, SCI estimated that after April 2016 its cost per treatment excluding costs for drugs and costs paid for by governments would generally be about $0.30.173 We are unsure how it calculated these estimates. In September 2015, SCI told us that it expects to improve its cost-effectiveness analysis by the end of the year.174 SCI shared updated cost per treatment estimates with us for a few country programs in early 2016, but we are also uncertain about how these estimates were produced.175

We discuss how the cost per treatment figure relates to how much it costs to improve a child's health and development at our report on mass treatment programs for schistosomiasis and STHs.

Our calculations and sources for our cost per treatment analysis are shown in more detail in this spreadsheet.

What is the cost per schistosomiasis treatment?

Our approach

Our general approach to calculating the cost per treatment is to identify comparable cost and treatment data and take the ratio. We prefer to have a broadly representative selection of treatments in order to mitigate possible distortions, such as using data from a new program, which may incur costs from advocacy, mapping, etc. before it has delivered any treatments.

It is our understanding that SCI generally intends to treat for STHs in all places where it treats for schistosomiasis, so the treatments SCI reports can generally be interpreted as combination schistosomiasis and STH treatments,176 though we are aware of several cases in which schistosomiasis-only treatments were delivered either by design or due to problems with implementation.177

To get the total cost, we attempt to include all partners (not just SCI) such that our cost per treatment represents everything required to deliver the treatments.178 In particular, we include these categories:

• SCI’s funding to country programs (e.g. to fund drug delivery).
• SCI’s headquarter costs (e.g., for management and technical salaries), including an estimate of costs paid by Imperial College (e.g., office space and some legal and administrative expenses).
• Cost of drugs. We include the full market cost of all praziquantel that is needed to deliver the treatments, regardless of whether SCI purchased it or used donated drugs.
• Costs incurred by the government implementing the program (e.g. for staff salaries when working on treatment programs).

SCI notes that cost per treatment calculations should include sensitivity analysis179 – analysis on the degree to which the cost per treatment varies when various assumptions vary. We have not yet completed such an analysis.

Our analysis

We analyzed several sources of data, which cover different time periods, and developed several estimates, two of which we summarize here. Full details in this spreadsheet.

1. SCI’s recent programs: $1.26 per treatment. The ICOSA program (the name for SCI's DFID-funded work) covers eleven country programs and has been funded primarily by SCI’s first DFID grant, with additional funding from both restricted and unrestricted sources. We have seen data that we think provide a relatively comprehensive picture of spending and treatments over the first five years of the program (October 2010 - March 2015). We have not included the cost of research programs (which are funded separately) in these countries. It is possible that some of this research funding contributed to the treatments we are counting. A more conservative estimate, including about $1.8 million of research expenses in the same countries, is $1.32 per treatment.
2. SCI’s early programs, adjusted for inflation: $1.76 per treatment. SCI previously shared with us a summary of the treatments it delivered and the costs it incurred during its early programs (from about 2003-2009). There is some ambiguity in the treatment data that we have interpreted conservatively (in other words, this estimate may be too high, as described in the footnote).180

Shortcomings of our analysis

While we believe the estimate described above is reasonable, we want to highlight specific reasons to interpret it with caution.

We rely on reported treatment data. Our understanding is that these data can overstate treatments, so we have discounted this data (by 10%) based on the differences between reported treatment rates and treatment rates found in the coverage surveys discussed above (see footnote for why this is an imperfect comparison).181

We rely on an estimate that 30% of overall program costs are attributable to the government. We derived this from an analysis of a single program in Niger (this footnote elaborates on the details and concerns).182

We do not have data that indicate what proportion of drugs are wasted. We expect that in some cases drugs are purchased or donated but expire before use. We do not know how common this is. In our analysis, we have assumed that 10% of drugs are wasted, which increases the cost per treatment by about $0.05.

We do not have data on Imperial College's expenses that support SCI. Based on a conversation with SCI, we have roughly estimated this support as 10% of SCI's expenses (excluding drugs and government contributions).183

We simply estimate an average cost across programs and do not account for variations in different contexts. SCI told us that costs can vary significantly, for example, due to increased transportation costs in some contexts.184

Baseline infection status in SCI's deworming locations compared to key deworming study locations

SCI's current and future programs may be less cost-effective than past programs or than programs discussed in our report on deworming because of lower baseline infection rates in current and future programs.

In 2014, SCI shared baseline data from countries it had recently started work in. We compare these rates to rates observed in the best evidence for the effectiveness of deworming, in order to understand how similar SCI's impact is likely to be to that observed in the studies. Schistosomiasis and STH prevalence and intensity in these countries was generally fairly low compared to the studies providing the best evidence for the benefits of deworming (Bleakley 2007, Croke 2014, and Miguel and Kremer 2004) and compared to some of SCI’s earlier prevalence and intensity studies (from Burkina Faso, Uganda, Niger, and Burundi).185

For the most part, baseline data was collected in schools that had been selected for prevalence and intensity studies. The baseline reports use methodologies that seem similar to the other SCI panel studies discussed above. With the exception of the study discussed above from Malawi, we have not examined how representative these schools are of the full treatment area or fully vetted the methodology used. For Ethiopia, we have used data collected during disease mapping and made several assumptions to make that data comparable to the data from other countries.186

In Malawi an error in data collection may have resulted in prevalence being underestimated.187 In Zanzibar, treatment has been ongoing,188 so the study does not reflect pre-treatment conditions.

Detailed results and sources in this spreadsheet.

Is there room for more funding?

In short:

• Funding SCI receives now will likely be allocated to its April 2017 to March 2018 activities. In April 2016, SCI allocated almost all of its available unrestricted funding to its current April 2016 to March 2017 budget year, so we do not expect that SCI currently has much funding available to allocate to its next budget year.
• Although SCI has room for more funding for its 2017-18 activities, we do not think that it is urgent to fill this funding gap. It is our understanding that SCI will plan its programs for its April 2017 to March 2018 budget year in February and March 2017, so it makes little difference whether SCI receives additional funding now or after our next update in November 2016.
• We have had challenges communicating with SCI about its room for more funding in the past, which caused us to substantially overestimate its room for more funding in November 2015.
• We have not asked SCI for a more recent update on its room for more funding.

Available and expected funding

SCI's current budget year began in April 2016 and will end in March 2017. It is our understanding that funding SCI receives during its current budget year is generally allocated to the next budget year.189 A donor giving funds to SCI now likely would not impact SCI's April 2016 to March 2017 activities, but would instead provide more funding for implementing SCI's April 2017 to March 2018 plans. It is possible that, if SCI received additional funding earlier in the 2016-2017 budget year, it might affect its discussions with partners and other preparations that could impact 2017-2018 and beyond; we have not discussed this possibility with SCI.

It is also our understanding that SCI currently has little unallocated unrestricted funding. As of May 2016, SCI told us that it was allocating all or nearly all of its available unrestricted funding (with the exception of funding SCI holds in reserve) to its 2016-17 budget year.190 At the time of this writing (June 2016) we would guess that SCI has received little funding that it can allocate to its 2016-17 plans since it is still early in its budget year. Absent a GiveWell recommendation, we expect that SCI would continue to receive some unrestricted funding from donors not influenced by GiveWell, but we have not attempted to estimate how much funding this would be.

Uses of additional funding in 2017-2018

We very roughly estimate that SCI has $10.1 million in room for more funding for its next budget year (between April 2017 and March 2018).191

For the reasons discussed above, we have not asked SCI for a more recent update on how it would use additional funding.

SCI notes that it allocates funding "to align with drug availability in country, capacity to deliver and ministry of health planned activities for the year."192

Sources of uncertainty

In the past, changing circumstances have caused SCI to update its target treatment numbers and its own estimate of its room for more funding. For example, political unrest delayed the program in Côte d'Ivoire for 18 months,193 in 2014 SCI was not yet ready to allocate additional funds to Mozambique because of lack of confidence in the program's ability to scale further at that time,194 and the Ebola outbreak has delayed work in Liberia.195 Factors that can shift SCI’s planned uses of unrestricted funding include political unrest, expiring drug supplies, additional donated drugs becoming available, delays and budget changes due to coordination with other actors, results of disease mapping, and grants from other donors.196 As such, we do not place much weight on the preliminary room for more funding estimates for SCI's work in 2017-2018. SCI notes that estimates are refined in collaboration with ministries of health on a country by country basis between January and March each year.197

Communication with SCI about its room for more funding

We have had challenges communicating with SCI about its room for more funding. In the past, our understanding was developed largely through conversations with SCI's leadership, supplemented with details from many other sources. In retrospect, our understanding of how SCI planned to use funds often did not match how SCI decided to allocate funds the next time it set program budgets.198

In October 2015, SCI sent us estimates of its room for more funding for its next three budget years.199 We later learned that these estimates did not include the full amount of restricted funding that DFID planned to provide, which caused us to overestimate its room for more funding for its 2016-2017 budget year in our November 2015 review.200

Global need for treatment

There appears to be a substantial unmet need for schistosomiasis treatment globally.

SCI shared an unpublished World Health Organization (WHO) estimate of the number of school-aged children and adults who require treatment for schistosomiasis and those who received treatment in 2014. We do not have permission to publish this data, but we note that only a small portion of those who WHO believes need treatment received it in 2014.201

SCI as an organization

• Track record: SCI has consistently gotten national deworming programs to go through, as discussed above. We know fairly little about how effectively these programs have delivered treatments.
• Self-evaluation: SCI’s self-evaluation is strong compared to the vast majority of organizations we have considered. That said, this evidence is incomplete and has quite a few limitations. In addition, we have a significantly different perspective than SCI on the strength of the evidential case for deworming (see our 2012 post on deworming and the comments that follow it).
• Transparency: SCI has consistently been strong in its commitment to transparency. It has generally provided the information we’ve asked for and has never hesitated to share it publicly (unless it had what we felt was a good reason). It has allowed a lot of public dialogue that other charities may have been uncomfortable with.
• Communication: In the past, we have struggled to communicate effectively with SCI representatives, and noted that we lacked important and in some cases basic information about SCI's finances. Our communication with SCI about its finances has improved substantially in 2016.

More on how we think about evaluating organizations at our 2012 blog post.

Sources

• 1

  "Objectives of SCI

  • To encourage development of sustainable schistosomiasis and STH control programmes in sub-Saharan Africa.
  • In the selected countries: to reach at least 75% of school-aged children (which in most countries would be from 6 to 15-year-old) and other high-risk groups with chemotherapy, namely PZQ and ALB; and thereby reducing prevalence and intensity of schistosomiasis and STH infections; as well as reducing schistosomiasis-related morbidity in high risk groups; and burdens due to STH infections in the targeted populations.
  • To create a demand for sustained schistosomiasis and STH control.
  • To promote access to anthelmintic drugs and good case management in the regular health system.
  • To develop a rigorous monitoring and evaluation plan which will generate the information required to determine whether or not the objectives have been met."

  Fenwick et al. 2009, pg. 3.

• 2

  See our November 2015 review here and here.

• 3

  "The move towards national control programmes in sub-Saharan Africa was facilitated by an award from the Bill and Melinda Gates Foundation (BMGF; http://www.gatesfoundation.org) Global Health Program in 2002, to the SCI for the implementation and evaluation of control of schistosomiasis." Fenwick et al. 2009, pg. 2. Amount at Gates Foundation, Imperial College London (June 2002).

• 4

  "Six countries were selected by October 2003 for full support: Burkina Faso, Mali, Niger, Uganda, Tanzania and Zambia. The countries each proposed a different implementation approach and management structure for their large-scale schistosomiasis control. This was readily accepted because the BMGF required SCI to test the ‘proof-of-principle’ of national scale, Ministry of Health (MoH)-led schistosomiasis control programmes. SCI is based in Imperial College London and operated with the principle that all programmes were country owned and run, with SCI staff offering technical and other assistance, but not as expatriates living in-country. Programmes were based in the MoH in the respective country, and SCI offered support to improve the national health system." Fenwick et al. 2009, pg. 2.

  Between 2003 and 2008, SCI provided treatment for schistosomiasis and soil-transmitted helminthiasis to the following number of people (Fenwick et al. 2009, pg. 3, Table 1).

  Number treated by country (millions)

  Year	Uganda	Burkina Faso	Niger	Mali	Tanzania	Zambia
  2003	0.43	-	-	-	0.10	-
  2004	1.23	1.03	0.67	-	0.44	-
  2005	2.99	2.30	2.01	2.60	2.95	-
  2006	1.51	2.82	1.56	2.18	0.38	0.56
  2007	1.81	0.75	2.07	0.65	2.65	0.25
  2008	1.50	2.70	5.28	-	1.24	-

• 5
  • "Current and future rounds of treatment in all six countries are being delivered in an integrated manner to include schistosomiasis, STH, lymphatic filariasis, onchocerciasis and trachoma." Fenwick et al. 2009, pg. 10.
  • The "six countries" refers to the six countries funded by SCI's first Gates Foundation grant.
  • Countries and dates from SCI Board management accounts (April 2010)

• 6

  SCI Burundi: Impact
  SCI Rwanda: Strategy
  SCI Board management accounts (April 2010)

• 7

  SCI Account summary (May 2011).

• 8

  SCI advisory board financial report (June 2014), pg. 3.

• 9

  "A team from Imperial College London has been awarded 25 million funding from the UK Government to continue its fight against neglected tropical diseases, it was announced this week. The money will enable the Schistosomiasis Control Initiative (SCI) to provide 75 million treatments to protect some of the world’s poorest children against schistosomiasis – an illness caused by parasitic worms – and soil-transmitted helminths (STH). £15 million of the funding will be spent directly on procuring drug treatments, through an organisation called Crown Agents. The rest will be administered by SCI." SCI Imperial initiative to protect children from tropical disease awarded ₤25m government backing.

• 10

  ”DFID has allocated 50 million GBP for control of neglected tropical diseases, of which 25 million is going through SCI for schistosomiasis control in 8 countries (including Malawi) over five years.” GiveWell's notes from visit to Malawi on October 17-19, 2011.

• 11

  "SCI will be assisted in their drug delivery by the Centre for Neglected Tropical Diseases at Liverpool School of Tropical Medicine via a subcontract through LATH (Liverpool Associates in Tropical Health). In six countries, this will lead to treatment for lymphatic filariasis – another worm disease – becoming integrated with schistosomiasis and STH treatment." SCI Imperial initiative to protect children from tropical disease awarded ₤25m government backing.

• 12

  SCI advisory board financial report (June 2014), pg. 4.

• 13

  As of July 2011, it had received about $580,000 in unrestricted funding (SCI IC Trust summary (September 2011)) and $108 million overall (SCI Gates Foundation final report (January 2011) Pg. 20).

• 14

  "For the smaller donor, we have two or three projects, which we have been supporting and which will hopefully lead to pilot project in their respective countries.

  1. In Côte d'Ivoire, we now have funding from the U.K. Department for International Development (DFID). There will eventually be a national program.
  2. In Mozambique, we have a doctor running a practice for 70,000 people. We have been funding her to test people, do surveys and give drugs to treat people. Up until now, that has taken all the individual funding that comes in."

  Alan Fenwick, SCI Director, phone conversation with GiveWell, February 16, 2011.

• 15

  Donations to SCI due to GiveWell's research through January 2016 are listed here and total about $14.2 million.

• 16

  We estimate that SCI received about $8.6 million due to GiveWell's recommendation between April 2013 and March 2015 (internal data). Over that period, SCI reports $12.4 million in unrestricted revenue. SCI financial statement 2013/14 and 2014/15 (revised October 2015). SCI's total may include some restricted funds. More details here.

• 17

  SCI allocation table 2016-2017, "Income analysis" sheet:

  • Unrestricted funding total: Income balance at IC Trust at 31 March 2016 (2,133,898 GBP) plus Income balance at L-account at 31 March 2016 (3,428,676 GBP) = 5,562,574 GBP. Cells B2 and B3.
  • Restricted funding total: "Committed from DFID" (4,285,439 GBP) plus "Committed from UBS" (257,753 GBP) plus "Committed from CIFF' (261,645 GBP) plus "Committed from END Fund" (908,860 GBP) plus "Committed from Score" (95,000 GBP) plus "Committed from Gates" (79,105 GBP) plus "Committed from MRC" (45,461 GBP) = 5,933,263 GBP. Cells B5:B11.
  • 5,562,574 /(5,562,574 + 5,933,263) = 48%

• 18
  • GiveWell's non-verbatim summary of a conversation with Alan Fenwick, SCI Director, June 17, 2010.
  • SCI Summary sheet of treatments instigated and overseen by SCI.
  • SCI Proposal by SCI, Imperial College to manage the Program for Integrated Control of Neglected Tropical Diseases in Côte d'Ivoire.
  • Conversations with four program managers, discussed below.

• 19

  Sudan

  • Sudan NTD concept paper (2015-2018)
  • Sudan PZQ and ALB treatments by locality (2015)
  • Sudan annual workplan (April 2015 to March 2016)
  • Sudan cash book
  • Sudan campaign photos
  • Sudan annual workplan for WHO (2015)
  • Sudan joint request for selected PC medicines

  Nigeria

  • Nigeria trip report (June 2015)
  • Nigeria stakeholders meeting summary (May 2015)
  • Nigeria NTD stakeholders meeting and potential SCI involvement scoping document

• 20

  "Prior to SCI’s work in Côte d'Ivoire, there were no large-scale schistosomiasis (SCH) treatment programs." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 21

  "SCI first contacted the government and started to discuss a SCH/STH program in 2010. A civil war delayed progress because there was political turmoil, a weakened health system, and dangerous conditions. In 2012 conditions improved, and the contracts between SCI and the Ministry of Health were set up. [...]
  Treatments started in December 2012:

  • In 2012: 649,859 school-aged children (SAC) received treatments in 12 districts, some of which had been mapped prior to SCI’s involvement.
  • In November 2013: 853,708 SAC received treatments in the areas that SCI mapped first, with the help of the Ministry of Health.
  • In May 2014: 1,425,461 SAC received treatment in the remaining areas of the first phase of mapping.
  • In November 2014: 3.5 million SAC are targeted for the next treatment, which would correspond to the second phase of mapping.

  The aim is that by the end of 2014, all SAC in the SCH-endemic districts will have been treated at least once. Some areas that were treated in 2012 were also treated again in May." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 22

  "In May 2014: 1,425,461 SAC received treatment in the remaining areas of the first phase of mapping. In November 2014: 3.5 million SAC are targeted for the next treatment, which would correspond to the second phase of mapping." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 23

  Alan Fenwick, email to GiveWell, November 3, 2015. Note this is slightly higher than we previously reported in our August 2015 update on SCI.

• 24

  SCI treatment data 2014-16, cell E5.

• 25

  "For the smaller donor, we have two or three projects, which we have been supporting and which will hopefully lead to pilot projects in their respective countries.

  1. In Côte d'Ivoire, we now have funding from the U.K. Department for International Development (DFID). There will eventually be a national program.
  2. In Mozambique, we have a doctor running a practice for 70,000 people. We have been funding her to test people, do surveys and give drugs to treat people. Up until now, that has taken all the individual funding that comes in."

  Alan Fenwick, SCI Director, phone conversation with GiveWell, February 16, 2011.

• 26

  SCI allocation table 2016-2017, "Implementation" sheet, cell B6.

• 27
  • Vitol: Côte d'Ivoire mapping; £102,152; grant period: 01/10/2012 to 31/12/2013. (pg. 3). SCI advisory board financial report (June 2014).
  • DFID ICOSA £177,000 in 2013-14. SCI advisory board financial report (June 2013).
  • DFID ICOSA: £200,000 in year 5 of the grant (pg. 9). (The grant is "A six year contract from October 2010 to March 2016," (pg. 7) and DFID's fiscal year is from April 1 to March (DFID glossary), so year 5 is April 2014 to March 2015.) SCI advisory board financial report (June 2014).
  • SCI has reported $525,264 in spending in Côte d'Ivoire from unrestricted funds between November 2011 and September 2014. GiveWell summary of SCI finances (October 2014), Sheet Combined with previous updates.

  As of November 4, 2014, Google states that £1 is worth $1.6.

• 28
  • SCI financial statement 2013/14 and 2014/15 (revised October 2015)
  • Note that SCI told us, "SCI used unrestricted funds to conduct a community health questionnaire in areas with low coverage. [...] This questionnaire was the main use of unrestricted funding in Cote d’Ivoire in 2014." GiveWell's non-verbatim summary of a conversation with Alan Fenwick and Najwa Al Abdallah, September 14, 2015.

• 29

  GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, cells I11 and K11.

• 30

  "If the 2014 targets are met, it will be an impressive achievement to have treated all SCH-endemic areas at least once since 2012. With just the funding allocated by DFID, the program could not have scaled up as quickly. In this scenario, it probably would have completed the mapping but delivered fewer treatments." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 31
  • "SCI conducted national mapping since late 2012, finishing in March 2014. In total, all 82 districts in Côte d'Ivoire were mapped. This was done in two phases because of limited funding. The first phase was done by the Ministry of Health with support from SCI. The second phase was led by Professor Eliézer N’Goran from the Université Félix Houphouët-Boigny with both technical and financial support from SCI. […]
    The country will start an integrated treatment program for several diseases in November 2014 […]
    Other neglected tropical disease (NTD) organizations have benefitted from the work SCI has done to build a SCH program. SCI has provided cars and developed staff capacity. Organizations have worked to coordinate when sharing these resources and operating an integrated program should make that easier." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.
  • "Cote d’Ivoire is working toward creating an integrated program for the treatment of lymphatic filariasis, onchocerciasis, schistosomiasis, and STH. This will involve collaboration and integration of tools and training, but drugs for all four diseases will not be administered simultaneously. So far, lymphatic filariasis and onchocerciasis treatments have been integrated, and schistosomiasis and STH treatments are administered together but currently delivered separately from lymphatic filariasis and onchocerciasis. [...] SCI's partners in Cote d'Ivoire include Sightsavers, the END Fund, and Helen Keller International." GiveWell's non-verbatim summary of a conversation with Alan Fenwick and Najwa Al Abdallah, September 14, 2015.

• 32

  GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 33

  "Dr. Nogaro has visited Côte d'Ivoire several times over the last year, often dealing with multiple activities on each trip:

  • One month last November to oversee the sentinel sites activity and obtain baseline information for both prevalence and intensity of infection, help with the MDA, and oversee the start of the second phase of mapping,
  • One month in January to work on the coverage survey, work on financial reporting, oversee mapping, and help with drafting the annual work plan and budget (alongside the Ministry of Health),
  • Three weeks in May for another MDA,
  • One week in June to work with WHO and other partners for the integrated MDA plan and budget work, and
  • Three weeks this month to follow up with CHWs in low coverage areas, oversee the sentinel sites activity, and oversee the first integrated MDA and training."

  GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 34

  "Program staff in each country create annual budgets in collaboration with SCI. The budget ensures money is being spent well and on activities defined in the annual work plan, which is submitted by the country to SCI. Generally, receipts are used to document expenses. In some cases, it takes a long time to get the receipts or receipts are not available at all. SCI hired an accountant and an auditor to better monitor finances." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 35

  "Data is entered at the country level. For example, data from mapping and sentinel sites can be handled by students at a local university. The data is double entered to ensure accuracy. The original forms stay in the country and the database is sent to SCI, which can then run analyses. SCI is planning to expand its data analysis team. In the spring, SCI’s biostatistician will teach a data cleaning session to program staff in Côte d'Ivoire. In the future, SCI will consider doing more advanced data analysis training in all the countries it supports." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

• 36

  "In 2007, prior to SCI’s involvement, Save the Children ran a one-time treatment for about 1 million children for schistosomiasis (SCH) and soil-transmitted helminths (STH)." GiveWell's non-verbatim summary of a conversation with Michael French, October 15, 2014.

• 37

  "In mid-2012, the University of Gondar invited SCI to an international symposium on SCH, which served as an introduction for SCI to Ethiopia. After this, Dr. Fenwick, Director of SCI, and other senior SCI staff established a relationship with senior Federal Ministry of Health (FMOH) officials.

  In 2013, SCI moved into more serious conversations with the Ministry of Health. In July 2013 SCI provided support for the launch of the FMOH’s National NTD Master Plan at an international symposium in Addis Ababa. Around the same time, SCI collaborated with the Ethiopian Health and Nutrition Research Institute (EHNRI; now renamed as the Ethiopian Public Health Institute), the technical arm of the FMOH, to plan and fund nationwide mapping. […]
  Last year (2013) SCI funded 1.4 million combined SCH/STH treatments. […]
  The mapping took longer than planned. Teams were trained in November 2013, and the mapping was completed in April 2014." GiveWell's non-verbatim summary of a conversation with Michael French, October 15, 2014.

• 38

  SCI Ethiopia treatment campaign summary report (April 2015).

  • "In total 2.9 million school-aged children received treatment against both schistosomiasis and STH (coverage of 74.0%), comprised of 2.3m enrolled SAC and 571,000 non-enrolled. These numbers are currently being validated using coverage validation surveys in representative districts. These results will be presented as soon as the data is analysed." pg. 1.
  • "Demographic and parasitological data were collected from 22 sentinel sites prior to the April 2015 campaign. Each sentinel site comprised of a school, with 125 children recruited from each school. Each child provided stool and urine samples on two consecutive days in order to estimate the level of parasite prevalence and average intensity of infection. These 22 sentinel sites will be supplemented by a further 15 sites prior to the October 2015 round of treatment to complete the picture on the baseline level of infection. These schools will be followed up annually, just prior to treatment, to understand changes in infection across the course of the programme." pg. 4.

• 39

  SCI treatment data 2014-16, cell F7. Total for April 2015-March 2016 is 7,912,288. ~7.9 million minus ~2.9 million delivered in April 2015 (see above) = ~5 million.

• 40
  • SCI Ethiopia annual treatments 2016, Cells H9:J9.
  • The figure reported in SCI allocation table 2016-2017, "Implementation" sheet is 13.2 million, the total number of children expected to receive schistosomiasis and/or soil-transmitted helminthiasis treatments.

• 41
  • GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet.
  • We discuss SCI's work in Ethiopia in our November 2012, October 2013, and June 2014 updates on SCI.

• 42

  Alan Fenwick, email to GiveWell, September 28, 2015. First transfers from END Fund and DFID funds occurred in 2015. Also:

  • "Getting sufficient funding from SCI this year was straightforward because it was primarily restricted funding from the END Fund. The first year of this 3-year grant ends in February 2015." GiveWell's non-verbatim summary of a conversation with Michael French, October 15, 2014.
  • We note in our June 2014 update on SCI that, as of April 2014, SCI was in the final stages of negotiating the grant from DFID that includes funding for Ethiopia.

• 43

  "He does not know if The END Fund/DFID would have gotten involved without the mapping data. At a minimum, the mapping effort significantly increased the probability of attracting external funding and accelerated the program by a few years." GiveWell's non-verbatim summary of a conversation with Michael French, October 15, 2014.

• 44

  "FMOH [Federal Ministry of Health] began working with SCI about 2.5 years ago. Mr. Shafi's predecessor began speaking to SCI about the possibility of working together on a schistosomiasis program. SCI was the only organization that the government spoke to about this, as it wasn't aware of any other organizations working on schistosomiasis. [...]
  Without SCI's involvement, the program would likely not have gone forward. With the exception of filling small funding gaps on an ad hoc basis, the government has not been able to allocate funding to NTDs. It was interested in doing disease mapping but did not have the funding to do so, and it likely would not have received drug donations without the mapping data." GiveWell's non-verbatim summary of a conversation with Oumer Shafi, November 4, 2014.

• 45

  "In July 2013 SCI provided support for the launch of the FMOH’s National NTD Master Plan at an international symposium in Addis Ababa. Around the same time, SCI collaborated with the Ethiopian Health and Nutrition Research Institute (EHNRI; now renamed as the Ethiopian Public Health Institute), the technical arm of the FMOH, to plan and fund nationwide mapping. […]
  The Deworm the World Initiative (DtWI) will be providing technical advice to Ethiopia, based on its experience in Kenya. […]
  With funding from Dubai Cares, the Partnership for Child Development (PCD) implemented a small pilot program in 30 schools in the southern region (SNNPR) to look at feasibility of a comprehensive school health and nutrition program, which includes deworming, nutrition, and WASH [water, sanitation, and hygiene] interventions and home-grown school-feeding. Related to this, the grant included funding for a region-wide situational analysis, covering an area with about 17 million people. Therefore, the decision for this to be included in the broader national mapping and to include the distribution of SCH, STH, WASH facilities, and school-feeding needs. SCI and PCD partnered on the design, implementation, and analysis of this national mapping. Overall, PCD provided both technical assistance and about 20%-25% of the funding for the national mapping." GiveWell's non-verbatim summary of a conversation with Michael French, October 15, 2014.

• 46

  "Examples of Dr. French's work to assist the Ethiopia program:

  1. Initial program setup activities: creating processes and scheduling meetings to lay groundwork for future activities; developing and finalizing contracts between funders, Imperial Collage [sic] and implementers (the contracts are more complicated than normal because there are 2 funders and 2 implementing agencies).
  2. SCH/STH action plan: This will provide a detailed plan for next 12 months and more general plan for 5 years. The FMOH have tasked Dr. French and SCI to drive this this forward, e.g providing an outline that can be adapted by the country.
  3. Mapping plan: Dr. French was the main technical advisor from SCI. He worked very closely with people from EHNRI to a) write the research protocol, b) advise on the recruitment of trainers, and c) train technicians, d) supervise activities, e) help with data collection and analysis.
  4. Last year’s mass drug administration (MDA): Dr. French was less involved with the MDA than with the mapping but was involved in developing the plans. The Ministry of Health was responsible for running the program and delegating work to regional governments. It had experience from other mass distribution programs. For technical training, it worked with health workers who already had extensive general experience and then provided NTD expertise largely from WHO guidelines, though SCI provided some technical guidance.

  Dr. French has been living in Ethiopia, which has helped him build relationships, enabled him to hear more activity details, and allowed him to provide timely advice. For example, he has stronger relationships with the officials in Ethiopia and a clearer sense of whom to go to for various requests. Also, by being close to the activities, some problems with data flowing back to central level have been clearer, which makes it easier for him to help develop solutions." GiveWell's non-verbatim summary of a conversation with Michael French, October 15, 2014.

• 47

  "Mike French, SCI's program manager for Ethiopia, works in the same office as the government NTD program." GiveWell's non-verbatim summary of a conversation with Oumer Shafi, November 4, 2014. In June 2016 when we sent a draft of this review to SCI, SCI edited this to "works closely with."

• 48

  "Prior to SCI’s involvement in 2010, there was a mapping project in 2008. Dr. Blair does not think there were any schistosomiasis (SCH) treatment programs (other than a small program funded by SCI). If there were, they would have been small-scale." GiveWell's non-verbatim summary of a conversation with Lynsey Blair, October 16, 2014.

• 49

  "The SCH program in Mozambique is run by the Center for Neglected Tropical Diseases (CNTD). Funding for the program is provided to SCI by DFID, and SCI funds CNTD as part of its agreement with DFID." GiveWell's non-verbatim summary of a conversation with Lynsey Blair, October 16, 2014.

• 50

  “When we bid for the DFID grant, I was advised that DFID favored grants that involved partnerships. CNTD does a lot of the work on lymphatic filariasis. DFID was providing 20 million pounds for lymphatic filariasis and 20 million for schistosomiasis. Working with CNTD demonstrated a degree of integration. CNTD does both lymphatic filariasis and schistosomiasis in Mozambique." Alan Fenwick, SCI Director, conversation with GiveWell, October 14, 2014.

• 51

  SCI treatment data 2014-16, cells E12:F12.

• 52

  SCI allocation table 2016-2017, "Implementation" sheet, cell B13.

• 53

  Dr. Wendy Harrison, Najwa Al Abdallah, and Dr. Lynsey Blair, conversation with GiveWell, April 6, 2016

• 54
  • DFID funds:
    • DFID ICOSA £649,514 in 2013-14. SCI advisory board financial report (June 2013), pg. 7, “Overseas country budgets - FY4.”
    • DFID ICOSA: £150,000 in year 5 of the grant (pg. 9). (The grant is "A six year contract from October 2010 to March 2016," (pg. 7) and DFID's fiscal year is from April 1 to March, so year 5 is April 2014 to March 2015.) SCI advisory board financial report (June 2014), pg. 9.
  • Unrestricted funds:
    • SCI draft financial statements for 2013/14 and 2014/15 reported that the unrestricted funds SCI spent in Mozambique summed to £595,421 in 2013-14 and £596,588 in 2014-15. SCI later told us that this document may not be accurate. SCI responses to GiveWell questions on financial statements (October 2015).
    • SCI did not spend any unrestricted funding on the program in Mozambique in its 2015-2016 budget year GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, cells J18:K18.

• 55

  "In January 2011, SCI were awarded three SCORE studies
  TITLE: Studies of Gaining and Sustaining Control of Schistosomiasis

  • Study: Schistosoma haematobium: Gaining and Sustaining Control of Schistosomiasis in Capdelgado, Mozambique
  • Study: Schistosoma haematobium: Gaining and Sustaining Control of Schistosomiasis in Kollo and Say Districts, Niger
  • Study: Schistosoma Haematobium: Gaining and Sustaining Control of Schistosomiasis in Loga District, Niger."

  SCI advisory board financial report (June 2014), pg. 16.
  As of June 2014, SCI had spent £366,937 of the £368,030 by SCORE, and expected to receive an additional $368,229. SCI advisory board financial report (June 2014), Pgs 4-5.

• 56

  See our June 2014 update on SCI.

• 57

  "SCI and CNTD are working closely with RTI International in Mozambique. This has been a great working relationship. RTI is funding some capacity development around drug logistics. It is working on trachoma; currently, this work is focused on mapping, but it will likely fund treatment when mapping is finished. RTI has staff in-country and reports to SCI and CNTD on issues, which can provide some verification that the program is progressing as expected. The organizations coordinate to ensure multiple sources of funding aren't allocated for the same purpose. For example, RTI emailed SCI and CNTD saying that the government had indicated that it was going to drop the SCH program; the staff from the organizations communicated quickly to sort out the confusion." GiveWell's non-verbatim summary of a conversation with Lynsey Blair, October 16, 2014.

• 58

  "The SCH program in Mozambique is run by the Center for Neglected Tropical Diseases (CNTD). Funding for the program is provided to SCI by DFID, and SCI funds CNTD as part of its agreement with DFID. [...] CNTD will continue to run the mass drug administration (MDA), but SCI has become more directly involved with the M&E as it has the technical expertise to manage these activities." GiveWell's non-verbatim summary of a conversation with Lynsey Blair, October 16, 2014.

• 59

  "SCI’s program in Uganda is fully funded by DFID (other than one use of unrestricted funding, discussed below). SCI had previously worked in Uganda with funding from the Bill & Melinda Gates Foundation. After this funding ended, USAID funded work by RTI International in Uganda. Around 2006, Uganda started an integrated NTD program in districts endemic with more than one NTD. Districts with only schistosomiasis (SCH) were not treated.
  When SCI re-entered Uganda, it started SCH programs in the districts not receiving treatment." GiveWell's non-verbatim summary of a conversation with Yolisa Nalule, October 14, 2014.

• 60

  See our August 2015 update on SCI.

• 61

  SCI report to GiveWell (September 2014), Annex 1.

• 62

  SCI treatment data 2014-16, cells E18:F18.

• 63

  SCI allocation table 2016-2017, "Implementation" sheet, cell B20.

• 64
  • DFID ICOSA £138,386 in 2013-14. SCI advisory board financial report (June 2013), pg. 7.
  • DFID ICOSA: £58,994 in year 5 of the grant (pg. 9). (The grant is "A six year contract from October 2010 to March 2016," (pg. 7) and DFID's fiscal year is from April 1 to March, so it's our under year 5 is April 2014 to March 2015.) SCI advisory board financial report (June 2014), pg. 9.
  • GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, cell I25.

  As of November 4, 2014, Google states that £1 is worth $1.6.

• 65

  "SCI’s program in Uganda is fully funded by DFID (other than one use of unrestricted funding, discussed below)...
  In deciding on the budget, there might be some disagreement around whether impact M&E (having sentinel sites) is worth the expense. The government might prefer to provide more treatments. Mrs. Nalule and other researchers argue for the value of M&E.
  Last year, M&E funding was augmented by £23,000 from SCI unrestricted funding. Without this, there likely would have been cuts in the number of sentinel schools. This M&E is enabling research to compare different treatment strategies in areas where prevalence is low. DFID was not interested in funding this research." GiveWell's non-verbatim summary of a conversation with Yolisa Nalule, October 14, 2014.
  Note that SCI has only reported $2,220 in unrestricted spending to us in financial reports. GiveWell summary of SCI finances (October 2014), Sheet Combined with previous updates.

• 66

  SCI draft financial statements for 2013/14 and 2014/15.

• 67

  GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, cell K25.

• 68

  "When SCI re-entered Uganda, it started SCH programs in the districts not receiving treatment. Also, in about 30 districts, USAID stopped treating SCH because prevalence dropped below 20%, so SCI supported the continuation of these treatments with an aim towards elimination. […]
  Also, SCI is taking over treatment from RTI in a 15 districts that have higher prevalence, and, in these districts, it will treat adults. Since prevalence is over 50%, WHO guidelines say to treat children and high risk adults (e.g. fisherman), though usually programs will just try to treat all adults." GiveWell's non-verbatim summary of a conversation with Yolisa Nalule, October 14, 2014.

• 69

  "In some cases (for example, Uganda and Niger), the government program manager understands SCH well and does not require a lot of technical guidance. [...] Uganda is an interesting example. The program had been relatively straightforward to manage. It is now focusing on elimination, which is unknown territory so will require much more guidance. There is a lot of research to do to determine guidelines for how often to treat." GiveWell's non-verbatim summary of a conversation with Lynsey Blair, October 16, 2014.

• 70

  GiveWell's non-verbatim summary of a conversation with Yolisa Nalule, October 14, 2014.

• 71

  GiveWell's non-verbatim summary of a conversation with Blandine Labry, December 15, 2015:

  • "Ms. Labry found it challenging to work in the academic setting of the Imperial College. She believes its procedures and controls are inflexible and not well suited to NGO grant management work. Ms. Labry’s work was also hindered by the fact that she had very limited contact with the Imperial College finance team."
  • "Ms. Labry feels that SCI’s data analysis and reporting capabilities are significantly limited by the Imperial College accounting system. She mentioned, for example, the difficulties involved in having to use spreadsheets to manage accounts and perform detailed analyses of large amounts of data."
  • "There is sometimes a backlog in the arrival of data from country partners, which makes it difficult for SCI to track and manage funds. For example, senior managers are less able to make informed decisions about future funds transfers if they have limited information about what amounts remain in-country."

• 72

  See our February 2016 research plan here.

• 73

  See our 2016 research plans blog post here.

• 74

  SCI allocation table 2016-2017, "Summary" sheet:

  • The sum of income carried forward for DFID, UBS, CIFF, END Fund, Score, Gates, and MRC is £5,933,263.
  • The sum of income carried forward for Unrestricted ICT and Unrestricted IC is £4,934,550
  • As of May 20, 2016, Google states that £1 is worth $1.45

• 75
  • SCI's total budget for its April 2016 to March 2017 budget year (£10,985,319) is £117,506 higher than its total amount of available funding (£10,867,813) SCI allocation table 2016-2017, "Summary" sheet, cells C4, C11, and C13.
  • A July 2015 transfer of $333,414 from GiveWell was initially misallocated by Imperial College; SCI was not aware of this funding until April 2016. This funding is not included in the total amount of available funding in SCI allocation table 2016-2017, "Summary" sheet.
  • SCI told us that it may use the July 2015 funding to fill its £117,506 funding gap. If SCI chose to do so, it would have £112,237 in unrestricted funding remaining ($333,414 is worth £229,743; £229,743 minus £117,506 equals £112,237). Dr. Wendy Harrison and Najwa Al Abdallah, conversation with GiveWell, May 4, 2016
  • As of May 22, 2016, Google states that $1 is worth roughly £0.69.

• 76
  • See our November 2015 review of SCI.
  • See our August 2015 update on SCI

• 77

  SCI notes, "Imperial College has charitable status as a UK Higher education establishment and is audited in line with the requirements of the UK charity commission." Comment provided in response to our draft of this page in June 2016.

  We have looked at Imperial College's financial statements and did not find that they helped us understand SCI's finances.

• 78

  Comment provided in response to our draft of this page in June 2016.

• 79

  Kieran Bird, SCI Finance Manager, conversation with GiveWell, October 16, 2014.

• 80

  "SCI plans to implement a new, standalone accounting software package starting April 1, which will allow SCI to be more flexible and to rely less on Imperial College London's accounting system. SCI only plans to use this system for financial data collected going forward (not for past data)." @GiveWell's non-verbatim summary of a conversation with Dr. Wendy Harrison and Najwa Al Abdallah, February 17, 2016@

• 81
  • In our November 2012 update on SCI (see footnote 1 of that report), we noted inconsistencies in the spending data SCI shared with us. In one case, we asked SCI about a discrepancy we have noticed and SCI's reply was implausible.
  • In our next update on SCI, in October 2013, we reported having a fuller view of how SCI had spent funds, though noted a discrepancy in funds held: "We reported (after checking this number with SCI) that as of October 2012, SCI held $2.65 million in unrestricted funds. As part of this update, we sought to understand SCI's use of unrestricted funds between October 2012 and August 2013. SCI told us that it had held $1.96 million in unrestricted funds as of October 1, 2012, about $660,000 less than we previously thought. We have assumed that we miscommunicated with SCI last year; one possibility is that the numbers we received included some restricted funds and we may not have fully excluded all restricted funds in our adjustment. We have used the more recent figure for this update."
  • In October 2014, SCI shared reports of spending since our last update and balance of unrestricted funds at the end of the period. Combining these figures with our previously published figure for balance of unrestricted funds as of our last update implied negative revenue over the period. We asked SCI to explain the discrepancy. SCI first told us that the discrepancy was due to counting some restricted funds in the unrestricted balance. At the same time, SCI provided data on unrestricted revenue over the period, giving us a double check on these figures. There was a discrepancy between actual revenue and revenue implied by the starting and ending balances and spending over the period of about £528,000. When we asked SCI about this discrepancy, SCI told us that there was an additional expenditure of £595,420 that was missing from the report previous, as well as a small amount of additional income, which reduced the discrepancy to £11,651.
  • In June 2015, SCI shared financial statements for 2013/14 with us that showed no spending in Ethiopia in that year (SCI draft financial statements for 2013/14 and 2014/15). We asked SCI about this because other communications with SCI indicated that SCI had been active in Ethiopia in that year. SCI noted that this was an error and was likely due to confusion by newer staff on how to interpret older financial data: "The three transfers which were made to Ethiopia during 2013/2014 were recorded to the L‐account which is an account the Imperial College currently use to record unrestricted income. In the past however, the L‐account was also used to record expenditures ‐ but later we changed our recording system whereby expenditures were accredited to 'NX' accounts which were created to capture country expenses. Because the transfers went straight from the L account [SCI staff member] did not identify them as being for Ethiopia." Alan Fenwick, email to GiveWell, October 1, 2015.
  • "In October 2015, SCI shared estimates for its April 2016-March 2017 budget year with GiveWell on target treatment numbers by country, amounts of funding available from DFID and other large donors, and the amounts of additional funding required to deliver the targeted number of treatments in each country and cover SCI's central expenditures (http://www.givewell.org/files/DWDA%202009/SCI/SCI_planned_SCH_treatment…). GiveWell interpreted the estimates for SCI's projected central expenditures in the document as part of SCI's funding gap for its 2016-2017 budget year.

    "Documents SCI sent GiveWell in March 2016 indicate that around $1.5 million more in funding from DFID is available to allocate for SCI's 2016-2017 budget year than GiveWell had previously expected. This is because the document from October 2015 included funding available from DFID that could be allocated to in-country programmatic expenditures, but did not include the funding from DFID that SCI planned to allocate to central expenditures (included in the March 2016 documents)." GiveWell's non-verbatim summary of a conversation with Dr. Wendy Harrison, Najwa Al Abdallah, and Dr. Lynsey Blair, April 6, 2016

  • In July 2015, GiveWell granted $333,414 to SCI, which included all donations received between February and March 2015. In March 2016, after reviewing a preliminary version of SCI L-account 2016, GiveWell told SCI that the July 2015 funding did not appear to be accounted for. SCI then learned that the funding had been misallocated by Imperial College. SCI received the funding in April 2016. Dr. Wendy Harrison and Najwa Al Abdallah, conversation with GiveWell, May 4, 2016

• 82

  See previous footnote.

• 83

  GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet.

• 84

  See our November 2015 review of SCI.

• 85
  • "Central programme costs include programme management, travel, monitoring &evaluation, payments to partners, and overhead costs. Previous figure Including 303K to LSTM related to Mozambique rolled from previous fiscal year." GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, Cell N7.
  • We believe that the "previous figure" referred to is from an earlier version of this document.

• 86

  GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, cell N16.

• 87

  "In 2015, SCI implemented the use of cashbooks by its country programs to report monthly spending. Countries where SCI receives funding from the Department for International Development (DFID) implemented this system first, and it has now been rolled out to all of SCI's country programs. A few countries are currently delayed in submitting cashbooks to SCI because of limited capacity." @GiveWell's non-verbatim summary of a conversation with Dr. Wendy Harrison and Najwa Al Abdallah, February 17, 2016@

• 88
  • SCI cashbook Niger 2015-16
  • SCI cashbook Côte d'Ivoire 2015-16
  • SCI cashbook DRC 2015-16
  • SCI cashbook Malawi 2015-16
  • SCI cashbook Tanzania 2015-16
  • SCI cashbook Uganda 2015-16
  • SCI cashbook summary 2015-16

• 89

  "In 2015, SCI implemented the use of cashbooks by its country programs to report monthly spending. Countries where SCI receives funding from the Department for International Development (DFID) implemented this system first, and it has now been rolled out to all of SCI's country programs. A few countries are currently delayed in submitting cashbooks to SCI because of limited capacity." @GiveWell's non-verbatim summary of a conversation with Dr. Wendy Harrison and Najwa Al Abdallah, February 17, 2016@

• 90

  SCI notes, "SCI carries out capacity development of and actively trains in-country accountants on the activity terminology and types of expenses that fall under each which are clearly defined e.g Drug distribution is for expenses related to conducting MDA at a site while drug logistics refer to the transportation of drugs from the source to MDA sites." Comment provided in response to our draft of this page in June 2016.

• 91
  • Total expenditure in 2015-16 was $0.8 million for Malawi and $1.4 million for Tanzania. GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, cells D16 and D24.
  • Country costs sub-total for Malawi is 528,675 GBP ($767,239). GiveWell's analysis of SCI cashbook summary 2015-16, "Malawi" sheet, cell F23.
  • Country costs sub-total for Tanzania is 265,973 GBP ($385,993). GiveWell's analysis of SCI cashbook summary 2015-16, "Tanzania" sheet, cell F23.

• 92

  "The reason for the differences: SCI BvA is prepared based on imperial college information system (ICIS) -expenditure is recorded when a transfer is made to a country while the cash book record when the money is being spent. Example of Tanzania 2 transfers were made: on 16 Dec. 2015 for the amount GBP 344,018 from DFID and GBP 406,696 from unrestricted. In the cash books Tanzania didn’t spend all the money received in Dec. 2015 as the MDA was delayed and took place during Jan-March 2016." Comment provided in response to a draft of this page in June 2016.

• 93
  • £1,458,998 ($2,139,438) in total spending. GiveWell's analysis of SCI cashbook summary 2015-16, "GW calcs" sheet, cell O16.
  • £6,251,569 ($9,428,510) in total country programs expenditure. GiveWell's analysis of SCI budget vs. actuals 2015-16 Redacted, "BVA in USD" sheet, cell L30.
  • According to Google, £1 is worth roughly $1.47 as of May 26, 2016

• 94
  • GiveWell's analysis of SCI cashbook summary 2015-16 "GW calcs" sheet.
  • Descriptions for these activities drawn from monthly spending reports in SCI cashbook Côte d'Ivoire 2015-16, SCI cashbook DRC 2015-16, SCI cashbook Malawi 2015-16, SCI cashbook Niger 2015-16, and SCI cashbook Niger 2015-16.

• 95

  SCI's summary of active accounts as of June 2014 lists six research grants totaling £2.6 million, or about $4.1 million over 2010-2014. SCI advisory board financial report (June 2014), pg. 5.

• 96

  "Once we have people that want to give at least $100,000, we talk to them directly. [...]
  Someone wanted to do something special with his money, so we're doing hydrocele surgery in Niger. He gave us $200,000 and we told him we could do ~1000 hydrocele surgeries.

  Alan Fenwick, SCI Director, phone conversation with GiveWell, February 16, 2011.
  
  
  Alan Fenwick, SCI Director, phone conversation with GiveWell, September 15, 2011. Note: "SCI generally doesn't do water and sanitation programs because of the expense. In Burundi they're doing water and sanitation programming because they have been successful there with running a program and treating schistosomiasis, but soil-transmitted helminth infections remain persistent." GiveWell's notes from visit to Malawi on October 17-19, 2011.

• 97

  Wendy Harrison and Sarah Knowles, SCI Managing Director and Biostatistician, conversations with GiveWell, April 9 and 14, 2014.

• 98

  LSTM Mozambique trip report (May 2015)

  • "With authorization of Dra Olga, I am sharing with you the EXCEL file with results (not the databases), and she reminds me that the data belongs to MISAU." pg. 2.
  • "The Team reviewed the results for 2012-2013-2014 and they confirmed that only 2 sentinel sites were strictly comparable for 2013-2014 (Mecula and Mandiba) and none of the Sentinel Sites for 2015 are going to be comparable to 2014. They decided to review the location of the new sentinel sites, to be able to have a better and stronger comparisons." pg. 2.

• 99

  Citations for all statements in this list can be found in this spreadsheet, Methods sheet, see comments.

• 100

  SCI told us, "Shehias were stratified by MoH treatment programme and by elimination programme; required sample size selected from each startum." Fiona Fleming, email to GiveWell, November 5, 2015. We don't know how to interpret this.

• 101

  "In other areas the survey was easier but in Chinde Islands where Dr Xose went from one island to the next to find the selected village and got lost for a few days because the district has been divided into two and we had to select again the villages." Fiona Fleming, email to GiveWell, November 5, 2015 (quoting from email from FPSU, which runs the program in Mozambique).

• 102

  Fiona Fleming, email to GiveWell, November 5, 2015 (quoting from email from FPSU, which runs the program in Mozambique).

• 103

  Fiona Fleming, email to GiveWell, November 5, 2015.

• 104
  • For Côte d'Ivoire, coverage reported by parents was lower than coverage reported by children. In Malawi, coverage reported by parents was similar to coverage reported by children. In Côte d'Ivoire, parents' answers were excluded from the reported results: "Calculation of validated coverage rates initially included answers from both proxy and direct interviews across all 4 districts. As shown in Figure 2, coverage rates calculated based on direct interviews were higher than those which included responses given by proxy (p < 0.001), perhaps due to the parent erring on the side of caution when giving their answer. As direct interviews are believed to be more robust, we omitted data from proxy interviews when calculating final coverage rates (Table 4)." SCI Côte d'Ivoire coverage survey 2014, pg. 10.
  • In Malawi, parents' answers were included in reported results. Proxy results were similar to results from direct interviewing in Malawi, and were included in the headline analysis. SCI Malawi coverage survey 2012, pg. 11, Figure 4.
  • About 9% of the responses in Côte d'Ivoire were from parents. SCI Côte d'Ivoire coverage survey 2014, pg. 9, Table 3.
    • Number of yes/no answers obtained in person (summed across four districts): 2178
    • Number of proxy yes/no answers obtained (summed across four districts): 228

• 105

  Fiona Fleming, email to GiveWell, November 5, 2015.

• 106
  • Côte d'Ivoire: “The surveyors may ask additional questions to remind the children and see if the answers seem believable, for example:
    • Show children the dose poll and ask if the children remember it
    • Ask the children what they remember about the pills (it is normal to remember praziquantel's bitter taste, size and smell)
    • Ask the children how many pills they took (while trying to judge how many they should have received by getting them to stand near the dose pole)."

    GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014.

  • For Malawi (2014), SCI told us that verification methods were used, but the data was not included in the survey report. Fiona Fleming, email to GiveWell, November 5, 2015.
  • For Uganda, Zanzibar, and Zambia, SCI told us that the survey "asked if [respondents] recognised pills / dummy pills and dose poles'." Fiona Fleming, email to GiveWell, November 5, 2015.

• 107

  "The surveyors only record if the children say they took the drug, didn’t take the drug, or are unsure; currently, the surveyors do not track answers or concerns from the additional questions, though going forward SCI will consider including these as well." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014

• 108

  "The actual sample included caretakers of 578 children." SCI Mozambique coverage survey 2015, pg. 5. Note that we have only received permission to publish a summary of this report. Quotation is from the full, unpublished report.

• 109

  SCI Mozambique coverage survey 2015, pg. 23. Note that we have only received permission to publish a summary of this report. Quotation is from the full, unpublished report.

• 110

  SCI Mozambique coverage survey 2015, pg. 24. Note that we have only received permission to publish a summary of this report. Quotation is from the full, unpublished report.

• 111

  "Pole doses were used as a proxy for the medication – rather than showing the medication we showed them the poles. Only parents (not teachers) can authorize medication to kids in Mozambique and they need to be involved and informed during the social mobilization which is conducted usually by the district officers and the activists, in the week preceding the distribution of the medication. The activists use posters and poles when they inform the community and the poles are very recognizable. The poles are also used in the integrated campaigns – and in most of these districts campaigns have been conducted for a number of years. So…they may be recognized as part of a preventive medication campaign rather than related to the PZQ only." Fiona Fleming, conversation with GiveWell, November 5, 2015 quoting from an email from FPSU.

• 112

  GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014

• 113

  See notes in the "data quality control" column of this spreadsheet, sheet Methods.

• 114

  Email from SCI's partner FPSU, quoted to us by SCI:
  "Further to our conversation, here is a recount of the supervision of the survey:

  1. Initially there were 6 teams with Cabo Delgado and Nampula divided into two. All of them were supervised by Don and myself during 2 days of intense data collection in Nampula Cidade to ensure the areas selected were visited, the random selection of houses and respondent and the correct administration of the questionnaire plus the team organization and supervision.
  2. Teams were then reduced to 4 as it made more sense to complete districts before traveling to the next.
  3. Each team had one or two supervisors, one being the provincial NTD or M&E staff in the case of Zambezia, the national supervisor being NTD or other MISAU Departments (Cabo Delgado, Zambezia). The district officers worked in teams so that most of the interviews in one district were done by the other district officers with no connection with the district. In fact in most cases the district officer of that particular district was solving logistical and administrative issues which are quite intense in Mozambique (permission by the provincial delegate, permission by the district and the locality chief letters of introduction all signed and stamped, motorcycles rental and bills etc).
  4. Nampula was identified as the most complex area and the team was supervised by me during the completion of Ilha and Mossuril districts plus Cidade Nampula while Don supervised District Nampula and again the finalization of Cidade Nampula which was pretty complicated. In other areas the survey was easier but in Chinde Islands where Dr Xose went from one island to the next to find the selected village and got lost for a few days because the district has been divided into two and we had to select again the villages. All other supervisors provided daily reports to Don. They also wrote a report of the questionnaires received every day, reviewed and sent.
  5. In most cases, data was sent to the database and reviewed every day. Once the supervisor had sent the questionnaire editing was only allowed to Don Whitson. Some areas did paper based questionnaires and this was again submitted and reviewed."

  Fiona Fleming, email to GiveWell, November 5, 2015.

• 115

  "MDA of PZQ across the 30 ICOSA districts were supported by the two partner organisations, SCI (ICOSA programme) and RTI (ENVISION programme), responsible for supporting SCH treatment in the country. ICOSA supported delivery in80 sub-counties and ENVISION in 47. Since the drug delivery may vary between the programmes, and thus potentially affect coverage, it was decided that stratification would be by the partner programme who supported the delivery of the MDA by the MoH i.e. ICOSA and ENVISION." SCI Uganda coverage survey 2014, pg. 6.

• 116

  "We found significant differences between districts in validated coverage rates, with three out of the four districts surveyed having an overall validated coverage rate above the target of 75% set by the WHO." SCI Côte d'Ivoire coverage survey 2014, pg. 3.

• 117

  Tohon et al. 2008. In 2014, SCI sent us a more recent report, SCI Niger panel study 2011. SCI Niger panel study 2011 is our only source of data on the second to fifth year followups. This analysis excludes three of the eight schools that were originally in the study as well as the 56% of participants lost to follow-up in the remaining five schools. Participants who were followed up for some years but lost to follow-up by the end of the study are completely excluded rather than being included in a separate analysis of the earlier follow-ups. The 3 schools were apparently dropped because they were not surveyed in some years:

  • "Of these eight sentinel sites, five had data available in all six years of the study up to 2010." SCI Niger panel study 2011, pg. 19.
  • "455 out of 1024 (44%) children recruited at baseline from November 2004 to April 2005 were successfully followed up for the full duration of the 6-year sentinel site monitoring." SCI Niger panel study 2011, pg. 25.

  For this reason, we feel that the results from SCI Niger panel study 2011 are hard to interpret, and we don't present them here, relying instead on Tohon et al. 2008.

• 118

  We have seen two reports on the Burundi study, Styles 2011 and Koukounari 2011, which use different methods of analysis and included different numbers of participants. Which individuals they included in their analyses is not always clear, however the fact they report similar results for prevalence provides some evidence that the results are not highly dependent on these choices. (Both Dr. Koukounari and Dr. Styles are statisticians who formerly worked at SCI.)

  • Pilot study: Styles 2011 and Koukounari 2011 have strengths and weaknesses in defining the sample for the pilot study. Styles 2011 includes a much larger number of individuals (Styles 2011, pg. 3, Figure 1b compared to 710 in Koukounari 2011, pg. 7, Figure 2). However, it is unclear whether the individuals in the follow-up were all in the original cohort or whether some were added to the sample later. The two reports indicate different numbers of students included in the study and retraced at each followup. We have seen two explanations for the differences:
    (1) "Dr. Koukounari only included students who were in first grade during the first year of the study and who were successfully surveyed every year of the study. In addition to the children counted by Dr. Koukounari, Dr. Styles included students who entered first grade and were added into the study in subsequent years, as well as students who were missing data from some years. Each of these strategies for data analysis has benefits and drawbacks. SCI initially planned to do a cross-sectional evaluation of sixth grade students every year, because each year the current sixth grade class would have received more rounds of treatment over the course of elementary school than the previous year. SCI did not complete this plan, but Dr. Koukounari included the data from the sixth grade students in the baseline data. Dr. Styles did not include this data." GiveWell's non-verbatim summary of a conversation with Giuseppina Ortu on June 20, 2014.
    (2) "In addition to the longitudinal studies at each follow-up newly recruited children were added to these surveys. At 1st follow-up (2008) 2288 newly recruited children were added to these surveys with range age: 6-21 years old and median age: 12 years old. Of these 2288, only 210 i.e. (9.18 %), were of age 6 and eligible to be included in the specific cross sectional data analysis. At 2nd follow-up (2009) 2311 newly recruited children were added to these surveys with range age: 5-20 years old and median age: 11 years old. Of these 2311, only 160 i.e. (6.92 %), were of age 6 and eligible to be included in this specific data analysis. Finally, at 3rd follow-up (2010) 2224 newly recruited children were added to these surveys with range age: 6-20 years old and median age: 12 years old. Of these 2224 only 189 i.e. (8.50 %) were of age 6 and eligible to be included in this specific data analysis." Koukounari 2011, pg. 6. Koukounari 2011 notes that it only includes the 20% of participants in the pilot study who were tracked through all follow-ups: "Finally at 3rd follow-up (2010) there were 713 children successfully followed-up (i.e. follow-up rate=19.71%). Longitudinal analyses for the 4 years are presented in the next pages for these 713 children." Koukounari 2011, pg. 6.
  • Other schools: Koukounari 2011 claims that 5,700 participants were recruited at baseline: "At baseline (2008) there were recruited 5700 children while the follow-up rate one year later was 53.42 % (3045/5700)." Koukounari 2011, pg. 16. Styles 2011 pg. 13, Table 8 claims there were 3,781 participants recruited at baseline. SCI later told us that this may be because Koukounari 2011 included data from a group of students who were surveyed for a concurrent cross-sectional study, which was not completed: "SCI initially planned to do a cross-sectional evaluation of sixth grade students every year, because each year the current sixth grade class would have received more rounds of treatment over the course of elementary school than the previous year. SCI did not complete this plan, but Dr. Koukounari included the data from the sixth grade students in the baseline data. Dr. Styles did not include this data." GiveWell's non-verbatim summary of a conversation with Giuseppina Ortu on June 20, 2014. Both studies include a similar number of participants at the first follow up: "At baseline (2008) there were recruited 5700 children while the follow-up rate one year later was 53.42 % (3045/5700)." Koukounari 2011, pg. 16; Styles 2011 pg. 13, Table 8 shows about 3030 students were included in the first year follow-up analysis.

• 119

  "The baseline impact survey was carried out in March 2012 with the 1st follow (FU1) up being done in March 2014 prior to the mass drug administration in April 2014. This report summarises and discusses the results from the 2nd follow-up survey (FU2) which was carried out in the 22 sentinel schools in the districts of Balaka,Blantyre, Chiradzulu, Lilongwe, Mwanza, N. and S. Mzimba, Neno, Ntcheu and Ntchisi in March 2015." SCI Malawi impact study – second follow up, pg. 3.

• 120

  Kabatereine et al. 2007

• 121

  Touré et al. 2008 and Koukounari et al. 2007

• 122

  SCI M&E timeline (May 2015). Reports with first follow up results expected:

  • Côte d'Ivoire: Not specified. Data collection expected in either May or November 2015.
  • Liberia: Delayed. New data not specified. Fiona Fleming, email to GiveWell, September 18, 2015.
  • Ethiopia: Not specified. Data collection expected in September 2015.
  • DRC: Not specified. Data collection expected in October 2015.
  • Mozambique: Low quality data collected so far. Next round of data collection expected to provide more information. LSTM Mozambique trip report (May 2015).
  • Tanzania: Was expected in September 2015. No recent update.
  • Zambia: Not specified. Data collection expected in July 2015.
  • Uganda: Not specified. Data collection expected in October 2015.
  • Niger: SCI notes that data was collected in 2011, 2012, 2013, and 2014. Report was expected in September 2015. No recent update.

• 123
  • Niger: “Praziquantel (using dose-pole corresponding to 40 mg/kg) and Albendazole (400 mg) were given to the target population regardless of infection status, during the mass drug administration campaign that took place 3–4 weeks after the surveys were conducted.” Tohon et al. 2008, pg. 3. “A total of 89% of the initial sample group were re-examined one year after baseline data collection and the first round of treatment with praziquantel and albendazole.” Tohon et al. 2008, pg. 4.
  • Burundi pilot: “At baseline (2007) there were recruited 3616 children. At 2008 the 1st follow-up took place where 1188 children were retraced since baseline (i.e. follow-up rate=32.85 %). At 2nd follow-up (2009) there were 1004 children successfully followed up since baseline (i.e. follow-up rate=27.77%). Finally at 3rd follow-up (2010) there were 713 children successfully followed-up (i.e. follow-up rate=19.71%). Longitudinal analyses for the 4 years are presented in the next pages for these 713 children.” Koukounari 2011, pg. 6.
  • Burundi other schools: “At baseline (2008) there were recruited 5700 children while the follow-up rate one year later was 53.42% (3045/5700).” Koukounari 2011, pg. 15.
  • Malawi: “A longitudinal survey design requires baseline data collection from schools prior to the initiation of large-scale distribution of praziquantel and albendazole or mebendazole through the school-based platform. Follow up surveys will be conducted immediately prior to subsequent rounds of treatment for the life of the programme to monitor the impact of the health intervention.” SCI Malawi panel study, pg. 3. “During the baseline survey, cohorts of 125 children from Standards 1, 2 and 3 (aged approximately 6, 7 and 8 years) were randomly selected in each of the schools and enrolled into the study. [...] This group of selected children, now in standards 2, 3, and 4, as well as a new group of 40 Standard 1 children, were re-tested to measure the same indicators during the 1st follow-up.” SCI Malawi panel study, pg. 4.
  • For all three studies, the methodology does not discuss a control group, and with context, it is sufficiently clear that there was not one.

• 124

  "This survey year saw a switch from a longitudinal survey design to a cross-sectional design which occurred following internal SCI reviews of the data and issues arriving from field surveys. The change in survey design led to the ages of children included in the study to be altered slightly to allow for like for like comparison over time and to capture those with the highest burden of infection." SCI Malawi impact study – second follow up, pg. 3.

• 125

  SCI notes that because children in a control group would be tested for infection, ethical guidelines would require that those found to be infected receive treatment and thus would no longer serve as controls.

• 126

  Sources for the data in the tables:

  • Niger: Tohon et al. 2008. SCI recently sent us a more recent study on the Niger program, SCI Niger panel study 2011. However, for reasons described elsewhere in this report, we continue to rely on Tohon et al. 2008.
  • Uganda: Kabatereine et al. 2007.
  • Burundi: Styles 2011, and Koukounari 2011. As we discuss elsewhere in this report, both analyses have methodological limitations, and it is not clear which is more reliable. The results are very similar for worm prevalence, and Styles 2011 reports an additional year of data, so we rely on the Styles 2011 data for worm prevalence. Styles 2011 does not report infection intensity, so we rely on Koukounari 2011 for that data.
  • Malawi: SCI Malawi impact study – second follow up, pg. 11, columns for 6-8 year olds only.

• 127“Before treatment, the overall prevalence of S. heamatobium infection was 75.4% of the 1,642 enrolled children...One year after a single-dose praziquantel treatment (administered using the WHO PZQ dose pole) co-administered with albendazole (400 mg single dose) for deworming, the prevalence of S. haematobium infection was 38%.”
  Tohon et al. 2008, Abstract.
• 128
  • "Before treatment...21.8% of children excreted more than 50 eggs/10 ml urine." Tohon et al. 2008, Abstract.
  • "The overall prevalence of S. haematobium infection was 38% and 4.6% of children had heavy-intensity infections; only three (4.6%) among the latter excreted more than 500 eggs/10 ml." Tohon et al. 2008, pg. 4.

• 129"S. mansoni infection was observed only in 2 schools: Sabon Birni (3%) and Sanguile ́ (1.1%)." Tohon et al. 2008, pg. 3
• 130"Hookworm infection was observed in 3 schools; Sabon Birni, where the prevalence was 18.8 and in 2 other villages were the prevalence was 0.6%. Hookworm infection was not observed in the schoolchildren of the 5 other villages." Tohon et al. 2008, pg. 3
• 131Alan Fenwick, phone conversation with GiveWell, November 28, 2011.
• 132Styles 2011, Table 2 pg. 2. The same means seem to be presented in Figure 1, pg. 3.
• 133Eggs per gram. All epg data are the average over all participants, including those not infected.
• 134Koukounari 2011, Figure 6, pg. 10.
• 135Styles 2011, Table 3, pg. 4.
• 136Koukounari 2011, Figure 6, pg. 10.
• 137Alan Fenwick, phone conversation with GiveWell, November 28, 2011.
• 138Styles 2011, Table 9, pg. 15.
• 139Koukounari 2011, Figure 21, pg. 20. Note that the values are only visible in the Word version of the document.
• 140Styles 2011, Table 10, pg. 16.
• 141Koukounari 2011, Figure 21, pg. 22.
• 142SCI Malawi impact study – second follow up, Table 3, pg. 11, columns for 6-8 year olds only.
• 143SCI Malawi impact study – second follow up, Table 3, pg. 11. The first follow up report stated that there were no heavy infections at baseline:
  • SCI Malawi panel study, Pg. 11, Table 7.1.
  • SCI Malawi panel study, Pgs. 19-20, Table 8.2, "Followed-up" column.

  We don't know the reason for the change in results.

• 1442.2% at baseline to 0.0% at one year to 1.5% at two years. SCI Malawi impact study – second follow up, Table 3, pg. 11, columns for 6-8 year olds only.
• 1450.2% at baseline to 0.0% at both one and two years. SCI Malawi impact study – second follow up, Table 3, pg. 11, columns for 6-8 year olds only.
• 146SCI Malawi impact study – second follow up, Table 3, pg. 11, columns for 6-8 year olds only.
• 147SCI Malawi impact study – second follow up, Table 3, pg. 11 columns for 6-8 year olds only.
• 148
  • Niger: "Very low prevalence (0.3 to 0.7%) of Ascaris lumbricoides infection was observed in 5 schools, while 3% of the schoolchildren were infected in 1 school (Sanguile) and no infection was observed in 2 schools (Kaou and Tabalak)." Tohon et al. 2008, pg. 3. “The very low prevalence of soil-transmitted helminth infection is in accordance with previous observations made in Niger [32] and in 2 other African countries, Mali and Chad [22,33], areas that are subjected to climatic conditions comparable to the most inhabited regions of Niger.” Tohon et al. 2008, pg. 5.
  • Malawi: No Ascaris or Trichuris infection reported. SCI Malawi impact study – second follow up, Table 3, pg. 11, columns for 6-8 year olds only.

• 149

  Results from Styles 2011. We report "as measured" results for Burundi; SCI also reports model results.

  	Baseline	Year 1	Year 2	Year 3	Year 4
  Ascaris in Burundi (pilot)	14.9%	12.9%	20.1%	10.6%	10.1%
  Trichuris in Burundi (pilot)	3.2%	1.8%	3.9%	1.5%	2.4%
  Ascaris in Burundi (other schools)	21.6%	11.7%	-	9.1%	-
  Trichuris in Burundi (other schools)	10.4%	10.0%	-	4.3%	-

• 150
  • Niger:
    • "Eight villages located in schistosomiasis endemic regions were randomly selected to represent the two main transmission patterns in Niger: six villages located near permanent (Tabalak, Kokorou) or semi-permanent (Kaou, Mozague, Rouafi, and Sabon Birni) ponds and two (Saga Fondo, Sanguile) located along the Niger River. The villages represented the south-western region (Tillabe´ry) and the central-northern region (Tahoua) of the country, with four villages from each region. One village is located in the Sudanian climatic zone and the seven others are in the Sahelian climatic zone." Tohon et al. 2008, pg. 2.
    • SCI told us that these locations "are not representative of the treatment population as a whole. They were selected to indicate the impact of treatment in schools with varying prevalence and intensity of both [types of schistosomiasis]." Anna Phillips, SCI Country Program Manager for Burkina Faso and Niger, email to GiveWell, October 13, 2011.
  • Burundi pilot survey: “More precisely, the 12 schools were chosen based on 3 zones-believed at the time that they would have the majority of NTDs. 4 schools were selected randomly so that they represent the ‘STHs +Schisto +oncho’ zone (these were Musenyi, Nyamibu, Munyika, Rukinga); then another 4 schools were selected randomly so that they represent the ‘STHs +oncho’ zone (Mirombero, Kizuga, Ruzibira, Mudende) and finally 4 schools were selected randomly so that they represent STHs only endemic areas (Gatwe, Ruko, Condi, Gitobo). Such decisions were based on available historic data. Thus, SCI Programme Manager advised not to stratify the statistical analysis by province and so such results (i.e. stratification by province) are not presented anywhere in this report.” Koukounari 2011 pg. 6.
  • Burundi other schools: it appears that schools were selected to be representative, though this is not fully clear in the reports we have seen.
    • Styles 2011 says, "The selection of schools was done randomly from the non-pilot provinces; taking into account 11 separate ecological zones." pg. 13.
    • Koukounari 2011 says, “At baseline (2008) there were recruited 5700 children while the follow-up rate one year later was 53.42% (3045/5700). For these set of studies as they were designed to cover almost all of the country, it is worthwhile to also examine stratifications of analyses by district and such results are also presented in the following subsections. However, in most of the districts the children were coming only from 1 school (see relevant graphs for district whenever n<200; when this is the case then this is only 1 school per district and thus results should be treated there with caution and programmatic decisions to be taken with reservations). Whenever/wherever this is the case, results should be interpreted with caution as just 1 school would be quite ‘risky’ to represent inference/decisions for a whole district.” pg. 16.
  • Malawi: The schools seem to have been selected in a way that makes them representative of districts with moderate to high prevalence, which are those districts that receive annual treatment (low prevalence districts receive more limited treatment). The schools were selected only from districts found to have moderate schistosomiasis prevalence in SCI's mapping: "Method of sentinel site selection: SCI’s protocol is to monitor only in those districts where prevalence of schistosomiasis is moderate or high i.e. SCI does not monitor in non-endemic or low prevalence districts where a full control program is not implemented. All districts except Mzuzu City surveyed in the mapping in February 2012 were determined to have moderate prevalence of S. haematobium, and consequently all districts except Mzuzu City were included in the selection of sites to be monitored for this species (see Table 1). S. mansoni infection was more focal and only present at moderate prevalence in Chiradzulu, Blantrye Rural, Lilongwe City and Lilongwe Rural East. Following district stratification by S. mansoni infection, such that the number of schools selected for S. mansoni monitoring, reflected the frequency of moderate risk areas in the monitoring areas, 22 schools were selected that would be monitor S. haematobium infection with a subset of 9 schools which also monitor S. mansoni infection. Due to the low prevalence of STH’s, STH infection was only monitored in those schools where the Kato-Katz slides were already prepared for S. mansoni." SCI Malawi panel study, pg. 4; SCI's senior biostatistician told us that the sampling method would produce a sample representative of the treated districts. Michelle Clements, SCI Senior Biostatistician, conversation with GiveWell, October 15, 2014.

• 151
  • Niger: "A total of 89% of the initial sample group were re-examined one year after baseline data collection and the first round of treatment with praziquantel and albendazole." Tohon et al. 2008, pg. 4.
  • Burundi:Pilot schools: 33%: “Without taking into consideration the parasitological exams, at baseline (2007) there were recruited 3616 children. At 2008 the 1st follow-up took place where 1188 children were retraced since baseline (i.e. follow-up rate=32.85 %).” Koukounari 2011. 50%: Styles 2011 pg. 1, Table 1. Other schools: 53%: “At baseline (2008) there were recruited 5700 children while the follow-up rate one year later was 53.42 % (3045/5700).” Koukounari 2011, pg. 16. 80%: Styles 2011 pg. 14, Table 8. The discrepancy between the populations included in Koukounari 2011 and Styles 2011 is described in more detail below.
  • Malawi: "Overall, the drop-out rate was higher than expected for both species of schistosomiasis. 48% of those pupils monitored for S. haematobium dropped out the study between baseline and follow-up, and 64% of those pupils monitored for S. mansoni dropped out of the study." SCI Malawi panel study, pg. 17. Since the most meaningful results from Malawi were for Schistosoma haemotobium, we focus on the follow-up rate for that species.

• 152

  Benjamin Styles, SCI Senior Biostatistician, phone conversation with GiveWell, August 12, 2011.

• 153

  "An initial analysis of approximately 2,000 individuals in selected sentinel sites has demonstrated an over 50% reduction in infection markers (both prevalence and intensity) following two rounds of treatment. Infection with either species of schistosomiasis has fallen from 19.8% at baseline to 8.3%, following two rounds of treatment, with less than 4% now harbouring heavy infections (and who are those most likely to develop severe morbidity)(Table 2)." Summary Technical Report: Schistosomiasis Control in Yemen (July 2014), pg. 4. Table with results for S. haematobium and S. mansoni is on pg. 5.

• 154

  "As part of this, a cohort of 8,000 individuals (6,500 school-aged children and 1,500 adults) from 36 representative districts across the country were sampled for demographic and parasitological markers at baseline prior to intervention and following treatment. An initial analysis of approximately 2,000 individuals in selected sentinel sites has demonstrated an over 50% reduction in infection markers. [...] To confirm these results, and to determine whether they are reflected elsewhere, a full round of impact evaluation in all 36 districts will be implemented in September 2014 prior to the next treatment campaign." Summary Technical Report: Schistosomiasis Control in Yemen (July 2014), Pgs 4-5.

• 155

  "At baseline in 2010, prior to the first treatment campaign, a small-scale mapping survey was carried out in in areas without previous mapping data. The results from these surveys (120 schools in 9 governorates) were combined with mapping surveys conducted from 2002-2010, along with ecological information relevant to schistosomiasis transmission obtained from the literature." Summary Technical Report: Schistosomiasis Control in Yemen (July 2014), pg. 3.
  Results from "2010" include 333 districts, which seems to indicate that only a portion of this data (120 of 333 districts) was collected in 2010 and the remainder was collected several years before. Mapping of Schistosomiasis and Soil-transmitted helminthiasis in Yemen, pg. 42.

• 156

  "After 2-3 rounds of treatment a prevalence re-mapping survey was conducted to map the distribution of infection and provide an overview of the programme’s impact on NTD prevalence from baseline levels in 2010, as well as set the control approach for upcoming years of the programme." Mapping of Schistosomiasis and Soil-transmitted helminthiasis in Yemen, pg. 9. Full results on pg. 42, Table 13.

• 157

  "The research summarised in this paper is based upon field-work funded by the Schistosomiasis Control Initiative (SCI), Imperial College, under the auspices of the Bill and Melinda Gates Foundation. [...] We are grateful to Professor Alan Fenwick, director of SCI, for his unfailing support, and his encouragement of our work, even when our findings made uncomfortable reading. We are also grateful to other staff from SCI, some of whom have disagreed with our analysis, but who have continued to provide assistance, including Lynsey Blair, Fiona Fleming and Artemis Koukounari." Allen and Parker 2011, pg. 114.

• 158

  We examined the references from Parker, Allen, and Hastings 2007, Allen and Parker 2011, Parker and Allen 2011, Allen and Parker 2012, and Parker and Allen 2014, Muhumuza et al. 2013 and Standley et al. 2009. We searched on Google Scholar for "niger schistosomiasis prevalence," “niger deworming uptake," "niger praziquantel uptake," "Burundi schistosomiasis prevalence," "Burundi deworming uptake," "Burundi praziquantel uptake," "schistosomiasis prevalence Africa," "deworming uptake Africa," "praziquantel coverage Africa," "Uganda schistosomiasis prevalence," and "Uganda praziquantel uptake."

• 159

  For example:

  • SCI Director Alan Fenwick and SCI-affiliated researcher Narcis Kabatereine were coauthors on Brooker et al. 2005 and Standley et al. 2010.
  • Narcis Kabatereine, SCI Africa Capacity Building Adviser, was a co-author on Standley et al. 2009.
  • Lynsey Blair, SCI Senior Project Manager, was a coauthor on Stothard et al. 2013.
  • Knopp et al. 2009 was funded in part by SCI: "Financial support: This study received funding from the Swiss National Science Foundation (projects PPOOB–102883 and PPOOB– 119129), the Natural History Museum (supported through The Health Foundation,United Kingdom),the World Health Organization (OD/TS- 07-00331), the European Union (FP6 STREP CONTRAST project, contract 032203), and the Schistosomiasis Control Initiative." pg. 1077.

• 160

  See the spreadsheet for details and citations.

• 161

  Wikipedia entry for Unguja

• 162

  "The study was designed specifically to target schools on an ad hoc basis as an 'ambush' strategy, to prevent visiting only schools which are commonly surveyed or well-known by local health officers. By doing so, some of the more overlooked schools may have been sampled here, which correspondingly had lower incidences of treatment, irrespective of their prevalence levels." Standley et al. 2009, pg. 50.

• 163

  
  SCI Côte d'Ivoire panel study baseline report, pg. 7.

• 164

  "Single-dose oral therapies can kill the worms, reducing ... infections by 99 percent ... Reinfection is rapid, however, with worm burden often returning to eighty percent or more of its original level within a year ... and hence geohelminth drugs must be taken every six months and schistosomiasis drugs must be taken annually." Miguel and Kremer 2004, pg. 161.

• 165

  GiveWell's notes from visit to Malawi on October 17-19, 2011. Updated in Alan Fenwick, SCI Director, email to GiveWell, November 24, 2014.

• 166

  GiveWell's notes from visit to Malawi on October 17-19, 2011.

• 167
  • Uganda: "The first country to implement a control programme on a national scale...Uganda implemented the SCI-supported control programme in April 2003." Kabatereine et al. 2007, pg. 91.
  • Burkina Faso: "Some small-scale control activities with treatment had taken place in some areas in the past,11,13 but the national control programme did not start until 2004." Touré et al. 2008, pg. 780.
  • Niger: "Niger’s National Schistosomiasis and Soil-Transmitted Control Programme (PNLBG) was launched at 2004." Fenwick et al. 2009, pg. 5.
  • Mali: "In the following years many planned activities were not implemented due to limited financial resources but finally in 2004 national control activities recommenced in the country with support from the SCI." Fenwick et al. 2009, pg. 5.
  • Tanzania: "The failure to embrace a national treatment programme has been due primarily to the costs involved in reaching the millions of individuals estimated to be at risk of infection, and the Ministry of Health was never able to support treatment within their budget. In October 2003, the Tanzanian National Plan was approved for funding by the SCI as a step towards developing a sustainable control programme." Kabatereine et al. 2006, pg. 334.
  • Zambia: "'The Zambian Bilharzia Control Programme' (ZBCP) was established in 2004 to develop a MoH and MoE joint strategy for bilharzia and worm control. The MoE was already in receipt of a grant from the United States Agency for International Development (USAID) for implementing training and treatment in some schools on a small scale in two provinces, Eastern and Southern, which was known as the ‘School Health and Nutrition programme’ (SHN)." Fenwick et al. 2009, pg. 4.

• 168

  "Without SCI's involvement, the program would likely not have gone forward. With the exception of filling small funding gaps on an ad hoc basis, the government has not been able to allocate funding to NTDs. It was interested in doing disease mapping but did not have the funding to do so, and it likely would not have received drug donations without the mapping data." GiveWell's non-verbatim summary of a conversation with Oumer Shafi, November 4, 2014.

• 169

  "Drug distribution channels:

  • School-based delivery for school children. School teachers will be trained to carry out drug distribution at schools.
  • Community-based delivery for school-aged children who are not attending school and for community adults at high risk. Community Drug Distributors (CDD) will be trained to deliver the drugs at community.
  • Health centre-based delivery. Drugs will be made available at health centres for those in the community who do not qualify for MDA and who request for treatment. Health workers at the centres will be trained.

  Drug distributors need a minimum of one day’s training to understand the basis for calculating dosages, the necessary actions to deal with side-effects and treatment record keeping and reporting." SCI Neglected tropical diseases in Mozambique, pg. 23.
  "For schistosomiasis and STHs, treatment will be conducted through schools by the teachers. For LF, treatment will be conducted through community directed treatment, by the CDDs and community health agents, managed by the district medical officer." SCI Proposal by SCI, Imperial College to manage the Program for Integrated Control of Neglected Tropical Diseases in Côte d'Ivoire, pg. 23.

• 170
  • "In Tanzania matters came to a head in places around Morogoro in 2008. Distribution in schools of tablets for schistosomiasis and soil-transmitted helminths provoked riots, which had to be contained by armed police. It became a significant national incident, and one of the consequences has been the delay in Tanzania adopting a fully integrated NTD programme, and the scaling back some existing drug distributions." Allen and Parker 2011, pg. 109.
  • "From these reports a number of problems with the MDA were raised which included fear of side effects from the tablets, particularly following the mass hysteria and death in Blantyre and Rumphi respectively and may explain some of the geographic heterogeneity seen. Furthermore most districts reported that MDA occurred after standard 8 students had finished exams and left school, and due to having inadequate resources for drug distribution...The side-effects incident in Blantyre and death in Rumphi had a large effect on districts and with many district reports stating that after the incidence many families refused to participate." SCI Malawi coverage survey 2012 Pgs 5, 21.

• 171

  Fiona Fleming, conversation with GiveWell, November 5, 2015.

• 172

  Alan Fenwick, SCI Director, email to GiveWell, November 24, 2014.

• 173

  SCI planned SCH treatment numbers by country by year (October 2015)

  • $0.30 comes from the "0.25 cents per treatment" plus an additional 20% for "Central cost required (normally 20%)" since these assumptions are applied to most programs from April 2016 through December 2019.
  • One exception is for Nigeria, where SCI estimates $0.40 per treatment and an annual central cost of $300,000 (roughly $0.50 per treatment from April 2016 - December 2019).
  • For comparison, our estimate of SCI's cost per treatment, excluding drugs and in-kind government contributions, based on SCI's recent programs, is $0.53 per treatment (a portion of our total estimate of $1.26, with the remainder accounting for drug costs and government contributions).

• 174
  • Fiona Fleming, SCI Senior Monitoring & Evaluation Manager, conversation with GiveWell, September 21, 2015.
  • SCI's first Value for Money Officer left in May 2015, after only 6 months in the role. It hired a new Value for Money Officer, who was expected to join SCI in October 2015. Fiona Fleming, SCI Senior Monitoring & Evaluation Manager, conversation with GiveWell, September 21, 2015.

• 175
  • "Happy to share cost per treatment estimates with you for several of our countries – Malawi £0.16, and Cote d’Ivoire £0.15. This includes all in-country related costs e.g. in-country expenditure and central level procurement." Fiona Fleming, email to GiveWell, March 3, 2016
  • Converted to USD, the cost per treatment estimate for Malawi is $0.23 and the estimate for Côte d'Ivoire is $0.22.
  • As of May 24, 2016, Google states that £1 is worth $1.46

• 176

  This understanding is from undocumented conversations with SCI from early in our investigation of SCI.

• 177
  • "Ordering of ALB is carried out by the Ministry Of Health Programme Manager for Lymphatic Filariasis (LF) Elimination. Previously the ALB had been used for the LF program and stocks had not been replenished in time for the SCH campaign which contributed to the low ALB coverage. Treatments were also carried out in conjunction with the child health days which may have caused confusion with who was eligible for treatment and prioritising the younger children. Furthermore due to the complexities of distributing PZQ compared to ALB there may have been more focus during the training and distribution on reporting and dispensing PZQ." SCI Malawi coverage survey 2012, Pgs 20-21.
  • "Miscommunication between the national Program Managers for lymphatic filariasis and schistosomiasis led many districts to believe that the ALB which they received should have been made available for the MDA but was not used." SCI Malawi coverage survey 2014, pg. 37.
  • "Unlike in Sub-Saharan Africa, where SCI distributes praziquantel for schistosomiasis and albendazole for STH in equal amounts, Sudan has many areas where schistosomiasis is a problem but STH are not, and SCI distributes more praziquantel than albendazole. However, a small number of areas including Kassala and eastern Sennar have STH and no schistosomiasis. In these areas, the Ministry distributes more albendazole than praziquantel." GiveWell's non-verbatim summary of a conversation with Alan Fenwick and Najwa Al Abdallah, September 14, 2015.
  • "Albendazole (ALB) was also distributed in some of the 30 districts under different partners, however ICOSA programme did not support its procurement, distribution or data collection in 2013/2014." SCI Uganda coverage survey 2014, pg. 4.

• 178

  We explain why we take this approach in this blog post.

• 179

  Alan Fenwick, SCI Director, email to GiveWell, November 24, 2014

• 180

  It is not clear to us how to interpret the 87.4 million “treatments for all NTDs” in this data (e.g. how many of these were deworming and if SCI paid for the treatments), so to be conservative we have not included them at all. A more optimistic way to interpret these data could be to include 24.2 million schistosomiasis treatments in Burkina Faso, Niger, and Uganda that were delivered as part of integrated treatment programs (SCI Gates Foundation final report (January 2011), pg. 9, Table 2; these countries account for 70 million of the 87.4 million “treatments for all NTDs”). Including these 24 million additional treatments lowers the estimate to $1.17 per treatment.

• 181

  See this spreadsheet for details. Note that this is not an ideal comparison because reported coverage rates are calculated from reported treatment numbers and an assumed target population. We would like to adjust the reported treatment numbers, rather than the reported coverage rate, but don't have the data to do so. It is possible that the difference in the reported coverage rate and the coverage rate from the coverage surveys is due to errors in the numbers used for the target population rather than the reported treatment numbers.

• 182

  We used Leslie et al. 2011, a study of the costs of a SCI-funded deworming program in four districts of Niger in 2004-2006, to estimate non-SCI contributions to SCI's deworming programs. Three of the authors of the study were affiliated with SCI. The study aimed to account for all costs of the program, including costs funded by the government and non-financial costs such as the value of volunteers' time:

  “This was a retrospective study which covered a two year period from April 2004 to May 2006, including the first and second years of MDA and related programme activities in four health districts. All data on first year costs at national, regional, district, and sub district levels were taken from the PNLBG accounts and receipts and records of staff missions or activities. Second year cost data for national and regional level activities were taken from receipts. District and sub district, school and community MDA resource use data for 2005 were collected in June 2006 through a retrospective survey…
  The main cost elements include: the programme specific expenditure; the opportunity cost or value of government contributions related to in-kind costs of using local government staff and vehicles and the value of CDD’s time (taken as the daily agricultural labour rate); and the international costs of programme co-ordination, reporting and technical support." Leslie et al. 2011, pgs. 2-3.

  The study is of a single country, looked at a program that was carried out years ago, and the program may differ in some ways from current programs, but overall it is of high quality and provides us with a sense for the portion of resources contributed by SCI versus non-SCI parties.

  Two examples of how the area where the study was conducted may not be representative of all areas in which SCI works:

  • Due to low school enrollment rates, a substantial portion of the program was through community distribution. Current SCI programs focus on school-based distribution. "The primary school net enrolment rate (NER) in 2004 in Niger was 41%... To achieve high treatment coverage in targeted school age children and at risk adults two treatment strategies, school-based and community-based distribution, were established." Leslie et al. 2011, pg. 2.
  • "The cost per treatment and prevalence figures relate to the study sample of four districts located in the Niger River Valley. This was and is an area of high disease prevalence and high population density relative to other parts of the country. The costs per person treated may be higher in lower density and more remote areas." Leslie et al. 2011, pg. 8.

  Non-SCI costs were 18% of the total cost of the program and 33% of the cost of school-based deworming (the program also included community-based deworming).

  • ”Programme cost: 75%
  • Government cost: 18%
  • International tech. support: 7%”

  Leslie et al. 2011, pg. 5, Table 2.

  It is our understanding from the paper and our past conversations with SCI that "programme expenditure" was fully funded by SCI. We believe that "international tech. support" refers to SCI staff time and travel costs; we're somewhat less confident in this than in our understanding of “programme expenditure.” Government costs are "related to in-kind costs of using local government staff and vehicles and the value of CDD’s time (taken as the daily agricultural labour rate)." Leslie et al. 2011, pg. 3.

  Calculating non-SCI costs of school-based delivery:

  • The average cost/treatment in the study was $0.58: “The total economic cost per treatment was $0.58. This includes programme, government and international costs.” Leslie et al. 2011, Author Summary. At 7% of the total cost, international tech. support accounts for $0.04/treatment.
  • “The full economic delivery cost of school based treatment in 2005/06 was $0.76, and community treatment was $0.46. If only programme costs are included these figures are $0.47 and $0.41 respectively.” Leslie et al. 2011, Pgs 7-8.
  • Therefore, non-program costs (government and international tech. support) are $0.29 ($0.76 - $0.47) of the $0.76 cost of each school based treatment. Since $0.04 is international tech. support, that leaves $0.25 of government costs, or 33% of the total cost.

  It is our understanding that in recent programs SCI has continued to do some community-based deworming but that most of its treatments are delivered through schools. Therefore, we conservatively estimate that non-SCI actors contribute 30% of the cost of a SCI deworming program.

• 183

  "Imperial College pays the majority of SCI’s overhead costs, including rent and utilities, and offers free services such as legal assistance. Imperial’s legal department prepares all of SCI’s contracts. The college also covers some risks and liabilities that SCI may face. In return, SCI pays Imperial 6% of its funding from DFID, but this does not cover the full cost of the services that SCI receives. The 6% of the DFID funding that is paid to the college is not included in SCI’s budget, but is included in the total size of the DFID grant. SCI may perform an analysis of the costs covered by Imperial College, but this is not currently a high priority." GiveWell's non-verbatim summary of a conversation with Wendy Harrison and Najwa Al Abdallah on September 8, 2015, pg. 4.

• 184

  “Program costs can vary. For example, in Sudan, it can be expensive to transport drugs. In Darfur, it might cost about 40-50 pence per treatment, while in Khartoum it could be 10 pence per treatment.” Alan Fenwick, SCI Director, conversation with GiveWell, October 15, 2014.

• 185

  Sources for this data:

  • Bleakley 2007: pg. 84, Table 1, "Hookworm-Infection Rate," "Whole Sample."
  • Croke 2014: Kabatereine et al. 2001, pg. 284, Table 1, Average for eastern districts. Kabatereine et al. 2001 was a disease mapping study conducted in southern Uganda in 1998, two years before the study underlying Croke 2014 began, I.e. these are not results from the individuals enrolled in the study underlying Croke 2014. Note that the districts in the eastern district average in Kabatereine et al. 2001 are not an exact match for the districts included in Croke 2014, though four of the five districts included in Croke 2014 (Busia, Iganga, Mbale, and Tororo) are included in the Eastern district average. Furthermore, the districts for Croke 2014 were selected because Kabatereine et al. 2001 had found high worm prevalence in eastern districts: "Five districts in the eastern region of Uganda were selected (Busia, Iganga, Mbale, Palissa, and Tororo) because a survey had indicated that about 60% of children aged 5-10 years were infected with intestinal nematodes, most commonly hookworm." Alderman et al. 2006, pg. 1.
  • Miguel and Kremer 2004: Miguel and Kremer 2004, pg. 168, Table 2.
  • Summary of SCI data from previous programs: Based on our discussion of the Niger, Burundi, Burkina Faso, and Uganda panel studies in our 2012 review of SCI.
  • Liberia: SCI Liberia panel study baseline report, pg. 4, "Summary table of baseline results."
  • Malawi: SCI Malawi panel study, pg. 11, Table 7.1.
  • Côte d'Ivoire: SCI Côte d'Ivoire panel study baseline report, pg. 11, Table 6.1.1
  • Zambia: SCI Zambia panel study baseline report, pg. 11, Table 7.
  • Uganda: SCI Uganda panel study baseline report pg. 10, Table 6.1.
  • Zanzibar: Knopp et al. 2013: "The baseline Schistosoma haematobium prevalence in school children and adults was 4.3% (range: 0-19.7%) and 2.7% (range: 0–26.5%) in Unguja, and 8.9% (range: 0–31.8%) and 5.5% (range: 0–23.4%) in Pemba, respectively. Heavy infections were detected in 15.1% and 35.6% of the positive school children in Unguja and Pemba, respectively." Abstract. This study also measured disease rates in adults and the treatment program will include adult treatments.

  Bleakley 2007, Croke 2014, and Miguel and Kremer 2004, are discussed in greater detail here.

  The WHO definition of "heavy" infection, which seems to have been used across all of the studies, is given in this table:

  
  SCI Côte d'Ivoire panel study baseline report, pg. 7.

• 186
  • For schistosomiasis, mapping data was given in the form of number of districts with schistosomiasis prevalence ≥50%, ≥10% and <50%, and ≥1% and &lt10%. As an estimate, we assumed that these ranges correspond to average prevalences of the midpoint of the range: 75%, 30%, and 5.5%, respectively.
  • For STH, mapping data was given in the form of number of districts with STH prevalence ≥50%, ≥20% and <50%, and ≥1% and &lt20%. As an estimate, we assumed that these ranges correspond to average prevalences of 75%, 35%, and 10.5%, respectively.
  • We assumed that SCI would provide treatment in all areas, including low risk STH areas. The mapping report notes that WHO does not recommend MDA for STH in low risk areas (pg. 9), but it is our understanding that SCI generally treats for STH wherever it treats for schistosomiasis, which may include some districts at low risk for STH.
  • We have assumed that no child has more than one STH and that prevalence of each STH is the same. This is unlikely to be accurate, but serves as a lower bound.

  SCI Ethiopia mapping of SCH and STH 2014.

• 187

  "In Malawi, urine volumes were not accurately recorded thus it is possible that data is indicating lower overall prevalence in sentinel sites. ICOSA will be undertaking further data analysis to quantify underestimates using mapping data from 2012 and baseline data in appropriate districts." SCI report to DFID (October 2013), pg. 15.

• 188

  "The NTD programme in Zanzibar has recently completed the 3rd round of MDA." SCI report to DFID (October 2013), pg. 17.

• 189

  Najwa Al Abdallah, email to GiveWell, May 18, 2016

• 190
  • As of April 2016, SCI had 10,867,813 GBP in restricted and unrestricted funding available to allocate to its 2016-17 budget year. The total budget for April 2016 to March 2017 is 10,985,319 GBP, leaving a funding gap of 117,506. SCI allocation table 2016-2017, "Summary" sheet, cells C4, C11, and C13.
  • A transfer of roughly $333,000 (236,573 GBP) from GiveWell to SCI in July 2015 was originally misallocated by Imperial College, and did not reach SCI until April 2016. SCI GiveWell income reconciliation 2016, cells O8554:P8554.
  • SCI told us that it may use the funding from the misallocated transfer to cover the funding gap for its 2016-2017 plans. The misallocated funding could cover the funding gap and leave around 119,000 GBP remaining (236,573-117,506=119,067). Dr. Wendy Harrison and Najwa Al Abdallah, conversation with GiveWell, May 4, 2016

• 191

  SCI planned SCH treatment numbers by country by year (October 2015)).

  • We subtract $2 million because shortly after SCI provided the estimate, a funder committed $2 million per year for three years to the Ethiopia deworming program. The grant is not to SCI and SCI notes that the uses of this funding may not fully overlap with its planned activities. We subtract this funding from SCI's room for more funding for the next budget year because we would guess that most of it will be used to fund activities that overlap with the activities SCI was planning to fund, though we are not confident in this interpretation and SCI disagrees with it to some extent. SCI also notes that it still aims to reach its previously stated fundraising goal because this would give it some more flexibility. For example, it is interested in spending $1 million to purchase schistosomiasis drugs for adults in Ethiopia: "This gap when we wrote in September did not account for any [funder name] funding, basically because at that time [funder name] had not finalised their agreement, and we were not certain until the contract was signed how much they would offer. Having said that I must now update you because Mike French tells me that for Ethiopia [funder name] have finally signed an agreement with the government and transferred the first years money ‐ $2 million. That therefore does indeed reduce the gap for all three years although the $2 million may not all be for what we would have spent money on so I hesitate to reduce our contribution by the full $2 million. We are expecting that [funder name] will disburse $2 million each year so that is good news. If we hit our fund raising target therefore we will have some extra flexibility and Mike has come back with some further needs because the programme in Ethiopia is hoping to target adults for treatment with schistosomiasis and of course the Merck praziquantel donation is only for treating children so he would like us to buy some if possible. (10 million tablets for $1 million). However we need to think this through, for all our countries and we have a meeting with DFID and WHO in the coming weeks because the praziquantel needs for adults is becoming a hot issue." Alan Fenwick, email to GiveWell, October 29, 2015.
  • We also subtract an additional $1.5 million, assuming that a similar amount of additional funding from DFID in 2017-18 as in 2016-17:

    "In October 2015, SCI shared estimates for its April 2016-March 2017 budget year with GiveWell on target treatment numbers by country, amounts of funding available from DFID and other large donors, and the amounts of additional funding required to deliver the targeted number of treatments in each country and cover SCI's central expenditures (http://www.givewell.org/files/DWDA%202009/SCI/SCI_planned_SCH_treatment…). GiveWell interpreted the estimates for SCI's projected central expenditures in the document as part of SCI's funding gap for its 2016-2017 budget year.

    Documents SCI sent GiveWell in March 2016 indicate that around $1.5 million more in funding from DFID is available to allocate for SCI's 2016-2017 budget year than GiveWell had previously expected. This is because the document from October 2015 included funding available from DFID that could be allocated to in-country programmatic expenditures, but did not include the funding from DFID that SCI planned to allocate to central expenditures (included in the March 2016 documents)." GiveWell's non-verbatim summary of a conversation with Dr. Wendy Harrison, Najwa Al Abdallah, and Dr. Lynsey Blair, April 6, 2016

• 192

  Comment provided in response to a draft of this page in June 2016.

• 193

  Alan Fenwick, SCI Director, conversation with GiveWell, October 15, 2014.

• 194

  Alan Fenwick, SCI Director, conversation with GiveWell, October 14, 2014.

• 195

  “In extreme circumstances, committed money could be shifted to other opportunities. For example, SCI had planned to do work in Liberia, but the Ebola outbreak has put that work on hold and so the committed funding could be reallocated.” Alan Fenwick, SCI Director, conversation with GiveWell, October 14, 2014

• 196
  • ”SCI first contacted the government [in Côte d'Ivoire] and started to discuss a SCH/STH program in 2010. A civil war delayed progress because there was political turmoil, a weakened health system, and dangerous conditions. In 2012 conditions improved, and the contracts between SCI and the Ministry of Health were set up.” GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014, pg. 1.
  • ”Using drugs that are about to expire is a key factor used to prioritize between programs in strategic budget decisions.” Alan Fenwick, SCI Director, conversation with GiveWell, October 15, 2014.
  • “Factors that could affect SCI’s plans include country capacity (countries not being prepared to use planned funding or asking to expand more quickly than planned), WHO praziquantel donations (countries getting more or less drugs than expected), and new countries asking for assistance. New restricted funding could shift unrestricted funding to another use.” Alan Fenwick, SCI Director, conversation with GiveWell, October 15, 2014.
  • Treatment frequency is determined by mapping results: "World Health Organization (WHO) guidelines suggest different treatment frequencies depending on prevalence:
    • Low (1-10% prevalence): treat twice during primary school (once every 3-4 years)
    • Moderate (10-50% prevalence): treat every other year, and
    • High (prevalence over 50%): treat every year.

    Only 3 districts have high prevalence so the plan for next year is to treat those areas, as well as those that need treatment every other year and were not treated in 2014. Others districts have moderate or low prevalence and would be treated again in future years, following WHO guidelines." GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014, pg. 2.

• 197

  Comment provided in response to a draft of this page in June 2016.

• 198
  • Comparing our conversations from 2013 to what happened in 2014 provides some context. In October 2014, SCI told us that if it had received an additional £1 million (about $1.6 million) at the beginning of 2014 it would have used the money to increase the size of each country's budget by 10-20%, which would have been used to make the programs more comfortable but would not have led to more treatments. Alan Fenwick, SCI Director, conversation with GiveWell, October 15, 2014. (As of November 4, 2014, Google stated that £1 is worth $1.6.) In November 2013, SCI told us that it would likely use additional funding to fund two existing programs and to start two additional programs. Expectations as of November 2013 for how it would spend an additional $4 million:
    1. Côte d'Ivoire: expected to need $400,000 and actually spent about $350,000.
    2. Mozambique: expected to need as much as $3 million and actually spent $1.4 million.
    3. Mauritania: expected to need $1 million and didn’t spend anything.
    4. Ugandan islands: expected to need about $600,000 and didn’t spend anything.
    5. The $1.75 million spent in Côte d'Ivoire and Mozambique represents about 80% of the spending.
    6. Expectations as of 2013 are listed in our November 2013 review of SCI.
    7. Actual spending is through mid-September 2014 from GiveWell summary of SCI finances (October 2014).
  • See also our discussion of budget revisions in 2015 in our August 2015 update on SCI.

• 199

  SCI planned SCH treatment numbers by country by year (October 2015)

• 200

  "In October 2015, SCI shared estimates for its April 2016-March 2017 budget year with GiveWell on target treatment numbers by country, amounts of funding available from DFID and other large donors, and the amounts of additional funding required to deliver the targeted number of treatments in each country and cover SCI's central expenditures (http://www.givewell.org/files/DWDA%202009/SCI/SCI_planned_SCH_treatment…). GiveWell interpreted the estimates for SCI's projected central expenditures in the document as part of SCI's funding gap for its 2016-2017 budget year.

  "Documents SCI sent GiveWell in March 2016 indicate that around $1.5 million more in funding from DFID is available to allocate for SCI's 2016-2017 budget year than GiveWell had previously expected. This is because the document from October 2015 included funding available from DFID that could be allocated to in-country programmatic expenditures, but did not include the funding from DFID that SCI planned to allocate to central expenditures (included in the March 2016 documents)." GiveWell's non-verbatim summary of a conversation with Dr. Wendy Harrison, Najwa Al Abdallah, and Dr. Lynsey Blair, April 6, 2016

• 201

  WHO schistosomiasis treatment gap data.