Overview of notable research affiliated with Innovations for Poverty Action

Published: November 2011

This page discusses the cases in which research affiliated with Innovations for Poverty Action (IPA) has (a) reached the point where IPA considers it to have proven a point of interest about aid or (b) had a substantial influence on the way aid is carried out (whether by governments, funders, or nonprofits).

Table of Contents

How we identified these cases

  • We examined the "Selected programs" area of the "Proven Impact" section of IPA's website, which lists interventions that IPA appears to consider very well supported (enough to promote scaling up and not just further research).1
  • We asked IPA for examples of the influence of its studies; it provided an "Influence on Funders" document, which we examined.2
  • We searched the grant databases of the funders listed in the "Influence on Funders" document, looking for cases where IPA was being funded to carry out a large-scale investigation of an idea pioneered in an existing IPA-affiliated study. (Note: we weren't able to search the grant databases of the multilateral organizations listed, only those of the foundations. We searched the multilateral organizations' websites in full, but didn't find anything of interest.)
  • We checked our results for apparently influential studies that we knew about independently (e.g., from informal conversation) and found that we had accounted for all such studies.
  • We also plan to show this report to Innovations for Poverty Action and ask whether there are any examples that should be added.

Format of this report

For each case, we:

  • Summarize the key program or idea that IPA classifies as proven or influential.
  • Summarize the different studies supporting the key program or idea. In most cases we have not thoroughly examined these studies, and so we do not discuss the details and quality of individual studies. We do list
    • The IPA study: the original study presenting the program or idea.
    • Direct replications: any replications of the original study in another setting, which provide evidence that the program or idea can be generalized and that it isn't entirely contingent on the specific circumstances under which the first study was carried out.
    • Other supporting evidence: any other evidence that supports the program or idea, even if it does so somewhat indirectly.
    • Ongoing further evidence generation: any further evidence relating to the program or idea that is in progress.
    • Our take on the evidence: our overall sense for the extent to which the program or idea has a diverse evidence base, making it suitable to consider it generally "proven" and worth scaling up. Again, we have not examined individual studies; our conclusions are based on the factors above and on cursory readings of the studies.
  • Summarize what we know about the influence of IPA (and affiliated organizations) on governments, nonprofits, or grantmakers with regard to the program or idea.

Programs and ideas that have arguably been influential

Microlending vs. microsavings

In its document on the influence of its work (Innovations for Poverty Action, "Influence on Funders," Pg 2), IPA lists a set of studies on microfinance3 that it was partly or fully responsible for.4 These studies showed mixed results for the traditional model of microfinance, which generally involves bringing very low-income people into groups and making relatively regular, repeated small loans to them. More-encouraging results were found for other interventions, particularly microsavings (the provision of formal savings accounts, rather than loans, to help low-income people manage their financial situations). These studies can be seen as collectively suggesting lower confidence in traditional microfinance, and greater interest in microsavings and further experimentation.

Evidence

IPA studies:

  • Banerjee et al. (2010) examined a randomized rollout of a traditional microfinance program in Hyderabad, India, and found mixed results: there were some changes in behavior for subgroups, but no detected effect overall on average spending or on "measures of health, education, or women's decision-making."5
  • Crépon et al. (2011) examined a randomized rollout of a traditional microfinance program in rural Morocco and found similar results to the study discussed immediately above: some changes in behavior for subgroups, but "little or no effect on average consumption as well as on other outcomes such as health, education, etc."6
  • Dupas and Robinson (2011) examined a program in rural Kenya that randomly gave people financial assistance in opening formal savings accounts.7 They found positive effects on estimated total expenditures8 (implying that those offered savings accounts ended up with more available to spend). They also argued that they had demonstrated "some suggestive evidence that the accounts made market women less vulnerable to illness shocks."9
  • Karlan and Zinman (2007) worked with a lender in South Africa to randomly approve some clients (who would have been rejected by the normal process, but were borderline) for consumer loans (note these were one-time bank loans, distinct from traditional microlending). They found "significant benefits for borrowers across a wide range of economic and wellbeing outcomes."10
  • Karlan and Zinman (2009) examined a similar situation in Manila to the situation they had examined in South Africa (bullet point immediately above) and found mixed results.11

For all of these except the Hyderabad study, IPA was fully responsible for fieldwork. The Hyderabad study was conducted by IPA research affiliates.12

Direct replications: it appears that the Morocco study discussed above is a fairly direct replication of the Hyderabad study discussed above, and that the two studies by Karlan and Zinman are structurally very similar.

Other supporting evidence: none that we know of.

Ongoing further evidence generation: IPA's research initiative on microsavings lists a large number of ongoing studies.13

IPA's take: we haven't seen strong statements on IPA's part regarding how strong this evidence is.

Our take: we feel that the work discussed above is epistemically valuable in raising questions about microfinance and encouraging more reflection and experimentation on the part of funders. We don't think that it has reached any conclusive results about the impact of either traditional microfinance or microsavings, but we don't perceive advocates as suggesting that it has.

Influence

IPA discusses the impact of its research on general dialogue (Google results, media articles)14 as well as on two specific groups:

  • The Consultative Group to Assist the Poor (CGAP), a consortium whose members include 16 government aid agencies (including the U.S.'s) and the World Bank.15 IPA points to the contrast between a 2003 CGAP writeup and a 2010 CGAP writeup on the impact of microfinance;16 The latter writeup cites two studies for which IPA was fully responsible and one more that was conducted by its research affiliates (out of 13 citations total),17 and informally speaking, we believe that these are among the more relevant studies to CGAP's apparent shift.
  • The Bill and Melinda Gates Foundation, which in late 2010 pledged $500 million for increasing access to savings worldwide.18 IPA states that press releases on this topic have cited IPA's research.19 The press release IPA pointed us to is not clear about which research it is citing,20 though it does include an announcement of $7 million in funding for IPA for further research on the topic.21 We note that it is difficult to disentangle the effect of research, as opposed to other factors, on the Foundation's decision: its strategy overview for the program cites high demand and underfunding for savings, as well as new savings technologies (such as savings through mobile phones), and does not specifically mention impact evaluation research.22

Chlorine dispensers

IPA "aims to radically increase the percentage of households that treat their drinking water by providing chlorine treatment at water sources through easy-to-use dispensers."23

Evidence

IPA study: IPA refers24 to Ahuja, Kremer, and Zwane (2010), which in turn references Kremer, Leino, Miguel and Zwane (2009), a study we have not located, as evidence that a program focused on the provision of chlorine dispensers at water sources - in addition to promoters to advocate their use - is a relatively inexpensive and effective way to get people to treat their water with chlorine.25 IPA was fully responsible for this study's fieldwork.26

Direct replications: none that we know of.

Other supporting evidence: IPA cites another study, Kremer et al. (2003), as evidence regarding the health impact of improving water quality. 27 IPA was fully responsible for this study's fieldwork.28 We have discussed this study on our blog and consider it to be encouraging evidence regarding the health impact of improving water quality in general, though it does not address chlorination specifically.

Ongoing further evidence generation: none that we know of.

IPA's take: IPA lists this intervention as "Proven."29

Our take: IPA's case for this intervention is based on only one study, so we have hesitations about whether it should be scaled up without further investigation. The inexpensiveness and intuitive appeal of the program lessen our hesitation to some degree.

Influence

IPA discusses four pilots of this program in Western Kenya, all with participation by the government, and also states that "extensive pilots are underway in Haiti, Bangladesh, India, Swaziland, and Ethiopia."30 We do not have information about the expense of these pilots, the plans for evaluating them, or the likelihood of further scaleup following the pilots.

Remedial education

IPA states that "Remedial education has been proven to be a cost-effective solution that directly addresses the issue of lags in student achievement" and lists several key points that it believes to be demonstrated by its research; in particular, children should be divided by initial achievement levels.31

The evidence

IPA study: Banerjee et al. 2006 examines a remedial education program run by Pratham in two different cities in Western India.32 It concludes that the program "increased average test scores of all children in treatment schools by 0.28 standard deviation, mostly due to large gains experienced by children at the bottom of the test-score distribution."33 IPA was not responsible for this study's fieldwork, though the study was conducted by IPA research affiliates.34

Direct replications: the program was run concurrently in two different cities (see immediately above).

Other supporting evidence: IPA cites35 a paper examining a randomized experiment in 121 primary schools in Kenya and concluding that student performance (as measured by test scores) was improved when students were separated into different classes by ability (rather than randomly).36

Ongoing further evidence generation: an evaluation is ongoing in Ghana; details are under "Influence" below.

IPA's take: IPA lists this intervention as "Proven."37

Our take: IPA's case for this intervention is based on only one study (in two different cities, but in the same general region at the same time with the same NGO), so we have hesitations about whether it should be scaled up without further investigation. However, the ongoing investigation in Ghana (discussed immediately below) may provide a more robust case.

Influence

IPA states that a national scale-up program was undertaken;38 we have not identified a national scale-up program, but have identified the following:

  • A news article from the Hindu Business Line reporting that " The programme has since been replicated across 20 cities."39 We do not have information about the expense of this rollout or the plans for evaluating it.
  • Another study in Ghana conducted in partnership with the Ghana government and the Children's Investment Fund Foundation among others. This study, also randomized, involves 500 schools.40 We do not have information about the expense of this study or the likelihood of further scaleup following it.

School-based deworming

IPA states that "For just pennies per treatment, deworming pills administered through school programs can improve health and increase attendance."41 (See our discussion of deworming for an explanation of the intervention.)

Evidence

IPA study:

Miguel and Kremer 2004 finds that deworming schoolchildren in Kenya caused an increase in their school attendance, and argues that deworming is an unusually cost-effective way to get these benefits.42 It argues that previous studies on deworming, which did not find educational benefits, were flawed.43 A followup study - Baird, Hicks et al. 2011 - concluded that deworming had conferred a variety of additional benefits including better health (self-reported) and substantially higher earnings.44 It appears that this study took place under unusual conditions: a sharp rise in moderate-to-heavy infections took place in the middle of the program due to "the extraordinary flooding in 1998 associated with the El Nino weather system."45

IPA did not exist at the time that work on this study began, but it took over responsibility for the fieldwork partway through the project.46

Direct replications: none that we know of.

Other supporting evidence: there are many high-quality studies regarding the benefits of deworming. We summarize the overall evidence at our writeup on deworming.
Ongoing further evidence generation: none that we know of.

IPA's take: IPA lists this intervention as "Proven."47

Our take: as discussed at our writeup on deworming, we feel the benefits of deworming remain unclear, and we note significant limitations to the studies discussed above. That said, the intervention also has strong intuitive appeal for us, since it is inexpensive (on a per-child basis), and involves direct administration of drugs with proven clinical efficacy.

Influence

IPA points to the Deworm the World Initiative,48 which is affiliated with IPA.49 We have reviewed Deworm the World.

According to Deworm the World's 2009 IRS Form 990 (2009 is the only year for which its Form 990 is available as of this writing), Deworm the World had $229,016 in revenues and $164,538 in expenses in 2009.50

Free bednets

IPA points to the contributions of its work to a question of whether insecticide-treated nets (ITNs) should be given out for free or sold for low prices.51

The evidence

IPA study: Cohen and Dupas 2008 examined the question of whether insecticide-treated nets (ITNs) should be given out for free or sold for low prices. The researchers examined 20 clinics in Western Kenya, and found that when nets were randomly given out for free vs. charged for, the group that received them for free had significantly higher take-up (i.e., whether they purchased/received an ITN), but not significantly different usage (i.e., whether, conditional on having taken an ITN, they were found to be using the ITN in a later check). The researchers concluded that "free distribution of ITNs could save many more lives … at a lesser cost per life saved [than charging for ITNs]."52 We do not have information about IPA's role in this study.

Direct replications: Dupas 2009 examined eight rural markets in a district of Western Kenya and came to a similar conclusion.53

Other supporting evidence: the discussion of this issue at our page on ITNs examines several other sources of evidence and concludes that the weight of the evidence supports giving out ITNs for free, consistent with the conclusion of the studies discussed above.

Ongoing further evidence generation: none that we know of.

IPA's take: IPA lists this intervention as "Promising" (not "Proven").54

Our take: the IPA studies looked at a fairly small number of units (20 clinics, randomized into five groups, in one study; 8 market areas in the other), all in Western Kenya, so we are hesitant to generalize these studies' results. When considering the results alongside the other evidence we discuss at our page on ITNs, we believe it is reasonable to provisionally conclude that ITNs should be given out for free, rather than sold, though we wouldn't conclude that this holds in all circumstances.

Influence

The website of the Poverty Action Lab describes a shift toward policies of free ITN distribution on the part of several entities: the U.K.’s Department for International Development (DFID), Save the Children UK, the United Nations Millennium Project, Commission for Africa, the World Bank, the World Health Organization (WHO), and Population Services International.55 We have not vetted these claims, aside from noting a shift toward free distribution in our 2009 review of PSI.

Programs and ideas that IPA lists as "proven" (excluding those from the previous section)

This section discusses only evidence, not influence; we are not aware of substantial influence that the programs and ideas in this section have had to date.

Investment vouchers

IPA study: Duflo, Kremer and Robinson 2009 examined a program in Western Kenya aiming to induce farmers to purchase more fertilizer by offering a small, time-limited discount just after harvest, and concluded that the program was successful in causing more purchase and use of fertilizer - more successful than a similar discount offered with different timing, and nearly as effective as a much larger discount.56 The study cited an earlier work as evidence that the use of fertilizer is likely profitable;57 we note, however, that this earlier work examined four different doses of fertilizer, finding only one to be profitable (in a statistically significant sense) and finding the Ministry of Agriculture's recommended package to have significant negative returns.58

We do not have information about IPA's roles in these studies.

Direct replications: none that we know of.

Other supporting evidence: IPA states, "Research of a conditional cash transfer (CCT) program in Colombia has proven the effectiveness of similar programs in sectors such as education."59 This study finds that the timing of a cash payment (made conditional on student attendance) can be significant for the likelihood of students' re-enrolling in school.60

Ongoing further evidence generation: IPA states that another study is ongoing, by the same researchers, also in Kenya.61

IPA's take: IPA lists this intervention as "Proven."62

Our take: the case for this program seems to rest mostly on a single study. This study empirically analyzes only fertilizer takeup and use, not profitability; the only evidence cited on profitability is another study with mixed results depending on precisely how fertilizer is used. Overall, we don't feel there is strong evidence that this program represents a strong humanitarian intervention.

Commitment savings

IPA study: IPA promotes63 two products, both piloted in the Philippines, aiming to help people place restrictions on their own savings:

  • Ashraf, Karlan and Yin 2006 found that savings were increased after 12 months for people offered a commitment savings product allowing them to place restrictions on their abilities to withdraw from savings (as opposed to a traditional savings account).64
  • Giné, Karlan, and Zinman 2010 tested a product specifically targeting smoking. This product "offered smokers a savings account in which they deposit funds for six months, after which they take a urine test for nicotine and cotinine. If they pass, their money is returned; otherwise, their money is forfeited to charity." The researchers concluded that this product helped people quit smoking, even well after the 6-month test.65

We do not have information about IPA's roles in these studies.

Direct replications: the "commitment savings" product was also examined in an experiment in Malawi, which concluded that it had - relative to a standard formal savings account - "large positive effects on deposits and withdrawals immediately prior to the next planting season, on agricultural input use in that planting, crop sales from the subsequent harvest, and on household expenditures in the months immediately after harvest."66 We do not have information about IPA's role in this study.

Other supporting evidence: as discussed in a previous section, Dupas and Robinson (2011) examined a program in rural Kenya that randomly gave people financial assistance in opening formal savings accounts.67 They found positive effects on estimated total expenditures68 (implying that those offered savings accounts ended up with more available to spend). They also argued that they had demonstrated "some suggestive evidence that the accounts made market women less vulnerable to illness shocks."69 This savings product had a "commitment" aspect - the presence of fees for withdrawal - though the authors appear to believe that this was a relatively weak commitment device and played a relatively minor role in the impact of the accounts.70

Ongoing further evidence generation: none that we know of.

IPA's take: IPA lists this intervention as "Proven."71

Our take: given the novelty of these products, we don't feel that the evidence (two studies on one product, one on the other) constitutes strong evidence that these products represent strong humanitarian interventions.

Sources

  • 1

    "100% of all donations to the Fund will be directed to proven programs. … All donations received by the Proven Impact Fund go towards programs that have rigorous evidence behind their success. … Each and every program funded by PIF is backed by substantial evidence of its impact." Innovations for Poverty Action, "Proven Impact Fund."

  • 2

    Innovations for Poverty Action, "Influence on funders (2011)."

  • 3

    Some IPA studies that we consider particularly influential to this subject:

    • Banerjee et al. 2010.
    • Karlan and Zinman 2009.
    • Crépon et al. 2011.
    • Dupas and Robinson 2011.
    • Karlan and Zinman 2007.

  • 4

    Nicole Mauriello, email to GiveWell, July 8, 2011 for the specifics of the role IPA played in each of these studies. It was fully responsible for fieldwork in four of the five, and appears to have had limited involvement in the fifth: Banerjee et al. 2010.

  • 5

    "Half of 104 slums in Hyderabad, India were randomly selected for opening of an MFI branch while the remainder were not. We show that the intervention increased total MFI borrowing, and study the effects on the creation and the profitability of small businesses, investment, and consumption. Fifteen to 18 months after lending began in treated areas, there was no effect of access to microcredit on average monthly expenditure per capita, but expenditure on durable goods increased in treated areas and the number of new businesses increased by one third. The effects of microcredit access are heterogeneous: households with an existing business at the time of the program invest more in durable goods, while their nondurable consumption does not change. Households with high propensity to become new business owners increase their durable goods spending and see a decrease in nondurable consumption, consistent with the need to pay a fixed cost to enter entrepreneurship. Households with low propensity to become business owners increase their nondurable spending. We find no impact on measures of health, education, or women's decision-making." Banerjee et al. 2010, abstract.

  • 6

    "This paper reports the results of a randomized experiment designed to measure the impact of microcredit in rural areas of Morocco. Within the catchment areas of new MFI branches opened in areas that had previously no access to microcredit, 81 pairs of matched villages were selected. The treatment villages, randomly selected within each pair, were offered microcredit just after Al Amana opened the branch, while the control villages were offered access only two years later. Al Amana program increased access to credit significantly. Its main effect was to expand the scale of existing self-employment activities of households, for both non-livestock agriculture and livestock activities. We find little or no effect on average consumption as well as on other outcomes such as health, education, etc. However, treatment effects are heterogeneous depending on whether the households had an existing self-employment activity at baseline. Households that had a pre-existing activity decrease their non-durable consumption and consumption overall, as they save and borrow to expand their activities. Households that had not a pre-existing activity increase food and durable expenditure and no effects on business outcomes are observed." Crépon et al. 2011.

  • 7

    "Those sampled for treatment were offered the option to open an account at the village bank at no cost to themselves – we paid the account opening fee and provided each individual with the minimum balance of 100 Ksh (US $1.43), which they were not allowed to withdraw. Individuals still
    had to pay the withdrawal fees, however. Those individuals that were sampled for the control group did not receive any assistance in opening a savings account (though they were
    not barred from opening one on their own)." Dupas and Robinson 2011, Pg 11.

  • 8

    Dupas and Robinson 2011, Table V, upper-left cell.

  • 9

    Dupas and Robinson 2011, Pg 5.

  • 10

    Karlan and Zinman 2007, abstract.

  • 11

    "We generate clean variation in access to microcredit by working with a lender to randomly approve some microenterprise loans within a pool of marginally creditworthy, first-time applicants. We then use an extensive follow-up survey to measure a wide range of impacts on households and their businesses … The impacts of FMB’s credit expansion on more ultimate outcomes are varied, diffuse, and surprising in many respects. Business investment does not increase; rather, we find some evidence that the size and scope of treated businesses shrink. We do find some evidence that profits increase, at least for male borrowers, and the mechanism seems to be that treated businesses shed unproductive employees. One explanation is that increased access to credit reduces the need for favor-trading within family or community networks. This hypothesis is consistent with other treatment effects that are consistent with less short-term diversification and hedging, better access to risk-sharing, and more long-term investment in human capital. The likelihood of other household members working (either in family or outside businesses) falls, as does the likelihood of someone working overseas. The use of formal insurance falls, while trust in one’s neighborhood and access to emergency credit from friends and family increase (i.e., microcredit seems to complement, not crowd-out, informal mechanisms). The likelihood of a household member attending school increases. We find no evidence of improvements in measures of subjective well-being; if anything, the results point to a small overall decrease." Karlan and Zinman 2007, abstract.

  • 12

    Nicole Mauriello, email to GiveWell, July 8, 2011.

  • 13

    Innovations for Poverty Action, "Microsavings Projects."

  • 14

    Innovations for Poverty Action, "Influence on Funders," Pgs 2-3.

  • 15

    CGAP, "Access to Finance for the Poor (2010)," Pgs 57-59.

  • 16

    Littlefield, Morduch, and Hashemi (2003) reviews the available evidence on the impact of microfinance in relation to the MDGs, though few of the studies reviewed include fully rigorous controls for selection biases.

  • 17

    From the references section of Rosenberg 2010:

    "Banerjee, Abhijit, Esther Duflo, Rachel Glennerster, and Cynthia Kinnan. 2009. 'The miracle of microfinance? Evidence from a randomized evaluation.' Cambridge, Mass.: MIT Poverty Action Lab, May.

    Dupas, Pascaline, and Jonathan Robinson. 2009. 'Savings Constraints and Microenterprise Development: Evidence from a Field Experiment.' Working Paper #14693. Cambridge, Mass.: National Bureau of Economic Research, January.

    Karlan, Dean, and Jonathan Zinman. 2009. 'Expanding Microenterprise Credit Access: Using Randomized Supply Decisions to Estimate the Impacts in Manila.' New Haven, Conn.: Innovations for Poverty Action, July."

    We believe these three studies correspond to the first, fourth, and second studies listed under the discussion of microfinance in Nicole Mauriello, email to GiveWell, July 8, 2011, respectively.

  • 18

    Bill and Melinda Gates Foundation, "Melinda Gates Challenges Global Leaders: Create Savings Accounts and Bring Financial Security to the World’s Poorest."

  • 19

    "Press releases by Melinda Gates have cited some of our research as part of the rationale on their focus on and financial commitment to savings. Additionally, BMGF Financial Services for the Poor program lists new research on savings, all of which are IPA studies." Innovations for Poverty Action, "Influence on Funders," Pg 4.

  • 20

    "Research shows that when they are offered side by side with loans, people chose savings over loans at rates of up to 12:1. In one study, Malawian farmers who were given the option to put aside some of their earnings toward the next planting season increased their farming inputs by 64 percent, resulting in 54 percent higher farm revenues, and 30 percent higher day-to-day expenditures. New research also shows that safe savings options can empower women, help people manage risks, like illness and job loss, and increase investment in livelihoods." Bill and Melinda Gates Foundation, "Melinda Gates Challenges Global Leaders: Create Savings Accounts and Bring Financial Security to the World’s Poorest."

  • 21

    "Yale University/Innovations for Poverty Action (2010-2014)
    The Microsavings and Payments Innovation Initiative (MPII): $7 million
    The MPII will strengthen understanding of the financial needs of the poor, and the economic benefits of financial products by launching more than 20 studies to identify the best ways to reach the poor with savings products and money transfer services. To support this effort and build capacity in developing countries, the program looks to expand its research network to include at least 15 more developing country researchers worldwide." Bill and Melinda Gates Foundation, "Melinda Gates Challenges Global Leaders: Create Savings Accounts and Bring Financial Security to the World’s Poorest."

  • 22

    "When we started our work in 2006, we sought to increase financial inclusion through a wide array of projects, including microcredit (offering small loans to poor people), micro- insurance, and savings. But we quickly became aware that savings is the most neglected financial service and the one that is in highest demand by poor people...While we acknowledge the important improvements microcredit programs have made in the lives of the poor, we believe that by increasing access to savings we can have a significant, positive impact on the lives of people living in extreme poverty. Our decision to focus on savings was also driven by the potential of new models for delivering quality, safe, and well-regulated financial services outside the walls of traditional banks." Bill and Melinda Gates Foundation, "Financial Services for the Poor: Strategy Overview (2010)," Pgs 3-4.

  • 23

    Innovations for Poverty Action, "Chlorine Dispensers: An Innovative Solution."

  • 24

    Innovations for Poverty Action, "Chlorine Dispensers: The Evidence." See the link in the sentence, "This study found that dispensers were highly popular with communities …"

  • 25

    "Kremer et al. (2009a) examine free provision of dilute chlorine via a point-of-collection system, which includes a container to dispense the product placed at the water source, a local promoter to encourage its use, and free provision of a supply of chlorine solution packed in bulk. This bulk supply dramatically reduces delivery costs relative to the retail approach that requires packaging chlorine in small bottles and relative to door to door distribution, which in addition significantly raises marketing costs; hence, bulk distribution to water sources makes free provision more realistic. Additionally, this delivery method also makes chlorine use very convenient. Users can treat drinking water when they collect it. The required agitation and wait time for chlorine treated water are at least partially accomplished automatically during the walk home from the source.…

    Take-up of chlorine provided through dispensers dramatically exceeded take-up of chlorine for treating water for in-home use. When communities were randomly assigned to treatment with a promoter and a community dispenser, take up was about 40 percent in the short run (three weeks) but had climbed to over 60 percent by the medium term (three to six months), representing 37 and 53 percentage point gains respectively over the control group. In contrast to the take-up levels achieved with the dispensers, the clinic-based coupon distribution approach proved initially promising, but resulted in much lower coupon redemption over time. Over 40 percent of households who were given coupons redeemed them eight months into the program in that sample, but this fell to 20 percent by 12 months. This suggests that the success of the dispenser is not due only to the zero price, but also to the reduction in the psychic cost of remembering to treat water that is achieved by source-based treatment as well as other attributes, like the visual reminders. Although take-up rates are slightly lower than those achieved in medical trials, the dispenser system relies far less on outside personal contact (e.g. from repeated household visits from enumerators) than do approaches used in medical trials; hence, costs are significantly lower." Ahuja, Kremer, and Zwane 2010, Pg 21-22.

  • 26

    Nicole Mauriello, email to GiveWell, July 8, 2011.

  • 27

    "The first study evaluated the effectiveness of protecting springs in rural areas where piped water is not cost-effective or feasible to implement. This study found that protecting a spring increase the water quality at the source by 66%, but that much of this gain was lost due to unhygienic water collection and storage practices. This finding highlights the importance of protecting against recontamination." Innovations for Poverty Action, "Chlorine dispensers: The evidence."

  • 28

    Nicole Mauriello, email to GiveWell, July 8, 2011.

  • 29

    Innovations for Poverty Action, "Proven Impact Initiative."

  • 30

    Innovations for Poverty Action, "Chlorine Dispensers: Our Programs."

  • 31

    "Remedial education has been proven to be a cost-effective solution that directly addresses the issue of lags in student achievement. Randomized evaluations conducted in both Kenya and India have provided a number of key insights on how programs offering remedial education in schools can successfully address this problem of under-performance.

    • It is essential to focus instruction at the level of the child. This can be done by splitting the class by achievement levels, for some of the class time or the entire class duration. In India, Pratham hired and trained balskhi (friends of the child), women from poor communities who had not gone further than secondary school. These women provided remedial education classes to low performing students in public schools for two hours a day, by pulling them out from the rest of the class. In another program, Read India, Pratham implemented a similar program in rural communities, where volunteers conducted remedial classes outside of school hours. In Kenya, extra teachers were hired for a number of schools, and their effect was only effective when the classes were divided by initial achievements levels.
    • A reliable, easy methodology is needed to identify the level of achievement of all students. Pratham uses a very simple test, ASER, to identify children’s levels. Such a test is easy to use by anyone and can be used to identify the children who need remedial education.
    • It is possible to teach students the basic skills in a relatively short time, using a simple methodology and low-cost materials. In fact, low qualified individuals (such as secondary school graduates or less) can be trained to do so. In India, the balsakhis and the volunteers often have not completed more than secondary education, and they are trained for a short amount of time (4 to 15 days), although they receive regular support after this initial training.
    • Empowering local school committees to hire community teachers and monitor their performance can maximize the benefits for children. In the Kenya study, the program was found to be more effective when the extra teachers were monitored by the School Management Committees. In schools where they were not trained to do so, regular teachers were coming less often."

    Innovations for Poverty Action, "Remedial Education: The Solution."

  • 32

    "The remedial education was run in Mumbai (formerly known as Bombay) and Vadodara (formerly known as Baroda), two of the most important cities in Western India." Banerjee et al. 2006, Pg 2-3.

  • 33

    Banerjee et al. 2006, abstract.

  • 34

    Nicole Mauriello, email to GiveWell, July 8, 2011.

  • 35

    "Dupas, P., E. Duflo and M. Kremer (2009) 'Peer Effects and the Impact of Tracking.'" Innovations for Poverty Action, "Remedial Education: The Solution."

  • 36

    " One hundred and twenty one primary schools which all had a single grade one class received funds to hire an extra teacher to split that class into two sections. In 60 randomly selected schools, students were randomly assigned to sections. In the remaining 61 schools, students were ranked by prior achievement (measured by their first term grades), and the top and bottom halves of the class were assigned to different sections. After 18 months, students in tracking schools scored 0.14 standard deviations higher than students in non-tracking schools, and this effect persisted one year after the program ended. Furthermore, students at all levels of the distribution benefited from tracking. A regression discontinuity analysis shows that in tracking schools scores of students near the median of the pre-test distribution score are independent of whether they were assigned to the top or bottom section. In contrast, in non-tracking schools we find that on average, students benefit from having academically stronger peers. This suggests that tracking was beneficial because it helped teachers focus their teaching to a level appropriate to most students in the class." Duflo, Dupas and Kremer 2008, abstract.

  • 37

    Innovations for Poverty Action, "Proven Impact Initiative."

  • 38

    "Research on both school attendance and learning improvements has led to some very marked policy outcomes. We haven’t gone into this case in as much detail but the same process can be done for both of the aforementioned examples. In both cases, national scale-up programs were undertaken, vastly increasing the number of children benefitting from these effective interventions." Innovations for Poverty Action, "Influence on Funders," Pg 4-5.

  • 39

    Hindu Business Line, "Something isn't working…"

  • 40

    Innovations for Poverty Action, "Remedial Education: Scale Up Activities."

  • 41

    Innovations for Poverty Action, "School-based deworming: The Solution."

  • 42

    "We evaluate a Kenyan project in which school-based mass treatment with deworming drugs was randomly phased into schools, rather than to individuals, allow- ing estimation of overall program effects. The program reduced school absenteeism in treatment schools by one-quarter, and was far cheaper than alternative ways of boost- ing school participation. Deworming substantially improved health and school partic- ipation among untreated children in both treatment schools and neighboring schools, and these externalities are large enough to justify fully subsidizing treatment. Yet we do not find evidence that deworming improved academic test scores." Miguel and Kremer 2004, abstract.

  • 43

    "Our approach can be distinguished from that in several recent studies in which treatment is typically randomized at the individual level and its educa- tional impact is estimated by comparing cognitive ability among those treat- ment and comparison pupils who attend a later testing session. Dickson et al. (2000) review these studies and conclude that they do not provide convincing evidence for educational benefits of deworming. However, these studies fail to account for potential externalities for the comparison group from reduced disease transmission. Moreover, if externalities benefit the comparison group, outcome differences between the treatment and comparison groups will un- derstate the benefits of treatment on the treated." Miguel and Kremer 2004, Pg 160.

  • 44

    "Self-reported health, years enrolled in school, and test scores improve significantly, hours worked increase by 12%, and work days lost to illness fall by a third in the treatment group. Treatment individuals report eating an average of 0.1 additional meals per day. Point estimates suggest substantial externalities among those living within 6 km of treatment schools, although significance levels vary due to large standard errors. Within the subsample working for wages, earnings are 21 to 29% higher for the treatment group. Most of the earnings gains are explained by sectoral shifts, including a doubling of manufacturing employment. Small business performance also improves among the self-employed." Baird et al. 2011, abstract.

  • 45

    "The rise in overall moderate-to-heavy helminth infections between 1998 and 1999 (refer to Table II) is likely to be due to the extraordinary flooding in 1998 associated with the El Niño weather system, which increased exposure to infected fresh water (note the especially large in- creases in moderate-to-heavy schistosomiasis infections), created moist conditions favorable for geohelminth larvae, and led to the overflow of latrines, incidentally also creating amajor outbreak of fecal-borne cholera." Miguel and Kremer 2004, Pg 174 (footnote 19).

  • 46

    Nicole Mauriello, email to GiveWell, July 8, 2011.

  • 47

    Innovations for Poverty Action, "Proven Impact Initiative."

  • 48

    "The Deworm the World Initiative works to expand school-based deworming globally by providing technical assistance, coordinating strategic support, and advocating for large scale and sustainable school-based deworming programs." Innovations for Poverty Action, "School-Based Deworming: Scale Up Activities."

  • 49

    "The Deworm the World (DtW) is a joint initiative of Innovations for Poverty Action and Partnership for Child Development." Deworm the World, "Our Story."

  • 50

    Deworm the World, "IRS Form 990 (2009)." See Part I, lines 9 and 17.

  • 51

    Innovations for Poverty Action, "Free Bednets: Problem and Solution."

  • 52
    • "It is often argued that cost-sharing—charging a subsidized, positive price—for a health product is necessary to avoid wasting resources on those who will not use or do not need the product. We explore this argument through a field experiment in Kenya, in which we randomized the price at which prenatal clinics could sell long lasting anti-malarial insecticide-treated nets (ITNs) to pregnant women. We find no evidence that costsharing reduces wastage on those that will not use the product: women who received free ITNs are not less likely to use them than those who paid subsidized positive prices. We also find no evidence that costsharing induces selection of women who need the net more: those who pay higher prices appear no sicker than the average prenatal client in the area in terms of measured anemia (an important indicator of malaria). Cost-sharing does, however, considerably dampen demand. We find that uptake drops by 75 percent when the price of ITNs increases from zero to $0.75 (i.e. from 100 to 87.5 percent subsidy), the price at which ITNs are currently sold to pregnant women in Kenya. We combine our estimates in a cost-effectiveness analysis of ITN prices on child mortality that incorporates both private and social returns to ITN usage. Overall, our results suggest that free distribution of ITNs could save many more lives than cost-sharing programs have achieved so far, and, given the large positive externality associated with widespread usage of ITNs, it would likely do so at a lesser cost per life saved." Cohen and Dupas 2008, abstract.
    • "In a field experiment in Kenya, we randomized the price at which 20 prenatal clinics could sell long-lasting ITNs to pregnant women. Four clinics served as a control group and four price levels were used among the other 16 clinics, ranging from 0 (free distribution) to 40 Kenyan shillings ($0.60). ITNs were thus heavily subsidized, with the highest price corresponding to a 90 percent subsidy, comparable to the subsidies offered by the major cost-sharing interventions operating in the area and in many other malaria endemic African countries. To check whether women who need the ITN most are willing to pay more for it, we measured hemoglobin levels (a measure of anemia and an important indicator of malaria in pregnancy) at the time of the prenatal visit. To estimate the impact of price variation on usage, we visited a sub-sample of women at home a few months later to check whether they still had the net and whether they were using it." Cohen and Dupas 2008, Pg 3.
    • "The experiment was conducted in 20 communities in Western Kenya, spread across four districts: Busia, Bungoma, Butere and Mumias." Cohen and Dupas 2008, Pg 10.

  • 53
    • "This paper tests the effects on the take-up of a preventative health product of two interventions based on behavioral models derived from psychology: varying the framing of the perceived benefits; and having people verbally commit to purchase the product. I find that none of these interventions had a significant effect (whether economically or statistically) on take-up, and that the gender of the household member targeted was also irrelevant. In contrast, I find that take-up is sensitive to price, as in Cohen and Dupas (2008), and is correlated with indicators of household’s wealth." Dupas 2009, Pg 224.
    • "The experiment was conducted in eight rural markets in a district in Western Kenya with endemic malaria. In each market area, a list of 150 to 200 households was compiled from school registers. Single-headed households were excluded. Remaining households on the list were randomly assigned to: a subsidy level; one of three “marketing” groups; one of two “commitment” groups; and one of three “targeting” groups (described below). After the random assignment (performed in office), trained enumerators visited each sampled household to administer a baseline survey. At the end of the interview, the household was given a voucher for an LL-ITN at the randomly assigned subsidy level. The subsidy level varied from 40 percent to 100 percent; there were 22 corresponding final prices faced by households, ranging from 0 to 300 Ksh (US $4.60). Vouchers could be redeemed within three months at participating local retailers (one per area). 1 The voucher indicated (1) its expiration date, (2) where it could be redeemed, (3) the final (postdiscount) price to be paid to the retailer for the net, and (4) the amount discounted from the recommended retail price." Dupas 2009, Pg 225.
    • "The experimental net was seen hanging in 57.7 percent of the households visited at followup, and the fraction is not significantly higher in any of the experimental groups. 4 Interestingly, usage is not higher among those who paid a higher price—if anything, higher prices seem associated with lower usage rates, suggesting that higher prices did not select those who needed the product more, but rather those for whom the marginal utility of cash is lower, presumably the wealthier. This result is consistent with the result observed among pregnant women in Cohen and Dupas (2008), suggesting that their result holds for the general population and over a larger price range. This result, however, is in sharp contrast with the results in Nava Ashraf, James Berry, and Jesse Shapiro (2008), who find that households that paid more for a water-treatment product are more likely to put it to use within two weeks than those who paid a lower price." Dupas 2009, Pg 228.

  • 54

    Innovations for Poverty Action, "Proven Impact Initiative."

  • 55

    "In the past few years, many organizations have reconsidered their policies to charge for health services, opting instead to distribute ITNs and other health products free of charge. The elimination of user fees is now strongly supported by a number of influential organizations including the U.K.’s Department for International Development (DFID), Save the Children UK, the United Nations Millennium Project and the Commission for Africa.3 In 2009, the British government cited the study by Cohen and Dupas in calling for the abolition of user fees for health products and services in poor countries. Many countries including Burundi, Nepal, Malawi, Zambia, Sierra Leone, Ghana and Liberia are responding to this call, taking major steps towards the provision of free services.

    Population Services International (PSI) is a leader in malaria prevention, providing malaria control support to national Ministries of Health in over 30 countries worldwide. Traditionally PSI has supported the practice of cost-sharing for ITNs as they believed that positive prices increase usage and promote sustainability. PSI has increasingly moved to free distribution of ITNs for pregnant women in Kenya. Today, PSI’s bednet delivery strategies include routine facility-based delivery, mass free distribution for rapid scale-up and continued engagement of the private sector in some locations. For example, in Kenya PSI delivers free ITNs to pregnant women through 3,000 public antenatal clinics, while at the same time subsidizing ITNs sold commercially. The World Bank has also started shifting away from this position, and the WHO recently endorsed free distribution of bednets." Poverty Action Lab, "Free Insecticidal Bednets."

  • 56
    • "Many farmers in Western Kenya fail to take advantage of apparently profitable fertilizer investments, but they do invest in response to small, time-limited discounts on the cost of acquiring fertilizer (free delivery) just after harvest. Later discounts have a smaller impact, and when given a choice of price schedules, many farmers choose schedules that induce advance purchase. Calibration suggests such small, time-limited discounts yield higher welfare than either laissez faire or heavy subsidies by helping present-biased farmers commit to fertilizer use without inducing those with standard preferences to substantially overuse fertilizer." Duflo, Kremer, and Robinson 2009, abstract. "The basic SAFI program worked as follows: a field officer visited farmers immediately after harvest, and offered them an opportunity to buy a voucher for fertilizer, at the regular price, but with free delivery. The farmer had to decide during the visit whether or not to participate in the program, and could buy any amount of fertilizer." Duflo, Kremer, and Robinson 2009, Pg 16.
    • "Both versions of the SAFI program offered free delivery. Our interpretation is that the resulting decrease in the utility cost of using fertilizer is small enough that it would be unlikely to induce large changes in fertilizer use in a purely time-consistent model. However, an alternative explanation is that the free delivery is a substantial cost reduction. To test this hypothesis, and to test prediction three in our model, we offered free delivery later in the season (corresponding to period 2). We also offered a 50 percent subsidy to a separate, randomly selected group of farmers at the same point in the season. As shown in table 1, panel B, free delivery later in the season did not lead to fertilizer purchases from ICS as often as under the SAFI program (20 percent under free delivery vs. 39 percent in the SAFI). The difference between the fraction of farmers who purchase fertilizer under free delivery late in the season and any of the other groups is significant at the 1 percent level, while all the other groups have similar levels of adoption. When offered a 50 percent subsidy late in the season, 46 percent of farmers bought fertilizer." Duflo, Kremer, and Robinson 2009, Pg 25.
    • "Table 2 (columns 3 and 4) presents the impacts of the different programs on fertilizer use, and shows very consistent results: the offer of free delivery late in the season increased fertilizer use by 9 to 10 percentage points (not significant), less than half the increase due to the SAFI program (or SAFI with ex ante timing choice) … Interestingly, a 50 percent subsidy in period 2 significantly increases fertilizer use (by 13 to 14 percentage points), which is very similar to the impact of the free delivery at harvest time (and statistically undistinguishable)." Duflo, Kremer, and Robinson 2009, Pg 25-26.

  • 57

    "In earlier work (Duflo, et al., 2008) we presented evidence that fertilizer is profitable in Western Kenya but many farmers do not use it." Duflo, Kremer, and Robinson 2009, Pg 34.

  • 58

    "A series of demonstration plot experiments in which treatment and control plots were
    randomly allocated within farms suggests that top dressing fertilizer, when used in
    appropriate quantities, is highly profitable, with mean returns of 36% over a season, and
    69.5% annualized. However, other levels of fertilizer use, including the official
    recommendations of the Ministry of Agriculture, are unprofitable for the average farmer in our sample." Duflo, Kremer, and Robinson 2008, Pg 11. See also Table 1 on Pg 6.

  • 59

    Innovations for Poverty Action, "Investment Vouchers: The Evidence."

  • 60

    "We evaluate multiple variants of a commonly used intervention to boost education in developing countries – the conditional cash transfer (CCT) – with a student level randomization that allows us to generate intra-family and peer-network variation. We test three treatments: a basic CCT treatment based on school attendance, a savings treatment that postpones a bulk of the cash transfer due to good attendance to just before children have to reenroll, and a tertiary treatment where some of the transfers are conditional on students’ graduation and tertiary enrollment rather than attendance. On average, the combined incentives increase attendance, pass rates, enrollment, graduation rates, and matriculation to tertiary institutions. Changing the timing of the payments does not change attendance rates relative to the basic treatment but does significantly increase enrollment rates at both the secondary and tertiary levels. Incentives for graduation and matriculation are particularly effective, increasing attendance and enrollment at secondary and tertiary levels more than the basic treatment. We find some evidence that the subsidies can cause a reallocation of responsibilities within the household. Siblings (particularly sisters) of treated students work more and attend school less than students in families that received no treatment. We also find that indirect peer influences are relatively strong in attendance decisions with the average magnitude similar to that of the direct effect." Barrera-Osorio et al. 2008.

  • 61

    "Based on evidence of the impact of these smart subsidies in a variety of environments, the Proven Impact Initiative will continue to work to advocate and support these types of programs.

    For example, the Proven Impact Fund is working to identify organizations marketing and offering ways to increase investment opportunities and technology adoption, particularly among small-scale and subsistence farmers.

    One such study is currently being conducted in Kenya by researchers Esther Duflo, Michael Kremer, and Jonathan Robinson." Innovations for Poverty Action, "Investment Vouchers: The Evidence."

  • 62

    Innovations for Poverty Action, "Proven Impact Initiative."

  • 63

    Innovations for Poverty Action, "Commitment Savings: Challenge and Solution."

  • 64

    "We designed a commitment savings product for a Philippine bank and implemented it using a randomized control methodology. The savings product was intended for individuals who want to commit now to restrict access to their savings, and who were sophisticated enough to engage in such a mechanism. We conducted a baseline survey on 1777 existing or former clients of a bank. One month later, we offered the commitment product to a randomly chosen subset of 710 clients; 202 (28.4 percent) accepted the offer and opened the account. In the baseline survey, we asked hypothetical time discounting questions. Women who exhibited a lower discount rate for future relative to current trade-offs, and hence potentially have a preference for commitment, were indeed significantly more likely to open the commitment savings account. After twelve months, average savings balances increased by 81 percentage points for those clients assigned to the treatment group relative to those assigned to the control group. We conclude that the savings response represents a lasting change in savings, and not merely a short-term response to a new product." Ashraf, Karlan, and Yin 2006, abstract.

  • 65
    • "We designed and tested a voluntary commitment product to help smokers quit smoking. The product (CARES) offered smokers a savings account in which they deposit funds for six months, after which they take a urine test for nicotine and cotinine. If they pass, their money is returned; otherwise, their money is forfeited to charity. Eleven percent of smokers offered CARES tookup, and smokers randomly offered CARES were 3 percentage points more likely to pass the 6- month test than the control group. More importantly, this effect persisted in surprise tests at 12 months, indicating that CARES produced lasting smoking cessation."
    • "We take some initial steps toward addressing the empirical viability and effectiveness of commitment devices for smoking cessation, using evidence from a field experiment in the Philippines."

    Giné, Karlan, and Zinman 2010, abstract.

  • 66

    Brune et al. 2010, abstract.

  • 67

    "Those sampled for treatment were offered the option to open an account at the village bank at no cost to themselves – we paid the account opening fee and provided each individual with the minimum balance of 100 Ksh (US $1.43), which they were not allowed to withdraw. Individuals still
    had to pay the withdrawal fees, however. Those individuals that were sampled for the control group did not receive any assistance in opening a savings account (though they were
    not barred from opening one on their own)." Dupas and Robinson 2011, Pg 11.

  • 68

    Dupas and Robinson 2011, Table V, upper-left cell.

  • 69

    Dupas and Robinson 2011, Pg 5.

  • 70
    • "In terms of the time consistency measures, if anything, we find that respondents who exhibit present-biased preferences were slightly less likely to deposit money than the omitted time-consistent group. This comes in contrast to the findings of Ashraf, Karlan, and Yin (2006), who study a commitment savings product in the Philippines, and Bauer, Chytilová, and Morduch (2010), who observe that present-biased Indian women who lack suitable saving devices tend to borrow from microcredit institutions, as a way to commit themselves to (costly) saving, by way of mandated, structured weekly repayments. This difference might be explained by the fact that the savings account used in our program offered a commitment device to avoid spending money once it had been deposited, but was not accompanied by a commitment to make regular deposits." Dupas and Robinson 2011, Pg 17.
    • "Consistent with this, we find that market women exhibiting time-inconsistent preferences in survey questions were somewhat less likely to use their accounts. This is not surprising, since the bank accounts offered very weak commitment (there were no formal withdrawal restrictions - other than the fee - on the account)." Dupas and Robinson 2011, Pg 31-32.

  • 71

    Innovations for Poverty Action, "Proven Impact Initiative."