GiveDirectly – November 2016 version

We have published a more recent review of this organization. See our most recent report on GiveDirectly.

Published: November 2016

Summary

What do they do? GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households (more).

Does it work? We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people. It appears that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected (more).

What do you get for your dollar? The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82% overall (more).

Is there room for more funding? We believe that GiveDirectly is highly likely to be constrained by funding next year. We expect GiveDirectly to have $19.8 million to spend on its standard cash transfer campaigns in its 2017 budget year. We estimate that if it received an additional $46 million (allowing it to commit $65.8 million) its chances of being constrained by funding would be reduced to 50%. (more).

GiveDirectly is recommended because of its:

• Focus on a program with a low burden of proof and a strong track record (more).
• Strong process for ensuring that cash is well-targeted and consistently reaches its intended targets (more).
• Documented success in transferring a high portion of funds raised directly to recipients (more).
• Standout transparency (more).
• Room for more funding. We believe that GiveDirectly can use substantial additional funding productively (more).

Major open questions include:

• While GiveDirectly has one major RCT of its activities in Kenya, there is still limited evidence on the impact of the type of transfers (large, one-time transfers; and, in the future, unconditional long-term income transfers) that GiveDirectly generally provides, particularly the long-term impact of such transfers. There are currently several ongoing experimental evaluations of GiveDirectly's programs, including a long-term RCT.
• GiveDirectly chooses who should receive cash on a household-by-household basis, as opposed to simply giving cash transfers to everyone in a village. We have doubts about the efficiency of this strategy, given the difficulties of finding criteria that effectively target the poorest households, the large amount of staff time that goes into vetting each household, and the possible offsetting impact of conflict and jealousy. GiveDirectly will soon test giving cash to every recipient in a village in its basic income guarantee program.
• We believe GiveDirectly's basic income guarantee program is likely less cost-effective than GiveDirectly's standard cash transfer campaigns. In 2016, GiveDirectly chose to fundraise extensively for the basic income study rather than for its standard cash transfers, leaving it with a large funding gap for standard cash transfer campaigns in 2017.

Our review process

To date, our review process has consisted of

• Conversations with GiveDirectly staff: Paul Niehaus (Director and President), Piali Mukhopadhyay (COO, International), Ian Bassin (COO, Domestic), Joy Sun (former COO, Domestic), Carolina Toth (Manager, Finance and Operations), and Stuart Skeates (former Uganda Field Director).
• Conversations with GiveDirectly board members: Rohit Wanchoo (Director), Michael Faye (Director), and Jeremy Shapiro (former Director).
• Reviewing documents GiveDirectly sent in response to our queries.
• In November 2012, we visited GiveDirectly's operations in Kenya, where we met with beneficiaries of its work and spoke with its local field staff.
• In 2014, we retained a journalist to visit GiveDirectly in Kenya. We published his report on our blog.
• In October 2014, we visited GiveDirectly's operations in Uganda, where we met with beneficiaries of its work, spoke with local field staff, and observed a cash out day.

All content on GiveDirectly, including updates, blog posts and conversation notes, is available here.

What do they do?

Overview

GiveDirectly transfers cash to poor households in developing countries primarily via mobile phone-linked payment services.1 It has operated since 2009 and is currently active in Kenya, Uganda, and Rwanda (launched in October 2016).2 To date, GiveDirectly has primarily provided large, one-time transfers. It expects to soon start a basic income guarantee program, in which recipients will receive long-term (over two or twelve years in the initial study), ongoing cash transfers sufficient for basic needs (more).

GiveDirectly's work of providing cash transfers to poor households may also include:3

• Experimentation: GiveDirectly runs or participates in studies on a) the impact of cash transfers and b) the costs and benefits of various program designs, with the goal of improving its own cash transfer program, improving other cash transfer programs, or encouraging the creation of new programs.4
• Partnership work: GiveDirectly pursues opportunities to partner with other organizations on cash transfer projects. Through these projects, GiveDirectly aims to encourage the evaluation of aid projects (often by using cash transfers as a standard of comparison) and ultimately influence funders to move resources from less effective aid programs to more effective ones.5

We discuss GiveDirectly's experimentation and partnership work to some extent below, but most of our review focuses on its direct impact, rather than the research or policy impact its programs might have. We focus on direct impact for historical and pragmatic reasons: in the past, GiveDirectly's direct work was the primary use of additional unrestricted donations, and direct impact is more quantifiable and evidence-backed than research or policy impact. More recently, a greater proportion of GiveDirectly's focus has been on research and policy impact; we are not sure if this trend will continue.

Below, we discuss:

• The structure of GiveDirectly's transfers
• GiveDirectly's process for identifying recipient households and delivering cash transfers
• GiveDirectly's staff structure
• GiveDirectly's experimentation work
• GiveDirectly's work on partnerships

Grant structure

GiveDirectly's standard model involves grants of approximately $1,000 (USD) over about four months, after which recipients become ineligible for further grants.6 GiveDirectly has told us that it adjusts its transfer sizes for purchasing power; as of late 2015, in Kenya, GiveDirectly transferred approximately $1,040 to each enrolled household, while in Uganda, it transferred approximately $875.7 We are not sure what the size of transfers in Rwanda will be, but expect it to also be about $1,000 per household.

This is a different approach from the approach we've seen in government cash transfer programs. One way of putting the difference (which has been reflected in GiveDirectly's communications with us) is that government programs aim for "income transfers" (small supplements to income over many years), whereas GiveDirectly's standard program aims for "wealth transfers" (large, one-off transfers that hopefully give people more flexibility to make large purchases and investments). GiveDirectly's basic income guarantee program (more below; expected to launch in late 2016) will be structured as "income transfers."

GiveDirectly's standard transfer schedule involves a small initial transfer (or "token" payment) of about $90 (USD), followed by two larger transfers of about $475 (USD).8 GiveDirectly aims to send these transfers over a period of approximately 4 months.9 GiveDirectly has an ongoing study of behavioral interventions that will allow some recipients the ability to choose when they receive their transfers.

Note that when we reviewed household data from Kenya several years ago, we found that household size varies substantially: while the mean household size was ~4.7 and the median size was 4, 16% of households had 1 or 2 people, ~20% had 6 or more, and the maximum household size was 16.10 We estimate that the average family receives $288 per capita from GiveDirectly, which is 121% of baseline annual consumption per capita for recipients in Kenya.11

GiveDirectly's process

GiveDirectly currently operates in Kenya, Uganda and Rwanda (more details about how GiveDirectly chose those countries in the footnote).12 GiveDirectly is not prioritizing expansion to other countries, as there remain many poor households to serve in the countries in which it operates.13 It does, however, consider opportunities in other countries when compelling reasons are presented, such as the opportunity to impact policy in a way that could only be done in that country or the opportunity to serve a significant number of poor households but only by operating in a donor’s preferred country.14

GiveDirectly's typical process is as follows:

1. Selection of a local region: Once GiveDirectly has selected a country, it narrows down the geographic region in which it would like to work based on a variety of factors, heavily weighting poverty statistics. For example:
   • GiveDirectly told us that it initially chose to work in western Kenya and eastern Uganda based on poverty statistics.15
   • GiveDirectly considers poverty data, population density, logistical and security factors, and the presence of other poverty-focused NGOs when it selects a district or county to work in.16
   • In early 2015, when selecting sub-counties and sub-locations in Kenya, GiveDirectly considered poverty data, the number of potentially eligible households, how easily it could transfer staff capacity to the new locations, and how rural each area was.17

   Note that we have reviewed the data GiveDirectly used in some of the examples above (see footnotes).

2. Selection of villages: GiveDirectly selects villages primarily based on poverty level and location.18 For details on how GiveDirectly has targeted villages historically, see this footnote.19 For recent campaigns in Kenya and Uganda, GiveDirectly has estimated poverty levels through census data.20
3. Obtaining permission from local officials: Before beginning to work in a given area, GiveDirectly obtains permission from local officials. This process can involve officials from the national to the village level and generally requires a series of conversations to get all the relevant stakeholders on board.21 GiveDirectly signs written agreements with or obtains approval letters from local officials to formalize permissions.22
4. Village meeting: A village meeting is held "introduce GiveDirectly and its programming to the village residents, to answer questions anyone may have about the program, and to clarify that [GiveDirectly is not] affiliated with a political party, government agency, etc."23
5. Enrollment process:
   • Census: GiveDirectly has field staff from its census team visit the village to create a census of all households.24 The field staff collect data about each household and note if the household is eligible for transfers (the criteria for eligibility in a campaign depend on where the campaign is located – more).25
   • Registration: GiveDirectly has a separate set of field staff from its registration team visit households marked as eligible in the census and register them.26 Registration involves a) helping recipients set up a payment system to receive transfers (if they don't already have such a system in place), and b) collecting an additional round of data from the household that can be checked against the initial data from the census (more detail in footnote).27 A registered household is formally enrolled only after all phases of enrollment (census, registration, back check, and audit) have been completed and the household has obtained a mobile money account (if necessary).28
   • Back check: GiveDirectly sends a separate team of field staff from its back check team to revisit every registered household and collect data about that household that can be compared to data collected during census and registration.29 GiveDirectly field staff also ask households if they were asked to pay a bribe to register.30 GiveDirectly is currently testing a more streamlined version of its program that does not include the back check step; GiveDirectly hopes to know by the end of 2016 if it can maintain the quality of its program without this step.31
   • Audits: GiveDirectly sends field staff to revisit a portion of the registered households for audits.32 GiveDirectly determines which households to audit based on the extent of the discrepancies between data collected at different phases in enrollment.33 GiveDirectly field staff resolve discrepancies during audits to determine whether households are eligible or ineligible. Households found to be eligible through this process are then considered formally enrolled, in addition to the households considered eligible after back check and not selected for audit.34

   GiveDirectly aims to enroll all eligible households.35 If eligible members of the household are not home during a phase of enrollment, GiveDirectly staff revisit the household several times until they can be found.36

   We have reviewed (and made public) data collected during each step of the enrollment process for most of GiveDirectly's campaigns, with deletions to preserve anonymity.37

6. Sending transfers to recipients: GiveDirectly sends transfers to recipients via mobile money providers (and, in one campaign, via a bank) (more).38 See above for more on the grant structure. GiveDirectly is currently testing a more streamlined version of its program that does not include the token payment; GiveDirectly hopes to know by the end of 2016 if it can maintain the quality of its program without this element.39
7. Conducting follow up calls: GiveDirectly field staff make multiple phone calls and, for vulnerable recipients, in-person visits, to all recipients as transfers are being sent.40 The schedule of follow up calls has varied somewhat by campaign.41 In 2016, GiveDirectly changed its policies such that recipients cannot receive their next transfer installment until they have been reached for follow-up.42 In addition to the follow-up calls, GiveDirectly maintains a phone "hotline" for recipients to call if they have any questions about the transfers or issues in obtaining funds.43 Recipients can also report issues to GiveDirectly field staff when they are in the village; GiveDirectly created a formal mechanism for recording these reports.44

Staff structure

GiveDirectly delivered its first cash transfers in 2011.45 Starting in January 2011 it had one full-time staff member.46 In early 2013 it hired a second full-time staff member to serve as COO (Domestic).47 GiveDirectly has since expanded its staff significantly. As of February 2016, its organizational structure in East Africa included:48

• Chief Operating Officer International (COO-I): The COO-I provides oversight and quality control of cash transfer programming and international operations. The COO-I oversees the Country Directors.
• Country Directors (CDs) and Field Directors (FDs): Both CDs and FDs are primarily in charge of overseeing field operations. The Country Directors oversee operations in a given country; the Field Director position is a slightly more junior role. Combined, GiveDirectly had four CDs and FDs in early 2016.49
• Field Managers and Associate Field Managers: The Field Managers supervise Associate Field Managers, focusing on quality control, management, and training of Field Officers.50 Associate Field Managers manage the logistics of transfer rounds and oversee Field Officers, as well as conduct high-level analysis of field operations and work on technology integration.51 GiveDirectly had 10 Field Managers and Associate Field Managers in early 2016.52
• Field Officers (FOs): FOs implement the steps required on the ground to enroll and follow up with households. They have the most face-to-face interaction with recipients and are all hired within the country of the transfers. There is a separate group of FOs for each of the first three pre-transfer stages: census, registration, and back checks. FOs are also hired to conduct audits and follow-up surveys with recipients post-transfers; some of the FOs hired for these roles may have previously worked on the census, registration, or back check phases.53 GiveDirectly had 71 Field Officers in early 2016.54

It is our impression that GiveDirectly has grown substantially in 2016 and the numbers above may no longer accurately represent its current size.

Segovia

In mid-2014, three members of GiveDirectly's board of directors began the for-profit technology company Segovia, which develops software that NGOs and developing-country governments can use to help implement their cash transfer programs.55 Paul Niehaus and Michael Faye, co-founders of GiveDirectly and Segovia, split their time between the two organizations.56 They previously told us that they track their time allocation to projects and would be able to share details of how much time they each spend on GiveDirectly and Segovia; however, when we requested this information, GiveDirectly told us that it had previously understood that we no longer wanted this information so it had stopped explicitly time tracking. This was a miscommunication, but we decided not to ask GiveDirectly to track this going forward. GiveDirectly estimates that the time Faye and Niehaus collectively spend on both organizations amounts to each organization having the equivalent of a full-time CEO.57 One other staff member who was previously working full-time at GiveDirectly now works part-time for each entity.58 We discuss potential risks from the overlap in staff in this blog post.

When Segovia was created, it expected to provide its services to GiveDirectly for free, in order to avoid conflicts of interest between the two organizations.59 However, in 2016, after realizing that providing free services to GiveDirectly was too costly for Segovia (customizing the product for GiveDirectly required much more Segovia staff time than initially expected), the two organizations negotiated a new contract under which GiveDirectly will compensate Segovia for its services.60 GiveDirectly wrote about this decision here. GiveDirectly told us that it recused all people with ties to both organizations from this decision and evaluated alternatives to Segovia.61 We have seen information on what portion of Segovia's revenues GiveDirectly accounts for, but we do not have permission to share that figure publicly.62

Although we believe that there are possibilities for bias in this decision and in future decisions concerning Segovia, and we have not deeply vetted GiveDirectly's connection with Segovia, overall we think GiveDirectly's choices were reasonable. However, we believe that reasonable people might disagree with this opinion, which is in part based on our personal experience working closely with GiveDirectly's staff for several years.

Evaluation and experimentation

GiveDirectly's goals for experimentation include increasing the evidence base for cash transfers, improving recipient returns and welfare (both in GiveDirectly's program and others), and developing capabilities necessary to implement larger-scale programs or programs in new contexts.63 When choosing which evaluations to run, GiveDirectly also considers the potential impact on policymakers.64 GiveDirectly has told us that it has increased its experimentation to the point where it aims to enroll every recipient in a study or a campaign variation.65 Below, we list the studies and campaign variations that GiveDirectly is currently working on, has completed, or has considered.

Ongoing experimentation

• Macroeconomic effects: Based on conversations with policymakers, GiveDirectly found that a key question relevant to government cash transfer programs is the impact they have on macroeconomic factors such as inflation and job creation.66 GiveDirectly is working to conduct an RCT examining the macroeconomic effects of GiveDirectly's program in Kenya.67 Details of the study are in this footnote.68 Endline data collection was expected to be completed by the end of 2016; as of September 2016, midline data was still being analyzed.69
• Behavioral interventions (Ideas42 study): GiveDirectly is conducting an RCT of two main behavioral interventions: (a) enabling recipients to decide when and how to receive their transfer payments, and (b) providing more information to recipients about spending options.70 Details of the study are in this footnote.71 This study began in late October 2014 and endline results are expected to be available in early 2017.72
• Gender contracts: GiveDirectly ran a small pilot of informal contracts between spouses receiving cash transfers in the spring of 2015.73 External research partners are evaluating the impacts of the contracts on domestic violence and female empowerment.74 After the initial study group was completed, GiveDirectly piloted a second round in early 2016.75 GiveDirectly has said that if the pilot is successful it will be expanded into a larger-scale project.76
• Aspirations study: GiveDirectly is running an RCT in 180 villages looking at the effects of showing recipients a motivational video before their participation in GiveDirectly's program.77 A pilot of the intervention was completed, and baseline data collection was nearly finished as of September 2016.78 GiveDirectly does not expect results from this study for several years.79
• Coffee study: GiveDirectly is implementing an RCT to study the effect of cash transfers on coffee farming communities, and as of September 2016 it was finishing enrollment for the study.80 The study is intended to provide insight into how recipients with high investment return opportunities (i.e., coffee farms) are affected by cash transfers.81 Results are expected in 2018.82
• High throughput campaign: GiveDirectly is currently testing a more streamlined version of its program that removes the back check and token payment steps of its process.83 It hopes to use the streamlined process in 2017 to increase the amount of cash it can transfer per staff member.84 GiveDirectly expects to have finished the majority of its testing by the end of 201685 and intends to implement this model in Rwanda.86

Previous experimentation

• RCT of GiveDirectly's Rarieda campaign: Innovations for Poverty Action (IPA) conducted a randomized controlled trial (RCT) of GiveDirectly's program in which eligible households were selected randomly to receive cash transfers.87 These transfers were made in Rarieda, Kenya in 2011-2012.88 GiveDirectly publicly provided the plan for collecting and analyzing data to determine the impact of these transfers. The RCT has been published; we discuss it in detail here.
• Small-scale RCT of cash transfers to young women: IPA conducted an RCT of GiveDirectly's Nike campaign, which provided transfers exclusively to young women ages 18-19.89 GiveDirectly shared IPA's survey instrument with us prior to the study.90 We did not see an analysis plan prior to the study, as we did with the Rarieda RCT.91 The study is now complete, and GiveDirectly has shared its write-up, as well as a qualitative piece on the perspectives of the young women involved in the study, which was prepared for GiveDirectly by an independent researcher; we have reviewed these documents.92
• Extended data collection by phone: IPA received a $30,200 grant to extend data collection in a sub-sample of participants from the Rarieda RCT using mobile phone-based data collection techniques.93 The goals of the project were to generate data on longer-term effects of cash transfers (up to two years after completion of the RCT), as well as to study the potential for using mobile phones as cost-effective, easily adaptable tools for data gathering.94 GiveDirectly has sent us the results from this study; they include information on the follow-up rates achieved by different types of surveys and on what participants in the study were thinking about before they were called or texted.95
• Broadening eligibility with more inclusive targeting: GiveDirectly conducted a small-scale study in Kenya to see whether more inclusive targeting criteria could reduce tension and conflict within villages. Details of the study are in this footnote.96 GiveDirectly found that data collected on adverse events was inconclusive, and that when faced with the decision of how to allocate limited resources, focus groups preferred to prioritize thatched-roof households.97 We put limited weight on these results due to the small sample size of the study and would be interested in seeing further research on this question.
• Community-based targeting: GiveDirectly piloted community-based targeting, where village residents help determine who should receive cash transfers. GiveDirectly does not expect to implement this targeting method more broadly.98
• Index-based crop insurance program: GiveDirectly and The Rockefeller Foundation developed a strategy for offering index-based insurance to cash transfer recipients (details on index-based insurance in footnote).99 GiveDirectly then ran a small-scale test of the program in western Kenya, simulating a government cash transfer program.100 GiveDirectly found that the cost of the program was lower than the cost of previous index-based insurance programs and a higher rate of people bought insurance.101
• Biometrics: GiveDirectly has tested the use of biometrics to enhance security in Uganda.102 GiveDirectly may continue to use biometrics in contexts where national IDs are uncommon.103
• Eligibility requirements in Homa Bay: GiveDirectly experimented with new eligibility requirements because a) it needed new eligibility requirements for Homa Bay County, where grass is scarce and thus thatch roofs are less common, and b) knowing how to use a number of different eligibility requirements increases GiveDirectly's ability to work in new areas.104 GiveDirectly chose new eligibility requirements for Homa Bay in October 2015 (more).

Future experimentation

Below we describe experimentation that GiveDirectly is planning for or might implement in the future.

Basic income guarantee study

GiveDirectly is planning to begin a study of providing long-term, ongoing cash transfers sufficient for basic needs ("basic income guarantee") in 2017; it launched a pilot of the program in October 2016.105 The study is expected to include approximately 30,000 individuals and provide a basic income for either 2 or 12 years to every adult enrolled (more details in footnote).106 The income will likely be close to $0.75 per day.107 GiveDirectly may solicit input from recipients when determining the timing of the basic income transfers; GiveDirectly suspects most recipients will want to receive larger, more infrequent payments.108

GiveDirectly told us that recently policymakers, academics, and others have shown an increased interest in universal basic income experiments and GiveDirectly believes the project could have significant policy impact.109 We and GiveDirectly believe that the direct impact of the program (excluding any potential policy impact) is likely to be less cost-effective than GiveDirectly's standard campaign (more).110

Other future experimentation

Other ideas that GiveDirectly has considered or is considering for future experimentation include:

• Providing cash transfers in an urban setting111
• Providing cash transfers as humanitarian relief112
• Providing cash transfers to sex workers, in part to examine the impact of cash transfers on HIV outcomes113
• Facilitating the pooling of recipient funds for public goods projects114

Partnership work

GiveDirectly has been exploring projects with a number of partners. The projects that GiveDirectly has partnered on or considered generally involve implementing cash transfers as part of a study funded by an institutional partner. GiveDirectly has also provided informal advice to those considering cash transfer programs. For a sample of smaller potential partnership projects that GiveDirectly has considered, see this footnote.115

In 2015 and 2016, GiveDirectly's President spent approximately 25% of his time on developing partnership projects.116 GiveDirectly has signed agreements or MOUs for the following partnership projects:

• Rwanda benchmarking project: In 2015, GiveDirectly finalized an agreement for a partnership project in Rwanda. GiveDirectly will be implementing cash transfers in two randomized controlled trials; the studies will cost $4 million and are co-funded by an institutional funder and Google.org.117 The studies will test cash transfers as a benchmark against other aid programs funded by the institutional funder.118 GiveDirectly started enrollment for one of the studies in 2016, for which it expects results in late 2017, and as of August 2016 was finalizing the structure of the second study.119 GiveDirectly has provided us with some additional details about the studies, which are not yet public.
• MOU with large institutional funder: In 2016, GiveDirectly signed an agreement with an institutional funder (whom it is partnering with for the Rwanda benchmarking project) which provides a mechanism through which multiple benchmarking projects (projects comparing cash transfers to other types of aid programs) can be launched.120 The funder and GiveDirectly have each offered to contribute up to $15 million (for a total of $30 million) to support four different benchmarking projects with GiveDirectly acting as the implementer for the cash arm.121 We describe what we know about the process of setting up the benchmarking projects in this footnote.122 We do not yet have details of which aid programs will be evaluated or how the evaluations will be carried out. By the end of 2016, GiveDirectly hopes to identify two countries in which it would like to do a benchmarking project.123

Additionally, GiveDirectly has told us that it has made progress in conversations with several other institutional funders about potential projects.124 If all of the partnership projects GiveDirectly is discussing came through (which GiveDirectly believes is unlikely), GiveDirectly would need $23 to $30 million to support all of them.125

Although partnership projects are now taking up a significant portion of Dr. Niehaus' time, GiveDirectly does not believe this has negatively affected its core operations.126 Over the last year, GiveDirectly has hired two additional high-level staff to help with its partnership work: Ian Bassin and Jo Macrae.127 We expect partnerships to continue to take up the President's time and to involve a significant portion of GiveDirectly’s funding over the next few years.128

We have not yet made a strong attempt to assess the value of the partnership projects beyond their direct impact (more). We can imagine cases where partnership projects might be very high leverage (e.g., enabling another organization to "benchmark" its current programming against cash, perhaps ultimately directing funding away from a less effective intervention to cash transfers) and also cases that may have more limited value (e.g., providing cash transfers at a higher cost given the coordination and other costs of partnership projects).

Does it work?

This section discusses the following questions:

• Generally speaking, are unconditional cash transfers a promising approach to helping people? We believe that this approach faces an unusually low burden of proof and that the available evidence is consistent with the idea that unconditional cash transfers help people.
• How effective and well-founded are GiveDirectly's criteria? The evidence we have suggests that GiveDirectly targets low-income recipients. We have reservations about the approach of giving cash transfers to only those who meet GiveDirectly's criteria.
• Is GiveDirectly effectively targeting people who meet its criteria? We believe GiveDirectly's enrollment process is a relatively effective way of targeting people who meet its criteria, although we note that GiveDirectly has experienced difficulties recently with some people in certain geographic areas refusing to enroll in its program.
• Does GiveDirectly have an effective process for getting cash to recipients? GiveDirectly's process seems to have been successful so far, with one notable exception.
• How do recipients spend their cash, and how does this spending impact their lives? We present a variety of evidence, including findings from a randomized controlled trial of GiveDirectly's work.
• Are the size and structure of the cash transfers well-thought-through and appropriate? We find GiveDirectly's approach to be defensible, but we look forward to seeing the results of GiveDirectly's experimentation with different approaches in the future.
• Are there negative or other offsetting impacts? GiveDirectly has taken some measures to address this question, and we believe that the evidence so far suggests that while the cash transfers do raise some problems, these problems are relatively minor.
• Does GiveDirectly have a broader impact on the international aid sector? We have chosen not to look at this question in depth. We have not seen compelling evidence that GiveDirectly has significantly affected the behavior of funders or other organizations, although GiveDirectly has shared some qualitative evidence that we have not followed up on.

Generally speaking, are unconditional cash transfers a promising approach to helping people?

We discuss this question more extensively in our report on cash transfers. In brief:

• The evidence most relevant to GiveDirectly comes from an RCT of a GiveDirectly campaign (available here). We discuss the findings of this RCT in our cash intervention report.
• Cash transfers are among the best-studied development interventions, though questions remain. These studies generally show substantial increases in short-term consumption, especially food, and little evidence of negative impacts (e.g., increases in alcohol or tobacco consumption). It is important to note that most of these studies are of income transfers; there is more limited evidence for programs with wealth transfer models like GiveDirectly's. This is a potential cause for concern and one of the reasons that we are particularly interested in GiveDirectly experimenting with and evaluating different approaches.
• There is also some evidence that recipients are able to invest cash transfers at high rates of return (e.g., ~20% per year), leading to long-term increases in consumption.
• We feel that this intervention faces an unusually low burden of proof, given that short-term poverty reduction is an outcome by definition, though donors' intuitive reactions to it may vary widely.

How effective and well-founded are GiveDirectly's eligibility criteria?

GiveDirectly currently uses two different sets of eligibility criteria for its standard campaigns:

• Assets and vulnerability status: In its campaign in Homa Bay County, Kenya, GiveDirectly uses an algorithm to determine eligibility; this algorithm uses a number of inputs related to household assets and the vulnerability of recipients.129 GiveDirectly developed this algorithm after testing a number of new potential criteria and expects to use similar algorithms for its other campaigns in the near future.130 It is our understanding that GiveDirectly is working to test and develop a similar algorithm for its eligibility criteria in Rwanda.131
• Thatched roofs: Until 2015, GiveDirectly used housing materials to select recipients in all of its standard campaigns, enrolling households who live in a house made of organic materials (thatched roof, mud walls, and a mud floor) and excluding households with iron roofs, cement walls, or cement floors.132 GiveDirectly still uses these criteria in Uganda.133

The assets and vulnerability status criteria

In 2015, GiveDirectly started to work in Homa Bay County in Kenya, where families are less likely to have thatch-roofed houses due to a scarcity of grass.134 Consequently, GiveDirectly changed its eligibility criteria for Homa Bay County to better capture the poorest households.135 The algorithm GiveDirectly uses to determine eligibility in Homa Bay takes into account a range of factors including household assets and the vulnerability status of potential recipients; we are unable to elaborate because GiveDirectly would prefer to keep the criteria confidential so as to prevent households from gaming the system (more detail in footnote).136 More detail on how the algorithm was developed is in this footnote.137 Note that GiveDirectly may adjust its eligibility criteria for other campaigns based on its experience in Homa Bay: for example, it is currently developing eligibility criteria using a similar process in Rwanda.138

GiveDirectly tried to choose criteria that (a) included recipients who would benefit the most from the transfer, (b) were difficult to fake, (c) were low cost to implement, and (d) were perceived as fair both by community members and by GiveDirectly staff.139 GiveDirectly believes that, compared to its previous criteria, the assets and vulnerability status criteria are more difficult to fake, somewhat more expensive to administer, and more difficult to explain (which might lead to people believing the criteria are not fair).140 Note that recipients are not made aware of the full criteria (as a measure to prevent cheating), which may also contribute to decreased perceptions of fairness.141 However, because the criteria explicitly put weight on vulnerability, they could also increase perceptions of fairness, or at least offset other fairness concerns.142

GiveDirectly's assets and vulnerability criteria may help GiveDirectly expand to new areas more easily and could provide valuable guidance for other cash transfer programs (although we are unsure if GiveDirectly will be able to share learnings from this project since it hopes to keep its algorithm confidential). However, our evidence for GiveDirectly's impact and for low rates of conflict within villages is based on previous campaigns in which GiveDirectly used different eligibility criteria, and it is possible—although we think unlikely143 —that these criteria will substantially change these outcomes.

The thatched roof and mud house criteria

As part of the baseline survey for the RCT of its program, researchers collected in-depth information on poverty levels of recipients. GiveDirectly has shared the full survey form used to interview participants, as well as its own summary of the data collected as of March 2012:144

GiveDirectly reports that recipients in Uganda have a slightly higher average daily consumption of $0.83.145

GiveDirectly also provided charts that show a clear difference in the consumption, expenditures, and assets of households in mud and thatch homes compared to those in cement homes, but fairly small differences between those living in mud and thatch homes and those living in mud and iron roof homes.146

End-line data on food consumption among control group recipients from GiveDirectly's RCT also suggests that the thatched-roof eligible households are extremely poor.147 This data shows that "20% [sic] of the control group reports that not all household members usually eat until they are content, 23% of respondents report sleeping hungry in the last week, and only 36% report having enough food in the house for the next day."148 Other results related to food consumption are measured as well, which are, in our view, consistent with the notion that recipients are extremely poor.

Concerns about GiveDirectly's eligibility criteria

How much poorer are those selected by GiveDirectly's criteria?
It is not clear to us that people in thatched-roof homes (eligible for transfers) are substantially and consistently poorer than people in iron-sheet-roofed homes with mud walls and floors (not eligible for transfers in a standard campaign). In community-based targeting pilots, GiveDirectly recipients identified households that did not meet GiveDirectly's standard targeting criteria but seemed comparably poor.149 GiveDirectly has also received feedback from field staff and recipients that using housing materials as the targeting criteria systematically misses some households that are viewed within communities as comparably poor to those in thatched-roof houses.150 GiveDirectly still feels that housing materials are an effective means of targeting the poorest of the poor, on average, in areas where it has worked to date.151

Note that the concern that GiveDirectly's criteria do not select the poorest households could also apply to the assets and vulnerability status criteria. However, from a sample of 423 people, GiveDirectly found that its algorithm selected recipients with an average consumption of $0.50 per day, compared to a community average consumption of $0.86 per day.152 We don't believe these numbers are highly reliable, but they lend some support to the claim that GiveDirectly on average targets poorer households.153

What do housing materials or assets indicate about financial management?
To the extent that there are differences in income or wealth between residents of eligible homes and those who live in non-eligible homes, it seems possible that these differences come down to fortune/luck (e.g., people in iron-sheet homes have been more fortunate and thus able to afford iron sheets), but we also think it may come down to differences in choices regarding financial management (e.g., people in iron-sheet homes may have demonstrated better financial management and planning, thus allowing them to acquire iron sheets). If the latter is the case, there is a potential risk that GiveDirectly is systematically targeting the people who are less likely to use additional money well. GiveDirectly comments: "The most informative data available on this point are the differential impacts we’re seeing within the set of eligible households – specifically, poorer families seeing bigger impacts on nutrition while richer households see bigger impacts on tangible investment."154

Are the benefits of targeting the poorest worth the costs?
We also wonder if attempting to target only the poorest members of a community (with any eligibility criteria) is worth the costs, given that we expect almost everyone in the communities that GiveDirectly works in to be quite poor. In addition to the cost of staff time needed to select eligible households and verify their eligibility, giving cash transfers to some members of a community and not others has the potential for increased conflict. GiveDirectly's follow up surveys demonstrate that cash transfers can lead to tension between recipients and non-recipients.155 Though follow up surveys report low levels of tension and conflict, we would expect these to be underreported by recipients to GiveDirectly staff, a dynamic that GiveDirectly has seen play out in past cases.156 GiveDirectly conducted a small-scale study in Kenya to see whether more inclusive targeting criteria could reduce tension and conflict within villages. We find the results inconclusive (more). However, we note that GiveDirectly will be testing universal enrollment again as part of its basic income guarantee study.157 When we spoke with three field staff in Uganda, two of them suggested that it would be better for GiveDirectly's transfers to reach more people in a village, even if it meant reducing the size of a standard transfer. According to the Assistant Field Manager, the current targeting model causes bragging and unrest in the communities, potentially motivating those who don't benefit to steal from those who do. He said it would be better for GiveDirectly to provide transfers to everyone in a village, even if some transfers were small.158

Anecdotal evidence from GiveWell's site visit to Kenya

In November 2012, GiveWell staff visited Kenya to view GiveDirectly's program in the field. See our notes and photographs from the site visit. We visited five locations (three in Siaya and two in Rarieda) where GiveDirectly had transferred funds or was in the process of enrolling recipients to receive funds. We visited approximately 15 households across the five locations (including two non-recipient households with metal roofs and cement walls and floors that did not qualify for GiveDirectly's program). For details on how homes we visited were selected, see this footnote.159 Note that when we visited, GiveDirectly was using thatched roofs and mud building materials as its criteria.

We would characterize the ~15 households we visited (as well as other households we saw while walking but did not speak with directly) as extremely poor. We summarize characteristics of these households in this footnote.160

Note that among the households we visited, many had already received part or all of their transfer from GiveDirectly, so our observations are based on a selection of households that include some newly-built or renovated structures in addition to older structures. Given that some of the recipients we met used transfers to build larger houses or buy livestock, our observations would likely over-estimate the assets of each household pre-transfer.

In addition, the homes we saw from afar in villages we visited and homes we passed while driving in the area appeared to be at a similar level of extreme poverty.

Is GiveDirectly effectively targeting people who meet its criteria?

GiveDirectly's process for identifying and enrolling households is described above. It involves multiple unannounced visits by different staff to each recipient home in order to confirm that recipients meet the criteria. (That is, if someone were to temporarily occupy a mud and thatch home in order to be enrolled, they would be unlikely to be sure of being present for future re-checks.) We have examined data collected by GiveDirectly from its enrollment process (registration, back checks, remote checks and audits) for most transfer campaigns; we have only spot-checked the data GiveDirectly shared with us in 2015 and 2016.161

Historically, if the information collected about a household at different stages of enrollment is inconsistent, GiveDirectly staff revisit the household for an audit.162 GiveDirectly tracks the percentage of households found to be ineligible at the back check and audit stages on its website; as of October 2016, it reported that 3% of initially registered participants in Kenya were found to be ineligible by the end of GiveDirectly's enrollment process, while that figure was higher at 6% in Uganda.163 We are not sure over what time period these figures are calculated.164

We believe GiveDirectly's current process to be generally effective at identifying households that meet its criteria. However, GiveDirectly has told us that in the future it plans to experiment with streamlining its enrollment process by excluding the back check step from its process.165 It is possible that this change will allow a greater number of recipients to game the system. However, given that we expect almost everyone in the communities that GiveDirectly works in to be quite poor, we do not believe this is cause for much concern.

Refusals in Homa Bay, Kenya

While we believe GiveDirectly currently has robust systems in place to identify recipients who are attempting to game its enrollment process, GiveDirectly has recently struggled to persuade some potential recipients to start the enrollment process, even if they are eligible.166

GiveDirectly started enrolling recipients from Homa Bay county, Kenya in mid-2015.167 There, it encountered unexpectedly high rates of refusals from potential recipients; while refusal rates in Uganda and Siaya, Kenya have historically been low (around 5%), refusal rates in Homa Bay have been about 45%.168 GiveDirectly believes the refusals are due to widespread skepticism towards GiveDirectly's program and rumors that GiveDirectly is associated with the devil.169

Although GiveDirectly has created an outreach team to address the issue, in part to prevent the problem of refusals from spreading, as of September 2016 it was still facing high rates of refusals.170 GiveDirectly told us that while the refusals have reduced GiveDirectly's efficiency somewhat, GiveDirectly was still ahead of its enrollment targets as of late August 2016.171 GiveDirectly plans to run its basic income guarantee experiment in Kenya, but not in Homa Bay county.172 We intend to continue to follow this issue, as we are concerned about what it could mean for GiveDirectly's ability to scale up in the future (more).173

Does GiveDirectly have an effective process for getting cash to recipients?

Mobile money providers and distribution models

GiveDirectly transfers funds to recipients through mobile money providers. In Kenya, the mobile money provider, M-PESA, allows users to receive, send, deposit, and withdraw funds on their mobile phones. When withdrawing funds, recipients must present ID along with their mobile phone number and a user-specified M-PESA PIN number to an M-PESA agent.174 Users enter the amount they want to withdraw on their own phone, and after each transaction, they can see their remaining balance, reducing the ability of agents to defraud clients of funds.175 GiveDirectly has told us that recipients are generally able to withdraw cash from mobile money agents located in or near their villages.176 Recipients must pay a small fee when they withdraw a portion of their transfer (around 1% for large withdrawals, and higher for small withdrawals).177

GiveDirectly works with a mobile money provider called MTN in Uganda.178 MTN has similar security measures to M-PESA: a user must present ID to an agent before making withdrawals, provide their phone or SIM card, and enter their PIN number. Users must pay a fee to withdraw, and confirmation messages are sent after withdrawals.179

In Uganda, the agent network is less robust; however, GiveDirectly has found that recipients are still able to withdraw cash from mobile money agents.180 GiveDirectly tracks the ability of recipients to withdraw cash in its follow-up surveys, in which it asks recipients if they have withdrawn their transfer, if they experienced any issues, and how long it took them to make the trip to withdraw.181

Additionally, the "coffee RCT" that GiveDirectly is running will be conducted in Uganda (more), and GiveDirectly intends to use data from this study as a more rigorous check on how easily recipients can withdraw their money in Uganda.182

Staff fraud

The most significant issue that GiveDirectly has had in making sure that cash gets to recipients is a case of staff fraud in its Uganda pilot campaign. In mid-2014, GiveDirectly experienced a case of large-scale crime, when two of its field staff colluded with mobile money agents to defraud recipients of funds. The staff and mobile money agents were able to steal a total of $20,500 in the form of $20 deductions from 85% of recipients and $100 deductions from 15% of recipients.183 GiveDirectly found out about the fraud through follow-up calls to recipients, which were accelerated after a separate issue had been reported to GiveDirectly's hotline.184 GiveDirectly has taken multiple measures to address the vulnerabilities exposed by this case of fraud (see footnote for details).185 We consider fraud to be an ongoing risk to the success of GiveDirectly's programs, but feel that the risk is mitigated by these measures as well as by GiveDirectly's monitoring. It shifted to a distributed cash out model in Uganda in late 2015, which may be somewhat more secure.186

As GiveDirectly scales, we would weakly expect greater awareness of its program and more attention to be paid to it by people outside of the villages in which it works.187 This could increase the risk of large-scale crime.188 GiveDirectly believes that additional security measures are unlikely to be particularly useful (details in footnote).189 In addition to harming recipients, crime would likely cause delays for GiveDirectly's work.

Other issues

Other possible issues with GiveDirectly's process for sending cash to recipients include:

• In Kenya, M-PESA agents could be overcharging or stealing some of recipients' funds.190 GiveDirectly recognizes that this is a common criticism from recipients who call into GiveDirectly's hotline, but believes it is likely that many recipients with this complaint are not fully aware of how to use their mobile money accounts.191 Results from GiveDirectly's follow-up surveys indicate that this problem is fairly rare.192
• In Uganda, some recipients have experienced delays in accessing their funds due to MTN not activating their accounts immediately.193
• Recipients who are unfamiliar with mobile phones or mobile money accounts may not know how to keep their information secure. Field Officers may provide assistance during back check visits.194 GiveDirectly checks the quality of its Field Officers' interactions with potential recipients by administering "quality audits" that test how well recipients understand GiveDirectly's program and ask how the Field Officer conducted himself or herself.195
• Some of the recipients that GiveDirectly serves are not able to fully understand how to use the mobile money payments system on their own, or do not have the mobility to go to agents or cash out days to withdraw their funds.196 For these recipients, GiveDirectly finds a trustee or helper who aids them with their cash transfers; GiveDirectly tries to ensure that this person is someone the recipient trusts.197

How do recipients spend their cash, and how does this spending impact their lives?

Findings from the RCT

We write extensively about the results from GiveDirectly's RCT in our intervention report on cash transfers. In the RCT, researchers collected data by surveying members of the treatment and control groups about their recent spending. All data that follows comes from participant self-reports. GiveDirectly recipients increased the value of their non-land assets and their monthly consumption.198 Their spending is broken down in more detail below (all dollar amounts are adjusted for US purchasing power).

• Total non-land assets.199 Receipt of large transfers increased households’ non-land assets by an average of $463 (95% CI: $378 to $549).200 Households receiving transfers (small or large) were 23 percentage points (95% CI: 17% to 29%) more likely to have an iron roof than the control households.201 Haushofer and Shapiro 2013 estimated that iron roofs cost about $564 based on a survey of one respondent in each of 20 villages.202 GiveDirectly ran a survey that sampled a respondent from each of 20 villages and found that iron roofs cost $418 on average.203 We do not know what explains this discrepancy.
• Business expenses. Households receiving large transfers spent about $13 per month (95% CI: $1 to $25) more than control households on business expenses, which were primarily made up of non-durable expenses on non-agricultural businesses.204 Recipients of small transfers also spent about $13 more per month (95% CI: $4 to $22).205
• Health expenditures. Recipients of large transfers spent about $3 (95% CI: -$1 to $6) per month more than control households on health expenditures.206 Recipients of small transfers also spent about $3 (95% CI: $1 to $5) more.207 This spending was also included within the estimate of spending on consumption, below.
• Education expenditures. Haushofer and Shapiro 2013 reports that treatment households receiving large transfers spent $1.89 (95% CI: $0.20 to $3.58) more than the control households on education expenditures and treatment households receiving small transfers spent $0.79 (95% CI: -$0.31 to $1.89) more.208 Education expenditures were also included within the estimate of spending on consumption, below.
• Consumption. Treatment households consumed about $51 more per month (95% CI: $32 to $70) than control households.209 About half of this additional consumption was on food.210 This additional consumption also included increased spending on social expenditures and various other expenditures.211
• Alcohol and tobacco. Treatment households did not increase their spending on alcohol or on tobacco.212

The RCT also found increases in food security, revenue, psychological well-being, and female empowerment for recipients of cash transfers.213 There was no significant effect found on health and education outcomes, profits, or cortisol levels.214

Data from follow-up surveys

For several of GiveDirectly's past campaigns, GiveDirectly staff surveyed recipients on how they used their cash transfers during the follow-up calls.215 The surveys were conducted at different points in the transfer cycle of each campaign.216 We summarize the data from the more recent campaigns in Kenya and Uganda below. The spending data from Kenya covers portions of the Kenya 2M, Kenya 1.2M, and Kenya rolling campaigns, and covers dates from February 2014 to September 2015. The spending data from Uganda covers some of the Uganda pilot campaign from October 2013 to April 2014.217 Note that we do not put much weight on this data, as it is all self-reported and we have no control group to compare it to.

Amount of reported funds spent, by category

Note that GiveDirectly has also presented some limited data on spending in a single village on its website.219 This data indicates that the vast majority of recipients (over 75%) in the village used their transfer to buy an iron roof.220 The next three largest categories of spending were on other home improvements, livestock, and furniture.221

Anecdotal evidence from our site visit

In our site visit to Kenya, we asked recipients about the value of items commonly purchased with transfer funds.222 Recipients reported that their thatched-roofs frequently leak when it rains and require replacement every 3-4 months at a cost of 1,000 Kenyan shillings ($11.68 based on the exchange rate as of November 15, 2012223 ) as well as time/labor. One recipient also reported that when it rains, she moves her family and their belongings into other structures to stay dry. Recipients reported buying livestock as an investment/savings device, hoping that they could (a) use the milk from the cow or goat for additional income and (b) sell the cow or goat and any offspring in the future if/when they needed additional funds (for e.g., secondary school fees for their children which are approximately 15,000 Kenyan shillings per year224 [$175.13 based on the exchange rate as of November 15, 2012225 ]).

Will the results be different in other campaigns?

GiveDirectly's RCT was conducted in Rarieda, Kenya. GiveDirectly now primarily works in Homa Bay, Kenya and Uganda, and recently started a standard campaign in Rwanda (in October 2016).226 . We guess that these contexts are similar enough that the impact of cash transfers on recipients will be roughly similar.

GiveDirectly has informed us that most potential recipients in Homa Bay County already have iron roofs.227 Additionally, Rwanda recently banned thatched roofs, so recipients are more likely to already have iron roofs there.228 To date, our estimate of investment returns from GiveDirectly's cash transfers has been based on the return to buying an iron roof (due to this being a particularly common purchase). The fact that iron roofs are already common in Homa Bay raises questions about how recipients will spend transfers and what returns on their investments they will get. GiveDirectly has noted that Homa Bay County is geographically very close to Rarieda and that the poverty rate in Homa Bay County is higher than it was in Rarieda, which could indicate that cash transfers will do more good in Homa Bay.229 We expect to learn more about the impact of cash transfers on recipients in Homa Bay from the results of the Aspirations study (more).

Are the size and structure of the cash transfers well-thought-through and appropriate?

GiveDirectly’s standard model is to grant about $1,000 (USD) to households over approximately four months, after which recipients become ineligible for future transfers.230 GiveDirectly has also experimented with different transfer sizes and structures and plans to continue doing so in the future.231 In the past, GiveDirectly has given the following rationale for the size of its standard transfers:232

We have reservations about the above reasoning:

• Regarding "fair:" Pre-cash-transfer wealth/income differences between eligible and ineligible recipients may exist for a number of reasons; we don't believe it's warranted to assume that a fair world would see the two groups with the same wealth/income due to an equalizing transfer, and more to the point, we don't believe that the ineligible households are likely to see the situation as fair. In addition, we are concerned that by aiming to equalize eligible and ineligible households, GiveDirectly takes on a substantial risk of its calculations being off in a way that leads to eligible households becoming systematically better off than ineligible households, which could distort incentives and lead to conflict.
• Regarding "well-understood:" GiveDirectly notes that its transfers are similar—in dollar terms—to those of government programs, but that they are likely much larger in "percentage of income" terms. We note that the cash transfer programs that have been studied to date seem to be in the range of 9-27% of recipients' annual consumption; by contrast, if GiveDirectly's recipients average $0.65 in daily per capita consumption and receive an average of $288 per person over the course of a year (see above), this implies that people receive an average of 121% of their annual consumption in the year in which they receive the transfer.233 The quote above states that the lower level of initial income is an argument for making the cash transfer larger, but to us, it also means that the risks of distorting incentives, causing conflict, etc. are likely to be greater than those of previously-studied programs, since the transfers are a substantially greater percentage of consumption. This issue is somewhat mitigated by the fact that GiveDirectly's transfers are designed as "wealth transfers" rather than as "income transfers": recipients receive funds over the course of a few months and then become ineligible, whereas the government programs GiveDirectly points to have longer periods of eligibility. GiveDirectly has also told us that its decision to make larger transfers over a shorter period of time is based on recipients' reported preferences.234

Perspectives of recipients and field staff

During our site visit to Uganda in 2014, we spoke with a small number of recipients and field staff about the size of transfers. We asked whether people felt it would be better for GiveDirectly to keep the transfer size the same or reduce the transfer size but provide transfers to more people.235 3 out of 4 recipients told us that the transfer size should stay the same (or be increased).236 One GiveDirectly field officer also held this view, saying that $1,000 is enough to help someone advance, but is not so much that it would distort incentives to work. Two other field officers suggested that it would be better for GiveDirectly's transfers to reach more people in a village, even if it meant reducing the size of a standard transfer.237

Merits of further research

GiveDirectly has considered experimenting with transfer size but does not view this as a high priority, in part because it feels that although further research on this question may improve GiveDirectly's program, it would be unlikely to influence other cash transfer programs.238 GiveDirectly is not concerned that people will run out of good uses of funds from $1,000 transfers.239 The Rarieda RCT included both a $300 transfer treatment group and a $1,000 transfer treatment group, but did not provide strong evidence on what the best transfer size would be, because of small sample sizes.240

Are there negative or offsetting impacts?

Below, we discuss questions about the possible negative effects of cash transfers and GiveDirectly's operations. For more, see our site visit notes from our visit to GiveDirectly's operations Kenya in November 2012, during which we spoke with recipients and non-recipients about potential problems.

Does distribution to some community members and not others result in jealousy, conflict, or related issues?

The RCT that Innovations for Poverty Action conducted of GiveDirectly's transfers in Rarieda found no significant effects of transfers on the rate of crime in treatment villages or on instances of physical, sexual, or emotional violence in treatment households as compared to control households in treatment villages.241

We have found very limited information about jealousy and conflict related to other cash transfer programs, but one study that found small levels of hostility towards recipients of an unconditional wealth transfer in Uganda is discussed in our cash transfer intervention report.

GiveDirectly has two primary mechanisms for tracking and resolving conflicts: its follow-up surveys and its hotline. GiveDirectly's follow-up surveys include questions like the following:242

• Have you heard complaints about GiveDirectly in your community? What complaints are you hearing? Who is upset/complaining? Who are they upset with?
• Has there been any shouting or angry arguments among people in your village about these transfers? If yes, describe.
• Has there been any violence, theft, or other crime in your village related to these transfers? If yes, describe.

Recipients can use GiveDirectly's hotline to report issues at any time. GiveDirectly has informed us that recently its hotline service was not effectively responding to everyone who called in; it is in the process of upgrading its hotline.243

Data from follow-up surveys
GiveDirectly has sent us results from follow-up surveys conducted in multiple transfer campaigns. In 2016, we asked for a sample of recent follow-up survey data. GiveDirectly sent us a database covering 3,329 follow-up calls from late July to mid-August 2016 across its campaigns in Kenya, Uganda, and Rwanda.244 This data indicated that reported issues were low: 7% of recipients reported some regrets about how they spent their transfer, 2% reported hearing complaints, and 1% reported thefts.245

Below, we summarize older survey data from several campaigns in Kenya and the pilot campaign in Uganda for some of the questions included in these surveys.

This table includes follow-up survey data primarily from the Kenya 2M, Kenya 1.2M, Kenya rolling enrollment, and Kenya behavioral optimization campaigns (survey results are from 2014 and 2015) and from the Uganda pilot campaign, the Uganda 2M campaign, and the Uganda model variations campaign (survey results are from 2013, 2014, and 2015). Note that recipients may have been surveyed more than once and would therefore be included more than once in the data presented.246 Percentages reported in this table represent the number of recipients who are marked as having responded "yes" (that they had the issue) out of those for whom a response is recorded in the data.247

Note that GiveDirectly surveys only cash recipients, not non-recipients, and all data is self-reported.

Data from hotline calls
We have reviewed records of calls made to GiveDirectly's hotline from May 2012 – August 2015, which provide anecdotal evidence of tension and conflict caused by the cash transfers, according to recipient reports, including marital disputes, fraud committed by helpers, trustees, or family members, and Village Elders requesting funds from recipients.250 In the most recent complete hotline call data that we have seen (from October 2014; in 2015 we asked for sample data only and in 2016 we did not ask for a sample), the most common type of adverse event recorded is household conflict, followed by theft.251 The number of issues reported was about 6% of the total households in the campaigns (though it is possible that single households account for more than one issue recorded).252

Do the cash transfers have negative effects on non-recipients?

There is suggestive evidence that cash transfer programs may have moderate negative short-term effects on the well-being and economic outcomes (e.g., consumption, assets, and business revenue) of non-recipient households living in the same areas as similar households that receive transfers.253 However, the evidence for these effects primarily comes from studies of a variant of GiveDirectly’s program that may differ from its core program in important ways. GiveDirectly notes that even though it has not identified significant evidence of negative effects on non-recipients, it now generally avoids conducting experiments that randomize at the individual level, to avoid situations in which one eligible household receives transfers while a similarly situated neighbor does not.254

Do the cash transfers lead to more frequent or more serious criminal activity?

The RCT that Innovations for Poverty Action conducted of GiveDirectly's transfers in Rarieda found no significant effects of transfers on the rate of crime in treatment villages.255 It is possible that cash transfers cause more serious crimes (in terms of damages) even if they do not cause more crimes; this seems plausible given that cash transfers create an influx of resources into villages. GiveDirectly notes that crime could become a more serious problem as its program becomes larger and more well-known, but GiveDirectly does not expect to see significantly higher rates of crime in the near future.256

Examples of attempted and/or successful criminal activity relating to GiveDirectly cash transfers include:

• People stealing cash and cell phones from recipient households257
• People contacting recipients and posing as GiveDirectly staff to defraud recipients of funds258
• Mobile money agents defrauding recipients of funds259
• GiveDirectly staff defrauding recipients of funds (we discuss one particularly large case of this above)

To mitigate the risk of small-scale crime, in its communications with recipients GiveDirectly emphasizes ways that recipients can keep their mobile money accounts and phones secure.260 It does not communicate with recipients via text message and tells recipients of this policy in order to protect against mass attempts at fraud, and it follows up with recipients who report crimes to try to resolve the issues.261

Do grants distort incentives and decision-making?

We have not seen information on the question of whether individuals who live in the areas served by GiveDirectly change their behavior in order to increase their chances of receiving transfers – for example, by spending more time at home to increase their chances of being at home when GiveDirectly staff visit, or by choosing to live in poorer quality housing in hopes of receiving transfers.262 The one-off nature of transfers (recipients are not eligible for a second round of transfers) may help to mitigate these effects among past and current recipients, though there is information to suggest that some recipients believe transfers could be given again in the future.263

Another way in which grants may distort decision making is if they are promised and not delivered in time (causing people to make plans that cannot be executed). We do not have data directly addressing this issue, but GiveDirectly provides some statistics on the speed with which transfers are received.264 It is possible that this will become more of an issue in the future, as GiveDirectly recently changed its model such that recipients cannot receive their next transfer until a GiveDirectly staff member has followed-up with them about their previous transfers, and it is possible this will increase delays.265

GiveDirectly told us that in its Kenya campaigns the key factor determining when a recipient receives funds is when he or she registers for M-PESA; recipients are told that they will not receive transfers until they have registered.266 In Kenya, for recipients receiving their first transfer in February 2016 (the last time we checked this), the average time for recipients between the census survey and their first payment was 67 days and 2.5% of recipients had transfers that had been delayed for over 3 months.267 GiveDirectly's records of calls to its Kenya hotline demonstrate that some recipients are delayed in registering for M-PESA or collecting transfers due to issues outside of their control (e.g., a recipient's SIM number was already registered to someone else's M-PESA account; another recipient reported that an agent mistakenly claimed that the recipient's account had expired).268

In Uganda, the agent networks of mobile money providers are not as robust, which means that recipients must travel farther, on average, to reach an agent.269 This may hamper recipients' ability to execute plans for how and when to use funds. In late 2015 (the last time we checked this), 81% of recipients in GiveDirectly's Uganda model variations campaign had received their transfers on time (within 15 weeks of enrollment) and 14% had experienced registration problems.270 In early 2016, GiveDirectly reported that transfers in Uganda were delayed due to elections, but did not state by how much.271

Do grants distort local markets?

It seems possible to us that a large infusion of cash into an area could alter economic opportunities for both recipients and non-recipients. Such effects could be positive (for example, by spurring investment and job creation or by increasing the availability of retail goods) or negative (for example, by leading primarily to local inflation). The limited evidence addressing this issue in the RCT of GiveDirectly's program in Rarieda and the broader literature on cash transfers points to no distortion. There is an ongoing RCT of GiveDirectly's program that is testing for macroeconomic effects.

Do cash transfers lead to large increases in spending on alcohol and tobacco?

The RCT of GiveDirectly's program in Rarieda did not find an increase in spending on alcohol or tobacco. As discussed in our intervention report on cash transfers, RCTs of other programs that report spending on alcohol or tobacco find no impact on spending on these goods.

Does GiveDirectly divert skilled labor away from other areas?

In February 2016, GiveDirectly had 94 total field staff members across Kenya, Uganda, and Rwanda: 4 Country Directors and Field Directors, 2 Data Managers and Operations Managers, 7 Administration and Finance staff, 10 Field Managers and Associate Field Managers, and 71 Field Officers.272

GiveDirectly recruits Field Officers through referrals from peer organizations, postings at universities, and job advertisements. The application process involves an interview with a Field Director and a language competency exam. GiveDirectly reports that it receives approximately six times the number of resumes as openings for Field Officer positions.273 Regarding its field staff in Kenya, GiveDirectly explained that successful candidates generally have a college education and are paid approximately $12 per day, in addition to expenses for travel and lodging while working.274 GiveDirectly reported greater language heterogeneity in the areas in which it works in Uganda, which made it harder to hire qualified field staff who also had the necessary language skills.275

Because GiveDirectly continues to easily hire additional staff and its compensation seems roughly in line with market value, we do not see diversion of skilled labor as a serious concern.

Does GiveDirectly have a broader impact on the international aid sector?

One of the aims of GiveDirectly's partnership and evaluation work is to influence the broader international aid sector to use its funding more cost-effectively.276 We have not yet seen compelling evidence that GiveDirectly is causing significant shifts within the international aid sector, although GiveDirectly has noted that we might find conversations with some of its partners to be qualitatively persuasive.277 GiveDirectly has provided evidence that weakly suggests that the international aid sector is moving towards benchmarking programs against cash.278 However, it is difficult to understand what portion of that shift is attributable to GiveDirectly. Below, we describe the types of examples GiveDirectly has provided in support of its impact on the sector:279

• Anecdotally, GiveDirectly has heard that some large funders are asking themselves "Is this better than cash?" before making grants.280 Additionally, several large funders partnering with GiveDirectly (or in discussions for future partnerships) have told GiveDirectly that they are having internal policy conversations around the idea of benchmarking programs against cash, in large part due to GiveDirectly.281
• GiveDirectly believes there has been an increase in demand from policymakers for evidence that compares programs to cash.282
• GiveDirectly believes there has been an increase in the number of studies that include cash arms (and GiveDirectly was invited to implement the cash arms of several new evaluations).283
• Anecdotally, GiveDirectly has heard that several new cash transfer programs, new evaluations, and increased transparency practices were inspired by GiveDirectly.284 GiveDirectly believes that, by executing an excellent program, it may put competitive pressure on other implementers to also perform effectively.285
• GiveDirectly has provided informal advice to new cash programs and studies.286
• GiveDirectly has participated in several high-level panels and roundtables.287
• GiveDirectly is used as an example in trainings and university courses.

We have created a spreadsheet with the examples of GiveDirectly's potential impact on the international aid sector that we are aware of.

It is easier to evaluate GiveDirectly's role in causing unique projects to happen, as opposed to its impact on the broader sector. We believe that the Rwanda project, which caused large donors to give $4 million to a study that will benchmark an intervention against cash transfers, would not have occurred without GiveDirectly and the media attention that GiveDirectly has attracted.288

We would guess that a large portion of any sector impact attributable to GiveDirectly comes from the fact that GiveDirectly has functioned as a proof of concept for cash transfers. Because GiveDirectly has already shown that implementing cash transfers broadly is feasible, we are unsure whether or not additional growth would have a similar sector impact. It is possible that some activities, such as policy-relevant experimentation or partnership projects, could cause significant sector impact in the future; we have not looked in-depth at the impact of these activities (beyond the direct impact on recipients).289 We remain highly uncertain of our ability to determine how much these activities sway policymakers' or funders' decisions, even if we put substantial time and effort into the question.

GiveDirectly notes that its standard cash transfer campaigns could also contribute to sector impact by attracting additional attention which later leads to partnership projects or changes in funders' behavior.290 While this is plausible, we do not see any clear way to verify the suggested causal connection.

What do you get for your dollar?

What percentage of GiveDirectly's expenses end up in the hands of recipients?

Cash grants make up 82.3% of GiveDirectly's all-time incurred expenses.291 This figure includes fundraising costs that are expected to generate revenue in the future and excludes some of the costs of following up with recent recipients.292 We do not have a detailed breakdown of projected future campaign costs, so we are unsure if the ratio of direct grant to total spending will look similar in the future. We believe it's likely to be slightly lower: for spending since June 2015, transfers have been closer to 80% of GiveDirectly's total spending.293

Below we break down GiveDirectly's total spending through July 2016 by activity; however, note that the table does not include data for the period July 2015 - February 2016, because GiveDirectly's breakdown of the data for that time period did not match the categories used at other times.295 Costs not included in GiveDirectly's total spending were at least some of the research costs of the independently-run studies of GiveDirectly's program (these costs are not funded by GiveDirectly)296 and the reserves that GiveDirectly had set aside to cover staff salaries in the event that GiveDirectly has a funding shortfall.297

For the period that is missing from the table above, we know that GiveDirectly spent $13,576,343 and that $10,832,798 of that went to direct transfers (79.8%).300

Note that GiveDirectly expects its efficiency to be higher in its Rwanda standard cash transfer campaign.301

Does GiveDirectly offer a large amount of humanitarian impact per dollar?

We have not conducted a cost-effectiveness analysis that attempts to quantify the benefits of cash transfers in humanitarian terms. Instead, in comparing cash transfers to the interventions conducted by our other top charities, we have attempted to monetize some of the benefits of the latter, in particular the “developmental effects” of deworming and bednets. (In the case of the comparison with bednets, for instance, this means quantifying the estimated impact of bednets on later-in-life income of children through a comparison with the effects of deworming, and then subjectively comparing the cost per life saved with the value of that amount of money as a cash transfer.)

In practice, these calculations are highly sensitive to assumptions, especially regarding:

• the investment returns to cash transfers;
• how much confidence one places in the developmental impacts of deworming; and
• the subjective assessment of the relative value of averting child mortality and improving incomes.

We guess that in purely programmatic terms, and given our values, bednet distributions and deworming are both more cost-effective than cash transfers. However, we think there are plausible values for these assumptions that would permit any ordering of these three programs.

We encourage readers who find formal cost-effectiveness analysis important to examine the details of our calculations and assumptions, and to try putting in their own values. To the extent that we have intuitive preferences and biases, these could easily be creeping into the assumption- and judgment-call-laden work we’ve done in generating our cost-effectiveness figures, and we’re not entirely confident that the figures themselves are adding substantial information beyond the intuitions we have from examining the details of them.

Our full cost-effectiveness model is available here. See also, our 2012 discussion of the cost-effectiveness of cash transfers and other interventions.

Are there significant differences in cost-effectiveness between GiveDirectly's various types of programs?

Basic income guarantee cost-effectiveness

We have limited information on the cost-effectiveness of GiveDirectly's basic income guarantee program. While we have modeled the basic income guarantee program in our cost-effectiveness analysis, we do not have a high degree of confidence in our results. Our model indicates that GiveDirectly's standard cash transfer campaigns are roughly 1.5 to 2 times more cost-effective than the basic income guarantee program.302 The difference in cost-effectiveness is partially driven by the fact that GiveDirectly is raising funding for the basic income guarantee study upfront and investing it in a low-risk portfolio. On average, because one arm of the study will receive cash transfers for 12 years, this funding won't be spent for multiple years, so the benefits from the funding are discounted.

It is possible that the basic income model will be less cost-effective than GiveDirectly's standard model because long-term, smaller transfers may reduce incentives or ability to invest the funds, or because it is possible that it will be more expensive per dollar transferred for GiveDirectly to deliver funds.303 However, it is also possible that the program will be significantly more cost-effective, perhaps by allowing participants to make longer-term plans or through influencing other funders and governments to implement basic income guarantees. We have not incorporated estimates of this type of impact into our model. GiveDirectly has told us that while it expects the cost-effectiveness of direct transfers through this project to be lower than its standard program, it believes the potential for beneficial policy impact, which is hard to quantify, outweighs any difference.304

Benchmarking partnership projects cost-effectiveness

As we have mentioned above, it is our impression that GiveDirectly has increased its focus on experimentation and partnerships. We expect the structure of benchmarking cash transfer programs could look substantially different from GiveDirectly's standard model in some cases. For example, GiveDirectly has told us that if it is benchmarking cash against a program that distributes food stamps, GiveDirectly might disburse smaller and more frequent payments (which recipients are more likely to spend on food) to make the programs more comparable.305 As with the basic income guarantee program, and for similar reasons, benchmarking projects could be more or less cost-effective than GiveDirectly's standard program.

Is there room for more funding?

We believe that GiveDirectly could effectively use more funding than it expects to receive and is very likely to be constrained by funding next year. We estimate that if it received an additional $46 million (allowing it to commit $65.8 million) its chances of being constrained by funding would be reduced to 50%.

In short, we calculate this from (more detail in the sections below):

• Total opportunities to spend funds productively: We believe that GiveDirectly could productively use between $66 million (50% chance of scaling to this level) and $144 million (5% chance of scaling to this level) for cash transfers in its 2017 budget year. This excludes consideration of what GiveDirectly could productively use on activities other than standard cash transfer campaigns (such as fundraising) because GiveDirectly has told us its other activities are fully funded. This estimate includes the costs of enrollment, transferring funds, and follow-up.
• Cash on hand: GiveDirectly holds approximately $66 million. It expects to spend or allocate all of this before February 2017.
• Expected additional funding: We estimate that GiveDirectly will raise $19.8 million in additional funding for its 2017 standard cash transfer campaigns.

Below, we also discuss:

• Past spending: In recent months, GiveDirectly has enrolled recipients at a rate corresponding to transferring $21 million per year.
• Additional considerations: GiveDirectly has a track record of success in scaling its operations quickly. Recently, it grew its capacity for cash transfers by a factor of almost two in a year. It is not clear whether it will be able to continue this trend. Over the last year, GiveDirectly has experienced a high rate of targeted households refusing to be enrolled in an area GiveDirectly was expanding into.

Details follow.

Available and expected funds

As of July 2016, GiveDirectly had $66 million on hand.306 By the end of GiveDirectly's current budget year (February 2017), GiveDirectly expects to have spent or allocated all of this funding, along with an additional $17.5 million that it expects to raise for its basic income study (collectively, $83.5 million):307

• $30 million will be allocated to the basic income guarantee project and granted out over the next 12 years.
• $14 million is allocated to partnership projects.
• $8.5 million is allocated to GiveDirectly's fundraising activities for the next three years.
• $2 million is set aside for salary reserves.
• $29 million will be committed to households for standard cash transfers by the end of February 2017.

Excluding GiveWell-influenced donors, we predict that GiveDirectly will raise $15.8 million in unrestricted funding through the first half of its 2017 budget year that it could use for its standard cash transfer campaigns in 2017.308 We expect that GiveDirectly will receive an additional $4 million from GiveWell-influenced donors who do not follow our recommendation exactly.309

Funding priorities

In the table below, we've briefly summarized the details of GiveDirectly's funding gaps; further detail follows the table. All figures in this section are inclusive of the costs of enrollment, transferring funds, and follow-up.310

Note that:311

• A standard cash transfer team consists of one "team lead" (a Field Director or Country Director) and a team of Field Managers, Associate Field Managers, and Field Officers. GiveDirectly expects that each team in 2017 will be able to transfer $12 million per year (up from a pace of $7 million per team per year in mid-2016 and $11 million per team per year that GiveDirectly expects to transfer by the end of 2016; more below). Half of a team in the table below represents that team working on standard cash transfer campaigns for half of the year.
• GiveDirectly has estimated its "throughput"—the amount of cash that GiveDirectly can commit to households within a given time frame—will be $33 million in 2016.

GiveDirectly's funding gaps for 2017312

Additional detail:313

• Operating below 2016 throughput: GiveDirectly expects to transfer $33 million in its 2016 budget year (3.1 fully trained teams) and to be on pace, if it raises enough funding, to transfer $48 million in 2017 (4 teams).314 We discussed several scenarios with GiveDirectly about what it would look like if it scaled down its operations below its 2016 level in 2017:
  • If GiveDirectly raises a total of $20 million for its standard cash transfer campaigns in 2017, it will have to downsize its staff for standard cash transfer campaigns by approximately 33%.315
  • At a total of $30 million, GiveDirectly would operate 2.5 full teams (likely one full team in Kenya, one full team in Uganda, and a half team in Rwanda).316 GiveDirectly told us that it would still need to downsize its staff.317 GiveDirectly estimates that it would take half a year to scale back up to its current pace after the losses in staff it would experience at the $30 million level.318
• Funding level at which GiveDirectly has a 50% chance of being constrained by funding: GiveDirectly is currently on pace (with no additional hiring) to have four full teams operating its standard cash transfer model in 2017 (details in footnote).319 However, it believes that it could easily scale to 5.5 teams (details in footnote) and, if it receives enough funding to do so, has a 70-80% chance of scaling to this level successfully.320 We guess that this probability is lower (about 50%), given that (a) GiveDirectly has other major priorities as well in 2017 (e.g., partnership projects and the large basic income study), and (b) that GiveDirectly does not yet have the ability to raise enough funds to maintain an operation of this size in the future, which might make it more hesitant to scale to that size in 2017—GiveDirectly has expressed concerns about the negative attention that might come with reducing its size.321
• Funding level at which GiveDirectly has a 20% and 5% chance of being constrained by funding: We asked GiveDirectly to estimate the point at which it believes it would only have a 5% chance of succeeding at scaling to that level in 2017. While this estimate is highly uncertain, GiveDirectly estimated that scaling to 16 additional full teams, 20 total, would have a low likelihood of success.322 We estimate that the 5% level is at 8 additional teams, 12 total. This estimate is very rough and relies on our intuitions. Assuming a linear relation between team size and chance of success, we estimate that GiveDirectly has a 20% chance of success at a scale of 10 teams.323

GiveWell's prioritization of GiveDirectly's funding gaps

We have tried to rank our top charities' funding gaps based on:

• Capacity relevance: how important the funding is for the charity's development and future success.
• Execution relevance: how likely it is that the charity's activities will be constrained if it does not receive the funding.

We believe that "capacity-relevant" gaps are the most important to fill, and "execution"-related gaps vary in importance. More explanation of this model is in this blog post.

We consider all of the funding gaps for GiveDirectly's current priorities to all be "execution" gaps.324 We assign execution level gaps a level (1, 2 or 3) that corresponds with how likely we believe it is that GiveDirectly would be constrained by funding (rather than other factors, such as an inability to grow staff capacity quickly enough) if it is unable to fill the funding gap. Level 1 is 50% chance of funding being the constraint, level 2 is 20% chance, and level 3 is 5% chance. These judgements are rough and largely based on intuitions formed from following GiveDirectly's scale up over several years (more in the next section).

Past enrollment rate

GiveDirectly's past rate of committing funds to recipients is lower than its projected rate for the remainder of 2016 and 2017. Its enrollment rate from March 2016 - July 2016 (the period of GiveDirectly's current budget year for which we have information) implies a transfer rate of about $21 million per year,325 or, assuming three full teams were in operation (two in Kenya and one in Uganda), about $7 million per team per year.326 Including the costs of delivering transfers, GiveDirectly has been transferring about $7.7 million per team per year.327

Note that GiveDirectly expected to transfer $29 million in the period August 2016 - February 2017; assuming that it has 4 teams during that time period, that would require a pace of $12.4 million per team per year, much faster than its pace in the first half of the year.328 When we asked GiveDirectly about this, it noted that it is on pace with its plan, which had included significantly more transfers in the second half of the year.329 If GiveDirectly manages this pace for the second half of 2016, then it should be able to transfer $12 million in cash transfers per team per year (which includes the costs of delivering transfers) in 2017.

In the past, GiveDirectly has successfully scaled up over time, recently increasing its rate of transfers by about a factor of 1.5 to 2 in a year,330 but it is unclear if it will be able to continue this trend. In the table below, we show how GiveDirectly's rate of commitments has increased recently.

Rate of funds committed331

In the past, with a lag of about four months, distributed transfers have generally kept pace with committed transfers.332

To scale up to any point beyond 4 full teams on standard cash transfer campaigns, GiveDirectly will need to hire additional team leads (Country Directors or Field Directors), and it takes GiveDirectly several months to hire a team lead.333 Historically, GiveDirectly has not expected hiring more junior staff to be a challenge.334

Risks to room for more funding

GiveDirectly believes it can grow extremely quickly. However, there are some risks that might impede its ability to grow as fast as it believes it can. The following are concerns identified by GiveWell or GiveDirectly:

• Refusals: As discussed above, GiveDirectly has experienced a high rate of people refusing to be enrolled in Kenya over the last year. GiveDirectly has told us that this has not slowed down its productivity because (a) the refusals only affect the activities of one team (the census team; though this doesn't take into account increased travel time as other teams have to travel further on average between each house) and (b) GiveDirectly is flexible enough that it can pivot to new areas when refusals are high and come back later if refusal rates seem like they will decrease (perhaps due to outreach efforts).335 However, it is possible that the high rates of refusals could create challenges for GiveDirectly in its relationship with the Kenyan government; GiveDirectly has been working to build relationships with the government to mitigate this possibility.336 High refusal rates could also force GiveDirectly to move to new areas sooner than it expected, which could cause challenges if GiveDirectly struggles to obtain permission from local leaders to work in new areas (see next bullet). Additionally, similar challenges in other locations in the future might also reduce GiveDirectly's ability to scale as quickly as it hopes to.
• Government permissions: In order to expand into new areas, GiveDirectly must obtain permission from government officials at many levels. This process could be held up by an official who refused to grant permission, causing delays and possibly preventing GiveDirectly from expanding into an area indefinitely. GiveDirectly has attempted to mitigate this risk by networking with people with expertise in navigating such government relationships and who could intervene if there were a problem.337 GiveDirectly feels that it now has a good understanding of the process for seeking government approvals and does not see this as a major risk.338 In fall 2016, GiveDirectly was beginning the process of assessing how many households remained for it to enroll in the areas it has historically worked in and considering which areas it should enter next.339 GiveDirectly told us that there were sufficient eligible households to enroll over the next year, even if GiveDirectly worked with 5.5 full teams (we did not ask about whether there were sufficient households for more teams than this).340
• Crime: Incidents of large-scale crime could cause delays and reduce GiveDirectly’s ability to transfer funds to recipients. The risk of crime could increase as GiveDirectly becomes better known in the regions in which it works. We discuss this risk more above. We consider this a low to moderate risk.
• Security: GiveDirectly notes that political violence and terrorism could hamper its ability to work in an area, and while these are risks in Kenya, they have not impacted Western Kenya (where GiveDirectly works) since 2008. GiveDirectly has attempted to mitigate this risk by working in multiple locations, so that it could shift its operations to Uganda or Rwanda if there were an issue.341 We know very little about security risks in Kenya, Uganda and Rwanda, but would guess based on GiveDirectly's assessment that it presents a low risk. As GiveDirectly continues to expand to other countries (e.g., Rwanda), we think this risk will be reduced because GiveDirectly will have more areas to redirect its work if necessary.
• Payment provider: Relying on one payment provider in each country introduces a risk that problems with the payment provider could cause delays. GiveDirectly feels that this risk is low, because if there were problems, it could switch to alternative providers.342 We would guess that this risk is low, as the mobile money providers that GiveDirectly uses are national networks, and GiveDirectly has identified alternatives. However, we note that GiveDirectly once tried working with an alternative provider in Uganda (Centenary Bank) and had difficulties in the partnership.343
• Maintaining staff quality as the organization grows: GiveDirectly noted that it has hired a number of new staff over the last year to scale up and prepare for additional scale-up in the future.344 It is possible that GiveDirectly will face issues if the new staff members learn slowly or turn out to be poor fits for their positions.345 So far, GiveDirectly believes that its hiring processes have been successful and that new staff are taking on responsibility quickly and competently.346
• Supporting operations as the organization grows: As GiveDirectly has grown in size, it has needed to expand its internal operations to support its larger team and activities. For example, GiveDirectly now uses Segovia (which we discuss above) to manage many of its enrollment, transfer, and follow-up activities. In 2015, GiveDirectly intended to hire a fundraising team leader to build out its fundraising operations.347 However, it did not manage to find someone for this position until late 2016, possibly affecting the amount of funding GiveDirectly will have available for standard cash transfer campaigns in 2017.348 We are unsure if GiveDirectly's internal operations and fundraising will be able to grow quickly enough to support its current size and continued expansion.
• Political instability or regulation: GiveDirectly has noted that there is some risk of political instability in the countries it works in. Elections could change how the government works with GiveDirectly or regulates NGOs.349 In the worst case scenario, GiveDirectly might be forced to move its operations out of one of the countries it is operating in.350 However, because GiveDirectly now operates in three different countries, it believes that this would be doable.351

Unrestricted vs. restricted funds

We prefer that GiveDirectly spend funds in the way that it believes will maximize its potential and, accordingly, do not recommend that GiveWell donors restrict their donations in any way. We plan to grant funds to GiveDirectly unrestricted (such that GiveDirectly may use funds for all purposes, including experimenting with its model and process and organizational capacity building).

GiveDirectly as an organization

GiveDirectly is a relatively young organization. It was founded in 2009 when its founders were graduate students in economic development; Paul Niehaus, President and co-founder of GiveDirectly, is also an Assistant Professor of Economics at the University of California, San Diego.352 Professor Niehaus was on sabbatical from his teaching position and working full time on GiveDirectly in 2014-2015.353 He returned to his professorship in fall 2015.354

We believe GiveDirectly to be an exceptionally strong and effective organization:

• Self-evaluation: GiveDirectly has invested heavily in self-evaluation from the start, and furthermore, the study design of its Rarieda RCT was pre-registered for additional accountability and credibility. It continues to demonstrate a strong commitment to rigorous analysis of its work.
• Track record: Although it is relatively young, we feel that GiveDirectly's first few years have gone well; GiveDirectly has successfully accomplished its goal of transferring cash to extremely low-income people at a fairly low expense ratio. We have also seen GiveDirectly refine its process over the years and take thoughtful measures in response to problems that arise, demonstrating a commitment to continuous improvement.
• Communication: GiveDirectly has always communicated extremely clearly and directly with us and given thoughtful answers to our critical questions. Generally, GiveDirectly seems to come to conclusions that we find reasonable on key questions.
• Transparency: GiveDirectly appears to value transparency as much as any organization we’ve encountered. We have not seen it hesitate to share information publicly (unless it had what we consider a good reason).

More on how we think about evaluating the leadership of organizations at our 2012 blog post.

Sources

• 1
  • "We move the money from our US bank to our account with Safaricom's M-Pesa mobile payment system using a foreign exchange broker. We then transfer money from our M-Pesa account to the recipient's M-Pesa account. As a security measure we only transfer funds to a recipient if the name in our records matches the name on the national ID document he or she used to register for M-Pesa. The recipient gets an SMS text message reminding him or her of the transfer and then collects the transfer from a local M-Pesa agent, who is typically a shopkeeper in the recipient's village or in the nearest town. The recipient transfers his or her electronic balance to the agent's phone in return for cash." GiveDirectly, How it works 2013
  • "In Uganda, we use MTN's mobile payment system to send recipients their transfers. The recipient can collect cash from an MTN agent, who organizes a ‘payday’ at the village level each month. Community-nominated ‘monitors’ help us oversee the payday." @GiveDirectly, What we do - Operating Model@, Uganda tab.

• 2
  • Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016
  • "In October 2016, GiveDirectly will launch a $5-million-dollar retail campaign in Rwanda. Enrollment will continue until the end of February 2017. This program will provide GiveDirectly with more stability in case problems arise in other countries of operation, and offers an additional platform for interesting projects." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, pg. 6.
  • "GiveDirectly was founded by Paul Niehaus, Michael Faye, Rohit Wanchoo and Jeremy Shapiro, who were studying economic development at Harvard and MIT at the time and also looking for the most effective way to give their own money to reduce poverty. They found that cash transfers had a strong evidence base, and that the rapid growth of mobile payments technology in emerging markets had opened the door to delivering cash transfers securely and efficiently on an unprecedented scale. They created GiveDirectly as a private giving circle in 2009 and opened it to the public in 2011 after two years of operational testing." GiveDirectly, FAQs 2015
  • As of November 2016, GiveDirectly had provided partial or full cash transfers to approximately 52,000 households in western Kenya and eastern Uganda, and was continuing to transfer funds to additional households in both places. Ian Bassin, edits to GiveWell's review, November 10, 2016

• 3
  • GiveDirectly, Update for GiveWell, May 2015, Pg 2.
  • GiveDirectly, Update for GiveWell, September 2015, Pg 3.
  • Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 4
  • "Design Lab Nature of impact: In transferring funds, GD generates knowledge that expands or improves existing $150B+ cash market… Related org. priorities: Research studies" GiveDirectly, Update for GiveWell, September 2015, Pg 3.
  • "Rigorous, experimental evaluation of impacts is rare among nonprofits. GiveDirectly collaborates with third-party researchers to measure the impacts of cash transfers and answer complex design questions. Researchers are fully independent and independently-funded. We report the results of our evaluations and also announce studies in progress before the data are in, so that we can be held accountable for the results." GiveDirectly - Evidence - Research at GiveDirectly
  • This understanding is from many conversations with GiveDirectly and following GiveDirectly's progress over time.

• 5
  • "Benchmark Nature of impact: Success of GD and cash transfers generally creates pressure for transparency, evidence, and for other approaches to prove they outperform cash… Related org. priorities: Institutional partnerships, real time transparency." GiveDirectly, Update for GiveWell, September 2015, Pg 3.
  • This understanding is from many conversations with GiveDirectly and following GiveDirectly's progress over time.

• 6
  • "We transfer recipient households roughly $1,000, or around one year's budget for a typical household." GiveDirectly, What We Do - Operating Model, Overview tab
  • In late 2015, GiveDirectly adjusted the size of its transfers for inflation, so that it now transfers ~$1,040 (USD) to recipients in Kenya and ~$875 (USD) to recipients in Uganda. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
    • "In Kenyan Shillings, they are 10K, 50K, 50K (assuming no phone purchase). Ugandan shillings is 3.2M but division into different transfers hasn't been decided yet." Carolina Toth, email to GiveWell, September 25, 2015
    • XE currency converter, Kenya shillings to US dollars, September 25, 2015
    • XE currency converter, Uganda shillings to US dollars, September 25, 2015
  • The amount of time that it takes for total transfers to be sent has varied between campaigns. Campaigns that involved an experiment (like Rarieda and Nike) and older campaigns like Siaya have differed from the standard schedule. Conversation with Carolina Toth, GiveDirectly, November 20, 2014
  • In November 2015, GiveDirectly informed us that it aims to send transfers within 4 months. Carolina Toth, email to GiveWell, November 10, 2015

• 7
  • GiveDirectly plans to continue to adjust its transfer sizes for inflation in the future, possibly reassessing every 6 months. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • "In Kenyan Shillings, they are 10K, 50K, 50K (assuming no phone purchase). Ugandan shillings is 3.2M but division into different transfers hasn't been decided yet." Carolina Toth, email to GiveWell, September 25, 2015
  • XE currency converter, Kenya shillings to US dollars, September 25, 2015
  • XE currency converter, Uganda shillings to US dollars, September 25, 2015
  • GiveDirectly, Inflation analysis - Kenya. Note that the inflation analysis does not include information on how the transfer size should be adjusted in Uganda, so we are not sure how GiveDirectly settled on the Uganda amount.
  • Note that GiveDirectly sets its transfer sizes in the currency of the country in which it operates. So, if the exchange rate between that country's currency and the dollar changes, the transfer size in U.S. currency may appear to fluctuate, even if GD is transferring the same amount to recipients.

• 8
  • "In Kenyan Shillings, they are 10K, 50K, 50K (assuming no phone purchase). Ugandan shillings is 3.2M but division into different transfers hasn't been decided yet." Carolina Toth, email to GiveWell, September 25, 2015
  • "1 KES = 0.00947200 USD" XE currency converter, Kenya shillings to US dollars, September 25, 2015. So 10K KES = $94.72, and 50K KES= $473.6.

• 9
  • The amount of time that it takes for total transfers to be sent has varied between campaigns. Campaigns that involved an experiment (like Rarieda and Nike) and older campaigns like Siaya have differed from the standard schedule. Conversation with Carolina Toth, GiveDirectly, November 20, 2014
  • In November 2015, GiveDirectly informed us that it now aims to send transfers within 4 months. Carolina Toth, email to GiveWell, November 10, 2015

• 10

  GiveWell Household size analysis. Note that this data is based on a small sample from one of GiveDirectly's first campaigns (the Siaya campaign).

• 11
  • GiveWell Household size analysis
  • Mean daily per capita consumption among eligible households = $0.65. GiveDirectly, Offering Memorandum (January 2012) Pg 24.
  • Note that recipients in Uganda have a slightly higher average daily consumption of $0.83, according to GiveDirectly's website. GiveDirectly, What We Do - Operating Model, see the Uganda tab.
  • Note that GiveDirectly estimates that it transfers approximately $200 (USD) per person, although we have not asked where this estimate comes from: "How much do recipients get? We send each recipient household roughly $1,000 over the course of one year, or $200 per household member for the average household." GiveDirectly, FAQs 2015

• 12
  • Kenya: GiveDirectly told us that it chose to work in Kenya due to the robustness of M-Pesa as a mobile banking platform and the large population of people meeting its criteria who have access to mobile technology:
    • "Kenya was selected due to 1) the robustness of M-Pesa as a mobile banking platform 2) large population of target poor with access to mobile tech." Email from Paul Niehaus, President, GiveDirectly, and Joy Sun, COO, Domestic, GiveDirectly, November 18, 2013
  • Uganda: In choosing a second country in which to work, GiveDirectly said that it considered whether there was a mobile money provider accessible to the very poor, how costly it would be to operate in the country, how politically stable the country is, and how common corruption is in government affairs. The ease of moving staff between Kenya and Uganda was also a factor, as GiveDirectly's current COO (International) oversees the work in both places. Conversation with Paul Niehaus, President, and Rohit Wanchoo, Director, GiveDirectly, March 18, 2013
  • Rwanda: GiveDirectly chose to work in Rwanda to pursue a partnership with a bilateral aid donor after staff at the Rwanda office of that donor approached GiveDirectly about a possible cash transfer program there. The employee heard about GiveDirectly through an NPR news piece and became excited about the prospect of GiveDirectly working in Rwanda. GiveDirectly and the donor were then able to secure interest from Google in co-funding such a project through mutual connections. Paul Niehaus and Carolina Toth, conversation with GiveWell, May 28, 2015 Rwanda also has a growing mobile money system and a large population of extremely poor potential recipients.
    • "While the results of this project will be applicable in many places, Rwanda is an ideal setting in which to conduct it. Rwanda features a) a burgeoning mobile money landscape, with 17% mobile money penetration in mid-2013 as per National Bank of Rwanda and cell phone penetration at 72% in September 2014, b) a sizeable population of poor households (45% below the poverty line), and c) a track record of prioritizing transparent and innovative aid programming." GiveDirectly, Rwanda technical application Pg 5.
  • GiveDirectly tries to ensure that if its operations in a country are severely hampered or shut down, it has other locations that it can move to and scale up quickly in. GiveDirectly currently works in East Africa and is interested in expanding to different areas of the world, so that it can be more resilient to potential developments in the East Africa region that could negatively affect GiveDirectly's operations. Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 13

  Ian Bassin, edits to GiveWell's review, November 10, 2016

• 14

  Ian Bassin, edits to GiveWell's review, November 10, 2016

• 15
  • Paul Niehaus, GiveDirectly Founder, email to GiveWell, November 20, 2012
  • "GiveDirectly selected a region of the [second] country to set up initial operations in based on poverty data." Conversation with Paul Niehaus, President, and Joy Sun, COO, Domestic, GiveDirectly, July 18, 2013
  • We have reviewed poverty data that GiveDirectly sent us for Uganda. We have not written up the process we used to review this data. GiveDirectly has not yet selected regions within Rwanda.

• 16
  • Kenya: To select counties in Kenya, GiveDirectly told us that its executive staff uses data on poverty, population density, security, and presence of poverty-focused NGOs (with the goal of avoiding overlapping with these organizations).
    • Paul Niehaus, GiveDirectly Founder, conversation with GiveWell, October 22 2012
    • "County level – we look at data on poverty, population density, presence of poverty-focused NGOs (which we try to avoid), and security."@GiveDirectly, Operational Process Overview@ Pg 1.
  • Uganda: For Uganda, GiveDirectly told us that it chose a county to target initially based on poverty statistics, logistical factors and security considerations.
    • "Factors that informed decision to locate initial campaign in […] County: poverty rate, […] logistical ease for set up activities and cross-country management, […] minimum security for staff." GiveDirectly, Uganda targeting data, July 22, 2013

    We have seen poverty data for Uganda. GiveDirectly sent us the poverty data, which we internally reviewed.

  • Rwanda: GiveDirectly has not yet selected districts in Rwanda to work in.

• 17
  • "We prepared a '100,000 households plan' in early 2015 as a roadmap to select sub-counties for the next 100,000 households for GiveDirectly to enroll after the households in Ugunja and Ugenya that are a part of the GE study. We wanted to start with households that could be enrolled with operations based out of our Kisumu office and with our current Luo-speaking staff members. In the far right of the '20160216 100,000 households plan', you can see that we calculated a ranking that drew from World Bank and Kenya National Bureau of Statistics poverty data. It weighted a given sub-county’s rural poverty, distance from Kisumu, and if the region speaks Luo. The manual adjustments put regions we were already operating in at the top...We selected Rachuonyo North in Homa Bay from this shortlist as it had one of the highest estimated poverty rate from the World Bank (see summary tab in that file) and the largest number of potential eligible households...After selecting the sub-county, we select sub-locations. For the current sub-county, Rachuonyo South, we selected sub-locations that had < 10% urban populations and > 40% 'poor' according to World Bank estimates in the attached '20151005 Rachuonyo S. Villages.'" GiveDirectly, Village selection process Kenya
  • We have spot-checked the poverty data that GiveDirectly used to select sub-counties in Homa Bay; we believe GiveDirectly's processes were reasonable. GiveDirectly, 100,000 households plan
  • We have previously reviewed poverty data that GiveDirectly sent us for divisions within Siaya district. GiveDirectly, Siaya poverty data by location
  • We have not seen examples of or asked GiveDirectly to describe in detail how it selected sub-county locations in Uganda or Rwanda.

• 18

  "Recently we have been relying on sublocation-level poverty data and looking at village names and locations to omit villages that would be too urban for our current model. We have also been working with satellite data to approximate village-level poverty." Conversation with Carolina Toth, GiveDirectly, November 20, 2014

• 19
  • Rarieda, Kenya: In Rarieda (the site of the RCT and the first transfers GiveDirectly provided), GiveDirectly sought poorer districts (based on 2005 census data) that were in places with sufficient accessibility, M-PESA usage, population density to make it more convenient, proximity to Innovations for Poverty Action (the RCT implementer) offices and where there would be a sufficient number of potential recipients in this district (# of thatched roof houses). GiveDirectly chose to work in Rarieda District because it was slightly poorer than the nearby Siaya district. In Rarieda, there are slightly more than 300 villages, and GiveDirectly conducted a census in each village to determine the number of eligible (thatched-roof) and ineligible households in each. It then selected the 100 with the highest proportion of thatched-roof to non-thatched-roof households, and randomly selected 60 of those to serve as the treatment and control groups in its trial. Faizan Diwan, Innovations for Poverty Project Associate, conversation with GiveWell, November 8, 2012
  • Siaya, Kenya: GiveDirectly shared the full details of its village selection process for Siaya, including data for each village and the method for weighting the different factors used to select villages in that campaign. GiveDirectly chose to work in Siaya District, the location of the three other sets of transfers GiveDirectly has initiated, because it shared local administration with the Rarieda District, making expansion easier; it chose not to remain in Rarieda because it did not want to overlap in areas in which the RCT was being conducted. Using administrative data, it chose 3 locations within Siaya that it believed had the highest poverty levels. It then ranked the 100 villages in these locations. See @GiveDirectly, Siaya Village Index@ for the calculations that GiveDirectly did to create "poverty scores" for different villages in Siaya. The weights placed on each indicator (in constructing the index) were determined using the process described in @GiveDirectly, Village Targeting Regression@: the more detailed "poverty scores" from GiveDirectly's Rarieda study were regressed on indicators such as "village population," "number of boreholes," etc. GiveDirectly was not able to contact all Village Elders to obtain data (staff estimate they reached 85 out of 100) and it excluded villages whose Village Elders it was not able to reach. In June 2012, it selected the 7 villages which its model ranked as highest poverty to receive the Siaya transfers. For the project funded by the Nike Foundation, GiveDirectly selected the next 36 villages in its ranked list of 100 villages in Siaya District, as this number of villages provided a sample size sufficient to meet their target size. In the Google transfers, GiveDirectly is continuing to work down the list of 100 villages to target those not already targeted in the Siaya or Nike Foundation transfers. Piali Mukhopadhyay, COO, International, GiveDirectly, conversation with GiveWell, November 8, 2012
  • Homa Bay, Kenya: We have reviewed the data GiveDirectly used to select villages within Homa Bay, and we believe GiveDirectly's process was reasonable. Once GiveDirectly had selected locations within Homa Bay County (see process above), it selected sublocations based on the poverty rate and the percentage of the population living in an urban area. For each sublocation that met GiveDirectly's criteria, every village within that sublocation was included as one that GiveDirectly would visit. See GiveDirectly, Rachuonyo S. Villages
  • Pilot campaign in Uganda: For the pilot campaign in Uganda, GiveDirectly relied on publicly available poverty data, which we have reviewed, as well as data that it received from local officials, which we have not reviewed. Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013 GiveDirectly also sent field staff to test cell phone reception and measure proximity to market centers, and used these as inputs into its village selection (optimizing for good reception and longer distances from market centers). Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013 GiveDirectly also considered population size in the villages, so that it could enroll all eligible households in each village and not exceed the budget for the Uganda pilot campaign, which was funded by part of its Google Global Impact Award.
    • Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013
    • GiveDirectly prefers not to enroll only part of a village. GiveDirectly slightly extended its Uganda 2M campaign after realizing that its target enrollment numbers would have caused it to stop enrollment halfway through a village. See the Ug-201404 (ext) campaign in GiveDirectly, Monthly operations report, August 2015. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 20
  • Previously, in the Kenya 1.2M campaign, GiveDirectly selected villages by manually estimating the proportion of thatch- to iron-roof homes with satellite imagery. In the Kenya rolling enrollment campaign, GiveDirectly used a machine learning algorithm to estimate thatch-iron proportions at the village level based on satellite imagery. In the Uganda 2M campaign, GiveDirectly relied on parish-level census data with poverty measures, as well as mobile money coverage. GiveDirectly, Update for GiveWell, April 2014, Pg 3.
  • In the Kenya rolling campaign, as GiveDirectly moved into Homa Bay, it selected villages using World Bank census data. See GiveDirectly, Rachuonyo S. Villages

• 21
  • Seeking government approvals for GiveDirectly cash transfer campaigns

    By now, GiveDirectly understands well the process for seeking government approvals in Kenya and does not see acquiring approvals as a major risk. GiveDirectly said that in Kenya it is important to maintain relationships with government officials at the county and district levels; district commissioners introduce GiveDirectly to chiefs, and chiefs introduce GiveDirectly to Village Elders. In Uganda, there are no counties, so GiveDirectly coordinates with a few people at the district level to acquire approvals, and from there connect with officials at the local level […] As part of its networking in Kenya, GiveDirectly staff have met with the Permanent Secretary for Devolution and Planning. This is someone who could help GiveDirectly acquire permission to work in new counties in Kenya. Conversation with Piali Mukhopadhyay, GiveDirectly, October 20-21, 2014

  • "When entering a new area, the COO meets with a series of officials to explain the project, obtain permission, and establish a relationship in case any problems arise:
    • District Commissioner
    • Chief
    • Assistant chiefs
    • Village elders"

    @GiveDirectly, Operational Process Overview@, Pg 1.

  • In Uganda, GiveDirectly had to spend more time meeting with officials early on in the process, because there is a greater bureaucratic structure than in Kenya. This engagement tapered off after the early stage, though GiveDirectly remains in touch with officials by phone to let them know when GiveDirectly has started conducting field activities. Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013

• 22
  • "Govn’t relations: Signed MOUs with local officials to maximize buy-in and formalize relationship" GiveDirectly, Update on process changes, August 28, 2013
  • Typical approval process
    Kenya:
    • "Seek buy-in from County and District Commissioner and sign written agreement w/district
    • Ensure Governor’s office and relevant Country admin officials informed of expansion activities"

    Uganda:

    • "Attain approval letter from Resident District Commissioner for natl renewal
    • Attain approval letters from RDC, District Security Officer, District Intelligence Officer, and District Development Officer for local renewal"

    GiveDirectly, Update for GiveWell, October 2014, Pg 9.

• 23

  @GiveDirectly, Operational Process Overview@, pg. 2, and Ian Bassin, edits to GiveWell's review, November 10, 2016

• 24

  Data collected during census can be found in enrollment databases, for example: GiveDirectly, Kenya 1.2M enrollment database

• 25
  • "Enumerators enter villages, engage a local to serve as guide for the day, and enumerate all households living in the village, noting which homes are eligible." @GiveDirectly, Operational Process Overview@, Pg 2.
  • Data collected during census can be found in enrollment databases, for example: GiveDirectly, Kenya 1.2M enrollment database
  • "Household level – we enroll households living in mud and thatch homes." @GiveDirectly, Operational Process Overview@, Pg 2.
  • "Dropped mud walls as eligibility requirement." GiveDirectly, Update for GiveWell, October 2014
  • "We've now completed our targeting pilots in Kenya, and have selected a new targeting criteria for Homa Bay. We looked at a range of alternatives from simple proxy means tests to more complicated rules to subjective scoring to community-based methods, and evaluated them along dimensions of accuracy, perceived fairness, gameability, and cost. The results were useful in the near term for Homa Bay, and also provide a framework and starting point for criteria evaluation in new geographies going forward." Carolina Toth, email to GiveWell, October 20, 2015
  • GiveDirectly also gives each household a "token" during the census: "During census, we provide each household with a card with a number on it (a “token”). When we then seek to enroll eligible households, we ask to see the token. This helps ensure that the household being enrolled is the same as the one that was censused and we deemed eligible based on that information." GiveDirectly staff, responses to monitoring questions, October 11, 2016, pg. 1.

• 26

  "Enrollment. A second, distinct enumerator returns to enroll households identified as eligible, give them a SIM card and instructions on how to register if needed." @GiveDirectly, Operational Process Overview@ Pg 1.

• 27
  • Kenya: In Kenya, registration involves giving the household member a SIM card (if they do not already have an M-PESA account), which is used to transfer funds through the M-PESA system, and collecting other data that can be checked against the initial data from the census. Recipients are also given the option of purchasing a cell phone from GiveDirectly at the time of registration, the cost of which is removed from the recipient's transfer (Conversation with Paul Niehaus, President, and Michael Faye, Director, GiveDirectly, October 6, 2012).
    • "GiveDirectly gave cash transfer recipients the option of spending some of the money that they receive to buy a phone provided by GiveDirectly." Conversation with Paul Niehaus, President, and Michael Faye, Director, GiveDirectly, October 6, 2012, pg. 3.
    • "Enrollment. A second, distinct enumerator returns to enroll households identified as eligible, give them a SIM card and instructions on how to register if needed… We check that the data collected in enrollment and the village census match." @GiveDirectly, Operational Process Overview@, pg. 1.
    • See process description in @GiveDirectly, Operational Process Overview@, pg. 3.
    • Data collected during registration can be found in enrollment databases, for example: GiveDirectly, Kenya 1.2M enrollment database
  • Uganda: In Uganda, GiveDirectly uses a similar registration process. Additionally, GiveDirectly helps recipients in Uganda obtain national ID cards and arranges for mobile money agents to visit villages to register recipients in the mobile money system:
    • "Do recipients need to have a mobile phone to participate?
      No. Households need at least a SIM card to participate, and we give SIM cards to households that do not already have one. We also give recipients the option of purchasing a phone from us at bulk rates in order to make it easier to communicate with them. When recipients choose this option we deduct the value of the phone from their transfer. Historically the large majority of recipients in both Kenya and Uganda have chosen to buy a phone." GiveDirectly, FAQs 2015
    • "How do you prevent corruption?
      The two main corruption risks that typically arise in transfer programs involve (a) manipulation of the list of eligible recipients and (b) diversion of transfers sent to eligible recipients. We address the first through a comprehensive audit process, using multiple independent checks to ensure that recipients are eligible and have not been charged bribes to get on the list. These checks include in-person visits by different staff members, in-person audits by senior management, remote audits of image and satellite image data, and phone calls with each recipient, all prioritized using modern analytics. We address the second through identity-matching between our records and those of our payment providers, through comprehensive follow-up calls to ensure money is reaching the intended recipients, and in some cases through direct staff monitoring of cash-out points." GiveDirectly, FAQs 2015
    • GiveDirectly told us that in Uganda, it is possible to purchase ID cards that can be formally approved by the signature of one's Local Councilperson. GiveDirectly helped recipients obtain ID cards by purchasing the cards, sending field staff to villages to take photographs of recipients, printing the photographs for the ID cards at a local printer, working with Local Councilpeople to approve the cards, and arranging for recipients to collect their cards.
      GiveDirectly told us that the mobile money agents did not charge a fee to visit villages to register recipients, and that GiveDirectly field staff were present to supervise the process. Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013
    • The logistics are significantly harder in Uganda than in Kenya. For example, when GiveDirectly enters a new village in Uganda, over 90% of recipients need SIM cards because they did not previously have cell phones, and about 70-80% of recipients need national IDs. GiveDirectly coordinates registration drives for people to get national IDs - they buy national ID booklets, print a photo of each recipient to put in the booklets, and have the Local Councilperson stamp the booklets to approve them. GiveDirectly was able to reach 85-90% of people through these registration drives, returning IDs within about 1 week of visiting eligible households. In the Uganda 2M campaign, there are 9 villages, and GiveDirectly was able to put them all through the national ID registration process within 1 month, so that 90% of eligible households were ready to receive transfers when payments started. (The remaining households will receive their transfers on a delayed schedule, once they complete registration.) GiveDirectly also facilitates recipients signing up for a mobile money account with MTN by having an agent visit the villages. Once recipients have signed up for an account, MTN generally activates their line within 2-3 weeks. By the time the backcheck team visits villages, most recipients’ lines are active. Conversation with GiveDirectly field staff, October 20-21, 2014
  • "During census, we provide each household with a card with a number on it (a “token”). When we then seek to enroll eligible households, we ask to see the token. This helps ensure that the household being enrolled is the same as the one that was censused and we deemed eligible based on that information." GiveDirectly staff, responses to monitoring questions, October 11, 2016, pg. 1.
  • "[GiveWell]: How are the questions about recipients being an imposter asked? (Columns DM and DN) Are these questions asked of neighbors, or judgment calls made by the enrollment team, or something else? [GiveDirectly staff]: These are usually asked of neighbors, the resident guiding us through the village or sometime a village elder. An example of the question asked is: 'Does so and so live here? How long have they lived here?'" GiveDirectly staff, responses to monitoring questions, October 11, 2016, pg. 2.
  • "Field officers are asked to record when they suspect a recipient of being dishonest. For example, a polygamous husband may try to claim that one of his daughters is actually an additional wife in order for her to qualify for an additional payment (since we enroll each wife in a polygamous household). A field officer might discover this when reviewing documents provided by the daughter or others in the household (e.g, if a document says the family has 3 daughters but they present as having only 2). Usually, the field officers also ask the neighbours and their guides, whose information they use to corroborate what the recipient told them during the interview." GiveDirectly staff, responses to monitoring questions, October 11, 2016, pgs. 2-3.

• 28
  • GiveWell site visit to GiveDirectly, October 2014
  • For example, see GiveDirectly, Uganda pilot enrollment database - Akumure. Note that even households that have passed the "Registration" phase (starts in column AB) must still go through the "Back check" phase (the "Census-registration comparison" (column EF), "Photograph check" (column EP), and "Remote Check" (column ES)), and a possible "Audit" (column FH) before their final enrollment status is determined (columns FK and FL).

• 29

  Data collected during back checks can be found in enrollment databases, for example: GiveDirectly, Kenya 1.2M enrollment database

• 30

  "Back-checking. Another enumerator, distinct from the census and enrollment workers, revisits each enrolled household to check that they eligible, didn’t have to pay a bribe to enroll, etc." @GiveDirectly, Operational Process Overview@, Pg 3.

• 31
  • "In early 2016, GiveDirectly allocated a fairly large sum of money to test a new implementation model in Kenya. A separate field director and team were hired approximately six months ago, and the experiment is taking place in an area close to GiveDirectly's other field operations. The primary aim is to double the program's size in 2017 without significantly increasing the management structure. The new model seeks to increase throughput per manager by eliminating the token payment and back check steps. GiveDirectly will assess gains in throughput as well as costs, which might occur in the areas of comprehension and fraud. Results and data should be available after a few months of disbursements. By the end of 2016, GiveDirectly hopes to have a blueprint for implementing a similar model in other countries." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pgs 4-5.
  • "Approach to checking quality for campaigns with steps removed:
    • Recipient Comprehension: With the removal of back check, we are tracking any deviation in recipient comprehension below 90% (our preferred rate). We are tracking this through additional survey questions in both audit and our follow up surveys where we ask recipients their understanding of our program. Additionally, Associate Field Managers will be conducting quality check surveys at audit stage for another data point on recipient comprehension.
    • Transfer Integrity & Adverse Events Detection. Given additional risk of two payment structure, we will be assessing (on a bi-weekly basis) the percent of transfers reversed and percentage of adverse events happening after first payment and after second payment. This will then be compared to our standard quality bar to assess significant deviations. Given a greater need for over the phone assistance, we will also be tracking through our follow up program the percentage of recipients who received no customer service and tried.
    • Fraud Detection. With the removal of back check and a two payment structure that makes each payment riskier, we will be checking the percentage of recipients flagged for additional audit stage and then the percentage of those deemed ineligible after audit is conducted. We also have an internal audit team in place that will be conducting surveys post-payment to see if fraud occurred."

    Piali Mukhopadhyay, COO, International, GiveDirectly, email to GiveWell, August 25, 2016

• 32
  • "How do you prevent corruption? The two main corruption risks that typically arise in transfer programs involve (a) manipulation of the list of eligible recipients and (b) diversion of transfers sent to eligible recipients. We address the first through a comprehensive audit process, using multiple independent checks to ensure that recipients are eligible and have not been charged bribes to get on the list. These checks include in-person visits by different staff members, in-person audits by senior management, remote audits of image and satellite image data, and phone calls with each recipient, all prioritized using modern analytics." GiveDirectly, FAQs 2015
  • In Kenya, the field staff who do audits are not involved in earlier enrollment activities: "[In Kenya] the follow up team sends some of its members to do audits and staff are not pulled from prior enrollment teams." Conversation with Carolina Toth, GiveDirectly, November 20, 2014.
  • In Uganda, the field staff who have done audits in past campaigns were from earlier enrollment teams. Conversation with GiveDirectly field staff, October 20-21, 2014 (Note that this point was not included in the notes from this conversation).
  • However, GiveDirectly has told us that it ensures that staff members never revisit households that they helped through previous steps of the enrollment process: "In GiveDirectly's standard operating model, each field officer is responsible for completing one type of step in the enrollment process; for example, a field officer might be assigned to the 'backcheck team.' In two of its experimental campaigns (the experiment to increase throughput in Kenya and its work in a coffee farming community), GiveDirectly is experimenting with a "wave" approach. In this scenario, the same officers perform multiple steps; however, for steps intended to provide a check on previous steps, officers do not perform multiple steps in the same village. This ensures that the same field officer does not visit a household twice during the enrollment process." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 5.

• 33
  • In August 2016, we asked GiveDirectly for an update on which discrepancies cause it to audit a household:
    • "GiveDirectly has adjusted some of its audit flags based on factors such as changes in enrollment numbers and eligibility criteria. Households might be flagged for an audit for a variety of reasons, including SIM card mismatches, GPS coordinate distance mismatches beyond a certain threshold, and, in programs where housing materials affect eligibility, the appearance of a metal roof on a recipient's home after registration." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 4.
    • "The audit flags differ depending on target criteria. Some standard flags include discrepancies between data collected across census and backcheck, including: GPS coordinates, phone number, name / spouse name, token number, face photo. In Kenya we have additional flags based on our targeting criteria (differences in housing size)." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, August 25, 2016
  • Previously, GiveDirectly's procedure for identifying households to audit:
    • GiveDirectly collects information about recipients during the first three stages of a campaign: census, registration, and backcheck. Some information, such as recipient name, GPS location, housing materials, and identifying photograph, is collected at more than one stage and then checked for mismatches. (These checks are currently conducted using Excel but will eventually be automated through Segovia technology. One exception is comparing identifying photographs of recipients, which is done using a crowdsourced work platform called Mechanical Turk.)
    • Each mismatch in recipient information is assigned a certain weight depending on how likely it is to be an indication of gaming. (GiveDirectly said that it determined the likelihoods of various mismatches indicating gaming by conducting an analysis of the mismatches present in past cases of gaming. We did not review this analysis.) GiveDirectly said that the mismatches with the highest weights are mismatches in identifying photographs and housing materials.
    • Each recipient is assigned a total mismatch "score" (the composite of all their weighted mismatches). Recipients with scores above a certain level are audited.

    Conversation with GiveDirectly field staff, October 20-21, 2014

• 34

  Conversation with GiveDirectly field staff, October 20-21, 2014
  GiveWell site visit to GiveDirectly, October 2014

• 35
  • Conversation with Carolina Toth, GiveDirectly, November 20, 2014
  • For an example of GiveDirectly's commitment to potential recipients, see GiveDirectly, Blog post, August 21, 2015

• 36

  “People have to be at home at registration and back check so that they can be given phones, have safety information explained etc. If they are not at home during the first attempt to visit, we re-visit them several times until they can be found." Conversation with Carolina Toth, GiveDirectly, November 20, 2014

• 37
  • GiveDirectly, Siaya enrollment database
  • GiveDirectly, Google enrollment database
  • GiveDirectly, Nike enrollment database
  • GiveDirectly, RCT Enrollment Database
  • GiveDirectly, Kenya 2M enrollment database, September 2013
  • GiveDirectly, Uganda pilot enrollment database - Kosile
  • GiveDirectly, Uganda pilot enrollment database - Kawo
  • GiveDirectly, Uganda pilot enrollment database - Kanyamutamu
  • GiveDirectly, Uganda pilot enrollment database - Akumure
  • GiveDirectly, Kenya 1.2M enrollment database
  • GiveDirectly, Kenya 2M census results, July 2013
  • GiveDirectly, Kenya, Uganda, and Rwanda enrollment database, 2016
  • GiveDirectly gave permission to publish the following databases on the condition that all personally identifying information was removed. Due to the size of the databases, we have not yet anonymized them and therefore have not published them. If you are interested in the content of these databases, please contact us at info@givewell.org.
    • GiveDirectly, Kenya rolling campaign enrollment database - Homa Bay
    • GiveDirectly, Kenya rolling campaign enrollment database - Siaya
    • GiveDirectly, Uganda 2M campaign enrollment database

• 38

  "We use electronic payment systems; typically, recipients receive an SMS alert and then collect cash from a mobile money agent in their village or nearest town." GiveDirectly, What We Do - Operating Model, Overview tab

• 39
  • "In early 2016, GiveDirectly allocated a fairly large sum of money to test a new implementation model in Kenya. A separate field director and team were hired approximately six months ago, and the experiment is taking place in an area close to GiveDirectly's other field operations. The primary aim is to double the program's size in 2017 without significantly increasing the management structure. The new model seeks to increase throughput per manager by eliminating the token payment and back check steps. GiveDirectly will assess gains in throughput as well as costs, which might occur in the areas of comprehension and fraud. Results and data should be available after a few months of disbursements. By the end of 2016, GiveDirectly hopes to have a blueprint for implementing a similar model in other countries." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pgs 4-5.
  • "Approach to checking quality for campaigns with steps removed:
    • Recipient Comprehension: With the removal of back check, we are tracking any deviation in recipient comprehension below 90% (our preferred rate). We are tracking this through additional survey questions in both audit and our follow up surveys where we ask recipients their understanding of our program. Additionally, Associate Field Managers will be conducting quality check surveys at audit stage for another data point on recipient comprehension.
    • Transfer Integrity & Adverse Events Detection. Given additional risk of two payment structure, we will be assessing (on a bi-weekly basis) the percent of transfers reversed and percentage of adverse events happening after first payment and after second payment. This will then be compared to our standard quality bar to assess significant deviations. Given a greater need for over the phone assistance, we will also be tracking through our follow up program the percentage of recipients who received no customer service and tried.
    • Fraud Detection. With the removal of back check and a two payment structure that makes each payment riskier, we will be checking the percentage of recipients flagged for additional audit stage and then the percentage of those deemed ineligible after audit is conducted. We also have an internal audit team in place that will be conducting surveys post-payment to see if fraud occurred."

    Piali Mukhopadhyay, COO, International, GiveDirectly, email to GiveWell, August 25, 2016

• 40
  • "GiveDirectly conducts follow-ups with all recipients after the token and first lump sum payments, respectively. In campaigns in which recipients receive larger numbers of payments, follow-ups might be done near the beginning, midpoint, and end of the process rather than after each payment...In both Kenya and Uganda, GiveDirectly conducts in-person follow-ups with recipients flagged as 'vulnerable.' Vulnerability flags include a recipient designating a trustee to receive payments on his or her behalf." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 3.
  • "What percentage of recipients receive in-person follow-ups? ~10-15%. This would include
    • everyone with trustees (~4%) [these are situations in which the transfer goes to someone other than the ultimate beneficiary]
    • everyone with helpers (7%) [these are situations where the transfer goes to the beneficiary, but a helper is identified because the beneficiary is vulnerable in some way]
    • few percent for folks not reached by phone"

    Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, August 25, 2016

  • There are short verification calls to confirm that the transfer was received and ask if the recipient experienced any problems after each transfer. Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013 There are also two longer surveys administered to a randomly selected sample of recipients after larger portions of the transfers have been sent.
    • Piali Mukhopadhyay, COO, International, GiveDirectly, conversation with GiveWell, November 7, 2012
    • "For Uganda 201404, the plan is to do monthly short surveys, and then long follow up after payday 2 and 9 or 10." Carolina Toth, GiveDirectly, email to GiveWell, November 14, 2014
    • Carolina Toth, email to GiveWell, November 10, 2015
  • In the longer surveys, GiveDirectly staff ask recipients a number of questions including whether they received the transfers or had any trouble withdrawing funds, how they spent the funds, and whether there were any problems in their community relating to the transfers.
    • GiveDirectly, Uganda pilot follow up data, April 2014
    • GiveDirectly, Kenya follow up data, November 2014
    • We have reviewed and made public data from these calls for ongoing transfer campaigns in Kenya and Uganda:
      • GiveDirectly, Rarieda verification stats
      • GiveDirectly, Siaya verification stats
      • GiveDirectly, Siaya follow-up data - disaggregated
      • GiveDirectly, Nike follow-up data - disaggregated
      • GiveDirectly, Google follow-up data - disaggregated (LS - long)
      • @GiveDirectly, Rarieda Top-up verification (short)@
      • GiveDirectly, Rarieda verification (top ups), May 26, 2013
      • GiveDirectly, Siaya verification, June 15, 2013
      • GiveDirectly, Google verification, September 2013
      • GiveDirectly, Nike verification (final), September 2013 (based on full survey given to recipients after whole transfer was sent)
      • GiveDirectly, Nike verification (combined), May 2013 (based on full survey given to recipients early in campaign, which GiveDirectly later stopped administering because it was too time-consuming with monthly transfers Conversation with Carolina Toth, Field Director, GiveDirectly, October 24, 2013)
      • GiveDirectly, Nike verification (short version), June 2013 (based on abbreviated survey given to recipients after each part of the transfer was sent, in place of "combined" survey Conversation with Carolina Toth, Field Director, GiveDirectly, October 24, 2013)
      • GiveDirectly, Kenya follow up data, November 2014
      • GiveDirectly, Uganda pilot follow up data, April 2014
      • GiveWell, GiveDirectly follow up surveys summary - Uganda, September 2015
      • GiveWell, GiveDirectly follow up surveys summary - Kenya, September 2015 (GiveDirectly gave permission to publish this database on the condition that all personally identifying information was removed. Due to the size of the database, we have not yet anonymized it and therefore have not published it. If you are interested in the content of the database, please contact us at info@givewell.org.)

• 41
  • "For Uganda 201305, we did long follow up surveys on a rolling basis and most recipients were followed up with twice-- this is the data you've already received. At the end of 201305, there was going to be one short survey, and this survey was accelerated as part of the fraud investigation. These are labelled _short and _short_crosscheck and are attached. There were no monthly short surveys. We've not yet decided when/if to fill in the rest of the short surveys for those not reached during the fraud investigation." Carolina Toth, GiveDirectly, email to GiveWell, November 14, 2014
  • "For Uganda 201404, the plan is to do monthly short surveys, and then long follow up after payday 2 and 9 or 10." Carolina Toth, GiveDirectly, email to GiveWell, November 14, 2014
  • In early transfer campaigns, GiveDirectly transferred a first, full installment to recipients before its first call; in its Google campaign, GiveDirectly implemented the initial small transfer to enable it to identify problems before transferring the larger amount. Piali Mukhopadhyay, COO, International, GiveDirectly, conversation with GiveWell, November 7, 2012.

• 42

  "For its standard Kenya and Uganda programs, GiveDirectly has implemented a rule in Segovia to block payments until the previous one has been confirmed: for example, first and second lump sum payments are blocked until token and first lump sum payments, respectively, have been confirmed via a follow-up call or visit. In an effort to ensure implementation quality, GiveDirectly seeks to keep the percentage of households with blocked payments under 5%. The delay between the first and second lump sum payments tends to be the longest. After the final payment, GiveDirectly aims to follow up with a geographically representative sample of roughly 50-60% of households. Staff in Kenya and Uganda have been focused on following up on the first two payments because of the new rule described above; they are in the process of catching up on the schedule for final follow-ups." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 4.

• 43
  • Piali Mukhopadhyay, COO, International, GiveDirectly, email to GiveWell, November 23, 2012
  • We have reviewed the records of calls made to GiveDirectly's Kenya hotline from May 2012 – September 2014.
    • GiveDirectly, Kenya hotline log, July 2013 (May 2012 – July 2013)
    • GiveDirectly, Follow-up tracker, July 2013 (July – October 2013)
    • GiveDirectly, Follow-up tracker, October 2014 (January 2013-September 2014). This file also contains issues reported through other channels, such as to field staff during field visits and during follow up surveys.

• 44
  • GiveDirectly told us that for the Kenya 2M campaign, its enrollment field staff conducted a short survey with anyone who approached the field staff to complain that they had been unfairly or mistakenly skipped during the census. Though [in campaigns prior to and including Kenya 2M] GiveDirectly [did] not add recipients after the census [had] been conducted, it intends to continue carrying out the surveys for future campaigns, as a way of tracking complaints, recognizing potential issues with the census, and assessing changes intended to improve GiveDirectly's census process. Conversation with Carolina Toth, Field Director, GiveDirectly, October 24, 2013. We have not reviewed the results of this survey. As of November 2015, GiveDirectly informed us that it still conducts these surveys, and that people who are skipped in error have an opportunity to be enrolled. Carolina Toth, email to GiveWell, November 10, 2015
  • Starting with the Kenya rolling enrollment campaign in early 2014, GiveDirectly now adds into the enrollment process households that complain about having been skipped at census. Conversation with Carolina Toth, GiveDirectly, November 20, 2014

• 45

  GiveDirectly FY 2011 Form 990

• 46

  "GD has had one person working full-time since January of 2011. Jeremy Shapiro was full time during 2011 and Piali Mukyopadhyay is full-time now." GiveDirectly, clarifications on GiveWell's draft review of GiveDirectly

• 47

  Conversation with Paul Niehaus, President, and Rohit Wanchoo, Director, GiveDirectly, March 18, 2013

• 48
  • Conversation with Paul Niehaus, President, and Joy Sun, COO, Domestic, GiveDirectly, July 18, 2013 (unpublished)
  • GiveDirectly, Update for GiveWell, July 2013
  • Conversation with Paul Niehaus, President, and Joy Sun, COO, Domestic, GiveDirectly, October 16, 2013
  • Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013
  • Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 49

  GiveDirectly, Monthly operations report, February 2016

• 50
  • "Introduced Project Associate role to strengthen focus on productivity, QC and professional development and increase FD leverage (Ke)" GiveDirectly, Update for GiveWell, April 2014
  • GiveDirectly changed the name of Project Associates to Field Managers in 2015. GiveDirectly, Estimate of personnel 2015

• 51
  • "Greater leverage for FD to work on tech integration, high-level analysis (e.g. “smart” audits)" GiveDirectly, Update for GiveWell, October 2014, Pg 6.
  • GiveDirectly changed the name of Senior Field Officer to Assistant Field Manager in 2015. GiveDirectly, Estimate of personnel 2015

• 52

  GiveDirectly, Monthly operations report, February 2016

• 53
  • Email from Paul Niehaus, President, GiveDirectly, and Joy Sun, COO, Domestic, GiveDirectly, November 18, 2013
  • In Uganda, GiveDirectly hired some of the highest performing FOs from the pre-transfer FO staffs to stay on as FOs who would conduct follow-up surveys. (Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013). It is our impression that FOs who conduct follow-up surveys are often hired out of the pre-transfer FO staffs.

• 54

  GiveDirectly, Monthly operations report, February 2016

• 55
  • "Three members of GiveDirectly‘s board of directors (Paul Niehaus, Michael Faye, and Chris Hughes) are planning to start a for-profit technology company, Segovia, aimed at improving the efficiency of cash transfer distributions in the developing world. Segovia plans to sell software to developing-country governments for use in implementing their cash transfer programs." (From GiveWell's update on GiveDirectly, June 20, 2014)
  • What GiveDirectly uses Segovia for: "Segovia, the product GiveDirectly uses for end-to-end data management, enables GiveDirectly to track recipients' progress through each step of its enrollment, payment, and follow-up processes and manage other tasks such as data merging, matching checks between different steps, audit flags, and roster generation. Currently, Segovia automatically generates lists of pending payments, but GiveDirectly must log in to a mobile payment portal, such as M-Pesa or MTN, to upload payments; this additional step takes time and could pose a security risk. Segovia is working to bring the entire payment process into Segovia by directly integrating GiveDirectly's platform with the payment portals. This new feature should be available within the next quarter. Segovia's monitoring and reporting tools, including its dashboards, provide significant value to GiveDirectly, particularly as its campaign numbers increase." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 1.

• 56

  "Mr. Faye and Dr. Niehaus split their time between Segovia and GiveDirectly, and make efforts to ensure that each organization always has the equivalent of one full-time CEO. For the sake of efficiency, staff member Melissa Harpool, who manages Mr. Faye's and Dr. Niehaus' schedules across Segovia and GiveDirectly, is employed by both organizations. All of the other staff members are employed by only one of the two organizations. There have been no issues related to staff leaving one organization for the other; the organizations' respective staff profiles are quite different." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 3.

• 57

  Ian Bassin, edits to GiveWell's review, November 10, 2016

• 58

  "For the sake of efficiency, staff member Melissa Harpool, who manages Mr. Faye's and Dr. Niehaus' schedules across Segovia and GiveDirectly, is employed by both organizations." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 3.

• 59

  We mention this in this blog post. If GiveDirectly pays for Segovia's services, then in some sense GiveDirectly is contributing additional income to its co-founders, who are also the co-founders of Segovia.

• 60

  "Segovia intended and did provide free services to GiveDirectly initially, but the scale of GiveDirectly's needs increased considerably. And the unique nature of different GiveDirectly campaigns required considerable ongoing technical assistance from Segovia that was not part of the original assumptions about what would be required in terms of ongoing, active support. Given that, it was both necessary for Segovia and fair for GiveDirectly to provide some compensation for those services." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, August 25, 2016

• 61
  • "Michael Faye and Paul Niehaus are co-founders and board members of both GiveDirectly and Segovia. In order to mitigate the potential legal and ethical risks of this situation, they are recused from GiveDirectly's decisions about Segovia. Ms. Mukhopadhyay and Gavin Walsh, Director of Finance and Information Systems, manage the decision to do business with Segovia and the details of that contract.

    During recent contract negotiations, they assessed the quality and cost implications of building a similar product to Segovia in-house, because no similar product is offered by any other vendors. They concluded that using Segovia's product was the best option and negotiated a pricing model with its product lead and head engineer. GiveDirectly received a discounted quote in line with the planned volume of transfers. Ms. Mukhopadhyay and Mr. Walsh's proposal was accepted by the other independent board members. In making this decision, board members reviewed information about the extent to which Segovia's growth relies on GiveDirectly's business to ensure that GiveDirectly was not too large a percentage of Segovia revenue; this was out of a concern that such a scenario could cause some to question the relationship even if the decision to use Segovia is independently sound." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pgs 2-3.

  • "Our independent board members did ask what % of Segovia's revenue our contract would represent in order to make sure it was not so high as to raise appearance concerns that could be detrimental to the GD brand and mission. They only considered that information in that regard, not otherwise. Because that information is not ours, however, and is Segovia's that was shared with our independent board members solely for that purpose, we're not at liberty to share it." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, August 25, 2016
  • "The current agreement term ends on February 28, 2017. The parties intend to discuss renewal beyond that point." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, August 25, 2016

• 62

  Paul Niehaus, email to GiveWell, October 11, 2016

• 63

  GiveDirectly, Update for GiveWell, July 2014, Pg 5.

• 64

  Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 65

  Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015 Note that we are calling rigorous tests of interventions "studies" and GiveDirectly's less rigorous, internal testing of new variations on its model (e.g, using biometric scans for additional security) "campaign variations."

• 66

  "Based on conversations with policymakers, GiveDirectly has identified gaps in the evidence-base for cash transfers that currently limit policymakers’ ability to implement cash transfer programs or to do so effectively. GiveDirectly has spoken with policymakers in Kenya and Indonesia, as well as representatives of the UK’s Department for International Development (DFID). The leading question that came out of these conversations was about the macroeconomic impacts, or “general equilibrium effects,” of cash transfers when conducted at a national scale. This includes factors such as enterprise structure, prices, local public finance, and local government activities. This question is the primary motivation for GiveDirectly’s top research priority: a study of general equilibrium effects." Conversation with GiveDirectly, July 7, 2014, Pg 2.

• 67

  "Objective:

  • Understand macro-economics impacts of transfers at scale (in-flation, job creation, etc.)
  • Measure impacts over a long time horizon (e.g., [less than[sic]] 5 years)

  Status:

  • Started baseline, with long term follow up mechanisms in place
  • Not fully funded– facing a gap of ~8M

  Partners:

  • Edward Miguel, Berkeley
  • Johannes Haushofer, Princeton

  Potential impact:

  • Increase government use of CT programs
  • Increase support for our particular model in proving LT impact"

  GiveDirectly, Update for GiveWell, October 2014, Pg 13.

• 68
  • This study may include a long-term follow up component that will provide information on the impacts of cash transfers several years after the transfer: "This study will potentially include long-term follow up as well, to address a separate question raised by policymakers about the long-term impacts of cash transfers. Professor Miguel previously worked on a study of the impacts of deworming (Miguel and Kremer 2004) that involved follow-up over a period of ten years and obtained a high response rate, so he has experience in setting up effective systems for tracking study participants over time." Conversation with GiveDirectly, July 7, 2014, Pg 3
  • The study is randomized at the village level, will involve 325 villages, and is expected to survey approximately 3,900 households and 4,875 enterprises. GiveDirectly, GE study design, Pgs. 4-5. Update: "Currently in the midst of endline data collection. Plan is to finish endline data collection by the end of the year, and hope to survey over 9,000 households, 700 village elders, 80 assistant chiefs, 200 school head teachers and 3,000 enterprises." GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.
  • Baseline data collection for the study began in August 2014 and was still in progress as of September 2015. Baseline data collection was slightly slower than expected, which meant that GiveDirectly had to delay some transfers (so that researchers could complete the baseline survey before recipients had received cash).
    • GiveDirectly, GE research and measurement plan, Pg 6.
    • "The GE study in Siaya is an example of how research studies can affect GiveDirectly’s timeline. In that case, GiveDirectly moved more quickly than anticipated, and so had to delay the token transfers for some of its recipients to give the GE team enough time to conduct its baseline survey, which had to be completed before the token transfers were sent." GiveWell's non-verbatim summary of a conversation with Paul Niehaus and Carolina Toth, September 7, 2015, Pg 8.
  • Endline data collection was expected to be completed by the end of 2016, although this may be delayed since baseline data collection has taken longer than expected. GiveDirectly, GE research and measurement plan, Pg 6.
  • Midline data was scheduled to be collected from November 2014 to early 2016. GiveDirectly, GE research and measurement plan, Pg 6.
  • Paul Niehaus, GiveDirectly's President, is serving as one of the Principal Investigators on this study, along with Edward Miguel (UC Berkeley), Johannes Haushoffer (Princeton), and Michael Walker (UC Berkeley). GiveDirectly, GE study design, Pg 2, and Carolina Toth, email to GiveWell, November 10, 2015.
  • In order to mitigate potential bias from his involvement with the research, GiveDirectly has applied a number of safeguards, including preregistration of plans for measurement and analysis: "Paul Niehaus, GiveDirectly’s Co-Founder and President, will serve as a Principal Investigator on the general equilibrium effects study. To mitigate potential bias when GiveDirectly staff are involved in research on the impacts of its programs, GiveDirectly has decided on a few safeguards: the research team conducting the study will 1) preregister their plans for measurement and analysis 2) use a (non-GiveDirectly staff) third party for measurement, and 3) include academic PIs who are not involved in GiveDirectly and have a reputation for honesty." Conversation with GiveDirectly, July 7, 2014, Pgs 2-3.

• 69

  "Currently in the midst of endline data collection. Plan is to finish endline data collection by the end of the year, and hope to survey over 9,000 households, 700 village elders, 80 assistant chiefs, 200 school head teachers and 3,000 enterprises.

  Pre-analysis plan for the midline data has been filed, and are actively working on finalizing pre-analysis plans for the local leader survey data and the rest of the endline data. Analysis has begun on the midline data and is ongoing." GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.

• 70

  "Objective:

  • Measure impact of providing information on spending options
  • Measure impact of getting to choose when and how to receive cash"

  GiveDirectly, Update for GiveWell, October 2014 Pg 13

• 71
  • The main outcome of interest in this study is the rate of return on spending. Conversation with Stuart Skeates, GiveDirectly, October 20-21, 2014 (unpublished)
  • GiveDirectly is conducting the data collection for this study internally, and the analysis will be done by independent researchers.
    • GiveDirectly notes:
      • "We have replaced IPA on this project with an RA hired by ideas42 + GD field staff, for cost reasons
      • This represents our first in-house data collection, and a change in policy: whenever treatments are variations on how to do cash (rather than: does cash work at all?) we do not face a conflict of interest and can be involved in data collection"

      GiveDirectly, Update for GiveWell, July 2014

    • "Partners:
      • Anandi Mani, Warwick
      • Sendhil Mullainathan, Harvard
      • Anuj Shah, Chicago Booth"

      GiveDirectly, Update for GiveWell, October 2014, Pg 13.

  • It is fully funded by an anonymous donor. GiveDirectly, Update for GiveWell, April 2014, Pg 6, and Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015.

• 72
  • Additionally, as of May 2015, baseline results and the initial cash transfers were completed. Conversation with Stuart Skeates, GiveDirectly, October 20-21, 2014 (unpublished) and GiveDirectly, Update for GiveWell, May 2015, Pg 5.
  • GiveDirectly, Update for GiveWell, February 2016, Pg 12.
  • "Data collection complete, currently finalizing pre-analysis plan to be completed in October
    Analysis to follow with final report expected in January 2017" GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.

• 73

  "Objective:

  • Test if informal contracts can help further reduce domestic violence and improve female empowerment

  Status:

  • Small pilot, spring 2015
  • If successful, grow into a more large-scale project

  Partners:

  • Simone Schaner, Dartmouth
  • Jessica Leight, Williams"

  GiveDirectly, Update for GiveWell, October 2014, Pg 13.

• 74

  "Test if informal contracts can help further reduce domestic violence and improve female empowerment" GiveDirectly, Update for GiveWell, October 2014, Pg 13.

• 75
  • GiveDirectly, Update for GiveWell, February 2016, Pg 12.
  • "Objective is to test whether various exercises or nudges about making plans for spending the cash could end up reducing violence against women and HH conflict, and improve women's empowerment.
    A small pilot was conducted (80 HH) earlier this year. Pilot results will be analyzed and the team may look to launch a more full scale project in the next year." GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.

• 76

  "Pilot results will be analyzed and the team may look to launch a more full scale project in the next year." GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.

• 77
  • GiveDirectly, Aspirations study proposal
  • Carolina Toth, email to GiveWell, November 10, 2015
  • Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016

• 78
  • Richard Sedlmayr, conversation with GiveWell, February 19, 2016
  • "The study is currently well underway; the team has baselined nearly all participants. Some study groups have already received token transfers" GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.

• 79
  • "Expecting roughly 6-10 months more of enrollment in Aspiration villages. After that GD will send transfers to Aspirations villages over several months. The research team does treatment between the LS1 and LS2." GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.
  • Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016

• 80
  • GiveDirectly intends to enroll ~3,600 households, and endline data is expected in December 2017. GiveDirectly, Coffee study design, Pgs 3-5 (note some pages have been redacted).
  • "Census completed and enrollment of households is well underway and expected to continue through October, at which point payments will commence" GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.

• 81
  • "[Redacted] is a German foundation very aligned with effective altruism and transparency principles
  • RCT aims to study how UCTs impact recipients with access to high investmet return opportunity, in this case coffee growing, which is an industry core to [Redacted]’s mission
  • Broader goal is using [Redacted] and study to introduce model to German and UK philanthropic sectors with evangelizing partner"

  GiveDirectly, Update for GiveWell, February 2016, Pg 12.

• 82

  "We have pushed endline back to early 2018 as recent fly harvests have been poor and we want to be sure to endline after the reliable December 2017 main harvest" GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 3.

• 83

  "In early 2016, GiveDirectly allocated a fairly large sum of money to test a new implementation model in Kenya. A separate field director and team were hired approximately six months ago, and the experiment is taking place in an area close to GiveDirectly's other field operations. The primary aim is to double the program's size in 2017 without significantly increasing the management structure. The new model seeks to increase throughput per manager by eliminating the token payment and back check steps. GiveDirectly will assess gains in throughput as well as costs, which might occur in the areas of comprehension and fraud. Results and data should be available after a few months of disbursements. By the end of 2016, GiveDirectly hopes to have a blueprint for implementing a similar model in other countries." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pgs 4-5.

• 84

  "The primary aim is to double the program's size in 2017 without significantly increasing the management structure." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 4.

• 85

  "By the end of 2016, GiveDirectly hopes to have a blueprint for implementing a similar model in other countries." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 5.

• 86

  "We are actually aiming to make Rwanda’s retail program more efficient. We are making two changes to achieve this: (1) we are eliminating token payments. Instead, the first payment issued will be the first lump sum, after which we will do follow up calls to ensure proper receipt; (2) we are eliminating backcheck. In its place, we are flagging any discrepancies between census and registration for individual audits and then, on top of that, adding an additional randomized selection of HHs for audit until we achieve 40% of hhs for audit. We think this will be more efficient while still ensuring accuracy and avoiding fraud. We may eventually migrate these changes to other countries but are starting in Rwanda." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 11, 2016

• 87
  • "We conducted a randomized controlled trial (RCT) of the unconditional cash transfer program implemented by the NGO GiveDirectly in western Kenya between 2011 and 2012, in which poor rural households received unconditional cash transfers through the mobile money system M-Pesa." Pg 1.
  • "In each chosen village, GD conducted a census, usually with the help of the village
    elder, which identified all households in the village that met this targeting criterion. Among the eligible households, treatment households were chosen randomly (details are described in Section 2). Households were aware that recipients would be chosen by lottery, but the actual selection was done privately by means of random number generation." Pg 4.
  • "We are grateful to the study participants for generously giving their time; to Marie Collins, Faizan Diwan, Conor Hughes, Chaning Jang, Bena Mwongeli, Joseph Njoroge, Kenneth Okumu, James Vancel, and Matthew White for excellent research assistance; to Innovations for Poverty Action for implementation." Pg 1.

  Haushofer and Shapiro 2013 Policy Brief

• 88

  GiveDirectly, Rarieda transfer schedule, August 2013

• 89
  • "Identified and surveyed 160 girls, age 18 - 19 at baseline and 140 girls at endline
  • Randomly assigned girls living in 18 villages to control group, 9 villages to $1,000 treatment group, and 9 villages $500 treatment group"

  GiveDirectly, Final report Nike girls study, Pg 3.
  

• 90

  GiveDirectly, Nike instrument

• 91

  GiveDirectly writes that "the pilot was designed at a small scale with the expectation that it would not produce statistically robust evidence, but would provide directional learnings to guide future investment and experimentation." GiveDirectly, Final report Nike girls study, Pg 3.

• 92
  • GiveDirectly, Final report Nike girls study
  • Jean Junior, The Perspectives of Young Women in Siaya County, Kenya: Their Lives and Their Thoughts on Cash Transfer Programs

• 93
  • The grant was from USAID via the Policy Design and Evaluation Lab at UCSD.
  • Johannes Haushofer and Paul Niehaus, DIL Demonstration Proposal
  • UCSD, Policy Design and Evaluation Lab, "Tracking the Impact of GiveDirectly Transfers with Mobile Surveys in Kenya"

• 94

  Johannes Haushofer and Paul Niehaus, DIL Demonstration Proposal

• 95

  Paul Niehaus and Johannes Haushofer, Optimizing Impact for the Mobile Era - Final Report

• 96

  In treatment villages, GiveDirectly applied a "saturation" model, which included households with iron roofs and mud walls as eligible. In total, about 85% of households in saturation villages received transfers. GiveDirectly collected its standard follow up survey data, which includes questions about tension and conflict, in the 19 treatment and 18 control villages, where GiveDirectly used its standard targeting criteria. It also conducted focus groups in 3 treatment and 3 control villages to elicit opinions about targeting strategies.

  • In its standard model, GiveDirectly provides cash transfers only to the households that have thatch roofs. GiveDirectly experimented with more inclusive targeting in 19 randomly selected villages, in which nearly all households received transfers (all except those made from fully permanent materials such as cement walls and iron roofs). GiveDirectly compared these villages to 18 villages in the same region where standard targeting was applied. The factors being compared were cases of conflict/tension reported in follow-up surveys and focus groups, and instances of gaming that were discovered by GiveDirectly field staff throughout the cash transfer process. Conversation with GiveDirectly, April 8, 2014
  • “GiveDirectly: As of March 2013, GiveDirectly had received $790,000 from GiveWell donors designated as “flexible funds.” This includes a $500,000 gift from Good Ventures. The research question we are most interested in is whether providing cash transfers to all households in a village, rather than targeting the poorest households, could reduce tension and improve social outcomes of the transfer campaigns. In order to address this question, we’ve created 3 groups of randomly assigned villages for GiveDirectly’s most recent campaign in Kenya:
    • Villages in which no households will receive transfers
    • Villages in which only mud-wall and thatch-roof households will receive transfers
    • Villages in which nearly all households will receive transfers (all households with mud walls and thatch or metal roofs will receive transfers, only households with cement walls and metal roofs will be excluded)

    We are currently finishing up enrollment for this campaign, so transfers will be sent soon. We plan to collect data by administering our standard phone surveys, which include questions about tension, disagreements with neighbors, etc. We expect to receive the first round of data within the next month or two. The “flexible funds” received from GiveWell donors are going to be used for transfers to villages in group 3, including households with mud walls and either thatch or metal roofs.” Conversation with Paul Niehaus, President, and Joy Sun, COO, Domestic, GiveDirectly, August 27, 2013

  • In GiveDirectly's Kenya 2M campaign, "saturation villages" are villages in which all households with mud walls and thatch or iron roofs will receive transfers. These households make up about 85% of the households in saturation villages. The other 15% of households are built with all man-made materials (e.g., cement walls and iron roof) and will not receive transfers. Conversation with Carolina Toth, Field Director, GiveDirectly, October 24, 2013

• 97
  • "Quantitative analysis is inconclusive: some metrics favorable in saturation villages (e.g. fewer overall complaints), others less favorable (e.g. larger % asked to pay a bribe)
  • FGDs suggests that conflict levels were low across both categories of villages (mainly rumors and awkwardness) and that when faced with the choice we have to make, people prioritize the poorest, who they feel are more deserving"

  GiveDirectly, Saturation analysis, Pg 1.

• 98
  • GiveDirectly, Update for GiveWell, July 2014, Pg 11.
  • "In Kenya, GiveDirectly experimented with a community-based targeting process, whereby residents gave input on households that they felt were deserving of transfers but had been excluded by GiveDirectly’s criteria. GiveDirectly felt that to do this process well required significant resources (staff time) and that the benefits were not worth the costs. In addition, some of the villages involved in this experiment gave feedback that they would prefer for GiveDirectly to make the decisions about targeting." Conversation with GiveDirectly field staff, October 20-21, 2014, Pg 4.

• 99
  • "Index insurance for individuals builds resilience and increases investment for
    stallholder farmers in a more effective way than traditional insurance
    • Remote triggers for payout (e.g., measurements at rain stations) prevent moral hazard and decrease cost of determining payout vs. traditional insurance
    • Index-insured households in Ghana significantly increased investment in
      agricultural inputs vs. control (Karlan et al 2013)" GiveDirectly, Rockefeller index insurance update, July 2015, Pg 3.
  • "GiveDirectly & Rockefeller co-created an innovative approach to index insurance delivery, embedding it into social protection programs already at massive scale… Governments could embed index insurance into these programs by allowing beneficiaries to choose to exchange part of their status-quo benefits for insurance." GiveDirectly, Rockefeller index insurance update, July 2015, Pg 4.

• 100

  "GiveDirectly constructed a cash transfer program simulating Gov’t of Kenya’s flagship Hunger Safety Net Program (HSNP)

  • ~$100 in two tranches, two months apart
  • GiveDirectly negotiated a 100-day index-based crop insurance policy with a local commercial insurer (APA) based on standard, pre-existing offerings
  • Beneficiaries of the cash transfer program were given a choice of how much of benefit to receive as cash v.s. insurance premiums
  • Neutrally framed as a choice of benefits (v.s. a purchase decision)
  • Priced in 0.1 acre increments
  • Half offered approximately actuarial price (200 Ksh / 0.1 acre), half offered 50% subsidy (100 Ksh / 0.1 acre)”

  GiveDirectly, Rockefeller index insurance update, July 2015, Pg 5. Note that this study and associated cash transfers were fully funded by the Rockefeller Foundation. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 101
  • "Take-up rates, even without subsidy, were higher than in other subsidized schemes." GiveDirectly, Rockefeller index insurance update, July 2015, Pg 8.
  • "Average acquisition costs were 75% lower than commercial insurer’s, and incremental costs were 94% lower." GiveDirectly, Rockefeller index insurance update, July 2015, Pg 7.
  • Note that we have only seen a summary of the results.

• 102
  • GiveDirectly purchased palm readers, collected palm prints during registration, and used palm scans as an additional identification measure during recipients' first cash out day; it also collected information on the level of comfort that recipients feel towards biometrics during its follow-up surveys. GiveDirectly, Update for GiveWell, September 2015, Pg 10, and GiveWell, GiveDirectly follow up surveys summary - Uganda, September 2015
  • Additional context from our 2014 review: "The government of Uganda started a large cash transfer program called Social Assistance Grants for Empowerment (SAGE), which provides $20 monthly transfers to eligible people in Uganda. The program currently serves 100,000 households in 17 districts and has plans to scale up; it is not currently active in [Bukedea], where GiveDirectly operates. The government of Uganda is working with the mobile money provider MTN to build the capability to use biometric authentication (fingerprinting) for transactions and account access. GiveDirectly is interested in running a pilot of biometric authentication with its own cash transfer recipients who are serviced by MTN." Conversation with GiveDirectly, April 8, 2014, Pg 5.

• 103

  "Biometrics: GD likely to use in contexts where national IDs are not present, paydays are necessary" GiveDirectly, Update for GiveWell, September 2015, Pg 10.

• 104
  • "Objective: Build expertise in a set of targeting competencies to rapidly identify poor locations and households across diverse settings, and structure an evaluation framework for targeting effectiveness
    Approach: Desk research followed by field piloting for most promising household targeting methods
    Geography: Homa Bay, where thatch is uncommon and an overly-restrictive proxy means test, and so change is already needed" GiveDirectly, Update for GiveWell, September 2015, Pg 9.
  • Recipients from approximately 55 villages were enrolled using several different targeting methods. Criteria being tested include being a widow, being an orphan, community based targeting, and subjective judgments about poverty level.
    • GiveDirectly, Update for GiveWell, September 2015, Pg 9 and Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015.
    • GiveDirectly, Uganda 2M campaign enrollment database (Note: GiveDirectly gave permission to publish this database on the condition that all personally identifying information was removed. Due to the size of the database, we have not yet anonymized it and therefore have not published it. If you are interested in the content of this database, please contact us at info@givewell.org.)

• 105
  • "Goals for 2016…GiveDirectly hopes to launch a test of a program that would guarantee a basic income to participants over the course of several years." Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016, Pg 1.
  • "This study could use the same control-group villages as the GE study." Conversation with GiveDirectly, September 5, 2014, Pg 6.
  • "The organization that we founded, GiveDirectly, has decided to try to permanently end extreme poverty across dozens of villages and thousands of people in Kenya by guaranteeing them an ongoing income high enough to meet their basic needs—a universal basic income, or basic income guarantee." Michael Faye and Paul Niehaus, Slate article, April 14, 2016
  • Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016
  • GiveDirectly, Blog post, September 22, 2016

• 106
  • As of September 2016, GiveDirectly had raised slightly more than $21 million. It would like to raise $30 million for the study. Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016
  • Sample size
    • "To do so, we’re planning to provide at least 6,000 Kenyans with a basic income for 10 to 15 years." Michael Faye and Paul Niehaus, Slate article, April 14, 2016
    • Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016 - noted that the study was expected to include ~12,000 recipients.
    • The study will cover 342 villages (including control) GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 2.
    • Ian Bassin, edits to GiveWell's review, November 10, 2016
  • Eligibility criteria
    • GiveDirectly will attempt to enroll everyone in the village. GiveDirectly wants its study to be as helpful to policymakers as possible, and policymakers interested in basic income programs are especially interested in universal programs. Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016
    • "Adult individuals, not households, are the relevant unit for GiveDirectly for all arms (i.e., each adult in a household has their own line)
      All full-time resident adults (>=18) will receive payments
      All individuals >=15 years old will begin to receive payments once they turn 18 for whatever remains of the payment period (e.g., 9 years for a 15 year old)
      Migration: Eligible recipients who migrate out of the village will continue to receive. Recipients that migrate into the village after enrollment has begun are not eligible to receive." GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 2.
  • Arms of the study
    • There will be four arms: control (100 villages), 12 years of basic income received monthly (41 villages), 2 years of basic income received monthly (95 villages), and 2 years of basic income received in "lump sums" like in GiveDirectly's standard model (106 villages). GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 2.
  • Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016
  • "The overall scale of the project remains incredibly ambitious: if fully funded, it will deliver transfers to over 26,000 individuals, including 6,000 receiving the full 12-year basic income transfers. This breaks down into 40 villages in the first treatment arm and 80 each in the other two, with roughly 120 to 150 individuals per village. Given this scale, we have reasonably good odds of detecting impacts not only on individuals, but also on village-level markets." GiveDirectly, Blog post, September 22, 2016

• 107
  • "The group is still finalizing details, but the payment is expected to be about $0.70 to $1.10 per person per day." Dylan Matthews, Vox article, April 15, 2016
  • "Snaswa: What is the minimum amount of cash that a household/individual can receive as Basic Income in the pilot study?... Paul Niehaus: "we're finalizing, looking at around $0.75 nominal = $1.50 PPP" Paul Niehaus, AMA on Reddit, May 31, 2016
  • Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016
  • GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 2.

• 108

  Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 109
  • Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016
  • Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016

• 110

  "We think the direct impact of the UBI program will be about ~60% as effective as the lump sum program, but that the potential policy impact more than makes up for that difference" GiveDirectly, Update for GiveWell on experimentation, September 2016, Pg 2.

• 111
  • GiveDirectly, Update for GiveWell, July 2014, Pg 5.
  • This was deprioritized for 2015: "Urban pilot –limited interest expressed by inst. donors, govt, private donors in urban" GiveDirectly, Update for GiveWell, February 2015, Pg 20.

• 112

  GiveDirectly, Update for GiveWell, July 2014, Pg 5.

• 113

  Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 114

  Cash transfers currently fund private goods. GiveDirectly is considering experimenting with a system whereby transfer recipients could propose public goods projects and individuals could pool their resources to fund projects they consider worthwhile. Conversation with GiveDirectly, September 5, 2014, Pg 5.

• 115

  Some of GiveDirectly's potential partnership projects (note that these projects are small compared to the projects GiveDirectly is currently interested in):

  • Uganda District: Request from an MP to discuss working in his district GiveDirectly, Check in with GiveWell, September 2014, Pg. 5. Update: "GD declined request to work in Uganda's [Redacted] district - no obvious opportunity for policy impact to offset relocation costs" GiveDirectly, Update for GiveWell, September 2015, Pg 6.
  • GiveDirectly has spoken with researchers in a multilateral-funded program to which the partners were considering adding unconditional cash transfers. Conversation with Piali Mukhopadhyay, GiveDirectly, October 20-21, 2014 (unpublished). Update: GiveDirectly declined to participate in this program despite high potential policy impact, because a) the program was located in an area GiveDirectly had never worked and GiveDirectly did not have enough time to run pilots of cash transfers before making its decision and b) GiveDirectly was worried that study results would not be rigorous due to low sample size for the cash transfer arm. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • Multilateral agency: GiveDirectly has also spoken with another agency, which is considering using cash transfers as a benchmark to assess the impacts of one of its nutrition programs. Conversation with Piali Mukhopadhyay, GiveDirectly, October 20-21, 2014 (unpublished). Update: "GD declined to pursue implementation of nutrition benchmarking study, but will provide advice." GiveDirectly, Update for GiveWell, September 2015, Pg 6.
  • Haiti evaluation: GiveDirectly is discussing experimental impact evaluation in Haiti to assess how cash performs relative to other reconstruction projects there. GiveDirectly, Update for GiveWell, September 2015, Pg 6.
  • Evaluation non-profit benchmarking initiative: GiveDirectly spoke with a representative at an evaluation non-profit about adding cash transfer arms to some of the non-profit's studies. A survey of 31 ongoing studies found that 6 have a cash arm currently and 20 would like to add one. GiveDirectly does not plan to be the implementing partner for these studies because there are often implementing partners that are already set up who could implement the cash transfers if they had enough funding. But GiveDirectly is spreading awareness of this opportunity through its network. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 116

  He was primarily focused on Rwanda (see below) and replicating the Rwanda model.

  • Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • Carolina Toth, email to GiveWell, November 10, 2015

• 117
  • Funding for the project came from an institutional funder and Google.org; each provided $2 million. Paul Niehaus and Carolina Toth, conversation with GiveWell, May 28, 2015
  • GiveDirectly, Rwanda technical application
  • @Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, August 12, 2016@

• 118
  • The project partners are still in the process of designing the project; they expect to have plans for the RCT ready by mid-November 2015. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • GiveDirectly, Rwanda technical application
  • A pilot is currently in progress and implementation is expected to begin in 2017. GiveDirectly, Update for GiveWell, February 2016
  • @Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, August 12, 2016@

• 119

  @Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, August 12, 2016@

• 120
  • Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016
  • "GiveDirectly has an agreement with a partner that would fall into these categories and could result in GiveDirectly running projects in up to 4 new countries." GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, August 12, 2016, Pg 4

• 121
  • Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016
  • GiveDirectly plans to make available up to $15 million of the grant it received from Good Ventures in 2015 to match funds committed by the major funder.
    • Email conversation with anonymous funder, May 2016
    • Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016
    • The $15 million that each has offered is not committed: "GiveDirectly is offering up to $15 million in matching funds on a first-come, first-served basis to fund these types of projects. GiveDirectly hopes that offering the matching funding will incentivize large funders to move quickly on partnership projects. If all of GiveDirectly’s partnership projects discussions came through, GiveDirectly would need to contribute $23-30 million. While GiveDirectly cannot fund all of the projects it is currently discussing, it would consider talking to partners about opportunities to fund any projects approved by these potential funders that are beyond GiveDirectly's funding limit." GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, August 12, 2016, Pg 5.
    • "GiveDirectly has an agreement with a partner that would fall into these categories and could result in GiveDirectly running projects in up to 4 new countries." GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, August 12, 2016, Pg 4.

• 122
  • Location: The benchmarking projects could occur in multiple different countries. GiveDirectly may decline to work in countries that it does not wish to enter. GiveDirectly intends to choose which countries it will work in according to:
    • How logistically simple it is to work in the country
    • How much working in the country would increase GiveDirectly's geographic diversity. GiveDirectly would like to test its program outside of East Africa.
    • How quickly GiveDirectly will be able to set up operations in the country. The easiest countries for GiveDirectly to set up in will likely be those in East Africa.
    • Whether or not GiveDirectly can find an on-the-ground staff member who is willing to push forward the benchmarking project. GiveDirectly's experience in Rwanda has taught it that having such a "champion" can be invaluable.
  • Process
    • According to its agreement with the institutional funder, GiveDirectly is not allowed to reach out to country-level staff to gauge their interest in a benchmarking program. Rather, first the staff must express interest in a partnership with GiveDirectly.
    • Once interest is expressed, GiveDirectly has a high-level conversation with the country staff.
    • If that conversation goes well, GiveDirectly will be invited to submit a proposal for a benchmarking project to the country staff.
    • GiveDirectly and the country staff then negotiate the details of the proposal and sign a contract. In Rwanda, GiveDirectly signed the contract while it still left many details to be worked out later, and this caused difficulties down the road. GiveDirectly now intends to bring up many details about the program during the negotiation stage.
    • Once the contract is signed, GiveDirectly and the country staff can enter the implementation phase.

  @Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, August 12, 2016@

• 123
  • As of August 2016, conversations had been initiated with approximately 8 countries. GiveDirectly thinks it is unrealistic to expect any contracts to be signed by the end of the year. Ideally, GiveDirectly would like to roll out the benchmarking projects in a sequential way to avoid stretching its management capacity too thin. @Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, August 12, 2016@
  • "One of GiveDirectly's strategic goals for 2016 is to finalize these negotiations, which it believes would lead to two new countries it will work in. It aims to have tentative agreements in place by the end of the year. GiveDirectly aims to balance its priorities between moving forward quickly with new projects and rolling them out sequentially, which is easier from a management standpoint. Starting and scaling up projects in 4 new countries at once would be a significant challenge." GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, August 12, 2016, Pgs 4-5.

• 124

  "GiveDirectly is offering up to $15 million in matching funds on a first-come, first-served basis to fund these types of projects. GiveDirectly hopes that offering the matching funding will incentivize large funders to move quickly on partnership projects. If all of GiveDirectly’s partnership projects discussions came through, GiveDirectly would need to contribute $23-30 million. While GiveDirectly cannot fund all of the projects it is currently discussing, it would consider talking to partners about opportunities to fund any projects approved by these potential funders that are beyond GiveDirectly's funding limit.

  One of the large funders is unlikely to begin providing funding until early 2017. This funder and GiveDirectly would each contribute roughly $7 million to a project, for a rough total of $14 million. It is possible, though unlikely, that GiveDirectly could get funding from this funder by the end of 2016. (Update: As of October 2016, GiveDirectly did not expect this project to happen.) GiveDirectly has had exploratory talks with another large funder."

  GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, August 12, 2016, Pg 5.

• 125

  "GiveDirectly is offering up to $15 million in matching funds on a first-come, first-served basis to fund these types of projects. GiveDirectly hopes that offering the matching funding will incentivize large funders to move quickly on partnership projects. If all of GiveDirectly’s partnership projects discussions came through, GiveDirectly would need to contribute $23-30 million. While GiveDirectly cannot fund all of the projects it is currently discussing, it would consider talking to partners about opportunities to fund any projects approved by these potential funders that are beyond GiveDirectly's funding limit." GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, August 12, 2016, Pg 5.

• 126
  • "In practice, we do not think there is much scope for additional senior leadership time devoted to fieldwork to materially improve the quality or efficiency of execution there." GiveDirectly, Update for GiveWell, September 2015, Pg 14.
  • Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 127

  @Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, August 12, 2016@

• 128

  For example, see GiveDirectly's intended use of Good Venture's $25 million grant here and here.

• 129
  • We are unable to be more specific because GiveDirectly would prefer to keep the particular inputs the algorithm uses confidential, to make it harder for people to game the eligibility requirements.
  • Carolina Toth, email to GiveWell, October 20, 2015

• 130
  • Carolina Toth, conversation with GiveWell, November 12, 2015
  • "A full set of asset questions are oftentimes only relevant when we are re-assessing poverty of a particular region, or assessing it for the first time when we enter a new region. In addition to assets, we may consider other factors such as housing materials, facilities (latrine, roof, floor) etc. The exact set of factors considered changes across communities to reflect the complex variations of poverty. We also continuously test and tweak our set of criteria based on our analysis. This is why you may see different sets of information (assets/facilities etc.) collected across different geographies." GiveDirectly staff, responses to monitoring questions, October 11, 2016, pg. 1.

• 131

  "We are currently in the process of developing a targeting criteria for Rwanda. For the first 500 households, we are using PPI but also collecting asset information and then will use the data we gather to determine an eligibility criteria." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 11, 2016

• 132
  • "We typically use building materials as eligibility criterion—organic materials like a thatched roof, mud walls, or mud floor have the advantage of being (a) a strong predictor of poverty, (b) easy for community members to understand, and (c) relatively easy to audit in a number of ways, including both digital imagery captured by our field staff and satellite imagery captured remotely." GiveDirectly, FAQs 2015
  • GiveDirectly has tweaked these criteria in the past, e.g., "Dropped mud walls as eligibility requirement." GiveDirectly, Update for GiveWell, October 2014
  • Carolina Toth, email to GiveWell, November 10, 2015

• 133

  Carolina Toth, conversation with GiveWell, November 12, 2015

• 134
  • "In Siaya, we also honed our process of targeting households that are most in need. In a new region, our old ways of targeting may not work. For example, in Homa Bay there is a scarcity of grass, so identifying the poorest families using thatched roofs may be less effective." GiveDirectly, Blog post, August 25, 2015
  • "Geography: Homa Bay, where thatch is uncommon and an overly-restrictive proxy means test, and so change is already needed" GiveDirectly, Update for GiveWell, September 2015, Pg 9.
  • "The ‘thatch roof, mud walls, mud floor’ eligibility criteria was not going to work in Homa Bay, as <3% households had thatch roofs" Carolina Toth, email to GiveWell, October 20, 2015

• 135
  • [GiveDirectly] is also considering modifying its targeting criteria to include certain types of people who may be especially vulnerable whether or not they live under an iron roof.
    Conversation with GiveDirectly field staff, October 20-21, 2014
  • GiveDirectly is considering expanding its eligibility criteria to include:
    • Widows living in iron-roofed houses
    • Houses with iron roofs that are severely corroded
    • Households with partially cemented floors

    Conversation with GiveDirectly, October 6, 2014

  • "Methods being piloted
    • Community based targeting (multiple variants)
    • Subjective rankings (by staff, external parties)
    • Proxy means tests
      • PPI (Progress out of Poverty Index)
      • MPI (Multidimensional Poverty Index)
      • Other additional proxies (e.g., widows)"

    GiveDirectly, Update for GiveWell, September 2015, Pg 9.

  • "We've now completed our targeting pilots in Kenya, and have selected a new targeting criteria for Homa Bay." Carolina Toth, email to GiveWell, October 20, 2015

• 136
  • Carolina Toth, email to GiveWell, October 20, 2015
  • "In the end, balancing our four criteria, we decided to use an algorithm that takes into account several factors, including housing (e.g. house size), assets (e.g. presence of a latrine), vulnerable recipient status (e.g. homelessness), and other criteria." GiveDirectly, Blog post, January 21, 2016

• 137
  • To test possible proxies for poverty to use as its new criteria, GiveDirectly attempted to determine the validity and replicability for each metric, and also solicited community feedback (more detail on GiveDirectly's process in the footnote).
    • Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
    • "Additional detail on process and excel file context:
      1. Conducted desk research on most promising targeting methods
      2. Piloted MPI, PPI, numerous types of subjective assessment, numerous types of CBT, different proxies, and blends of these different approaches across 50 villages in Homa Bay
      3. Collected recipient and non-recipient feedback after token transfers were sent (See “Targeting Focus Group results”). The focus groups were done on three of the most successful and widely used criteria: [redacted]
      4. Collected consumption data as part of our household census for ~500 households in Homa Bay (see “Consumption data for targeting work” which includes the most complete survey versions)—the monthly per-capita consumption figure gathered from this was the measure of each household’s poverty.
      5. Analyzed consumption data to see if there were strong predictors of poverty among household’s observable characteristics. Requested analysis and advice of several data scientist volunteers. Unfortunately, there was no single, strong predictor of poverty like thatch.
      6. Developed multiple possible criteria (i.e. models) (See “Targeting criteria analysis summary”) based on the piloting and analysis experience, using factors that were the strongest predictors of poverty, fair, and difficult to game.
      7. Selected one of the models based upon our priorities of accuracy, perceived fairness, lack of gameability, and cost."
      GiveDirectly, Targeting process overview
    • "To inform and structure this decision, we tested out several different targeting methods and eligibility criteria to evaluate their pros and cons, and the circumstances in which we thought they would be most and least useful. For example, we tested: a variety of proxies, such as per-capita housing space and housing materials; community-based targeting, where members of a village nominate, through various means, which they think are the poorest households; points-based systems such as the Progress out of Poverty Index (PPI) and the Multidimensional Poverty Index (MPI); subjective assessments like our field officers rating on a 1-5 or 1-10 scale the poverty of a household or the quality of their house; and various blends of these different approaches." GiveDirectly, Blog post, January 21, 2016
    • GiveDirectly put together focus groups to solicit feedback about the criteria it was testing, asking questions about how well GiveDirectly did at identifying the poorest people in the community and whether or not GiveDirectly missed particularly poor households or included not-poor households. GiveDirectly, Targeting focus group results and GiveDirectly, Targeting criteria analysis summary
  • For example, GiveDirectly tested the same criterion on the same group of people at different times to see if respondents gave consistent answers that led to the same group of eligible recipients each time.
    • GiveDirectly gave an example of one inconsistent criteria model that it tested: some versions of community-based-targeting were inconsistent. GiveDirectly found that in some community-based targeting models, unique community groups would rate the same family's level of poverty quite differently; while one community group would claim the family was extremely poor, another would tell GiveDirectly that the family was comparatively wealthy. These discrepancies led to GiveDirectly discarding several community targeting models as eligibility models. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 138
  • Carolina Toth, conversation with GiveWell, November 12, 2015
  • Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 11, 2016
  • "A full set of asset questions are oftentimes only relevant when we are re-assessing poverty of a particular region, or assessing it for the first time when we enter a new region. In addition to assets, we may consider other factors such as housing materials, facilities (latrine, roof, floor) etc. The exact set of factors considered changes across communities to reflect the complex variations of poverty. We also continuously test and tweak our set of criteria based on our analysis. This is why you may see different sets of information (assets/facilities etc.) collected across different geographies." GiveDirectly staff, responses to monitoring questions, October 11, 2016, pg. 1.

• 139
  • GiveDirectly, Eligibility criteria presentation
  • Carolina Toth, email to GiveWell, October 20, 2015
  • "We wanted to evaluate how well different techniques worked along several dimensions. We looked for accuracy (did the method actually identify poor households), perceived fairness (did community members think that the method was fair), gameability (how easy was it to cheat the system), and cost (how expensive was the method to administer). We will write more about the targeting project on our blog over the coming months.

    Besides testing and evaluating different techniques, the project allowed us to settle on usable eligibility criteria for our new home in Homa Bay. We found that there was no single, objective replacement for thatched roofs as a criterion (e.g. mud floors or the presence of a latrine) in our new location that also met the bar for accuracy, perceived fairness, gameability, and cost. We also found that some methods that might work well in other contexts didn’t work well in Homa Bay. For example, community-based targeting was perceived as fair and was cost-effective, but it was not particularly accurate, in part due to the high prevalence of clanism in Homa Bay – villagers were sometimes nominating those in their own clan over the poorest households in the community."
    GiveDirectly, Blog post, January 21, 2016

• 140
  • "There are major differences between this method and the thatch and homeless criteria we used in Siaya. Because the new criteria take into account many factors, it’s harder to game the system. Also, the new criteria include a broader range of vulnerable statuses than the criteria in Siaya by adding widows and child-headed households into the algorithm. This is likely to increase the perceived fairness of our eligibility criteria. The new criteria also have some weaknesses: it’s slightly more expensive from an operational standpoint, because the criteria involve more questions, and it is more difficult to explain to community members why a given household was or was not eligible. This may counteract the perceived fairness of enrolling more vulnerable-status recipients.
    Taking all of these dimensions into account, our new eligibility criteria will allow us to identify and serve the poorest of the poor in Homa Bay. Even though the new criteria make this system slightly more expensive, that cost will likely be mitigated by a decrease in gameability and an increase in accuracy. And although the new algorithm may be perceived as harder to understand, we hope that by accounting for vulnerable-status groups the criteria will be perceived as more fair in the communities where we work."
    GiveDirectly, Blog post, January 21, 2016
  • "Major differences between this criteria in Kendu vs thatch criteria in Siaya
    Pro: harder for all parties to game, as algorithm is not known
    Pro: includes vulnerable groups important to fairness perceptions
    Con: more expensive to administer as there are more questions to answer. Cost savings from reduced gaming may offset this.
    Con: difficult to explain to communities why someone was or was not included, decreasing fairness perception
    Taking all of these: we think cost and fairness differences are probably going to be about the same, and so the major gain is the fact that it is difficult to game" Carolina Toth, email to GiveWell, October 20, 2015.

• 141
  • "Pro: harder for all parties to game, as algorithm is not known" Carolina Toth, email to GiveWell, October 20, 2015.
  • "The new criteria also have some weaknesses: it’s slightly more expensive from an operational standpoint, because the criteria involve more questions, and it is more difficult to explain to community members why a given household was or was not eligible. This may counteract the perceived fairness of enrolling more vulnerable-status recipients." GiveDirectly, Blog post, January 21, 2016

• 142

  "And although the new algorithm may be perceived as harder to understand, we hope that by accounting for vulnerable-status groups the criteria will be perceived as more fair in the communities where we work." GiveDirectly, Blog post, January 21, 2016

• 143

  We asked GiveDirectly if it has heard of any increased conflict in locations where they new criteria are being used, and GiveDirectly noted: "Our field directors have not seen or heard any evidence of this." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 8, 2016

• 144
  • @GiveDirectly, Survey for Randomized Controlled Trial@
  • GiveDirectly, Offering Memorandum (January 2012), Pgs 23-24.

• 145

  GiveDirectly, What We Do - Operating Model, see the Uganda tab.

• 146

  GiveDirectly, Offering Memorandum (January 2012), Pg 25.
  

• 147

  Haushofer and Shapiro 2013 Policy Brief

• 148
  • The data reported for this variable in Table 4 of Haushofer and Shapiro 2013 Policy Brief, Pg 32, appears to demonstrate that 21% (not 20%) of the control group reports that not all household members usually eat until they are content.
  • Haushofer and Shapiro 2013 Policy Brief, Pg 18, corresponding data in Table 4, Pg 32.

• 149
  • "Majority of households are widows, almost half have one family member that is disabled or very sick.”
  • ”Households seem to be as or more needy than typical GD recipients"

  GiveDirectly, Update for GiveWell, July 2014, Pg 11

• 150
  • "Some (but not all) mabati [iron-roofed] or even permanent HH are as deserving as thatched HH […] 6/6 groups mentioned deserving special cases." GiveDirectly, Saturation analysis, Pg 4.
  • Mr. Ekeu thinks that roofs are too rough a way to target poverty because some people may live under an iron roof but actually be very poor (e.g., someone who inherited an iron-roofed house from his grandfather, or a widow whose late husband built her an iron-roofed house long ago). Ms. Mukhopadhyay said that GiveDirectly hears this kind of feedback from a lot of its field staff, but believes that building materials are still a good criteria on average.
    Conversation with GiveDirectly field staff, October 20-21, 2014

• 151

  Mr. Ekeu [the Senior Field Officer in Uganda] thinks that roofs are too rough a way to target poverty because some people may live under an iron roof but actually be very poor (e.g., someone who inherited an iron-roofed house from his grandfather, or a widow whose late husband built her an iron-roofed house long ago). Ms. Mukhopadhyay said that GiveDirectly hears this kind of feedback from a lot of its field staff, but believes that building materials are still a good criteria on average. It is also considering modifying its targeting criteria to include certain types of people who may be especially vulnerable whether or not they live under an iron roof. Conversation with GiveDirectly field staff, October 20-21, 2014, Pg 4.

• 152

  GiveDirectly, Consumption data for targeting work

• 153

  Our impression is that the average consumptions were calculated based on surveys where recipients were asked to report what they had spent in the last year in a number of categories. These amounts were summed, then divided by 12 and the number of people per household to obtain the monthly consumption per person. We believe that self-reported responses, especially about what has been spent in the last year (which is difficult to recall), are unlikely to be highly accurate. GiveDirectly, Consumption data for targeting work

• 154

  Paul Niehaus, GiveDirectly Founder, email to GiveWell, November 20, 2012.

• 155
  • GiveDirectly, Kenya hotline log, July 2013
  • GiveDirectly, Follow-up tracker, July 2013
  • GiveDirectly, Follow-up tracker, October 2014

• 156
  • One adverse event reported in the Kenya follow-up tracker involved a case of domestic abuse resulting in the death of the mother and child, where the particular instance of conflict may have been related to the use of transfer funds. When GiveDirectly investigated the event, the parents of the deceased expressed that there was not anything GiveDirectly could have done to prevent this adverse event. People in the community, including the parents of the deceased, also said that they did not want to report the incident because they were afraid that it would cause GiveDirectly to cancel the cash transfer program, which they did not want to happen. Conversation with GiveDirectly, July 7, 2014, Pg 5.
  • "During this process there were some reports of problems during paydays - recipients were hesitant to come forward initially."Carolina Toth, GiveDirectly, email to GiveWell, September 12, 2014

• 157

  Ian Bassin, edits to GiveWell's review, November 10, 2016

• 158

  We asked the field officers what they think about the current transfer size ($1000), and whether they’d choose to keep it at that level, increase it, or decrease it, given the effects that an adjustment would have on how many people GiveDirectly would be able to serve.

  • Mr. Okello: Typically there are multiple households on one compound, each inhabited by relatives of the same family, and any household that meets the targeting criteria can receive transfers. Mr. Okello said that it may make more sense for GiveDirectly to group some households on a compound together so that transfers are shared across them, rather than each eligible household receiving the full $1000.

    Mr. Okello also said that if GiveDirectly increased the size of the transfers, that could create a high level of dependency. One of the messages that field officers send is that people should use the $1000 transfers to develop themselves as much as possible, but if someone knew they were getting $2000, they may stop farming, for example. With $1000 people can get some things but not everything; it is the right amount.

  • Mr. Ekeu: Mr. Ekeu prefers reducing the amount of money in each transfer and expanding the recipient base to reach everyone in the village. He said that the current targeting model causes bragging and unrest in the communities. The people who don’t benefit may be brought to use force to get some of the money, such as by breaking into recipients’ homes. Mr. Ekeu suggested that it would be better for GiveDirectly to provide all households in a village with some amount of money, even if it was less for households that are currently deemed ineligible (e.g., $100). This way, each of the households would be busy figuring out how they would spend their own money rather than how to get money from another.
  • Mr. Olinga: Mr. Olinga said that to reduce extreme poverty the bigger transfer is better, but he didn't have a strong opinion on $1000 transfers to some people versus $500 transfers to twice as many. [This is how we posed the question to Mr. Olinga.]

  Conversation with GiveDirectly field staff, October 20-21, 2014, Pg 5.

• 159
  • Several recipient households had been selected by GiveDirectly for our visit as representative of how recipients use funds. 2 locations (and the households within them) were selected as a function of GiveDirectly's activities that day -- an area of Rarieda where the end-line survey for the RCT was being implemented and an area of Siaya where enrollment was being undertaken. In both cases, we don't know whether enrollment and surveying activities were taking place elsewhere that day which would have given GiveDirectly discretion in choosing these areas.
  • On the final day of our visit, we asked GiveDirectly whether we could randomly select 3 households to visit. GiveDirectly sent us a list of 15 households in a location in Rarieda where the end-line survey for the RCT was complete and therefore we could question recipients without interfering with the RCT. We don't know whether GiveDirectly had discretion in choosing these 15 households. We selected 5 households from the list using Excel's RAND() function and visited 3 of them. (GiveDirectly made appointments with the households in advance and could not reach 2 of them.)
  • We would characterize all the households we visited -- those that GiveDirectly fully selected for us, those over which GiveDirectly had less discretion, and those we selected randomly -- as extremely poor. We did not see any significant differences in wealth between them.

• 160
  • Most homes are made up of three rooms.
  • The main room is a sitting area. In the homes we visited, this room varied in size from approximately 10'x12' to 12'x20'. This room has 3 doors: one to the outside; one to what appeared to be a storage room; one to a bedroom. The husband and wife sleep in the bedroom and the children sleep in the storage room or in the kitchen.
  • The kitchen is often a separate structure, most often thatched-roof (even for homes that have tin roofs). Some households have no kitchen structure and cook outside. Others have a small kitchen in place of a storage room.
  • There are no doors in between rooms in the house, just hung curtains.
  • The living room has many chairs and couches for sitting. They often almost fully cover the wall area (aside from doors). There are also coffee tables in the middle of the room. Poorer homes had less furniture; one home we saw had a single chair and a single broken table.
  • People have wall hangings for decoration. The most common hanging we saw was old calendars (e.g., from 2003) that have pictures and can be used for decoration.
  • Most houses had 1-2 kerosene lamps that provide light since they don't have electricity. One home (a non-recipient we visited) had two electric lights, which were powered with a solar panel.
  • People we spoke with reported walking 5-20 minutes daily to obtain drinking water, which one recipient reported doing 1-2x per day.
  • Most households owned a bicycle.
  • Some households have radios (both of the non-recipients we saw had radios; one had a TV). Of the others, 3/14 had radios. These were most often powered with what looks like a car battery.
  • Most people owned one cell phone pre-GiveDirectly.
  • Households tend to own some livestock. Most commonly, we saw 1-2 cows and 4-5 chickens each. They reported selling the milk or saving it for personal use, and many mentioned being able to sell their cows in the future to pay for their kids to attend secondary school.

• 161
  • @GiveDirectly, Siaya Enrollment Database@
  • GiveDirectly, Google enrollment database
  • GiveDirectly, Nike enrollment database
  • GiveDirectly, RCT Enrollment Database
  • GiveDirectly, Kenya 2M enrollment database, September 2013
  • GiveDirectly, Uganda pilot enrollment database - Kosile
  • GiveDirectly, Uganda pilot enrollment database - Kawo
  • GiveDirectly, Uganda pilot enrollment database - Kanyamutamu
  • GiveDirectly, Uganda pilot enrollment database - Akumure
  • GiveDirectly, Kenya 1.2M enrollment database
  • GiveDirectly gave permission to publish the following databases on the condition that all personally identifying information was removed. Due to the size of the databases, we have not yet anonymized them and therefore have not published them. If you are interested in the content of these databases, please contact us at info@givewell.org.
    • GiveDirectly, Kenya rolling campaign enrollment database - Homa Bay
    • GiveDirectly, Kenya rolling campaign enrollment database - Siaya
    • GiveDirectly, Uganda 2M campaign enrollment database
  • GiveWell, spot checks of Segovia registration sample 2016
  • GiveDirectly staff, responses to monitoring questions, October 11, 2016

• 162

  GiveDirectly's procedure for identifying households to audit:

  • GiveDirectly collects information about recipients during the first three stages of a campaign: census, registration, and back check. Some information, such as recipient name, GPS location, housing materials, and identifying photograph, is collected at more than one stage and then checked for mismatches. (These checks are currently conducted using Excel but will eventually be automated through Segovia technology. One exception is comparing identifying photographs of recipients, which is done using Mechanical Turk.)
  • Each mismatch in recipient information is assigned a certain weight depending on how likely it is to be an indication of gaming. (GiveDirectly said that it determined the likelihoods of various mismatches indicating gaming by conducting an analysis of the mismatches present in past cases of gaming. We did not review this analysis.) GiveDirectly said that the mismatches with the highest weights are mismatches in identifying photographs and housing materials.
  • Each recipient is assigned a total mismatch "score" (the composite of all their weighted mismatches). Recipients with scores above a certain level are audited.

  Conversation with GiveDirectly field staff, October 20-21, 2014

• 163

  GiveDirectly, What We Do - Operating Model, October 2016

• 164

  However, they seem to roughly match other data we have seen. Between September 2013 and July 2015, 3.5% of recipients initially eligible after registration were found to be ineligible after the back check or audit stage.

  • See GiveDirectly, Eligibility check, which we downloaded from GiveDirectly's website on September 14, 2015. GiveDirectly, What We Do - Operating Model, Overview tab. Note that the document does not include all households that GiveDirectly has ever enrolled (e.g. there is one household from 2012 and only a sample from the other years).
  • GiveDirectly used to track these metrics in its monthly operations reports. For example, see "Gaming detection" in GiveDirectly, Monthly operations report, October 2014.

• 165
  • "In early 2016, GiveDirectly allocated a fairly large sum of money to test a new implementation model in Kenya. A separate field director and team were hired approximately six months ago, and the experiment is taking place in an area close to GiveDirectly's other field operations. The primary aim is to double the program's size in 2017 without significantly increasing the management structure. The new model seeks to increase throughput per manager by eliminating the token payment and back check steps. GiveDirectly will assess gains in throughput as well as costs, which might occur in the areas of comprehension and fraud. Results and data should be available after a few months of disbursements. By the end of 2016, GiveDirectly hopes to have a blueprint for implementing a similar model in other countries." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pgs 4-5.
  • [GiveWell]: What is GiveDirectly checking to determine the quality of the campaigns with removed steps (especially the Kenya campaign with no backcheck and token payment)?
    [GiveDirectly]:
    • "Recipient Comprehension: With the removal of back check, we are tracking any deviation in recipient comprehension below 90% (our preferred rate). We are tracking this through additional survey questions in both audit and our follow up surveys where we ask recipients their understanding of our program. Additionally, Associate Field Managers will be conducting quality check surveys at audit stage for another data point on recipient comprehension.
    • Transfer Integrity & Adverse Events Detection. Given additional risk of two payment structure, we will be assessing (on a bi-weekly basis) the percent of transfers reversed and percentage of adverse events happening after first payment and after second payment. This will then be compared to our standard quality bar to assess significant deviations. Given a greater need for over the phone assistance, we will also be tracking through our follow up program the percentage of recipients who received no customer service and tried.
    • Fraud Detection. With the removal of back check and a two payment structure that makes each payment riskier, we will be checking the percentage of recipients flagged for additional audit stage and then the percentage of those deemed ineligible after audit is conducted. We also have an internal audit team in place that will be conducting surveys post-payment to see if fraud occurred."

    Piali Mukhopadhyay, COO, International, GiveDirectly, email to GiveWell, August 25, 2016

• 166

  "Recently, however, we’ve seen lower than usual participation rates in parts of Kenya. In July 2015 we entered Homa Bay, a new county and our first venture outside of Siaya. In Homa Bay and the neighboring areas, roughly 45% of the households we speak with decline to be enrolled into the program." GiveDirectly, Blog post, September 5, 2016

• 167

  "In July 2015 we entered Homa Bay, a new county and our first venture outside of Siaya." GiveDirectly, Blog post, September 5, 2016

• 168

  "In general, an overwhelming majority of eligible recipients opt to receive cash transfers from GiveDirectly. In Siaya, where GiveDirectly Kenya has operated from 2011 to the beginning of 2016, over 95% of recipients who are given the opportunity to be a part of the program accept it. In Uganda and Rwanda more than 96% of eligible recipients have opted in, respectively.

  These figures are high relative to participation rates in typical development programming, which is not surprising given the unconditional nature of our transfers. For example, Manuela Angelucci and Orazio Attanasio found that Oportunidades, a conditional cash transfer program in Mexico, had take up rates of roughly 50%. Oriana Bandiera et al. report that a BRAC training program targeted at adolescent girls in Uganda expected participation rates of roughly 20%.

  Recently, however, we’ve seen lower than usual participation rates in parts of Kenya. In July 2015 we entered Homa Bay, a new county and our first venture outside of Siaya. In Homa Bay and the neighboring areas, roughly 45% of the households we speak with decline to be enrolled into the program. As it turns out these challenges have been common for NGOs working in the area. Other development programs focused on HIV, water and sanitation, agricultural development, education, and female empowerment have also faced community resistance."
  GiveDirectly, Blog post, September 5, 2016

• 169
  • "We’ve done a lot to try to understand what has led to this increase in refusals. We’ve found that people typically refuse out of skepticism. Potential recipients find it hard to believe that a new organization like GiveDirectly would give roughly a year’s salary in cash, unconditionally. As a result, many people have created their own narratives to explain the cash, including rumors that the money is associated with cults or devil worship." GiveDirectly, Blog post, September 5, 2016
  • "While GiveDirectly is still determining the root causes of this problem, it has developed a few hypotheses. These include the presence of vocal religious or political leaders who might believe they have reason to mobilize the community against the payments (such as politicians who worry the organization or payments may be aligned with their opponents), or a history in the area of hostility to NGOs born either of bad prior experiences or otherwise. Recently, GiveDirectly engaged in productive conversations with some of these stakeholders. The problem might also be exacerbated by the upcoming national elections." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 5.
  • "GiveDirectly has seen an uptick in the rate of refusal to participate in its cash transfer program in Homa Bay. The root of this development is not clear, and GiveDirectly has not yet identified a solution. In some cases, community members are led by local religious leaders or local government to mistrust the program. In Siaya County, this issue did not arise, possibly because GiveDirectly covered such a large portion of the county that in any new area it entered, people were already aware of the program and knew that it was trustworthy. GiveDirectly has attempted to allay these suspicions by meeting with local government and religious leaders and speaking on local radio shows to explain the program’s purpose. In villages where the refusal rate was high, GiveDirectly put some recipients on an accelerated schedule so they could receive their transfers more quickly and serve as an example to others. After returning to these villages, however, GiveDirectly found no change in public opinion. For this reason, it has not attempted to re-enroll those who refused, but might do so in the future." GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, February 23, 2016, Pg 5.

• 170
  • "We’ve debated how much effort to expend trying to address the refusal issue. We considered leaving Homa Bay since working in an area with lower refusals would be better for efficiency targets and staff morale. However, ultimately we decided to stay because we are in the process of implementing a large scale research project in Homa Bay, and leaving the area would result in meaningful delays and cost overruns for our research partners. We also believe it’s important to try to address the issue here or else rumors could spread to other areas we’ve not yet been.

    So far we’ve tested several different approaches, with varying degrees of success. We’ve significantly expanded our outreach effort by creating a team dedicated to grassroots mobilization. We’ve also gone on the radio several times, enlisted the help of local opinion leaders, leveraged testimonials of past recipients, and much more. In some areas the results were promising. In parts of Nyando subcounty we saw refusal rates dip from 80% to 20% within a two month period. Other areas proved more difficult; in another subcounty (Rachuonyo East) refusal rates have stubbornly hovered around 40% for weeks."
    GiveDirectly, Blog post, September 5, 2016

  • "GiveDirectly created an outreach team to focus on this issue, which has been operating for three months. The team is employing a variety of different tactics: for example, it is communicating with relevant community stakeholders and using media to disseminate recipient testimonial videos and other materials. It will use A/B testing to assess the effectiveness of different strategies. During a period of a couple of months, the refusal rate dropped from an average of 70% to 20-30%, but it has rebounded significantly in the last couple of weeks." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 5.

• 171

  "High refusal rates primarily impact the productivity of census teams; for example, if 50% of households refuse registration, the team must replace them with new households. In order to continue to reach enrollment targets in Kenya, GiveDirectly has provided additional support by increasing census team sizes and adjusting team members' locations. As lists generated by census teams only include eligible and interested recipients, refusal rates are much lower during subsequent steps.

  In Kenya, GiveDirectly's enrollment numbers are still roughly 10% above target; this margin is even greater in its Uganda program."
  GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 6.

• 172
  • "However, ultimately we decided to stay because we are in the process of implementing a large scale research project in Homa Bay, and leaving the area would result in meaningful delays and cost overruns for our research partners." GiveDirectly, Blog post, September 5, 2016
  • GiveDirectly noted that the basic income guarantee experiment will be run in one old district (where it had acceptance rates of 98% for its program) and one new district. It is possible that refusals in the new district would mean that GiveDirectly would not be able to enroll everyone in an entire village; however, GiveDirectly would still be able to learn about the impacts of basic income on those who chose to participate. Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016

• 173

  "GiveDirectly is aware that similar drivers might arise elsewhere, though refusal rates in its other countries of operation are very low; for example, in Uganda, they are less than 1%. In Rwanda, refusal rates have also been very low. In a few cases where community members have been skeptical, the government has offered supportive backing to validate the program." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 6.

• 174
  • Piali Mukhopadhyay, COO, International, GiveDirectly, conversation with GiveWell, November 8, 2012
  • GiveWell visit to M-PESA agent, November 8, 2012

• 175

  GiveWell visit to M-PESA agent, November 8, 2012

• 176

  "We then transfer money directly to the recipient's account […] The recipient collects the transfer from a local M-Pesa agent, who is typically a shopkeeper in the village or the nearest town. The recipient transfers his or her electronic M-Pesa balance to the agent using his or her mobile phone in return for cash." GiveDirectly, How it works 2014

• 177
  • "Mobile money payments are subjected to a small, less than 1%, withdrawal fee; GiveDirectly absorbs this cost by increasing the transfer amount accordingly." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 4.
  • The withdrawal fee table for M-PESA is here (archive).

• 178
  • GiveDirectly worked with two different mobile money providers in its pilot campaign in Uganda: EZEE Money and MTN (745 recipients were assigned to EZEE Money, 215 recipients to MTN). Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013
  • After assessing the relative performance of these two providers, GiveDirectly chose to work exclusively with MTN in the next campaign: "Selected MTN as preferred provider in Uganda after assessing performance of Ezee/MTN (building relationship with Airtel so as to have an additional hedge)" GiveDirectly, Update for GiveWell, July 2014, Pg 9.
  • GiveDirectly has also tested working with a different payment provider (Centenary Bank) in Uganda and experienced difficulties.
    • GiveDirectly wanted to test using a bank as its payment provider partner (as opposed to a telecommunications company):
      • "Modification Use bank (vs telco) as payments vendor
      • Potential benefit 1.3% efficiency gain, Lower vulnerability to fraud given stronger protocols, accountability
      • Potential cost FD time required to build/manage partnership, Van could be unreliable"

      GiveDirectly, Update for GiveWell, February 2015, Pg 16

    • "Bank as payments provider
      • Partnered with Centenary Bank, which offered lower transaction fees than MTN
      • Difficulty scheduling cash delivery logistics (e.g., reserving vans) has pushed back first lump sum payment by a month
      • Significant amount of FD time spent managing weak counterparts at bank"

      GiveDirectly, Update for GiveWell, September 2015, Pg 10.

• 179
  • Observations from GiveWell site visit to GiveDirectly, October 2014, Pg 5.
  • "Mobile money payments are subjected to a small, less than 1%, withdrawal fee; GiveDirectly absorbs this cost by increasing the transfer amount accordingly." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 4.
  • The withdrawal fee table for MTN withdrawals is here (archive).

• 180
  • Because of this, GiveDirectly used to arrange "cash out days" in Uganda, during which GiveDirectly's mobile money provider partner would send an armored vehicle with large amounts of cash, security personnel, and multiple agents to a location close to recipients' villages, so that recipients could easily come and withdraw their funds. However, in late 2015 GiveDirectly tested and then switched to a "distributed cash out" model in Uganda (the same model that it uses in Kenya).
    • "Large volumes of cash [are] brought into villages on cashout days, making them a possible target for robbery. However, the banks or mobile money providers that GiveDirectly works with provide armored vehicles, security personnel, and valuable cash handling expertise for the cash out days." GiveWell's non-verbatim summary of a conversation with Paul Niehaus and Carolina Toth, September 7, 2015, Pg 7.
    • "In Uganda, transfers are sent in ten equal installments, and GiveDirectly works with the mobile money providers to coordinate pay out days, as their agent networks are less robust than those in Kenya." Conversation with GiveDirectly, April 8, 2014, Pg 4.
    • "Pilot of distributed cashout with MTN in lieu of paydays underway as well." GiveDirectly, Update for GiveWell, September 2015, Pg 10.
    • Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016
  • Carolina Toth, email to GiveWell, November 10, 2015
  • GiveDirectly hopes that communicating intensively to recipients about where the nearest MTN mobile money agents are will make it possible to use the distributed cash out model in Uganda.
    • Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
    • Note that GiveDirectly has not previously tested the distributed cash out model in Uganda. It originally developed and implemented the cash out day model for Uganda because it thought that some recipients would be too far away from MTN mobile money agents to (e.g., some recipients would have to make a three hour round trip to withdraw their funds). Now GiveDirectly thinks that with targeted communication about where the nearest MTN agents are, GiveDirectly might be able to spare recipients these longer trips under the distributed cash out model. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • "GiveDirectly has often found the 'payday' payment model too restrictive. In its Uganda program, it has successfully tested a 'cash-out' model. Recipients and agents were able to overcome liquidity constraints, and in Uganda, GiveDirectly distributes $700,000 to $1 million per month without any increases in fraud." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 5.

• 181
  • For example, see GiveWell, spot checks of Segovia follow-up data sample, 2016
  • When GiveDirectly first started its "distributed cash out" model in Uganda (instead of hosting "cash out days" GiveDirectly conducted some quick follow-up phone calls with vulnerable recipients in Uganda; in the sample of 67 call records GiveDirectly sent us, only 9 vulnerable recipients had already withdrawn their funds successfully (although many had received the transfer and were planning to withdraw it soon, and 22 responses were ambiguous). GiveDirectly, Distributed cash out follow up with vulnerable recipients. Note that some of the comments indicate that the person surveyed was not the recipient, but someone close to the recipient or the recipient's helper. We are not sure how indicative this data is of difficulties obtaining funds.

• 182

  Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 183
  • GiveDirectly 's blog post on the staff fraud: GiveDirectly blog, Fighting fraud in Uganda
  • "Two GD staff members stole 2% of transfers on Google campaign ($20,500 in total) in form of 50,000 UGX deductions from penultimate or final transfers (85%) or theft of entire final transfers (15%)"GiveDirectly, Check in with GiveWell, September 2014

• 184

  "Discovery of the fraud

  • A GiveDirectly recipient had given their SIM card to the SFO [Senior Field Officer] (whose contract had been terminated in April due to an unrelated issue involving a fraudulent receipt he brought to GiveDirectly for reimbursement). The recipient asked the SFO to replace his SIM card (recipients have to travel about 4 hours round trip to get this done) and the SFO had not returned it. This report was made to the hotline that the OM [Office Manager] was answering three months after the recipient had given over their SIM card. Mr. Skeates audits the logs of these hotline calls.
  • GiveDirectly had been planning a full round of follow-up surveys as part of its normal process, but accelerated the follow-up surveys in response to this issue. GiveDirectly’s backcheck team paused their work on enrollment for the Uganda 2M campaign and called all the recipients in that village (Kosile) to ask whether they had received all of their transfers, had any problems withdrawing, and whether GiveDirectly currently had any of their documents (e.g., SIM cards, IDs).
  • During this process, there were some reports of problems during paydays. Recipients were initially hesitant to come forward.
  • Because of the reports of payday problems, GiveDirectly began calling another village, Kawo, the following day to gather more information. Recipients in Kawo were far more forthcoming with information when asked specific questions about payday problems.
  • GiveDirectly continued follow-ups (conducted by a new Field Officer brought on after the SFO's and OM’s dismissals) until it had spoken to about 92% of its recipients across all villages. GiveDirectly also conducted in-person visits to villages."

  Conversation with GiveDirectly, September 5, 2014

• 185

  Changes implemented in response to staff fraud
  In the Uganda pilot campaign, GiveDirectly cash out days were managed by the Uganda Senior Field Officer, the Uganda Office Manager who also managed the GiveDirectly hotline, and mobile money agents. After these people fraudulently diverted funds from recipients, GiveDirectly implemented a series of changes:

  • Terminated the GiveDirectly staff who had been involved in the fraud; started working with new mobile money agents.
  • Removed all of its staff from the cash out day process except the Uganda Field Director. The Uganda Field Director had previously been making planned visits to oversee some of the cash out days; he now actively manages all of them along with new mobile money agents.
  • Appointed community-nominated monitors to assist the Uganda Field Director on the cash out day with translation, observe transactions between recipients and mobile money agents, and report any issues they see. GiveDirectly compensates the monitors with 10,000 UGX (~$4) for their time during a cash out day.
  • Developed networks of English-speaking informants who are not formally announced within the villages, but are tasked with also reporting any issues they see regarding transfers. To date, 4 of the 9 informants have provided GiveDirectly with helpful information, such as identifying that households in the enrollment process were actually ineligible, and telling GiveDirectly that someone had taken a recipient's phone after the recipient passed away.
  • Moved the GiveDirectly call center (hotline) to Kampala, to increase the separation of call center staff from field staff, who are based in Mbale.
  • Tasked the call center with calling a randomly selected 10% of the village during a cash out day to see if it is going smoothly.
  • Changed the contractual agreement GiveDirectly has with mobile money agents to include an indemnity clause, so that in the case of stolen funds, GiveDirectly could remove funds directly from a mobile money agent's account.

  Conversation with Stuart Skeates, GiveDirectly, October 20-21, 2014

• 186
  • GiveDirectly has piloted a few changes that would increase security, including the use of biometrics (more) and partnering with a banking partner for cash out days (more).
  • In late 2015, GiveDirectly piloted the distributed cash out model in Uganda that it now uses. It is possible that a distributed cash out model is more secure from large-scale crime than a payday model because a) without the cash out days, funds are not as concentrated in one location and b) it's easier to obscure who is a recipient when recipients withdraw their funds at different times from different places. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015.

• 187
  • Note that sometimes as GiveDirectly scales and moves into new areas, it could end up being less well known. For example, when GiveDirectly moved most of its Kenya operations from Siaya County to Homa Bay County, it experienced a high rate of people refusing to be enrolled. GiveDirectly thinks this may be because many people in Homa Bay had not heard of GiveDirectly before and were suspicious of the program.
  • "GiveDirectly has seen an uptick in the rate of refusal to participate in its cash transfer program in Homa Bay. The root of this development is not clear, and GiveDirectly has not yet identified a solution. In some cases, community members are led by local religious leaders or local government to mistrust the program. In Siaya County, this issue did not arise, possibly because GiveDirectly covered such a large portion of the county that in any new area it entered, people were already aware of the program and knew that it was trustworthy." Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016, Pg 5.

• 188

  The cash out days that GiveDirectly used to administer in Uganda seemed to be particularly easy targets for large-scale theft, as there was a substantial amount of cash in one location (although it is our understanding that GiveDirectly's partners sent security personnel to the cash out days to mitigate this risk). Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015. If GiveDirectly decides to run cash out days again, either in Uganda or a new location, we will slightly increase our concern about theft.

• 189

  GiveDirectly notes: "...we see two possible forms of larger scale crime/interference. The first is interference or expropriation by other institutional actors. As we have relayed previously we have tried to hedge against this by improving governmental contacts. The second is some larger scale organized crime, but we do not see this as a threat that meaningfully increases as we scale as any attempt at theft would still need to target small individual disbursed transactions, which is the same fundamental risk to our current program that we believe our current structure has had a strong track record in mitigating." Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016

• 190

  Despite the mobile money security measures, Lydia Tala, an Assistant Field Manager who has been responsible for making post-transfer phone calls to recipients in Kenya, reports that one of the most common client complaints is the belief that M-PESA agents are overcharging or stealing funds. Lydia Tala, GiveDirectly Field Assistant, conversation with GiveWell, November 7, 2012

• 191
  • Ms. Tala (an Assistant Field Manager, see footnote above) believes that these reports are incorrect and that approximately 10% of recipients are not fully aware of how to use M-PESA, so they may withdraw funds without checking their balance, ultimately being surprised when they have drawn down their account. Lydia Tala, GiveDirectly Field Assistant, conversation with GiveWell, November 7, 2012
  • GiveDirectly told us that it recognizes this issue and maintains a hotline to provide recipients with assistance in navigating the M-PESA system: GiveDirectly helps recipients with issues using M-PESA via our hotline. Staff sometimes contact M-PESA agents to notify them one of our clients will be visiting and may need assistance. Piali Mukhopadhyay, COO, International, GiveDirectly, email to GiveWell, November 23, 2012

• 192

  After the transfers are sent, GiveDirectly also administers follow up surveys that ask recipients if they have collected their funds and if they had any trouble doing so.

  • @GiveDirectly, Siaya Verification Stats@
  • GiveDirectly, Rarieda verification stats
  • GiveDirectly, Siaya follow-up data - disaggregated
  • GiveDirectly, Nike follow-up data - disaggregated
  • GiveDirectly, Google follow-up data - disaggregated (LS - long)
  • @GiveDirectly, Rarieda Top-up verification (short)@
  • GiveDirectly, Rarieda verification (top ups), May 26, 2013
  • GiveDirectly, Siaya verification, June 15, 2013
  • GiveDirectly, Google verification, September 2013
  • GiveDirectly, Nike verification (final), September 2013
  • GiveDirectly, Kenya follow up data, November 2014
  • GiveDirectly, Uganda pilot follow up data, April 2014
  • GiveDirectly gave permission to publish the following databases on the condition that all personally identifying information was removed. Due to the size of the databases, we have not yet anonymized them and therefore have not published them. If you are interested in the content of these databases, please contact us at info@givewell.org.
    • GiveWell, GiveDirectly follow up surveys summary - Uganda, September 2015
    • GiveWell, GiveDirectly follow up surveys summary - Kenya, September 2015

  The percentage of recipients who report issues withdrawing funds is consistently low (<5%) across campaigns. See the table below for details. Follow up surveys also ask recipients what size of transfer they received. These amounts generally appear to vary slightly among cohorts of recipients. For example, in follow-up surveys of recipients in Kenya from 2014, recipients reported receiving various amounts between 37,000 KES – 40,000 KES. This is based on data GiveWell reviewed in 2014. GiveDirectly, Kenya follow up data, November 2014. Other than the mobile phone purchase deduction, we do not know the causes of this variance.

• 193

  "There were four types of issues that were responsible for most of the people who were unable to withdraw funds at the cash out day: […] MTN (the payment provider) had not yet activated the funds in the person's account. Mr. Skeates said that this was not a common problem in the previous campaign in Uganda, but it affected many people at this cash out day. He estimated that people affected by this issue would receive their first installment of funds in another 2-3 weeks. […]" GiveWell site visit to GiveDirectly, October 2014, Pg 6.

• 194
  • A challenge of backchecks is that field officers often end up teaching recipients how to use their cell phones and mobile money accounts, so that they can access their money and are less likely to be scammed. Field officers will teach recipients how to check their balance and distinguish messages that say they received money from other messages. Field officers will sometimes write out instructions for recipients in the local language that describe step-by-step how to operate the phones and mobile money accounts. Recipients often do not understand the importance of keeping their PIN numbers secure. Some elderly recipients do not want a trustee to manage their transfers, but they are unable to remember their PIN numbers or read the messages on their phone, so they are more likely to have issues receiving transfers. Some people are still skeptical that the money will actually come, even after they have received messages on their phone, so they don’t pay attention to the instructions about how to use the mobile money account. Conversation with GiveDirectly field staff, October 20-21, 2014, Pg 3.
  • Mr. Skeates, the Uganda Field Director, made announcements at the start of the event, translated by the 2 community monitors. The announcements included reminders about how to keep account information secure (e.g., after entering your PIN number, make sure to press "send" before handing your phone back to the agent; make sure you have received a confirmation message after withdrawing and that it states the correct amount; count the cash immediately after receiving it.) GiveWell site visit to GiveDirectly, October 2014 Pg 5.

• 195
  • Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • The summary results of these quality audits for Kenya can be seen in some monthly operations reports (e.g. GiveDirectly, Monthly operations report, August 2015). GiveDirectly has sent us results from these audits for the Ug-201503 campaign as well, which we have briefly reviewed but have not published. GiveDirectly, Uganda model variations quality audits - census and GiveDirectly, Uganda model variations quality audits - registration

• 196

  Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 197
  • A "trustee" is someone who is registered for the mobile money payments on behalf of the recipient. A "helper" is someone who is not registered for the payments, but who helps the recipient with the process (e.g. assisting with transportation to the locations one can withdraw cash or helping to use the phone properly).
  • Roughly, the process for choosing a trustee or helper is to get the recipient alone (out of earshot of family) and ask who that recipient trusts the most. This choice is typically validated with some neighbors (ensuring that that person is regarded as trustworthy). Generally, GiveDirectly prefers to choose trustees and helpers who are already recipients themselves, so that they have less of an incentive to steal the transfer and so that GiveDirectly can stop transfers to them if they are not performing their role appropriately.

  Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015. Note that GiveDirectly has offered to send us the protocol used to determine helpers and trustees; we have not yet reviewed this protocol.
  

• 198

  Table 28, Haushofer and Shapiro 2013 Appendix, Pg. 54.

• 199
  • Non-land assets includes “the value, in USD PPP, of all moveable assets the household owns, including savings, plus the value of the roof of the home.” Haushofer and Shapiro 2013 Policy Brief, pg. 10.
  • Haushofer and Shapiro 2013 Appendix, Pgs. 1-2.

• 200
  • Small transfers: $210 (95% CI: $158 to $263). Table 28, Haushofer and Shapiro 2013 Appendix, Pg. 54
  • The largest categories of asset increases were livestock ($131, 95% CI: $79 to $183), durable goods ($100, 95% CI: $71 to $129; primarily furniture), and savings ($18, 95% CI: $9 to $27).
    • Small transfers, livestock: $68 (95% CI: $35 to $100)
    • Small transfers, durable goods: $36 (95% CI: $18 to $54)
    • Small transfers, savings: $7 (95% CI: $2 to $13)

    Table 32, Haushofer and Shapiro 2013 Appendix, Pg. 58.

• 201
  • Table 5, Haushofer and Shapiro 2013, Pg. 53
  • Though Haushofer and Shapiro 2013 doesn't report the change in likelihood for recipients of large transfers alone, recipients of large transfers were 23 percentage points (95% CI: 13% to 33%) more likely to have iron roofs at end-line than recipients of small transfers: Table 5, Haushofer and Shapiro 2013, Pg. 53

• 202

  "To this end, we conducted a separate survey of one respondent from each of 20 villages to obtain estimates for the costs of purchasing and maintaining metal and thatch roofs. The purchase of a metal roof represents an expenditure of on average USD 564, or 75 percent of the average transfer value." Haushofer and Shapiro 2013, Pg. 34

• 203

  "Based on the anonymized individual-level survey data, an iron roof costs $418 on average, thatch roof replacement (including the cost of grass for making the roof and the labor) costs $95 on average, and thatch roof repair (including the cost of grass for making the roof and the labor) costs $107 on average. These numbers appear to conflict with the full paper and the policy brief. It may be that the results were from a different survey. Haushofer and Shapiro have not yet finished verifying which data were used." GiveWell's non-verbatim summary of a conversation with Carolina Toth, GiveDirectly, October 1, 2014

• 204

  Table 44, Haushofer and Shapiro 2013 Appendix, Pg. 70

• 205

  Table 44, Haushofer and Shapiro 2013 Appendix, Pg. 70

• 206

  Table 36, Haushofer and Shapiro 2013 Appendix, Pg. 62

• 207

  Table 36, Haushofer and Shapiro 2013 Appendix, Pg. 62

• 208
  • Table 36, Haushofer and Shapiro 2013 Appendix, Pg. 62
  • We're not sure of the time period over which this estimate is calculated. Haushofer and Shapiro 2013 also reports that treatment households receiving large transfers spent $16.26 (95% CI: -$6.50 to $39.02) more than control households on education expenditures in the past month and treatment households receiving small transfers spent $19.41 (95% CI: -$12.22 to $44.74) more: Table 52, Haushofer and Shapiro 2013 Appendix, Pg. 78 We're not sure if the difference between the two estimates is due to the difference in the samples used to calculate them (they have different sample sizes) or the different time periods over which they might be calculated or some other explanation.
    • N = 1372. Table 36, Haushofer and Shapiro 2013 Appendix, Pg. 62
    • N = 1174. Table 52, Haushofer and Shapiro 2013 Appendix, Pg. 78

• 209

  Small transfers: $31 (95% CI: $18 to $43). Table 28, Haushofer and Shapiro 2013 Appendix, Pg. 54

• 210
  • Large transfers: $25 (95% CI: $11 to $39). $25/$51 = about 50%
  • Small transfers: $18 (95% CI: $9 to $27). $18/$31 = about 60%
  • Table 36, Haushofer and Shapiro 2013 Appendix, Pg. 62

• 211
  • Large transfers, social: $3 (95% CI: $1 to $5)
  • Small transfers, social: $2 (95% CI: $1 to $3)
  • Large transfers, other: $19 (95% CI: $13 to $24)
  • Small transfers, other: $7 (95% CI: $3 to $11)
  • Table 36, Haushofer and Shapiro 2013 Appendix, Pg. 62

• 212
  • Large transfers, alcohol: -$2.07 (95% CI: -$4.6 to $0.5)
  • Small transfers, alcohol: -$0.51 (95% CI: -$2.7 to $1.7)
  • Large transfers, tobacco: -$0.38 (95% CI: -$1.0 to $0.2)
  • Small transfers, tobacco: -$0.08 (95% CI: -$0.6 to $0.4)
  • Table 36, Haushofer and Shapiro 2013 Appendix, Pg. 62

• 213

  See our intervention report on cash transfers.

• 214

  See our intervention report on cash transfers.

• 215

  "Following its Google-funded campaign, GiveDirectly surveyed recipients in detail on how they spent their transfers. Given the limitations of this kind of self-reported data, GiveDirectly has not continued this practice. It prefers to rely on more accurate data gleaned through randomized controlled trials (RCTs), and expects to collect more [of] this type of information in future studies or campaigns, such as its ongoing RCT in coffee farming communities." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 4.

• 216
  • GiveDirectly, Kenya follow up data, November 2014
  • GiveDirectly, Uganda pilot follow up data, April 2014
  • GiveDirectly, Rarieda verification (top ups), May 26, 2013
  • GiveDirectly, Siaya verification, June 15, 2013
  • GiveDirectly, Google verification, September 2013
  • GiveDirectly, Nike verification (final), September 2013
  • GiveWell, GiveDirectly follow up surveys summary - Uganda, September 2015
  • GiveWell, GiveDirectly follow up surveys summary - Kenya, September 2015 (Note: GiveDirectly gave permission to publish this database on the condition that all personally identifying information was removed. Due to the size of the database, we have not yet anonymized it and therefore have not published it. If you are interested in the content of this database, please contact us at info@givewell.org.)

  Note that in some cases, we have cleaned this data (where it was obvious what the error was in the data entry). These cases have been marked with cell comments. In other cases, when it was not clear what the error was (for example, when an amount of funds was mistakenly placed in a spending category column, and it was not clear whether the amount was intended to be placed in the former or latter column), we deleted the data and commented on the cell to indicate that data had been deleted. Deletions were made no more than 5 times in any given spreadsheet.

• 217
  • The tables include follow up survey data from the Kenya 2M, Kenya 1.2M, Kenya rolling enrollment, and Kenya behavioral optimization campaigns and from the Uganda pilot campaign. Note that recipients may have been surveyed more than once and would therefore be included more than once in the data presented.
    • GiveWell, GiveDirectly follow up surveys summary - Kenya, September 2015 (Note: GiveDirectly gave permission to publish this database on the condition that all personally identifying information was removed. Due to the size of the database, we have not yet anonymized it and therefore have not published it. If you are interested in the content of this database, please contact us at info@givewell.org.)
    • GiveWell, GiveDirectly follow up surveys summary - Uganda, September 2015
  • We have seen some spending data for the Ug-201404 campaign (the Uganda 2M campaign), but we have not analyzed it. We have not seen spending data from the Ug-201503 campaign (the Uganda model variations campaign) yet; it is our understanding that GiveDirectly did not collect spending data for that campaign: "Please note that for the Uganda 201503, we no longer collect spending data. I believe Carolina has mentioned this before - we felt the data from our RCT was more reliable on this and we had enough of it to feel confident we understood spending trends without needing to continue to collect it in a form less optimal than in the RCT." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, August 17, 2016

• 218In follow up surveys administered in Uganda, recipients were asked about spending on large household items and small household items. The figure reported here is the combination of those two categories.
• 219

  GiveDirectly, What We Do - Who We Serve, September 2016

• 220

  GiveDirectly, What We Do - Who We Serve, September 2016 See the chart in the upper left section of the web page.

• 221

  GiveDirectly, What We Do - Who We Serve, September 2016 See the chart in the upper left section of the web page.

• 222

  For full details of our interviews with recipients, see GiveWell Site visit notes.

• 223

  Provided by Google via Citibank N.A. on November 15, 2012.

• 224

  Lydia Tala, GiveDirectly Field Assistant, conversation with GiveWell, November 7, 2012.

• 225

  Provided by Google via Citibank N.A. on November 15, 2012.

• 226

  Ian Bassin and Piali Mukhopadhyay, conversation with GiveWell, August 23, 2016

• 227
  • "The ‘thatch roof, mud walls, mud floor’ eligibility criteria was not going to work in Homa Bay, as <3% households had thatch roofs." Carolina Toth, email to GiveWell, October 20, 2015
  • GiveDirectly has told us that this is because there is very little grass in Homa Bay County. GiveDirectly thinks that people in Homa Bay have spent more money historically on their buildings (because the cost of thatch roofs was not cost-competitive).
  • "However, more people in Homa Bay have metal roofs than in Siaya. This is likely because the grass for thatched roofs does not grow in Homa Bay, so the price of thatch is less competitive." Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016, Pg 6.

• 228

  Paul Niehaus and Ian Bassin, conversation with GiveWell, September 15, 2016

• 229

  "Physically closer to Rarieda than Siaya was
  Poverty rate is higher than in Rarieda (50% vs 46%)"
  GiveDirectly, Update for GiveWell, February 2016, Pg 13

• 230
  • The amount of time that it takes for total transfers to be sent has varied between campaigns. The Google transfer campaign likely gives the best approximation of timing for a standard campaign, because it does not involve an experiment (like Rarieda and Nike), and it is more recent than the Siaya campaign. In the Google campaign, the planned schedule for transfers lasts approximately eight months for any given recipient, not including the time for census and enrollment. GiveDirectly, Google transfer schedule, July 2013
  • "GiveDirectly stops issuing transfers after two years and clearly informs recipients that they should expect this." GiveDirectly, Offering Memorandum (January 2012), Pg 33.
  • Update [November 2014]: the two year timeline mentioned above is now outdated; transfers are generally sent over a period of 8 months in Kenya and 10 months in Uganda. It is still the case that recipients can only receive one full transfer, and that they are ineligible for additional transfers thereafter. Conversation with Carolina Toth, GiveDirectly, November 20, 2014
  • In November 2015, GiveDirectly informed us that it now aims to send transfers within 4 months. Carolina Toth, email to GiveWell, November 10, 2015
  • Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • "In Kenyan Shillings, they are 10K, 50K, 50K (assuming no phone purchase). Ugandan shillings is 3.2M but division into different transfers hasn't been decided yet." Carolina Toth, email to GiveWell, September 25, 2015
  • XE currency converter, Kenya shillings to US dollars, September 25, 2015
  • XE currency converter, Uganda shillings to US dollars, September 25, 2015

• 231
  • For example, see the Behavioral interventions study.
  • Carolina Toth, email to GiveWell, November 10, 2015

• 232

  @GiveDirectly, Contextualizing Transfer Size@

• 233
  • $0.65 in pre-cash-transfer consumption per person per day implies (365*$0.65) = $237.25 per person per year. If each person receives $288 in a year from GiveDirectly, that's (288/237.25) = 121%.
  • Note that recipients in Uganda live on $0.83, according to GiveDirectly's website (we believe this is referring to consumption because GiveDirectly's comparable figure for Kenya is $0.65). GiveDirectly, What We Do - Operating Model, see the Uganda tab. So the calculation in Uganda would be (365*$0.83)=$302.95. ($288/$302.95)=95%.
  • Note that GiveDirectly has told us that recipients in Homa Bay have a slightly lower average daily consumption of $0.50. GiveDirectly, Consumption data for targeting work So, the calculation in Homa Bay would be (365*$0.50)=$182.50. ($288/$182.50)=158%.

• 234

  Paul Niehaus, GiveDirectly Founder, email to GiveWell, November 20, 2012. We have not reviewed the data GiveDirectly used to reach this conclusion.

• 235

  In our conversations with recipients and field staff, we phrased this question two different ways:

  • Do you think it would be better for GiveDirectly to provide $1000 transfers to households in one village or $500 transfers to households in two villages?
  • Do you think that GiveDirectly should keep the transfer size the same or reduce the transfer size but provide transfers to a greater number of people?

  GiveWell site visit to GiveDirectly, October 2014
  Conversation with GiveDirectly field staff, October 20-21, 2014, Pgs 5-6.

• 236

  GiveWell site visit to GiveDirectly, October 2014

• 237

  We asked the field officers what they think about the current transfer size ($1000), and whether they’d choose to keep it at that level, increase it, or decrease it, given the effects that an adjustment would have on how many people GiveDirectly would be able to serve.

  • Mr. Okello: Typically there are multiple households on one compound, each inhabited by relatives of the same family, and any household that meets the targeting criteria can receive transfers. Mr. Okello said that it may make more sense for GiveDirectly to group some households on a compound together so that transfers are shared across them, rather than each eligible household receiving the full $1000.

    Mr. Okello also said that if GiveDirectly increased the size of the transfers, that could create a high level of dependency. One of the messages that field officers send is that people should use the $1000 transfers to develop themselves as much as possible, but if someone knew they were getting $2000, they may stop farming, for example. With $1000 people can get some things but not everything; it is the right amount.

  • Mr. Ekeu: Mr. Ekeu prefers reducing the amount of money in each transfer and expanding the recipient base to reach everyone in the village. He said that the current targeting model causes bragging and unrest in the communities. The people who don’t benefit may be brought to use force to get some of the money, such as by breaking into recipients’ homes. Mr. Ekeu suggested that it would be better for GiveDirectly to provide all households in a village with some amount of money, even if it was less for households that are currently deemed ineligible (e.g., $100). This way, each of the households would be busy figuring out how they would spend their own money rather than how to get money from another.
  • Mr. Olinga: Mr. Olinga said that to reduce extreme poverty the bigger transfer is better, but he didn't have a strong opinion on $1000 transfers to some people versus $500 transfers to twice as many. [This is how we posed the question to Mr. Olinga.]

  Conversation with GiveDirectly field staff, October 20-21, 2014, Pgs 5-6.

• 238

  Conversation with GiveDirectly, September 5, 2014

• 239

  Conversation with GiveDirectly, September 5, 2014

• 240

  Conversation with GiveDirectly, September 5, 2014

• 241
  • Haushofer and Shapiro 2013 Policy Brief, Table 10, Pg 38.
  • "Additional variables in table 9 show the frequency of any episode of physical, sexual or emotional violence in the last six months, and the percentage of respondents who believe that domestic violence is justified in some instances. The point estimates for these variables suggest a reduction in domestic violence, although none are individually different from zero at conventional significance levels." Haushofer and Shapiro 2013 Policy Brief, Pg 21.

• 242
  • GiveDirectly sent 8 versions of the survey forms it used, as the form was modified over time. For the most recent version we have seen, see GiveDirectly, Kenya verification template, August 2013. We expect that the questions have changed slightly since this version.
  • For earlier versions, see @GiveDirectly, Verification Template (October 1, 2012)@ and @GiveDirectly, Verification Template (November 7, 2011)@ (corresponding data for the latter: GiveDirectly, Verification data (November 17, 2011))

• 243
  • "For example, following concerns about missed inbound calls, it decided to upgrade its call center technology. This process is ongoing; GiveDirectly expects to see progress in this area within the next few months." Ian Bassin and Piali Mukhopadhyay, conversation with GiveWell, August 23, 2016, Pg 2.
  • [GiveWell]: "How did GiveDirectly become aware that it might have been missing some incoming calls to the hotline and how will the new call center technology fix this issue?"
    [GiveDirectly]:
    • "Hotline phones keep a record of missed calls and we were seeing more than usual
    • FOs in the field would hear anecdotally that recipients tried the hotline number and failed to reach
    • The new call center will have a centrally controlled hotline system where inbound calls are routed directly to the first available agent (right now they are being routed sequentially). New technology will also allow us to monitor call volumes and staff the hotline dynamically as certain times of days and days of the month see higher volumes"

    Piali Mukhopadhyay, COO, International, GiveDirectly, email to GiveWell, August 25, 2016

• 244

  GiveWell, spot checks of Segovia follow-up data sample, 2016 Note that we did not specify to GiveDirectly which sample to send and they did not say how they selected this particular sample.

• 245
  • GiveWell, spot checks of Segovia follow-up data sample, 2016 While there were at least 2 events reported for every adverse event that GiveDirectly staff asked about, the rate of other adverse events was effectively 0%.
  • The rates of issues we've seen reported by GiveDirectly are typically also quite low:
    • GiveDirectly's website reports that only 0.3% of recipients in Kenya were asked for a bribe; we are not sure over what time period or from what sample this figure was calculated. GiveDirectly, Performance - Quality of Service, September 2016
    • In August 2016, GiveDirectly mentioned that its Uganda campaign only has a 0.52% complaint rate from follow-up calls; again, we are not sure from what sample this figure is calculated: "The most common complaints/comments are from people seeking transfers, either people hoping GD will come to their area, people seeking a greater transfer, or ineligibles or people whose transfers have been delayed for some reason seeking to receive. The other recently common complaint are people saying the money is evil in some way. For context on size, Uganda rolling currently has a 0.52% complaint rate of for all follow up calls." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, August 25, 2016
  • Note that GiveDirectly has told us complaints tend to be higher when it first enters a new area: "When GiveDirectly enters a new area, complaint rates tend to be relatively high. This is because GiveDirectly records as "complaints" callers who request payments but are not eligible for its program. When GiveDirectly initially enters a new area, word spreads that GiveDirectly is distributing funds but people misunderstand the program, so these call volumes tend to be high. For example, the initial rate in GiveDirectly's new Rwanda campaign was 30.94%. Field teams are responsible for identifying the source of and addressing high complaint rates." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 2.

• 246
  • GiveWell, GiveDirectly follow up surveys summary - Kenya, September 2015 (Note: GiveDirectly gave permission to publish this database on the condition that all personally identifying information was removed. Due to the size of the database, we have not yet anonymized it and therefore have not published it. If you are interested in the content of this database, please contact us at info@givewell.org.)
  • GiveWell, GiveDirectly follow up surveys summary - Uganda, September 2015
  • For a list of follow-up survey questions from 2013, see GiveDirectly, Kenya verification template, August 2013. Note that this template may now be out of date.

• 247

  Note that not every follow-up survey asks about precisely the same issues (e.g. some do not ask about domestic violence or trouble collecting), which is part of what accounts for the different sizes of each group of respondents.

• 248In the Uganda follow up data, this issue is denoted "stole_item."
• 249In the Uganda follow up data, we identified 4 issues that we believe all asked about bribes ("bribe," "pay to collect," "others_bribes," "agent_bribe"). We only include the "bribe" issue in this table for Uganda (which has the highest rate of complaints out of the four).
• 250
  • GiveDirectly, Kenya hotline log, July 2013 (May 2012 – July 2013)
  • GiveDirectly, Follow-up tracker, July 2013 (July – October 2013)
  • GiveDirectly, Follow-up tracker, October 2014 (January 2013-September 2014). This file also contains issues reported through other channels, such as to field staff during field visits and during follow up surveys.
  • GiveDirectly, Kenya top 10 adverse events 2015 (Note: we are unsure if all of the issues reported in this file were reported through the hotline.)
  • GiveDirectly, Uganda top 10 adverse events 2015 (Note: we are unsure if all of the issues reported in this file were reported through the hotline.)
  • GiveDirectly, Kenya randomized sample of adverse events, 2014-2015
  • GiveDirectly, Uganda randomized sample of adverse events, 2014-2015
  • We have not checked to see whether the issues reported in these files are overlapping, though note that the time ranges are overlapping.

• 251

  GiveDirectly, Follow-up tracker, October 2014 Sheet: "Summary" In 2015 we did not ask GiveDirectly to send us a follow-up tracker because it is our understanding that sharing the follow-up tracker databases take a significant amount of effort on GiveDirectly's part (much of the effort goes into anonymizing the entries). Instead, we requested a random sample of adverse events and a sample of some of the most serious adverse events from the previous year. We have reviewed these samples, but have not made them public (they are not anonymized); the issues they cover are broadly similar to the types of issues we have seen in previous years.

• 252

  GiveDirectly, Follow-up tracker, October 2014 Sheets: Summary; GiveWell notes.

• 253

  See our intervention report on cash transfers.

• 254

  Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016

• 255

  Haushofer and Shapiro 2013 Policy Brief, Table 10, Pg 38.

• 256
  • The RCT of GiveDirectly’s program in Rarieda did not find an increase in crime, so at that scale it does not seem to be an issue. It’s possible that crime would be a more serious problem if GiveDirectly became a substantially larger and better-known organization. Conversation with GiveDirectly, December 7, 2013
  • GiveDirectly has become very well-known in Siaya County, Kenya, but has not seen a significant increase in crime rates there. As GiveDirectly begins to work in Homa Bay County, it expects crime rates to be lower, because the context is similar to Siaya but fewer people know about GiveDirectly in Homa Bay. If GiveDirectly were to start working in more urban areas, where crime rates tend to be higher, GiveDirectly would put more time into strategizing about crime and security. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 257Example: "The recipient was given all the cash withdrawn as she requested.. then as she {deidentified} she was just outside the house since she's blind and her door is not lockable, she came to find her money missing but she doesn't know who might have stolen the KES."GiveDirectly, Follow-up tracker, October 2014 Sheet: "Tracker" (text removed in deidentification.)

• 258

  Example: "He was phoned by unknown person who posed as GD staff and requested for 500/= bribe to hasten the processing of his transfer."GiveDirectly, Follow-up tracker, October 2014 Sheet: "Tracker" (text removed in deidentification.)

• 259Example: "She lost the phone, and in the process of renewing the line the Agent transfer the money to another line in order to withdraw later."GiveDirectly, Follow-up tracker, October 2014 Sheet: "Tracker" (text removed in deidentification.)
  
• 260

  Some recipients, especially elderly ones, have to learn how to use cell phones for the first time in order to manage the GiveDirectly transfers in mobile money accounts. These people have a more difficult time understanding how to keep their phones secure; for example, they often keep the phone in its original packaging and do not conceal it. Another problem with security is that some recipients will share the PIN numbers for their mobile money accounts, either intentionally or unintentionally by handing the phone to a mobile money agent before pressing "Send" (so the PIN number is still apparent on the screen of the phone.) This makes recipients more vulnerable to people who wanted to steal money from their accounts. Teaching PIN safety has long been a priority, and GiveDirectly has added additional emphasis on the topic (e.g., emphasis during village meetings, additional trainings given by the mobile provider) Improved security is a reason why GiveDirectly is interested in piloting biometric authentication for mobile money accounts, though it does not currently have plans to do so. Conversation with Stuart Skeates, GiveDirectly, October 20-21, 2014, Pgs 2-3.

• 261
  • Conversation with Stuart Skeates, GiveDirectly, October 20-21, 2014 (unpublished)
  • GiveDirectly, Follow-up tracker, October 2014;GiveDirectly, Kenya hotline log, July 2013 (May 2012 – July 2013); GiveDirectly, Follow-up tracker, July 2013 (July – October 2013)

• 262

  In Siaya, GiveDirectly experienced some difficulty with people pretending to live in poorer quality housing: "[GiveWell]: There were multiple comments about recipients switching from iron-roofed houses to grass-roofed houses in order to be enrolled. Is this becoming a more common problem? How does GiveDirectly discover these instances? [GiveDirectly staff]: This was a common problem in Siaya -- where everyone knew, from our work there, that our criteria relied on housing materials, therefore they’d try to pose as living in such a house to be eligible. There are a number of ways this can be discovered -- either by asking neighbors, or observing that a recipient does not have at hand items (like vaccination forms) that would be in their possession if they were actually at their home, suggesting they don’t actually live where they are claiming to live." GiveDirectly staff, responses to monitoring questions, October 11, 2016, pg. 3.

• 263
  • "GiveDirectly stops issuing transfers after two years and clearly informs recipients that they should expect this." GiveDirectly, Offering Memorandum (January 2012), Pg 33.
  • Update [November 2014]: the two year timeline mentioned above is now outdated; transfers are generally sent over a period of 8 months in Kenya and 10 months in Uganda. It is still the case that recipients can only receive one full transfer, and that they are ineligible for additional transfers thereafter. Conversation with Carolina Toth, GiveDirectly, November 20, 2014
  • "Attrition was not a significant concern in this study because it became evident early on in GD’s work in Kenya that respondents were highly interested in maintaining relations with GiveDirectly in the hope of receiving future transfers (although these are never promised)." Johannes Haushofer and Jeremy Shapiro, Welfare Effects of Unconditional Cash Transfers: Pre-Analysis Plan, June 27, 2013, Pgs 5-6.

• 264

  In the Rarieda campaign, 67% (359 of 536) of recipients waited less than a month, 84% (448 of 536) waited 3 months or less, and 6% (34 of 536) waited 6 months or more. In the Siaya campaign (a later campaign), 188 of 193 recipients waited less than a month, and the remaining 5 waited 2-3 months.

  • GiveDirectly, Enrollment speed of distributions - Siaya and Rarieda
  • GiveDirectly commented: "We were able to accelerate [the time it took for recipients to register for M-PESA] significantly for two reasons: (a) we gave clearer instructions, and (b) we let recipients designate which household member they wanted to receive the transfers, which gives them flexibility to choose someone who already has a National ID; in the Rarieda round we could not do this as we were randomizing recipient gender. I expect the Nike cohort will take longer to register as that project focuses on 18-19 year old women, many of whom will not yet have IDs." GiveDirectly, Updated data (March 31, 2012)

• 265

  "For its standard Kenya and Uganda programs, GiveDirectly has implemented a rule in Segovia to block payments until the previous one has been confirmed: for example, first and second lump sum payments are blocked until token and first lump sum payments, respectively, have been confirmed via a follow-up call or visit." GiveWell's non-verbatim summary of a conversation with Ian Bassin and Piali Mukhopadhyay, GiveDirectly, August 23, 2016, Pg 4.

• 266
  • Paul Niehaus, GiveDirectly Founder, email to GiveWell, November 20, 2012.
  • Email from Paul Niehaus, President, GiveDirectly, and Joy Sun, COO, Domestic, GiveDirectly, November 18, 2013

• 267
  • GiveDirectly, Monthly operations report, February 2016.
  • GiveDirectly's website notes that, in Kenya, "On average, it takes us 9 weeks from enrolling a recipient to starting his or her transfers." GiveDirectly, Performance - Quality of Service, September 2016. We are not sure over which time period this figure was calculated.

• 268
  • The hotline records also document the steps that GiveDirectly has taken to follow up on recipients' concerns; we have reviewed some of these cases and find the steps taken to resolve issues to be reasonable and appropriate.
  • GiveDirectly, Kenya hotline log, July 2013
  • GiveDirectly, Follow-up tracker, July 2013
  • GiveDirectly, Follow-up tracker, October 2014
  • Data from GiveDirectly indicates that on average 11% of recipients in Kenya campaigns are delayed due to registration issues. GiveDirectly, Monthly operations report, August 2015

• 269

  "The main downside to both of the mobile money services in [Uganda] as compared to MPESA in Kenya is that there are fewer mobile money agents in the rural areas that GiveDirectly is targeting. In response, GiveDirectly has been more proactive in coordinating with the mobile money service for the transfers that have begun, for example, by giving the mobile money service advanced notice before sending the funds so that agents could be prepared. In some cases, agents traveled to the villages in which recipients live to reduce recipient travel time." Conversation with Paul Niehaus, President, and Joy Sun, COO, Domestic, GiveDirectly, July 18, 2013

• 270

  Carolina Toth, email to GiveWell, September 14, 2015

• 271

  GiveDirectly, Monthly operations report, February 2016.

• 272
  • GiveDirectly, Monthly operations report, February 2016
  • As of August 2015, GiveDirectly had 60 staff: 3 Field Directors, 3 Field/Finance Managers, 5 Associate Field Managers, 46 Field Officers, and 3 other staff (@GiveDirectly, Monthly Operations report, August 2015@), meaning that it has grown by an additional 50% in just 6 months.

• 273

  GiveDirectly has so far received about six applications for every one FO [Field Officer] position it has open, which it sees as an indicator that the necessary talent is available for it to scale its operations. Conversation with Paul Niehaus, President, and Joy Sun, COO, Domestic, GiveDirectly, July 18, 2013 (unpublished)

• 274
  • Paul Niehaus, GiveDirectly Founder, conversation with GiveWell, October 22 2012.
  • GiveDirectly, Budget summary, July 2013
  • $12 per day seems very roughly to be in line with market value:
    • $12 per day * 5 days a week * 52 weeks per year = $3,120 per year
    • This salary site indicates that lower-skilled workers are paid ~20,000 - 50,000 KES per month ($198 - $497 per month, according to Google as of May 5, 2016), which comes out to $2,376 -$5,964 per year.
    • We would guess that GiveDirectly's Field Officer position is generally lower-skilled (e.g., it involves significant surveying of recipients, which we'd expect to be paid similarly to other types of administrative assistant roles).

• 275

  Conversation with Piali Mukhopadhyay, COO, International, GiveDirectly, October 22, 2013

• 276

  GiveDirectly has told us that it seeks to influence both the official development assistance that high-income countries provide and individual donor contributions. Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 277

  Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016

• 278

  Some of that evidence must be kept confidential. Note that we have not vetted the examples GiveDirectly has provided.

• 279

  GiveDirectly, Update for GiveWell, May 2015 (the slide with details of the examples mentioned has been redacted).
  GiveDirectly, Update for GiveWell, September 2015 (two slides with details of the examples mentioned have been redacted).

• 280

  Note that this includes Good Ventures: see this blog post.

• 281

  Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016

• 282

  For example.

• 283

  For example:

  • "Survey of 31 ongoing studies found that 6 have a cash arm currently and 20 would like to add one."
  • "GD declined to participate in impact evaluation of cow distribution; study will proceed, may or may not include a cash arm"
  • "GD declined to pursue implementation of nutrition benchmarking study, but will provide advice."
  • "Discussing multi-country comparison of current conflict & jobs programming to cash transfers, using matching funds"
  • "[W]e are seeing growing momentum behind cash transfers"

  GiveDirectly, Update for GiveWell, September 2015, pgs. 5-6.

• 284

  For example: "Indonesian government and World Bank ([Redacted]). WB pushing for an RCT comparing cash to other approaches with conditional funding from DIV and GDL. Pending Indonesian government’s buy-in. Motivated by GD model, GD visited to present on impacts & methods" GiveDirectly, Update for GiveWell, September 2015, pg. 6.

• 285

  Ian Bassin and Carolina Toth, email to GiveWell, June 14, 2016

• 286

  For example: "Indonesian government and World Bank ([Redacted]). WB pushing for an RCT comparing cash to other approaches with conditional funding from DIV and GDL. Pending Indonesian government’s buy-in. Motivated by GD model, GD visited to present on impacts & methods" GiveDirectly, Update for GiveWell, September 2015, pg. 6.

• 287

  For example: the Brookings Blum Roundtable and a DFID high-level panel on cash transfers. GiveDirectly, Update for GiveWell, September 2015, pg. 6.

• 288

  This is based on GiveDirectly's description of how the project started. Paul Niehaus and Carolina Toth, conversation with GiveWell, May 28, 2015

• 289

  GiveDirectly has told us that, although comparing the cost-effectiveness of the programs involved in the Rwanda benchmarking experiment is part of the experiment, doing so is challenging, in part because one of its partner organizations may not have high quality data on its expenses. Challenges like this make us unsure how likely it is that even experimentation designed to be policy-relevant will end up impacting funders. Paul Niehaus, Carolina Toth, and Ian bassin, conversation with GiveWell, August 12, 2016

• 290

  Paul Niehaus, Carolina Toth, and Ian Bassin, conversation with GiveWell, February 23, 2016

• 291

  GiveWell, GiveDirectly financials - 2016, "2016-efficiency ratio" sheet.

• 292

  Carolina Toth, conversation with GiveWell, November 12, 2015

• 293

  GiveWell, GiveDirectly financials - 2016, "2016-efficiency ratio" sheet.

• 294

  Carolina Toth, email to GiveWell, November 10, 2015 We believe the points apply to our more recent estimates as well.

• 295

  GiveDirectly sent us its July 2015 - February 2016 financial documents with a different breakdown than we have worked with before, so have not combined the data into one comprehensive summary.

  • GiveWell, GiveDirectly financials 2015 Sheet: "2015-Total spend + efficiency." Note that the incurred "direct grants to households" includes transfers that GiveDirectly has already officially committed to, but not yet sent.
  • GiveWell, GiveDirectly financials - May 2016 Sheet: "2016 May - Total expenses"

• 296

  For example, the National Institutes of Health funded Innovations for Poverty Action to conduct the RCT of GiveDirectly's program: "GiveDirectly is conducting a longer-term evaluation to provide more detailed, context-specific evidence on how its recipients use cash transfers. The study is coordinated by an external research organization, Innovations for Poverty Action, led by Dr. Johannes Haushofer of Harvard University, and funded by the National Institutes of Health." GiveDirectly, Offering Memorandum (January 2012), Pg 26.

• 297
  • Conversation with Paul Niehaus, President, and Joy Sun, COO, Domestic, GiveDirectly, October 16, 2013
  • GiveDirectly, GW scratch sheet
  • In late 2015, GiveDirectly had set aside roughly $2.4 million in reserves. GiveWell, GiveDirectly financials 2015 Sheet: "2015-Balance Sheet."
  • Note that in budget year 2016, GiveDirectly decided it only need $1 million in reserves. See GiveWell, GiveDirectly financials - May 2016 Sheet: "Budget and RFMF"

• 298

  GiveWell, GiveDirectly financials - 2016 Sheet: "2016-efficiency ratio."

• 299Includes Core Operations and Core Operations-general. Excludes fundraising.
• 300

  GiveWell, GiveDirectly financials - 2016 Sheet: "2016-efficiency ratio."

• 301

  "We are actually aiming to make Rwanda’s retail program more efficient. We are making two changes to achieve this: (1) we are eliminating token payments. Instead, the first payment issued will be the first lump sum, after which we will do follow up calls to ensure proper receipt; (2) we are eliminating backcheck. In its place, we are flagging any discrepancies between census and registration for individual audits and then, on top of that, adding an additional randomized selection of HHs for audit until we achieve 40% of hhs for audit. We think this will be more efficient while still ensuring accuracy and avoiding fraud. We may eventually migrate these changes to other countries but are starting in Rwanda." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 11, 2016

• 302
  • See our 2016 cost-effectiveness model.
  • GiveDirectly's internal analysis predicted that the basic income study will be 61% as cost-effective as the standard cash transfer campaigns. GiveDirectly, UBI cost-effectiveness estimate

• 303

  Note that GiveDirectly estimates that the efficiency of the basic income program will be similar to that of GiveDirectly's standard program:

  • "We expect efficiency for the UBI program to be modestly worse than our typical program, averaging across the 3 arms to 84% (v. ~90% typically).
  • Long-term UBI:
    • Transfers per recipient are larger for the long-term arm (~$3K). We expect nominal enrollment costs per adult recipient to be modestly more than for our typical program (but to be a smaller %).
    • FX costs maintain the same percentage, and mobile money costs are moderately worse (because smaller withdrawals are charged higher %s).
    • We expect to incur roughly the same follow-up costs as our lump sum program for each year of the long-term UBI. As a result, follow-up costs make up a larger proportion of the overall costs of the project than they do typically.
  • The short-term and lump sum arms are less efficient than the long-term arm, because they incur equal nominal enrollment costs, and comparable % follow-up, management, and transfer fees, relative to a total transfer amount that is lower.
  • We will refine these estimates as we pilot the project in one village this month and finalize our workplan for the full study enrollment."

  Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 8, 2016

• 304

  Ian Bassin, edits to GiveWell's review, November 10, 2016

• 305

  This is also because GiveDirectly wants to structure payments such that the experiment is as informative as possible for its partners. GiveDirectly believes this strategy is most likely to lead to policy impact. Paul Niehaus, Carolina Toth, and Ian bassin, conversation with GiveWell, August 12, 2016

• 306

  GiveDirectly, Room for funding update for GiveWell, October 2016, Pg 6.

• 307
  • Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 17, 2016
  • GiveDirectly, Room for funding update for GiveWell, October 2016, Pg 6. Note that GiveDirectly had already raised $9.5 million of the $17.5 it expected as of October 2016 (see Pg 3).
  • GiveDirectly, Room for funding update for GiveWell, October 2016, Pg 6. It is our understanding from previous conversations with GiveDirectly that even if it does not raise the full amount it would like for its basic income study ($30 million total), it will move ahead with the project with the funding it has available, though the power of the study will not be as high.

• 308
  • We expect GiveDirectly to raise about $6 million in unrestricted funding available for transfers in 2017 over the rest of its budget year (note our estimate differs from GiveDirectly's).
    • GiveDirectly's estimate: "How much would GD raise through 2/28 absent any recommendation from GW:
      • We estimate absent GW we would raise ~$6.1m in retail revenue through 2/28. Of that, $3m is slated to be transferred before 2/28, leaving us with $3.1m in available funding for 2017.
      • We got this figure by trying to estimate how much of our revenue during the 2015 holiday period came from non-GW related sources, removing a large outlier, and then applying our current rate of YOY growth to project how much that figure is likely to be in 2016. We'd be happy to walk through those figures in detail if helpful.
      • It’s also likely that some donors previously influenced by GW would continue to donate to GD even absent a future GW recommendation. We do not have an estimate for this figure so the true number we would raise would likely be somewhat higher."

      Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016

    • GiveWell's estimate:
      • At the end of February 2016, GiveDirectly had about $29.2 available for its standard cash transfers (excluding institutional funders' funding) :
        • $0.2 million was restricted to Kenya.
        • $8.3 million was restricted to "flexible funding" that must be used for cash transfers.
        • $49.5 million was unrestricted. Of that, $25 million was set aside for partnership activities and fundraising, and $1.7 million was set aside for Global Management costs, and another $2.1 was set aside for fundraising, leaving $20.7 million available for standard cash transfers.
        • 0.2 + 8.3 + 20.7 = 29.2.

        GiveDirectly, Update for GiveWell, February 2016 Pgs 5-6.

      • Our understanding is that this was raised in ~September 2015 to end of February 2016 (since funds raised before that were all scheduled to be committed to households before then). We add $1.8m to this to get an estimate of the total GD raised in that period because it looks like about that much was allocated to non-transfer expenses in that period (changes in reserves, global management, and fundraising). Total raised estimate: ~$31 million.
      • The amount we tracked as due to GiveWell in that period, including Good Ventures ($9.8m), two large funders ($9m), and and others ($3.2m): $22m
      • So, the amount that GiveDirectly raised excluding GiveWell-influenced funds: ~$9m ($31 - $22)
      • We then exclude a large $3m donation that GiveDirectly told us was one-off and assume 50% growth over last year: ~$9m [(9-3)*1.5)] GiveDirectly, Room for funding update for GiveWell, October 2016, Pg 6.
      • Finally, GiveDirectly told us that it intended to spent $3 million of what it raised over giving season on cash transfers, bringing us to $6 million (9-3): "We estimate absent GW we would raise ~$6.1m in retail revenue through 2/28. Of that, $3m is slated to be transferred before 2/28..." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016
  • We estimate that GiveDirectly will raise about $9.8 million in unrestricted funding between March 2017 and August 2017.
    • GiveDirectly's revenue between March 2016 and July 2016 was about $12.5 million. Excluding GiveWell-influenced donors ($0.5 million) and grants for partnership projects we believe GiveDirectly received in that time period ($4.5 million and and $2 million), GiveDirectly raised $5.5 million.
    • $5.5 million over 5 months implies that GiveDirectly raises about $1.1 million per month.
    • So, between March 2016 and August 2016, we estimate that GiveDirectly raised about $6.6 million in funding for its standard cash transfer campaigns.
    • Assuming a 50% increase in fundraising next year, we estimate that GiveDirectly will raise $9.8 million (6.6*1.5) between March 2017 and August 2017.

    GiveWell, GiveDirectly financials - 2016, "2016 - Commitments" sheet.

  • $6 million + $9.8 million = $15.8 million.
  • In the past, GiveDirectly has raised significantly more than we expected. For example:
    • In fall 2015, GiveDirectly predicted that it would raise $4 million in retail donations during the 2015 giving season (Sep 2015 - Feb 2016). GiveWell, GiveDirectly financials 2015, Sheet: "2015-RFMF scenarios"
    • Excluding grants from Good Ventures and a grant that GiveDirectly had informed us was expected, it raised roughly $12.2 million in donations. GiveDirectly, Revenue by referral source 2015
    • The $12.2 million figure excludes donations in February; in April 2015, GiveDirectly told us that it raised an additional $1.5 million in February - also more than it expected. Carolina Toth, email to GiveWell, May 3, 2016
    • Note: If GiveWell were to continue recommending GiveDirectly and if GiveWell donations to GiveDirectly grew at approximately the same pace they have grown in previous years, then GiveDirectly would expect to raise $11 million through February 2017. GiveDirectly, Room for funding update for GiveWell, October 2016, Pg 6. Subtract the $17.5 million for the basic income study from $28.5 million raised to arrive at $11 million.

• 309

  This is based on our records of how much we influenced to GiveDirectly last year, when our main recommendation, after accounting for grants from Good Ventures, was to give to the Against Malaria Foundation.

• 310
  Conversation with GiveDirectly, October 6, 2014

• 311
  • GiveDirectly, Room for funding update for GiveWell, October 2016
  • GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 312

  See this spreadsheet for our analysis.

• 313

  You can also see our analysis here.

• 314

  GiveDirectly, Room for funding update for GiveWell, October 2016, Pgs 2 and 4.

• 315

  "$20m and $11m: In both these cases, the damage would potentially be more severe... When successful companies lay off 15% of staff it makes news. Even companies our size make news when they lay off 10% of staff. A contraction like this would require a 33-50% reduction in our staff."
  Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016

• 316
  • "$30m: ...This would look similar to the $36m scenario we discussed on the call, but one team would operate at half capacity. It would take roughly half a year to scale back up from a hit this size." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016
  • Our guess about the countries GiveDirectly would operate in and at what capacity is based on GiveDirectly staff, conversation with GiveWell, October 6, 2016.

• 317

  Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016

• 318

  "$30m: ...This would look similar to the $36m scenario we discussed on the call, but one team would operate at half capacity. It would take roughly half a year to scale back up from a hit this size." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016
  

• 319

  Two full teams in Kenya, one full team in Uganda, half a team in Rwanda (because the Rwanda team's capacity is partially being used on the Rwanda benchmarking project), and half a team (location unknown) managed by team leads that GiveDirectly expects will be occupied by partnership projects in the latter part of the year. GiveDirectly staff, conversation with GiveWell, October 6, 2016.

• 320

  The 5.5 teams would include: Two full teams in Kenya, two full teams in Uganda, half a team in Rwanda (because the Rwanda team's capacity is partially being used on the Rwanda benchmarking project), and, collectively, a full team (location not known) managed by team leads that GiveDirectly expects will be occupied by partnership projects in the latter part of the year. GiveDirectly staff, conversation with GiveWell, October 6, 2016.

• 321
  • "$20m and $11m: In both these cases, the damage would potentially be more severe... When successful companies lay off 15% of staff it makes news. Even companies our size make news when they lay off 10% of staff. A contraction like this would require a 33-50% reduction in our staff. We would expect the news to be public and could spur a "what went wrong for GiveDirectly?" narrative." Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016
  • GiveDirectly told us that it intends to try to end its 2017 budget year with more funding on hand for retail cash transfer campaigns that it has in the past, because its strategy of trying to commit as much to its retail campaigns as it has available leaves it highly dependant on how much it can raise during giving season each year. GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 322

  "What would an aggressive stretch scenario look like in which we'd have a 95% chance of hitting a non-funding related barrier before accomplishing it in full:

  • While it is hard to estimate what growth rate we'd only have a 5% chance of achieving given the many uncertainties (e.g., when and how smoothly the USAID new countries come online, how quickly we can source new team leads, whether any fail to work out after a trial period, etc.), we would have low confidence in our ability to hire 4x our current retail team, or 16 team leads.
  • 16 more teams at 12m each would mean an additional $192m in retail capacity on top of the $66m we presented that we could move in our max scenario ($258m in retail capacity in total, and just over $300m when you add in partnerships).
  • We'd give ourselves a 5% chance of being able to do that if funding were not a constraint."

  Ian Bassin, COO, Domestic, GiveDirectly, email to GiveWell, October 14, 2016

• 323

  See this spreadsheet.

• 324

  We want GiveDirectly to be in a position where it can scale relatively quickly, up to the point where it can transfer $100 million within two years if necessary (i.e., in such a case where our other top charities have limited room for more funding). We believe that at $15 million, GiveDirectly could scale up to $40 million within a year. After scaling to this level, we believe GiveDirectly would be in a good position to potentially hit $100 million the following year.

• 325

  $1.78m/month x 12 months = $21.4m/year. GiveWell, GiveDirectly financials - 2016, "2016-Commitments" sheet.

• 326

  It is our understanding that GiveDirectly has been operating with approximately 3 full teams this year. GiveDirectly, Room for funding update for GiveWell, October 2016, Pg 2.

• 327

  We assume the cost of delivering transfers is an additional 10%; this is what GiveDirectly has suggested we use in the past.

• 328

  We assume that GiveDirectly will have 4 teams for the latter half of the year because it recently started its standard cash transfer campaign in Rwanda. $29 million over 7 months is $4.1 million per month. That's slightly over $1 million per team per month, or a rate of $12.4 million per team per year.

• 329

  GiveDirectly staff, conversation with GiveWell, October 6, 2016.

• 330

  See GiveDirectly, Room for funding update for GiveWell, October 2016, Pg 2. In 2015, GiveDirectly transferred $15.9 million after transferring $10.3 million in 2014, scaling by a factor of 1.5. If GiveDirectly successfully transfers $38.4 million in 2016, it will have scaled by a factor of 2.4.

• 331
  • GiveWell, GiveDirectly financials - May 2016 "2015-Rate of money moved" and "2016 May - Commitments + revenue" sheets.
  • GiveWell, GiveDirectly financials - 2016 "2016-Commitments" sheet.

• 332

  GiveWell, GiveDirectly financials 2015, Sheet: "2015 - Rate of money moved." See chart for "Committed and distributed transfers to date."

• 333

  GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 334
  • Paul Niehaus, Michael Faye, and Piali Mukhopadhyay, conversation with GiveDirectly supporters, August 11, 2015
  • In the past, GiveDirectly has noted that it has a pipeline of several Field Directors whom it could hire if additional Field Directors were needed. It is easy to find additional Field Managers, Assistant Field Managers, and Field Officers in Kenya and Uganda because unemployment rates are high, and many qualified candidates are looking for jobs. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015
  • GiveDirectly has had more difficulty building up its domestic capacity. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 335

  GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 336

  GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 337

  "GiveDirectly is also meeting with senior or retired government officials who can provide guidance on navigating the government and connect GiveDirectly to allies on the public sector side." Conversation with GiveDirectly, April 8, 2014, Pg 11.

• 338

  "By now, GiveDirectly understands well the process for seeking government approvals in Kenya and does not see acquiring approvals as a major risk." Conversation with Piali Mukhopadhyay, GiveDirectly, October 20-21, 2014, Pg 3.

• 339

  GiveDirectly staff, conversation with GiveWell, October 6, 2016. As of late 2015, GiveDirectly had obtained permissions to enroll a cumulative capacity of about 100,000 households across Kenya and Uganda:

  • "Bukedia district [Uganda] still has 27,000 un-visited, eligible households, and country-wide registration is in process that will provide approval for all 7.3m households in Uganda." GiveDirectly, Update for GiveWell, May 2015, Pg 3.
  • "County-level government approval equivalent to ~70 K additional eligible households in Siaya and Homa Bay counties" GiveDirectly, Update for GiveWell, May 2015, Pg 3.
  • GiveDirectly already has approval to work in Bukedia district in Uganda, so even if it did not obtain country-wide permission in Uganda, it would still be able to work in Uganda for a long time. Most of the households that GiveDirectly has permission to enroll in Kenya are in Homa Bay County, although GiveDirectly still needs to speak to some of the districts in Homa Bay to get approval at a more local level. GiveDirectly could also go back and enroll the houses that are controls in the General Equilibrium study in Siaya County once that study is complete. Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 340

  GiveDirectly noted that it could go to some counties that were slightly further away from its current offices and that it was planning to open a new office in Kenya, which would allow it to enroll additional households in new areas. In Uganda, GiveDirectly has secured permissions to work in the district where the coffee RCT is taking place, so it could easily expand there and enroll more households for its standard cash transfer campaign if needed. GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 341
  • Political violence and terrorism are both risks in Kenya. Western Kenya has not been impacted since 2008 election violence
  • Operations in Uganda provide an alternative, and funds could be shifted more heavily toward Uganda

  GiveDirectly, Update for GiveWell, October 2014, Pg 16.

• 342
  • "Dependency on a single provider: low risk. In Uganda, we have Ezee Money and Airtel as a back-up, and in Kenya could use Equity Bank."
    GiveDirectly, Update for GiveWell, July 2014 Pg 12.
  • "Selected MTN as preferred provider in Uganda after assessing performance of Ezee/MTN (building relationship with Airtel so as to have an additional hedge)"
    GiveDirectly, Update for GiveWell, July 2014 Pg 9.
  • GiveDirectly continues to seek other mobile money providers who it might want to partner with for future campaigns in Uganda (e.g., Airtel).
    Conversation with GiveDirectly, April 8, 2014 Pg 5.

• 343

  Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015

• 344

  GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 345

  GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 346

  GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 347

  "GiveDirectly has already begun to interview candidates to lead its marketing activities and expects to hire someone for the position by the end of 2015. It expects the rest of the marketing team (3-5 people) to be hired soon after the lead is on board." See here.

• 348

  "GiveDirectly plans to use part of the grant from Good Ventures to hire a vice president of marketing. Progress on finding the right person has been slow, and GiveDirectly has not yet hired anyone for this position. (Update: in September 2016, GiveDirectly hired Matt Johnson, former CMO of Tough Mudder and VP of marketing at Seamless, for this role.) GiveDirectly recently began to work with a recruiting firm to assist with the hiring process."
  GiveWell's non-verbatim summary of a conversation with Paul Niehaus, Carolina Toth, and Ian Bassin, August 12, 2016, Pg 1.

• 349

  There will be a national election in Kenya in 2017. GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 350

  There will be a national election in Kenya in 2017. GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 351

  There will be a national election in Kenya in 2017. GiveDirectly staff, conversation with GiveWell, October 6, 2016

• 352

  "They created GiveDirectly as a private giving circle in 2009 and opened it to the public in 2011 after two years of operational testing." GiveDirectly, FAQs 2015
  GiveDirectly, Team
  Paul Niehaus, GiveDirectly Founder, email to GiveWell, November 20, 2012

• 353

  Conversation with Paul Niehaus, November 14, 2014

• 354

  Paul Niehaus and Carolina Toth, conversation with GiveWell, September 7, 2015