Evidence Action's No Lean Season - November 2017 Version

We have published a more recent review of this organization. See our most recent report on No Lean Season.

Published: November 2017; Last Updated: January 2018

Summary

What do they do? Evidence Action's No Lean Season (evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the time of seasonal income and food insecurity ("lean season") between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on an adult household member stating their intent to temporarily migrate to urban or other rural locations to seek short-term employment. The program is implemented by RDRS Bangladesh (http://www.rdrsbangla.net/). Evidence Action (https://www.evidenceaction.org) provides strategic direction for No Lean Season, mobilizes resources, provides technical assistance, conducts program monitoring independent of RDRS, and coordinates partnerships on the program. (More)

Does it work? We believe there is strong evidence for the positive impact of seasonal migration subsidies on household income and consumption. The program has been evaluated in several randomized controlled trials (RCTs) in Bangladesh, and an additional RCT is ongoing. We have remaining questions about whether the monitoring processes that Evidence Action is developing will be able to detect ways in which program implementation differs from the program as implemented in the trials. (More)

What do you get for your dollar? We estimate that the program will cost No Lean Season, on net, about $18 per participating household in 2018-2020. This estimate relies on a number of uncertain assumptions. Our cost-effectiveness analysis estimates the impact per dollar of the program on household income. (More)

Is there room for more funding? We believe that No Lean Season is very likely to be constrained by funding. Over the next three years, we expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive in that time to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and to do further research in Bangladesh and other locations. Funding beyond this level would allow Evidence Action to build reserves for No Lean Season. (More) December 2017 update: In November 2017, we recommended that Good Ventures give $11.5 million to Evidence Action’s No Lean Season program, which we expect to fill its room for more funding for its work in Bangladesh over the next three years.

No Lean Season is recommended because of its:

• Focus on a program with a strong evidence base and strong cost-effectiveness.
• Standout transparency – it has shared significant, detailed information with us.
• Room for more funding – we believe No Lean Season will be able to use additional funds to scale and implement its program in northern Bangladesh and to explore whether its model would be effective in other contexts.

Major open questions include:

• We have somewhat limited information on how the program will be monitored to answer questions such as whether loans are targeted at low-income households, whether funds reach the intended households in full, whether recipients are pressured into participation, and whether participants find work when they migrate. We expect to see additional data from RDRS and Evidence Action in the future, but note that we put less weight on monitoring plans than on a demonstrated track record of producing high-quality monitoring. Our confidence in Evidence Action's plans is increased by its track record of producing high-quality monitoring for the Deworm the World Initiative, another GiveWell top charity.
• The program as implemented in 2017 and going forward differs in some ways from the program as evaluated in previous RCTs (more). For example, some of the RCT evidence is from programs in which subsidies were given as grants rather than loans and only a portion of eligible households were invited to participate. We expect the 2017 RCT to provide additional evidence on the impact of the program in its current form.

Our review process

To date, our review process has consisted of:

• Making several grants to No Lean Season as part of our work to support the development of potential future GiveWell top charities (now known as GiveWell Incubation Grants). For more information, see our reports on a March 2014 grant to No Lean Season and several subsequent grants.
• Conversations with Evidence Action staff who work on No Lean Season and with Mushfiq Mobarak, a lead researcher on the studies that have informed our view of the No Lean Season program.1
• Reviewing documents No Lean Season sent in response to our queries.
• In September 2017, GiveWell visited No Lean Season in Bangladesh.

All content on No Lean Season, including past grants, blog posts, and conversation notes, is available here.

What do they do?

No Lean Season provides no-interest loans to poor rural households between planting and the major rice harvest in rural northern Bangladesh. This annual period, from approximately September to November, is often the time of greatest income insecurity and food insecurity for poor households (the "lean season").2 The No Lean Season loan is contingent on an adult member of the household engaging in temporary labor migration and is intended to increase employment opportunity, income, and household consumption. The loan amount of 1500 Bangladeshi taka (approximately $18.70 USD) is intended to be sufficient for round-trip transportation and to set aside some money for family members staying behind.3

No Lean Season was developed and is managed by Evidence Action, which provides strategic direction and technical assistance, mobilizes resources, conducts independent monitoring, and coordinates partnerships. The program is implemented by RDRS Bangladesh. The program is fairly new. The first randomized controlled trial (RCT) of conditional subsidies for seasonal labor migration in northern Bangladesh was conducted in 2008. Additional RCTs were conducted in 2011, 2013, and 2014; there is an ongoing RCT beginning in 2017.4 The program operated on a small scale in 2016 (out of 15 RDRS branch offices)5 and grew in 2017 to 50 branch offices.6 The No Lean Season program has only operated in and been evaluated in Rangpur Division.7

See also our report on conditional subsidies for seasonal labor migration in northern Bangladesh.

This section describes the roles played by organizations involved in the program, details how the program is implemented, and describes how funding for the program is spent. Note that when we refer to program activities in a particular year, this refers to the implementation of the program targeting the lean season in the fall of that year, with program activities extending into approximately February of the following calendar year. For example, loan recovery for the No Lean Season program in 2017 extends into March 2018.

Partners and staffing

No Lean Season is implemented by RDRS Bangladesh and most staff who work on No Lean Season are employed by RDRS. Evidence Action developed No Lean Season and plays a range of ongoing roles in the program, described below. Innovations for Poverty Action conducts evaluation activities, of which Yale professor Mushfiq Mobarak is the lead Principal Investigator.

RDRS Bangladesh

No Lean Season is implemented by RDRS Bangladesh, a microfinance and development agency that works primarily in Rangpur Division in northwest Bangladesh.8 As of March 2016, RDRS operated in every upazila (subdistrict) of every district in Rangpur Division and operated 171 total local branch offices.9 RDRS is expanding both within and beyond Rangpur: ten new branch offices were opened in 2016, three of which were in Rangpur Division.10 No Lean Season plans to scale to additional existing RDRS branch offices in future years, and may also scale to additional RDRS branch offices as they open.11

RDRS Bangladesh provides the operational infrastructure for the No Lean Season program in Bangladesh, and hires, trains and manages field staff.12 RDRS also conducts process and performance monitoring of the No Lean Season program.13

Evidence Action

Evidence Action is developing No Lean Season as a part of its Beta department (more below). Evidence Action leads the strategic direction and program management of No Lean Season, provides technical assistance, mobilizes resources, and conducts program monitoring independent of RDRS. Evidence Action also coordinates research partnerships for ongoing experimental evaluation of the program and for further optimization of the program design. Two Evidence Action staff members are based in Bangladesh and work full time on No Lean Season.14 In 2017, Evidence Action hired a team of part-time monitors. (See below for more on Evidence Action's monitoring of No Lean Season.) Evidence Action is in the process of exploring whether the program could scale in other contexts (more).

Evidence Action was created out of Innovations for Poverty Action (IPA) in 2013 with a mission to scale cost-effective and evidence-based programs. Two IPA initiatives, the Deworm the World Initiative (one of GiveWell's top charities) and Dispensers for Safe Water, were spun off from IPA to be managed by Evidence Action.15 It has since built a department for investigating, testing, and considering new programs for scaling up called Evidence Action Beta;16 one program in the Beta portfolio is No Lean Season. We focus this review on No Lean Season and discuss the room for more funding implications of No Lean Season being a program of a larger organization below.

More on Evidence Action as an organization below.

Staffing

RDRS employs the following types of employees for the No Lean Season program:

• Migration Organizers are the No Lean Season staff who are primarily responsible for interacting with program participants at each stage of the program. Two Migration Organizers are assigned to each RDRS branch office that is delivering the No Lean Season program, and divide the targeted villages in that office's catchment area between them, such that each Migration Organizer is, on average, responsible for six villages per year.17 Migration Organizers deliver the program in several phases: conducting village surveys and household surveys, informing prospective migrants about seasonal migration in the local lean season, making loan offers to eligible households, assisting in loan disbursement, monitoring migration compliance of the loan recipients, and following up with households to recover loans.18 Migration Organizers participate in RDRS-led training prior to each phase of the program, from approximately September to December of each year.19

  Evidence Action Beta's monitoring plans include monitoring of the effectiveness of staff training and of staff's knowledge of how to implement the program, by administering a phone-based test to a sample of the trainees.20 We have not evaluated the details of training materials, the protocols for the phone-based evaluations, or results from phone-based evaluations to date; instead, we focus below on overall program effectiveness.

  RDRS employed 30 Migration Organizers in 2016 and 102 in 2017.21 21 (70%) of the 2016 Migration Organizers were retained in 2017.22 See footnote for more details on RDRS's hiring process.23

• Monitoring Officers. In addition to Migration Organizers, RDRS hires, trains, and manages a smaller number of Monitoring Officers. Monitoring Officers monitor Migration Organizers' performance, including field observations of work, training replacement staff, answering questions and providing technical support.24
• Management staff. RDRS field management, i.e. Branch Managers, Area Managers, and Program Managers, provide overall management support for the program. RDRS has a dedicated Project Manager for No Lean Season. RDRS senior management staff such as the Director of Microfinance and four Microfinance Coordinators also provide support for the program.25

Evidence Action employs the following types of employees for the No Lean Season program:

• Global support staff. Several Evidence Action Beta staff members based outside of Bangladesh provide full and part-time strategic, management, technical, and monitoring support for No Lean Season.26
• Monitoring staff. At the time of GiveWell's 2017 site visit to No Lean Season, Evidence Action had hired a team of nine part-time staff to monitor the quality of program implementation by RDRS. All of these staff are masters students in local Bangladeshi universities.27 We describe Evidence Action's monitoring plans for each phase of program implementation below.
• Local management and support staff. In 2016, Evidence Action Beta employed one employee in Bangladesh working full-time on No Lean Season.28 In 2017, another full-time staff member was hired to lead independent monitoring efforts and otherwise support the project.

Mobile phone data management system

No Lean Season field staff are provided with a mobile phone and trained in use of a mobile program implementation and data collection application.29 The mobile application includes modules for each stage of the program (described below): targeting surveys (village survey and household survey), invitations to offer meetings, making loan offers to households, loan applications and disbursement, migration compliance monitoring, loan repayment, and the migration debrief survey.30 The mobile application includes scripts and checklists to remind Migration Organizers of key information and procedures to communicate to program participants, as well as answers to Frequently Asked Questions based on field experiences to date.

Evidence Action has backend access to the mobile application's database, allowing it to observe program implementation and program statistics in close to real time.

When there are problems with the mobile data collection application, Migration Organizers use pre-developed paper forms as a backup.31

Implementation phases of the program

In brief, the program involves the following stages:

• Targeting: Selection of villages and households within selected villages, based on proxies for level of poverty
• Offer: Communications with eligible households to explain the conditions of the loans (no interest and conditional on stated intent to migrate by an adult member of the household)
• Disbursement: Loan disbursement to the interested eligible households
• Migration Compliance Monitoring: Visits to households to determine whether they have sent an adult migrant and whether the migrant found work at their destination
• Recovery and debrief: Loan recovery from returning successful migrants and administration of a migration debrief survey

Details follow.

Targeting: Village selection

RDRS begins No Lean Season's annual implementation cycle by compiling a list of all villages within an hour's bicycle travel of each participating branch office.32 RDRS Migration Organizers survey a village elder in each village in order to assess the program's viability in the village and to establish the exact location of the village with GPS coordinates.33

No Lean Season plans to annually operate in one third of the villages in each RDRS branch office's catchment area, rotating which villages are targeted each year, so that each village is reached once every three years.34 This is based on evidence that migration subsidies can have an effect on re-migration for several years without further subsidization (more). In 2017, No Lean Season operated out of 50 RDRS branch offices, serving about 600 villages.35

No Lean Season estimates that there are an average of 35 villages per branch office catchment area. As approximately one-third of villages in a catchment area are served per year, No Lean Season serves approximately 12 villages per RDRS branch office per year.36 As there are two Migration Organizers assigned to each branch office, each Migration Organizer serves approximately six villages per year.

Targeting: Household eligibility

Households are eligible to participate if they report owning less than a certain amount of land or having regularly skipped meals in a period just before the survey.37

Household eligibility census

A complete household census is conducted by Migration Organizers in all villages that are selected to be reached by the program in a given year. The household census aims to identify which households will be eligible to receive a loan offer.38 The 2017 census collects data from an adult household member39 regarding (a) land ownership and (b) missed meals in a period just prior to the survey.40 We have seen the 2016 survey protocols for this survey, as well as unpublished survey results from 2016.41

Evidence Action monitoring staff verify the quality of this census by re-visiting a random42 sample of households and administering a Household Targeting Verification Survey to check the accuracy of the data from the original census conducted by No Lean Season Migration Organizers. In 2017, Evidence Action re-surveyed approximately 0.4% of households.43 Evidence Action has told us that this sample size has been determined to meet 95% confidence level and 5% margin of error. We have seen preliminary results of the 2017 Household Targeting Verification Survey,44 which we discuss below.

Determination of household eligibility

Results from the household eligibility census are processed by Evidence Action staff.45 Data is cleaned for blank entries, duplicates, and entries missing data.46

In 2017, households were eligible for the No Lean Season program if they reported owning 50 or fewer decimals of cultivable land or reported any incidence of skipped meals in the two weeks prior to the survey.47 These eligibility criteria are somewhat different from the eligibility criteria used in the studies which inform our view of the intervention (more).

Based on preliminary data from the 2017 household eligibility census, Evidence Action estimates that approximately 77% of surveyed households are eligible for the program.48 In the 2014 RCT, about 57% of censused households were eligible for the program, using the eligibility criteria of that study.49

Based on the compiled list of eligible households, Evidence Action adds each eligible household to a database, identified by a unique household ID, village name, and name of the head of household. Each of these households is a "case file" in the mobile application, which is updated through future interactions with the household. It is not possible for Migration Organizers to create or delete households through the mobile data collection application, or to edit a household's identifying information (unique household ID, village name, and household name).50

Offers to participate in the program

Migration Organizers inform eligible households about the No Lean Season program, extend offers for participation in the program to each eligible household, and record the household's level of interest in the program. The offer process proceeds in several stages: eligible households are invited to a public meeting about the program, the program is explained at this public meeting, and Migration Organizers follow up with eligible households who did and did not attend the offer meeting in order to extend the offer and to gauge interest in the program.

Invitation to an offer meeting

After being assigned a list of eligible households, each Migration Organizer travels to their assigned villages and visits every household on their list. The purpose of this visit is to inform households of their eligibility for the program and to invite them to attend a public group meeting about the program.51 If an adult household member is not present, the Migration Organizer is expected to revisit the household at a later time.52

Offer meetings

Offer meetings are held in public locations and are open to the public. One hour before the scheduled start time of a meeting, Migration Organizers are instructed to revisit households invited to this meeting and remind them of the scheduled meeting.53

At the beginning of each offer meeting, the Migration Organizer uses the mobile application to take a photo of the meeting as additional verification that the meeting took place. The Migration Organizer takes attendance by calling the names of the up to 50 households assigned to this offer meeting. A household is marked as attending the offer meeting if any member of the household over 18 years of age is present at the meeting.54 Migration Organizers are instructed to wait to begin the meeting until at least 70% of the invited households are present at the meeting.55

The Migration Organizer leading the meeting explains the purpose of the No Lean Season program.56 Migration Organizers are trained to base their verbal presentation on an offer meeting script, which communicates that migration subsidies are offered as interest free conditional loans, and that the condition for receiving the money is a commitment to temporarily migrating in the coming lean season. The script also includes discussion of the risks and benefits associated with temporary migration.57 No Lean Season has shared with us the full English text of the No Lean Season 2017 Offer Meeting Script. Additionally, at the offer meeting Migration Organizers are instructed to inform attendees of the scheduled loan disbursement days at their local RDRS branch office.58

After describing the program, Migration Organizers publicly answer any questions that meeting attendees may have.59 In particular, Migration Organizers are trained to communicate carefully about the limited liability conditions of the loan. Migration Organizers are permitted to say that if a migrant dies or is seriously injured or ill while migrating, they will not be required to pay back the loan. However, Migration Organizers are not permitted to say that if a migrant fails to find work while migrating, they will not be required to pay back the loan; they are instructed to answer more vaguely that such cases will be reviewed on a case-by-case basis by RDRS staff.60 More on communicating about loan forgiveness here.

At the end of the offer meeting, the Migration Organizer asks each household assigned to the meeting whether, at this time, they accept the loan offer, are not interested in participating in the program, or are potentially interested in the program, and records the response in the case management application.61 The script also instructs Migration Organizers to encourage eligible households to discuss the offer with their friends and neighbors and to note that they will be revisited at a later time to ask if they would like to accept the offer.62

Offer meetings are monitored in the following ways:

• RDRS Monitoring Officers attend a sample of offer meetings. RDRS managerial staff, such as Branch Managers and the No Lean Season Project Manager, also attend some offer meetings.63
• Evidence Action monitoring staff attend a sample of offer meetings and track whether Migration Organizers convey key pieces of information.64
• Evidence Action monitoring staff conduct a post-offer-meeting survey asking about attendees' comprehension of the program and perception of any pressure involved in the offer meeting. The goals of this exercise include monitoring Migration Organizers' compliance with the offer script and attendee's perceptions and comprehension.65 Evidence Action monitoring staff sample meeting attendees both from meetings that were observed by monitoring staff and from meetings that were unobserved by monitoring staff.66

GiveWell staff observed a portion of an offer meeting while on a site visit in 2017.67

Post-meeting household offers

After the offer meeting stage of the program, Migration Organizers revisit all eligible households: those that attended the offer meeting and those that did not. If a household states that they are interested in the program, but not ready to accept the offer at this time, or if a household states that they are not interested in the program, the Migration Organizer is instructed to make the offer to the household a total of three times (including the offer meeting if the household attended). If a household does not accept to participate in the program after the third offer, they are not revisited again that year, and their case management file is closed.68

Evidence Action's current monitoring plans do not include activities designed to monitor Migration Organizers' post-meeting household-level interactions.69

Subsidy disbursement

Migration Organizers fill out an electronic loan application form70 for every household which accepts the subsidy offer. The first part of the form consists of basic and identifying information, including taking a photograph of the applicant and of the applicant's identity documentation,71 and is filled out at the household.72 Our impression is that this generally occurs during the household re-visit when the household accepts the offer, as described above.

According to the offer script, Migration Organizers explain the loan disbursement process to households as follows:73

• Loans are disbursed from the local RDRS branch office. Evidence Action has clarified that for villages located further than 12 kilometers from a branch office, disbursements are arranged to take place in the village. (Villages eligible for the program were selected on the basis of being less than one hour's bicycle ride from an RDRS branch office.)74
• There is a window of opportunity for collecting the loan disbursement: in 2017, Evidence Action has told us that the loan availability period has been extended through the end of December.
• The person intending to migrate must themself apply to receive the loan disbursement, must be over 18 years of age, and must have identification. We are not certain to what extent the identification requirement for loan disbursement is a barrier to uptake. Our impression is that possession of some acceptable form of ID (such as a national ID card, a birth registration certificate, or a voter number) is common in the target population.75
• Households are asked to notify the Migration Organizer one day in advance of traveling to the branch office to receive the loan, so that RDRS can manage the supply of funds and volume of visitors.
• Our understanding is that RDRS branch offices may offer No Lean Season loan disbursements on only some days of the week.76

Three RDRS staff members are involved in disbursement at the branch office: the Migration Organizer, the Branch Manager, and an accountant. During our visit to the program in 2017, we observed the following process for loan disbursement:

• The Migration Organizer works with the loan applicant to fill out the second part of the loan application on the Migration Organizer's smartphone.77 This includes verifying the applicant's identity, and asking the applicant to report their expected migration destination.78 The RDRS Branch Manager observes every loan application, and the Migration Organizer, the Branch Manager, and the loan applicant must all sign the electronic loan application form on the smartphone.79
• The Migration Organizer provides some advice on migration to the loan applicant.80
• The loan applicant proceeds to an RDRS accountant, who manages a paper registry book. Our understanding is that maintaining this paper registry book is a legal requirement.81 A duty stamp (record of tax paid) is also legally required on records of debt in Bangladesh; for No Lean Season loans, this duty is 10 taka.82 The accountant collects the loan applicant's signature or fingerprint in the registry book alongside the record of the date, loan amount disbursed, and duty stamp.83 Then, the accountant disburses the loan funds (1500 Bangladeshi taka) to the applicant.84

The amount of cash disbursed as part of the No Lean Season program is tallied by each branch office and aggregated up to the area office level and then to the district (unit) office level.85

For villages that are farthest from an RDRS branch office, the No Lean Season program may coordinate a loan disbursement meeting in the village.86 No Lean Season told us that, in 2017, 1% of villages qualified for a loan disbursement meeting (because they were further than 12 kilometers from a branch office).87

Evidence Action observes disbursements by making random, unannounced visits to branch offices during the disbursement period. These visits seek to verify that Migration Organizers are adhering to procedures, and to interview recipients to assess their understanding of loan conditions.88 As of this writing, we have not seen results of these monitoring activities, as they are ongoing. Evidence Action’s goals for the loan disbursement verification survey are to verify the proportion of travel subsidy recipients who:89

• attended an offer meeting
• come from eligible households
• report receiving the correct amount of cash and who match the photo/ID in RDRS records
• are 18 years of age or above
• can accurately relay information related to the subsidy and loan repayment

Migration compliance monitoring: verification and post-migration survey

Migration Organizers verify program participants' migration status via regular household visits. First, Migration Organizers ask approximately three neighbors about the migration status of a household; then, Migration Organizers visit the household.90

When a household visit indicates that a migrant has returned from migration and that they do not intend to migrate again this year, the Migration Organizer administers a migration debrief survey. The survey asks specific questions about the migrant's ability to find work during their first migration episode this season, as well as their overall migration experience and earnings this season. If a household did not send a migrant, the Migration Organizer records the reasons they report for not migrating.91

Evidence Action also plans to monitor migration compliance by surveying a sample of households. This monitoring plan is still under development, and may involve surveying both eligible and ineligible households in order to understand migration throughout the village. Evidence Action may also conduct this migration debrief survey in villages not reached by the intervention in a given year in order to assess the rate of re-migration in the absence of an additional subsidy.92

Loan recovery and loan forgiveness

Program participants are asked to repay the loan amount within one week of returning from migration.93 After confirming that a program participant has returned from migration, Migration Organizers make household visits in order to remind them of the need to repay within a week and to collect the repayment. Migration Organizers provide the household with a signed repayment acknowledgement receipt for each repayment.94

Migration Organizers are told that loans may be repaid in up to three installments;95 our understanding is that this is conveyed to program participants if they ask.96

No Lean Season loans are to some degree limited liability; our understanding is that this is conveyed to program participants if they ask.97 According to Evidence Action, the following are non-exhaustive examples of circumstances under which loan forgiveness is considered and granted:98

• The prospective migrant was unable to travel because of illness, injury, death, or other catastrophic event.
• The migrant traveled but did not find work.
• The migrant traveled, found work, but earned less than twice the amount of the loan.
• The migrant traveled but incurred a serious accident or illness that restricted them from working.99

RDRS staff discussed with us the challenges of communicating about loan forgiveness.100 Our understanding is that Migration Organizers do not have the authority to grant loan forgiveness. Rather, Evidence Action has told us that Migration Organizers collect information supporting why the participant requests loan forgiveness and the decision is made centrally.101 We are uncertain about the details of how management decides whether or not to grant loan forgiveness. In 2016, 97.81% of loan capital was repaid,102 suggesting that loan forgiveness is rare, but it is unclear if this is due to high rates of migration success, lack of knowledge among recipients about the possibility of loan forgiveness in certain circumstances, possible pressure by Migration Organizers, or branch management denying loan forgiveness applications. Whether or not the loan is forgiven, the Migration Organizer will cease to visit that household for loan repayment after the end of the program cycle, at which point outstanding loan balances are written off as unreceivable.103

It is our understanding that Migration Organizers are on salary and do not receive financial incentives to achieve higher repayment rates.104 We have not seen information on whether Migration Organizers who achieve higher repayment rates are evaluated more favorably or are more likely to be retained.

Evidence Action plans to conduct a loan recovery verification survey to monitor loan repayment rates, Migration Organizers' adherence to the protocol, and any pressure reported by program participants.105 We have not seen the questionnaire for this monitoring survey.

Breakdown of No Lean Season's spending

We summarize No Lean Season's 2016 spending in GiveWell's summary of No Lean Season's 2016 spending.106

In short:

• No Lean Season spent approximately $930,000 in 2016.
• Of that spending, approximately 19% was loan capital, 73% was spent on program management and implementation, and 8% was spent on ongoing research into the program's effectiveness.
• Staffing costs made up a little less than half of management and implementation spending.

We note that 2016 was an early-stage version of the No Lean Season program. We therefore expect this cost breakdown to change as the program scales. One significant difference we anticipate in future years is that repaid loan capital will be "rolled over" to support the program's implementation in the subsequent year.

We have seen budget projections of No Lean Season's spending for 2017 and future years.107

Does it work?

We discuss the evidence for the impact of conditional subsidies for seasonal labor migration in northern Bangladesh in a separate report. We believe that there is strong evidence that conditional subsidies for seasonal labor migration in the region in which No Lean Season operates increase household income and consumption during the lean season in which participants receive subsidies and for 1 to 3 years following.

Additionally, the 2017 cycle of the program is being evaluated with an RCT, for which preliminary results are expected in 2018.

The program has produced limited independent monitoring data to date. We have seen 2016 data from RDRS on what proportion of loan recipients migrated, what proportion of migrants found work, and what wages migrants received. Evidence Action is currently developing and piloting monitoring procedures and has shared fairly detailed plans with us. We have seen initial results of Evidence Action's monitoring of 2017 household targeting. Given the limited monitoring data to date, we have remaining questions about whether the monitoring processes that Evidence Action is developing will be able to detect ways in which the program could fail to have the expected impact or have a negative impact.

What is the evidence for the effect of conditional subsidies for seasonal labor migration on household income and consumption?

We discuss the evidence for the impact of conditional subsidies for seasonal labor migration in northern Bangladesh in a separate report. We believe that there is strong evidence that conditional subsidies for seasonal labor migration increase household income and consumption during the lean season in which participants receive subsidies. We also believe there is fairly strong evidence that the program causes higher rates of migration in the 1 to 3 years following program participation, without further subsidies, and that migration in subsequent years increases household consumption. This conclusion is based on three randomized controlled trials (RCTs) conducted in the region in Bangladesh in which No Lean Season operates, two of which found an effect on income or consumption.

How is the program different from the intervention studied?

As described in our report on conditional subsidies for seasonal labor migration, we rely on the following studies to inform our view of the effectiveness of the intervention: A 2008 RCT with follow-up in 2009, 2011, and 2013; an unpublished 2013 RCT about which we have relatively little information; a 2014 RCT with follow-up regarding remigration in 2015; and a forthcoming 2017 RCT. In this section, we discuss how similar the program is to the intervention in these studies, with an emphasis on the 2008 and 2014 RCTs, the published studies taking place in the lean season. We believe that the No Lean Season program is highly similar in context and operation to the intervention in the key studies that inform our evaluation.

• Geographic context. All of the studies that we rely on took place in Rangpur Division, Bangladesh.108 No Lean Season predominantly plans to scale up within Rangpur Division, though it may plan to scale up in Bangladesh outside of Rangpur in the future.109 Our understanding is that there is a seasonal decline in consumption for low-income households throughout Rangpur Division.110 We are uncertain about the heterogeneity of relevant factors within Rangpur Division, such as the degree of seasonality in consumption, or the baseline prevalence of seasonal migration. We have not examined the village-level heterogeneity of the RCT results.
• Eligibility criteria. The eligibility criteria in the studies that inform our evaluation were somewhat different from the eligibility criteria used in the No Lean Season program; the eligibility criteria of the No Lean Season program are strictly more inclusive.111
• Subsidy type and amount. No Lean Season offers a conditional no-interest loan of 1500 taka (approximately 18.70 USD).112 The 2014 study offered a conditional cash transfer of 1000 taka (approximately 13.00 USD).113 The 2008 study included a study arm in which households were offered a conditional cash transfer of 600 taka (approximately 8.50 USD) and a study arm in which households were offered the same amount as a conditional no-interest loan; in both arms, migrants could earn an additional 200 taka (approximately 3 USD) for checking in with study staff at their migration destination, for a total subsidy of 8.50 - 11.50 USD.114
• Targeting intensity. In each village where No Lean Season works, the program offers a conditional no-interest loan to all households which meet the eligibility criteria. In the 2014 study, one "low-intensity" study arm offered conditional cash subsidies to approximately 14% of the eligible population, while another "high-intensity" arm offered conditional cash subsidies to approximately 70% of the eligible population.115 In the 2008 RCT, the intensity of treatment was similar to the low-intensity 2014 treatment (approximately 14% of the eligible population).116 The results from the 2014 RCT using this variation in treatment intensity suggest that the program's effect on migration rate per offer increases with treatment intensity, but that the program's effect on total income per household offered is not higher at higher treatment intensity.117
• Additional conditionalities. In the 2008 experiment, some households were assigned a migration destination or a migration group.118 This is not the case in the No Lean Season program.
• Implementing partner. No Lean Season is implemented by RDRS Bangladesh. RDRS was the sole implementing partner for the 2014 RCT.119 The 2008 RCT was implemented by a number of NGOs, including RDRS.120

Is No Lean Season effectively targeting people who meet its criteria?

Households eligible for the No Lean Season program in 2017 are those households which report land ownership of 50 decimals or less or report any incidence of skipped meals in the two weeks prior to the household targeting census conducted by Migration Organizers.121

Evidence Action's monitoring plans include employing independent monitors to survey a sample of households in a sample of villages targeted by the program in order to verify the completeness and accuracy of the Household Targeting Survey implemented by RDRS.122 On our 2017 site visit we spoke with these monitors, who had recently conducted this monitoring activity.123 We have seen preliminary results of this monitoring for 2017.124 This monitoring activity provides evidence as to how effectively No Lean Season targets people who meet its criteria:

• Does the Household Targeting Survey reach every household? The Household Targeting Survey is intended to be a complete village census and to collect data relevant to No Lean Season's eligibility criteria from every household.125 Evidence Action's monitoring provides verification of the comprehensiveness of the Household Targeting Survey by randomly sampling households in a village independently from the list of households visited by RDRS Migration Organizers,126 thereby estimating the proportion of households in a village that have been missed by the Household Targeting Survey. According to preliminary verification survey results, out of 748 randomly selected households visited by monitoring staff, 747 (99.9%) had also been captured in the Household Targeting Survey. It is possible that particularly hard-to-reach households would be skipped by both the Household Targeting Survey and the verification survey.127
• Are households' self-reports of land ownership and skipped meals accurate? Evidence Action's verification survey provides some check on the reliability of households' self-report of land ownership by comparing households' responses in the Household Targeting Survey to responses in the verification survey. According to preliminary results of the verification survey, 64% of households in the verification survey reported exactly or nearly exactly the same amount of owned land as they reported in the Household Targeting Survey.128 Because Evidence Action's verification survey takes place at a later date, and reported incidence of skipped meals may be prone to recall error, it is not feasible to verify the consistency of households' reported incidence of skipped meals with this verification survey. We also expect that it is infeasible to verify households' land ownership or incidence skipped meals in a way that does not rely on self-reports.
• How reliable is the determination of program eligibility? Evidence Action's verification survey provides some check on the reliability of the determination of program eligibility. Based on preliminary results of the verification survey on self-reported land ownership, discrepancies in recorded self-reported land ownership affected the eligibility status of 8% of households reached in the verification survey: 5% were deemed ineligible according to the Household Targeting Survey but would have been eligible based on the verification survey ("false negatives") while 3% were deemed eligible according to the Household Targeting Survey but would have been ineligible based on the verification survey ("false positives").129

We note that Migration Organizers, who conduct the surveys that determine eligibility, do not decide which households are eligible.130 RDRS told us that Migration Organizers are not told the program's eligibility criteria.131 Migration Organizers collect household data in the household eligibility survey. Evidence Action staff then analyze the data and create a list of eligible households that is sent back to Migration Organizers.132

Do loans reach the people targeted?

In the majority of cases, loans are disbursed at RDRS branch offices, and three RDRS staff are present during all disbursements. For villages that are far from a branch office, RDRS organizes disbursement days in the village; we do not know how these are supervised.133

We think it is fairly likely that we would find out about cases in which intended loan recipients do not receive the funds or receive the incorrect amount. Evidence Action plans to conduct household loan disbursement verification surveys, which will ask participating households whether they received a maximum of one loan of the correct amount and verify that the recipient matches the photo and ID in RDRS's records.134 We have not yet seen this questionnaire or any results indicating whether beneficiaries report receiving their loans (as this piece of monitoring is underway as of November 2017).

We are less confident that we would learn about cases in which Migration Organizers or other RDRS staff collect more than the loan amount in loan repayments, for example, by failing to record in the system that the loan has been repaid and asking recipients to repay the loan twice, or telling migrants that they are expected to repay the loan with interest.

Do recipients understand the conditions of the loan? Are they pressured into accepting or repaying the loan?

Evidence Action plans to collect the following monitoring data on whether beneficiaries understand the terms of the loan and whether they are pressured into accepting or paying back the loan:

1. Observing a sample of offer meetings and interviewing participants from eligible households to check whether they understand the key information about the loan and to monitor for coercive practices by migration organizers.135 We have not yet seen this questionnaire or any results. Considering that Migration Organizers may change their behavior when Evidence Action monitors are present at an offer meeting, Evidence Action's post-meeting survey samples households from observed offer meetings and from unobserved offer meetings.136
2. Making random, unannounced visits to branch offices during the disbursement process to verify that the loan disbursement procedures are being followed and to interview recipients to determine whether they have an accurate understanding of the terms of loan repayment.137
3. Surveying a sample of households to verify repayment rates and to monitor whether recipients felt pressured into paying back the loan.138

Our understanding is that Migration Organizers are on salary and do not receive financial incentives to achieve higher take-up or repayment rates.139 We have not seen information on whether Migration Organizers who achieve higher take-up or repayment rates are evaluated more favorably or are more likely to be retained.

A 2016 small focus group study commissioned by Evidence Action found that some beneficiaries reported feeling some pressure to repay the loan (including disliking the frequency of visits from Migration Organizers), but no one in the sample reported being coerced to accept the loan.140 Other respondents reported positive experiences with loan repayment.141 Some respondents had been briefed by RDRS staff prior to focus group discussions.142 We put limited weight on the results of this small study.

Do people who receive loans successfully migrate and find work?

RDRS conducts a migration compliance survey to determine whether people who received loans have migrated143 and surveys migrants after they return to measure the percentage of migrants who are able to find work at their destination.144 In 2016, 87% of loan recipients migrated and 99.2% of recipients who migrated were able to find work, according to data from RDRS.145 RDRS reported following up with over 99% of households for the 2016 lean season.146 Migrants reported an average daily wage of 385 taka. The 25th percentile wage was 300 taka and the 75th percentile was 450 taka.147 No Lean Season has shared the questionnaire for this survey and the underlying data;148 we are not aware of any audits that were done to check the accuracy of the 2016 data.

These numbers may be inflated due to reporting bias; since stated intent to migrate is a condition of the loan, recipient households have an incentive to falsely report that someone migrated. On the other hand, some migrants may feel more inclined to report unsuccessful migration attempts in order to seek loan forgiveness. RDRS has told us that as a check, Migration Organizers ask up to three neighbors whether the loan recipient migrated; we are uncertain about the details of this procedure.149 RDRS and Evidence Action staff told us that neighbors are very likely to know whether a loan recipient has migrated; it seems possible to us that, since loan conditions are likely well-known in the village, neighbors might falsely report that a loan recipient has migrated.

Evidence Action plans to also survey households on migration compliance as a check on the data it receives from RDRS. This monitoring plan is under development and involves surveying both eligible and ineligible households in order to better understand migration throughout the village.150 Evidence Action also plans to conduct this migration debrief survey in villages not reached by the intervention in a given year in order to assess the rate of re-migration in the absence of an additional subsidy.151

Are people who migrate exposed to greater physical and other risks during migration than at home?

We have seen limited information addressing this question. The migration debrief survey that RDRS Migration Organizers conduct after the migrant returns home includes questions about whether the migrant experienced an illness, injury, or family emergency that prevented them from migrating or working.152 We have seen data from the 2016 migration debrief survey.153

Evidence Action's monitoring plans include loan recovery verification activities which aim to verify repayment rates and to check that households were not unduly pressured into paying back the loan, as well as migration debrief verification, which includes questions about the migrant's migration experience.154 We have not seen the questionnaires for these surveys and we have not seen how Evidence Action specifically plans to ask about negative experiences during migration.

Are there additional negative or positive effects of the program?

We discuss possible negative and positive effects of the program in a separate report. In short, we speculate that there may be additional impacts of the program, but we have little or no evidence about these possible additional impacts.

What do you get for your dollar?

In this section, we discuss the cost per participating household for the No Lean Season program.

See our most recent cost-effectiveness model for more detail on how we evaluate the cost-effectiveness of No Lean Season, and for how the program's cost-effectiveness compares to that of our other top charities. We discuss major judgement calls or uncertainties in our model here.

Cost per participating household

We estimate that, on average in 2018-2020, the net cost per loan made by No Lean Season will be $18.44.155

There are a number of sources of uncertainty in this estimate:

• When possible, we prefer to rely on past results over projected future results, as projections can be overly optimistic. For No Lean Season, we have used projected costs and numbers of households reached because our understanding is that the cost structure of the program in the past was considerably different from what is expected in the future.156
• No Lean Season's projections assume that 90% of the loan capital from each cycle will be repaid and available for the following cycle. If the loan repayment rate falls in the future, net costs would increase.
• No Lean Season's projections assume that 77% of households in selected villages will be eligible for the program and 60% of those eligible will choose to participate. Lower eligibility and participation rates could lead to higher costs per participating household as Migration Organizers could need to visit additional villages to enroll the same number of households or as fixed costs of the program could be spread across fewer households.
• Some research costs may be incurred by other organizations. We have not yet determined the amount of such costs or how to account for them in order to be consistent across our top charities.

Is there room for more funding?

We believe that No Lean Season could productively use more funding than it expects to receive and that is it very likely to be constrained by funding.

In short:

• Total opportunities to spend funds productively: No Lean Season projects that it could use up $11.9 million for implementing and monitoring the program in Bangladesh in 2018-2020, and an additional $4.5 million over that period to conduct additional research on the program and explore the possibility of expansion to other countries.
• Cash on hand: Evidence Action expects to have $400,000 available for No Lean Season in Bangladesh for 2018. In addition, we project that RDRS will have repaid loan capital available for 2018, which has been subtracted from the projected costs of the program. Evidence Action expects to have $657,750 available in matching funds for research outside of Bangladesh in 2018.
• Expected additional funding: Evidence Action told us that it has limited fundraising prospects for supporting No Lean Season in the next few years.
• Additional considerations: Below, we also discuss implications of No Lean Season being a program within the larger Evidence Action organization.

In sum, we expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Funding beyond this level would allow Evidence Action to build reserves for No Lean Season.

Details in this spreadsheet.

December 2017 update: In November 2017, we recommended that Good Ventures give $11.5 million to Evidence Action’s No Lean Season program, which we expect to fill its room for more funding for its work in Bangladesh over the next three years.

Available and expected funds

To date, Evidence Action has received funding for No Lean Season from the Global Innovation Fund (GIF; one grant for $2.5 million), Y Combinator (one-off grant of $100,000), and GiveWell-directed funds. It expects to have $400,000 from the GIF grant remaining for supporting the program in Bangladesh in 2018. If it raises matching funding, it expects to receive $657,750 from GIF for a pilot and RCT in Indonesia.

RDRS will also hold some funding at the end of the 2017 cycle from repaid loans that will be available for funding the program in 2018. The budget for the program in 2018-2020 projects that $1.4, $1.7, and $2.8 million, will be available in 2018, 2019, and 2020, respectively, from the previous year's loan repayments.

Evidence Action told us that it has only speculative or early-stage prospects for raising funding for No Lean Season from sources other than GiveWell-directed funding. No Lean Season has not yet been invited to apply for additional funding from GIF and, given the length of the application process, it is unlikely that additional funding from GIF will be available for the 2018 and 2019 cycles in Bangladesh. Evidence Action told us that there are a few other institutional funders that could potentially be interested in funding No Lean Season, but that it is not in active conversations with any funders currently. It is also interested in fundraising from individual donors, but does not yet have specific plans for this work.157

Uses of additional funding

No Lean Season estimates that it could use up $11.9 million for its work in Bangladesh in 2018-2020, and an additional $4.5 million over that period to conduct additional research on the program and explore the possibility of expansion to other countries.

We ask top charities to consider GiveWell-directed funds to be multi-year grants. The amount of GiveWell-directed funding that a top charity receives can vary greatly from year to year, and spending the funds over two to three years can help smooth these fluctuations.

Details of how No Lean Season expects to spend additional funding are in this spreadsheet, sheet "Spending opportunities." In short:

• No Lean Season projects that it could scale to 92,000 loans disbursed in 2018 ($2.5 million total budget), 151,000 in 2019 ($3.7 million), and 235,000 in 2020 ($5.7 million), up from an expected maximum of 84,000 in 2017 ($2.7 million). Evidence Action notes that it does not expect to expand No Lean Season in Bangladesh significantly in 2018 because it would like to see the results of the 2017 cycle RCT and spend time setting up an office in the country.158
• At $11.9 million for its work in Bangladesh, No Lean Season would budget $470,000 for research on the program, including completing an RCT begun in 2017. No Lean Season notes that this budget would "include all monitoring and evaluation costs related to assessing the path-to-scale and ongoing delivery-at-scale of the program in Bangladesh" and exclude "costs that are related to exploration of, adaptation to, and expansion into other geographies" and "costs that are related to broader learning agendas about the theory behind the program."159
• If No Lean Season received sufficient funding for its work in Bangladesh, it would use additional funding for "identifying and exploring new geographic contexts [including pilot programs and monitoring and evaluation], global policy engagement and communications, [and] broadening the donor base in anticipation of future growth."160 It estimates that it could use up to $4.5 million for this additional work in 2018-2020. December 2017 update: In November 2017, we recommended that Good Ventures give $11.5 million to Evidence Action’s No Lean Season program, which we expect to fill its room for more funding for its work in Bangladesh over the next three years.

The total of the above is $16.4 million.

In addition, No Lean Season – and Evidence Action more broadly – would like to increase its reserves. Evidence Action has drawn down some of its reserves in the past year, and aims to hold more funding in reserve to protect against underestimation of costs and overestimation of revenue from other sources.161

We believe that there is some risk that No Lean Season will encounter limitations to its ability to scale, other than funding. For example, the ability of the program to scale successfully is dependent on RDRS's capacity and ability to hire high-quality staff. Our understanding is that this is relatively unlikely to be a limitation. We note that No Lean Season's plan is to scale in current RDRS branches only through 2020 (scaling projections for 2021 onward assume that RDRS would expand, opening additional branch offices).

The budget also makes assumptions about the rate of eligible households (77% of all households) and take up in targeted villages (60% of eligible households). We note that in 2017, No Lean Season underestimated the number of households that would be eligible to receive loans, and possibly also underestimated take-up rate. GiveWell recommended funding and a funding guarantee to cover cost overruns to ensure that the program could meet its commitments without drawing down unrestricted funding.162 The budget also assumes that 90% of loan capital from each year will be available for further loans in the subsequent year, based on 98% loan recovery in 2016.163

No Lean Season as a program of Evidence Action

No Lean Season is a program of Evidence Action. Evidence Action supports other programs in addition to No Lean Season, including GiveWell top charity Deworm the World Initiative and GiveWell standout charity Dispensers for Safe Water.164

This has some implications relevant to No Lean Season's room for more funding: donations to Evidence Action, even if restricted to No Lean Season, might change the actions that staff take to fundraise (i.e., which grants they pursue and what type of funding they ask for). We've seen some evidence that this has been the case in the past and provide details on our page with additional information about Deworm the World.

Evidence Action as an organization

We believe that Evidence Action is a strong organization:

• Track record: Evidence Action's track record of funding and monitoring the No Lean Season program is fairly limited. We have separately discussed its track record of implementing the Deworm the World program.
• Self-evaluation: Evidence Action has demonstrated a commitment to evaluating No Lean Season with RCTs—it is supporting an RCT in 2017 to build upon the evidence collected in multiple previous trials. No Lean Season has described fairly detailed ongoing monitoring plans, but we have so far seen limited monitoring from the program. We have separately discussed Evidence Action's commitment to monitoring in the context of its Deworm the World program.
• Communication: No Lean Season staff (Evidence Action staff and RDRS staff) have generally communicated clearly and directly with us, given thoughtful answers to our critical questions, and shared significant, substantive information.
• Transparency: Evidence Action has consistently been strong in its commitment to transparency. It has provided the information we’ve asked for and has not hesitated to share it publicly (unless it had what we felt was a good reason).

More on how we think about evaluating organizations at our 2012 blog post.

Sources

Document	Source
Akram, Chowdhury and Mobarak 2017	Source (archive)
Bryan, Chowdhury and Mobarak 2014	Source (archive)
Evidence Action 2016-2017 revenue and expenditures	Source
Evidence Action memo to GiveWell, 23 October 2017	Unpublished
Evidence Action webpage: The solution: No Lean Season	Source (archive)
Evidence Action webpage: What is Evidence Action Beta?	Source (archive)
GiveWell's 2012 site visit to Dispensers for Safe Water	Source
GiveWell's 2017 site visit to No Lean Season	Unpublished
GiveWell's summary of No Lean Season's 2016 spending	Source
Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh	Unpublished
Information System of the Laws of Bangladesh	Source (archive)
Information System of the Laws of Bangladesh: Bangladeshi Stamp Act of 1899	Source (archive)
Information System of the Laws of Bangladesh: Chapter II section 3 of the Stamp Act of 1899	Source (archive)
Information System of the Laws of Bangladesh: Schedule I of the Stamp Act of 1899	Source (archive)
Innovations for Poverty Action webpage: Evidence Action launch press release	Source (archive)
Khandker 2012	Source (archive)
Khandker and Mahmud 2012	Source (archive)
Mobarak, Reimão and Shenoy 2017	Source
No Lean Season 2013 note about hartals and low migration rates	Source
No Lean Season 2016 Analysis Plan for Selecting Target Households	Source
No Lean Season 2016 Household Targeting Questionnaire	Source
No Lean Season 2016 Loan Application Form	Source
No Lean Season 2016 Migration Debrief Survey	Source
No Lean Season 2016 program costs	Unpublished
No Lean Season 2016 Program Data Codebook	Source
No Lean Season 2016 summary of data	Source
No Lean Season 2016 summary of data (guide)	Source
No Lean Season 2016 Survey Guidelines & FAQ for the Household Targeting Questionnaire	Source
No Lean Season 2017 CommCare Module Translation	Unpublished
No Lean Season 2017 Offer Meeting Script	Source
No Lean Season 2017 Preliminary Monitoring Results: Household Targeting	Source
No Lean Season 2017 Process and Performance Monitoring Framework	Source
No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet	Source
No Lean Season Budget Projections	Source
No Lean Season CommCare Design Graphic	Source
No Lean Season Household Survey Verification Questionnaire	Source
No Lean Season Research Timeline	Source
RDRS Bangladesh 2016 Annual Report	Source (archive)
RDRS Bangladesh webpage: RDRS in Brief	Source (archive)
RDRS Bangladesh webpage: Where We Work	Source (archive)

• 1
  • Notes from a conversation with Mushfiq Mobarak, November 8, 2012
  • Notes from a conversation with Alix Zwane and Mushfiq Mobarak, April 18, 2014
  • Notes from a conversation with Alix Zwane and Mushfiq Mobarak, January 22, 2015
  • Notes from a conversation with Alix Zwane, February 3, 2015
  • Notes from a conversation with Mushfiq Mobarak and Evidence Action staff, October 22, 2015
  • Notes from a conversation with Karen Levy, February 4, 2016
  • Notes from a conversation with Jeff Brown and Karen Levy, February 26, 2016
  • Notes from a conversation with Karen Levy and Guillaume Kroll, August 2, 2016

  We have had additional conversations for which we have not published conversation notes.

• 2
  • "Although the change in the rice crop cycle has helped reduce the seasonal spread in rice prices, the traditional lean season preceding the aman harvest and spreading from September to November has changed little in its characteristics. It is the season of the least crop-related activity with no major crops planted or harvested, as can be seen from the crop calendar shown in figure 3.4. Although the wheat-growing season starts in November, the crop is grown in only about 5 percent of the cultivated land area. Moreover, the aftermaths of natural disasters like floods, drought, or excessive rains are usually felt most severely in this season.

    The cropping patterns in Bangladesh are delicately balanced within the natural cycles of rains and annual floods. Thus, farmers’ production options and perception of risk are often determined by the physical environment: the degree of seasonal flooding, the timing and quantity of rainfall, and the soil characteristics (Mahmud, Rahman, and Zohir 1994, 2000)." Khandker and Mahmud 2012, Pgs 40-41. See especially figure 3.4, "Crop Calendar of Major Crops," which illustrates that no other major crop activities occur while the aman rice crop is in its growing period from September to November.

  • For background on seasonal variation in consumption in rural northern Bangladesh, see Khandker 2012 and Khandker and Mahmud 2012.
  • In particular, Khandker and Mahmud 2012 discuss seasonal patterns of income, expenditures, and poverty on Pgs 43-61, based on the authors' analysis of data from the Household Income and Expenditure Survey of the Bangladesh Bureau of Statistics, showing marked dips in income and consumption in the lean season. See especially data on seasonal employment patterns on Pg 60, showing that household employment in the farm sector varies seasonally from a peak of approximately 17 person-days per month to a low of approximately 8 person-days per month in the lean season.
  • Bryan, Chowdhury and Mobarak 2014, Pg 1676 shows seasonal variation in total expenditures per capita, food expenditure per capita, price of rice, and quantity of rice per household based on the Household Income and Expenditure Survey of the Bangladesh Bureau of Statistics.

• 3
  • "No Lean Season addresses seasonal poverty by offering a migration subsidy of about $20 to participating households, covering the cost of a round-trip bus fare to nearby urban areas that do not experience the same seasonal fluctuations in job opportunities." Evidence Action webpage: The solution: No Lean Season.
  • From conversations with Evidence Action Beta staff, it is our understanding that the costs of transportation may vary and that the loan amount is intended to be large enough to additionally allow migrants to set aside some money for family members staying behind.
  • "Following the success of the loan provision for seasonal migrants, we have piloted the project NLS in 15 branches for 7000-9,000 seasonally-unemployed adults who can borrow Tk 1,500 each on an interest-free basis." RDRS Bangladesh 2016 Annual Report, Pg 49.
  • The loan amount of 1500 taka is built in to No Lean Season Budget Projections, unpublished tab.
  • We use the currency conversion rate 1 USD / 80.22 Bangladeshi taka, as used in No Lean Season Budget Projections, unpublished tab.

• 4

  No Lean Season Research Timeline illustrates the history of research studies related to the No Lean Season project prior to 2017.

• 5

  "In 2016, we decided to work with 15 of 164 Branch Offices (as agreed in the MOU with RDRS). This was an increase from previous years, but a number that felt appropriate in terms of overall budget and implementation capacity." No Lean Season 2016 summary of data (guide), Pg 2.

• 6

  No Lean Season Budget Projections, tab "Budget Projections." "Total # of Branch Offices" for 2017 is 50.

• 7
  • In 2016, No Lean Season operated out of 15 RDRS branch offices "geographically dispersed throughout the target program area in northern Bangladesh." No Lean Season 2016 summary of data (guide), Pg 2.
  • 2014 RCT:
    • "The landless poor supplying agricultural labor on others’ farms are especially affected when demand for agricultural labor falls. They constitute around 56% of the population in our sample area, and are the target of the seasonal migration encouragement intervention that we design. Our sampling frame is representative of this landless population in the Rangpur region of Northern Bangladesh. According to the Bangladesh Bureau of Statistics, there are roughly 15.8 million such inhabitants in Rangpur (BBS, 2011)." Akram, Chowdhury and Mobarak 2017, Pg5.
    • "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur," Akram, Chowdhury and Mobarak 2017, Pg 7.
  • 2008 RCT (and its follow-up surveys): "Our experiments were conducted in 100 villages in two districts (Kurigram and Lalmonirhat) in the seasonal-famine prone Rangpur region of northwestern Bangladesh." Bryan, Chowdhury and Mobarak 2014, Pg 1675.

• 8
  • "Established in 1972 to assist with relief and rehabilitation of greater Rangpur-Dinajpur region immediately following the War of Independence of Bangladesh in 1971, the RDRS program evolved into a sectoral then comprehensive effort. Formerly the Bangladesh field program of the Geneva-based Lutheran World Federation/Department for World Service, RDRS became a national development organization in 1997... RDRS is now a major multisectoral development agency focused on disadvantaged northwest Bangladesh." RDRS Bangladesh webpage: RDRS in Brief
  • "RDRS is unusual is maintaining a concentrated geographic program, focusing on 8 districts and 57 Upazilas (sub-districts) mainly in deprived north-west Bangladesh in Rangpur Division, far from the overcentralized economic and political powerbase of Dhaka, Chittagong, even Rajshahi." RDRS Bangladesh webpage: Where We Work, which also includes a map of areas where RDRS works, showing a geographic emphasis on Rangpur Division.
  • No Lean Season operates under the microfinance arm of RDRS: "No lean season (NLS); Following the success of the loan provision for seasonal migrants, we have piloted the project NLS in 15 branches for 7000-9,000 seasonally-unemployed adults who can borrow Tk 1,500 each on an interest-free basis." RDRS Bangladesh 2016 Annual Report, Pg 49, under the heading Microfinance.

• 9
  • See the table at the bottom of the RDRS Bangladesh webpage: Where We Work. The first eight listed districts are the eight districts of Rangpur Division.
  • See table row "Total Branches," RDRS Bangladesh 2016 Annual Report, Pg 53.

• 10

  "As our Sylhet projects are doing well, we have had to open 7 new branches in this tea-growing area and 3 more branches were opened in Dinajpur." RDRS Bangladesh 2016 Annual Report, Pg 49. Sylhet District, Sylhet Division is in northeastern Bangladesh. Dinajpur District is in Rangpur Division.

• 11

  No Lean Season Budget Projections, tab "Budget Projections" shows projected growth in "Total # of Branch Offices." The number of branch offices in which No Lean Season projects operating by 2022 exceeds 171, the total number of RDRS branch offices as of 2016 (see table row "Total Branches," RDRS Bangladesh 2016 Annual Report, Pg 53), and continues to grow at a rate of 10 per year.

• 12

  See GiveWell's summary of No Lean Season's 2016 spending for a summary of RDRS's 2016 budget, including line items for "Salary and allowances" and other operating costs.

• 13

  "Both process and performance monitoring involve the collection of two interrelated sets of data:

  • Administrative data is collected by the RDRS in the course of implementation. This includes training attendance, household listing, offer attendance, migration compliance checks, loan recovery etc. Some key forms are digitized and collected using electronic data collection.
  • Verification data is collected by Evidence Action from a sample of respondents or site observations in order to confirm that various program process have been performed effectively and in adherence to protocol. The sampling plan, which is still under development, details the sampling approach to be used for verifying each phase of the program and is summarized on the third tab here."

  No Lean Season 2017 Process and Performance Monitoring Framework, Pg 2.

• 14

  We met these two staff members during GiveWell's 2017 site visit to No Lean Season.

• 15

  "Two IPA initiatives that touch millions of people in Africa and Asia - Dispensers for Safe Water and the Deworm the World Initiative - will spin off from IPA to be managed by Evidence Action." Innovations for Poverty Action webpage: Evidence Action launch press release

• 16

  "We identify and pressure-test evidence-based innovations that reduce the burden of poverty, and then design and build the most cost-effective programs for massive scale." Evidence Action webpage: What is Evidence Action Beta?

• 17
  • GiveWell's 2017 site visit to No Lean Season
  • No Lean Season Budget Projections, unpublished tab "Assumptions" shows the distribution of the number of villages per branch office based on 2017 data, for an average of 35 total villages per branch office, or 12 villages targeted per branch office per year.

• 18

  "RDRS’ Migration Organizers deliver the program at the last mile. They conduct village and household surveys, inform prospective migrants about the benefits and risks of migration, extend offers to eligible households and participate in loan recovery." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 2.

• 19
  • "It is critical that Migration Organizers have a strong understanding of how to conduct each of these processes. To that end, Migration Organizers undertake training prior to each phase of the program." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 2.
  • No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "FullFramework" shows that post-training verification surveys are planned to take place at five times between September and December.

• 20
  • "To monitor the effectiveness of the trainings, Evidence Action will conduct phone-based tests after each training session using a sample of Migration Organizers. These tests will enable us to measure the impact of training on Migration Organizers’ knowledge, and to assess the knowledge retention of Migration Officers following training." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 2.
  • No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tabs "FullFramework" and "Sampling plan" show the dates and targets for these knowledge verification surveys. In 2017, Evidence Action aims to sample 41 Migration Organizers (40%) per survey (tab "Sampling plan"), and considers the target for program success to be 90% accuracy in Migration Organizer understanding of key program elements (tab "FullFramework").

• 21
  • "In 2016 there were 30 migration organizers, 21 of whom were invited to continue this year." GiveWell's 2017 site visit to No Lean Season, Pg 16.
  • "In 2017, RDRS hired 102 migration organizers for No Lean Season out of about 1,000 applicants and left roughly one candidate in the reserve pool for every candidate hired." GiveWell's 2017 site visit to No Lean Season, Pg 16.

• 22

  "In 2016 there were 30 migration organizers, 21 of whom were invited to continue this year." GiveWell's 2017 site visit to No Lean Season, Pg 16.

• 23

  "When recruiting for migration organizers, RDRS initially selects more candidates than the number of positions it is trying to fill so that positions can be filled more quickly (vacancies are typically filled within 1-2 days). In 2017, RDRS hired 102 migration organizers for No Lean Season out of about 1,000 applicants and left roughly one candidate in the reserve pool for every candidate hired. The monitoring officer observes each new recruit one-on-one for 1-3 days. If a migration organizer is performing poorly, they will be observed and evaluated.

  So far in 2017, RDRS has replaced seven of the 102 migration organizers it hired this year. Since migration organizers are skilled workers, the usual reason for replacing them is because they leave for a better job. In 2016 there were 30 migration organizers, 21 of whom were invited to continue this year."

  GiveWell's 2017 site visit to No Lean Season, Pg 16.

• 24

  GiveWell's 2017 site visit to No Lean Season

• 25

  Evidence Action provided us with this list of management staff. We have seen RDRS budgets for No Lean Season including salary & allowance line items for Senior Management, a Project Manager, Branch Managers, and Area Managers (unpublished). We met several of these individuals during GiveWell's 2017 site visit to No Lean Season.

• 26

  No Lean Season Budget Projections, tab "EA MgmtEval for BD" includes "Ongoing management and leadership costs at the country, regional, and global levels necessary to support ongoing program delivery and growth as per three year plan. Focus on establishing and building scale at minimum marginal cost in existing context; maximizing impact and efficiency; mitigating risk and minimizing unintended consequences" and "Ongoing communications, donor management, policy engagement, and other operational costs related to the three-year plan in Bangladesh." The table shows to what extent various staffing categories are full-time staff or part-time staff.

• 27

  GiveWell's 2017 site visit to No Lean Season

• 28

  GiveWell's 2017 site visit to No Lean Season

• 29

  No Lean Season uses CommCare (https://www.dimagi.com/products/), an open source mobile data collection and case management platform. In 2017, Evidence Action Beta worked with AgImpact (https://agimpact.org/) to develop a No Lean Season mobile data collection app on the CommCare platform. CommCare was selected for its features such as low reliance on data connectivity and support for the Bengali alphabet. GiveWell's 2017 site visit to No Lean Season

• 30

  No Lean Season CommCare Design Graphic. Although this graphic does not include targeting surveys, Evidence Action has told us that the CommCare mobile application also includes modules for targeting surveys (village survey and household survey).

• 31

  During GiveWell's 2017 site visit to No Lean Season, we heard from several people about a recent CommCare outage lasting approximately three days, in which data collection reverted to the paper system. Sohel Rana, senior Evidence Action staff member in Bangladesh, described this as a good, though unintentional, test of the backup paper system. GiveWell's 2017 site visit to No Lean Season

• 32

  "Village surveys are used to profile and geolocate all the prospective villages in which No Lean Season may operate. RDRS identifies the list of villages within each particular catchment area. A catchment area includes all villages located within an hour’s bicycle distance from a specific RDRS branch office." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 3.

• 33

  "Village surveys: Village surveys are used to profile and geo-locate all the prospective villages in which No Lean Season may operate. RDRS identifies the list of villages within each particular catchment area. A catchment area includes all villages located within an hour’s bicycle distance from a specific RDRS branch office.
  Migration Organizers then conduct a survey of a Village Elder in each village according to predefined guidelines." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 3.

• 34

  "At scale, we estimate that one third of the villages in any particular catchment area will participate in the program each year. The program will rotate to include different villages each year, beginning the cycle again once all villages in a catchment area have been reached." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 3.

• 35

  No Lean Season Budget Projections, tab "Scenarios." For 2017, "Total # of BOs" is 50 and "Total # of villages per year" is listed as 600 with a clarifying note: "Actual 608; this will increase after the survey completes but maybe over the 624 village number because we expanded to 1/2 villages in some branches and we should figure out how to account for that this year and consider whether it will happen in the future."

• 36

  No Lean Season Budget Projections, tab "Assumptions" includes these estimates, as well as a frequency chart showing the distribution of the number of villages per branch office, based on 2017 program data.

• 37

  Evidence Action staff confirmed these 2017 eligibility criteria.

• 38

  "Household surveys are conducted in all the villages that are selected to participate in the program. Household surveys are used to identify which households are eligible to receive the travel subsidies. Household surveys are conducted by RDRS Migration Organizers using the household survey tool." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 3.

• 39
  • We have not seen the 2017 household eligibility census questionnaire. However Evidence Action staff have told us that, like the 2016 household eligibility census, the survey collects data from an adult household member.
  • We have seen guidelines for the 2016 household eligibility census: "You have to ensure that you collect data from a working member or at least from someone who knows most about the working member of the HH. Working members are those that participate in the labor market for wage labor or work/manage their own enterprise (e.g. farm, shop, cottage industry, etc). You, for instance, SHOULD talk to the wife or the elder son in case the working member is not at home. You, however, SHOULD NOT interview under-aged (less than 16 years old) member of the HHs. Also, you MUST NOT talk to neighbor or any other persons to get information on the Question # 9-22. However, you may fill in the first eight (08) questions (until the Question # 8) by asking the closest neighbors/ relatives of the household." No Lean Season 2016 Survey Guidelines & FAQ for the Household Targeting Questionnaire, Pg 3.

• 40

  We have not seen the 2017 household eligibility census questionnaire. However Evidence Action staff have told us that the survey collects data regarding (a) land ownership and (b) missed meals in a period just prior to the survey. We have seen the 2016 household eligibility census questionnaire, and discuss it in the footnote below.

• 41
  • Note that in 2016, this survey was conducted on paper. In 2017, this survey was conducted using the CommCare mobile application.
  • No Lean Season 2016 Household Targeting Questionnaire. Note that in 2016, No Lean Season experimented with alternative eligibility criteria, and thus the 2016 Household Targeting Questionnaire includes a question about past migration.
  • No Lean Season 2016 Survey Guidelines & FAQ for the Household Targeting Questionnaire
  • Results from the 2016 household targeting survey are not publishable at this time.

• 42
  • "For household survey, we will use two stage sampling where we first select the village and then the household. Samples for these events will be selected to meet 95% confidence level (CL) and 5% margin of error (MoE)." No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "Sampling plan."
  • In September 2017, GiveWell spoke with Evidence Action's nine monitoring staff members in Bangladesh. They had recently completed the Household Targeting Verification Survey and described the process for this monitoring activity. Evidence Action randomly selected a sample of households from the full list compiled in the course of the household eligibility census, and communicated this list to the monitoring staff. Monitors were required to begin sampling at each of the randomly selected households, then to follow a "left hand rule" to count houses to the left of the starting house, and to subsequently sample the fifth and tenth houses. Evidence Action monitors that houses are counted as clearly visible buildings (if multiple households live in one structure, they would still be counted as one house for the purposes of counting towards the next house to sample). Evidence Action monitors told us that they are required to sample the first household (the household selected by Evidence Action), and hence will return if no adult household member is present at the time of the survey; they may ask neighbors or children over 12 when an adult household member will return. However, if no one is present in the fifth house, they are permitted to instead sample the sixth house (and so on), and subsequently to begin counting five more houses from that point in order to sample a third house in the sampling series. Evidence Action monitors described to us that when counting left-hand houses, if no more houses remain in a village cluster, the monitor turns and from that point begins counting on their new left side (formerly their right side). Evidence Action monitors told us that if they are unsure whether a house belongs to a village cluster (for example, it is remotely visible from other houses), they will inquire with nearby neighbors, and told us that neighbors are generally clearly aware of which houses are part of their village cluster or not. If an adult household member declines to participate in the survey, monitors move on to sample the next household to the left. One male monitor told us that he had experienced a small number (3 to 4 out of 118) of refusals, because adult household members present during the survey are more likely to be women than men. The female monitors told us they had never experienced refusal to participate in the survey. Evidence Action monitors told us that each survey takes 5 to 6 minutes. GiveWell's 2017 site visit to No Lean Season

• 43
  • "This year, a total of 748 households were randomly sampled for the migration compliance verification survey using a two-stage random sampling, similar to that of the Household Targeting verification survey. In other words, we randomly sample 123 villages and randomly select 6 households per village." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 5.
  • Evidence Action projected that 183,000 households would be reached in the household eligibility census. No Lean Season Budget Projections, tab "Budget Projections," row "Total # of HHs per year" for 2017. We note that this projection may differ from the actual number of households reached with the eligibility census in 2017. 748 / 183,000 = 0.41%
  • We have seen Evidence Action use a denominator of 206,774 households reached in the household eligibility census, which we expect is an update to the projected 183,000 households reached. 748 / 206,774 = 0.36%.

• 44

  No Lean Season 2017 Preliminary Monitoring Results: Household Targeting

• 45

  We have seen an outline of this process from 2016: No Lean Season 2016 Analysis Plan for Selecting Target Households.

• 46

  In 2016, (1) household listings with missing entries for critical eligibility criteria were removed from the dataset and not considered for eligibility and (2) households with duplicate listings were considered eligible if at least one of the listings was eligible:

  "1,381 households (not removing duplicates) had at least one of the three eligibility criteria missing/inputted incorrectly and/or had an issue where the migration history for the listed working members were not provided - these households were removed from the eligibility tests and 32,080 households (not removing duplicates) were considered for eligibility... Removal of duplicate households was addressed after eligibility was determined: For the households with duplicate observations, the eligibility for each observation within one household was checked for consistency - if one household had inconsistent results for eligibility (e.g. one observation states ‘eligible’ and the other states ‘ineligible’ or one observation states ‘eligible’ and the other is ‘missing’), the household was marked as ‘eligible’" No Lean Season 2016 Analysis Plan for Selecting Target Households, Pg 1. Further detail and discussion is available in that document.

• 47

  Evidence Action staff confirmed that these are the eligibility criteria used in 2017.

• 48

  No Lean Season Budget Projections, tab "Budget Projections," column "2017," ratio of "Total # of eligible households" to "Total # of HHs per year" (households targeted by the eligibility survey): 140,061 / 183,000 = 77%. These figures are estimates made by No Lean Season.

• 49

  "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur. We first conducted village censuses to identify all households that would be “eligible” to receive this intervention in each of these villages. A household was deemed eligible if (1) it owned less than 0.5 acres of land, and (2) it reported back in 2008 that a member had experienced hunger (i.e., skipped meals) during the 2007 monga season. We focused on landownership because land is the most important component of wealth in rural Bangladesh, and it is easily measurable and verifiable. We used the second question on skipping meals to avoid professional, non-agricultural households (who may not own much land, but who are comparatively well off). Our census data suggest that about 57% of households in these villages were eligible to receive the intervention after applying these two criteria." Akram, Chowdhury and Mobarak 2017, Pg 7.

• 50

  GiveWell's 2017 site visit to No Lean Season

• 51

  More detail: In order to keep offer meetings at a manageable size, Migration Organizers limit the number of households invited to each offer meeting to 50. Therefore, in some villages multiple offer meetings may be scheduled. As offer meetings are open to the public, and others in the village may be interested to learn about the program, attendance at offer meetings is not limited and may exceed 50 people.

  GiveWell's 2017 site visit to No Lean Season

• 52
  • No Lean Season 2017 CommCare Module Translation, tab "module1_form1" includes an option for the Migration Organizer to mark "Eligible person not available."
  • See also GiveWell's 2017 site visit to No Lean Season.

• 53

  "REMIND the invited eligibles once again just an hour before the meeting," No Lean Season 2017 CommCare Module Translation, tab "module2_form3." We also heard this mentioned during GiveWell's 2017 site visit to No Lean Season.

• 54

  GiveWell's 2017 site visit to No Lean Season

• 55

  "[Note to MO: Make sure that more than 70% of the invited eligibles are present in the meeting before starting the meeting]" No Lean Season 2017 Offer Meeting Script, Pg 1, and No Lean Season 2017 CommCare Module Translation, tab "module2_form1."

• 56
  • See the No Lean Season 2017 Offer Meeting Script.
  • GiveWell's 2017 site visit to No Lean Season

• 57

  "With a list of eligible households available from Evidence Action, RDRS Migration Organizers communicate offers to eligible households using an Offer Script. The subsidies are offered as interest free conditional soft loans. The condition for receiving the money is a commitment to migrating. Migration Organizers extend the offer while communicating the risks and benefits associated with migration. Eligible households who are approached with an offer are free to decline the offer. The Migration Organizers record acceptances and refusals of offers in the offer form." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 3.

• 58
  • No Lean Season 2017 CommCare Module Translation, tab "module2_form2," instructions prior to offer meeting invitations include "Confirm respective DISBURSEMENT DATE from BO."
  • GiveWell's 2017 site visit to No Lean Season

• 59

  Migration Organizers are trained on how to answer frequently asked questions, and some answers are included in the mobile data collection application for reference. See No Lean Season 2017 CommCare Module Translation, tab "module2_form2."

• 60
  No Lean Season 2017 CommCare Module Translation, tab "module2_form2." Translation provided by Evidence Action staff:
  • "Q: What if I die after taking the loan? Or get seriously injured/sick? A: Firstly, we do not wish you to die! However, in such case (god prohibits), you (or your household) will not be asked to pay us back the money."
  • "Q: What if after migration I cannot find work and therefore, cannot earn any money? A: We suggest you to try out at this moment. We hope that you will be able to find work and earn, if you are serious enough. However, we cannot guarantee that you will get work and earn handsome amount through seasonal migration. We will see if you really made a try to work but couldn’t manage a good fortune; we will take measures accordingly then. Now we cannot give you exact information, but we can guarantee that each situation will be processed by RDRS on a case-by-case basis, taking into account your situation and any difficulties you faced by no means you will be harmed (financially) participating in the project. But again, we hope that you will be able to find work and earn after you migrate."

• 61

  At this stage, each household acquires two new properties in the CommCare case management system: offer status, tracking whether the household has accepted, rejected, or expressed interest in the offer, and offer count, tracking how many times the household has been contacted with the offer to participate in the program. See No Lean Season 2017 CommCare Module Translation, tabs "module3" and "module3_form1."

• 62

  "Please take the time necessary to think about this offer, and discuss it with your family and friends. Note that it is common for people to migrate in groups. You may want to talk to other people in your village and other people at this meeting about their plans to migrate. We can wait for your decision until November 30 this year, unless you live in a village that is far from a Branch Office. If that is the case, we will organize another group meeting in your village where you will be able to get the loan instead of having to travel to a Branch Office.

  However, those who have taken a decision already, please tell me your name. I will make sure that we have a faster disbursement for you.

  If you are interested, you are welcome to come to our Branch Office between to receive the loan. Before that, however, I will come back to your village again to fill out a short loan application form for those who are interested." No Lean Season 2017 Offer Meeting Script, Pg 3.

• 63

  GiveWell's 2017 site visit to No Lean Season

• 64
  • "To verify that households have been provided with all the relevant information necessary for making an informed decision about whether to accept the offer, Evidence Action carries out offer meeting observations for a random sample of offer meetings, and then interviews a random sub-sample of participants post-meeting.
    Offer meetings were randomly sampled for the observation survey. During observations, Evidence Action enumerators collect data on whether the Migration Organizer followed protocol and covered key aspects of the script for the offer meeting, as well as collect data on attendance to ascertain who and how many people are attending offer meetings." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.
  • Evidence Action's monitoring plans include visits to approximately 104 offer meetings. No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "Sampling plan."
  • GiveWell's 2017 site visit to No Lean Season

• 65
  • "To verify that households have been provided with all the relevant information necessary for making an informed decision about whether to accept the offer, Evidence Action carries out offer meeting observations for a random sample of offer meetings, and then interviews a random sub-sample of participants post-meeting.
    Offer meetings were randomly sampled for the observation survey. During observations, Evidence Action enumerators collect data on whether the Migration Organizer followed protocol and covered key aspects of the script for the offer meeting, as well as collect data on attendance to ascertain who and how many people are attending offer meetings. Additionally, four eligible households were sampled per Migration Organizers for the post-meeting survey; two participants from the observed offer meeting and two other eligible households that were scheduled to attend another offer meeting, but where the offer meeting is unobserved. Evidence Action uses these surveys to assess offer recipients’ understanding of key information about the subsidy and checks that they were not pressured into accepting the subsidy. In total, this year, Evidence Action randomly sampled 104 offer meetings for observation and 416 eligible households for post-meeting surveys." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.
  • No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "Sampling plan" shows a sample of ˜104 offer meetings and 416 offer meeting participants in support of the verification goals of "Migration Organizers adhere to offer meeting script (e.g. no coercion in taking-up loan offer); Eligible households attend offer meetings; Eligible households correctly understand the offer and loan conditions."
  • When we spoke with Evidence Action monitoring staff, they described selecting two offer meeting attendees, checking against a list of eligible households assigned to this meeting, and replacing them if they did not represent eligible, assigned households. GiveWell's 2017 site visit to No Lean Season
  • Evidence Action has told us that it is also monitoring offer meetings by sampling eligible, invited households on a subsequent day, and asking questions about comprehension and pressure without the Migration Organizer in the near vicinity.
  • As of this writing, we have not seen results of this monitoring.

• 66

  "Additionally, four eligible households were sampled per Migration Organizers for the post-meeting survey; two participants from the observed offer meeting and two other eligible households that were scheduled to attend another offer meeting, but where the offer meeting is unobserved. Evidence Action uses these surveys to assess offer recipients’ understanding of key information about the subsidy and checks that they were not pressured into accepting the subsidy." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

• 67

  GiveWell's 2017 site visit to No Lean Season

• 68

  "You have now provided an offer to this household 3 times and they have indicated they are interested in the program. You will not return to this household again so please confirm whether they would like to accept the offer or not as this is their last chance to participate in the program," and "You have provided an offer to this households 3 times and they are not interested in participating in the program. Please thank this household for their time. Select the close button below to close this households case," and "Close Households Case." No Lean Season 2017 CommCare Module Translation, tab "module3_form1."

• 69

  Evidence Action's monitoring plans for the offer stage of the program are described in No Lean Season 2017 Process and Performance Monitoring Framework, Pgs 3-4 and in No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "FullFramework," lines 19-21. These describe monitoring of offer meetings but do not describe activities specifically aimed at monitoring Migration Organizers' post-meeting household-level interactions.

• 70

  For examples, see the paper version, No Lean Season 2016 Loan Application Form, and the electronic version, No Lean Season 2017 CommCare Module Translation, tab "module5_form2."

• 71

  No Lean Season 2017 CommCare Module Translation, tab "module5_form1" includes the collection of detailed family information, as well as the applicant's National ID (NID) number, "National Identification Card/ Birth Certificate Image Capture," and "Take Photo of applicant."

• 72

  Migration Organizers are instructed to mention that part of the loan application form will be filled out during a household visit: "If you are interested, you are welcome to come to our Branch Office between to receive the loan. Before that, however, I will come back to your village again to fill out a short loan application form for those who are interested." No Lean Season 2017 Offer Meeting Script, Pg 3.

• 73

  "We can wait for your decision until November 30 this year, unless you live in a village that is far from a Branch Office. If that is the case, we will organize another group meeting in your village where you will be able to get the loan instead of having to travel to a Branch Office.

  However, those who have taken a decision already, please tell me your name. I will make sure that we have a faster disbursement for you.

  If you are interested, you are welcome to come to our Branch Office between to receive the loan. Before that, however, I will come back to your village again to fill out a short loan application form for those who are interested. When you come to the Branch Office to collect the loan, you must bring your National ID Card with you.

  Anytime between August and November, you can come to the Branch Office to get the loan amount. But just ONE DAY before coming, please give me a phone call so that I can arrange the money for you.

  ...

  If you want this migration loan for another working member of your family (instead of yourself), please bring him/her with you to the Branch Office. Please make sure that s/he is 18 or more years old and s/he brings a copy of his/her National ID Card with him/her too. If that member doesn’t have any National ID Card, please ask him/her to bring his/her Birth Certificate/ Voter Number document. The money will be disbursed only to the person who will apply and ideally, the same person would migrate afterwards. It is not possible to get the money on someone else’s behalf."

  No Lean Season 2017 Offer Meeting Script, Pg 3.

• 74

  "Village surveys are used to profile and geolocate all the prospective villages in which No Lean Season may operate. RDRS identifies the list of villages within each particular catchment area. A catchment area includes all villages located within an hour’s bicycle distance from a specific RDRS branch office." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 3.

• 75
  • GiveWell's 2017 site visit to No Lean Season
  • "Interviewees were asked about their use of services traditionally provided by their union parishad, a small administrative unit usually covering a handful of villages. Unlike many other developing country contexts, almost all respondents in this sample have received a voter ID card as well as a birth certificate from this entity." @Mobarak, Reimão and Shenoy 2017@ Pg 4.
    See the table on Pg 5:
    • "Have you ever received a voter card from your union parishad?": 0.99
    • "Have you ever received a national ID card from your union parishad?": 0.52
    • "Have you ever received a birth certificate from your union parishad?": 0.98

• 76

  GiveWell's 2017 site visit to No Lean Season

• 77
  • "Fill-out Part 2: Subsidy Disbursement on CommCare for the interested candidates right on the disbursement spot.
  • Ensure that the applicant and recipient are the same person by verifying National ID card/picture on CommCare
  • Ensure that the recipient is above 18 years old
  • Fill-out Part 2: Subsidy Disbursement in CommCare for the interested candidates at the time of disbursement."

  No Lean Season 2017 CommCare Module Translation, tab "module4_form1."

• 78

  No Lean Season 2017 CommCare Module Translation, tab "module5_form2."

• 79
  • "Please hand tablet to applicant to read and sign"
  • "I, hereby, declare that on the date of < output value="format-date(date(/data/subsidy_disbursement/date_subsidy_received), ' % a, % b % e, % Y')"/ >" I got information about an interest-free loan facilities from RDRS Bangladesh. I am fully agreed with the condition and thereby, interested to take the interest-free loan of amount 1500 Tk (One thousand five hundred taka).
    I also declare that after taking the interest-free loan, I will migrate outside of my upazila for at least seven days for working there. RDRS won't be liable for any of my illegal activities/accidents during my migration. I vow to return the interest-free loan back by < output value="/data/subsidy_disbursement/date_repayment"/ > "
  • "Migration Organiser Signature"
  • "Branch Manager Signature"
  • "Recipient Signature Required"

  No Lean Season 2017 CommCare Module Translation, tab "module5_form2."

• 80

  "Sensitize recipients with DOs and DON’Ts of migration:

  • DON’T travel on the roof of bus/train
  • DON’T consume foods/drinks from unknown persons while traveling
  • DRINK safe water in the destination
  • USE safe sanitation in the destination
  • DON’T get engaged in any immoral/illegal/criminal/political activities in the destination places
  • KEEP contact number of the MO, Union Parishad Chairmen and Ward Member and CONTACT to them in case you have any administrative troubles
  • During migration, KEEP a photocopy of your national ID card (or birth certificate) duly attested by the respective ward member or union parishad chairman and produce that to the authority in case you have any problem. The document, in addition, must have the phone number of the attesting ward member or union parishad chairman"

  No Lean Season 2017 CommCare Module Translation, tab "module4_form1."

• 81

  GiveWell's 2017 site visit to No Lean Season

• 82
  • GiveWell's 2017 site visit to No Lean Season
  • The Information System of the Laws of Bangladesh: Bangladeshi Stamp Act of 1899 as found in the online Information System of the Laws of Bangladesh. The Information System of the Laws of Bangladesh: Chapter II section 3 of the Stamp Act of 1899 establishes that "Subject to the provisions of this Act and the exemptions contained in Schedule I, the following instruments shall be chargeable with duty of the amount indicated in that schedule as the proper duty therefore, respectively, that is to say- (a) every instrument mentioned in that schedule which, not, having been previously executed by any person, is executed in Bangladesh on or after the first day of July, 1899..." The first instrument listed in this schedule describes: "ACKNOWLEDGEMENT of a debt exceed, in One thousand Taka in amount or value, written or signed by, or on behalf of, a debtor in order to supply evidence of such debt in any book (other than a banker’s pass book) or on a separate piece of paper where such book or paper is left in the creditors possession: Provided that such acknowledgement does not contain any promise to pay the debt or any stipulation to pay interest or to deliver any goods or other property." The duty listed in this table for this stamp is ten taka. Information System of the Laws of Bangladesh: Schedule I of the Stamp Act of 1899, Pg 1. This page can be found through the Information System of the Laws of Bangladesh's page on the Information System of the Laws of Bangladesh: Bangladeshi Stamp Act of 1899. At the bottom of this page is a link to the associated Information System of the Laws of Bangladesh: Schedule I of the Stamp Act of 1899.
  • RDRS pre-purchases duty stamps and the accountant collects 10 taka from the loan applicant in order to cover this duty. GiveWell's 2017 site visit to No Lean Season
  • "Collect a duty stamp of 10 taka value for the Registry Book from the interested candidate." No Lean Season 2017 CommCare Module Translation, tab "module4_form1."

• 83

  "Collect signature/fingerprint of the interested candidate on the Registry Book." No Lean Season 2017 CommCare Module Translation, tab "module4_form1." We observed an example of this registry book during GiveWell's 2017 site visit to No Lean Season.

• 84

  "Disburse the money to the candidate." No Lean Season 2017 CommCare Module Translation, tab "module4_form1." We observed this process during GiveWell's 2017 site visit to No Lean Season. We were told that the denomination of bills may vary: applicants may receive one ten-thousand taka note and one five-hundred taka note, or three five-hundred taka notes. GiveWell's 2017 site visit to No Lean Season

• 85

  GiveWell's 2017 site visit to No Lean Season

• 86
  • "We can wait for your decision until November 30 this year, unless you live in a village that is far from a Branch Office. If that is the case, we will organize another group meeting in your village where you will be able to get the loan instead of having to travel to a Branch Office." No Lean Season 2017 Offer Meeting Script, Pg 3.
  • "[For remote villages only] We will organize another group meeting in your village where you will be able to get the loan instead of having to travel to a Branch Office. The meeting will take place on at . Please try to attend this meeting with necessary documents if you want to get our interest-­free loan. If you are not able to attend, you will have to travel to the Branch Office to get the loan. We won’t be able to arrange such disbursement meetings twice in your village." No Lean Season 2017 Offer Meeting Script, Pgs 3-4.

  • 87

    "At-village distribution occurs specifically in villages which are farther than 12 km from the branch office. The date and location of the at-village disbursement is communicated in advance at the offer group meeting(s). At-village disbursements will occur in 10 villages this year. This is approximately 1% of the total villages included in this year's program." Natalie Duarte, Evidence Action Beta Research Analyst, email to GiveWell, November 14, 2017

  • 88

    “At the time of disbursement, it is important that subsidy recipients are able to accurately recall key information about the subsidy - including the terms of loan repayment. To verify this key step, Evidence Action makes random, unannounced visits to branch offices during the disbursement process. During these visits, loan disbursement procedures are observed and verified, and loan recipients are interviewed to assess whether they have a correct understanding about the terms of the loan repayment.

    Similar to the verification activities for offer meetings, Evidence Action conducts an observation survey and a loan recipient post-survey. Both the observation and recipient surveys assess whether the proper protocols were carried out during the disbursements and whether the recipient understood the conditions of the loan. In addition, these verification activities check that only eligible households receive the loan, that each eligible household received a maximum of one stipend of the correct amount, and that the recipient matches the photo and ID in RDRS records.

    This year, we randomly sampled 104 disbursement days (or 2 disbursement days per branch) and randomly sampled an estimated 416 loan recipients (or an average of 4 recipients per disbursement day). Loan disbursement verification is currently an on-going activity and will conclude December 31, 2017.” No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

  • 89

    No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "FullFramework." Lines with data source "Loan disbursment verification survey" are the following "Indicators of Achievement":

    • "% of loan recipients who attended offer meeting"
    • "% of travel subsidy recipients who come from eligible households (independent monitoring)"
    • "% of travel subsidy recipients who report receiving the correct amount and who match the photo / ID in RDRS records"
    • "% of travel subsidy recipeints who are 18 years of age or above (independent monitoring)"
    • "% of recipients who understand and can accurately relay information related to the subsidy and loan repayment (independent monitoring)"

  • 90

    Evidence Action staff confirmed for us that Migration Organizers visit approximately three neighbors to ask whether a loan recipient has migrated, then visit the loan recipient's household. See also No Lean Season 2017 CommCare Module Translation, tab "module6_form1."

  • 91

    For a paper version used in 2016, see No Lean Season 2016 Migration Debrief Survey. For the electronic version used in 2017, see No Lean Season 2017 CommCare Module Translation, tab "module8_form1."

  • 92
    • "Migration Compliance: Migration Organizers use the migration compliance survey to ascertain whether subsidy recipients actually migrate.

      This key performance metric will be verified by Evidence Action by visiting a sample of households and conducting a supplementary interview to assess the incidence of migration. In this stage, we may draw the sample from the entire set of HHs surveyed (both eligible and ineligible households) in order to understand migration rates across the entire community. In steady state, we would also like to estimate the rate at which beneficiaries of the program re-migrate in the years following their receipt of the subsidy, even without the program intervention." No Lean Season 2017 Process and Performance Monitoring Framework, Pgs 4-5.

    • No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "FullFramework" refers to "Migration verification survey" as the source of monitoring for the indicators "% of households receiving a travel subsidy that send a migrant during a lean season (independent monitoring)," and "% of non-eligible and/or non-recipient households in a village sending a migrant." Additionally, this document refers to "% of households that received a travel subsidy and migrated, that continue to migrate in subsequent years" as measured by "Migration verification survey (in later years)."
    • No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "Sampling plan" states the verification goals of migration verification as "Working age member of the eligible households who received the loan migrate" and "Migration Organizers adhere to migration compliance visit script."

  • 93

    GiveWell's 2017 site visit to No Lean Season

  • 94

    "Please confirm a receipt has been given for this payment," "If you have not yet provided a receipt please do so before proceeding." No Lean Season 2017 CommCare Module Translation, tab "module7_form1."

  • 95
    • "How will the repayment be paid back?" includes the response "Instalments," and "How many instalments will the applicant pay back the loan in?" includes the note "The maximum instalments an applicant can pay the loan back in is 3." No Lean Season 2017 CommCare Module Translation, tab "module7_form1."
    • Evidence Action has told us that loans may be repaid in up to three installments within two months of a migrant's return.

  • 96

    There is no mention of the option to repay in installments in the No Lean Season 2017 Offer Meeting Script. However, Migration Organizers' guidance for frequently asked questions includes: "Q: Can I repay the loan on installment basis? A: Yes. We prefer to recover the money at once upon your return from migration, but if you need to pay on installments, we can allow maximum 3 installments for this amount within the stipulated time (January 31st)." No Lean Season 2017 CommCare Module Translation, tab "module2_form2."

  • 97

    There is no mention of the limited liability nature of the loan in the No Lean Season 2017 Offer Meeting Script, however, Migration Organizers' guidance for frequently asked questions includes: "Q: What if I die after taking the loan? Or get seriously injured/sick?
    A: Firstly, we do not wish you to die! However, in such case (god prohibits), you (or your household) will not be asked to pay us back the money… Q: What if after migration I cannot find work and therefore, cannot earn any money? A: We suggest you to try out at this moment. We hope that you will be able to find work and earn, if you are serious enough. However, we cannot guarantee that you will get work and earn handsome amount through seasonal migration. We will see if you really made a try to work but couldn’t manage a good fortune; we will take measures accordingly then. Now we cannot give you exact information, but we can guarantee that each situation will be processed by RDRS on a case-by-case basis, taking into account your situation and any difficulties you faced by no means you will be harmed (financially) participating in the project. But again, we hope that you will be able to find work and earn after you migrate." No Lean Season 2017 CommCare Module Translation, tab "module2_form2."

  • 98

    "Loan Recovery: Migration Organizers visit households to recover the subsidy amount from households that are able to pay. Migration Organizers record their visits to beneficiary households on the loan recovery form. Loan forgiveness is considered and granted, per agreed upon criteria, for recipients who:

    • a. Were unable to travel because of illness, injury, death, or other catastrophic event
    • b. Traveled and tried to find work, but were unsuccessful and would thus be worse off if they were obligated to repay the loan, and
    • c. Traveled, found work, but did not earn enough (i.e. twice as much as the loan amount) to repay the loan without incurring substantial financial hardship."

    No Lean Season 2017 Process and Performance Monitoring Framework, Pg 5.

  • 99

    Evidence Action provided this additional example of a circumstance under which loan forgiveness would be considered and granted.

  • 100

    GiveWell's 2017 site visit to No Lean Season

  • 101

    See also (from Migration Organizers' guidance on frequently asked questions about the program): "Q: What if after migration I cannot find work and therefore, cannot earn any money? A: We suggest you to try out at this moment. We hope that you will be able to find work and earn, if you are serious enough. However, we cannot guarantee that you will get work and earn handsome amount through seasonal migration. We will see if you really made a try to work but couldn’t manage a good fortune; we will take measures accordingly then. Now we cannot give you exact information, but we can guarantee that each situation will be processed by RDRS on a case-by-case basis, taking into account your situation and any difficulties you faced by no means you will be harmed (financially) participating in the project. But again, we hope that you will be able to find work and earn after you migrate." No Lean Season 2017 CommCare Module Translation, tab "module2_form2."

  • 102

    No Lean Season 2016 summary of data, tab "(5) Recovery," row "Total Amount of Funds Recovered."

  • 103

    GiveWell's 2017 site visit to No Lean Season

  • 104

    GiveWell's 2017 site visit to No Lean Season

  • 105
    • "Loan recovery verification survey" is listed as the source of data for the indicator "% of loans recovered from households that sent a migrant (independent monitoring)." No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "FullFramework."
    • "The terms of loan repayment are intentionally soft. It is important that households do not feel pressured into re-paying the subsidy amount. Evidence Action plans to assess loan repayment activities to verify repayment rates and to check that households were not unduly pressured into paying back the loan. The loan recovery verification survey will be conducted with a random sample of loan recipient households. Sample size calculations have not been completed as of yet, but will allow for a 95% confidence interval and a 5% margin of error." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 6.
    • No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "Sampling plan" lists the sample for the "repayment" monitoring as "TBD."

  • 106

    GiveWell's summary of No Lean Season's 2016 spending is based on No Lean Season 2016 program costs.

  • 107

    No Lean Season Budget Projections

  • 108
    • 2014 RCT:
      "The landless poor supplying agricultural labor on others’ farms are especially affected when demand for agricultural labor falls. They constitute around 56% of the population in our sample area, and are the target of the seasonal migration encouragement intervention that we design. Our sampling frame is representative of this landless population in the Rangpur region of Northern Bangladesh. According to the Bangladesh Bureau of Statistics, there are roughly 15.8 million such inhabitants in Rangpur (BBS, 2011)." Akram, Chowdhury and Mobarak 2017, Pg 5.
      • "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur." Akram, Chowdhury and Mobarak 2017, Pg 7.
    • 2008 RCT (and its follow-up surveys): "Our experiments were conducted in 100 villages in two districts (Kurigram and Lalmonirhat) in the seasonal-famine prone Rangpur region of northwestern Bangladesh." Bryan, Chowdhury and Mobarak 2014, Pg 1675.

  • 109

    No Lean Season Budget Projections, tab "Budget Projections" shows projected growth in "Total # of Branch Offices." The number of branch offices in which No Lean Season projects operating by 2022 exceeds 171, the total number of RDRS branch offices as of 2016 (see table row "Total Branches," RDRS Bangladesh 2016 Annual Report, Pg 53), and continues to grow at a rate of 10 per year.

  • 110

    For background on seasonal variation in consumption in rural northern Bangladesh, see Khandker 2012 and Khandker and Mahmud 2012.
    In particular, Khandker and Mahmud 2012 discuss seasonal patterns of income, expenditures, and poverty on Pgs 43-61, based on the authors' analysis of data from the Household Income and Expenditure Survey of the Bangladesh Bureau of Statistics, showing marked dips in income and consumption in the lean season. See especially the figures in this section, showing seasonal trends in both Rangpur and the rest of Bangladesh.

  • 111
    • Evidence Action has confirmed for us that households were eligible for the No Lean Season program in 2017 if they reported owning 50 or fewer decimals of land or reported any incidence of skipped meals in the two weeks preceding the eligibility survey.
    • Eligibility criteria in the 2014 RCT: "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur. We first conducted village censuses to identify all households that would be 'eligible' to receive this intervention in each of these villages. A household was deemed eligible if (1) it owned less than 0.5 acres of land, and (2) it reported back in 2008 that a member had experienced hunger (i.e., skipped meals) during the 2007 monga season. We focused on landownership because land is the most important component of wealth in rural Bangladesh, and it is easily measurable and verifiable. We used the second question on skipping meals to avoid professional, non-agricultural households (who may not own much land, but who are comparatively well off). Our census data suggest that about 57% of households in these villages were eligible to receive the intervention after applying these two criteria." Akram, Chowdhury and Mobarak 2017, Pg 7.
    • Eligibility criteria in the 2008 RCT: "The two districts where the project was conducted (Lalmonirhat and Kurigram) represent the agro-ecological zones that regularly witness the monga famine. We randomly selected 100 villages in these two districts and first conducted a village census in each location in June 2008. Next, we randomly selected 19 households in each village from the set of households that reported (a) that they owned less than 50 decimals of land, and (b) that a household member was forced to miss meals during the prior (2007) monga season. [footnote] Seventy-one percent of the census households owned less than 50 decimals of land, and 63% responded affirmatively to the question about missing meals. Overall, 56% satisfied both criteria, and our sample is therefore representative of the poorer 56% of the rural population in the two districts." Bryan, Chowdhury and Mobarak 2014, Pgs 1677-78.

  • 112
    • "Following the success of the loan provision for seasonal migrants, we have piloted the project NLS in 15 branches for 7000-9,000 seasonally-unemployed adults who can borrow Tk 1,500 each on an interest-free basis." RDRS Bangladesh 2016 Annual Report, Pg 49
    • The loan amount of 1500 taka is built into No Lean Season Budget Projections, unpublished tab.
    • We use the currency conversion rate 1 USD / 80.22 Bangladeshi taka, as used in No Lean Season Budget Projections, unpublished tab.

  • 113

    "The basic form of our intervention was the offer of a cash grant worth 1,000 Taka ($13.00 USD) to rural households in northern Bangladesh to cover the round-trip cost of travel to nearby cities where there are job opportunities during the lean season. This was a conditional transfer, where the subsidy is conditional on one person from the household agreeing to out-migrate during the lean season. As offers were made, we let households know that they may have a better chance of finding work outside of their village, but we did not offer to make any connections to employers. No requirement was imposed on who within the household had to migrate, or what city they had to go to." Akram, Chowdhury and Mobarak 2017, Pgs 6-7.

  • 114

    "Seven hundred three households in 37 randomly selected villages were offered cash of 600 Taka (∼US$8.50) at the origin conditional on migration, and an additional bonus of 200 Taka (∼US$3) if the migrant reported to us at the destination during a specified time period. We also provided exactly the same information about jobs and wages to this group as in the information-only treatment. Six hundred Taka covers a little more than the average round-trip cost of safe travel from the two origin districts to the four nearby towns for which we provided job information. We monitored migration behavior carefully and strictly imposed the migration conditionality, so that the 600 Taka intervention was practically equivalent to providing a bus ticket. The 589 households in the final set of 31 villages were offered the same information and the same Tk. 600 + Tk. 200 incentive to migrate, but in the form of a zero-interest loan to be paid back at the end of the monga season. The loan was offered by our partner micro-credit NGOs that have a history of lending money in these villages. There is an implicit understanding of limited liability on these loans since we are lending to the extremely poor during a period of financial hardship." Bryan, Chowdhury and Mobarak 2014, Pg 1678.

  • 115

    "We randomly assigned the 133 villages into three groups: (a) Low Intensity – 48 villages where we targeted migration subsidies to roughly 14% of the eligible (landless, poor) population. This is comparable to the Bryan et al. (2014) treatment. (b) High Intensity – 47 villages where we targeted roughly 70% of the eligible population with migration subsidy offers. (c) Control – 38 randomly selected villages where nobody was offered a migration subsidy.
    The high vs. low intensity design was chosen to generate significant variation in the size of the emigration shock, but the precise target (14% vs 70%) varied a little across villages within treatment arms. This is because our village population estimates were dated (from 2008) for most (100) villages, and imprecise in the 33 other villages, which made it difficult for us to precisely estimate the ratio (offers/eligible population) in each village." Akram, Chowdhury and Mobarak 2017, Pg 7.

  • 116

    "(a) Low Intensity – 48 villages where we targeted migration subsidies to roughly 14% of the eligible (landless, poor) population. This is comparable to the Bryan et al. (2014) treatment." Akram, Chowdhury and Mobarak 2017, Pg 7, referring to the treatment in the 2008 experiment published in Bryan, Chowdhury and Mobarak 2014.

  • 117

    In 2014 villages treated with cash at low intensity, migration rate was 24.8 percentage points higher than the average migration rate in the control group (34.2%). In villages treated at high intensity, migration rate among those offered was 39.8 percentage points higher than the average migration rate in the control group. Table 1, Akram, Chowdhury and Mobarak 2017, Pg 36. For comparison: in the 2008 RCT, in treatment villages (all at low-intensity), migration rate among those offered was 23.0 percentage points higher in the cash treatment group than in the pure control group (36.0%) and 20.8 percentage points higher in the credit treatment group than in the pure control group. Bryan, Chowdhury and Mobarak 2014, Pg 1683.
    In 2014 villages treated with cash at low intensity, total income among those offered was 2,589 taka higher than the average in the pooled control group (23902.688 taka). In villages treated at high intensity, total income among those offered was 2,105 taka higher than the average in the pooled control group. Table 5, Akram, Chowdhury and Mobarak 2017, Pg 41.

  • 118

    "In the 68 villages where we provided monetary incentives for people to seasonally out-migrate (37 cash + 31 credit villages), we sometimes randomly assigned additional conditionalities to subsets of households within the village. A trial profile in Figure 2 provides details. Some households were required to migrate in groups, and some were required to migrate to a specific destination. These conditionalities created random within-village variation, which we use as instrumental variables to study spillover effects from one person to another." Bryan, Chowdhury and Mobarak 2014, Pgs 1678-79.

  • 119

    "All of the implementation activities – the offers and marketing, grant disbursement, and monitoring to ensure adherence to the conditionality, were conducted by RDRS, a local NGO with 40 years of engagement in Rangpur, and substantial presence in the region." Akram, Chowdhury and Mobarak 2017, Pg 8.

  • 120
    • "In August 2008, we randomly allocated the 100 villages into four groups: Cash, Credit, Information, and Control. These treatments were subsequently implemented on the 19 households in each village in collaboration with PKSF through their partner NGOs with substantial field presence in the two districts." Bryan, Chowdhury and Mobarak 2014, Pgs 1677-78.
    • GiveWell's 2017 site visit to No Lean Season

  • 121

    Evidence Action confirmed these 2017 eligibility criteria.

  • 122

    "For household survey, we will use two stage sampling where we first select the village and then the household. Samples for these events will be selected to meet 95% confidence level (CL) and 5% margin of error (MoE)." No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "Sampling plan."

  • 123

    In September 2017, GiveWell spoke with Evidence Action's nine monitoring staff members in Bangladesh. They had recently completed the Household Targeting Verification Survey and described the process for this monitoring activity. Evidence Action randomly selected a sample of households from the full list compiled in the course of the household eligibility census, and communicated this list to the monitoring staff. Monitors were required to begin sampling at each of the randomly selected households, then to follow a "left hand rule" to count houses to the left of the starting house, and to subsequently sample the fifth and tenth houses. Evidence Action monitors that houses are counted as clearly visible buildings (if multiple households live in one structure, they would still be counted as one house for the purposes of counting towards the next house to sample). Evidence Action monitors told us that they are required to sample the first household (the household selected by Evidence Action), and hence will return if no adult household member is present at the time of the survey; they may ask neighbors or children over 12 when an adult household member will return. However, if no one is present in the fifth house, they are permitted to instead sample the sixth house (and so on), and subsequently to begin counting five more houses from that point in order to sample a third house in the sampling series. Evidence Action monitors described to us that when counting left-hand houses, if no more houses remain in a village cluster, the monitor turns and from that point begins counting on their new left side (formerly their right side). Evidence Action monitors told us that if they are unsure whether a house belongs to a village cluster (for example, it is remotely visible from other houses), they will inquire with nearby neighbors, and told us that neighbors are generally clearly aware of which houses are part of their village cluster or not. If an adult household member declines to participate in the survey, monitors move on to sample the next household to the left. One male monitor told us that he had experienced a small number (3-4 out of 118) of refusals, because adult household members present during the survey are more likely to be women than men. The female monitors told us they had never experienced refusal to participate in the survey. Evidence Action monitors told us that each survey takes 5-6 minutes. GiveWell's 2017 site visit to No Lean Season

  • 124

    "At the beginning of the program year, we selected 575 villages in 52 branches to be included in No Lean Season 2017," No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 1. "Verification data was collected in a random subset of villages (n = 123) and a random subset of households within these villages (n = 748)," Pg 5.

  • 125

    "The Household Targeting Survey is meant to be a complete census of each village selected," No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 1.

  • 126

    "In each of the villages selected for verification activities, we would wait until the household targeting activity was complete to randomly select one household as starting point for the verification survey. The enumerator then used the left hand rule to interview two additional households skipping 4 households from the one interviewed. After conducting interviews in three households in the same area of the village, the enumerators then moved to the nearest para (i.e. Bangladesh administrative unit within villages) and randomly selected one households to be his starting point. For villages that did not have Paras, the enumerators walked for about 500 meters and then selected a starting household at random. In total, 748 households were visited during the households survey verification exercise." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 5.

  • 127

    Evidence Action notes this possibility: "It may be the case that particularly ‘hard to reach’ households or sets of households would be similarly skipped by both the original survey and the verification survey. In the future we may use GPS data and satellite imagery to identify missing clusters of households and small villages. But overall, the results of this verification survey give us a relatively high level of confidence that the Household Targeting Survey successfully achieved extremely high coverage of households within participating villages." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 2.

  • 128

    "Collecting reliable eligibility criteria, in particular the amount of cultivable land owned, is critical to the program’s integrity and impact. We verified the administrative data collected on this criteria using verification data. Results from the analysis of this cross-check show that 64% of the randomly selected households reported exact or near-exact data on cultivable land owned to the independent enumerators." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 4. A footnote on Pg 5 specifies that "Near‐exact means less than a 8 decimal difference in reporting." The table on Pg 4 shows that 480 out of 747 households surveyed reported exact or near-exact amounts of cultivatable land owned.

  • 129

    "Of the 480 observations that were exact or near-exact, 421 were exact matches. For the 36% of households (267 households) with differences, we analyzed whether the discrepancies led to any changes in eligibility between surveys. In this analysis, 24% (63 households) of households’ eligibility status would have changed -- 41 households ‘false negatives’ and 22 were ‘false positives’. In total, false positives and false negatives account for 8% of the total sample (63 out of 747 households). In the context of the program goals, ‘false negatives’ are more of a concern than ‘false positives’. This is because providing access to a travel subsidy for a non-eligible household may be slightly inefficient, but it is not reducing the potential impact of the program. On the other hand, not serving a household that should have been eligible is a greater concern. The verification survey shows that this is an issue that we will need to monitor over time, but that the incidence of false negatives is quite low -- only 41 out of 747 cases, or 5%." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 4.

  • 130

    Evidence Action determines household eligibility using the process laid out in this document: No Lean Season 2016 Analysis Plan for Selecting Target Households.

  • 131

    GiveWell's 2017 site visit to No Lean Season

  • 132

    We have seen an outline of this process from 2016: @No Lean Season 2016 Analysis Plan for Selecting target Households@.

  • 133
    • "We can wait for your decision until November 30 this year, unless you live in a village that is far from a Branch Office. If that is the case, we will organize another group meeting in your village where you will be able to get the loan instead of having to travel to a Branch Office.
      However, those who have taken a decision already, please tell me your name. I will make sure that we have a faster disbursement for you.
      If you are interested, you are welcome to come to our Branch Office between to receive the loan. Before that, however, I will come back to your village again to fill out a short loan application form for those who are interested. When you come to the Branch Office to collect the loan, you must bring your National ID Card with you.
      Anytime between August and November, you can come to the Branch Office to get the loan amount. But just ONE DAY before coming, please give me a phone call so that I can arrange the money for you.
      ...
      If you want this migration loan for another working member of your family (instead of yourself), please bring him/her with you to the Branch Office. Please make sure that s/he is 18 or more years old and s/he brings a copy of his/her National ID Card with him/her too. If that member doesn't have any National ID Card, please ask him/her to bring his/her Birth Certificate/ Voter Number document. The money will be disbursed only to the person who will apply and ideally, the same person would migrate afterwards. It is not possible to get the money on someone else's behalf."
      No Lean Season 2017 Offer Meeting Script, Pg 3.
    • GiveWell's 2017 site visit to No Lean Season

  • 134

    "The household-level verification survey will check that each eligible household received a maximum of one stipend of the correct amount, and that the recipient matches the photo and ID in RDRS records." No Lean Season 2017 Process and Performance Monitoring Framework, p 4.

  • 135

    "Evidence Action will observe a sample of offer meetings, and then will interview a sub-sample of participants from the list of eligible households post-meeting. Evidence Action will assess offer recipients' understanding of key information about the subsidy and check that they were not coerced into accepting the subsidy." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

  • 136

    "Offer meetings were randomly sampled for the observation survey. During observations, Evidence Action enumerators collect data on whether the Migration Organizer followed protocol and covered key aspects of the script for the offer meeting, as well as collect data on attendance to ascertain who and how many people are attending offer meetings. Additionally, four eligible households were sampled per Migration Organizers for the post-meeting survey; two participants from the observed offer meeting and two other eligible households that were scheduled to attend another offer meeting, but where the offer meeting is unobserved. Evidence Action uses these surveys to assess offer recipients’ understanding of key information about the subsidy and checks that they were not pressured into accepting the subsidy. In total, this year, Evidence Action randomly sampled 104 offer meetings for observation and 416 eligible households for post-meeting surveys." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

  • 137

    "Evidence Action will make random, unannounced visits to branch offices during the disbursement process. During this visit, loan disbursement procedures will be verified, and loan recipients will be interviewed to assess whether they have a correct understanding about the terms of the loan repayment." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

  • 138

    "It is important that households do not feel pressured or coerced into re-paying the subsidy amount. Evidence Action verifies loan repayment for a sample of households to verify repayment rates and to check that households were not unduly coerced into paying back the loan. This component is under development." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 5.

  • 139

    GiveWell's 2017 site visit to No Lean Season

  • 140
    • Some beneficiaries reported feeling some pressure to repay the loan, including disliking the frequency of visits by migration organizers:
      • "Some people talked about this type of force from both sites. Apart from this kind of force, all respondents mentioned that they received the reminder from RDRS staff within a month of receiving the loan. Many respondents mentioned that they received frequent reminders from RDRS staffs. Some of them told that they were asked to re-pay in a certain date of the month." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 15.
      • "I took another loan of 1000 BDT to pay off this loan. Poor people are unable to save money. the staff forced me in many ways. He told me that ‘I would lose the job if I delay paying the amount. He also told that it is not my duty to see how did you spent the amount, you have taken the loan and you have to pay off the loan’. I felt like, I have to migrate even if I die. I then lend it from my neighbor and paid it. " Excerpt from a day laborer from Kurigram district, focus group discussion. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 15.
      • "The staff told me that ‘uncle, I may lose my job if you delay to pay off the loan’. I replied I will not cause you that damage and paid off the loan by taking another loan." Excerpt from a farmer from Kurigram district, focus group discussion. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 15.
      • "Some people mentioned that staffs frequent visited the loan recipients household that created a stress to repay the loan. After taking out loan, program organizers visited respective household frequently that many loan recipients did not appreciate. Loan recipients saw it as a type of indirect force to repay the loan. They saw it as a lack of trust between loan recipients and implementers. Since people took out loan after submitting relevant documents with a formal agreement, they need not to create any mental pressure for the loan recipients. Frequent visit by the implementer was not appreciated by many respondents and even by the female members of the family." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pgs 15-16.
      • "If someone [RDRS staff] faces any problem because of me and he visits my home every day for money, it does not feel good. It creates a pressure for us to pay the loan as soon as possible. I then questioned myself that why have I taken the loan? I took a loan from another person and returned the loan of RDRS. " Excerpt from discussion with an agricultural day laborer from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 16.
      • "How many times do you have to visit us, we already have returned the loan. If every time people start coming to our house, how could we live in the house. We have many things to do and are very busy with rice plantation. Why do you come repeatedly?" Excerpt from an informal discussion with the wife of a recipient from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 16.
      • "Program staffs need to think about developing an effective mechanism of monitoring loan recipients avoiding annoyance among the recipients. It was clear from the field research that loan recipients did not like the tendency of RDRS staff who visited them frequently to monitor. Although RDRS staff mentioned them that they just came to see them not to monitor them, people took it as a negative activity of the program that was discussed in the findings section in detail. Therefore, the frequency of visiting the household can be reduced, especially when the household head is away, because people from the community do not like outsiders to interact with their female counterpart frequently." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pgs 25-26.
    • "An incident was found during the field trip where the son of a loan recipient did not like the behavior of one staff of RDRS. A younger son of a loan receiver reported that the PO was forcing them to return the loan without giving extra time. The person was sick on that day and requested to extend the time but the program organizers did not listen to their request. In addition, it was only one month of getting the loan, they added. This was the only incident that we experienced during the field visit that shows a clear dissatisfaction." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 16 and described again: "No one reported about malpractice by the program organizers of RDRS in the Sonahar of Panchagar District. However, one incident was found where a son of a loan receiver reported that the Program Organizer was forcing them to return the loan without giving extra time and later on, we discovered many more incidents like this. The person was sick on that day and requested to extend the time but the program organizers did not listen to their request. In addition, it was only one month of getting the loan, they added." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 25.

    • 141
      • "Most of the people told the research team that they had returned the loan right after coming back to their village from their destination place." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 14.
      • "It was in my head that I have to pay back the loan when I would go back home. I started to save 20-30 BDT everyday from my daily wage for paying back the loan, because if I go back home without saving money for this loan, it would be very difficult [he can spend the entire amount for household tasks] for me to arrange money. I saved about 800-900 BDT. After working couple of days more, I arranged the entire amount to pay off the loan. However, I did not return the entire amount at a time, but by two installments. The staff of RDRS listened to my request." Excerpt from an informal discussion with a day laborer in oil production from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 14.
      • "I found it very difficult to pay off the loan, as I did not go out to work. I did not find suitable work to do during this time. I took some loan from my neighbor and saved some money by working here and there to return the loan. The staffs of RDRS were very kind to me not to pressurize for paying off the loan." Excerpt from an informal discussion with a 35-year-old small business man from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 17.

    • 142

      "A different perception was notice during the second field trip where a few people shared their views about some activities of the implementers, where implementers told some loan recipients to talk about good side of the program. A respondent from Sonahar, Panchagar, mentioned in this context that "RDRS staff told that someone from Dhaka would come to talk to us soon. If I talk to them in a good manner and say good things about the project, I would get loan again." - A Day Laborer, Sonahar, IDI, Panchagar" This is the only response that we discovered from the field research that can be seen as an isolated event. The researchers conducted interviews in absence of RDRS staff. The research team found that RDRS staff briefed many loan recipients prior to the visit of the research team. It did not affect the findings of the research as the research team engaged with them for a longer time building a good rapport. Development of a good rapport helped the team to discover this behavioral pattern of some RDRS staff in the field. This was not discovered in the first field trip, the research team found this type of comments during the second trip. The research team did not informed RDRS staff before hand of the second field trip." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pgs 17-18.

    • 143

      "Migration Organizers use the migration compliance survey to ascertain whether subsidy recipients actually migrate.

      "This key performance metric will be verified by Evidence Action by visiting a sample of households and conducting a supplementary interview to assess the incidence of migration. In this stage, we may draw the sample from the entire set of HHs surveyed (both eligible and ineligible households) in order to understand migration rates across the entire community. In steady state, we would also like to estimate the rate at which beneficiaries of the program re-migrate in the years following their receipt of the subsidy, even without the program intervention."

      No Lean Season 2017 Process and Performance Monitoring Framework, Pgs 4-5.

    • 144

      No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, "FullFramework" sheet.

    • 145

      No Lean Season 2016 summary of data, "Migration" sheet.

    • 146

      No Lean Season 2016 summary of data, "Migration" sheet.

    • 147

      More detail here: No Lean Season 2016 summary of data, "Migration" sheet.

    • 148
      • No Lean Season 2016 Household Targeting Questionnaire
      • No Lean Season 2016 summary of data is a summary of the data obtained from this survey. No Lean Season has shared the underlying data with GiveWell. However that data is not publishable at this time.
      • No Lean Season also shared this related analysis plan: No Lean Season 2016 Analysis Plan for Selecting Target Households

    • 149

      GiveWell's 2017 site visit to No Lean Season

    • 150

      "Migration Compliance: Migration Organizers make regular visits to the households that receive the subsidy to check migration compliance. They use the migration compliance survey to ascertain whether subsidy recipients actually migrate.
      This key performance metric is verified by Evidence Action by visiting a sample of households and conducting a supplementary monitoring survey to assess the incidence of migration, verify loan and migrant details, and detect any coercion from Migration Organizer. In this stage, we draw a random sample from the entire set of HHs surveyed in the household targeting survey (including both eligible and ineligible households) in order to understand migration rates across the entire community.
      This year, a total of 748 households were randomly sampled for the migration compliance verification survey using a two-stage random sampling, similar to that of the Household Targeting verification survey. In other words, we randomly sample 123 villages and randomly select 6 households per village. After calculating the sample size, we then create three groups from which to sample for this survey (2 households from each group in each village). These three groups are: ‘Accepted Loan’ (households that took out the loan); ‘Eligible, Non-Loan’ (households that were eligible for the loan but did not take the loan yet); and, ‘Not eligible’ (households that were not eligible for the loan’). This design allows us to capturing the rate of the migration in these communities, regardless of whether the household took out a No Lean Season loan." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 5.

    • 151

      "Migration Organizers use the migration compliance survey to ascertain whether subsidy recipients actually migrate.

      "This key performance metric will be verified by Evidence Action by visiting a sample of households and conducting a supplementary interview to assess the incidence of migration. In this stage, we may draw the sample from the entire set of HHs surveyed (both eligible and ineligible households) in order to understand migration rates across the entire community. In steady state, we would also like to estimate the rate at which beneficiaries of the program re-migrate in the years following their receipt of the subsidy, even without the program intervention." No Lean Season 2017 Process and Performance Monitoring Framework, Pgs 4-5.

    • 152

      "Rigorous Investigations on the Emergency Cases … Was this migrant sick, injured or dealing with a family emergency which restricted him/her from migration/work? … Please provide details of reason for restriction (If available take photos of any necessary documentation)." No Lean Season 2017 CommCare Module Translation, sheet "module8_form1," cells B77, B78, and B81.

    • 153

      See No Lean Season 2016 summary of data and No Lean Season 2016 summary of data (guide). We have also seen unpublished underlying data from this survey.

    • 154

      "It is important that households do not feel pressured into re-paying the subsidy amount. Evidence Action plans to assess loan repayment activities to verify repayment rates and to check that households were not unduly pressured into paying back the loan... In addition to the loan recovery verification activities, Evidence Action plans to conduct migration debrief verification. Upon migrants’ return, Migration Organizers follow-up at the household to collect loan repayments and to conduct a debriefing about the household’s migration experience. This debrief survey conducted by Migration Organizers collects data on migration dates, employment, wages, and costs of migration. The migration debrief verification survey will verify the data collected by the Migration Organizers is consistent and reliable. It is planned that these activities will accompany the loan recovery verification as these two steps are implemented together by RDRS, and verification surveys should be carried out in a similar manner." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 6.

    • 155

      For calculations, see our cost-effectiveness model of No Lean Season.

    • 156

      In 2014, as part of a research agenda, the program enrolled 50% of eligible households in some villages and 10% of households in other villages. In 2016, the program operated at a very small scale. Spending data from 2017 is not yet available because disbursements are ongoing as of this writing.

    • 157

      Karen Levy, Evidence Action Director of Global Innovation, conversation with GiveWell, November 6, 2017 (unpublished).

    • 158

      Karen Levy, Evidence Action Director of Global Innovation, conversation with GiveWell, November 6, 2017 (unpublished).

    • 159

      No Lean Season Budget Projections, tab "EA MgmtEval for BD."

    • 160

      No Lean Season Budget Projections, tab "RFMF."

    • 161
      • No Lean Season Budget Projections
      • "As greater visibility has been obtained into the organizational financial position as part of the clean up process, it has become evidence that the level of available and uncommitted unrestricted funding is very low– at the end of June 2017, unrestricted funds covered approximately 4 weeks of the organization's annual expenses. Unrestricted funding serves as the sole organizational backstop with the exception of Deworm the World reserves. Moreover maintaining a robust level of flexible unrestricted reserve is seen as particularly important during this period when we are still dealing with the implications of the issues discovered during the ongoing clean-up. Looking forward, Evidence Action is seeking to develop a further organizational backstop through the development and institutionalization of an organization-wide reserve policy." Evidence Action 2016-2017 revenue and expenditures, tab "Unrestricted commitments."
      • "After further internal discussion we would prefer to convert all reserves to organization-wide reserves. By maintaining central organizational reserves we anticipate we will actually require a lower amount of cumulative total reserves over time, since we diversify risk across programs. [...] It should be noted that we expect few funders besides GiveWell to contribute to reserves based on our knowledge of current donors." Evidence Action memo to GiveWell, 23 October 2017, unpublished.
      • Kanika Bahl and Grace Hollister, Evidence Action CEO and Deworm the World Director, conversation with GiveWell, October 16, 2017 (unpublished).

    • 162
      • In September 2017, Elie Hassenfeld, Executive Director of GiveWell and manager of the EA Fund for Global Health and Development, recommended that the EA Fund grant $150,000 to Evidence Action for No Lean Season.
      • In October 2017, GiveWell recommended that Good Ventures backstop up to $1.5 million above No Lean Season's existing level of funding to ensure that it was able to disburse loans to enrolled households for the 2017 season.

      • 163

        No Lean Season Budget Projections, unpublished tab.

      • 164
        • "Our current programs are:
          • Dispensers for Safe Water is an entrepreneurial program for more than 4.7 million people in rural Eastern and Southern Africa. It is a proven innovation that dramatically expands access to water treatment at an extremely low cost.
          • Deworm the World Initiative scales up school-based deworming programs worldwide to improve children’s health, education, and long-term development. It currently supports deworming of more than 200 million children annually.
          • Evidence Action Beta explores what programs with proven impact might work next for millions of people. Similar to beta testing for software, we are pressure-testing promising interventions to determine their suitability for massive scale-up."
        • GiveWell conducted a site visit to Dispensers for Safe Water in November 2012 and published notes: GiveWell's 2012 site visit to Dispensers for Safe Water.