GiveDirectly’s Cash for Poverty Relief Program
In a nutshell
GiveDirectly’s Cash for Poverty Relief program involves sending one-off unconditional cash transfers of ~$1,000USD (nominal) via mobile money platforms to households living in poor regions of low-income countries in sub-Saharan Africa. We estimate that this program is ~3-4x more cost-effective than we had previously estimated, and around ~30-40% as cost-effective as our marginal funding opportunity.
We think that the main benefits of the Cash for Poverty Relief program come from consumption gains to households that receive cash transfers. We also think there are benefits to households in nearby villages that don’t receive cash transfers, as increased spending by recipient households stimulates local economic activity. Finally, we think there are health benefits from the program – e.g. reductions in child mortality and morbidity – though we think these are less important than consumption gains that accrue to recipient and nearby non-recipient households.
Our main uncertainties concern: i) the persistence of consumption gains to recipient households; ii) the magnitude of consumption gains to nearby non-recipient households; iii) how much we ought to value increasing consumption vs. averting deaths or improving other health outcomes. All of these assumptions are based on relatively few empirical studies, and we may update them as new evidence comes to light.
Cost-effectiveness analysis accompanying this report: Link Note: The figures in this report are from our October 2024 cost-effectiveness analysis, and represent our best guess at that moment in time. At the moment, we have no concrete plans to update this analysis, though we expect new evidence on GiveDirectly’s Cash for Poverty Relief program (and unconditional cash transfers more generally) to emerge over the coming months and years. We may update this analysis in light of this. |
Published: November 2024
Previous version of this page:
- November 2020 report on GiveDirectly
- December 2018 report on spillover effects of GiveDirectly's cash transfer programs
- November 2018 report on cash transfers
Table of Contents
- In a nutshell
- Summary
- 1. The basics of the program
- 2. How GiveWell estimates cost-effectiveness
- 3. How many people are reached?
-
4. What impact does the Cash for Poverty Relief program have?
- 4.1 Summary
- 4.2 Consumption benefits to recipients
- 4.3 Consumption benefits to non-recipients (spillovers)
- 4.4 Mortality benefits to recipients
- 4.5 Additional benefits and downsides
- 4.6 How are donations allocated across countries?
- 5. Additional perspectives beyond our cost-effectiveness model
- 6. Sources
- 7. Appendix
Summary
Basics
GiveDirectly’s Cash for Poverty Relief program involves sending one-off cash transfers of ~$1,000USD (nominal) via mobile money platforms to households living in poor regions of low-income countries in sub-Saharan Africa. These transfers are unconditional: GiveDirectly places no restrictions on what people can spend their money on, and eligibility isn’t conditional on certain behaviors (e.g. sending children to school). Recipients are informed that these transfers are one-off – i.e. that they shouldn’t expect additional transfers from GiveDirectly in the future.
Currently, the program is operational in 5 countries: Kenya, Malawi, Mozambique, Rwanda, and Uganda. In deciding where to distribute transfers, GiveDirectly targets villages they haven’t targeted before in regions with high absolute poverty rates, but also considers neglectedness, government priorities and permissions to operate, and factors related to operational feasibility (e.g. whether there is good cell network coverage). Targeting is universal within villages where they expect >80% of residents live below the extreme poverty line: every household is typically eligible to receive a transfer, provided there is at least one member that permanently resides in the village and is over the age of 18, and nearby villages receive transfers more or less simultaneously. Transfers are sent via mobile money platforms; if households aren’t in possession of a mobile phone, GiveDirectly offers to give them one and then deduct the cost from the transfer amount.
How cost-effective is it?
As of October 2024, our best guess is that donations to GiveDirectly’s Cash for Poverty Relief program are ~3-4x more cost-effective than we had previously estimated, and around ~30-40% as cost-effective as our marginal funding opportunity.1 Our best guess varies from ~3x in Kenya to ~4x in Malawi. Cross-country differences are driven by our estimate that the average recipient is poorer in Malawi vs. Kenya, and our assumption that absolute consumption gains are more welfare-enhancing for poorer households (i.e., that there is diminishing marginal utility of consumption). We may update this estimate in future in light of research that’s currently in progress.
We think that around half the benefits of GiveDirectly’s Cash for Poverty Relief program come from consumption gains in households that receive cash transfers, which have been extensively researched through several randomized trials. Intuitively, a cash transfer enables people to spend more on food, furniture, and productive assets like farm tools. We don’t think there is credible evidence that GiveDirectly’s cash transfers increase the consumption of temptation goods (or ‘bads’) like alcohol or cigarettes.
With GiveDirectly’s current default program design (universal within-village targeting and simultaneous disbursement), we think households in nearby villages that don’t receive transfers are also likely to see consumption gains, as they benefit from a virtuous spending cycle brought about by an increase in local demand. Intuitively, households that receive cash are likely to spend some of it in nearby villages, which puts more money in the pockets of small business owners, who then spend it on other locally produced goods and services. Our estimates of these ‘spillover’ effects are informed by a recent randomized evaluation of GiveDirectly’s Cash for Poverty Relief program in Kenya (Egger et al., 2022), which finds large positive consumption spillovers to non-recipient households. We adjust these results downwards towards a more skeptical prior, which is partly informed by the broader evidence base, and to account for how likely we think these results are to generalize to more saturated program designs in poorer contexts, which we think characterizes the current programmatic context.
We also expect there to be health benefits from the Cash for Poverty Relief program – for example, with more cash in their pocket, recipient households might purchase more food and medicine, or invest in health-related home improvements (e.g. new toilets or floors). We model under 5 child mortality effects based on preliminary results we’ve seen from a randomized controlled trial (RCT) of a Cash for Poverty Relief program in Kenya, which finds a 46% reduction in all-cause under 5 mortality. We make a steep 50% adjustment to this result, but even if we took it at face-value, it doesn’t make a big difference to our overall cost-effectiveness estimates. Intuitively, we don’t think the Cash for Poverty Relief program represents a cost-effective opportunity to save lives because, unlike the programs supported by our Top Charities (e.g. seasonal malaria chemoprevention), the program is expensive per household and isn’t as targeted at medically vulnerable populations.
We quantify these benefits using a cost-effectiveness analysis, which allows us to compare across different programs. Here is a sketch, using Rwanda (our median cross-country estimate) as an example:
Best guess | Confidence intervals (25th - 75th percentile) | Implied cost-effectiveness | |
Program reach (more) | |||
---|---|---|---|
Donation to Cash for Poverty Relief (nominal USD, arbitrary value) | $1,000,000 | ||
Transfer size per household (nominal USD) | $1,100 | ||
Transfers as a % of total cost | 85% | 80 - 90% | 3.0 - 3.4x |
Total households receiving a transfer | 850 | ||
Average household size | 4.3 | ||
Total people directly reached | 3,323 | ||
Baseline annual consumption per capita (PPP 2017) (more) | $533 | $453 - $613 | 3.6 - 2.9x |
Transfer size per capita (PPP 2017) | $715 | ||
% increase in consumption per person in year 1 (more) | 76% | 65 - 88% | 2.9 - 3.5x |
Year 2-10 consumption gains (as % of year 1 consumption gains) (more) | 134% | 80 - 188% | 2.5 - 3.9x |
Moral weight assigned to doubling consumption for 1 person for 1 year (more) | 1 | 0.8 - 1.2 | 2.5 - 3.9x |
Units of value from recipient consumption gains | 5,736 | ||
Initial cost-effectiveness estimate (x prev estimate) | 1.7 | ||
Consumption benefits to recipients (units of value) | 5,736 | ||
Consumption spillovers to non-recipients (units of value) (more) | 3,630 | 338 - 5,673 | 2.2 - 3.8x |
Mortality benefits (units of value) (more) | 856 | 439 - 1,282 | 3.1 - 3.3x |
Additional benefits and downsides (units of value) (more) | 818 | ||
Overall cost-effectiveness estimate (x prev estimate) | 3.3 | 2.0 - 4.1x | |
How could we be wrong?
Our main uncertainties, in rough order of importance, are as follows:
- How big are consumption spillovers to non-recipients?
We think consumption benefits to non-recipient households are likely to be positive, and a sizable fraction (~60-70%) of the consumption benefits to recipient households. This is informed by a recent evaluation of a GiveDirectly lump sum program in Kenya which found large consumption spillovers to non-recipients (Egger et al., 2022). If we took this result at face-value, it would imply that consumption spillovers comprise ~180% of the consumption benefits of recipients, which would shift our cost-effectiveness estimates to ~4-6x. Though we think this paper is high quality, we adjust this result downwards because of: i) large standard errors on the estimates; ii) the paper's inability to observe net exports (which we think is likely think to bias their estimates slightly upwards); iii) the result seeming incongruous with previous spillover estimates of GiveDirectly’s lump sum program; iv) and the results being surprisingly large; larger than what our prior would have been. The adjustments we apply for this are speculative, and we think people could reasonably disagree about how much weight to place on this finding (more)
- How likely are these spillovers to generalize to different contexts?
The Egger et al. paper evaluates a GiveDirectly lump sum program in Western Kenya that employed within-village means-testing, with only households with a thatched roof being eligible for a transfer. To estimate spillovers in current programmatic contexts, a key question is how we expect these results to generalize to more saturated program designs (where everyone in a village is eligible) and in potentially poorer, more remote settings (e.g. rural Malawi). We’d expect spillovers to be slightly smaller in these contexts, though our adjustments for this also speculative (more)
- How persistent are consumption gains to recipients?
Our best guess is that households see a sharp increase in consumption following receipt of transfers, but that these consumption gains fade-out quickly over time. We’ve been sent preliminary 5-7 year follow-up data from the experiment studied in Egger et al. (2022) which shows more persistent consumption gains compared to what’s implied by our assumptions. If we put these preliminary results in at face-value, our cost-effectiveness estimates would change to ~4-6x. At the moment, we aren’t putting a lot of weight on these results because they haven’t been subject to external scrutiny and seem inconsistent with other long-run follow-ups of unconditional cash transfer programs. We may change our mind about this as this result goes through academic peer review (more)
- How much should we value raising consumption vs. saving lives?
Our overall cost-effectiveness estimate of the Cash for Poverty Relief program depends critically on how much we value consumption gains as opposed to saving lives. We convert these outcomes into a consistent unit of value using our ‘moral weights’, which are based on surveys of donors, staff, and beneficiaries in Kenya and Ghana that present hypothetical thought experiments designed to elicit these trade-offs. We think there are significant limitations of using the preferences stated on these surveys, and think people could reasonably disagree about the relative importance of these outcomes (more)
- Are we modeling spillovers consistently across programs?
GiveWell makes relative funding recommendations: we try to identify the best funding opportunities, relative to other options. Because of this, we try to be consistent about how we model different benefit streams across programs. If we think there are positive consumption spillovers from the GiveDirectly program, one outstanding question we have is whether we should also expect positive spillovers from other livelihood programs we’ve looked at, or from health programs that we expect to increase recipients’ consumption. We plan to look into this further after publishing this update.
- How are consumption gains distributed across households?
A key assumption that underpins our cost-effectiveness modeling of livelihoods programs is diminishing marginal utility of consumption. While we think it’s intuitive that given consumption gains are more impactful for poorer households compared to richer households, we feel uncertain about how consumption gains from the Cash for Poverty Relief program are distributed. Our best guess is that long-run consumption gains and spillovers to non-recipient households are likely to flow disproportionately to relatively richer households (e.g. small business owners vs. subsistence farmers), since these households seem better positioned to capture the gains from increased economic activity. However, our adjustments for this are speculative, and we haven’t dived deeply into the household-level data of the GiveDirectly trials (more)
- How poor is the average household in villages GiveDirectly targets?
Since we assume diminishing marginal utility of consumption, our model is very sensitive to how poor we think the average recipient household is. Our best guess is that the average household that receives Cash for Poverty Relief transfers is below the international extreme poverty line in each of the 5 countries where GiveDirectly operates, though we expect the average household is meaningfully poorer in Malawi and Mozambique vs. Kenya. Our estimates are based on government consumption surveys and baseline consumption measures of RCTs of the GiveDirectly program, but both these inputs are generally outdated and don’t always agree with each other (more)
1. The basics of the program
1.1 What is GiveDirectly?
GiveDirectly is a US-based not-for-profit focused predominantly on delivering unconditional cash transfers (UCTs) to poor people living in low-income countries. It currently operates 6 program categories:
- Cash for Poverty Relief: transfers an unconditional one-off transfer of ~$1,0002 via mobile money to households living in poor regions of low-income countries (LICs).3 The program is currently operational in 5 sub-Saharan Africa countries4
- Basic Income: transfers monthly unconditional transfers of ~$40 to households living in poor regions of LICs. The program is currently operational in 4 sub-Saharan African countries5
- Emergency Relief: transfers unconditional cash transfers to households recently affected by global crises or natural disasters. The size and location of these transfers depends on the crisis (though is typically between $200-$500).6 Since 2017, GiveDirectly has delivered emergency cash remotely and in-person to 13 countries in Africa, Asia, and North America, across crises ranging from natural disasters to civil conflict7
- Refugees: transfers unconditional cash transfers to recipients registered as refugees in African countries.8 These transfers are typically between $800-$2,000, and can be either lump sum or spread out via monthly transfers9
- Climate Survival: transfers unconditional cash transfers to recipients in non-US countries where GiveDirectly operates. Donations can go towards post-disaster emergency relief or resilience building10
- Cash in the US: transfers unconditional cash transfers to low-income households in the USA. This work is funded by US-restricted and general unrestricted donations11
Via their website, GiveDirectly offers donors the opportunity to donate to any cash transfer program in Africa or earmark to one of these 6 program. This report focuses on the marginal impact of donations to GiveDirectly’s Cash for Poverty Relief program, which has historically (and continues to) absorb the majority of GiveDirectly’s funding.12 GiveDirectly’s funding allocations across programs in 2023 are below:
1.2 How does their Cash for Poverty Relief program work?
GiveDirectly’s flagship Cash for Poverty Relief program entails transferring an unconditional one-off lump sum transfer of ~$1,000 (nominal)13 via mobile money to households living in poor regions in 5 sub-Saharan Africa countries: Kenya, Malawi, Mozambique, Rwanda, and Uganda.14 To date, the number of households GiveDirectly has reached with lump sum transfers in these countries ranges from 5,000 – 130,000;15 significantly less than the number of households the World Bank estimates to be living in extreme poverty. This section describes how this program functions.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Population estimate (million) | 54.0 | 20.4 | 33.9 | 14.1 | 47.2 |
Average rural household size | 4.2 | 4.5 | 4.4 | 4.3 | 4.8 |
Estimated number of households (million) | 12.9 | 4.5 | 7.7 | 3.3 | 9.8 |
Poverty headcount rate (%) | 29% | 70% | 75% | 52% | 43% |
Estimated number of households living in extreme poverty (million) | 3.8 | 3.2 | 5.7 | 1.7 | 4.2 |
Number of households that have received large one-off GD transfers | 122,997 | 78,579 | 5,485 | 120,393 | 78,929 |
Max % of households in extreme poverty that could have received a lump-sum GD transfer16 | 3.3% | 2.5% | 0.1% | 7.1% | 1.9% |
How does GiveDirectly decide which countries to work in?
GiveDirectly works in countries which: i) have a high burden of absolute poverty; ii) are feasible to operate in; iii) where they have express permission from the government to operate.17 The Cash for Poverty Relief program has been operational in Kenya since 2011; programming in Uganda, Rwanda, Malawi, and Mozambique began in 2013, 2016, 2019 and 2023, respectively.18 The amount of funding that has been allocated to the Cash for Poverty Relief program across each of these countries since 2018 is shown below.
How does GiveDirectly allocate funding across countries?
When deciding which countries to allocate marginal Cash for Poverty Relief funding to, GiveDirectly considers: i) program efficiency (and whether an expansion of programming could increase efficiency through scale economies); ii) country capacity constraints; and iii) ‘big bets’ – i.e. whether they think giving to a certain country could demonstrate impact or crowd in more funding.19
GiveDirectly uses similar targeting criteria within countries, targeting regions that have high rates of absolute poverty, are feasible to operate in while maintaining efficiency targets, and aren’t already receiving large amounts of aid from the government or other non-government organizations.20
Local poverty rates are usually the most important targeting criterion.21
To rank regions according to poverty rates, GiveDirectly will typically rely on consumption estimates from government surveys, though will occasionally take into account other metrics (e.g. stunting rates) depending on the government's priorities.22
When assessing operational feasibility, one thing GiveDirectly considers is cell network availability, since this is required to receive mobile-money based cash transfers.
How are households enrolled in the program?
Once regions are identified, GiveDirectly staff (known as ‘field officers’) host a community sensitization meeting in each village being targeted.23 The field officer explains what GiveDirectly is, how the program works, and what people need to do if they wish to participate. They also explain that the transfer is unconditional – i.e. people can spend the money on whatever they want – and that it is one-off – i.e. they shouldn’t expect additional transfers in the future.24 At least one member from each household in the village typically participates. Following the sensitization meeting, field officers go door-to-door to cross-check households against residency lists usually given to them by local leaders.25
After a census of households has been taken, registration takes place. All recipients must consent to receive a transfer: since 2020, the average refusal rate has been 0.2% across Cash for Poverty Relief programs.26 Participants also fill out an eligibility survey. Previously, eligibility was determined by living in a house with a thatched roof, which was used as a rough means-test for poverty.27 Since 2017, GiveDirectly has moved towards universal within-village targeting.28 This is now their default design, but occasionally they will layer on some form of needs-based targeting if mandated by the government.29
To be eligible for a transfer, participants have to permanently reside in the village being targeted.30 In Malawi, everyone over the age of 18 is eligible to receive a $550 transfer,31 which means a ~$1,000 transfer for a two-adult household. In Kenya, Mozambique, Rwanda and Uganda, transfers are targeted at households rather than individuals, with each household eligible to receive a transfer of ~$1,000, though the exact amount can vary because of exchange rate fluctuations.32
In theory, individuals or households could be missed due to administrative error, or if no one is available during the multi-day enrollment process.33 GiveDirectly doesn’t specifically collect data on this, though expects this to be very low. The average ‘written-off’ rate is 0.91% for lump sum programs since 202034 – this category includes those that were eligible for transfers but did not receive them, but also those who did receive transfers but were lost during follow-up because they moved location.35
GiveDirectly’s standard process is to register and pay households in nearby eligible villages more or less simultaneously.36
How is the cash disbursed?
All Cash for Poverty Relief transfers are dispersed through a SMS-enabled banking technology called mobile money.37 This requires people to have access to a mobile phone and to a cell network, but internet access is not required. If recipients don’t own a mobile phone, GiveDirectly offers to give them one and deduct the cost from the transfer amount.38 Recipients are required to register with the mobile money platform (e.g. M-PESA) if they don’t have an account; this typically requires a form of formal identification (e.g an ID card or passport).39 GiveDirectly field officers will assist people with this if necessary, and in some cases will coordinate with the government to issue IDs during the enrollment.40
About a month after registration, households are sent the transfer in two parts through mobile money. The total amount disbursed per household is typically ~$1,00041 sent in two chunks – the first ‘token’ transfer is typically 20-30% of the total transfer size, with the remainder being sent a month later.42 Once the money has been received, recipients can withdraw the mobile funds as cash with local agents, who receive commission from the mobile money platforms. GiveDirectly estimates there are 334,000 mobile money agents in Kenya, 145,000 in Rwanda, 472,000 in Uganda, and 24,000 in Mozambique.43
How are disbursements monitored?
After funds are disbursed, GiveDirectly staff call each recipient to verify whether they received the funds and ask questions about their experience of the program. Their internal audit team then follows up with a subset of recipients to identify and investigate potential fraud.44 These are randomized 1:1 follow-ups to check the work of the main enrollment teams, plus targeted investigations if there's a report of a suspected adverse event filed by a recipient or staff member.45 GiveDirectly’s approach to fraud is discussed more in this blog post.
What do recipients spend the money on?
Based on survey data sent to us by GiveDirectly, between 2021 and 2024, the following percentage of recipients of their Cash for Poverty Relief program across Rwanda, Kenya, Liberia, Malawi, Uganda, and Mozambique reported spending some of their cash transfer on the below categories.46
Category | Percentage of recipients |
---|---|
Housing | 63% |
Food | 62% |
Household essentials | 46% |
Livestock | 36% |
Education | 27% |
Savings | 24% |
Agriculture | 16% |
Business investment | 10% |
Other | 10% |
Healthcare | 9% |
We don’t think there’s evidence that recipients use their transfer to buy temptation goods (or ‘bads’) like alcohol and cigarettes. Of the 5 randomized trials of GiveDirectly lump sum transfers that we found, 3 measured effects on temptation goods specifically, and none found that the transfers led to statistically significant or meaningful increases.47 The lack of positive effect is consistent with the more general finding of Evans and Popova (2017), who review 19 studies and find that on average, cash transfers have a negative effect on total expenditures on temptation goods.48
2. How GiveWell estimates cost-effectiveness
GiveWell recommends programs that we believe save or improve lives as much as possible for as little money as possible. To estimate this, we produce a cost-effectiveness analysis (“CEA") which aims to produce a best guess of the overall impact of a program per dollar donated. We use "moral weights" to quantify the benefits of different impacts (e.g., increased consumption vs reduced deaths). These moral weights are based on a combination of staff, donor, and beneficiary stated preferences, and reflect highly subjective value judgments. We benchmark to a value of 1, which we define as the value of doubling someone’s consumption for one year. The main moral weights we use for our analysis of GiveDirectly are in the table below.
Benefit | Moral weight(units of value per outcome) |
---|---|
Doubling consumption for one person for one year | 1 |
Increasing ln(consumption) by one unit for one person for one year49 | 1.44 |
Averting the death of a child under five | 116 |
Other parameters | |
Discount rate on future consumption gains50 | 4% |
In this report, we estimate separate cost-effectiveness estimates for each country where GiveDirectly’s Cash for Poverty Relief program is currently operating. We also estimate an aggregated cost-effectiveness estimate, which takes a weighted average of these country estimates based on how we expect marginal donations to be allocated across countries. This represents our best-guess of the cost-effectiveness of a donation to the Cash for Poverty Relief program via GiveDirectly’s website. At the time of writing (October 2024), all donations to ‘Poverty Relief - Africa’ are earmarked for the Cash for Poverty Relief program.51
This report and accompanying cost-effectiveness analysis include 25th - 75th percentile confidence intervals for specific parameters. See the summary table above and this sheet of our cost-effectiveness analysis. These intervals are based on GiveWell staff members’ subjective levels of uncertainty for each parameter; we consider them very speculative, though helpful in portraying which parameters we feel most uncertain about.52
3. How many people are reached?
We estimate that each $1m spent by GiveDirectly on Cash for Poverty Relief programming leads to ~740 to ~970 households receiving a lump sum transfer, which directly affects ~3,300 to ~4,100 people. This varies by country depending on the average size of the transfer per household and program efficiency – i.e. the % of each dollar donated that reaches recipients (as opposed to going towards program overheads).
Our calculations are outlined in this spreadsheet and below. Transfer size and program efficiency data are both based on data sent to us by GiveDirectly.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Arbitrary donation (nominal USD) | $1,000,000 | $1,000,000 | $1,000,000 | $1,000,000 | $1,000,000 |
Transfers as a percentage of total cost | 84% | 84% | 75% | 85% | 71% |
Total size of transfer per household (nominal USD) | $865 | $1,000 | $1,000 | $1,100 | $955 |
Number of households receiving transfer | 971 | 840 | 750 | 773 | 743 |
Average rural household size | 4.2 | 4.5 | 4.4 | 4.3 | 4.8 |
Number of people directly reached | 4,079 | 3,780 | 3,300 | 3,323 | 3,569 |
4. What impact does the Cash for Poverty Relief program have?
4.1 Summary
Our cost-effectiveness analysis models 3 main benefits from the Cash for Poverty Relief program:
- Consumption benefits to people that receive transfers
- Consumption benefits to nearby people that don’t receive transfers (‘spillovers’)
- Reductions in child and adult mortality
We also include smaller supplementary adjustments to account for additional benefits, such as cash transfers improving the health of those that receive them (morbidity effects), and possible downsides, such as money being misappropriated or lost to fraud.
This section walks through how we calculate ‘units of value’ – what we use to measure impact – for each of these benefit streams. A breakdown of each stream's contribution to our overall cost-effectiveness estimates are outlined in this spreadsheet and below. We think the most important benefits from the Cash for Poverty Relief program come from consumption gains of recipient households, followed by spillovers to nearby non-recipient households.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Consumption benefits to recipients | 48% | 53% | 49% | 52% | 49% |
Consumption benefits to non-recipients | 32% | 31% | 29% | 33% | 31% |
Child and adult mortality benefits | 12% | 9% | 14% | 8% | 13% |
Additional benefits and downsides | 7% | 7% | 7% | 7% | 7% |
Across all benefit streams, our key uncertainties are:
- Our estimates of consumption benefits to non-recipients depend on how we expect previous experimental estimates to generalize to current program contexts, and we feel very uncertain about our adjustments (more)
- These estimates also depend on how much weight we choose to place on a particularly influential study (Egger et al.) vs. our prior and the wider evidence base (more)
- Our estimates of the consumption benefits to recipients depend on how persistent we expect these to be. We currently assume consumption gains to recipients fade out across time, but we’ve seen preliminary results from one study potentially at odds with this assumption (more)
4.2 Consumption benefits to recipients
Summary
We expect recipients of Cash for Poverty Relief transfers to see a sharp increase in their consumption in the 12 months immediately following the transfer. We expect consumption gains to last for 10 years – as recipients purchase more food, livestock, and productive assets – though expect consumption gains to fade out quickly after the initial spike. We think these assumptions match the general shape of the evidence base, which shows large but transient consumption gains to recipients following the disbursement of cash (more).
This section of our cost-effectiveness analysis calculates units of value from consumption gains to recipients. A summary is outlined below. The cross-country differences are driven by:
- Differences in baseline consumption across countries (more)
- Program efficiency being higher in some countries vs. others, which allows more households to be reached for a given donation size (more)
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Baseline annual consumption per capita (PPP 2017) | $652 | $470 | $450 | $533 | $626 |
Size of transfer per person (PPP 2017) | $488 | $651 | $639 | $715 | $589 |
Percentage of transfers consumed in year 1 | 57% | 57% | 57% | 57% | 57% |
Implied increase in consumption per person in year 1 | $278 | $371 | $364 | $407 | $336 |
Percentage increase in year 1 consumption over baseline | 43% | 79% | 81% | 70% | 54% |
Immediate increase in ln(consumption) per person | 0.36 | 0.58 | 0.59 | 0.57 | 0.43 |
Value assigned to increasing ln(consumption) by one unit for one person for one year | 1.44 | 1.44 | 1.44 | 1.44 | 1.44 |
Units of value from year 1 consumption gains per person | 0.51 | 0.84 | 0.86 | 0.82 | 0.62 |
Number of people reached given arbitrary donation | 4,079 | 3,780 | 3,300 | 3,323 | 3,569 |
Units of value from year 1 consumption gains given program reach | 2,090 | 3,175 | 2,825 | 2,723 | 2,213 |
Value of year 2-10 consumption returns as % of year 1 consumption returns | 123% | 136% | 140% | 134% | 127% |
Units of value from year 1-10 recipient consumption gains | 4,663 | 7,496 | 6,772 | 6,374 | 5,019 |
Distributional adjustment (consumption gains accruing to richer recipients) | 90% | 90% | 90% | 90% | 90% |
Units of value from year 1-10 consumption gains post adjustments | 4,196 | 6,746 | 6,095 | 5,736 | 4,517 |
Our main uncertainties are:
- How persistent are consumption gains to recipient households? We’ve been sent preliminary (unpublished) results from a 5-7 year follow-up of a randomized evaluation of a GiveDirectly lump sum program in Kenya, which suggests more persistent consumption gains compared to what’s implied by our assumptions (more)
- Do richer households see greater consumption gains from cash transfers? We think there’s evidence of this, but the adjustment we make for it is speculative (more)
How poor is the average Cash for Poverty Relief recipient?
We value consumption gains relatively: our moral weights assign a value of 1 doubling consumption for one person for one year (more). Concretely, this means we think a $1,000 annual consumption gain for someone consuming $1,000 worth of goods and services a year is more valuable than a $1,000 gain for someone consuming $2,000 worth a year.53
This means it matters how poor we think recipients are before they receive cash transfers.
Our best-guess estimates are presented below. Overall, we think the average Cash for Poverty Relief recipient is likely below the international extreme poverty line54
in each country that the program operates. We expect there to be meaningful variations across countries, with recipients in Malawi and Mozambique being poorer than recipients in Kenya.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
GiveWell estimate of annual consumption per capita of GiveDirectly recipients | $652 | $470 | $450 | $533 | $626 |
Implied consumption per day | $1.79 | $1.29 | $1.23 | $1.46 | $1.71 |
International extreme poverty line | $2.15 | $2.15 | $2.15 | $2.15 | $2.15 |
Implied poverty gap (%)55 | 17% | 40% | 43% | 32% | 20% |
Our estimates are based on two inputs: bottom quintile consumption estimates from government consumption surveys and baseline consumption surveys from randomized controlled trials (RCTs) of GiveDirectly’s Cash for Poverty Relief program. We use average consumption of households in the bottom quintile of the overall consumption distribution to proxy Cash for Poverty Relief recipients, since GiveDirectly deliberately targets poor regions in each country.
The step-by-step process we take to convert these inputs into our best guess is described in this section of the Appendix. In this section, we also subject our best guess estimates to basic sense checks, such as comparing them to other international poverty benchmarks and checking whether the relative ordering of countries makes sense, given other cross-country differences in other plausible indicators of material well-being (e.g. stunting rates).
Overall, we feel uncertain about these estimates as both the inputs we use are outdated and don’t perfectly triangulate with each other. We can also think of reasons why bottom quintile consumption estimates from government surveys may not be a perfect proxy for Cash for Poverty Relief recipients – e.g. if operational and efficiency considerations prevent GiveDirectly from targeting the very bottom of the consumption distribution.56 With this being said, we generally feel more confident in these estimates compared to our assumptions about expenditure gains following the receipt of cash transfers. These are discussed below.
How much does a Cash for Poverty Relief transfer increase expenditure?
Summary
We think Cash for Poverty Relief recipients see large gains in consumption immediately after receiving the transfer, as they spend just over half (~60%) of the transfer in the first year. We expect consumption gains to persist for 10 years, but to fade out after this initial spike, as recipients spend down the rest of the transfer and the increase in local economic activity starts to fizzle out.
We think these assumptions seem mostly consistent with the findings of 7 studies (across 5 randomized trials) of the Cash for Poverty Relief program, and studies of other UCT programs. Short-run (<1 year) follow-ups of the program generally find large consumption gains to recipients compared to non-recipient households (Haushofer and Shapiro, 2016; McIntosh and Zeitlin, 2024; Aggarwal et al., 2024). Medium-run (1-3 year) follow-ups of the program also find positive consumption gains, but these seem generally smaller than short-run gains (Egger et al., 2022; Haushofer and Shapiro, 2018; Banerjee et al., 2023). We found no published trials with long-run (3+ year) follow-ups of the Cash for Poverty Relief program, but long-run follow-ups of other UCT programs generally suggest a fade-out in recipient consumption returns over time (Blattman et al., 2016; Blattman et al., 2020).
Our biggest uncertainty stems from preliminary results we’ve been sent from a 5-7 year follow-up of Egger et al. (2022), one of the randomized trials we use to estimate recipient consumption gains. These preliminary results suggest much more persistent recipient consumption gains compared to what’s implied by our assumptions and the broader evidence base. At the moment, we aren’t putting much weight on these results as they’ve not been externally scrutinized, but may update as these results get peer reviewed (more).
What evidence is there on recipient consumption gains?
From a literature review of GiveDirectly’s lump sum transfer program,57 we found 7 studies related to 5 different randomized evaluations. Two of the studies were follow-ups to experiments initially evaluated in other studies. All of these trials evaluated GiveDirectly’s Cash for Poverty Relief Program against a control arm which didn’t receive cash transfers. Trials differed in terms of where they were conducted, program design, the size of the transfers being disbursed, and the timing of endline.
Three of the studies (Egger et al. 2022; Egger et al. (preliminary) and Banerjee et al. 2023) have co-founders of GiveDirectly as listed coauthors.58 We asked GiveDirectly what Paul Niehaus’ involvement was in the Egger et al. paper, and they said other members of the study team took the lead in writing the narrative sections of the paper, and also published a pre-analysis plan before analyzing the data to mitigate risk of biasing results.59 His involvement doesn’t change our interpretation of these papers.
Paper | Country | Program design | Transfer size per household | Timing of endline | Measures spillovers? |
---|---|---|---|---|---|
Haushofer and Shapiro (2016) | Kenya | Within-village means testing | $300-$1,000 ($709 avg) | ~9 months | Yes |
Haushofer and Shapiro (2018) | ~3 years | Yes | |||
McIntosh and Zeitlin (2024) | Rwanda | Within-village means testing60 | ~$500 | ~13 months | Yes |
Aggarwal et al. (2024) | Malawi and Liberia | Universal targeting within villages | $250-$750 | 0-24 months | No |
Egger et al. (2022) | Kenya | Within-village means testing | $1,000 | ~18 months | Yes |
Egger et al. (preliminary results) | ~5-7 years | Yes | |||
Banerjee et al. (2023) | Kenya | Universal targeting within villages | ~$1,00061 | ~2 years | Yes |
To trace expenditure gains across time, we break these papers down by their timing of follow-up:
Short-run (<1 year) expenditure impacts of the Cash for Poverty Relief program
- Haushofer and Shapiro (2016) find that, 9 months after cash was disbursed, recipient households (across both large and small transfer sizes) saw 22% higher monthly expenditure compared to control households62
- McIntosh and Zeitlin (2024) finds that, 13 months after cash was disbursed, household expenditure increases by 30%63
- Aggarwal et al. (2024) collects granular month-by-month expenditure data, which allows them to dynamically trace the spending response across time. They find large initial expenditure gains in Malawi, which fizzle out over the course of 24 months.64 The authors don’t detect a statistically significant expenditure response in Liberia65
Medium-run (1-3 year) expenditure impacts of the Cash for Poverty Relief program
- Egger et al. (2022) find that, 19 months after cash was disbursed (median timing of follow-up),66 annualized expenditure in recipient households was 13% higher than the expenditure of households in distant control villages67
- Banerjee et al. (2023) study the effects of GiveDirectly’s Universal Basic Income program, but also include a Cash for Poverty Relief arm. Compared to the control group, recipients that received Cash for Poverty Relief transfers saw 6% higher annual consumption ~2 years after the transfers had been disbursed68
- Haushofer and Shapiro (2018) follow-up on the same experiment studied in Haushofer and Shapiro (2016) 3 years later. They find that monthly consumption of recipient households was 9% higher vs. non-recipient households in control villages,69 though this difference is not statistically significant
Long-run (3+ year) expenditure impacts of the Cash for Poverty Relief program
The only long-run consumption estimates of the Poverty Relief program we’ve seen are preliminary results from a 5-7 year follow-up of the same experiment studied in Egger et al. (2022). These results aren’t available online, but were sent to us via email correspondence with the authors (NB: GiveWell funded this follow-up study).70 These initial results imply persistent consumption gains: annualized expenditure in recipient households was 12% higher compared to households in distant control villages – implying almost no fade-out in recipient consumption gains across time.71 At the moment, we don’t place a lot of weight on these results because they’ve not been externally scrutinized; we discuss this more in our key uncertainties section.
Since we lack published long-run evidence on the Cash for Poverty Relief program, we did a brief literature review for long-run follow-ups of lump sum UCT programs more generally. Crosta et al. (2024) conduct a meta-analysis of UCTs in low and middle-income countries, and find suggestive evidence of smaller consumption effects for lump sum (but not ongoing) UCTs in long-run.72 However, their sample is also limited: they only have two data points of lump sum transfers with >3 year follow-ups, both of which evaluated an unconditional lump sum transfer of ~$400 in Uganda (Blattman et al. 2016; Blattman et al. 2020). These papers generally find evidence of a fade-out of consumption gains: in contrast to significant consumption differences found 2 and 4-years post transfers, the 9-year follow-up found that treatment and control groups had converged in consumption (and employment and earning) levels.73
Beyond the papers included in this meta-analysis, we found two other papers from a Google search which seem relevant:
- Fiala et al. (2022) is a working paper that tracks the same Ugandan households studied in Blattman et al. 12 years later, during the COVID-19 pandemic. They find that some positive effects resurface: treated men are significantly more likely to be engaged in an income generating activity, though this does not translate into higher food security. 74 The paper doesn’t measure consumption directly, and finds no effects for women75
- Blattman et al. (2022) look at the impact of an unconditional $300 cash transfer to unemployed youth in Ethiopia.76 After 1 year, recipient earnings were higher than non-recipients, but by year 5, there was almost complete convergence in employment and earnings.77
How we convert consumption gains to units of value
Our overall impression of this evidence is that it points to large initial consumption gains following the receipt of cash transfers, which gradually fade-out across time. To build a cost-effectiveness estimate which fits the shape of this pattern, we make the following assumptions:
- Consumption gains from cash transfers are shared evenly within households – i.e. if household consumption increases by 25%, every member's consumption increases by 25%. We discuss the limitations of this assumption in this section
- Households save or invest78 ~40% in the first year; the remainder of the transfer goes towards immediate household consumption (e.g. spending on food, fuel, durables). This seems consistent with the short-run expenditure patterns documented in Haushofer and Shapiro (2016) and Egger et al. (2022)79
To estimate consumption returns in years 2-10, we make three additional assumptions which we think generates a function that broadly maps to the experimental estimates.80 We assume:
- In year 2, households consume 40% of what was initially saved/invested. This could be through consumption gains realized through spending down transfer savings, or through returns from productive investments (e.g. greater yields through investing in farming equipment)
- In years 2-10, household consumption decays at a rate of -20% a year, leading to a gradual fade-out in consumption gains (e.g. as they spend down the entirety of the transfer or increases in local economic activity start to fizzle out)
- Counterfactually – i.e. in the absence of any cash transfers – consumption of recipient households would have grown by 1-2% per year81
These assumptions generate the following consumption returns profile in Kenya:
To convert these into a cost-effectiveness estimate, we combine these relative consumption gains with our estimates of program reach and our moral weights. To estimate year 1 consumption benefits in Kenya, our calculations are here and summarized below:
Kenya | |
Program reach | |
---|---|
Arbitrary donation (nominal USD) | $1,000,000 |
Total size of transfer per household (nominal USD) | $865 |
Number of households receiving transfer | 971 |
Average household size | 4.2 |
Number of people reached | 4,079 |
Consumption gains to recipients | |
Baseline annual consumption per capita (PPP 2017) | $652 |
Total size of transfer per household (PPP 2017) | $2,049 |
Total size of transfer per individual (PPP 2017) | $488 |
Percentage of transfer that is invested/saved | 43% |
Percentage of transfer that is consumed in year 1 | 57% |
Year 1 consumption gains per person | $278 |
Increase in ln(consumption) per person in year 1 | 0.36 |
Value assigned to increasing ln(consumption) by one unit for one person for one year | 1.44 |
Units of value from year 1 consumption gains per person | 0.51 |
Units of value from year 1 consumption given program reach | 2,090 |
To estimate consumption gains in years 2-10, we use assumptions 3-5) to estimate likely consumption patterns post-transfers vs. the counterfactual. In Kenya, the implied year 2-10 consumption gains are here and as follows:
Kenya | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Year 6 | Year 7 | Year 8 | Year 9 | Year 10 |
---|---|---|---|---|---|---|---|---|---|---|
Counterfactual consumption per capita | $652 | $665 | $678 | $692 | $706 | $720 | $734 | $749 | $764 | $779 |
Consumption gains | $278 | $84 | $67 | $54 | $43 | $34 | $27 | $22 | $18 | $14 |
Increase in ln(consumption) | 0.36 | 0.12 | 0.09 | 0.07 | 0.06 | 0.05 | 0.04 | 0.03 | 0.02 | 0.02 |
Discounted increase in ln(consumption) | 0.36 | 0.11 | 0.09 | 0.07 | 0.05 | 0.04 | 0.03 | 0.02 | 0.02 | 0.01 |
Discounted units of value | 0.51 | 0.16 | 0.13 | 0.10 | 0.07 | 0.06 | 0.04 | 0.03 | 0.02 | 0.02 |
Total units of value given people reached | 2,090 | 672 | 513 | 391 | 297 | 226 | 171 | 129 | 98 | 74 |
Relative consumption gains (%) | 43% | 13% | 10% | 8% | 6% | 5% | 4% | 3% | 2% | 2% |
Year 2-10 consumption gains (as % of year 1 consumption gains) | 124% | |||||||||
Key uncertainties
Our main uncertainty stems from preliminary results we’ve been sent from a long-run follow-up to Egger et al (2022). These results suggest that, 5-7 years later, recipient households had 12% higher consumption compared to households in distant control villages – almost the same difference as at the 2 year follow-up. If we take these results at face-value, and assume 0% fade-out in consumption gains after year 2, our overall cost-effectiveness estimates change as follows:82
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Final CE estimate with best-guess recipient consumption returns | 2.6 | 3.8 | 3.7 | 3.3 | 2.8 |
Final CE estimate with Egger et al. implied recipient consumption returns | 3.7 | 5.5 | 5.3 | 4.8 | 3.9 |
At the moment, we don’t put much weight on these results because they haven’t been subjected to external scrutiny, and seem generally inconsistent with other experimental findings. However, if these results make it through academic peer review, we would probably update towards them.
If these results survive, two outstanding questions we have are:
- How can we reconcile these results with Aggarwal et. al., Haushofer and Shapiro, and Blattman et al.? While we think there are compelling reasons to think Egger et al. might be better-placed to measure spillovers to non-recipients than Haushofer and Shapiro (more), we don’t think these reasons carry over to consumption gains to recipients,83 so we’d would want to know more about whether this study offers methodological advantages over others
- What are the theoretical mechanisms through which consumption gains could have been sustained? One way to view these results is that these one-off transfers alleviated some kind of ‘poverty trap’, which led to persistent gains. If this is the hypothesis, we’d want to know more about what the poverty trap might have been, and how this story fits with the wider evidence on UCTs, which we think generally finds limited evidence of ‘threshold’ poverty trap models84
Do richer households see larger consumption gains?
Summary
The treatment effects cited in the previous section are average treatment effects, estimated across the entire treated population. Since we value consumption gains more highly for poorer households, we need to take a stance on how we expect these average treatment effects to be distributed across households.
Based on intuition and one experimental study we’re aware of, we think richer households are more likely to see larger consumption gains from cash transfers (in both absolute and relative terms), and hence contribute disproportionately to average effect sizes. We make a 90% adjustment (a -10% downweight) to account for this.
What evidence is there on the distributional effects of cash transfers?
We found one RCT of the Lump Sum Transfer program that looks into this in detail: Haushofer et al. (2022) use the same data as Egger et al. (2022) to look at how recipient consumption gains vary according to baseline assets, which they proxy for material well-being. They find that, ~2 years after transfers, those estimated to be in the bottom 50% most deprived households at baseline had a 20% lower average consumption treatment effect (over the last 12 months) vs. the average, and a 40% lower average consumption treatment effect vs. the richest 50% of households.85
Theoretically, we might expect richer households to see greater returns because they have better investment opportunities, or are better-placed to capture the gains from increased localized spending. For example, a local textiles worker might have better investment opportunities than a subsistence farmer (e.g. purchasing a new sewing machine), and nearby recipients of cash transfers might be more likely to frequent his shop compared to a neighboring farm.
We apply a 90% adjustment to account for recipient cash transfers more likely accruing to richer households. We don’t think a larger adjustment than this is warranted because we expect ~60% of the transfer to fuel immediate consumption increases, and we’d expect this to be pretty evenly distributed given everyone receives the same transfer amount.
Key uncertainties
Our 90% adjustment is speculative – one way we could get a better handle on it is to look at the microdata from Haushofer et al. (2022) to look at the distribution of consumption gains more granularly. We didn’t do this because we didn’t expect it to make a big difference to our adjustment.
4.3 Consumption benefits to non-recipients (spillovers)
Summary
In theory, GiveDirectly’s cash transfers could affect the behavior and consumption of households that didn’t receive cash. If increased expenditure by recipient households sparks a general increase in economic activity in the region, non-recipient households might also benefit from this. For instance, a local shopkeeper that didn’t receive a transfer could benefit from having more customers come to their shop (a positive spillover). On the other hand, the increase in demand might cause the shopkeeper to raise their prices, which could harm other customers that didn’t receive transfers (a negative spillover).
Our best guess is that these general equilibrium effects point in the positive direction, and comprise ~60-70% of the direct consumption benefits to recipient households. This is informed by a recent paper by Egger et al. (2022), which finds large and positive consumption spillovers to non-recipients from a randomized evaluation of a GiveDirectly lump sum program in Kenya. If we took this estimate at face-value, we think it would imply a ~180% adjustment – i.e. almost tripling the direct consumption gains to recipients.
While we think this paper is high-quality, we don’t think we should take this result as-is. First, we don’t think this paper completely supersedes previous papers that have sought to estimate spillovers from lump sum GiveDirectly programs, and generally find much more muted effects. Second, the program evaluated in this study employed within-village means-testing, leading to a less concentrated cash injection vs. GiveDirectly’s current Cash for Poverty Relief program.86 We’d generally expect smaller spillovers under i) more saturated program designs and ii) in poorer/more remote contexts than Western Kenya, so make downwards adjustments to account for this. Finally, we think we ought to account for richer households (e.g. small business owners) being disproportionately likely to capture the benefits of increased economic activity, so make a further downward adjustment.
How these adjustments feed into our bottom-line are outlined here and summarized below. Altogether, we expect consumption spillovers to non-recipients to comprise ~60-70% of the consumption benefits to recipients. This is our best guess of the average effect across all programming sites in each country. We’d expect this average to mask a lot of heterogeneity – with some regions seeing much larger spillover effects than others – given the heterogeneous results of the underlying evidence base.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Units of value from recipient consumption benefits | 4,196 | 6,746 | 6,095 | 5,736 | 4,517 |
Egger et al. (2022) multiplier estimate | 2.5 | 2.5 | 2.5 | 2.5 | 2.5 |
Adjustment to remove recipient consumption benefits | 70% | 70% | 70% | 70% | 70% |
Internal validity adjustment | 60% | 60% | 60% | 60% | 60% |
External validity adjustment | 80% | 70% | 70% | 75% | 75% |
Distributional adjustment | 80% | 80% | 80% | 80% | 80% |
Consumption benefits to non-recipients (as a % of consumption benefits to recipients) | 68% | 59% | 59% | 63% | 63% |
Units of value from year 1-10 non-recipient spillover benefits | 2,833 | 3,985 | 3,600 | 3,630 | 2,859 |
Our biggest uncertainties are:
- How much weight we should put on the Egger et al. results (vs. previous studies or a more skeptical prior) (more)
- How we should expect this result to generalize to current program contexts (poorer remote regions and more saturated program designs) (more)
- How the gains from increased economic activity brought about by cash transfers are likely to be distributed across households (more)
This second point may be informed by ongoing research in Malawi, which is evaluating the inflationary response from a much more saturated GiveDirectly lump sum program. We expect to have updates in 2025, and may update our best guess in light of these.
What evidence is there of spillover effects to non-recipients of the Cash for Poverty Relief program?
Summary
Of the 7 randomized evaluations we found of GiveDirectly’s lump sum program, 6 try to measure consumption spillovers to non-recipient households. 4 of these papers find null or slightly negative effects, while two (both the Egger et al. papers) find positive spillovers.
The evidence
The headline findings of these papers are summarized below:
- Egger et al. (2022) and preliminary results: ~18 months after transfers were disbursed, this paper finds that nearby non-recipient households had 13% higher consumption compared to distant non-recipient households (who are presumed to be affected by the cash transfers). This increase is similar to recipient households.87 Importantly, the paper also finds minimal (0.1%) price inflation.88 5-7 years later, the preliminary results we’ve been sent suggest non-recipient households still have 8% higher consumption than distant non-recipients, though this difference is not statistically significant89
- Haushofer and Shapiro (2016) and (2018): both the 9 month and 3 year follow-up find evidence of negative consumption spillovers on non-recipients. At 9 months, households which didn’t receive transfers in treated villages had 4% lower consumption than households in control villages, where no one received transfers.90 At 3 years, this difference had widened to 16%91
- McIntosh and Zeitlin (2022):92 finds no statistically significant evidence of spillovers (positive or negative) to households that didn’t receive transfers within villages that did93
- Banerjee et al. (2023): fails to reject a null of no consumption spillovers to non-recipient households in neighboring villages that didn’t receive transfers,94 though doesn’t report point estimates
We don’t place much weight on the preliminary follow-up results of Egger et al. since they’ve not been externally scrutinized. Nonetheless, the published results (Egger et al. 2022) stand in stark contrast to the other papers, which find null or mildly negative spillovers. While this might imply heavy discounts are warranted to Egger et al., we also think this study is generally better-placed to measure spillovers compared to the other studies:
- First, we think it has more credible identifying assumptions than Haushofer and Shapiro, which relies on the assumption of no spillovers across short (~1km) distances to identify unbiased treatment effects.95 For both theoretical and empirical reasons, this seems like a dubious assumption,96 and one we’d expect to bias treatment effects downwards.97 We think Egger et al. (2022) relies on a more credible identifying assumption,98 so think it’s more likely to produce unbiased treatment effects
- Second, Egger et al. is able to leverage a lot more experimental variation – not only do they distribute a large amount of cash,99 they also deliberately vary treatment intensity above the village level,100 unlike Banerjee et al. (2023) and McIntosh and Zeitlin (2024). The authors of both of these papers note that their lack of statistical power means their failure to reject a null of ‘no spillovers’ may not be that meaningful101
Given these advantages, we think Egger et al. serves as a credible top-line to base our spillover estimates on, though we think we ought to make adjustments to it. These adjustments are walked through in the next section.
Our approach
To convert the spillover effects to non-recipients in Egger et al. into a cost-effectiveness estimate, we need to take into account the number of non-recipients per recipient treated. Intuitively, if recipients and non-recipients both see a 13% increase in consumption (as Egger et al. find),102 but there are a lot more non-recipients compared to recipients, then the total benefits to non-recipients should outweigh the benefits to recipients.
To account for the relative share of recipients vs. non-recipients in the context that they study, Egger et al. combine their experimental treatment effects with a census taken before the cash transfers took place. They estimate that for every household that received a transfer, around 4 nearby103 non-recipient households were indirectly affected.104 They use this census information to convert their consumption spillover estimates into an economic multiplier,105 which they estimate to be 2.5 at an ~18 month follow-up.106 Intuitively, this means that for every $1 that was injected into the study region, the authors estimate that $2.50 of economic activity was generated. We use this as our top-line to estimate spillover benefits to non-recipients.
Adjustment to strip out recipient consumption benefits
Egger et al. economic multiplier includes both: i) consumption gains to recipients and ii) consumption gains to non-recipients (spillovers). Since we’re capturing consumption gains to recipients elsewhere in our model (as explained here), we should strip these out to avoid double-counting. To do this, we apply a 70% adjustment to the 2.5 multiplier estimate.
To get to this adjustment, we:
- Estimate the increase in economic activity implied by a 2.5 economic multiplier for a given transfer amount
- Estimate how much of this increase is ‘explained’ by recipient consumption gains, based on our assumptions in this section
- Assign the rest of the multiplier to consumption gains to non-recipients
When we do this, we estimate that recipient consumption returns explain ~30% of a 2.5 economic multiplier, implying ~70% of the multiplier comes from non-recipient consumption gains. This is very similar to what Egger et al. estimate themselves.107
Adjustment to strip out recipient consumption gains | Notes | |
---|---|---|
Total transfer amount per person (denominator) | $488 | |
Transfer multiplier | 2.5 | Egger et al. result |
Implied economic activity generated (numerator) | $1,220 | Implied |
Recipient consumption returns by year 2 | $362 | Calculated in this section |
Implied adjustment | 70% | |
Internal validity adjustment
Summary
We apply internal validity (IV) adjustments to shrink experimental treatment effects towards our prior/what we would intuitively expect. Typically, the size of the adjustment we impose depends on: i) the quality of the estimate (taking into account things like sample size, plausibility of identification strategy, whether the strategy was pre-registered etc.); ii) how credible the theoretical explanation seems; and iii) how the estimate fits with the broader evidence base.
Overall, we apply a 60% IV adjustment to the Egger et al. estimates, which shrinks them 40% of the way towards 0.108 We put a lot of weight on Egger et al. because of the methodological advantages discussed above and some further advantages discussed below. We don’t put all of our weight on Egger et al. because the estimates have large standard errors, and the authors can’t observe imported goods in their study, which we’d expect to slightly bias the estimates upwards. We also don’t think their finding completely supersedes other (more muted) spillover effect estimates, and think their results are surprisingly large compared to our prior.
Reasons we put weight on Egger et al.
We think Egger et al. (2022) is a high-quality experimental estimate – in addition to the advantages discussed above, the study also has many hallmarks of a well-conducted experiment, such as baseline balance,109 low attrition,110 and pre-registered outcomes.111 One thing we find especially compelling is that they have two separate estimates of the economic multiplier – as well as taking a consumption-based approach to measuring economic activity,112 they also take an income-based approach.113 These represent two very standard but independent approaches to measuring economic activity, and the fact that these yield very similar estimates makes us less concerned about possible measurement error.114
Given the importance of this study as our top-line estimate, we funded an external reanalysis of the paper, conducted by Michael Wiebe. The final report is uploaded here. The report finds no major problems with their analysis and the author generally trusts their main results,115 which gives us further reassurance.
We also think the theoretical explanation the authors put forward to rationalize their findings seems plausible. The authors hypothesize that the large spillovers they observe are due to a demand-led economic expansion,116 facilitated by excess ‘slack’ in the local economy.117 This theory suggests that the cash injection stimulates demand, which businesses can meet without raising prices due to underutilized capacity. This story is spelled out in more detail in this section of the Appendix.
This story seems plausible to us, for two reasons:
- Two of the key building blocks of their story – expenditure multipliers and aggregate demand expansions leading to ‘real’ increases in economic activity – are well-established ideas in economics. While there is a lot of empirical debate about the size of economic multipliers, their theoretical possibility under certain conditions seems quite uncontroversial118
- In a follow-up working paper, Egger et al. run a survey to try and measure the amount of ‘slack’ in the same experimental context. They find convincing evidence of this: from observing shopkeepers in rural Kenya, only 50% of the time was spent on productive activity, and when they asked small businesses how much their costs would increase to expand output by 10%, 40% said zero.119 This lends empirical support for the theoretical mechanism they hypothesize
Reasons we don’t put all our weight on Egger et al.
We don’t think we should put all our weight on Egger et al. for several reasons:
- The multipliers have large standard errors: The expenditure multiplier (2.6) has a standard error of 1.4 and the income multiplier (2.5) has a standard error of 1.7.120 When the authors plot the 95% confidence interval on both multipliers,121 both intervals comfortably contain 1 – which is what we might expect if there were no consumption spillovers to non-recipients. We think we should put less weight on noisier experimental estimates compared to our prior
- The paper doesn’t observe net imports: As the authors note, one limitation of their study is that they can’t observe whether respondents are consuming goods produced inside or outside the study area.122 If imports increase following a cash transfer – as we and Egger et al. would intuitively expect123 – this would bias their multiplier estimates upwards.124 We think would probably make a small difference125 as we expect a lot of consumption is on locally produced goods,126 but it makes us less inclined to fully update towards their result
- The paper can’t observe ambient effects across the study area: by comparing outcomes in households nearby treated villages vs. households in distant control villages, Egger et al. (2022) assumes there were no effects across the study area that affected households in both nearby and distant control villages.127 If ambient effects positively (negatively) affected distant households, this would bias the spillover treatment effects downwards (upwards).128 In a follow-up paper, the authors argue that ambient effects were more likely to be negative, as inflation in distant control villages appears more quantitatively important than income gains.129 Since these net effects are negative, we’d expect the spillover effects estimated in Egger et al. (2022) to be biased upwards, and put less weight on them as a result
- We don’t think it completely supersedes previous studies: While we think Egger et al. has important methodological advantages over previous studies, we don’t think it entirely supersedes them, either.130 Since these generally find much smaller spillovers, this makes us put less weight on the results of Egger et al.
- The multiplier seems surprisingly large: two results we find particularly surprising are: i) consumption gains to non-recipients being twice as large as consumption gains to recipients and ii) their finding of no meaningful price inflation, including for perishable, hard-to-transport products like eggs.131 We find this surprising for both theoretical132 and empirical133 reasons
Taking these considerations into account, our best guess is to apply a 60% internal validity adjustment (i.e. a -40% downweight) to the Egger et al. spillover results.
Key uncertainties
The ultimate adjustment we settle on is highly subjective, and we think reasonable people could place more or less weight on the Egger et al. estimate. We had discussions with several academic experts about this paper – while everyone agreed that this was a high-quality experimental estimate, our impression is that opinions differ on how much this paper updates the message of the wider literature. This is discussed further in this section.
External validity adjustment
Summary
Our external validity adjustments account for how the experimental results we use are likely to generalize to real-world programmatic contexts. In this case, two differences between the context of Egger et al. (2022) and the context of GiveDirectly’s current Cash for Poverty Relief program seem especially salient. First, unlike the experiment studied in Egger et al. – where, on average, roughly 1/3 of households were eligible for a transfer within a targeted village134 – GiveDirectly now does universal within-village targeting, where every household in a targeted village is eligible. Second, while the experiment Egger et al. is conducted in Siaya county in Kenya, GiveDirectly now operates the Cash for Poverty Relief program in other parts of Kenya,135 as well as 4 other countries. As argued in this section of the Appendix, we think Siaya county is likely richer, less agricultural, and less physically remote than more typical contexts of GiveDirectly’s current Cash for Poverty Relief program.
Our best guess is that spillovers to non-recipients are likely to be smaller (but still positive) under more saturated program designs and in poorer and more remote contexts. This is based on some empirical evidence, which finds a greater inflationary response to cash transfers in these settings, and some theoretical arguments, such as labor reallocation frictions making the slack story seem less likely in poorer, more agricultural economies. We apply an 80% external validity adjustment in Kenya (i.e. a -20% downweight) to account for this, and slightly steeper 70-75% adjustments in Malawi/Mozambique/Rwanda/Uganda.
These adjustments bring the Egger et al. results down to the average spillover effects we’d anticipate across programming sites in each of these countries. We’d expect these average effects to mask substantial heterogeneity, with some regions seeing a more pronounced uptick in economic activity and less inflation than others following the disbursement of cash. This seems consistent with two papers discussed below, which find heterogenous general equilibrium effects across different regions in Mexico and Philippines from government-led cash transfer programs.
Reasons to expect similar/larger spillovers in more remote and saturated contexts
One argument for this comes from a recently published working paper by the same authors of Egger et al., (Walker et al., 2024) who try to answer the question of how their experimentally estimated multipliers might generalize to larger cash injections and in poorer/more remote contexts. To do this, they build a spatial general equilibrium model that embeds ‘integer constraints’136 – the idea that you can’t divide a worker or milling machine in half – which is what they hypothesize as a driver of slack in their original paper.137
Two predictions of this theoretical model are especially worth mentioning:
- The model predicts that multipliers will be larger in more remote contexts.138 Intuitively, this is because there is more underutilized capacity (or ‘slack’) in these regions, as high transport costs mean that each mill owner or mechanic (for instance) can only service a very small area of effective demand139
- The model predicts that, under certain conditions (e.g. rural contexts and workers freely moving between the agricultural and non-agricultural sector), real economic multipliers remain fairly constant as the size of the cash injection scales. The authors argue this is because there is so much underutilized capacity in these contexts – e.g. unproductive workers in agriculture – that you can scale up the size of the demand shock by a lot before inflation starts to erode real consumption gains140
One limitation of this model is that it only zooms in on a single mediator of the relationship between the concentration of the program and the real multiplier – integer constraints.141 While the authors argue that this explains more than half of their experimental result,142 it also leaves a sizable fraction unexplained. We don’t have a good intuition about how other frictions proposed by the literature on macroeconomic multipliers (e.g. nominal rigidities or ‘sticky prices’) are likely to scale with the size of transfer.
Reasons to expect smaller spillovers in more remote and saturated contexts
Two reasons we expect smaller spillovers in more remote and saturated contexts are:
- Empirically, two recent papers have found concentrated inflationary effects in remote, saturated regions from government-led cash transfer programs in LMICs. We’d expect inflation to erode real consumption gains
- Theoretically, under certain conditions, Walker et al. (2024)’s model also predicts smaller real multipliers under more saturated program designs. We think these conditions are likely to hold (to some extent) in GiveDirectly’s current programmatic contexts
Empirical evidence on the relationship between inflation and saturation/remoteness
For inflation to erode the value of the real multiplier, we’d have to believe that inflationary pressure increases non-linearly as the size of the cash transfer scales up.143 We don’t have direct empirical evidence on this in the context of the Cash for Poverty Relief program. However, from a brief literature review, two recent experimental estimates from government-led cash transfer programs in Mexico and the Philippines seem relevant:
- Cunha et al. (2019) studies a means-tested government cash transfer program in Mexico, which entailed transfers of $20 per month144 for eligible households.145 They find no detectable price increases on average across the study region.146 However, when they zoom in on the most deprived villages – proxied by physical remoteness and low average income147 – they find that food prices increase.148 We’d expect the relative cash injection to be larger in more deprived villages, given the lower levels of baseline income and higher proportion of households likely to be eligible149
- Filmer et al. (2023) studies a means-tested government cash transfer program in the Philippines, which entailed transfers of $11-32 per household per month150 for eligible households.151 They find minimal price inflation on average, but a large 6-8% increase in the price of perishable products (e.g. fish, eggs) in the most highly saturated and remote villages.152 In these markets, the authors find that this increased child stunting by 11 percentage points153
We don’t think these studies provide conclusive evidence that inflation is likely to erode real multipliers under more saturated versions of GiveDirectly’s Cash for Poverty Relief program. First, the programs studied in these papers are quite different, entailing predictable monthly transfers rather than one-off lump sums.154 Second, what matters for the real multiplier is not inflation per se, but whether inflation scales non-linearly as the cash injection is scaled up,155 and we find this difficult to infer from these studies.
Nonetheless, the relatively large inflation response in certain markets found in Filmer et al. (2023) – and the fact that the impacts of program saturation is exacerbated by physical remoteness156 – makes us cautious, and we think a conservative adjustment seems appropriate until more direct evidence from the Cash for Poverty Relief program emerges.157 We also think these results suggest that, within countries, there’s likely to be a lot of heterogeneity underlying the average effects that we estimate.
Theoretical reasons to expect smaller multipliers in more remote/saturated contexts
The result in Figure 6 of Walker et al. (2024) (pasted above) relies on the frictionless reallocation of workers between the agricultural and non-agricultural (‘slack’) sector.158 When the authors relax this assumption and assume no reallocation between sectors, it’s no longer the case that the real multiplier remains constant under larger cash injections. For instance, according to their model, a cash transfer of 60% of GDP without labor reallocation leads to inflation of 24% and a real multiplier of only 1.1.159
We expect the real world to be somewhere in between these modeled scenarios: workers are unlikely to be fixed in their sectors, but frictionless reallocation also seems unlikely. For example, in the contexts that GiveDirectly operates in, we’d expect transport costs,160 search costs,161 contracting frictions,162 and heterogeneous preferences163 to all hinder the seamless transition of workers from agriculture to the manufacturing and retail sectors following a demand shock. These barriers are unlikely to be fixed in the long-run, but we think the theoretical explanation of Walker et al. (2024) requires a fairly timely transition, as people need to move to the ‘slack’ sectors – assumed to be manufacturing and retail164 – before the demand shock fizzles out.165
Key uncertainties
On balance, we find the arguments in favor of expecting smaller spillovers under more saturated programs in more remote contexts more convincing, so we apply conservative downweights. However, these are speculative, and we think more direct evidence on the inflationary impacts of the Cash for Poverty Relief program in these contexts would allow us to make better-informed adjustments. As discussed here, an ongoing trial in Malawi may inform an update to this parameter in the future.
Distributional adjustment
Similar to our estimates for recipient consumption gains, we need to decide whether the gains from a demand-led increase in economic activity are more likely to accrue to richer households. Since our non-recipient consumption benefits are pegged to our recipient consumption benefits, the question is whether we think the distribution of spillovers is likely to be more unequal compared to the distribution of recipient consumption gains.
Overall, we think spillovers to non-recipients are more likely to accrue to richer households compared to consumption gains to recipients, and so we apply a 80% distributional adjustment (a -20% downweight) to account for this.
We think spillovers are more likely to accrue to richer households because:
- Egger et al. (2022) find that the economic expansion they observe is concentrated in the retail and manufacturing sectors,166 and we’d expect relatively richer households to be more likely to own businesses in these sectors
- Egger et al. (2022) find no changes in village-level Gini coefficients167 for consumption and wealth in villages that received transfers, and a slight increase in the wealth Gini coefficient in nearby untreated villages.168 In treated villages, since transfers were deliberately targeted at poorer households – those with a thatched roof169 – an unchanged Gini coefficient suggests the additional gains from economic activity disproportionately accrue to richer households170
- When they disaggregate non-recipient spillovers by eligible and ineligible households, Egger et al. (2022) find that non-recipient spillovers accrue almost entirely to ineligible households. These households see a $411 increase in annualized consumption, while eligible households see a (not statistically significant) $21 increase.171 This could be consistent with the small negative spillovers found in Haushofer and Shapiro (2018), as the authors only collected data on eligible non-recipients.172 Since, in both studies, eligibility was defined by thatched roof ownership, we’d expect ineligible households to be materially better-off than eligible ones
- In follow-up work we commissioned, Michael Wiebe found that non-recipients households in the highest quintile of baseline assets saw the largest consumption gains.173 However, the results are noisy and don’t all point in the same direction, so we put less stock in this additional analysis compared to the findings of the paper
A separate but related question is whether any households are made worse-off by nearby Cash for Poverty Relief programming. We think this is possible, given the possibility of inflation and the findings of Filmer et al. (2023)174 and Haushofer and Shapiro (2018)175 – though we think there are reasons the latter results could be biased downwards and overstate negative spillovers into non-recipients (as discussed here in more detail).
Our guess is that the number of people made worse-off from the Cash for Poverty Relief program is, on average, small relative to the number of people that directly or indirectly benefit. We think this for two reasons:
- Walker et al. (2024) use their theoretical model to estimate how the real income effects of the experiment studied in Egger et al. (2022) were distributed across villages. Out of the 653 villages they model, they estimate only 6 non-recipient villages saw declines in real income due to inflation.176 Most non-recipient villages benefitted on average due to positive consumption spillovers
- Unlike in Egger et al. (2022), real world Cash for Poverty Relief programming employs universal within-village targeting and doesn’t deliberately leave out nearby villages to serve as controls. Since more people receive cash, we think it’s less likely that some people just experience inflation without having the increased purchasing power, so the possibility of net losers seems less likely177
What do academic experts think?
We spoke to the authors of the Egger et al. paper (Dennis Egger, Ted Miguel, Johannes Haushofer, and Michael Walker), as well as other people who have worked on academic studies of cash transfer programs in LMICs (Berk Ozler, Craig McIntosh, Rossa O’Keeffe O’Donovan, Jesse Cunha, and Eeshani Kandpal). Much of this section was heavily informed by these discussions: for example, Berk Ozler pointed out to us the ways the Egger et al. results could and could not be reconciled with the Haushofer and Shapiro results, while Eeshani Kandpal flagged to us the importance of physical remoteness, perishability, and program saturation in mediating the inflation response of cash transfer programs.
Our estimates rely on a number of highly subjective judgments about how much weight to put on different papers and how experimental effect sizes are likely to generalize to different contexts. Due to this subjectivity, we think people can reasonably disagree with our estimates. For example, amongst the experts we spoke to:
- Dennis Egger thinks our adjustments seem reasonable but skew conservative,178 and points to other countervailing factors that might lead to larger multipliers outside of the Egger et al. experimental context. For example, credit constraints might be more biting outside of Kenya (which could induce a sharper investment response)179 and sectoral reallocation might be smoother outside of Kenya (contrary to what we expect)180
- Berk Ozler thinks our internal validity and distributional adjustments may be slightly too generous, given the findings of previous GiveDirectly spillover papers and Egger et al.'s finding that spillovers accrue disproportionately to ineligible non-recipients.181 His perspective is outlined in more detail in this blog post. Both he and Craig McIntosh think spillovers to non-recipients are likely to be of second-order importance compared to the direct effects to recipients182
- Eeshani Kandpal and Rossa O’Keeffe O’Donovan think our adjustments seem reasonable – not too bullish or too conservative. Eeshani thinks that underlying our average estimates is substantial heterogeneity reflecting the importance of program context183
Key uncertainties
Overall, consumption gains to non-recipients are our biggest uncertainty about our modeling of GiveDirectly’s Cash for Poverty Relief program. Two things we feel especially uncertain about are:
- How much we ought to update based on the findings of Egger et al. (2022) (more)
- How we should expect these findings from Western Kenya to generalize to more saturated program models in different contexts (more)
In this section of the Appendix, we run some additional sense-checks of our estimates, such as comparing our implied economic multipliers to those that have been estimated from other government-led cash transfer programs.184 Overall, the fact that (mostly non-experimental) papers have estimated multipliers of >1 (which implies positive spillovers) seems reassuring for the direction of our results, but we still feel very uncertain about the magnitude.
We may update our current estimates if more evidence emerges on the magnitude of spillovers and inflation from large-scale lump sum cash transfer programs. One ongoing trial that seems particularly relevant is in Malawi, where transfers amounting to 90% of local GDP are being distributed in the north of the country. While the study is not measuring consumption spillovers to non-recipients,185 it is measuring the price response, so we think this will likely be informative about the likelihood of inflation under more intense programming in a poorer context. We expect to see results from this trial in 2025, and may update our model in light of this.
4.4 Mortality benefits to recipients
Summary
Our best-guess is that receipt of a Cash for Poverty Relief cash leads to a 23% reduction in the risk of all-cause under 5 mortality. This is based on published results of the Cash for Poverty Relief from McIntosh and Zeitlin (2024) and preliminary results sent to us by Egger et al., which imply large reductions in all-cause under 5 mortality (70% and 46%, respectively).
We apply steep discounts to these results because they seem much higher than the mortality effects estimated from a meta-analysis of government cash transfer programs, and because we don’t have a clear theoretical understanding of how effects could be this large, given other GiveDirectly RCTs have found muted effects on health investments and health-seeking behavior.
We’ve generally spent less time scrutinizing these estimates because they don’t make a big quantitative difference to our bottom-line. Even if we took the 46% all-cause mortality estimate at face-value, our cost-effectiveness estimate in Kenya would only move from 2.5x to 2.8x. Intuitively, a large mortality effect size doesn’t translate to a large cost-effectiveness estimate because GiveDirectly’s Poverty Relief is relatively expensive – costing ~$1,200 (nominal) per household186 – and untargeted – as not every household that receives a transfer has young children.
Our calculations of the mortality benefits of GiveDirectly’s Poverty Relief program are outlined in this spreadsheet, and summarized below:
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Number of people reached | 4,079 | 3,780 | 3,300 | 3,323 | 3,569 |
Share of u5s in targeted population | 13% | 15% | 16% | 14% | 17% |
Number of u5 children reached | 525 | 572 | 532 | 456 | 594 |
Estimated u5 all-cause mortality rate in GiveDirectly target population | 5.1% | 4.9% | 8.2% | 4.7% | 5.0% |
Egger et al. estimated reduction in all-cause u5 mortality from the GiveDirectly program | 46% | 46% | 46% | 46% | 46% |
IV adjustment | 50% | 50% | 50% | 50% | 50% |
GiveWell estimated reduction in all-cause u5 mortality | 23% | 23% | 23% | 23% | 23% |
u5 deaths averted | 6.1 | 6.5 | 10.0 | 4.9 | 6.8 |
Value assigned to averting an under-5 death | 116 | 116 | 116 | 116 | 116 |
Units of value from child mortality benefits | 710 | 754 | 1,159 | 570 | 792 |
Adult mortality benefits (as % of child mortality benefits) | 50% | 50% | 50% | 50% | 50% |
Units of value from child and adult mortality benefits | 1,065 | 1,131 | 1,739 | 856 | 1,188 |
We may update our views on the mortality effect size as the preliminary results of Egger et al. go through academic peer review. However, we think updates here are much less likely to meaningfully affect our overall view of the program compared to updates on consumption benefits to recipients and non-recipients.
What is the evidence on the mortality effects of cash transfers?
Of the 7 trials we found of the Cash for Poverty Relief programming, 2 looked at child mortality outcomes. None looked at adult morality, though Egger et al. plan to look at this as part of their long-run follow-up.187
- McIntosh and Zeitlin (2024) find that large lump sum cash transfers lead to a 70% reduction in all-cause under 5 mortality in Rwanda. This is based on a small sample (15 observations), though the difference between treatment and control groups is statistically significant188
- Egger et al. have sent us preliminary results from a 8-9 year follow-up189 of the original Kenya experiment studied in Egger et al. (2022). These indicate a statistically significant 46% reduction in all-cause under 5 mortality within recipient households during the ~2 years that cash transfers were being disbursed relative to the control mean.190 The authors don’t find evidence of a child mortality reduction in non-recipient households, nor do they find evidence of a mortality reduction among children born in the years after the cash transfer treatment ended.191
The effect of cash transfers on mortality in LMICs is examined more generally by a recent meta-analysis by Richterman et al. (2023). This paper combines administrative mortality data across 37 LMICs against the rollout of government-led cash transfer programs,192 mostly introduced in a non-randomized fashion.193 It finds that cash transfers were associated with a statistically significant 8% reduction in under 5 mortality and an insignificant 4% and 7% reduction in aged 5-9 and 10-17 mortality, respectively.194 When they zoom in on large cash transfer programs with high coverage (similar to GiveDirectly’s Poverty Relief program), the effect for children under 5 and children aged 10-17 increases to (a statistically significant) 14% and 20% respectively, but the effect for children aged 5-9 remains statistically insignificant. Among adults, large programs with high coverage were associated with a 30% and a 23% reduction in all-cause mortality.195
Our approach
To estimate child mortality, we use the preliminary results sent to us by Egger et al. because:
- These results are based on a much larger sample than McIntosh and Zeitlin (2024). The paper is able to track ~90% of the households identified in the initial census, corresponding to nearly 65,000 households.196 Overall, they observe ~200 child deaths197 during the period cash transfers were disbursed (2015-17)
- Richterman et al. (2023) is a meta-analysis of government-led cash transfer programs, which seem quite different to the Cash for Poverty Relief program (e.g. most entail continuous transfers).198 It also doesn’t leverage experimental variation, and we generally put less weight on non-experimental estimates because it’s easier to think of confounding factors that could bias the results199
Internal validity adjustment
We impose a 50% internal validity adjustment to the Egger et al. result. After this adjustment, our best-guess is that the Cash for Poverty Relief program leads to a 23% reduction in all-cause under 5 mortality.
We impose a relatively steep IV adjustment because:
- This result seems surprising in light of the patterns of consumption and under 5 mortality across countries and within Kenya. As this section in the Appendix notes, recipient households in this trial were still likely poorer than the average Kenyan, yet this mortality treatment effect implies an under 5 mortality rate 33% below the national average
- The preliminary results we’ve seen don't include mechanisms analysis, so we can’t examine the channels through which this mortality reduction might have come about. When we try to hypothesize some, we struggle to come up with ideas that could rationalize the size of this effect, and which are consistent with the patterns of previous findings (see footnote for details).200
To convert Egger et al.’s all-cause under 5 mortality treatment effect into units of value, we estimate:
- The number of u5 children reached (given our assumptions about program reach) and demographic characteristics of GiveDirectly households. We assume that households that receive GiveDirectly Poverty Relief transfers have the same age distribution as the general population201
- Estimate the baseline all-cause mortality risk for under 5s in GiveDirectly households across each country. To do this, we take country average estimates produced by UNICEF and then inflate these upwards to take account of children in GiveDirectly’s households likely being at higher risk (since these households are poorer). We use the difference between the all-cause under 5 mortality rate reported in the control group in Egger et al. (5.7%)202 against the country-wide estimate produced by UNICEF across the same time period (4.1%)203 to make this adjustment
These assumptions combine with our assumptions about program reach, under 5 mortality treatment effects, and moral weights as follows:
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Number of people reached | 4,079 | 3,780 | 3,300 | 3,323 | 3,569 |
Share of u5s in targeted population | 13% | 15% | 16% | 14% | 17% |
Number of u5 children reached | 525 | 572 | 532 | 456 | 594 |
Average u5 mortality rate in each country (%) | 4.1% | 4.0% | 6.6% | 3.8% | 4.1% |
Adjustment for GD u5 population being at higher risk | 123% | 123% | 123% | 123% | 123% |
u5 mortality rate in GD target population | 5.1% | 4.9% | 8.2% | 4.7% | 5.0% |
Egger et al. estimated reduction in all-cause u5 mortality from the GiveDirectly program | 46% | 46% | 46% | 46% | 46% |
IV adjustment | 50% | 50% | 50% | 50% | 50% |
GiveWell estimated reduction in all-cause u5 mortality | 23% | 23% | 23% | 23% | 23% |
u5 deaths averted | 6.1 | 6.5 | 10.0 | 4.9 | 6.8 |
Value assigned to averting an under-5 death | 116 | 116 | 116 | 116 | 116 |
Units of value from child mortality benefits | 710 | 754 | 1159 | 570 | 792 |
Adult mortality benefits (as % of child mortality benefits) | 50% | 50% | 50% | 50% | 50% |
Units of value from child and adult mortality benefits | 1,065 | 1,131 | 1,739 | 856 | 1,188 |
Since child mortality benefits don’t make a large difference to our bottom-line, we include a one-line adjustment for adult mortality benefits rather than modeling these out explicitly. Across countries, we guess that adult mortality benefits are 50% of the child mortality benefits of the GiveDirectly Poverty Relief program. Intuitively, we’d guess that adult mortality benefits are a less important benefit stream because: i) in a given year, most adults are less likely to die than under 5 children (i.e. their baseline mortality risk is lower)204 and ii) our moral weights consider saving the lives of adults as less valuable as saving the lives of children.205
Key uncertainties
We feel very uncertain about whether the adjustments we’re applying to the preliminary Egger et al. results are appropriate, and expect to get a better sense of this as the authors do more mechanisms analysis to rationalize their findings.
However, our uncertainty around mortality benefits is far less quantitatively important than our uncertainty around consumption benefits to recipients and non-recipients. Even if we take the Egger et al. all-cause mortality result at face-value, it doesn’t make a meaningful difference to our overall assessment of the cost-effectiveness of GiveDirectly’s Cash for Poverty Relief program, as our cost-effectiveness estimate in Kenya moves from 2.6x → 2.9x. Intuitively, this is because this program is a relatively expensive and untargeted way to save under 5 lives, and so on this metric alone, doesn’t look competitive with our Top Charities.206
4.5 Additional benefits and downsides
Summary
Our cost-effectiveness analysis includes a number of additional benefits and downward adjustments that we have opted not to explicitly model. Instead, we incorporate them as rough percentage best guesses. In general, we’ve spent less time reviewing the evidence behind these additional factors because we think they’re likely to be less quantitatively important than the benefit streams we’ve modeled.
To keep things consistent with how we model other programs, we sum these adjustments together to increase our estimate of the cost-effectiveness of the GiveDirectly Poverty Relief program by 8% (a 108% adjustment). These adjustments can be found in this section of our cost-effectiveness model and are summarized below. We apply the same adjustments for each country.
What we’re estimating | Estimated effect on cost-effectiveness |
---|---|
Reduced morbidity | 10% |
Developmental effects | 5% |
Leverage & funging | 2% |
Risk of wastage/fraud | -2% |
Within-org fungibility | -5% |
Psychological spillovers | -2% |
Total adjustment factor | 108% |
Additional benefits
Morbidity effects
Morbidity effects refer to health improvements brought about by the GiveDirectly program other than the reduction in mortality. For example, if cash transfers enabled people to eat healthier diets so they got sick less often, this benefit would show up here. This benefit stream refers to the adult population; GiveDirectly’s impact on child health is captured in our development effects discussion below.
We expect morbidity benefits to comprise a small share of overall benefits because there isn’t good evidence that the Poverty Relief program improves the physical health of adult recipients.207
While there is more robust evidence that cash transfers improve recipients’ mental health (see footnote for details),208
we think much of this benefit is already conceptually captured in the value we assign consumption gains.
- Our moral weights are how we convert consumption and mortality benefits into a consistent unit of value.
- These values are derived from surveys we ran on our staff, donors, and beneficiaries (households in Kenya and Ghana), which use discrete choice experiments to elicit respondent preferences over monetary gains (e.g. $1,000 in cash) vs. a reduced risk of dying.209
- In asking people how much they value cash, we think it’s likely people are considering all the things cash could be useful for; for example, someone might imagine the psychological relief receiving $1,000 would bring. This interpretation is speculative, because we don’t have insight into exactly what respondents were and weren’t accounting for when stating their preferences for cash.
- If this benefit is already ‘baked-in’ to their stated preference, then modeling psychological benefits as separate to consumption benefits would mean double-counting.
In theory, this concern could also apply to e.g. child mortality benefits, which we currently model separately. If, in our surveys, our respondents are imagining the additional health products they could purchase with cash – and the likely reductions in child mortality that would result – this benefit stream could also be baked into their stated preference. This seems less likely to us, as compared to the psychological benefits of cash, child mortality benefits seem less intuitively obvious.
Development effects
In our cost-effectiveness models, development effects refer to long-run consumption gains that might be realized through improving the health or education of children. For example, if cash transfers lead to an improvement in children’s growth and cognitive skills – and if these drive consumption gains in the future – this benefit should be captured here. We apply a small 5% upwards adjustment to account for this. We think these benefits should be relatively small because while there is evidence that cash transfers increase school attendance, effects of children’s cognitive skills appear weaker and evidence for effects on children’s height and weight (which are thought to be correlated with cognitive development) is mixed (see footnote for details on these evidence bases).210
As a sense check, we estimate how impactful it would be if, hypothetically, a one-time Poverty Relief transfer increased consumption for children in recipient households by 2% each year in adulthood. We’d consider this a large effect – for context, our best guess is that preventing a case of clinical malaria in childhood increases adult income/consumption by 0.6% a year,211 and that ridding a child of parasitic worms for a year increases adult income/consumption by 1.1%.212 If we hypothetically assume that receiving a GiveDirectly cash transfer in childhood leads to a 2% counterfactual increase in adult income, this only leads to a small change (0.2-0.3x) in cost-effectiveness. Intuitively, this is for a similar reason that a large all-cause under 5 mortality effect doesn’t lead to a large cost-effectiveness estimate: GiveDirectly is expensive per household, and doesn’t specifically target households with children.
Leverage and funging
In our cost-effectiveness models, ‘leverage’ refers to money a charity is able to crowd in from other sources (hence leveraging the potential impact of a given donation), while ‘funging’ refers to the possibility that a given donation crowds out funding that would otherwise have been donated. We think it’s important to account for the impact of donations on the spending of other actors, since we try to maximize the counterfactual impact of marginal dollars. For example, we’d consider funding a bed net campaign less cost-effective (all else equal) if, in our absence, another funder would have stepped in and funded the campaign.
In the context of GiveDirectly’s Poverty Relief program, we don’t think that funging seems like a big concern. Hypothetically, if a donor were to fund a Poverty Relief campaign in village X, it may be true that another donor would have otherwise funded it. However, by funding the campaign in village X, this donor would ‘free up’ the contributions of other donors to expand the program in village Y. So long as there are no shortages of poor villages that GiveDirectly could target, crowding out other people’s donations doesn’t seem like a big concern. As this section notes, there are likely many poor villages that GiveDirectly has not yet targeted in the countries that it operates, as GiveDirectly’s total reach is <10% of the total number of households estimated to live in extreme poverty.
Theoretically, private donations to GiveDirectly could be leveraged if they crowd in institutional funding. GiveDirectly estimates that they’ve crowded in around $23m of institutional funding (mostly from USAID) towards Cash for Poverty Relief transfers between 2018-2023, a period where they’ve directly allocated $416m themselves.213 We shouldn’t think of this $23m as fully additive, since it’s trading off against other programs USAID could theoretically fund. Since this is also a very small share (~5%) of their directly allocated funds, we don’t think we should make a large adjustment for this, so we apply a 2% adjustment.
Offsetting impacts
Within-org fungibility
Our within-org fungibility adjustment accounts for the possibility that donations earmarked for GiveDirectly’s Poverty Relief program crowds out flexible internal funding towards other (potentially less cost-effective) programs. We think this is unlikely to be a big concern because:
- According to GiveDirectly, only ~1% of their flexible expenditure has been allocated in their US program in 2024.214 We’d intuitively expect their US program to be the least cost-effective given American households are much richer than recipients in low-income countries
- GiveDirectly’s flexible allocation to Cash for Poverty Relief programming has been fairly consistent even as the organization has scaled. The question we’re ultimately interested in is: what would happen to flexible funding (the orange segments of the graph below) if we increase the donations earmarked for the Cash for Poverty Relief program (the dark blue segment)? This basically happened 2022-2023 – dollars earmarked to Poverty Relief rose from $10.9m to $47.7m, yet flexible funding allocated to Poverty Relief remained fairly stable in both absolute (going from $79.6m → $73.0m) and relative (going from 81% → 82% of flexible funding) terms. We think this is a vote against large injections of earmarked donations earmarked for the Cash for Poverty Relief program prompting a large internal reallocation of flexible funding
Overall, our best guess is to apply a small downwards adjustment for the possibility of within-org funging (-5%), which is the same adjustment we apply for Malaria Consortium’s seasonal malaria chemoprevention program215 and Helen Keller International’s vitamin A supplementation program.216 Like GiveDirectly, both of these organizations support additional programs beyond the ones we model.
Risk of wastage/fraud
Our risk of wastage/fraud adjustment accounts for the possibility that money donated to GiveDirectly to Poverty Relief gets misappropriated before reaching the recipients. There is some precedent of this happening: for example, in 2022, GiveDirectly estimates that $1.2m was illegally diverted from its operations in the Democratic Republic of Congo.217
While instances of fraud and theft do happen, our impression is that these are rare occurrences in the scale of GiveDirectly’s overall programming, and we feel reassured by GiveDirectly’s practice of publishing these incidents when these occur. For example, GiveDirectly estimates that, of the $144m they directed globally in 2022, approximately 1% of this funding was lost due to fraud (including the DRC case).218 Our best-guess is to apply a small (-2%) negative adjustment for this.
Negative psychological spillovers
We apply a small downwards adjustment (-2%) to account for the possibility of negative psychological spillovers to non-recipient households. Conceptually, this adjustment is supposed to capture possible feelings of resentment among households that aren’t selected to receive cash transfers. There is some evidence of this: for example, Haushofer et al. (2015) find that non-recipient households in villages that received transfers had worse mental health than non-recipient households in villages that didn’t receive cash.219
We think this is unlikely to warrant a big adjustment, for two reasons. First, since the experiment studied in Haushofer et al., GiveDirectly has moved away from within-village targeting and now gives transfers to everyone in targeted villages (provided they consent). We think this makes resentment among non-recipients seem less likely, because it’s no longer the case that certain households in tight-knit village communities are excluded. Second, as this section argues, we expect there to be positive consumption spillovers to untargeted villages that are nearby targeted ones, which we’d expect to improve the mental health of recipients. We think this is (largely) captured by the moral weight we assign to consumption gains to nearby non-recipients.
While this makes us think negative psychological spillovers are unlikely to make a big difference, we still think a small adjustment is warranted. The positive consumption spillovers that Egger et al. document are highly localized – i.e. they mostly accrue within a 2km radius of treated villages. Hence, we think it’s unlikely that villages ~10 km away from treated villages see benefits from this, but it’s likely that they are still aware that the program is taking place.220 We think the idea of these households harboring feelings of envy/resentment towards recipient households feels plausible, and so we make a small downwards adjustment.
Factors we have excluded
Effects on labor supply
In theory, if we thought households increased the hours that they work following cash transfers, this might warrant a small negative adjustment, insofar as it would mean that they have less time to spend on leisure activities. We don’t make an adjustment for this because we don’t think there is clear evidence of a directional change in hours worked.221
Exchange rate impacts
One thing we briefly considered was whether GiveDirectly’s Poverty Relief program could cause a harmful appreciation of a country’s exchange rate. To get money to recipients, GiveDirectly needs to convert donations (e.g., in US dollars) to the local currency of whichever country it is targeting (e.g. Kenyan shillings). To do this, it needs to purchase Kenyan shillings, which could cause the price of Kenyan shillings to go up. This could have harmful macroeconomic consequences: for example, an appreciated currency would make Kenyan exports relatively more expensive,222 which may stifle export-led economic growth. In economics, this phenomenon is known as Dutch disease, and it is sometimes hypothesized as a potential risk of sudden natural resource windfalls or foreign aid more generally.223
At the scale it currently operates, we think it’s very unlikely that GiveDirectly is having meaningful impacts on countries' exchange rates. In 2023, GiveDirectly directed ~$50m towards Povery Relief in Malawi (the country that received the most funding).224 In 2023, Malawi’s total GDP was estimated to be $14.1bn.225 Hence, Poverty Relief cash transfers amounted to <0.5% of total GDP. At this scale, we think effects on the exchange rate seem very unlikely: the discussion of Dutch disease we’ve seen talks about it in the context of much larger macroeconomic shocks (e.g. 10-20% of GDP).226 If GiveDirectly were to scale its programming significantly – i.e. billions of dollars in a short amount of time – these macroeconomic impacts are something we would reconsider.
Combining our benefit streams into a final cost-effectiveness estimate
To get to a final country-specific cost effectiveness estimate for the Poverty Relief program, we sum up the units of value from the 4 categories of benefits we model to get total units of value that we think are generated from an arbitrary donation size. We divide this by the size of the donation to get estimates in units of value per $ terms. We then divide this by our previous cost-effectiveness estimate for GiveDirectly (also expressed in units of value per $), to get a final cost-effectiveness estimate as a multiple of our previous benchmark.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Donation size (arbitrary) | $1,000,000 | $1,000,000 | $1,000,000 | $1,000,000 | $1,000,000 |
Consumption benefits to recipients (units of value) | 4,196 | 6,746 | 6,095 | 5,736 | 4,517 |
Consumption benefits to non-recipients (units of value) | 2,833 | 3,985 | 3,600 | 3,630 | 2,859 |
Mortality benefits to recipients (units of value) | 1,065 | 1,131 | 1,739 | 856 | 1,188 |
Total adjustment factor for additional benefits and downsides | 105% | 105% | 105% | 105% | 105% |
Additional benefits and downsides (units of value) | 648 | 949 | 915 | 818 | 685 |
Total units of value | 8,742 | 12,810 | 12,349 | 11,040 | 9,249 |
Total units of value per $ | 0.008 | 0.013 | 0.013 | 0.010 | 0.009 |
Units of value per $ (previous GiveDirectly benchmark)227 | 0.003355 | 0.003355 | 0.003355 | 0.003355 | 0.003355 |
Cost-effectiveness estimate (x times our previous benchmark) | 2.6 | 3.8 | 3.7 | 3.3 | 2.8 |
4.6 How are donations allocated across countries?
GiveDirectly has told us that if donors email them requesting that their Cash for Poverty Relief donation be earmarked for a certain country, they will accommodate that request.228 However, at the time of writing, there is no option to earmark Cash for Poverty Relief donations to a country on GiveDirectly’s website, so we estimate how the marginal dollar is likely to be allocated across countries to come up with an aggregated cost-effectiveness estimate for donations to the Cash for Poverty Relief program via the website.
To come up with this, our best guess is to use GiveDirectly’s planned flexible allocations for the Cash for Poverty Relief program in 2024,229 which we think is the best signal of how GiveDirectly is currently thinking about across-country allocations. This data was sent to us by GiveDirectly. Marginal donations today are unlikely to be allocated to Mozambique at significant scale because current programming is largely institutionally funded and already restricted to Mozambique, though this may change.230 We expect country-by-country allocations to change over time, so expect these estimates to become outdated.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Disaggregated cost-effectiveness estimate (x prev benchmark) | 2.6 | 3.8 | 3.7 | 3.3 | 2.8 |
Estimate of where marginal $ goes | 16% | 33% | 2% | 30% | 18% |
Weighted average cost-effectiveness estimate (x prev benchmark) | 3.3 | ||||
5. Additional perspectives beyond our cost-effectiveness model
In theory, our cost-effectiveness analysis intends to capture the total impact of a program per dollar spent. But we recognize that our cost-effectiveness calculations are not able to capture every factor that could make a program more or less impactful. Focusing only on our cost-effectiveness model may mean we’re missing things that are difficult to quantify. This section outlines some of the ‘outside the model’ questions we considered when evaluating this program.
5.1 Could GiveDirectly delay the scale-up of national social security programs?
One question we considered was whether Cash for Poverty Relief programming might delay the scale-up or strengthening of state-led social security programs. All else equal, we might prefer cash transfers to be administered by the government rather than a foreign NGO because i) there could be benefits to the state targeting poor recipients (e.g. they could then layer on other welfare programs like employment support); and ii) receiving benefits from the state might promote hard-to-quantify benefits like better civic engagement from citizens.
We think the question of whether NGO programming crowds out state programming is an important one, but we don’t think it’s especially pertinent to GiveDirectly compared to some of the other programs we recommend. For example, one critique that we’ve heard about funding vertical health campaigns (e.g. bed net distributions, national deworming days) is that it creates parallel distribution systems, and removes ‘learning by doing opportunities’ for the government.
At the moment, we think these concerns seem important, but we can also think of countervailing arguments in the opposite direction. For example, GiveDirectly’s programming could delay the scaling of social security programs, but it could also accelate it, by setting up the necessary infrastructure and providing proof-of-concept. We’ve heard similar arguments about vertical health programming, where successful NGO-led programs can be transitioned to the government at a larger scale than if the government had executed it alone. We hope to look into this question more comprehensively in the future.
5.2 How do recipients view GiveDirectly transfers?
GiveWell is interested in finding (and potentially funding) more research on how the recipients of programs view the programs themselves. In the case of GiveDirectly, we found an anthropological study in Western Kenya which interviewed 30 women: 10 that had received a $1,000 transfer, 10 that had received a $500 transfer, and 10 that had not received a transfer at all.231 Overall, the transfers were viewed positively by the recipients, and were generally preferred over other types of common development programs (e.g. being given free farming inputs).232 However, a sizable minority of recipients preferred education or livelihood support, such as a job or assistance with farming or starting a business. One reason for this preference was the desire for a more sustainable income source than that which they thought the cash transfers could provide.233
5.3 Are there intuitive reasons to think delivering free health commodities could be better than delivering cash?
Our bottom-line cost-effectiveness estimate implies that we think the programs implemented by our Top Charities – which deliver free health programs and commodities to people in low-income countries – represent more cost-effective giving opportunities than GiveDirectly’s Cash for Poverty Relief program. One critique of this recommendation we sometimes hear is that if these commodities are really as cost-effective as we think, GiveDirectly recipients would just buy them with their cash transfer, and so we shouldn’t necessarily think free mass distribution is better than giving cash.
We think this is a good line of questioning – when evaluating programs, we often ask ourselves why a problem a program is purporting to solve couldn’t be solved by providing people with cash. In the case of getting health commodities to people, we can think of 3 reasons why we’d expect uptake to be much lower in the absence of mass distribution campaigns:
- Demand: there’s evidence that demand for health products falls discontinuously when the price increases nominally above zero. For example, this paper shows that providing free chlorine tablets at the point of water collection (in addition to a local promoter) can increase the uptake of chlorine from less than 10% to 60%,234 while this paper finds that the uptake of insecticide treated bednets falls by 60 percentage points when the price increased from $0 to $0.60235
- Supply: private sector supply of many of the commodities we support is patchy in LMICs,236 and in the absence of mass campaigns, we’re skeptical that the private sector would be able to quickly step in and achieve similar levels of coverage. This is partly due to market failures – for example, we’d imagine it’s difficult to transport commodities reliably and continuously to remote pharmacies (transaction costs), and it may be difficult to enforce supply contracts in the context of weak legal systems (contracting frictions)
- Externalities: even if both the demand and supply side were functioning according to textbook economic theory, we’d expect uptake of these commodities to be lower than what is optimal due to externalities: indirect costs or benefits to third parties that result from another party's actions. All of the commodities our Top Charity programs distribute target infectious diseases,237 which are strongly associated with externalities. For example, if a person sleeps under a bednet, not only does this make them less likely to catch malaria, it also makes their neighbors less likely to catch malaria too. In contexts where externalities are present, free or subsidized distribution might produce more socially beneficial outcomes
6. Sources
7. Appendix
7.1 Baseline consumption estimates
Methodology
To arrive at our best-guess estimates, we combine two sources of information: government consumption surveys and baseline consumption surveys from randomized controlled trials (RCTs) of GiveDirectly’s Cash for Poverty Relief program (where available). We then do some sense-checks to see how our estimates compare to other benchmarks of deprivation, and whether our relative ordering of countries makes sense given their correlation with other plausible indicators of material well-being.
Government consumption surveys238
In the countries we study, government consumption surveys typically take place every ~5 years and are administered by the Ministry of Statistics (or equivalent body). They entail visiting a randomly selected sample of households across the country and asking respondents about their consumption habits: how much food they’ve consumed in the last week, any major asset purchases they’ve made, and how much they spend on other goods and services like fuel and health center visits. Estimates are expressed in the local currency (e.g. Kenyan shillings).
Each survey publishes mean estimates (i.e. average consumption in the country), then disaggregates this by segments of the population. To proxy Cash for Poverty Relief recipients, we use the average consumption of households in the bottom quintile of the overall consumption distribution. We use the bottom quintile mean rather than the mean because GiveDirectly deliberately targets the poorest regions of the country. Broadly speaking, they try to work up the consumption distribution, while also taking into account factors like operational feasibility and government priorities (more).
Since the latest government consumption survey in each country was conducted at different times – and reports consumption across different units and in different currencies – we try to make these estimates comparable as follows:
- Convert adult equivalent estimates into per capita estimates239
- Account for consumption growth between the time of the last survey and today. We assume that among GiveDirectly recipients, consumption grows by 1-2% per year240
- Convert the estimates into USD 2017 PPP by using official exchange rates and local currency unit (LCU) exchange rate estimated by the World Bank241
Using this methodology, we estimate that the average annual consumption of households in the bottom quintile of the consumption distribution in these countries is as follows:
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Annual consumption per adult equivalent in lowest quintile (local currency, latest year available) | 31,583 | 86,000 | 441,096 | ||
Adjustment to express estimates in per capita terms | 85% | 93% | 92% | ||
Annual consumption per capita in lowest quintile (local currency, latest year available) | 26,818 | 76,823 | 2,688 | 79,588 | 406,335 |
Years since last measure | 3 | 5 | 5 | 8 | 2 |
Consumption growth per year (%) | 2% | 1% | 1% | 2% | 2% |
Annual consumption per capita in lowest quintile (local currency, 2024) | 28,459 | 80,742 | 2,825 | 93,249 | 448,627 |
Official USD exchange rate in 2017 | 103 | 730 | 64 | 832 | 3,611 |
LCU conversion factor in 2017 | 44 | 249 | 22.6 | 298 | 1,219 |
Bottom quintile annual consumption per capita (2017 USD PPP) | $652 | $324 | $125 | $313 | $368 |
Baseline consumption surveys from GiveDirectly RCTs
We think these estimates are a good starting point, but the cross-country differences seem implausibly big to us and some estimates seem implausibly low.242 To triangulate these estimates, we also look at baseline consumption surveys from RCTs of GiveDirectly’s Cash for Poverty Relief program, which have been conducted in 4 out of 5 of the countries we study. An advantage of these estimates is that they’re probably a more direct measure of GiveDirectly recipients; a disadvantage is that most are quite outdated now, and in some cases it’s not clear exactly what units baseline consumption is being expressed in.243 To convert these to comparable metrics, we similarly account for economic growth since the time of time data collection and convert all estimates into USD PPP 2017.
Our calculations can be found here and are presented below:
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Annual consumption per capita (nominal) | $88 | $128 | |||
Annual consumption per capita (PPP adjusted, year of data collection) | $604 | $249 | $489 | $589 | |
Adjustment for growth since time of data collection | 128% | 96% | 136% | 111% | |
Years since last measure | 10 | 5 | 8 | 8 | |
Consumption growth per year (%) | 2% | 1% | 2% | 2% | |
Annual consumption per capita (2017 USD PPP) | $736 | $272 | $420 | $690 |
GiveWell estimates
To come up with our best guess, we do some ad hoc smoothing of these inputs. Our calculations can be found in the Baseline tab of our cost-effectiveness model. In each country, our best guess of recipient consumption is higher than what’s implied by the bottom quintile of government consumption surveys. This feels reasonable to us, as we think it’s unlikely that GiveDirectly can perfectly target the bottom of the consumption distribution. To receive transfers via mobile money, households need to be within reach of a mobile network. According to this 2023 blog post by GiveDirectly, 16% of Africans do not live within reach of a mobile network, and we’d expect this to be highly correlated with poverty rates.244 GiveDirectly also does universal targeting within-villages, and while we’d expect poverty to be correlated in geographic clusters,245 we’d also expect a lack of means-testing within villages to hinder their ability to perfectly work up the consumption distribution, as there are probably some (relatively) rich households living in poor communities.
Our best-guess consumption estimates, and how these compare to what’s implied by these inputs, is illustrated below:
Sense checks
How do our estimates compare to international poverty benchmarks?
Our best-guess estimates imply that we think the average Cash for Poverty Relief recipient is living below the international extreme poverty line in each country. This feels reasonable to us, given the scale of extreme poverty in these contexts: as shown in this section, the poverty headcount rate (defined as the % of people living in extreme poverty) ranges from 30-75% in these countries.
Another benchmark we consider are the World Bank’s Food for Nutrition program’s estimates of the amount of money required for an energy-sufficient diet, defined as meeting basic calorie needs (assumed to be ~2,300 calories per day).246 The estimates below are 2021 estimates expressed in 2021 PPP USD. While this makes them not directly comparable to our estimates (which are 2024 estimates expressed in 2017 PPP USD), we nonetheless think they make for a useful sense check. Overall, all of our consumption estimates are above what the World Bank estimates is required for a calorie sufficient diet, though some estimates are quite close. This doesn’t seem inconceivable to us, since we don’t think this represents a literal starvation threshold. For example, in 2021, Our World in Data (using data from the World Bank) estimated that 1.1bn people lived below this benchmark.247
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
International extreme poverty line (PPP 2017) | $2.15 | $2.15 | $2.15 | $2.15 | $2.15 |
GiveWell estimated consumption per day (PPP 2017) | $1.79 | $1.29 | $1.23 | $1.46 | $1.71 |
Daily cost of a calorie sufficient diet (PPP 2021) | $1.14 | $0.64 | $0.76 | $1.05 | $1.25 |
How do the relative ordering of our estimates compare to other measures of deprivation?
To sense-check whether the relative ordering of these estimates makes sense, we look at how these countries compare on other measures of deprivation or material well-being. We consider four other measures: GDP per capita, the World Bank’s Poverty and Inequality platform estimates of median consumption, the multidimensional poverty index and child stunting rates. While these data points don’t perfectly triangulate, we think they point in a generally consistent direction: across each dimension people in Kenya appear the least deprived while people in Malawi and Mozambique appear the most deprived (see the right half of the below chart for comparison across these metrics, with Kenya as the comparator).
Kenya | Malawi | Mozam- bique | Rwanda | Uganda | Kenya | Malawi | Mozam- bique | Rwanda | Uganda | |
---|---|---|---|---|---|---|---|---|---|---|
Best guess of consumption per capita of GiveDirectly recipients | $652 | $470 | $450 | $533 | $626 | 100% | 72% | 69% | 82% | 96% |
PIP median consumption estimates (per day) | $3.10 | $1.53 | $1.29 | $2.07 | $2.44 | 100% | 50% | 42% | 67% | 79% |
GDP per capita | $5,700 | $1,494 | $1,683 | $3,030 | $2,793 | 100% | 26% | 30% | 53% | 49% |
% of people not living in multidimensional poverty | 55% | 22% | 43% | 48% | 100% | 40% | 78% | 87% | ||
% of children not stunted | 81% | 63% | 62% | 67% | 72% | 100% | 78% | 77% | 84% | 89% |
7.2 Recipient consumption gains
Sense checks
How does the marginal propensity to consume implied by our assumptions compare to experimental estimates?
By year 2, our assumptions imply that an individual receiving $488 of a cash transfer will increase their consumption by $362, implying a marginal propensity to consume of 0.74 in Kenya. This is similar to Egger et al. (2022)’s estimate, who at a 19 month follow-up, estimate an MPC of 0.76 in their experimental context.248
How do our assumptions about relative consumption gains through time compare to experimental estimates?
Our assumptions imply the following dynamics of relative consumption gains in Kenya. When we plot this against the experimental trials with multiple follow-ups, we think this maps the general fade out of effects implied by the evidence. We think we should be careful about reading too much into the y axis of this graph, since relative consumption gains will vary by transfer size, which differ across studies. Nonetheless, the fact that these data points imply a downward sloping relationship between i) relative consumption gains and ii) the elapsed time since the transfer seems consistent with our assumptions. The 5-7 year follow-up to Egger et al. is a possible exception to this – we discuss this further in our uncertainties section.
Is the average Cash for Poverty Relief recipient meaningfully different now that GiveDirectly has switched to universal within-village targeting?
5 of the 7 RCTs we found of the Cash for Poverty Relief program studied a version with within-village means testing – households were only eligible to receive transfers if they had a thatched roof. GiveDirectly now does universal within-village targeting, so the households being studied in these RCTs may not be entirely representative of current Cash for Poverty Relief recipients.
If we think: i) the average recipient is slightly richer now that GiveDirectly has moved towards universal targeting and ii) richer recipients are likely to see greater consumption gains (as argued in this section), we might think the average treatment effects estimated in these trials might understate the ‘true’ treatment effect of current programming.
We don’t think this is likely to make a big difference because we don’t think the average Cash for Poverty Relief recipient is meaningfully different after the shift in targeting criteria. Intuitively, we’d expect poverty to have a fairly high degree of spatial correlation, with poverty concentrated in certain villages/regions of the country. Empirically, when we look at differences in consumption and assets between eligible and ineligible (control) households in the raw data of Egger et al.,249 we also don’t see meaningful differences in consumption between households with and without a thatched roof,250 though there are larger differences in baseline assets.251
Eligibles in the control group | Ineligibles in the control group | |
---|---|---|
Annualized consumption (PPP) | $2,491 | $2,560 |
Baseline assets (PPP) | $647 | $1,295 |
7.3 Spillovers to non-recipients
Methodology
The experiment studied in Haushofer and Shapiro employs two-stage randomization: first, they randomize at the village-level (e.g. some villages get transfers while others don’t); second, they randomize at the household-level (e.g. within villages selected for transfers, some households randomly receive transfers will others don’t).252 To measure consumption spillovers to non-recipients, they compare the consumption of non-recipient households in villages that receive cash (treatment villages) to the consumption of non-recipient households in villages that don’t receive cash (control villages).253
For this to yield unbiased treatment effect estimates, they have to assume that non-recipient households in control villages weren’t ‘treated’ in some way by the nearby cash injection.254 This might be a bad assumption if: i) treatment and control villages are geographically close to each other; and ii) treatment and control villages interact/trade with each other, which might mean households in control villages are affected by the increased spending nearby.
If across-village consumption spillovers were positive, we’d expect this to bias the spillover treatment effects downwards. By comparing non-recipients in treatment and control villages at the endline, these regression specifications will miss out on ambient effects that affect both groups. Technically, this is called the ‘missing intercept’ problem, and is illustrated below.
Egger et al. also employ two-stage randomization, but unlike Haushofer and Shapiro, they randomize at the village and supervillage level, rather than the village and household level. Not only do they randomize which villages receive transfers (as is typical), they also randomize how likely nearby villages are to receive cash transfers (i.e. the saturation of the program in different clusters).255
Broadly speaking, this creates villages which fall into 3 categories: a) villages that received cash; b) villages that didn’t receive cash but were nearby those that did; c) villages that didn’t receive cash and weren’t nearby those that did. To estimate consumption gains accruing to a) recipient households and b) non-recipient households that are nearby recipient households, they essentially compare a) to c) and b) and ci). Hence, c) represents the ‘pure’ control group. By setting up the experiment this way, they are able to estimate the direct effect of spillovers to nearby villages (orange arrows).
Their actual approach is more sophisticated than this: i) they construct continuous measures of saturation intensity (essentially by summing up cash dispersed to nearby villages)256 and ii) use a Bayesian information criterion to establish the optimal definition of ‘nearby’.257 Essentially, since adding in villages >2km away to their model doesn’t make much difference to their results,258 they define ‘nearby’ as villages within 2km (i.e. at a pretty localized level).
By constructing continuous measures of treatment intensity – and randomly varying this intensity by experimental design – their key identifying assumption to produce unbiased treatment effects is that there is a linear relationship between the amount of cash injected and the outcome measure (e.g. household consumption).259 One way this assumption would be satisfied is if spillover effects gradually dissipate with distance – i.e. the further one gets from a village that receives cash. This assumption seems more reasonable to us than the identifying assumption in Haushofer and Shapiro. One reason this might not be satisfied is if there are ambient effects that affect the entire study region (e.g. inflation that affects everyone). This is discussed more in our internal validity section.
Egger et al. hypothesize that the large spillovers they see to non-recipient households are due to a demand-led expansion in economic activity.260 For example, a cash injection might increase the demand for a mechanic, as people having more cash in their pockets means they are able to finally repair their broken machinery they previously couldn’t afford to fix. An increase in demand means more business for the mechanic, putting more money in her pocket, which she then spends on other goods and services, incentivizing other people to increase their production, and so on. This concept of an increase in spending leading to knock-on increases in spending throughout the economy is known as an ‘economic multiplier’. It has a long pedigree in economics,261 and is sometimes used to justify expansionary fiscal policy in high-income countries as a means to ‘kickstart’ or stimulate the economy during a recession.
A possibly surprising aspect of the Egger et al. results is that this demand-led expansion coexisted with a lack of meaningful price inflation. Going back to our example, Egger et al. find that not only does the mechanic meet the additional demand, she does so without raising her prices. To explain this, a hypothesis Egger et al. put forward is the idea that the economy they study has excess ‘slack’ – i.e. an underutilization of factors of production.262 In the case of the mechanic, before the cash injection, we might imagine she spent most of her time waiting around for customers while her machines sat idle.263 When demand suddenly increased, she was able to costlessly meet this demand without buying new machines or hiring new workers.
In economic terms, Egger et al. speculate that the aggregate supply curve in this context is highly elastic.264 When there is a sudden shift in aggregate demand (e.g. from a virtuous spending cycle), the economy is able to meet this demand without firms needing to hire new machines or workers, which would put an upwards pressure on prices. In basic macroeconomic terms, the story Egger et al. are telling is essentially the one below:
The main building block of the Egger et al. multiplier are the average treatment effects they estimate in Table 1, which show how the consumption of recipient and nearby non-recipient households compare to distant non-recipient households ~18 months after the transfers began. These results are pasted below: consumption in distant non-recipient households was $2,536 per year, while consumption in recipient and nearby non-recipient households was $338 and $334 higher (under their preferred estimates).
Their economic multiplier essentially measures the change in local economic activity following the disbursement of cash transfers. The denominator is the amount of cash that was transferred into the study area; the numerator how much local economic activity (i.e. GDP) changed since the transfers were disbursed:265
GDP is defined by a standard national accounts definition:266
GDP = C + I + G + NX267
To get to here from the results in Table 1, the authors essentially do 3 things:
- Ignore government consumption (G) and net exports (NX). They exclude G because they find elsewhere that the program had a precise null effect on it.268 They exclude NX because they essentially can’t measure it,269 though do some robustness checks to see how the multiplier would change under more/less conservative assumptions
- Convert the static/annualized estimates in Table 1 to cumulative measures. To do this, they exploit another dimension of randomization: time. They use the fact that the time of follow-up was random to re-estimate the regressions underlying Table 1 in quarterly chunks. That is, they estimate consumption/asset responses at 3 month, 6 month, 9 month etc. follow-up,270 then sum these up to estimate the cumulative consumption response. One limitation is that they have very sparse data in the immediate months following the transfer, as they acknowledge271
- Apply these treatment effects (estimated on a sample) to the entire population. Egger et al. are trying to estimate GDP effects across the entire study area (65,000 households), but only survey a subset of households within this (~5,000 recipients, ~5,000 non-recipients). To go from sample estimates → population estimates, the authors combine these treatment effects with a population census they conducted before the transfers went out.272 Within their study area (3 sub-counties of Siaya), this census suggested that ~10,000 households received transfers,273 ~45,000 households274 didn’t receive transfers but were in the same village as or within <2km of a village that did; ~10,000 households275 didn’t receive transfers and weren’t nearby those that did
Since they ignore G and NX, the equation Egger et al. are estimating essentially boils down to: GDP=C+I. Egger et al. find that most (2.0) of the 2.5 multiplier is being driven by increases in consumption (C).276 Within consumption, they find that most (80%) is being driven by consumption gains to nearby non-recipients277
To sense-check this, we reconstructed Egger et al.’s multiplier from the ‘bottom-up’, using the same treatment effects reported in Table 1 above. These calculations can be found here and are walked through below. Specifically, we assume:
- 10,500 households receive a $1,871 PPP transfer, in-line with what the paper says.278 This corresponds to the denominator of the multiplier (i.e. the total transfer amount), which is $19.6m
- Assume recipient households see annualized consumption gains of $338 PPP at endline, in-line with the results in Table 1 above
- Since median follow-up was at 19 months, we multiply these gains by 2, making the simplified assumptions that: i) endline captures years 1-2; and ii) consumption gains were similar in years 0-1. This is a simplification, though Egger et al. note that they find a fairly flat expenditure response279
- All in, we estimate that consumption gains to recipients amount to $7.1m at the endline of Egger et al.
- Since consumption doesn’t include asset expenditure, we assume that recipients invest the remainder of their cash transfer that they haven’t spent on consumption (19.6m - 7.1m) = $12.5m. Intuitively, this figure captures things like spending on iron roofs
To estimate consumption gains to non-recipients, we assume:
- The total population in the study area = 65,000 households, and nearby non-recipients correspond to 67% of the total (43,550). This corresponds to what’s reported in the paper280
- Assume non-recipient households see annualized consumption gains of $338 PPP at endline, in-line with the results in Table 1 above
- Assume that these results pertain to years 1-2, and that there’s a similarly ‘flat’ returns profile between then and years 0-1
- All in, we estimate that consumption gains to non-recipients amount to $29.1m at the endline of Egger et al.
To get to a total economic multiplier, we sum up recipient consumption ($7.1m) and investment/asset gains ($12.5m) with consumption gains to non-recipients ($29.1m), to get a total estimated increase in economic activity of $48.7m. When we divide this by the total transfer amount ($19.6m), we get an economic multiplier estimate of 2.5 – consistent with what Egger et al. find in their paper.
When we break this down, we find that 15% of the increase in economic activity comes from consumption gains to recipients; 26% comes from asset/investment gains to recipients (e.g. iron roofs); and 60% comes from consumption gains to non-recipients. Reassuringly, this is similar to Egger et al.’s own decomposition. They find that consumption contributes around 80% to their overall multiplier,281 and that non-recipients contribute 82% of the consumption gain.282 This suggests that (0.8*0.82) 66% of the multiplier comes from consumption gains to non-recipients and (0.8*0.18) 14% comes from consumption gains to recipients. If we similarly assume no changes in non-recipient assets, this suggests 20% of the multiplier comes from asset gains to recipients. Our decomposition is illustrated side-by-side by Egger et al.’s decomposition below:
Experimental context
In our external validity adjustment of our non-recipient spillovers section, we argue that Siaya county (the setting of Egger et al. 2022) is likely richer and less remote than the current contexts that GiveDirectly’s Cash for Poverty Relief program operates in. This section outlines some basic descriptives to support this claim.
Poverty rates
In Kenya, GiveDirectly currently operates its Lump Sum Program in Baringo and Kilifi counties.283 We expect these to be poorer than Siaya county: the latest Kenya Poverty report estimated a poverty headcount rate of 34.2% in Siaya vs. 47.5% and 49.2% in Baringo and Kilifi counties.284 Additionally, we’d expect average recipients of the Cash for Poverty Relief program to be poorer in Malawi/Mozambique/Rwanda/Uganda compared to Kenya, as argued in this section.
Physical remoteness
We think Siaya county has unusually high population density and unusually good road access, which likely makes it less remote than other contexts GiveDirectly operates in.
For example, at the time of the Egger et al. study, Siaya had a population density of 393 people per km².285 This is larger than national averages in Kenya/ Malawi/ Mozambique/ Uganda, which probably overstates the population density in places where GiveDirectly operates, since we’d expect population density to be lower in rural villages (which GiveDirectly predominantly targets). Participants in the Egger et al. study also seem close to markets: the average village had 0.7 markets located within 2 km and 2.3 markets within 4 km, and recipients’ average commuting time to their preferred market was 31 minutes, with over 80% of people traveling on foot.286
Siaya | Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|---|
People per km2 | 393 | 93 | 210 | 41 | 546 | 228 |
In addition to being close to each other/markets, participants also probably had unusually good road access: as Egger et al. note, the main national road running from the port of Mombasa to Nairobi and on to Kampala passes through the study area, likely helping to integrate it into the regional economy.287 According to the World Bank’s Rural Access Index, 98% of rural people in Siaya live within 2km of an all-season road, compared to an average of 70% in Kenya and 23% in Malawi.
Siaya | Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|---|
Rural Access Index | 98% | 70% | 23% | 20% | 55% | 53% |
Sense checks
How do our estimates compare to wider economic multiplier estimates?
Overall, we estimate that consumption benefits to non-recipients from GiveDirectly’s Cash for Poverty Relief program comprise ~60-70% of the consumption benefits to recipients. One way we sense-check this is by comparing it to economic multipliers that have been estimated from other cash transfer programs, which are summarized in this recent World Bank systematic review (Gassman et al., 2023).
To do this, we reverse the logic of our previous calculations to go from a spillover estimate back to an economic multiplier. Specifically, we:
- Take our estimates of the total transfer amount per person as our denominator
- Estimate consumption gains to recipients based on our assumptions about recipient consumption returns
- Apply our best guess of the consumption benefits to non-recipients as a % of the consumption benefits to recipients
- Strip out our distributional adjustment. We needed this to convert things from $ terms to units of value terms, taking into account diminishing marginal utility of consumption. Since economic multipliers are expressed in $ terms, we no longer need this adjustment
- Divide consumption gains to recipients & non-recipients by the total transfer amount to get to the economic multiplier our assumptions imply
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Total transfer amount per person (denominator) | $488 | $651 | $639 | $650 | $589 |
Recipient consumption/investment returns by year 2 | $362 | $483 | $474 | $482 | $437 |
Non-recipient consumption benefits (as % of recipient consumption benefits) | 68% | 59% | 59% | 63% | 63% |
Distributional adjustment | 80% | 80% | 80% | 80% | 80% |
Non-recipient consumption/investment returns by year 2 | $305 | $357 | $350 | $420 | $346 |
Recipient + non-recipient consumption & investment returns by year 2 (numerator) | $668 | $840 | $824 | $950 | $783 |
Implied multiplier by year 2 | 1.4 | 1.3 | 1.3 | 1.3 | 1.3 |
Overall, our assumptions imply that we expect an economic multiplier of 1.3-1.4 two years after the cash is distributed. When we compare these to the results summarized by Gassman et al. (2023), this seems broadly in the same ballpark as other estimates: almost all the studies report economic multipliers of >1, and the Egger et al. (2022) estimate of 2.5 is the second largest one reported. We don’t want to place much stock on these external estimates because we think Egger et al. seems higher quality,288 and because Gassman et al. explicitly warn against comparing outcomes across studies,289 given different periods of follow-up and definitions of economic multiplier. Nonetheless, the fact that most multipliers are estimated to be >1 (which implies positive spillovers) seems broadly reassuring for the direction of our estimates.
Could there be spillovers across time we’re missing?
The economic multiplier in Egger et al. is a cumulative measure: the numerator essentially sums up all economic activity that occurred since the start of the cash transfers up until the point of endline, which in Egger et al. (2022) was at 19 months (median follow-up time).290 We can’t think of a good reason to expect spillovers to stop beyond this, so one concern with using this estimate could be that we’re missing out on spillovers across time.
We think this is addressed in principle by our pegging of non-recipient consumption gains to recipient consumptions, which we model across 10 years. Since we assume non-recipient consumption gains are a constant fraction of recipient consumption gains, this means we effectively assume that spillovers will continue to accrue over 10 years, and will dissipate at the same rate as the consumption benefits to recipients. Concretely, our economic multiplier in Kenya is assumed to grow as follows:
Kenya | Year 2 | Year 3 | Year 4 | Year 5 | Year 6 | Year 7 | Year 8 | Year 9 | Year 10 |
---|---|---|---|---|---|---|---|---|---|
Cumulative consumption gains of recipients | $362 | $429 | $483 | $526 | $560 | $588 | $610 | $627 | $641 |
% increase from previous year | 19% | 13% | 9% | 7% | 5% | 4% | 3% | 2% | |
Implied consumption gains of non-recipients | 19% | 13% | 9% | 7% | 5% | 4% | 3% | 2% | |
What this implies about by expected multiplier growth | 1.4 | 1.6 | 1.8 | 2.0 | 2.1 | 2.2 | 2.3 | 2.4 | 2.4 |
While we think our approach accounts for spillovers across time in principle, in practice, we’re very uncertain about the rate at which spillovers fade-out, and whether these fade-out at the same rate as consumption gains to recipients. Egger et al. also plan to measure consumption spillovers in their 5-7 year follow-up, and preliminary results we’ve seen suggest: i) spillovers to non-recipients fading out at a slightly faster rate as recipient consumption gains but ii) both appearing much more persistent than what’s implied by our current assumptions. As explained in this section, we’re hesitant to put too much weight on these results at the moment as they haven’t been externally scrutinized and seem inconsistent with other longer-term follow-ups to cash transfer programs, though we may revisit this after these results get peer reviewed.
Should our estimates depend on the share of non-recipients vs. recipients in a given program area?
GiveDirectly’s current Cash for Poverty Relief program employs a more saturated model than the program studied in Egger et al. (2022), so for any given geographic space, we would expect the ratio of non-recipients to recipients to be smaller. If the ratio of non-recipients to recipients is smaller, we think consumption spillovers (as a % of recipient consumption gains) are likely to be smaller.
However, we think this is offset by a countervailing consideration. If a cash injection generates excess economic activity through a virtuous spending cycle, this excess economic activity has to flow somewhere. If there are less non-recipients in the area, this is likely to mean benefits are more likely to accrue back to recipients. What were spillover effects in the Egger et al. study will become ‘spillback’ effects under more saturated program designs. We think it makes more sense to think of the benefit stream being discussed here as the more encompassing “consumption benefits from increased local economic activity" and while we think we ought to account for this being larger or smaller under more saturated programs, this is already being accounted for in our external validity discussion.
7.4 Mortality benefits to recipients
Sense checks
How do the experimental results square with the cross-country relationship between consumption and under 5 mortality?
To sense-check the Egger et al. result, we first looked at the cross-country relationship between consumption and under 5 mortality. In the experiment Egger et al. study, treated households receive a $1,871 (PPP) transfer,291 and are estimated to consume 76% of it by the ~18 month follow-up.292 Household consumption is roughly $2,500 per year (PPP) in the control group. If we assume: i) resources are shared equally among household members and ii) the consumption gains are spread out evenly over the 18 months,293 this implies that consumption per capita in recipient households increased from $604 per year to $830 per year (a 37% increase) in the ~2 years after transfers were disbursed. Over the same period (2015-2017), the all-cause under 5 mortality rate in recipient households was 26.7 deaths per 1,000 births – 46% lower than the all-cause under 5 mortality rate in control (non-recipient) households.294
Altogether, these assumptions imply the following differences in annual consumption and u5 mortality rates between recipient (treated) and non-recipient (control) households:
Annual consumption per capita (PPP) | u5 mortality rate (%) | |
---|---|---|
Egger et al - non-recipients | $604 | 5.7% |
Egger et al - recipients | $830 | 3.1% |
To check whether the implied relationship between consumption gains and mortality reduction seems plausible, we downloaded country-level data on annual consumption and u5 mortality from the World Bank and UNICEF. This data is plotted below. The under-5 mortality rate found among recipient households in this trial seems like an outlier – below all the country-level averages, even for countries where average consumption seems significantly higher compared to GiveDirectly recipients (both pre and post transfer). For example, even after receiving the transfers, we think the GiveDirectly recipients were still meaningfully poorer than the average person in Kenya – yet the under 5 population had a 33% lower all-cause mortality rate.295
We think we should be careful about reading too much into this graph – the World Bank may have had different methods of data collection compared to Egger et al., and some of the data points correspond to different time periods to the experimental timeframe.296 More importantly, this graph plots a correlational relationship that isn’t necessarily causal – for instance, there could be other factors that are: i) correlated with per capita consumption and ii) also have an effect on u5 mortality, which complicates causal inference on the basis of this graph. Still, when we think about the likely direction of omitted variable bias, it tends to make the Egger et al. result seem even more remarkable. For example, as well as having higher consumption, the average Kenyan probably had access to better sanitation and health services vs. the recipients in Egger et al.,297 who lived in a poorer-than-average region of Kenya.298 Hence, it seems even more surprising that the u5 mortality rate among recipients in Egger et al. was lower than the country average.
7.5 Where is our update coming from?
Our best guess is the donations to GiveDirectly’s Cash for Poverty Relief program are ~3-4x our previous estimate. Quantitatively, the most important drivers of this update are: i) modeling the program in countries beyond Kenya and ii) an updated best-guess of consumption spillovers on non-recipients. This section gives more context on what caused us to update on the latter.
Kenya | Malawi | Mozambique | Rwanda | Uganda | |
---|---|---|---|---|---|
Previous best-guess | 1.0 | ||||
Previous + baseline consumption update | 1.1 | 1.7 | 1.5 | 1.4 | 1.2 |
Previous + long-run consumption update | 1.3 | 2.1 | 1.9 | 1.8 | 1.4 |
Previous + spillovers update | 2.1 | 3.1 | 2.8 | 2.7 | 2.2 |
Previous + mortality update | 2.4 | 3.5 | 3.4 | 3.0 | 2.6 |
New best-guess (x previous estimate) | 2.6 | 3.8 | 3.7 | 3.3 | 2.8 |
What we thought before
Previously, we had a -5% adjustment for consumption spillovers to non-recipients – i.e. we assumed non-recipients would be made slightly worse-off by not receiving a transfer. Our rationale for this is published here. We were basing our adjustment off 5 papers that had tried to estimate spillovers to non-recipients: Haushofer et al., 2015; McIntosh & Zeitlin 2020; Haushofer & Shapiro 2016; Haushofer & Shapiro 2018; and preliminary results of the paper that would become Egger et al. 2022. At the time, we thought that the weight of the evidence suggested that nearby non-recipients were made slightly worse off by not receiving a Cash for Poverty Relief transfer, so we applied a small (-5%) adjustment to account for this.
What we think now
Since this previous update, the full results of Egger et al. (2022) have been published, which has changed our mind in two ways:
- The published results imply larger spillovers than the preliminary results we were sent, for reasons discussed below
- We now put more weight on these findings than we did before, since they’ve: i) been through academic peer review; ii) seem robust to a reanalysis we commissioned; iii) are underpinned by a reasonably compelling theoretical explanation (discussed here)
Preliminary results vs. published findings
The preliminary results we were sent are uploaded here. These results imply small, statistically insignificant increases in consumption to nearby non-recipient households compared to control households in more distant villages.299 By contrast, the published results imply large, statistically significant increases in consumption in nearby non-recipient households vs. distance controls.300
We think the main reason for this difference is because different identifying equations were used in the preliminary vs. published findings. To estimate spillovers, the preliminary analysis estimates the following regression equation:
Where: T is an indicator for households residing in a treated village, E is an indicator for whether the household is eligible for transfers, and H is an indicator for living in a high-saturation sublocation. In this equation, 3 and 6 capture the impact of: i) not receiving cash transfers but ii) living in a region that did.
This regression is different to how spillovers to non-recipients are estimated in Egger et al. (2022), which are estimated as follows:
The novel term is here Amt, which is essentially the amount of cash (per capita) transferred to other villages within a 2km radius of the households own village.301 This is a much more precise measure of the size of the cash injection in nearby communities: it measures variation in nearby treatment intensity continuously, whereas the preliminary results measure this using blunt categorical variables.302 Since measures of nearby treatment intensity were measured very bluntly in the preliminary results, we’d expect the coefficients on these variables to be noisily estimated and biased downwards, as measurement error in independent variables causes regression coefficients on these variables to be biased towards 0.
- 1
- We currently fund new opportunities that are at least 10 times as cost-effective as (our previous estimation of) unconditional cash transfers. 3x to 4x is 30%-40% of 10x.
- “We use GiveDirectly's unconditional cash transfers as a benchmark for comparing the cost-effectiveness of different funding opportunities, which we describe in multiples of "cash." Thus, if we estimate that a funding opportunity is "10x cash," this means we estimate it to be ten times as cost-effective as unconditional cash transfers…As of August 2024, our bar for funding Top Charities is 8x cash, and our bar for funding other programs is 10x." GiveWell, GiveWell's Cost-Effectiveness Analyses.
- 2
“Every household in a village receives ~$1,000 with no strings attached." GiveDirectly, Large transfers for poverty relief.
- 3
- “As of February 2024, our poverty relief program has reached over 440,000 households in rural villages in Kenya, Liberia, Malawi, Mozambique, Rwanda and Uganda." GiveDirectly, Large transfers for poverty relief.
- “We primarily use government data to target villages where most or all residents are living in extreme poverty, defined by the World Bank as earning under $2.15/day." GiveDirectly, Large transfers for poverty relief.
- 4
The Cash for Poverty Relief program is currently operational in Kenya, Malawi, Mozambique, Rwanda, and Uganda. There are no active Cash for Poverty Relief programs in Liberia currently running. GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 5
The Basic Income program is currently operational in Kenya, Malawi, Mozambique, and Liberia. See GiveDirectly’s Basic Income page.
- 6
“The transfer amount varies from program to program. Emergency relief transfers are smaller ($200-$500 range)." GiveDirectly, Cash for Refugees.
- 7
The 13 countries are: USA, Nigeria, Liberia, DRC, Rwanda, Uganda, Kenya, Malawi, Mozambique, Turkey, Yemen, Morocco, and Bangladesh. See GiveDirectly’s Emergency Relief page.
- 8
“Donations made through this page will be directed exclusively to recipients who are registered refugees living in an African country." GiveDirectly, Cash for Refugees.
- 9
“Emergency relief transfers are smaller ($200-$500 range), whereas resilience-building transfers are larger ($800-$2,000) and are sometimes broken down into monthly payments." GiveDirectly, Cash for Refugees.
- 10
“The transfer amount varies from program to program. Emergency relief transfers are smaller ($200-$500 range), whereas resilience-building transfers are larger ($800-$2000) and are sometimes broken down into monthly payments." GiveDirectly, Climate Survival Fund.
- 11
See GiveDirectly’s cash in the US page.
- 12
See a breakdown of GiveDirectly’s funding allocations by program here. Data sent to us by GiveDirectly.
- 13
- The exact amount varies due to exchange rate fluctuations and slight differences in program design. For example, the Kenya transfer size has always been 110,000 Ksh. This has historically been ~$1,000, but is currently closer to ~$865 due to exchange rate fluctuations. In Malawi, transfers are targeted at the individual – each adult receives a transfer of $550 (nominal), which corresponds to a transfer of $1,100 assuming a two-adult household.
- “Kenya transfer size has always been 110,000 Ksh which historically has been ~$1,000 – currently closer to ~$865 due to fx volatility."
- “RW [Rwanda] transfer design is ~$1,100 per household even if it's a single-headed HH, but MW [Malawi] is ~$550 per person." Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 14
GiveDirectly also has other cash transfer programs running in Bangladesh, DRC, Liberia, Morocco, and Nigeria. See GiveDirectly’s website for more details.
- 15
GiveDirectly has also reached households in Liberia through their Cash for Poverty Relief program. This report only focuses on the 5 countries with ‘live’ Cash for Poverty Relief programs (as of October 2024).
- 16
This figure assumes that 100% of transfers to date have been targeted to households in extreme poverty. Since we’d guess the true figure is lower than this, because of imperfect targeting, we consider this an upper bound.
- 17
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 18
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 3, 2024 (unpublished).
- 19
GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 20
“For more detail - regional assessments typically include multiple dimensions, including: Poverty levels; Neglectedness, i.e. whether communities are already receiving aid from the gov’t or other NGOs; Government priorities and permissions to operate…We also have to consider funding constraints and operational feasibility / efficiency in these decisions.
All else held equal, we aim to reach the poorest - but these communities are generally in more remote / less secure geographies that cost more to reach (e.g. if we need to expand mobile money coverage like in Liberia), so we have to weigh these tradeoffs." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished). - 21
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 22
“For more detail - regional assessments typically include multiple dimensions, including: … Government priorities and permissions to operate – e.g. in Rwanda reducing stunting has been a top priority so our team considered relative childhood stunting levels alongside monetary poverty levels in regional scoping." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 23
“In-country GiveDirectly staff – not contractors – go door-to-door enrolling one member of each household in the entire village. They also hold large meetings to prepare the community for the coming transfers." GiveDirectly, Large Transfers for Poverty Relief.
- 24
- Videos of these sensitization meetings can be found here.
- “Each eligible household will receive their cash transfers in two tranches. Once you receive the two tranches, there will be no other additional transfers." GiveDirectly, Baraza - Registration, 2021 (translated).
- 25
“Our enrollment process spans multiple days, in part to minimize exclusion of likely eligible recipients. It generally includes: (i) a community sensitization meeting led by GiveDirectly staff, which is typically attended by at least one member of every household in the village (ii) village census where field staff go door-to-door cross-checking households against residency lists from local leaders." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 26
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 27
“We’ve used the material that people’s roofs are built from as an eligibility criterion: people with thatch roofs turn out to be much poorer than those with metal roofs in areas where we’ve been working, and it’s even possible to confirm roof type from space using satellite imagery." GiveDirectly, “How do we tell who is poor?" 2016.
- 28
“In 2017, we decided that — once we identified a village that met GiveDirectly’s eligibility criteria — we would transfer funds to every household in the community (i.e., we would “saturate" the community), as opposed to targeting only the very poorest families in that village…To this end, saturation will continue to remain our default enrollment method." GiveDirectly, “Reflecting on the Last Decade: 10 Things We Got Right & Wrong," 2019.
- 29
“We began using the saturation model in 2017. This is our default design today, but there are still many programs where we layer some form of need-based targeting, especially if we expect that >20% of households are above the extreme poverty line, or where it's mandated by the government." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 30
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 31
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 32
For example, the Kenya transfer size has always been 110,000 Ksh. This has historically been ~$1,000, but is currently closer to ~$865 due to exchange rate fluctuations.
- 33
GiveDirectly will return to a village multiple times to ensure all eligible recipients have the opportunity to enroll. “Staff will typically visit a village 2-3+ times to help ensure all eligible recipients are registered. In some contexts they will also have multiple phases of enrollment with a few months in between, e.g. to include people who relocate for seasonal work." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 34
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 35
“We don’t specifically collect data on ‘missed’ recipients but expect this would be similarly very low, especially in regions where we have been working for some time. Average ‘written off’ rate is 2% but this category includes people who did receive transfers but were lost during follow-ups because they moved, changed phone numbers, etc." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 36
“[The] standard process across countries is to register and pay neighboring, eligible villages within active districts at roughly the same time…there are definitely exceptions to this in practice given operational realities and a largely fixed number of field staff going village-to-village, but we don’t intentionally stagger this timeline or treatment intensity (like in Aggarwal et al) unless there is an RCT running." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 37
“About a month later, the funds are sent in two parts through a SMS-enabled banking technology called mobile money." GiveDirectly, Large transfers for poverty relief.
- 38
“In our main program, GiveDirectly gives those who don’t already have a mobile phone the option to buy one, subtracting the ~$15 cost from their first transfer, an offer taken by 90% of recipients." GiveDirectly, “The power of getting a mobile phone," 2023.
- 39
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 40
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 41
For households where each adult receives $550, this assumes a two-adult household.
- 42
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 43
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 44
“Follow-up & audit: We then call each person to verify they received the funds and evaluate their experience with our program. We also provide toll-free call centers to offer support. Our internal audit team follows up with a subset of recipients to identify and investigate potential fraud." GiveDirectly, Large transfers for poverty relief.
- 45
“Since 2016, this fully independent team has run randomized 1:1 follow-ups to check the work of our enrollment teams, surface any inconsistencies or concerns from community members, and conduct investigations." GiveDirectly, “How GiveDirectly prevents, addresses, and reports risks," 2024.
- 46
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 8, 2024 (unpublished).
- 47
- “We can reject meaningful increases in reported spending on temptation goods, consistent with Evans and Popova (2017)." Egger et al. (2022), page 16
- “The treatment effects on alcohol and tobacco expenditure are negative and insignificant, although a lack of power does not allow us to rule out reasonably sized increases." Haushofer and Shapiro (2016), page 3
- “While estimated effects on self-reported measures are approximately zero (and rule out large changes), effects on measures of drinking by others are significantly negative: respondents reported seeing fewer of their neighbors drinking daily, and were less likely to perceive drinking as a problem." Banerjee et al. (2023), page 21
- 48
“This article reviews 19 studies with quantitative evidence on the impact of cash transfers on temptation good expenditure, as well as 11 studies that surveyed whether respondents reported they used transfers to purchase temptation goods. We conduct a meta-analysis to gauge the average impact of transfers on temptation goods. Results show that on average cash transfers have a significant negative effect on total expenditures on temptation goods, equal to −0.18 standard deviations." Evans and Popova (2017), page 1
- 49
This is mathematically equivalent to assigning a value of 1 to doubling consumption.
- 50
For more on our discount rate, see here.
- 51
“As of today, we'll be restricting "Poverty relief - Africa" donations to our lump sum programs, which we're now calling "Cash for Poverty Relief" here for simplicity.” Kayla Fishman, Senior Growth Manager, GiveDirectly, email to GiveWell, October 25 2024 (unpublished).
- 52
One oversimplification is that we assume uniform distributions for each parameter. We don’t think the resulting cost-effectiveness intervals should be taken literally.
- 53
Intuitively, we’d expect $1,000 to ‘go further’ for someone poorer, as it could be the difference between having their basic food, shelter and healthcare needs being met vs. not
- 54
- “The $2.15-a-day poverty line in the 2017 PPP updates the $1.90-a-day poverty line in the 2011 PPP and is often referred to as the “extreme poverty line." World Health Organization, “Population with impoverishing health expenditures, at the 2017 PPP $2.15 a day poverty line."
- “Absolute [poverty] lines aim to measure the cost of certain “basic needs," which are often interpreted as physiological minima for human survival; nutritional requirements for good health and normal activity levels are widely used to anchor absolute lines." Ravallion (2010), page 5
- 55
The poverty gap is the ratio by which the mean income of the poor falls below the poverty line.
- 56
We make slight adjustments for this, but they’re speculative.
- 57
We sourced these papers based on conversations with GiveDirectly and recent meta-analyses published of UCT programs: Leight et al. (2024); Crosta et al. (2024).
- 58
Paul Niehaus, GiveDirectly co-founder and US director, is a coauthor of all 3 papers; Michael Faye, GiveDirectly executive chair and co-founder, is a coauthor on Banerjee et al. 2023.
- 59
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 60
“We enrol households into the study if they are defined as poor by the Government of Rwanda, and contain either pregnant women or children under five years of age." McIntosh and Zeitlin (2024), page 2
- 61
“Each adult over the age of 18 received a one-time payment of about US $500 nominal." Banerjee et al. (2023), page 10
- 62
“We find a strong consumption response to transfers, with an increase in household monthly consumption from $158 PPP to $193 PPP nine months after the transfer began." Haushofer and Shapiro (2016), page 1. 193-158=35. 35/158 = 22%
- 63
See the 0.3 coefficient on consumption in the ‘Large’ column of Table 3 of McIntosh and Zeitin (2024). Since the consumption variable is reported as an inverse hyperbolic sine, this can be interpreted as a percentage change.
- 64
“For expenditures and large purchases, treatment effects are substantial initially, but then fall to being indistinguishable from zero within 10-12 months." Aggarwal et al. (2024), page 13
- 65
“The effects on expenditure are fairly modest relative to Malawi (and to the total transfer size) and indistinguishable from zero." Aggarwal et al. (2024), page 13
- 66
“The 5th/95th percentiles of timing ranged from 12 to 27 months, and the median survey was conducted 19 months after the experimental start month." Egger et al. (2022), page 7
- 67
“Despite not receiving transfers, non-recipient households also exhibit large consumption expenditure gains: their annualized consumption expenditure is 13% higher eighteen months after transfers began, an increase roughly comparable to the gains contemporaneously experienced by the recipient households." Egger et al. (2022), page 3
- 68
See Table 5 in Banerjee et al. (2023). The point estimate on the lump sum arm is $514, 6% of the control group mean ($7,866)
- 69
See row 2, column 3 in Table 5 of Haushofer and Shapiro (2018). 17/187 = 9%
- 70
GiveWell, University of California, Berkeley – Follow-up for Cash Transfers Study, 2022
- 71
Michael Walker, Research Economist, University of California, Berkeley, email to GiveWell, July 16, 2024 (unpublished).
- 72
“We do, however, note suggestive evidence of smaller consumption effects for lump sum transfers in the long run." Crosta et al. (2024), page 24
- 73
“After nine years, we find these gains have dissipated. Grantees’ investment leveled off; controls eventually increased their incomes through business and casual labor; and so both groups converged in employment, earnings, and consumption levels." Blattman et al. (2020), page 1
- 74
“Treated men are also significantly more likely to be engaged in an income generating activity, though this does not translate into higher food security." Fiala et al. (2022), page 1
- 75
“We find no effects for women." Fiala et al. (2022), page 1
- 76
“We study two interventions for poor and underemployed Ethiopian youth: a $300 grant to spur self-employment, and a job offer to an industrial firm." Blattman et al. (2022), page 1
- 77
“After 1 year, the grant lead to higher earnings and self-employment, the job lead to worse health outcomes… after 5 years, we observe almost complete convergence in employment, earnings and health for both groups." Blattman et al. (2022), page 1
- 78
We think the distinction between investment and savings is blurred in this context – for example, buying livestock could be a way to both save and invest.
- 79
“Households invest part of the transfers: the value of nonland assets increased by $302 PPP on average; this represents 61% of the control group mean of $495 PPP and 43% of the average transfer." Haushofer and Shapiro (2016), page 26
"The pattern of expenditure effects by category is broadly consistent with earlier work
(Haushofer and Shapiro (2016))." Egger er al. (2022), page 16 - 80
See this section of our Appendix for a visualization of how the function implied by our assumptions matches to some of the experimental evidence.
- 81
This is generally lower than the GDP growth rates in some of these countries (e.g. Kenyan GDP growth was 5% in 2022). In general, we’d expect consumption in the lowest quintile to grow more slowly than aggregate GDP, because we’d expect gains from economic growth in low-income countries to be disproportionately captured by richer households (e.g. city dwellers, factory workers). This is based on Simon Kuznets’ inverted U-shaped hypothesis on the relationship between economic growth and inequality.
- 82
Part of the reason more optimistic assumptions about long-run persistence don’t translate into larger cost-effectiveness gains is because we apply a 4% discount rate to future consumption benefits. See this section for more details.
- 83
The control groups of previous studies may have been contaminated by positive spillovers across-villages, which may bias spillover treatment effects downwards (as explained in the next section). However, we think this is unlikely to explain fadeout in the studies with multiple follow-ups (e.g. Haushofer and Shapiro 2016 and 2018; Blattman et al. 2016 and 2020), since if the control group was contaminated this should affect both the short and long-run evaluations. For us to attribute fadeout to this possibility, we’d have to believe across-village spillovers got larger the more time that had elapsed since the cash was dispersed, which doesn’t seem very plausible to us.
- 84
“Thus we do not find evidence for “threshold" poverty trap models, at least for thresholds within the range of transfer amounts where our evidence is robust. But absence of evidence is not evidence of absence, particularly in this case, as this is a fairly weak test for the poverty trap theory given this is examining patterns at the study-level across markets and countries, rather than a household-level micro examination that attempts to incorporate household level heterogeneity which inevitably affects any such threshold." Crosta et al. (2024), page 26
- 85
Panel A, Table 2, Haushofer et al. (2022). I compare the difference between column 2 and column 1 to get relative consumption differences between the average and the bottom 50% of households. Since there are 50% of households in each, I assume the average household in the top 50% of households is the same distance from the pooled mean as the average household in the bottom 50% of households.
- 86
The current Cash for Poverty Relief program employs universal within-village targeting. “Every household in a village receives ~$1,000 with no strings attached." GiveDirectly, Large transfers for poverty relief.
- 87
“Despite not receiving transfers, non-recipient households also exhibit large consumption expenditure gains: their annualized consumption expenditure is 13% higher eighteen months after transfers began, an increase roughly comparable to the gains contemporaneously experienced by the recipient households." Egger et al. (2022), page 3
- 88
“For outputs, we document statistically significant, but economically minimal, local price inflation. Average price inflation is 0.1%, and even during periods with the largest transfers, estimated price effects are less than 1% and precisely estimated across all categories of goods." Egger et al. (2022), page 3
- 89
Michael Walker, Research Economist, University of California, Berkeley, email to GiveWell, July 16 2024 (unpublished).
- 90
We have taken the results from column (1) in Table III, page 2004 (the regression specification which does not include controls for baseline covariates). When including controls, the effect becomes slightly larger in absolute magnitude and statistically significant at the 10% level (see column (2)). The point estimate in column (1) a $7.77 decrease in monthly non-durable expenditure. From a control group mean of $182 (see column (1) in Table I, page 1992), this is a 4% decrease. Haushofer and Shapiro (2016).
- 91
"The point estimates suggest spillover households spend USD 30 PPP less than pure control households, or about 16% based on a pure control mean of USD 188 PPP." Haushofer and Shapiro (2018), page 3
- 92
Note: spillover results are reported in a separate paper (McIntosh and Zeitlin, 2022) to the recipient consumption results (McIntosh and Zeitlin, 2024).
- 93
“Looking at our primary outcomes, we find no evidence that spillovers from any treatment or to any treatment group are present." McIntosh and Zeitlin (2022), page 5
- 94
“We also test the null of no cross-village spillovers and, for most outcomes, we do not reject this null." Banerjee et al. (2023), page 17
- 95
- Haushofer and Shapiro (2016) employ two-stage randomization: first, they randomize which villages receive transfers, and then which individuals receive transfers within these villages. To estimate spillover effects, they compare the consumption of non-recipients in treatment villages to non-recipients in control villages. For this to produce unbiased treatment effects, they have to assume control villages weren’t ‘treated’ in some way by the cash transfers.
- While the paper doesn’t report geographic distances between treatment and control villages, Johannes Haushofer sent us this information via email. “I started with the household GPS coordinates, computed village centroids, used the geodist command in Stata to calculate pairwise distances, and then took the minimum for each village. The code, the resulting minimum distances, and a histogram, are below. Units are km. Mean and median are about 1km, so I think you're correct." Johannes Haushofer, Professor of Economics, Stockholm University, email to GiveWell, August 20 2024 (unpublished).
- 96
Theoretically, this assumption seems dubious because nearby households/firms likely trade with each other. Empirically, Egger et al. (2022) find evidence of positive spillovers accruing within 2km of treated villages. “For market-level price inflation, we include the amount of cash received by recipients living within 2 and 4 kilometers of a market. Spatial specification of this kind require a procedure for deciding how far spillovers are to be taken into account. Egger et al. use a pre-specified algorithm that typically selects a maximum buffer of either two or four kilometers (depending on the outcome), implying that the effect of cash transfers on villages located further away is zero." Walker et al. (2024), page 31
- 97
Egger et al. (2022) find positive across-village spillovers – i.e. non-recipient households saw consumption gains when nearby villages received cash. If across-village spillovers were also positive in Haushofer and Shapiro, this would inflate consumption in the control group, which we’d expect to bias treatment effects downwards. This is explained further in this section of the Appendix
- 98
Egger et al. (2022) assumes that its outcome variables (e.g. consumption) respond linearly to the amount of cash distributed in nearby villages, which is deliberately randomized according to the experiment’s design. One way this assumption could be satisfied is if consumption spillovers decay linearly with distance. This identifying assumption is discussed in more detail in this section of the Appendix.
- 99
- The experiment studied in Egger et al. (2022) distributed cash to 10,500 households. By contrast, the experiment studied in Haushofer and Shapiro (2016) distributed cash to 503 households.
- “We provided one-time cash transfers of about USD 1000 to over 10,500 poor households across 653 randomized villages in rural Kenya." Egger et al. (2022), page 1
- “After baseline, the research team randomly chose half of the eligible households to be transfer recipients (second stage of randomization). This process resulted in 503 treatment households." Haushofer and Shapiro (2016), page 1983.
- “Specifically, 258 of the 503 treatment households were assigned to the monthly condition and 245 to the lump-sum condition." Haushofer and Shapiro (2016), page 8
- 100
See this section of our Appendix for an illustration of the experimental variation in Egger et al. (2022).
- 101
- “This is not surprising from a power perspective. Compared to Egger et al. (forthcoming), we are half as dense, with about half as many villages and no super-village intensity variation." Banerjee et al. (2023), page 18
- “That said, the amount of variation we had in this project was relatively small, the youth involved made up a pretty small fraction of the population of their villages, and so this was a low-powered way of looking at spillovers and I’m not sure that our failure to reject is particularly meaningful." Craig McIntosh, School of Global Policy and Strategy, University of California, San Diego, email to GiveWell, August 19 2024 (unpublished).
- 102
“Despite not receiving transfers, non-recipient households also exhibit large consumption expenditure gains: their annualized consumption expenditure is 13% higher eighteen months after transfers began, an increase roughly comparable to the gains contemporaneously experienced by the recipient households." Egger et al. (2022), page 3
- 103
In their context, ‘nearby’ = within 2km. They use a Bayesian information criteria to decide on the optimal definition of ‘nearby’. Essentially, because their regression results don’t change much when they start to add households that were >2km away, they assume only households that were <2km are indirectly affected by the transfers.
“Because we had no a priori knowledge of the relevant distances over which general equilibrium effects might operate, we pre-specified an approach in which we first estimate a series of nested models varying the outer limit R of the spatial bands from 2 km to 20 km in steps of 2 km, and then select the one which minimized the Schwarz Bayesian Information Criterion (BIC). As it turns out, this algorithm selects only the innermost 0–2 km band for almost all outcomes." Egger et al. (2022), page 9 - 104
For more details on how these numbers are derived from Egger et al.’s census estimates, see this section of our Appendix, where we reconstruct Egger et al’s multiplier estimate from the bottom-up.
- 105
More details of how they construct the multiplier from their spillover treatment effects can be found in this section of our Appendix.
- 106
“We estimate a local transfer multiplier of 2.5." Egger et al. (2022), page 1
“The 5th/95th percentiles of timing ranged from 12 to 27 months, and the median survey was conducted 19 months after the experimental start month." Egger et al. (2022), page 7 - 107
“We find that non-recipient households account for 82% of the household contribution to the expenditure multiplier and 85% of the contribution to the income multiplier." Egger et al. (2022), page 29
Table 5 in Egger et al. (2022) suggests that consumption comprises 80% of their overall multiplier.
Combining these two, Egger et al. estimate that consumption gains to non-recipients comprise (0.82*0.8) = 66% of their overall multiplier. - 108
In this case, we think a skeptical null effect prior is consistent with the results other papers that have sought to estimate the spillovers of the Cash for Poverty Relief program imply. So, placing 60% weight on Egger et al. (2022) feels similar to placing 40% weight on these other studies.
- 109
“A large array of baseline characteristics are balanced across treatment and control villages." Egger et al. (2022), page 7
- 110
“We achieved high tracking rates at endline, reaching over 90% of eligible and ineligible households in both treatment and control villages, and these rates do not systematically vary by treatment status." Egger et al. (2022), page 7
- 111
“We filed several pre-analysis plans for this project." Egger et al. (2022), page 6
- 112
The consumption-based approach: Gross domestic product (GDP) = consumption (C) + investment (I) + government expenditure (G) + net exports (NX). Details on how Egger et al. construct their multiplier can be found in this section of the Appendix.
- 113
“Since we also measure household and enterprise income, we can construct a dual income-based measure of local GDP as the sum of factor payments and profits: GDP = W + R + t + Tax − NFIt , where W is the total household wage bill, R are rental expenses of local enterprises (assuming those are paid to capital owners within our study area), t are enterprise profits, and Tax is total enterprise taxes." Egger et al. (2022), page 27
- 114
“Two independent calculations of the local transfer multiplier using consumption data and income data yield estimates of approximately 2.5." Egger et al. (2022), page 36
- 115
“Overall, I trust the main results. I did not find any conceptual problems or coding errors, and my reanalyses suggest different interpretations rather than challenging the main findings." Michael Wiebe, Reanalysis of Egger et al. (2022), 2024, page 1
- 116
“Both patterns suggest a demand- rather than investment-led expansion in economic activity." Egger et al. (2022), page 3
- 117
“One plausible, albeit speculative, possibility is that the utilization of these factors was “slack" in at least some enterprises." Egger et al. (2022), page 4
- 118
“There is a wide range of views about this statistic [the economic multiplier] in the literature. On the one hand, the recent American Recovery and Reinvestment Act (ARRA)—perhaps the largest fiscal stimulus plan in US history—was motivated by a relatively high estimate of the multiplier of 1.6 (Romer and Bernstein 2009). Other studies argue that the multiplier is substantially smaller and potentially close to zero." Nakamura and Steinsson (2014), page 1
- 119
“We asked firms about the amount of (marginal) costs they would incur in order to increase their output by 10%. 40% of rural market firms report that they would incur zero additional costs to expand their output by 10%, while only 3% of firms in Nairobi report the same… Less than 50% of the observed time in rural villages (62% in rural markets) was spent on productive activities, versus 85% of time in Nairobi markets, indicating a substantial amount of idle time." Walker et al. (2024), page 9
- 120
See Table 5 in Egger et al. (2022)
- 121
See Figure 1 in Egger et al. (2022)
- 122
“The expenditure- and income-based measures of GDP we generate are based on unusually rich underlying data, but each has potential limitations. In particular, each may misattribute transactions between agents located in the study area and counterparties located outside." Egger et al. (2022), page 31
- 123
“Intuitively, we might expect net exports to fall following a large external income transfer: since many local firms are retail establishments, imports of intermediate goods (including packaged consumer goods ready for sale) would likely increase." Egger et al. (2022), page 31
- 124
To give an extreme example: if the cash transfers caused no changes to productive activity whatsoever, and recipients spent their money importing goods from other regions, then all that would happen is that goods would be redistributed in Kenya from outside to inside the study area. Egger et al. (2022) would pick up an increase in consumption in their consumption surveys, but this would be misattributed to an increase in economic activity, biasing their multiplier estimates upwards. This is acknowledged by the authors: “This suggests that the expenditure-based approach might be upwardly biased." Egger et al. (2022), page 31
- 125
One thing we find reassuring is that Egger et al. try to bound this bias, using product-level data from their enterprise surveys. In this data, ~20% of durable and nondurable intermediate goods are imported, suggesting that a large share of household consumption in rural areas consists of locally produced food and other basic necessities. They use this to derive a lower bound expenditure multiplier estimate of 2.05.
“As noted in Section 4.2, this conservative methodology yields an upper bound of 20% of local spending that may reflect expenditure on imported intermediate goods. If imports scale linearly with expenditure, this suggests a transfer multiplier of at least 2.05 on local expenditure alone." Egger et al. (2022), page 31 - 126
“We find that most consumption is in fact of locally produced goods, in line with the well-known fact that a large share of household consumption in rural areas consists of locally produced food and other basic necessities." Egger et al. (2022), page 20
- 127
This is known as the stable unit treatment value assumption (‘SUTVA’).
- 128
“It is well-known that treatment effects estimated by regressing an outcome on treatment may be biased in the presence of spillovers. This problem is often described as the “missing intercept" problem, as places or individuals without any direct treatment exposure might still experience some treatment effect from a program, and therefore do not represent a “pure" control." Walker et al. (2024), page 30
See this section of the Appendix for a further explanation of the missing intercept problem. - 129
“Notably, the ambient effects appear to be more meaningful in magnitude for inflation than for income." Walker et al. (2024), page 32
- 130
Berk Ozler, a World Bank economist that has worked on cash transfers and spillovers before, also shares this perspective. “Hence, my view is that it should rightly take its place among them and be recognized as important, rather than displacing everything that came before it." Berk Ozler, Who benefits from the indirect effects of cash transfer programs?
- 131
See Appendix Figure B4 in Egger et al. (2022) for a decomposition of price impacts by product.
- 132
Eggs are perishable, difficult to transport over long distances, and hard to costlessly and quickly increase the supply of, because you need to breed more hens. If supply the supply of eggs is relatively inelastic in the short-run, we’d expect to see price increases following an increase in demand; the fact that Egger et al. don’t observe this feels surprising to us
- 133
Empirically, a recent paper (Filmer et al., 2023) found that cash transfers led to localized price increases concentrated in perishable products like fish and eggs. This paper is discussed more below
“Cash transfers led to a sustained increase in the prices of perishable foods in some markets." Filmer et al. (2023), page 2 - 134
“Approximately one-third of all households were eligible." Egger et al. (2022), page 5
- 135
“Our current operations in Kenya are based in Kilifi and Baringo counties, which are relatively poorer than Siaya – we haven’t enrolled new recipients in Siaya since the study launch in ~2017." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 136
“In the model, input indivisibilities take the form of “integer constraints." meaning input amounts are constrained to come from the set of integers – i.e., firms can only hire in increments of 1.0 full labor-units." Walker et al. (2024), page 3
- 137
“One plausible, albeit speculative, possibility is that the utilization of these factors was “slack" in at least some enterprises (Lewis (1954)). This seems plausible because in the retail and manufacturing sectors, where output responses were concentrated, the typical firm has a single employee (i.e., the proprietor), suggesting that integer constraints may often bind." Egger et al. (2022), page 4
- 138
“With regional targeting: multipliers are larger in rural, remote, high-slack regions compared to urban, high-utilization regions." Walker et al. (2024), page 5
- 139
“In the rural study setting, high transportation costs exacerbate slack as remote markets face very low levels of effective demand. Urban markets, by contrast, are able to coordinate demand because they are often relatively easy to reach by a large share of the population." Walker et al. (2024), page 35
- 140
“A large reservoir of available labor in agriculture reallocating to the non-tradable sector helps mute inflationary pressures, maintaining the effectiveness of demand-side interventions… Crucially, however, this mechanism is not unbounded. As firms begin hitting capacity constraints, inflation sets in." Walker et al. (2024), page 4
- 141
“The model only zooms in on a single mediator of the relationship between the concentration of the program and the real multiplier – slack. While the authors argue that this explains more than half of their experimental result, it also leaves a sizable fraction unexplained." Walker et al. (2024), page 28
- 142
“The model estimates nominal and real multipliers of 1.8 and 1.5, respectively… The figure also includes the empirical multiplier estimate of 2.5 and associated 90% confidence interval from Egger et al. While the model prediction is substantially lower – albeit within the confidence interval of the empirical estimate – the model includes only one friction." Walker et al. (2024), page 28
- 143
In theory, inflation could be higher and this need not erode the real multiplier, provided the elasticity of inflation with respect to spending remains constant. For example, suppose the direct effect of a cash injection is a 100% increase in spending, and the elasticity of inflation with respect to the size of cash injection is constant at 0.1. If 50% of households are treated, that's a 50% increase in local GDP/ spending, and this leads to 5% inflation. The real effects are 90% as large as the nominal effects. If 100% households are treated, then you'd get 10% inflation, but real effects are still 90% of nominal effects.
This is illustrated in the Figure above from Walker et al. (2024) – where inflation increases from 1.5% to 7.2% as you move along the x-axis, but the real multiplier remains relatively constant. - 144
“The market value of the food transfer was about 200 pesos (20 US dollars) per household per month." Cunha et al. (2019), page 2
- 145
“Villages are eligible to receive PAL if they have fewer than 2,500 inhabitants, are highly marginalized as classified by the Census Bureau, and do not receive aid from either Liconsa, the Mexican subsidized milk programme, or Oportunidades, the conditional cash transfer programme… Households
within programme villages are eligible to receive transfers if they are classified as poor by the national government." Cunha et al. (2019), page 8 - 146
“We find no detectable increase in prices under cash transfers (though the point estimate
suggests a small increase)." Cunha et al. (2019), page 3 - 147
“The exception is the less developed villages in our sample, as proxied by low average income,
small population, and physical remoteness." Cunha et al. (2019), page 3 - 148
“For the less developed villages, in-kind transfers cause prices of the transferred goods to fall
by 5% relative to cash transfers. In addition, cash transfers lead to a 1.5% increase in overall food
prices." Cunha et al. (2019), page 3 - 149
“The vast majority of households in the villages, 89% on average, were eligible for
the programme." Cunha et al. (2019), page 2. We’d expect this to be closer to 100% in the most deprived villages. - 150
“Eligible households receive a combination of health and education grants every two months, ranging from ₱500 to ₱1,400 (approximately US$11 to US$32) per household per month, depending on their number of eligible children and compliance with program conditions." Filmer et al. (2023), page 12
- 151
“ As the program is targeted at the household level, eligibility is set through a Proxy Means Test that estimates household resources on the basis of a limited number of observed characteristics." Filmer et al. (2023), footnote of page 3
- 152
“This increase in village income raised the local prices of protein-rich perishable foods, such as eggs and fresh fish, by 6% to 8% while leaving the prices of staples unchanged… even at high saturation, the prices of perishable foods change only in more remote village markets." Filmer et al. (2023), page 1
- 153
“Can cash aid harm nonrecipients by raising local prices? We show that a household-targeted cash transfer in the Philippines increases the prices of perishable foods in some markets and raises stunting among nonbeneficiary children by 11 percentage points (34%)." Filmer et al. (2023), page 1
- 154
- Eeshani Kandpal told us that the inflationary response of cash transfers could be mediated by the permanence of the program.
- “I think it [the inflationary response] comes down to these four factors: perishables, place (remoteness), program saturation and permanence." Eeshani Kandpal, Senior Fellow, Center for Global Development, conversation with GiveWell, June 24, 2024 (unpublished).
- 155
In Figure 6 from the Walker et al. (2024) paper, inflation is higher when the cash transfer scales to 60% of GDP, but the real multiplier remains constant because the elasticity of inflation with respect to spending is assumed to remain constant.
- 156
“Leveraging the randomized evaluation of a cash transfer in the Philippines, we show that perishable and therefore less easily traded protein-rich foods become more expensive after program introduction when the proportion of eligible households in the local population is high—which we term high program saturation—especially if the village is remote." Filmer et al. (2023), page 1
- 157
The Egger et al. ongoing RCT in Malawi may provide more direct evidence on this.
- 158
“Households are freely mobile between sectors and can choose between wage employment and starting a business." Walker et al. (2024), page 16
- 159
“Yet a four times larger transfer (again at 60% of baseline GDP as above) without labor reallocation leads to inflation of 24% and a real multiplier of only 1.1." Walker et al. (2024), page 29
- 160
For example, it may not be easy for farmers to relocate to the manufacturing/retail sector if they live out of town, and there are poor road links connecting them to where these businesses are located.
- 161
For example, if a garage wants to hire more workers following an increase in demand, relaying this information to farmers may not be trivial, especially if they are in remote areas and lack internet access.
- 162
For example, weak contract enforcement may cause manufacturing and retail firms to hesitate in bringing on new workers.
- 163
Walker et al. (2024)’s model assumes farmers are profit maximizers – i.e. they move to whichever job/sector they can earn the most in. If people have innate preferences for working on their farms rather than in retail (e.g. if it means they can double-up on childcare), this might inhibit economically efficient reallocation.
- 164
“The starting point is a standard two-sector model with monopolistic competition in non-tradables and only one input factor - labor. The key innovation of the model is the presence of indivisibilities in labor input choice in non-tradable services. In line with our stylized facts and the discussion in section 2.2, one can think of this sector as encompassing all non-agricultural firms in the Kenya study setting, including local services as well as on-demand manufacturing and local retailers." Walker et al. (2024), page 11
Given the authors assume slack in manufacturing and retail and no slack in agriculture, a key question is whether it seems reasonable to assume no slack in agriculture. Intuitively, we find slack much harder to imagine in this sector. For if we imagine a subsistence farmer in Malawi, it’s not clear to us what inputs are being underutilized or could be ‘unlocked’ via cash transfers. When we think about what’s constraining their production, we think it’s more likely to be small plot sizes, poor soil quality and short growing seasons, rather than idle machines and workers caused by a lack of demand. - 165
If the demand shock was sustained, we’d expect workers to eventually relocate to the manufacturing/retail sectors. However, if the demand shock is temporary – as we’d expect from a one-off cash transfer program – timely relocation seems important.
- 166
“This seems plausible because in the retail and manufacturing sectors, where output responses were concentrated." Egger et al. (2022), page 4
- 167
The Gini coefficient measures the inequality among the values of a frequency distribution, such as levels of income. A Gini coefficient of 0 reflects perfect equality, where all income or wealth values are the same, while a Gini coefficient of 1 (or 100%) reflects maximal inequality among values.
- 168
“Indeed, we find no significant reductions in village-level Gini coefficients for consumption expenditure or wealth in treatment villages, and a small and significant (p < 0.05) increase for wealth in control villages (Table B.7)." Egger et al. (2022), page 33
- 169
“For the purpose of this study, to be eligible for transfers, households had to live in homes with thatched roofs, a simple means-test for poverty." Egger et al. (2022), page 5
- 170
“Indeed, we find no significant reductions in village-level Gini coefficients for consumption expenditure or wealth in treatment villages… Overall, the patterns underscore the large spillover gains for non-recipient households: wealthier non-recipients benefit along with recipients, on some dimensions so much that inequality may slightly increase." Egger et al. (2022), page 33
- 171
See Table B8, Egger et al. (2022) Appendix.
- 172
“As I explained in excruciating detail in the blog posts above more than six years ago, HS18 did not collect any data on ineligible HHs. So, the spillovers they estimated were on eligible HHs only, i.e., on less than 10% of the population that looked identical to those who did receive cash… it is clear that the spillovers of large cash transfers to approximately a fifth of all HHs in T accrued almost entirely to ineligible HHs in nearby areas [in Egger et al.]." Berk Ozler, Who benefits from the indirect effects of cash transfer programs?
- 173
“While top quintile households have the largest effect, there is not a clear trend with richer households benefiting more." Michael Wiebe, Reanalysis of Egger et al. (2022), 2024, page 1.
- 174
This paper finds evidence of concentrated inflationary impacts in high saturation and remote areas.
“In high saturation areas, particularly if remote, either geographic or community-based targeting or complementary programs like in-kind transfers may mitigate such negative impacts." Filmer et al. (2023), page 3 - 175
This paper finds negative consumption spillovers on eligible non-recipients in treated villages.
“We do find some spillover effects. Households impacted by spillovers have lower consumption and food security than pure control households, perhaps due to the sale of productive assets." Haushofer and Shapiro (2018), page 1 - 176
“Indeed, the support of this distribution is nearly non-negative; only six control villages lose in terms of real income (each losing less than 1% of purchasing power), due to high local price inflation. This highlights how providing fiscal stimulus to slack economies can create nearly universal real income gains, at least over the range of transfers studied in this setting." Walker et al. (2024), page 32
- 177
“And, since now everyone gets money, there's also no-one that will experience just inflation without having the increased purchasing power." Dennis Egger, Associate Professor of Economics, University of Oxford, email to GiveWell, September 17 2024 (unpublished).
- 178
“Of course, the exact weights to put on each study as well as your priors, as well as whether, how much, and in which direction to update, and how to take into account uncertainty is somewhat subjective. While you don't fully update towards our findings and generally err on the side of being conservative, I think you argue convincingly why you do so." Dennis Egger, Associate Professor of Economics, University of Oxford, email to GiveWell, September 30 2024 (unpublished).
- 179
“We follow your arguments here but would like to mention that it is not entirely obvious to me that spillovers would be smaller / inflation larger in other contexts. For instance, Kenya has a very well-developed mobile loans system, and therefore relatively few credit constraints, which might have dampened the overall positive supply/investment response relative to other places." Dennis Egger, Associate Professor of Economics, University of Oxford, email to GiveWell, September 30 2024 (unpublished).
- 180
“My sense is that many of these small farms require very small startup costs, most households already do these kind of activities for some part of the year. So in most settings where GD operates, I'd expect the reallocation to be similarly easy as in Kenya. In fact, if at least some people use their cash to start non-ag businesses, this may be even more smooth." Dennis Egger, Associate Professor of Economics, University of Oxford, email to GiveWell, October 8 2024 (unpublished).
- 181
“I think that your IV (65% weight to Egger) and your distributional adjustment (20% downweight for spillovers accruing almost exclusively to ineligibles) are slightly generous. I don’t like doing this as it wades into subjective territory without an exact welfare function, but it feels wrong that a new paper should be twice as important as everything else that came before it (IV) and that we care that much about the spillovers to ineligibles. I’d lower and increase these figures, respectively, but I can’t tell you exactly how much." Berk Ozler, Lead Economist, World Bank, email to GiveWell, October 10 2024 (unpublished).
Note: we tightened this adjustment to 60% after receiving this email. - 182
- “So while I wouldn’t personally perceive the multiplier to be a first-order benefit of CTs (the key argument there is of course the huge benefit for the direct recipient), I am now more of the mindset that the relatively muted community spillover effects are in the positive direction." Craig McIntosh, School of Global Policy and Strategy, University of California, San Diego, email to GiveWell, August 19 2024 (unpublished).
- “I agree with Craig that we should update our priors, period, and especially for sufficiently large cash infusions into economies. He is also very right that the spillovers are a bit of a red herring – the main issue is the benefits to the beneficiaries." Berk Ozler, Lead Economist, World Bank, email to GiveWell, August 28 2024 (unpublished).
- 183
“I took a close look at the entire document you shared and I don’t think you are too conservative or too bullish. I think the pushback you’re getting from opposite ends on this is a good sign because it means you’re squarely in the middle. This said, I wonder if you could do more to stress the underlying heterogeneity in arriving at an “average" estimated spillover." Eeshani Kandpal, Senior Fellow, Center for Global Development, email to GiveWell, October 9 2024 (unpublished).
- 184
This evidence is summarized in a recent World Bank systematic review, Gassmann et al., 2023.
- 185
The study is not well-placed to study these effects because they don’t have super-village level variation in the intensity of the cash transfer program. Dennis Egger, lead author of the study, Associate Professor of Economics, University of Oxford, conversation with GiveWell, 2024 (unpublished).
- 186
This includes the $1,000 nominal transfer plus a ~20% overhead. See our overhead estimates here.
- 187
We’ve not yet seen these results.
- 188
“There is a substantial decrease in child mortality of almost 1 percentage point (or 70% off of the baseline value). To contextualize these effects using unweighted numbers, the control group eligibles saw thirteen cases of child mortality out of 2,596 children (0.5%), while the GD Large arm saw two cases out of 1,200 children (0.16%)." McIntosh and Zeitlin (2024), page 15
- 189
This 8-9 year follow-up (‘endline 3’) sits alongside consumption data collected at 5-7 years (‘endline 2’)
- 190
Preliminary results shared with GiveWell (unpublished).
- 191
Preliminary results shared with GiveWell (unpublished).
- 192
“Here we evaluated the effects of large-scale, government-led cash transfer programmes on all-cause adult and child mortality using individual-level longitudinal mortality datasets from many low- and middle-income countries." Richterman et al. (2023), page 1
- 193
One exception is the PROGRESA program in Mexico, which had randomized roll-out.
- 194
Richterman et al. (2023), Figure 2
- 195
“We also found that programmes with higher coverage and larger cash transfer amounts were associated with the largest reductions in mortality, with these types of programmes being associated with significant reductions among women (ARR 0.70, 95% confidence interval 0.62–0.79), men (ARR 0.77, 95% confidence interval 0.71–0.84), children aged less than 5 years (ARR 0.86, 95% confidence interval 0.81–0.93) and children aged 10–17 years (ARR 0.80, 95% confidence interval 0.65–0.97), but not for children aged 5–9 years (ARR 0.94, 95% confidence interval 0.83–1.07)." Richterman et al. (2023), page 4
- 196
- “The analyses focus on households that were present at baseline (i.e., the start of the program), and that remains present at endline 3. These constitute over 90% of all baseline households." Preliminary results shared with GiveWell (unpublished).
- “Data collection from a new round of household censuses – the endline 3 household census – was completed in mid-November 2023, with nearly 65,000 households censused. Over 92% of resident households have completed the survey (including the birth history and child mortality modules), and reassuringly, these rates are balanced across treatment and control villages." Preliminary results shared with GiveWell (unpublished).
- 197
- “An estimate of the number of under-5 deaths averted is: (The estimated treatment on under-5 deaths) x (The number of births among recipient treatment households during 2015-2017), which is (-26.66 deaths/1000 births) x (3,397 births in treatment villages) ≈ 91 deaths." Preliminary results shared with GiveWell (unpublished).
- “For U5MR, we estimate a decline of 26.7 deaths per 1000 births during 2015-17, representing a substantial 46% fall in U5MR relative to the control mean." Preliminary results shared with GiveWell (unpublished).
- To back out the total number of deaths, we divide the total number of deaths averted by the treatment effect: 91/0.46 = ~200.
- 198
The cash transfer programs studied are summarized here.
- 199
In this instance, we’d be worried that cash transfers might be rolled out first to poorer regions that are also receiving other forms of government support (e.g. housing subsidies; healthcare investment), and that mortality effects from these other forms of support might be misattributed to causal effect of cash transfers.
- 200
For example:
- One way cash transfers could have lowered child mortality is if households used the cash to make durable health investments (e.g. purchasing new toilets, better roofs, or concrete floors). However, Aggarwal et al. (2024) find no changes in health investment behavior from their RCT of GiveDirectly lump sum transfers in Malawi and Liberia. None of the other RCTs we found reported effects on this outcome. Also, if health investments were an important channel, we’d expect to pick up mortality effects in younger siblings, but Egger et al. find no evidence of the under 5 mortality reduction enduring beyond the period that cash was disbursed
- Another way cash transfers could have lowered child mortality is by increasing households expenditure on food or medicine. If increased expenditure were an important mechanism, we’d have expected to see mortality reductions in nearby non-recipient households, who Egger et al. (2022) separately estimate to have experienced similar increases in expenditure due to positive spillovers. However, Egger et al. find no evidence of mortality spillovers on nearby non-recipients
- Cash transfers could have led to a mortality reduction if they led to a change in health-seeking behavior amongst recipient households. We find it hard to imagine why cash transfers would have led to large behavior changes (needed to rationalize the large all-cause mortality reduction), given the cash transfers weren’t conditional on any behaviors. McIntosh and Zeitlin (2024) also find that large cash transfers led to no significant changes in sanitation practices or health knowledge in their RCT of the GiveDirectly program in Rwanda. None of the other RCTs of GiveDirectly lump sum transfers that we found measured health practice or health knowledge outcomes
- 201
This may not be an accurate assumption if e.g. rural households skew older, as young people migrate to the cities. We haven’t looked into this as we don’t think it’s consequential to our bottom-line.
- 202
Preliminary results shared with GiveWell (unpublished).
- 203
All-cause under 5 mortality estimates from UNICEF are downloaded here.
- 204
For example, in Kenya, the Institute for Health Metrics and Evaluation estimate that under 5s are twice as likely to die from any cause in a given year compared to 20-54 year olds. Intuitively, mortality risk is higher in early life because human bodies and immune systems are fragile and take time to develop.
- 205
See our moral weights write-up for an explanation for why we value lives saved in this way.
- 206
In contrast to the Cash for Poverty Relief program, the programs implemented by our Top Charities (e.g. seasonal malaria chemoprevention) are much cheaper (~$6 per child per year), and specifically targeted at the under 5 population, and so can generate a much lower cost-per-life-saved.
Based on our current assumptions, our cost per life saved estimates of the Cash for Poverty Relief program range from ~$65,000 to ~$125,000. By contrast, our cost per life saved estimates from Malaria Consortium’s seasonal malaria chemoprevention program range from ~$2,200 to ~$8,800. - 207
For example, Egger et al. (2022) find insignificant and close to 0 effects on the physical health outcomes of recipients, and Haushofer and Shapiro (2016) also find insignificant and close to 0 effects on a physical health index. Physical health outcomes for adults aren’t measured in other RCTs of the GiveDirectly program (e.g. McIntosh and Zeitlin, Aggarwal et al.).
- 208
While evidence of the impact of the Poverty Relief program on the physical health of adults appears thin, there is much stronger evidence on its impact on mental health. For example, Egger et al. (2022) find positive and significant effects on the psychological well-being of cash transfer recipients, Haushofer and Shapiro (2016) find positive and significant effects on their psychological wellbeing index, and Banerjee et al. (2023) finds that lump-sum cash (as well as continuous monthly transfers) led to significant improvements in mental health. McGuire et al. (2022) conduct a systematic review of the subjective well-being and mental health impacts of cash transfers in LMICs, and find that cash transfers have a small but statistically significant positive effect on both subjective well-being and mental health.
- 209
These surveys are discussed in this GiveWell blog post.
- 210
There is some evidence that GiveDirectly’s Poverty Relief program leads to an improvement in school attendance. For example, Egger et al. (2022) find that cash transfers led to a statistically significant improvement in their education index (which includes attendance), while Aggarwal et al. (2024) find that cash transfers improved school attendance in Malawi.
An outstanding question we have is whether this improvement in school attendance is likely to improve children’s cognitive skills – which we think of as a more ‘downstream’ outcome more relevant for labor market returns, and hence more important in determining future consumption. None of the GiveDirectly RCTs we've seen look at children's cognitive skills as an outcome. In a review article of cash transfers more generally, Bastagli et al. (2016) state that the evidence of the impact of cash on school attendance is stronger than its impact on cognitive outcomes. One reason for this may be because poor quality schooling means increased attendance doesn't proportionately translate to learning gains.
Evidence on the impact of the Poverty Relief program on children’s anthropometric outcomes (e.g. height and weight) appears mixed. On the one hand, McIntosh and Zeitlin (2024) find that large transfers reduce stunting and wasting by 6 and 5 percentage points (significant at the 10% level). On the other hand, Banerjee et al. (2023) detect no changes in child anthropometrics in the lump-sum arm of their trial. - 211
GiveWell, Malaria income effect size, 2023.
- 212
GiveWell, Malaria income effects benchmarking, 2023.
- 213
Based on data GiveDirectly has sent us, downloaded and analyzed here.
- 214
Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 215
See line 159 of our seasonal malaria chemoprevention CEA.
- 216
See line 119 of our vitamin A supplementation CEA.
- 217
“Our investigation concluded that just under $1.2M was illegally diverted from our D.R.C. operations in 2022." Give Directly, Fraud in D.R.C. – our apology and response, 2023.
- 218
“Including other cases of loss, this means ~1.1% of the money we delivered last year was lost to fraud, our highest amount to date." Give Directly, Fraud in D.R.C. – our apology and response, 2023.
- 219
“We find that increases in neighbors’ wealth strongly decrease life satisfaction and moderately decrease consumption and asset holdings. The decrease in life satisfaction induced by transfers to neighbors more than offsets the direct positive effect of transfers, and is largest for individuals who did not receive a direct transfer themselves." Haushofer et al. (2015), page 1
- 220
We’d generally expect information to travel further than consumption spillovers.
- 221
For example, Egger et al. (2022) finds that recipients work slightly more following the receipt of transfers (though this isn’t statistically significant), and Haushofer and Shapiro (2016) and Banerjee et al. (2023) find null effects on the labor supply of recipients. One exception is Aggarwal et al. (2024), who find suggestive evidence of a decrease in labor supply in Liberia. This stands in contrast to the findings of Banerjee et al. (2017), who analyze data from 7 (smaller) government-led cash transfer programs in 6 low-income countries, and find no systematic evidence that cash transfer programs discourage work.
- 222
To purchase Kenyan goods, other countries would have to first purchase Kenyan shillings. With an appreciated exchange rate, they’d get fewer shillings per dollar, so the goods would be relatively more expensive.
- 223
“It is sometimes claimed that an increase in aid might cause Dutch Disease—that is, an appreciation of the real exchange rate which can slow the growth of a country’s exports— and that aid increases might thereby harm a country’s long-term growth prospects." Barder (2006), page 1
- 224
Based on data GiveDirectly has sent us, downloaded and analyzed here.
- 225
World Bank estimate.
- 226
“A recent model of Ethiopia suggests that, with no productivity benefits from aid, if aid doubled from 20% of GDP to 40% of GDP, exports might fall by 6% of GDP over ten years." Barder (2006), page 17
- 227
See line 87 of our previous GiveDirectly CEA.
- 228
“Correct that “Poverty relief" [Cash for Poverty Relief] donations aren’t earmarked to specific countries, but we will do this at donors’ request if they email us, include it in a grant agreement." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 229
This data was sent to us by GiveDirectly, and is uploaded here.
- 230
“Marginal donations today are unlikely to be allocated to Mozambique at significant scale because current programming is largely institutionally funded and already restricted to Mozambique, though this may change." Kayla Fishman, Senior Growth Manager, GiveDirectly, comments on a draft of this page, October 2, 2024 (unpublished).
- 231
“Qualitative methods were used, including in-depth interviews with 30 women: 10 receiving $1,000, 10 receiving $500, and 10 not receiving a UCT." Junior et al. (2016), page 1
- 232
“I would rather prefer a cash transfer. ... For instance, if somebody were to give me seeds, and apparently my need at that point was not necessarily seeds, for instance, if my baby was sick ... I might not be able to sell the seeds to make money to take my baby to the hospital." Junior et al. (2016), page 16
- 233
“Across both recipient groups, however, there was a sizable minority of interviewees who preferred education or livelihood support, such as a job or assistance with farming or starting a business. An important reason for such preferences was the desire for a more sustainable income source than that which UCTs could provide." Junior et al. (2016), page 16
- 234
“In particular, providing dilute chlorine solution free at the point of water collection, together with a local promoter, can increase take-up of water treatment from less than 10% to approximately 60%." Ahuja et al. (2010), page 3
- 235
“We find that uptake of insecticide treated bednets falls by 60 percentage points when the price increased from zero to $0.60." Cohen and Dupas (2010), page 1
- 236
A brief review of private sector commodity availability in LMICs can be found here. Coverage of key health commodities seems patchy – for example, only 5-9% of pharmacies stocked insecticide-treated bed nets, and availability of amodiaquine (one of the antimalarials used in seasonal malaria chemoprevention campaigns) ranged from 4% to 95%.
- 237
Insecticide treated bed nets and seasonal malaria chemoprevention predominantly target malaria; New Incentives supports childhood vaccinations which target infectious diseases; vitamin A supplementation is associated with a reduction in diarrhea and measles, both of which are infectious diseases.
- 238
See this data, and our analysis, here.
- 239
3/5 of the countries (Kenya, Rwanda, and Uganda) report consumption in terms of adult equivalents rather than per capita. Adult equivalent metrics scale absolute consumption estimates by different requirements of household members at different ages. For example, children are assumed to require less resources, so ‘count’ less to the overall household size. Kenya and Rwanda both state explicitly the equivalence scale they use, which I use to back-out per capita estimates. Uganda doesn’t report its adult equivalence scale; for simplicity, I assume Uganda is using the same scale as Rwanda.
- 240
This is generally lower than the GDP growth rates in some of these countries (e.g. Kenyan GDP growth was 5% in 2022). In general, we’d expect consumption in the lowest quintile to grow more slowly than aggregate GDP, because we’d expect gains from economic growth in low-income countries to be disproportionately captured by richer households (e.g. city dwellers, factory workers). This is based on Simon Kuznets’ inverted U-shaped hypothesis on the relationship between economic growth and inequality.
- 241
The LCU conversion factor is defined as the number of units of a country's currency required to buy the same amounts of goods and services in the domestic market as a U.S. dollar would buy in the United States. It is used to convert currencies into purchasing power parity terms.
- 242
The Mozambique estimates in particular seem implausibly low: they imply that someone in the bottom quintile of Mozambique’s consumption distribution lives off of $0.33PPP a day, and is 3x as poor as an equivalent person in Malawi. We think it’s possible we’re making a mistake in our interpretation of the official estimates.
- 243
For example, in both the Rwanda and Uganda RCTs, the authors do not specify if the estimates are in PPP or nominal terms (we’re assuming nominal, given how low the estimates seem).
- 244
We’d expect for-profit mobile phone companies to be more likely to build masts in richer and more densely populated areas, as there’s probably more customers to gain there.
- 245
At a very high level, we’d expect people living in rural areas to be poorer than those living in urban areas. Zooming in, we’d expect poverty to be concentrated in certain regions and villages – e.g. those located far away from cities and without good market access.
- 246
“Cost of the least expensive starchy staple for energy balance for a representative person at 2,330 kcal/day." See Our World in Data, “Daily cost of a calorie sufficient diet." Underlying data taken from the World Bank.
- 247
“By comparing the cost of diets with income distributions across the world, researchers estimated that 1.1 billion people could not afford the most basic energy-sufficient diet in 2021." Three billion people cannot afford a healthy diet," 2021.
- 248
“Combining estimated import shares with our preferred MPC estimate yields a marginal
propensity to consume on local value added of approximately 0.76 in this context." Egger et al. (2022), page 21 - 249
We downloaded the raw data for this paper from here. Our own analysis code is uploaded here.
- 250
Our guess is that this is because thatched roof ownership is a fairly blunt method of means-testing.
- 251
This is reassuring, as a certain type of asset ownership (thatched roof) was used to determine eligibility.
- 252
“We carried out a two-stage randomization, one at the village level, resulting in treatment and control villages, and another at the household level, resulting in “treatment" and “spillover" households in treatment villages, and “pure control" households in control villages." Haushofer and Shapiro (2018), page 2
- 253
“Here, the sample includes only non-treatment households (in treatment and control villages). Thus, β1 identifies within-village spillover effects by comparing control households in treatment villages to control households in pure control villages." Haushofer and Shapiro (2018), page 24
- 254
This is known as the stable unit treatment value assumption, or ‘SUTVA’.
- 255
“First, we generated substantial spatial variation in the intensity of transfers by deliberately randomizing the allocation of cash transfers not just across households or villages (as is typical), but also across geographic sublocations (groups of 10–15 villages)." Egger et al. (2022), page 2
- 256
“The novel terms here are the amount Amtv of cash per capita transferred to one’s own village v over the entire study, and the amount Amt¬v vr of cash per capita transferred to villages other than v in a series of bands with inner radius r − 2 km and outer radius r km around the village centroid." Egger et al. (2022), page 8
- 257
“Because we had no a priori knowledge of the relevant distances over which general equilibrium effects might operate, we pre-specified an approach in which we first estimate a series of nested models varying the outer limit R of the spatial bands from 2 km to 20 km in steps of 2 km, and then select the one which minimized the Schwarz Bayesian Information Criterion (BIC)." Egger et al. (2022), page 9
- 258
“As it turns out, this algorithm selects only the innermost 0–2 km band for almost all outcomes." Egger et al. (2022), page 9
- 259
The assumption of a linear relationship is the result of the functional form they assume in their main regression equations (equations 2 and 3).
- 260
“Both patterns suggest a demand- rather than investment-led expansion in economic activity." Egger et al. (2022), page 3
- 261
The concept of economic multipliers dates back to (at least) John Maynard Keynes.
- 262
“One plausible, albeit speculative, possibility is that the utilization of these factors was “slack" in at least some enterprises (Lewis (1954))." Egger et al. (2022), page 4
- 263
Egger et al. (2022) refer to this as an ‘integer constraint’ – where the ubiquity of one-person businesses’ means both the worker (and her machines) spend a lot of time being underutilized.
- 264
“If increasing utilization is possible without requiring additional factors of production, slack implies highly elastic aggregate supply curves at least over some range." Walker et al. (2024), page 1
- 265
“We define this multiplier M as the cumulative effect of transfers on local real GDP, relative to the total amount T transferred in real terms, over a given time interval." Egger et al. (2022), page 26
- 266
“Following national accounts definitions, the expenditure-based measure of local GDP is GDP = C + I + G + NX." Egger et al. (2022), page 26
- 267
C = consumption; I = investment; G = government expenditure; NX = net exports (exports – imports)
- 268
“We exclude local government expenditure, G, as Walker (2018) showed that the intervention had a precisely estimated null effect on it." Egger et al. (2022), page 27
- 269
“In the expenditure case, the main concern is that we do not directly observe net exports (NXt)." Egger et al. (2022), page 31
- 270
“To estimate the multiplier, we extend the cross-sectional analysis by estimating and
then integrating effects on components of GDP over time… For a flow variable x (e.g., consumption, investment, etc.), we first estimate the following specification… where Amtv(t−s) is the amount transferred to village v in quarter t − s." Egger et al. (2022), page 14 - 271
“A limitation of our approach is that we observe partial data in the months immediately following
the transfers, which reduces the precision of some estimates." Egger et al. (2022), page 4 - 272
“We then aggregate the quarterly estimates across all villages (using inverse population weights from our household and enterprise census) to compute the study area-wide IRF for each flow component." Egger et al. (2022), page 15
- 273
“ We designed and carried out a large-scale experiment in rural Kenya that provided one-time cash transfers worth roughly USD 1000 (distributed by the NGO GiveDirectly) to over 10,500 poor households." Egger et al. (2022), page 2
- 274
“Our household and enterprise censuses of the study area count 65,383 households." Egger et al. (2022), page 2
“We find that non-recipient households account for 82% of the household contribution to the expenditure multiplier and 85% of the contribution to the income multiplier, both of which are somewhat higher than their share in the local population of 67%." Egger et al. (2022), page 29
0.67*65k = ~45k - 275
65k - ~10k - ~45k = ~10k
- 276
Egger et al. (2022), Table 5, page 2630
- 277
“We find that non-recipient households account for 82% of the household contribution to the expenditure multiplier and 85% of the contribution to the income multiplier." Egger et al. (2022), page 29
- 278
- “We designed and carried out a large-scale experiment in rural Kenya that provided one-time cash transfers worth roughly USD 1000 (distributed by the NGO GiveDirectly) to over 10,500 poor households." Egger et al. (2022), page 2
- “These households received a series of three transfers totaling KES 87,000, or USD 1871 PPP (USD 1000 nominal)." Egger et al. (2022), page 5
- 279
“The increase in GDP is fairly stable over time; in fact, we cannot reject that the expenditure response is constant across all quarters (p-value of 0.53)." Egger et al. (2022), page 29
- 280
“Our household and enterprise censuses of the study area count 65,383 households." Egger et al. (2022), page 2
“We find that non-recipient households account for 82% of the household contribution to the expenditure multiplier and 85% of the contribution to the income multiplier, both of which are somewhat higher than their share in the local population of 67%." Egger et al. (2022), page 29
0.67*65k = ~45k - 281
Egger et al. (2022), Table 5, page 2630
- 282
“We find that non-recipient households account for 82% of the household contribution to the expenditure multiplier and 85% of the contribution to the income multiplier." Egger et al. (2022), page 29
- 283
“Our current operations in Kenya are based in Kilifi and Baringo counties, which are relatively poorer than Siaya – we haven’t enrolled new recipients in Siaya since the study launch in ~2017." GiveDirectly, GiveWell Questions for GiveDirectly, 2024 (unpublished).
- 284
Kenya National Bureau of Statistics, The Kenya Poverty Report, 2023, page 40, Table 4.4.
- 285
“The population in Siaya is predominantly Luo, the second largest ethnic group in Kenya, and while rural, it is also relatively densely populated, with 393 people per km2 compared to a Kenyan average of 88." Egger et al. (2022), page 5
- 286
“The average village had 0.7 markets located within 2 km and 2.3 markets within 4 km, again indicating the rather high density of settlement. Household respondents report an average commuting time to their preferred market of 31 minutes, where over 80% walk to the market." Egger et al. (2022), page 12
- 287
“The main national road running from the port of Mombasa to Nairobi and on to Kampala passes through the study area, likely helping to integrate it into the regional economy." Egger et al. (2022), page 5
- 288
None of the macro or meso-level papers employ experimentally-induced randomization except Egger et al. (2022). Many take the form of simulation-based predictions based on creating a social accounting matrix to capture transactions between agents (‘LEWIE’ models). An external expert we consulted (Rossa O’Keeffe O'Donovan) also said: “I briefly looked at these papers a few years ago and think their methods are much less rigorous (and empirically grounded) than Egger et al. So I wouldn't weight them that much." Rossa O’Keeffe-O’Donovan, Lead Researcher, OpenPhilanthropy, comments on a draft of this page (unpublished).
- 289
“As analysts, we would love to compare the findings, but a meta-analysis of the results was not possible. While there is an established methodology for evaluating the impacts of cash transfers on recipients, the methods used to measure multiplier effects are manifold as this review has shown. Not only do methods differ, it is also interesting to note that multipliers are uniquely conceptualized given the country context and focus of analysis. There is no shared definition of the main intended outcomes or the unit of measurement." Gassmann et al. (2023), page 28
- 290
“The median survey was conducted 19 months after the experimental start month." Egger et al. (2022), page 7
- 291
“These households received a series of three transfers totaling KES 87,000, or USD 1871 PPP (USD 1000 nominal)." Egger et al. (2022), page 5
- 292
“These estimates are broadly in line with what a simple model would imply from households’ marginal propensity to consume local value added, which we estimate to be approximately 0.76 over the study period." Egger et al. (2022), page 4
- 293
Both of these assumptions are overly simplistic, but we think they suffice for a basic sense check.
- 294
“For U5MR, we estimate a decline of 26.7 deaths per 1000 births during 2015-17, representing a substantial 46% fall in U5MR relative to the control mean." Preliminary results shared with GiveWell (unpublished).
- 295
See the calculations in this cell.
- 296
We took available consumption and u5 mortality estimates that were closest to the 2015-17 timeframe that the Egger et al. experiment took place. However, in some cases, these were a few years before or after 2015-17.
- 297
Egger et al. imply that the cash transfers had no effect on government spending in the region. “We exclude local government expenditure, Gt, as Walker (2018) showed that the intervention had a precisely estimated null effect on it." Egger et al. (2022), page 27
- 298
In the 2023 Kenya Poverty Relief report, based on survey data from 2021, total expenditure in Siaya county was 5,476 Kenyan shillings (vs. a national average of 7,393). Kenya National Bureau of Statistics, The Kenya Poverty Report, 2023, page 29, Table 3.2
- 299
“We do not find any statistically significant cross-village spillover effects on assets or expenditure; this generally holds both when pooling all households and estimating the effect of being in a high versus low saturation sublocation, and when looking at high versus low sublocations by treatment and eligibility categories." Haushofer et al. (2018), page 18
- 300
“Despite not receiving transfers, non-recipient households also exhibit large consumption expenditure gains: their annualized consumption expenditure is 13% higher eighteen months after transfers began." Egger et al. (2022), page 3
- 301
“The novel terms here are the amount Amt¬v vr of cash per capita transferred to villages other than v in a series of bands with inner radius r − 2 km and outer radius r km around the village centroid… As it turns out, this algorithm selects only the innermost 0–2 km band for almost all outcomes." Egger et al. (2022), page 9
- 302
This point is addressed in Egger et al. (2022), where they talk about the limitations of defining treatment status by sublocation. “Overall, we view Equation (1) as a useful benchmark but unlikely to capture the spatial variation in treatment intensity evident in Figure A.2. This is because, in our study area, villages are relatively close to each other; sublocation boundaries are not “hard" in any sense nor reflective of salient ethnic or social divides; and because our data indicate that there is extensive economic interaction in nearby markets regardless of sublocation." Egger et al. (2022), page 8