Published: May 2019; Last Updated: December 2019
In December 2019, we published the results from this work. You can read more on this page.
Note: This page summarizes the rationale behind a GiveWell Incubation Grant to IDinsight. IDinsight staff reviewed this page prior to publication.
Summary
As part of GiveWell's work to inform the moral weights inputs to our cost-effectiveness analyses, in March of 2019, IDinsight received a GiveWell Incubation Grant of $474,374 to support a scaled-up study surveying potential beneficiaries of our top charities about the relative value they place on different good outcomes, such as averting a death or doubling a household's income. This is a renewal of one project under an April 2018 grant to support the work of IDinsight's "GiveWell embedded team."
IDinsight also previously received Incubation Grants in May 2017, October 2016, and June 2016.
Table of Contents
Background
Some of the most subjective and uncertain inputs into our cost-effectiveness analyses involve the relative value we place on different good outcomes (e.g., how to value averting a death vs. increasing income, or how to value averting deaths at different ages). GiveWell does this by assigning quantitative moral weights to different outcomes in our cost-effectiveness model.
Information about how people living in poverty in low-income countries would make these tradeoffs would be a potentially useful input that could change the relative value we place on these outcomes. However, the information available on this question is limited, as most surveys assessing people's moral values have been conducted in high-income countries. We are aware of very little research on this question in sub-Saharan Africa. We discuss the literature that we are aware of in this report.
About the grant
This grant supports research to assess how potential beneficiaries of GiveWell's top charities value different good outcomes.1 This research aims to inform our approach to two questions:2
- Valuing health vs. income: For example, how much should we value averting the death of a one-year-old relative to doubling the income of an extremely poor household?
- Age-weighting: For example, how much should we value averting the death of a one-year-old relative to averting the death of a 30-year-old?
Pilot research
The pilot phase of this research was supported by an April 2018 grant to IDinsight. Over the past year, IDinsight conducted four small pilot studies and tested a variety of methods to solicit the relative moral values of potential beneficiaries.3 We found that many of these methods have significant limitations that make it difficult to obtain reasonable and consistent answers to the above questions. Results from the pilot studies show that responses are highly sensitive to how questions are framed.4
Scaled-up study
The current grant will support a scaled-up study in Kenya and Ghana5 that uses the most promising methods identified in the pilot studies to survey potential beneficiaries about their moral values. Due to the significant methodological limitations mentioned above, the study's results may not be highly reliable.6 Nevertheless, we expect these results to have a reasonable chance of influencing our moral weights and thus contributing to our cost-effectiveness analyses and funding allocation decisions. We also expect them to add to the limited existing literature on the relative moral values of people living in poverty in low-income countries.
Budget
IDinsight expects its total budget of $474,374 to break down roughly as follows:7
- IDinsight staff costs: $364,160
- Fieldwork: $104,214
- Other costs: $6,000
Open questions
- How reliable will this study's results be relative to the results of similar studies conducted in high-income countries?
- To what extent will this study's results influence our approach to valuing health vs. income or age-weighting?
- How can we encourage more research to be conducted on the moral values of people living in poverty in low-income countries?
Internal forecasts
For this grant, we are recording the following forecast:
| Confidence | Prediction | By time |
|---|---|---|
| 35% | At least four GiveWell staff members with inputs in our cost-effectiveness model change their moral weights for either valuing health vs. income or age-weighting by at least 25%, and they attribute that change to this research | End of 2020 |
Sources
- 1
Note that the populations surveyed were selected to be more or less representative of the beneficiaries of GiveWell's top charities, but are not literally beneficiaries of our top charities.
"The largest numbers of GiveWell beneficiaries, and active top charities, are in East and West Africa. We balanced the need for demographic diversity, representativeness, and feasibility of data collection to choose Kenya and Ghana for scale-up in 2019." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 5.
- 2
"At scale, we plan to collect data to inform the two principal components of the GiveWell moral weights...1. The value assigned to averting the death of an individual relative to doubling consumption for one person for one year...2. The value assigned to averting the death of an individual under-5 relative to an individual over-5." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 4.
- 3
- "IDinsight piloted several methods in Kwale County, Kenya to determine which were most promising for measuring beneficiaries’ preferences in a low-income country context. The goal was to test the appropriateness and accuracy of these options with the following questions in mind...Which method for measuring beneficiary valuations of interventions and outcomes performs best in terms of implementation and respondent understanding?...Are respondents comfortable with and capable of putting monetary values on certain outcomes?...How varied are the responses and do they seem realistic?" IDinsight, beneficiary preferences pilot final report, May 18, 2018, Pg 4.
- "In 2018, we conducted iterative piloting to test methods that elicit beneficiaries' valuation of life compared to interventions that increase consumption or income. All piloting took place in Kwale county, Kenya. We conducted the following activities:
- A large initial pilot (n= 166, 5 weeks, Feb-Mar 2018)
- First field test (n=71, 1 week, Aug 2018)
- Second field test (n=34, 1 week, Oct 2018)
- Third field test (n=152, 2 weeks, Nov 2018)...
The main output from the piloting is a set of methods that can collect reliable and relevant data for GiveWell's moral weights in a scale-up." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pgs 8-9.
- 4
For more details on these methods and their limitations, see:
- 5
"The largest numbers of GiveWell beneficiaries, and active top charities, are in East and West Africa. We balanced the need for demographic diversity, representativeness, and feasibility of data collection to choose Kenya and Ghana for scale-up in 2019." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 5.
- 6
- "We have identified three primary methods that directly capture data on beneficiary preferences to inform the intrinsic value of life theme. These methods will be included in a primary questionnaire." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 9.
- For more details on these methods and their limitations, see IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pgs 9-18.
- 7
Unpublished, internal version of "IDinsight, beneficiary preferences proposal for scale-up," Pg 22.