IDinsight — Beneficiary Preferences Survey (2019)

Published: May 2019; Last Updated: December 2019

In December 2019, we published the results from this work. You can read more on this page.

Note: This page summarizes the rationale behind a GiveWell Incubation Grant to IDinsight. IDinsight staff reviewed this page prior to publication.

Background

Some of the most subjective and uncertain inputs into our cost-effectiveness analyses involve the relative value we place on different good outcomes (e.g., how to value averting a death vs. increasing income, or how to value averting deaths at different ages). GiveWell does this by assigning quantitative moral weights to different outcomes in our cost-effectiveness model.

Information about how people living in poverty in low-income countries would make these tradeoffs would be a potentially useful input that could change the relative value we place on these outcomes. However, the information available on this question is limited, as most surveys assessing people's moral values have been conducted in high-income countries. We are aware of very little research on this question in sub-Saharan Africa. We discuss the literature that we are aware of in this report.

About the grant

This grant supports research to assess how potential beneficiaries of GiveWell's top charities value different good outcomes.1 This research aims to inform our approach to two questions:2

1. Valuing health vs. income: For example, how much should we value averting the death of a one-year-old relative to doubling the income of an extremely poor household?
2. Age-weighting: For example, how much should we value averting the death of a one-year-old relative to averting the death of a 30-year-old?

Pilot research

The pilot phase of this research was supported by an April 2018 grant to IDinsight. Over the past year, IDinsight conducted four small pilot studies and tested a variety of methods to solicit the relative moral values of potential beneficiaries.3 We found that many of these methods have significant limitations that make it difficult to obtain reasonable and consistent answers to the above questions. Results from the pilot studies show that responses are highly sensitive to how questions are framed.4

Scaled-up study

The current grant will support a scaled-up study in Kenya and Ghana5 that uses the most promising methods identified in the pilot studies to survey potential beneficiaries about their moral values. Due to the significant methodological limitations mentioned above, the study's results may not be highly reliable.6 Nevertheless, we expect these results to have a reasonable chance of influencing our moral weights and thus contributing to our cost-effectiveness analyses and funding allocation decisions. We also expect them to add to the limited existing literature on the relative moral values of people living in poverty in low-income countries.

Budget

IDinsight expects its total budget of $474,374 to break down roughly as follows:7

• IDinsight staff costs: $364,160
• Fieldwork: $104,214
• Other costs: $6,000

Open questions

• How reliable will this study's results be relative to the results of similar studies conducted in high-income countries?
• To what extent will this study's results influence our approach to valuing health vs. income or age-weighting?
• How can we encourage more research to be conducted on the moral values of people living in poverty in low-income countries?

Internal forecasts

For this grant, we are recording the following forecast:

Confidence	Prediction	By time
35%	At least four GiveWell staff members with inputs in our cost-effectiveness model change their moral weights for either valuing health vs. income or age-weighting by at least 25%, and they attribute that change to this research	End of 2020

Sources

Document	Source
IDinsight, beneficiary preferences field test findings, November 2018	Source
IDinsight, beneficiary preferences field test findings, October 2018	Source
IDinsight, beneficiary preferences high level results summary, 2018	Source
IDinsight, beneficiary preferences methods overview	Source
IDinsight, beneficiary preferences pilot final report, May 18, 2018	Source
IDinsight, beneficiary preferences proposal executive summary, February 22, 2019	Source
IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019	Source
IDinsight, responses to GiveWell questions, February 6, 2019	Source
IDinsight, responses to GiveWell questions, January 11, 2019	Source

• 1

  Note that the populations surveyed were selected to be more or less representative of the beneficiaries of GiveWell's top charities, but are not literally beneficiaries of our top charities.

  "The largest numbers of GiveWell beneficiaries, and active top charities, are in East and West Africa. We balanced the need for demographic diversity, representativeness, and feasibility of data collection to choose Kenya and Ghana for scale-up in 2019." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 5.

• 2

  "At scale, we plan to collect data to inform the two principal components of the GiveWell moral weights...1. The value assigned to averting the death of an individual relative to doubling consumption for one person for one year...2. The value assigned to averting the death of an individual under-5 relative to an individual over-5." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 4.

• 3
  • "IDinsight piloted several methods in Kwale County, Kenya to determine which were most promising for measuring beneficiaries’ preferences in a low-income country context. The goal was to test the appropriateness and accuracy of these options with the following questions in mind...Which method for measuring beneficiary valuations of interventions and outcomes performs best in terms of implementation and respondent understanding?...Are respondents comfortable with and capable of putting monetary values on certain outcomes?...How varied are the responses and do they seem realistic?" IDinsight, beneficiary preferences pilot final report, May 18, 2018, Pg 4.
  • "In 2018, we conducted iterative piloting to test methods that elicit beneficiaries' valuation of life compared to interventions that increase consumption or income. All piloting took place in Kwale county, Kenya. We conducted the following activities:
    1. A large initial pilot (n= 166, 5 weeks, Feb-Mar 2018)
    2. First field test (n=71, 1 week, Aug 2018)
    3. Second field test (n=34, 1 week, Oct 2018)
    4. Third field test (n=152, 2 weeks, Nov 2018)...

    The main output from the piloting is a set of methods that can collect reliable and relevant data for GiveWell's moral weights in a scale-up." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pgs 8-9.

• 4

  For more details on these methods and their limitations, see:

  • IDinsight, beneficiary preferences methods overview
  • IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pgs 9-18.

• 5

  "The largest numbers of GiveWell beneficiaries, and active top charities, are in East and West Africa. We balanced the need for demographic diversity, representativeness, and feasibility of data collection to choose Kenya and Ghana for scale-up in 2019." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 5.

• 6
  • "We have identified three primary methods that directly capture data on beneficiary preferences to inform the intrinsic value of life theme. These methods will be included in a primary questionnaire." IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pg 9.
  • For more details on these methods and their limitations, see IDinsight, beneficiary preferences proposal for scale-up, January 24, 2019, Pgs 9-18.

• 7

  Unpublished, internal version of "IDinsight, beneficiary preferences proposal for scale-up," Pg 22.