GiveWell's Change Our Mind Contest

The Change Our Mind Contest concluded on October 31, 2022, and we announced winners here on December 15, 2022. Thank you to everyone who participated!

GiveWell is launching a contest for critiques of our cost-effectiveness analyses. Excellent entries may lead to changes in how we allocate millions of dollars of funding, contributing to our ultimate goal of recommending programs that save or improve lives the most per dollar.

Cost-effectiveness is the single most important input in our decisions about what programs to recommend. For each of the programs we evaluate, we create a cost-effectiveness analysis, estimating how much good donations to that program will do. Despite the effort we put into these analyses, we believe it's very possible that we're missing important considerations or making mistakes that lead to suboptimal allocation decisions. We make our analyses publicly available because transparency enables people to evaluate our work and conclusions.

We're inviting you to identify potentially important mistakes or weaknesses in our existing cost-effectiveness analyses and tell us about them!

Entries are due by October 31, 2022. See below for details on requirements, what constitutes a strong entry, and how to submit. If you have questions that aren't answered on this page, please email change-our-mind@givewell.org or comment on our blog post.

The first-place winning entry will receive $20,000, the runner-up will receive $10,000, and the honorable mention will receive $5,000. We may offer multiple runner-up and honorable mention prizes if the quality of submissions is particularly high.

All other entries that meet our criteria will receive a participation prize of $500, capped at a total of 50 participation prizes for the first 50 submissions.

We will also publish winning entries online (anonymously, if requested). Winners will be announced by December 15, 2022.

We're running this contest because the recommendation decisions we make are extremely important, and we want to incentivize feedback that will improve our work, thereby enabling us to do more good. We hope you'll consider participating!

Table of Contents

The basics

What can I submit?

Requirement 1: Entries should focus on potential empirical, mathematical, or logical issues in GiveWell's existing cost-effectiveness analyses.

A mix of empirical evidence, interpretation, and intuition is acceptable; we recognize that our cost-effectiveness analyses are in part subjective. An entry that addresses the prompt might be grounded in empirical evidence that the author thinks we should incorporate; it might instead focus on the mathematical or logical approach we take and either identify a clear-cut mistake or suggest a more accurate conceptual approach.

Requirement 2: Entries should relate to at least one of our existing cost-effectiveness analyses.

We encourage authors to focus on one of the following six analyses:1

Each of the above analyses relates to a program to which we have directed substantial funding (generally tens of millions of dollars) and which we intend to consider for substantial future funding. As a result, entries that focus on one of the above programs will likely have a higher chance of meaningfully influencing GiveWell's overall allocation of funding. We expect that most strong entries will focus on one of the above programs.

If you prefer, you may focus on an existing GiveWell cost-effectiveness analysis that is not listed above. Our All Grants and Research on Programs pages are good resources for exploring other programs we've investigated.

Entries may relate to a program in a specific location covered by our cost-effectiveness analysis—for example, net distribution in Democratic Republic of Congo rather than in all countries where AMF works.

Requirement 3: Entries should, if incorporated, have a substantial impact on GiveWell's cost-effectiveness estimates.

A competitive entry will propose a concrete change to our cost-effectiveness analysis with meaningful implications for how we should change the way we allocate funding. Roughly, a critique should imply at least a 10% change to our bottom-line cost-effectiveness estimate for a given program.

You may submit work you've already completed; work that has already been published is eligible only if it was published within the last six months (i.e., since March 1, 2022) and was not shared with GiveWell prior to that date. Work that was submitted to the EA Criticism and Red Teaming contest is eligible as long as it meets the requirements stated on this page; authors are also welcome to submit an edited version of work they submitted to that contest, and might want to do so in order to better tailor their work to the prompt on this page.

What are the prizes?

First-place: $20,000.

Runner-up: $10,000. We may distribute multiple runner-up prizes if the quality of entries is particularly high.

Honorable mention: $5,000. We may distribute multiple honorable mention prizes if the quality of entries is particularly high.

We will distribute participation prizes of $500 to the first 50 entries that (a) meet all requirements above and (b) do not win another prize.

We will announce the winning entries by December 15, 2022.

What's outside the scope of this contest?

In this contest, we are not looking for:

  • Suggestions of programs we haven't investigated that you think we should recommend. See the requirements for more detail.
  • Purely presentational suggestions, such as ways to make our cost-effectiveness analyses easier to understand that would not change the conclusions. Presentational suggestions that could change the output of our model are acceptable.
  • Purely subjective arguments, such as a proposed change to a given figure that is rooted solely in intuition rather than empirical evidence. A mix of empirical evidence, interpretation, and intuition is acceptable.

Feedback outside the scope of this contest is always welcome and can be sent to info@givewell.org, but will not be eligible for one of these prizes.

Depending on the results of this contest, we may run other contests in the future, either with this same prompt or with other prompts.

What makes an entry strong?

A strong entry will have the following characteristics:

  1. Clarity: Clearly present the author's reasoning and conclusion, and practice reasoning transparency. Formal language isn't necessary as long as the message is clear. For entries that are more than a couple pages long, please include a one-page summary.
  2. Accuracy: Demonstrate sufficient understanding of the relevant cost-effectiveness analysis to present an accurate critique.
  3. Importance: Raise an issue that could substantially affect our cost-effectiveness estimates. As a general rule, we're looking for an implied change of at least 10% in the cost-effectiveness of a given intervention.
  4. Impact: Pose a good chance of ultimately changing how GiveWell allocates funds, largely due to the combination of the above characteristics. All else equal, entries that impact larger amounts of funding will score more highly than those that impact smaller amounts of funding.

Entries need not be written in "GiveWell" style, nor do you need to understand every detail of our cost-effectiveness analyses in order to enter.

As one example of a strong critique that meets the criteria for this contest, see this post by the Happier Lives Institute, which proposes a different approach to modeling the long-term benefits of deworming programs. A successful entry does not need to follow this format, nor does it need to be as long as that post is; we're sharing this as just one example of the type of issue a successful critique might discuss.

How do I submit?

To submit an entry, please fill out this form. When you fill out the form, you will be asked to either:

  1. share a link to your entry, if it is published online or available via, e.g., a link to a Google Doc, or
  2. upload your entry as a Word document.

Entries should take the form of a Word document, Google Doc, or blog post (e.g., a post on the Effective Altruism Forum). Each entry may include one supporting spreadsheet in Excel or Google Sheets.

The entry form will also ask you to confirm that you are eligible to participate in this contest and confirm that your entry meets the requirements.

All entries must be received by 11:59pm Pacific time on October 31, 2022.

How will the winners be determined?

Each entry will be reviewed by at least one GiveWell researcher who has substantial experience with our cost-effectiveness analyses. They will evaluate the entries based on the criteria above, with particular emphasis on whether the entry poses a critique that should ultimately change how we allocate funding; they will also consider whether the entry stands out in ways we haven't anticipated with the above criteria.

We think excellent entries will demonstrate the type of skills that we look for in GiveWell researchers. We may reach out to the authors of outstanding entries about their interest in applying to work for GiveWell; research recruiting is a major priority for us, and we'd be excited if this contest resulted in us hiring new researchers.

Other questions

Will my entry be public?

We will publish entries that win a first-place, runner-up, or honorable mention prize. Entries will be published anonymously if requested. You are very welcome to publish your own entry, for example on the Effective Altruism Forum, but this isn't required.

How will GiveWell engage with entries?

We're really looking forward to seeing what people submit. We make our research work transparent in part so that people can engage critically with our work, form their own conclusions, and help us improve.

In late 2022 and early 2023, we expect to make decisions about how to allocate a large portion of the overall funds we direct for the year. The timing of this contest will allow us to consider all entries prior to those decisions, so that we can prioritize making improvements that might lead to a more optimal allocation of funds.

We will carefully read and consider all entries. Due to limited research capacity, we likely won't be able to share a full response to all entries; it's time-consuming for us to write thorough responses, and we aren't sure how many entries we will receive. We will share a full, substantive response to the winning and runner-up entries.

In all cases, we're very grateful for people taking the time to engage with our work and share their views; we believe this kind of engagement makes our recommendations stronger and is important even when it doesn't lead to a change in our bottom line.

Who is eligible to participate?

You don't need to have any particular qualifications in order to participate. We're interested in hearing from people who come from all different backgrounds.

In order to be eligible for a prize:

  • You must be at least 18 years old.
  • You must live in the U.S. (excluding Puerto Rico), U.K., or another location in which this contest is not prohibited.
  • You cannot be employed by GiveWell, be the immediate family member of someone employed by GiveWell, or share a household with someone employed by GiveWell.

See the full terms and conditions for more information.

Can multiple people work together?

Multiple people may co-author a single entry, up to a total of ten people. If an entry is co-authored and wins a prize, the team of co-authors will receive a single prize to share among themselves.

Can I submit more than one entry?

Any individual may submit up to three entries (including any co-authored entries); if one individual submits multiple entries, those entries should be meaningfully different in spirit (e.g., not raise the same critique with a slight difference) in order for each to be eligible for a participation prize.

What if I have questions that aren't answered here?

Feel free to email us at change-our-mind@givewell.org or comment on our blog post!

Source URL: https://www.givewell.org/research/change-our-mind-contest