Clinton Health Access Initiative – CHAI Incubator (August 2022)

• We think that working with CHAI may help GiveWell improve our understanding of programs that have so far not been a large part of our funding recommendations. In partnership with Evidence Action’s Accelerator program, GiveWell has recently started recommending funding to more technical assistance (TA) programming.17 Evidence Action’s strengths in TA delivery and close knowledge of how GiveWell works yielded a productive collaboration that led us to adopt a more flexible modeling approach that could account for this type of upstream work. It also led to a more nuanced understanding of program delivery in general that has benefited GiveWell investigations beyond those with Evidence Action. Furthermore, certain TA interventions we’ve recommended funding to have been very highly cost-effective.18 We would likely not have identified or funded these opportunities without the push from Evidence Action to demonstrate that they could be justifiable within GiveWell’s approach to grantmaking.

  We expect that the Incubator partnership with CHAI may result in similar learnings. While we have some reservations about CHAI’s upstream focus, we expect that we will learn more about not just TA, but also likely how to think about and model the value of market shaping work, a particular strength of CHAI’s.19

• We think it will be beneficial to diversify the set of organizations that we work with to incubate new cost-effective programs. The Evidence Action Accelerator has been successful at identifying and scaling cost-effective funding opportunities that are above our bar.20 We think that diversifying the organizations with which we have this type of partnership will increase the number of opportunities we consider and will also prevent risks that might arise from any single organization constituting a substantial portion of our giving.21

We think CHAI is likely to be a strong partner:

• CHAI recently hired Neil Buddy Shah, former GiveWell Managing Director, as its CEO.22 We see this as a strong signal that CHAI is interested in prioritizing work that has a significant impact via the implementation of cost-effective and evidence-backed programming.

CHAI’s reach and orientation towards rapid execution indicates to us that working with CHAI may be an unusually effective means by which to achieve substantial scale of high-impact programs quickly. CHAI has offices in more than 30 countries.23 We have also heard in conversations with CHAI, as well as individuals with experience working with CHAI that CHAI is oriented toward rapid execution of new programming.24

CHAI has had success scaling large and evidence-based programs.25 Some of CHAI’s work includes reducing the global cost of HIV26 and tuberculosis treatment medications;27 scaling availability of zinc and oral rehydration salts (ORS) for children with diarrhea;28 training home birth attendants on emergency response practices;29 and working with governments to update malaria treatment protocols based on updated World Health Organization recommendations.30
• It’s our impression that the core CHAI team understands our modeling approach and we are able to communicate clearly and transparently about potential opportunities. Our impression in meeting with the core CHAI team was that team members were transparent about organizational strengths and weaknesses and where our organizations are more and less aligned. CHAI team members also engaged thoughtfully with several of our cost-effectiveness models, asking questions that reflected a strong understanding of our methods.31
• We think CHAI technical teams32 and country teams may bring us opportunities that we haven’t yet identified. We are hopeful that given CHAI’s expertise and collaboration between its technical and country teams, the CHAI Incubator team will be able to bring opportunities that we wouldn’t have identified through our typical channels. We saw some initial indication of this in early meetings.33

Risks and reservations

Our primary reservations about this grant are documented below.

• We don't have a longstanding working relationship with CHAI. It’s possible that the team may be less aligned with GiveWell’s values, and as a result it may take more effort to find programs that overlap to meet both organizations’ priorities. In conversations about this grant, we have identified a number of promising ideas to begin investigating,34 and CHAI has also brought ideas that GiveWell has not yet modeled in depth.35 How and whether those succeed in bringing funding opportunities, and whether GiveWell and CHAI are meeting grant goals, will signal our alignment on core values.
• There could be disagreement across CHAI’s leadership structure about scaling GiveWell-aligned ideas. CHAI is a matrixed organization with a leadership structure comprised of both program and country experts.36 There is a risk that even if the CHAI incubator core team advocates for ideas aligned with GiveWell’s approach, there may be some disagreement across CHAI’s wider leadership group, which could result in fewer cost-effective interventions being scaled. This could limit the extent to which CHAI’s very large footprint leads to substantial room for more funding.
• There may not be enough cost-effective interventions for CHAI to scale. The CHAI Incubator core team and GiveWell will both be working to identify large, cost-effective interventions suitable for CHAI. If we are unable to identify enough opportunities that clear our cost-effectiveness bar, then this grant will be less likely to be cost-effective. We think this isn’t very likely given the long list of mutually interesting programs we’ve identified in early discussions.37
• CHAI may not be the right organization for a given intervention of interest. It’s possible that GiveWell and CHAI might disagree on whether CHAI is best suited to implement a particular program. In that case, GiveWell may prefer to bring a program identified and/or scoped by CHAI to another implementing organization. We will try to mitigate this risk by maintaining alignment at a very early stage of investigation around the promisingness of opportunities and their fit for CHAI specifically. CHAI also expressed that if a program were moved to another implementing partner, that would still be considered a success.38

Plans for follow up

Because this is an early partnership, we aim to stay in sync with the CHAI team at regular intervals to discuss the programs that are in their pipeline.

Specifically, we will engage with CHAI via monthly meetings between the CHAI Incubator core team and GiveWell staff, have annual big-picture check-ins on progress towards our goals, and schedule project-specific calls for any interventions that are at or beyond the scoping and design phase (phase four).

Forecasts

We measure success by the identification of cost-effective implementation opportunities, how cost-effective those opportunities are, and the amount of additional funding that those scaled programs could absorb in the future (“room for more funding”) (more).

We think there is a 50% chance that CHAI will have identified at least 5 years of $30 million annual room for more funding at a minimum of 12x cash by 2025, or the equivalent (e.g., higher cost-effectiveness but lower room for more funding, leading to the same total units of value).

Our process

In the course of investigating this grant, GiveWell:

• Spoke with experts who previously worked at CHAI or had experience working with CHAI.39
• Had a series of conversations with CHAI senior leadership and country leaders in countries where CHAI may work as part of this grant.40
• Received a budget, Incubator structure, and notional targets from CHAI.41
• Made a rough model of cost-effectiveness to compare to our bar this year (back-of-the-envelope calculations).
• Had a GiveWell Senior Research Associate and a Program Officer review the case for this grant and provide feedback.

Sources

Document	Source
CHAI, "AIDS was killing millions of people. CHAI was founded to make treatment equitable," 2022	Source (archive)
CHAI, "CHAI Board of Directors announces appointment of Dr. Neil Buddy Shah as Chief Executive Officer," 2022	Source (archive)
CHAI, "How trained midwives are saving lives in rural Zambia," 2022	Source (archive)
CHAI, "Our Programs"	Source (archive)
CHAI, "Our Values"	Source (archive)
CHAI, "Saving lives through scaling up severe malaria treatment in Uganda," 2017	Source (archive)
CHAI, "Where we work"	Source
CHAI, Annual Report 2020	Source (archive)
CHAI, Incubator detailed budget, 2022 (redacted)	Source
CHAI, Incubator narrative, 2022 (redacted)	Source
GiveWell, "Evidence Action — Syphilis Screening and Treatment in Pregnancy," 2021	Source
GiveWell, "Evidence Action Accelerator – Renewal grant for 2022-2025," 2022	Source
GiveWell, CHAI Incubator threshold analysis BOTEC, 2022	Source
GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022	Source
Schroder et al. 2019	Source (archive)
Unitaid, "New patient-friendly tuberculosis preventive treatment to be rolled out in five high-burden TB countries at affordable price," 2021	Source (archive)

• 1

  See the "Assessment approach" section on p. 2 of CHAI, Incubator narrative, 2022 (redacted).

• 2

  “​​The core team will schedule periodic check-ins monthly (frequency to be adjusted as necessary) with the GiveWell team. The teams will review new opportunities CHAI is evaluating with a view to aligning on the cost-effectiveness analysis to ensure that investigations on programs that are not likely to pass GiveWell review are stopped before resources are expended on scoping and testing in-country. The call will also allow CHAI to get updates on new investigations by the GiveWell team (as part of the “Sourcing” phase outlined below).” CHAI, Incubator narrative, 2022 (redacted), p. 2.
  

• 3

  See the "Assessment approach" section on p. 2 of CHAI, Incubator narrative, 2022 (redacted).

• 4

  “GiveWell’s experience with Evidence Action indicates that taking one program that GiveWell has already evaluated from idea to implementation takes approximately one year. There may be an opportunity to accelerate this timeframe in geographies where CHAI already has a strong presence on the ground.” CHAI, Incubator narrative, 2022 (redacted), p. 2.
  

• 5

  The core team will be comprised of a quantitative cost-effectiveness modeler and three dedicated program associates. The team will be headed by Dr. Cohen and Neel Lakhani, a public health expert with experience strategically developing and scaling numerous multi-country health programs, with substantial engagement from senior CHAI leaders, including CEO Dr. Buddy Shah and Chief Science Officer Dr. David Ripin. The core team will manage coordination and interactions across CHAI’s existing technical expert teams and country teams, and with GiveWell.” CHAI, Incubator narrative, 2022 (redacted), p. 1.
  

• 6CHAI, Incubator detailed budget, 2022 (redacted).
  
• 7

  "New programs that GiveWell has less experience modeling—such as market shaping—could be avenues for future collaborative research and cost-effectiveness analyses, using GiveWell’s research approach and CHAI’s technical expertise. This specific type of partnership, using both organizations’ strengths, could lead to significant future funding opportunities that GiveWell has not yet had the opportunity, expertise, or capacity to model." GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022.

• 8

  "This list represents an initial set of interventions to be evaluated through the CHAI incubator. It includes, first, areas identified as promising by GiveWell's research, which also overlap with CHAI's technical and geographic strengths:

  1. Maternal and child health interventions including safe birth kits
  2. High-frequency training for improving facility-based births
  3. Participatory learning and action groups for antenatal care
  4. Under-five contact management for tuberculosis
  5. Switching from iron/folic acid to multiple micronutrient supplements during pregnancy
  6. Birth dose of hepatitis B vaccine
  7. Eave tubes and/or housing improvements
  8. Water quality interventions"

  CHAI, Incubator narrative, 2022 (redacted), p. 5.

• 9

  "Second, some additional areas GiveWell has evaluated positively but not highlighted as top priorities for CHAI:

  9. Zinc/ORS
  10. Community health worker/ICCM platforms for delivery of ACTs and other cost-effective drugs
  11. Tuberculosis elimination
  12. Malaria chemoprevention (under expanded WHO recommendations)
  13. HPV vaccine
  14. Eyeglasses for workers

  Third, additional tentative ideas from CHAI:

  15. Development of referral networks and systems for severe malaria and MNCH programs
  16. ACTs in high endemic regions through the private sector
  17. Additional screening during ANC visits (e.g., for TB)
  18. Indoor residual spraying
  19. Reconsideration of Hep C treatment based on revised commodity and delivery costs

  Fourth, ideas currently on hold, to be revisited if and when success is demonstrated in one of the three preceding categories:

  20. Market shaping for key commodities identified as promising but not currently cost-effective
  21. Flexible fund to address gaps in funding for crucial cost-effective commodities"

  CHAI, Incubator narrative, 2022 (redacted), p. 5.

• 10

  Our current bar for directing funding to programs is that they are roughly 10 times as cost-effective as GiveDirectly's cash transfer program, or 10x cash.

• 11

  Our threshold analysis begins with a “conservative” estimate for the cost-effectiveness of this grant at 10x cash (our current bar), and works backward to determine the amount of annual room for more funding (RFMF) the CHAI Incubator will need to have identified by the end of the grant period in order to achieve this cost-effectiveness estimate. Our “best guess” is that CHAI will actually have identified twice as much annual RFMF as in our conservative estimate, which leads to a best guess cost-effectiveness for this grant of 20x cash.

• 12

  "The incubator will target the identification of ~$40[million] per year in new cost-effective investments across two programs by 2025/26" (CHAI, Incubator narrative, 2022 (redacted), p. 3), which is higher than what we’ve modeled in our threshold analysis here.

• 13

  In our threshold analysis, we guess that the opportunities CHAI identifies will be 12x more cost-effective than cash transfers on average, and that without the Incubator we would otherwise direct that funding to opportunities that are 10x as cost-effective as cash transfers. See here.
  

• 14

  We guess there’s a 25% chance that we would have recommended funding to opportunities scaled up via the CHAI Incubator without it. See here.
  

• 15We’ve assumed that scaled-up programs are funded for five years in this model. See here.
  
• 16Based on our discussions with CHAI, our impression is that the cause areas it plans to prioritize will be those that are relatively neglected by other actors in global health.
  
• 17For example, a grant to support technical assistance to the Liberian government to support the scale up of syphilis testing and treatment in pregnancy. GiveWell, "Evidence Action — Syphilis Screening and Treatment in Pregnancy," 2021.
  
• 18For example, in July 2022, we recommended a $15 million grant to Evidence Action's maternal syphilis screen-and-treat program in Zambia and Cameroon. As of this writing, the grant page for this grant, which will detail our cost-effectiveness estimate of 35x cash, has not yet been published.
  
• 19“CHAI’s core strengths are in government technical assistance and market shaping (i.e., reducing the price of medical commodities and treatment by negotiating with private sector stakeholders), which comprise most of its work.” GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022.
  
• 20
  • “We previously made a 2-3 year grant of $5 million to establish the Accelerator. We think that the results thus far are promising.” GiveWell, "Evidence Action Accelerator – Renewal grant for 2022-2025," 2022. See more details in footnote 25 on that page.
  • In July 2022, we recommended a $4.4 million grant to Evidence Action’s in-line chlorination program in Malawi (which we estimate to have a cost-effectiveness of 14x cash) and a $15 million grant to its maternal syphilis screen-and-treat program in Zambia and Cameroon (which we estimate to be 35x cash). We also recommended a $9.2 million grant in August 2022 to provide technical assistance to the Government of India to increase uptake of iron and folic acid supplementation among school-aged children (which we estimate to be 11.5x cash). These programs were all developed as part of the Evidence Action Accelerator. These grant pages are being drafted and have not yet been published.

• 21“We may be over-invested in Evidence Action. We have recently recommended grants to Evidence Action for its Deworm the World Initiative, Dispensers for Safe Water, Syphilis Screening and Treatment in Pregnancy, and Iron and Folic Acid Supplementation programs. If Evidence Action constitutes an increasing portion of our money moved, Evidence Action might have undue influence on how GiveWell makes decisions. We hope to mitigate this risk by diversifying our portfolio of partners with whom we are able to build new programs.” GiveWell, "Evidence Action Accelerator – Renewal grant for 2022-2025," 2022.
  
• 22CHAI, “CHAI Board of Directors announces appointment of Dr. Neil Buddy Shah as Chief Executive Officer,” 2022.
  
• 23
  • As of October 2022, "CHAI currently has offices in 34 countries." CHAI, Incubator narrative, 2022 (redacted), p. 1.
  • See https://www.clintonhealthaccess.org/#where-we-work for CHAI’s countries of operation.

• 24"CHAI cultivates a strong in-country presence in order to use local knowledge and partnerships to quickly implement and scale programs." GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022.
  One of CHAI’s core values is “Urgency”: “​​People are dying unnecessarily from AIDS, malaria, tuberculosis, and other preventable and treatable conditions. We recognize that every day we delay, people die. Therefore, we work with the utmost speed to build a strong foundation for sustainable impact. The faster we act the more lives we can save.” CHAI, "Our Values".
  Several people told us that CHAI will do “whatever it takes” to get programs running:
  • For a pediatric HIV program that lacked funding, CHAI worked out a pricing deal and created a new system for reporting results in order to run the program. Former staff member, CHAI, conversation with GiveWell, June 29, 2022 (unpublished)
  • CHAI is very committed to its projects and will stay actively engaged on them until they're completed. Government officials, conversation with GiveWell, July 6, 2022, (unpublished)

• 25Our understanding is that many of the programs CHAI has supported are effective and backed by evidence, though we have not extensively reviewed the evidence base for the programs it has supported.
  
• 26“At the beginning of the 2000s, treating HIV cost over US$10,000 per person per year. CHAI’s pioneering work in negotiating price reductions and generic licenses, together with critical efforts from partners like PEPFAR and the Global Fund, has dramatically reduced that to under US$60 per person per year in 2022.” CHAI, "AIDS was killing millions of people. CHAI was founded to make treatment equitable," 2022.
  
• 27For example, CHAI’s market shaping work on rifapentine/isoniazid (3HP), which was recommended by the WHO as “the new fixed-dose combination TB preventive treatment that will result in reducing the pill burden for people with TB infection, enabling better adherence and outcomes.” Unitaid, "New patient-friendly tuberculosis preventive treatment to be rolled out in five high-burden TB countries at affordable price," 2021.
  
• 28“Oral rehydration salts (ORS) and zinc could avert an estimated 93% of deaths, but progress to increase coverage of these interventions has been largely stagnant over the past several decades. The Clinton Health Access Initiative (CHAI), along with donors and country governments in India, Kenya, Nigeria, and Uganda, implemented programs to scale-up ORS and zinc coverage from 2012 to 2016. . . . Across CHAI-supported focal geographies, average ORS coverage across the program areas increased from 35% to 48% and combined ORS and zinc coverage increased from 1% to 24%.” Schroder et al. 2019, p. 1.
  
• 29“In Zambia, CHAI partnered with the government to implement its integrated SRMNH strategy in Northern Province, covering 146 health facilities and a total population of 1.5 million. A key component of the strategy is building capacity and skills of midwives to deliver effective and high-quality care and referral. CHAI worked with the government to develop skills-based trainings and structured mentoring to strengthen skillsets, knowledge, confidence, and practice to effectively identify, manage, and treat emergency childbirth complications before they become life-threatening and address or escalate issues such as equipment and medication stock outs and failures.” CHAI, "How trained midwives are saving lives in rural Zambia," 2022.
  
• 30For example, in Uganda: “For decades, intravenous quinine was the recommended treatment for severe malaria, but it sometimes resulted in side effects like hypoglycemia (low blood sugar) and it was difficult to administer. However, the release of the World Health Organization’s (WHO) revised malaria treatment guidelines in 2011 provided a new recommendation to treat severe malaria with artesunate, a medication with higher efficacy, fewer side effects, and easier administration directly into the vein. In support of this policy change, CHAI worked with the government of Uganda to revise their treatment guidelines and adopt the use of injectable artesunate.” CHAI, "Saving lives through scaling up severe malaria treatment in Uganda," 2017.
  
• 31“GiveWell shared its typical research and grantmaking processes with CHAI, including an in-depth session on our cost-effectiveness modeling approach. CHAI leadership asked detailed questions that signified a strong understanding of our analyses.” GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022.
  
• 32

  CHAI’s major program areas with technical teams are: Infectious Diseases, Women and Children’s Health, Non-Communicable Diseases, and Universal Health Coverage. CHAI, "Our Programs".
  

• 33See CHAI, Incubator narrative, 2022 (redacted), p. 5, for a list of discussed potential areas to start investigating.
  
• 34For example: Safe birth kits, high-frequency training for facility-based births, childhood tuberculosis contact tracing, hepatitis B vaccine for newborns, zinc and oral rehydration solution for childhood diarrhea, and community health worker platforms. See CHAI, Incubator narrative, 2022 (redacted), p. 5, for a complete list of discussed potential areas to start investigating.
  
• 35For example: Indoor residual spraying for malaria mosquitoes, Hepatitis C treatment, severe malaria and maternal and child health referral networks, and screening during antenatal care visits. See CHAI, Incubator narrative, 2022 (redacted), p. 5, for a list of discussed potential areas to start investigating.
  
• 36
  • “CHAI teams operate at the global level (e.g., senior and regional leadership), technical level (e.g., malaria, digital health, non-communicable diseases), and country level. These teams interact to set priorities, make decisions, solve problems, obtain and allocate funding, and work with government and funding partners. Some employees serve roles on multiple teams. Thus, decision-making about programs is made by a variety of teams and people, at different levels of the organization. For example, country teams have autonomy to pursue programs based on needs and government interest, while the CHAI global team may have high-level priorities that cause it to support programs that cut across multiple technical and country teams.” GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022.
  • Neel Lakhani, Comments on a draft of this page, October 12, 2022 (unpublished).

• 37CHAI, Incubator narrative, 2022 (redacted), p. 5.
  
• 38“Additionally, transition of a program from CHAI to another implementing organization would be considered a success.” GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022.
  
• 39To encourage transparency among respondents, these conversations were not published.
  
• 40For example, see GiveWell's non-verbatim summary of conversations with CHAI, June 6 and 14-16, 2022.
  
• 41CHAI, Incubator detailed budget, 2022 (redacted); CHAI, Incubator narrative, 2022 (redacted).