IRD Global and Johns Hopkins University — Planning Grants for RCT of Zero TB Program (August 2022)

Note: This page summarizes the rationale behind a GiveWell grant recommendation to IRD Global. IRD staff reviewed this page prior to publication.

Published: June 2023

Background

IRD Global (IRD) runs a mass tuberculosis (TB) search-treat-prevent program, Zero TB, in Karachi, Pakistan.1 This program involves deploying mobile vans across the city to systematically test individuals for latent TB infection and active TB disease.2 Those found to have TB, either active disease or latent infection, are initiated on treatment.3 IRD has also launched a “WellCheck Clinic,” which will help provide care for those individuals who are missed by the mobile vans, and plans to expand the number of WellCheck Clinics in Karachi under the implementation plan.4

The planning grant

During the planning grant period of September 2022 to March 2023, IRD plans to:

• Conduct desk research to develop a detailed evaluation proposal and budget, in collaboration with Dr. Dowdy.
• Conduct field pilots to test the feasibility of incorporating new screening tests, X-ray tests, and clinical tests that have become available recently into the RCT.5

The total cost for the planning grant to IRD is $1,016,599. This breaks down as follows:6

• Staffing: $453,877. This includes full-time support for global, technical, and in-country staff, as well as consultant support7
• Meetings and travel: $53,800
• Field pilots: $400,000
• Indirect costs (12% of direct costs): $108,921

The planning grant to Johns Hopkins University will cover the following activities during the planning grant period of October 2022 to March 2023:8

• Based on reviewing the scientific literature and mathematical modeling, decide how to best measure impacts on TB incidence in the study and determine the sample size necessary to achieve sufficient statistical power.
• Convene a panel of experts to provide input on this approach.
• Develop a model to project observed reductions in the TB incidence in the trial to longer-term reductions in TB incidence and mortality. (This will inform GiveWell’s cost-effectiveness estimates after receiving RCT results.)
• Develop an outline of a statistical analysis plan for the RCT.

The total cost for the planning grant to Johns Hopkins University is $102,732. This breaks down as follows:9

• Staffing for Dr. Dowdy and two additional researchers, Dr. ​​Sourya Shrestha and Dr. Tess Ryckman: $83,39310
• External Consultants: $10,000
• Indirect Costs (10% of direct costs): $9,339

The output of the combined planning grants will be a proposal for an RCT in Karachi, Pakistan, which, as of August 2022, we roughly expect to cost around $10-$20 million, for both implementation of the program and research costs, but planning is still in early stages and the budget may end up outside this range.

The RCT

• Randomization will be at the sub-city (i.e., Union Council) level. Roughly 30% of the city of Karachi, or 6-7 million people, are expected to be covered under the RCT’s treatment and control arms. The city is divided into smaller sections, referred to as Union Councils. There are expected to be 20 or more Union Councils in treatment and an equal number in control. Clusters will be placed in city peripheries where the TB burden is higher, there are few large or medium sized TB treatment centers, and there is less interaction with the Karachi central business district. This may aid in limiting spillovers between treatment and control groups.11
• The primary outcome will be the rate of conversion in M. tuberculosis infection status from negative to positive in a random sample of individuals tested immediately prior to the start of the intervention, and immediately following the intervention. Infection status will be assessed using Interferon-Gamma Release Assays (IGRA) blood tests,12 which do not distinguish between latent and active TB.13
• The secondary outcome is TB mortality. This will be measured via data gathered at hospitals and local governments. IRD has suggested validating the data via phone interviews.14
• We do not yet know what the minimum detectable effects will be for TB infection or mortality. Shrestha et al. 2021, a paper modeling TB screening and treatment, predicts a roughly 25% reduction in both TB incidence and mortality.15 Dr. Dowdy says he is confident that he and IRD will be able to design a trial that has sufficient power to detect the reduction in TB infection from IGRA tests.16 We’re guessing it won’t be well-powered enough to detect an effect on mortality.
• We expect the trial to take place over two years, with TB mortality data to be gathered beyond that time frame.17

The RCT will be led by Dr. Aamir Khan and Dr. David Dowdy. As part of planning, they will gather input from other leading TB researchers.18

We expect IRD and Dr. Dowdy to submit a full RCT proposal to GiveWell by March 2023, and for GiveWell to make a grant decision in April to May 2023.19

Case for making a planning grant

We are recommending this grant because we think there’s a good chance we’ll want to fund the full Zero TB RCT. Specifically:

• There is a good chance a well-conducted trial could update us toward believing Zero TB and programs like it are as cost-effective or more cost-effective than programs we expect to fund in the future. There is very limited high-quality direct evidence of similar mass screen-and-treat TB programs; evidence from these programs finds mixed effects on TB prevalence, but the interventions studied differ substantially from Zero TB.20 As a result, our estimates of the program’s impact largely derive from Shrestha et al. 2021, a paper that models the effects of a one-time, community-wide screening campaign in a mid-sized city in India over 20 years.21 Based on applying some fairly large discounts to this study, our best guess is that IRD’s Zero TB program is around four times as cost-effective as unconditional cash transfers.22 This is far below our best guess for our future cost-effectiveness bar, which we guess is roughly 10 times as cost-effective as unconditional cash transfers.23 If we took the results of this study at face value, our best guess is that Zero TB would be approximately 7x cash.24 We also do not include the impact of wellcheck clinics as a form of health system strengthening, which Shrestha et al. 2021 estimates could double cost-effectiveness, which would bring this opportunity to approximately 15x cash.25 Given this, we think there’s a 30% chance the program is at least 8x cash, 20% chance it’s at least 10x cash, and 10% chance it’s at least 12x cash.26
• We think the trial will be well-conducted, and we’ll have a strong signal on that prior to making a decision about funding the full trial. The research will be led by Dr. David Dowdy, a leading TB researcher, and Dr. Aamir Khan, who has decades of experience in implementing TB interventions. Dr. Dowdy and Dr. Khan also plan to engage other TB researchers on trial design.27 We think the primary outcome, the rate of conversion from TB-negative to TB-positive in a random sample of individuals, will allow us to see if the program is following the modeling paper’s predictions and, in turn, update our adjustment on the effect of the program.28 We also believe this outcome will avoid problems introduced by other commonly measured study outcomes.29 In addition, this trial will attempt to measure mortality.30
• If we update toward believing this is above our cost-effectiveness bar, there are a moderate amount of funding opportunities for Zero TB programs, and we guess these funding gaps are real. IRD estimates room for more funding (RFMF) of approximately $240 million over nine years (2022-2030) across several large cities. These are cities and provinces where IRD claims it is ready to implement this programming in short order.31 There are likely other cities as well that could achieve similar levels of cost-effectiveness to the cities we’ve modeled.32 We would consider recommending funding to expand Zero TB to these areas if we update to believing they are cost-effective in the future based on the RCT. We roughly estimate there is $425 million33 in room for more funding over nine years (2022-2030) in the cities and provinces IRD has identified and in other settings that could achieve similar levels of cost-effectiveness, but this is a rough guess. We also think there is a real funding gap for this type of program. It is our impression that the Global Fund will not fund programs such as Zero TB at scale, since it mainly focuses on commodity procurement and TB control (e.g., via treatment of cases that present in health centers) rather than active case-finding in the community, treatment of latent cases, and prevention of new cases via preventive treatment of household contacts.34
• Because we would be funding implementation of the program, we guess the trial would provide some direct impact in addition to the learning opportunity.35
  Our current best guess is that in addition to these combined planning grants of $1,119,331, the trial would include approximately $11 million in costs to implement the program and approximately $3 million in research costs.36 Given IRD’s challenges in obtaining funding (see above), we guess it’s unlikely that the Zero TB program would be funded without GiveWell funding it as part of this trial.
• An RCT of the Zero TB program could provide a meaningful contribution to the research literature and be beneficial to other funders as well, though this is speculative. We have not spoken to any other funders of TB programs about whether they would find an RCT of this program useful in their decision-making. However, given the lack of RCTs on this type of program,37 this could make a meaningful contribution to TB research.
• IRD appears to be good at this. IRD is a founding member of the Zero TB Initiative, launched in October 2015. We’ve spoken to one of IRD’s funders, the Stop TB Partnership, who told us that IRD is one of the best global organizations doing this work.38 In the area of Karachi in which IRD has worked, IRD’s data demonstrates an initial increase in TB cases identified and then a reduction over time in TB cases identified, as it purportedly rooted out cases in the community.39
• We estimate the cost-effectiveness of funding an RCT of the Zero TB program to be right at our current cost-effectiveness bar of 10x cash (we estimate the trial would be 10x cash). However, our model is rough, and we don’t put a substantial amount of weight on it.
• The planning grant will enable IRD to keep its TB team in place. Without additional funding, this team would need to be either let go or reassigned to other projects within IRD.40 As a result, we think this planning grant provides some option value—if we did not provide this planning grant, it would be more challenging to explore funding opportunities for IRD's Zero TB team in the future.

Risks and reservations

• There are still many details about the trial to be worked out, and we may find out after spending $1 million that this isn’t worth doing. For example, we may find there’s insufficient power for measuring reductions in TB infection (see above), the trial is more expensive than we anticipate, our cost-effectiveness bar is higher than we anticipate, or some other factor. This still seems like it’s worth the risk to find out.
• We guess that the trial won’t provide us with a reliable measure of the effect of the program on long-run mortality, which is the main outcome we model in our cost-effectiveness analysis (CEA) of the Zero TB program. The primary source of benefits in the CEA is from reduction in deaths in the short- and long-term, including deaths from TB and post-TB sequelae.41 While we expect to have a measure of mortality, we have some reservations about how well this will be measured (since it will rely on mortality measurement from existing data sources that may not capture all deaths),42 the extent of bias from the trial (by raising awareness of TB, the Zero TB program may inflate the number of deaths classified as TB, biasing mortality effects downward),43 and potentially statistical power (though we don’t have information about this yet). As a result, we will have to rely on assumptions about the effect of reductions in infection on reductions in TB-related deaths. In addition, because the trial would only be planned for two years, we will not be able to measure long-term reductions in TB-related infections and deaths, which comprise 90% of the benefits of the program.44 As a result, we will have to rely on extrapolating short-term reductions in infections to long-term reductions in mortality.
• There is some chance the effect of the program on TB infection is muted by spillovers into control areas. We’re not sure how much this will weaken the estimated effect on TB infection or whether we’ll be able to take additional steps to mitigate this issue (beyond spacing treatment and control clusters farther apart). As mentioned above, clusters will be placed in city peripheries where TB burden is higher and there is less interaction with the Karachi central business district; this may aid in limiting spillovers between treatment and control. Dr. Dowdy said he does not expect substantial spillovers but is also highly uncertain.45
• The cost-effectiveness of the potential RCT is right at our current threshold for funding.46 It relies on our modeling of the value of information, which we still have a lot of questions about. While our value of information cost-effectiveness estimate is not the only factor we will consider when deciding whether to fund the potential RCT, we do put some weight on it. We have a lot of open questions about our modeling of the value of information. This includes how much to account for the risk that the trial is successful but we are wrong about the benefits of Zero TB (which would lower cost-effectiveness), the potential for the results of the trial to affect funding allocations from other funders of TB (which we believe would increase cost-effectiveness), and/or the risk that this study or one like it would be funded without us.
• The value of the trial depends on several parameters we’re highly uncertain about. This includes the extent of room for more funding for Zero TB or Zero TB-like programs, probability the trial is successful, probability we decide the program is above our bar once we receive the results, our cost-effectiveness bar in the future, and benefits of the program during the trial. We’ve tried to put in best guesses on these parameters, but they are rough guesses and have the potential to shift cost-effectiveness and our view of the value of the grant a lot.
• TB screen-and-treat is a new area for us, and we don’t understand it very well, since it’s complicated. As a result, there’s a decent chance we’re missing something important about this program that could make it look much less promising. We expect to have more conversations with other TB researchers and funders during the planning period to help narrow this gap.
• We may be diverting IRD’s attention from other programs that could be more cost-effective. During planning and especially if we decide to fund the trial, IRD will likely allocate time from its senior leadership to Zero TB and away from other programs. If we think these other programs are more cost-effective, this could offset impact.

Internal forecasts

• We think there’s a 50% chance we will fund the RCT by May 2023. This is based on the following:
  • We think there’s a 50% chance our cost-effectiveness bar will be 10x cash in May 2023. Conditional on our bar being 10x cash, we think there’s a 50% chance we will fund the RCT.
  • We think there’s a 25% chance our cost-effectiveness bar will be 8x cash in May 2023. Conditional on our bar being 8x cash, we think there’s a 70% chance we will fund the RCT.
  • We think there’s a 25% chance our cost-effectiveness bar will be 12x cash in May 2023. Conditional on our bar being 12x cash, we think there’s a 30% chance we’ll fund this RCT.
• Conditional on funding the RCT, we think there’s an 85% chance the RCT is successful enough for us to put substantial weight on the results (i.e., the program is implemented and data are collected without serious issues, there are not major concerns about data quality).
• Conditional on funding the RCT and the RCT being successful, we think there’s a:
  • 30% chance we believe IRD’s Zero TB program is 8x cash after seeing the RCT results.
  • 20% chance we believe IRD’s Zero TB program is 10x cash after seeing the RCT results.
  • 10% chance we believe IRD’s Zero TB program is 12x cash after seeing the RCT results.

What we’ll learn during the planning grant period

We expect the planning grant to answer the following questions:

• How much will the trial cost?
• What is the minimum-scale implementation that would give us a good signal of program effectiveness?
• Will the trial have appropriate statistical power?
• Will TB test conversion substantially validate the modeling paper? What secondary outcomes will bolster findings?
• Do other researchers think this trial and approach is high-quality and an update to the existing literature? Dr. Dowdy will speak with other researchers during the planning period, and we plan to have separate conversations about this as well.
• How will the field pilots impact the program design and costs?
• How well will mortality be measured?
• How will the team deal with potential spillovers?

GiveWell plans to separately explore the following questions during the planning grant period:

• Do TB funders view this trial as valuable? What is their view of IRD’s Zero TB program, and what is their view of funding gaps for this type of TB program? We have not yet spoken to the Global Fund, and this seems like an important next step.
• Would the Global Fund ever consider funding TB elimination work? If no, why not? A cornerstone of this case is that we don’t believe the Global Fund will fund this work. That is based on one quick conversation with a TB stakeholder. We need to triangulate that via a call with the Global Fund and other implementers and partners in this space. It’s also worth understanding why it is unlikely to fund Zero TB programming, and understand any critiques of this approach from the Global Fund perspective.

Sources

• 1

  When previously implemented, IRD’s Zero TB program provided comprehensive tuberculosis (TB) care through 15 different interventions, including treatment for comorbidities, mental health, and diabetes, among others. IRD’s current model is much narrower; it focuses on finding patients, treating active TB, and preventing TB. IRD, conversation with GiveWell, December 13, 2021 (unpublished).

• 2
  • "Persons with latent TB infection do not feel sick and do not have any symptoms. They are infected with M. tuberculosis, but do not have TB disease. The only sign of TB infection is a positive reaction to the tuberculin skin test or TB blood test. Persons with latent TB infection are not infectious and cannot spread TB to others. Overall, without treatment, about 5 to 10% of infected persons will develop TB disease at some time in their lives. . . . The general symptoms of TB disease include unexplained weight loss, loss of appetite, night sweats, fever, fatigue and chills. The symptoms of TB of the lungs include coughing for 3 weeks or longer, hemoptysis (coughing up blood), and chest pain. Other symptoms depend on the part of the body that is affected. Persons with TB disease are considered infectious and may spread TB bacteria to others. . . . TB disease is a serious condition and can lead to death if not treated." CDC, "The difference between latent TB infection and TB disease"
  • IRD identifies cases of TB disease using mobile X-ray vans with X-ray machines and computer-aided detection (CAD). Patients with CAD X-ray abnormality results of 70% or higher and a sufficiently high personalized risk score (based on history of disease, age, gender, disease exposure, and geographic location) would then be given an Xpert test of their sputum, which confirms active pulmonary TB. Once a TB patient has been identified, IRD then screens that person’s entire household, using chest X-rays and, as appropriate, Xpert tests. Household members who are diagnosed with active TB disease receive TB treatment. The remaining household members five years old and older are tested using the interferon-gamma release assay (IGRA) blood test, which identifies whether they have TB infection and need preventive treatment. IRD, conversation with GiveWell, December 13, 2021 (unpublished).

• 3

  Household members who are diagnosed with active TB disease receive TB treatment. The remaining household members five years old and older are tested using the interferon-gamma release assay (IGRA) blood test, which identifies whether they have TB infection and need preventive treatment. IRD’s use of the IGRA test substantially reduces the need for tuberculosis preventive treatment (TPT) by identifying those who have latent TB infection, thus increasing cost-effectiveness and reducing the number of adverse events associated with the preventive treatment. In addition, people are more likely to consent to preventive treatment when an infection has been confirmed. Children in the household who are younger than five years old automatically receive preventive treatment per national and WHO guidelines, as they are judged to be at high risk. IRD, conversation with GiveWell, December 13, 2021 (unpublished).

• 4

  IRD explained that clinics would allow the program to reach more people than door-to-door treatment and symptom monitoring. IRD believes this approach would lead to more efficiencies and lower costs. IRD also believes that the wellness clinics may catch cases of active extrapulmonary TB disease that test negative on the Xpert tests. IRD, conversation with GiveWell, March 18, 2022 (unpublished).

• 5
  • IRD does not yet have the experience with these new technologies to understand their impact. Calculating intervention size, power, and assumptions for a large-scale evaluation of Zero TB will require first having data on the impact, if any, of these new technologies. IRD and Dr. David Dowdy, conversation with GiveWell, July 22, 2022 (unpublished).
  • New X-ray technologies that can detect a higher percentage of TB cases were endorsed by Stop TB in 2021. They require a high initial investment, but the cost of the software to run them has gone down. However, there are several open questions about this technology, including its impact on screening costs, consumption of diagnostic tests, and overall cost-effectiveness. IRD, conversation with GiveWell, July 5, 2022 (unpublished).
  • A new blood-based screening test for TB was endorsed by WHO in 2022. The test can measure latent TB infection, not just active TB disease. There are several open questions regarding this test as well, including its cost-effectiveness, uptake, and impact on ability to diagnose TB infection and disease. IRD, conversation with GiveWell, July 5, 2022 (unpublished).

• 6

  See IRD, Planning Grant Budget Summary, 2022.

• 7

  FTE support for Global Strategic & Technical Team ($337,564) + FTE support for core country teams in target countries ($89,313) + Global consultant support for key areas ($27,000) = $453,877. IRD, Planning Grant Budget Summary, 2022.

• 8

  “Our proposed approach over the six-month planning phase (October 2022 through March 2023) would be as follows:

  1. Review the scientific literature to inform a likely range of possible IGRA/skin-test conversion (and reversion) rates and appropriate IGRA cutoff values in the study area.
  2. Use a simple mathematical model to estimate the levels of reduction in conversion rates that would likely correspond to a 25% reduction in incidence and mortality over 5 years (comparing the intervention to the control arm), as well as the appropriate timing of outcome measurement to correlate to this effect.
  3. Based on the results of (1) and (2) above, perform sample size calculations to inform the size of a serial IGRA/skin-test survey that would be required to detect a difference corresponding to a 25% reduction in incidence and mortality.
  4. Convene (virtually) a panel of TB experts and statisticians to obtain external review of our assumptions and methods. This would include one pre-analysis call (in November) to obtain initial thoughts on the approach, and one post-analysis call (in January) for review of calculations and estimates.
  5. Provide the framework for a more detailed model of TB transmission (expanding on the simple model in step 2 above) that could use observed reductions in IGRA/skin-test conversion to project longer-term reductions in TB incidence and mortality.
  6. In consultation with IRD, GiveWell, and other external consultants, identify secondary outcomes (e.g., changes in TB mortality, post-trial TB prevalence) that could bolster confidence in our primary outcomes and link with the model outlined in step 5 above.
  7. Develop the outline of a Statistical Analysis Plan for a 'TB Step Change' trial, including input on the appropriate geographic scale of randomization and analysis.”

  Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 2.

• 9

  Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 3.

• 10

  $14,495 (David Dowdy, PI) + $7,571 (Sourya Shrestha, Assistant Scientist) + $61,417 (Tess Ryckman, Assistant Scientist) = $83,393. Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 3.

• 11

  IRD and Dr. David Dowdy, conversation with GiveWell, August 19, 2022 (unpublished).

• 12

  “We propose to explore the following quantity as the primary outcome of a potential “TB Step Change” trial: the annual rate of IGRA (or skin test) conversion. This quantity would be estimated by performing a relatively small, population-based IGRA survey in the intervention and control arms before the onset of the trial. This would likely happen in “rolling” fashion –in other words, study clusters would be allocated to different waves. Those in Wave 1 would first have the pre-intervention IGRA survey performed. Then, while the intervention clusters in Wave 1 receive the intervention, the pre-intervention IGRA survey would move to Wave 2, etc. One year after the pre-intervention survey in each wave, the survey team would return to those clusters (intervention and control) for the post-intervention survey. In this fashion, all clusters in the trial would receive a pre-intervention and post-intervention survey, one year apart –thereby enabling an estimate the annual risk of TB infection in the two arms and of the difference between arms. We would then link these estimates to a mathematical model of TB transmission to project future changes in TB incidence and mortality over a longer (e.g., 5-and 10-year) time horizon.” Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 1.

• 13

  “There are two kinds of tests used to detect TB bacteria in the body: the TB skin test (TST) and TB blood tests. A positive TB skin test or TB blood test only tells that a person has been infected with TB bacteria. It does not tell whether the person has latent TB infection (LTBI) or has progressed to TB disease.” CDC, "TB testing & diagnosis"

• 14

  IRD and Dr. David Dowdy, conversation with GiveWell, August 19, 2022 (unpublished).

• 15

  “Even in the absence of lasting effects on care delivery by the health system, this one-time intervention was projected to result in a 26.6% (95% range 25.7–27.4%) reduction in city-wide annual TB incidence (Fig. 4A, C, red lines) and a 26.8% (25.9–27.6%) reduction in annual TB mortality (Fig. 4B, D, red lines), at 10 years following implementation. This impact was achieved immediately—with an estimated 26.7% (25.9–27.5%) reduction in TB incidence and 31.5% (30.0–33.4%) reduction in TB mortality within 1 year—and persisted over time, lowering the TB incidence by 25.3% (24.1–26%) and TB mortality by 25.3% (24.2–26%) over the next 20 years following the intervention.” Shrestha et al. 2021, p. 7.

• 16

  “So all to say, with a total population of 5 million, I'm pretty confident that an intelligent trial could be designed that would have sufficient power to detect a 25% reduction in IGRA conversion. But it won't just be a standard, six-cluster-per-arm, straight-up, intervention vs control design. I can speculate as to how this could be done - but any specific idea I have will undoubtedly raise specific concerns.” Dr. David Dowdy, Email to GiveWell, August 27, 2022 (unpublished).

• 17

  A TB infection may not result in death until several years later, so an impact on TB mortality may not be evident in the same time frame that an impact on TB transmission would be. An ideal scenario would involve putting in place a reporting mechanism that could continue to track TB mortality even after the trial is completed. Dr. David Dowdy, conversation with GiveWell, August 11, 2022 (unpublished).

• 18
  • The JHU team will "convene (virtually) a panel of TB experts and statisticians to obtain external review of our assumptions and methods."
  • "Dr. Dowdy will be responsible for team management and oversight of the [trial planning] and will take final responsibility for all deliverables."

  Johns Hopkins University, Analytic preparation for TB trial, 2022, pp. 2, 3.

• 19

  "Our proposed timeline and deliverables would include: . . . Mar 31, 2023: Final report, including framework for a transmission model to project longer-term outcomes from IGRA/skin-test conversion rates and outline of a Statistical Analysis Plan." Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 2.

• 20
  • A 2021 systematic review of active case-finding (ACF) interventions for tuberculosis finds mixed evidence from three high-quality RCTs that ACF leads to reductions in TB prevalence. "The more intensively delivered door-to-door active case-finding intervention of ACT3 in Vietnam . . . reported a statistically significant relative reduction in the prevalence of microbiologically confirmed tuberculosis of 45%. By contrast, the less intensive enhanced case-finding intervention in the ZAMSTAR trial in Zambia and South Africa . . . did not show an effect. The before-after evaluation that pooled data from both intervention groups of the DETECTB trial in Zimbabwe . . . showed a relative reduction in culture-confirmed tuberculosis of 41%." Burke et al. 2021 p. e297 (For more information on these three RCTs, see Table 4: RCTs evaluating the effect of ACF on tuberculosis prevalence. For more information on how these RCTs were selected, see the "Search strategy and selection criteria" section on p. e284).
  • None of the three active case-finding RCTs (Table 4, Burke et al. 2021) resemble Zero TB’s screen-and-treat strategy. "Two cluster-randomised trials compared the effects of active case-finding versus no active case-finding on tuberculosis prevalence in general populations (table 4). One further cluster-randomised trial allocated urban clusters in Zimbabwe to one of two types of active case-finding, and also evaluated change in tuberculosis prevalence before and after implementation of active case-finding, a non-randomised comparison." Burke et al. 2021, p. e293. None of the three RCTs involved testing and treating latent TB.
    • "The ZAMSTAR study was a cluster-randomised trial in 24 communities in Zambia and South Africa. The active case-finding intervention (referred to as enhanced case-finding) included community mobilisation, education about tuberculosis in schools, fast-track sputum collection points in health-care facilities, and mobile community sputum collection points. Tuberculosis diagnosis in the active case-finding intervention was based on smear microscopy." Burke et al. 2021, p. e293.
    • “In the DETECTB study in Harare, Zimbabwe,11 the prevalence of culture-positive tuberculosis among a random sample of 12% of households in each of 46 clusters (23 allocated to mobile van active case-finding and 23 to door-to-door screening with symptoms and smear) before the active case-finding intervention was compared with prevalence after five rounds of active case-finding.” Burke et al. 2021, p. e295.
    • “In the ACT3 study, Marks and colleagues evaluated an active case-finding intervention in Vietnam that involved 3 years of annual household tuberculosis screening using sputum Xpert MTB/Rif assays for all people aged 15 years or older, regardless of symptoms, in 120 communities.” Burke et al. 2021, p. e295.

• 21

  "We developed a compartmental model that resembled TB transmission and epidemiology of a mid-sized city in India, the country with the greatest absolute TB burden worldwide. We modeled the impact of a one-time, community-wide screening campaign, with treatment for TB disease and preventive therapy for latent TB infection (LTBI)." Shrestha et al. 2021, Abstract, p. 1.

• 22
  • For the discounts we applied to the Shrestha et. al study, see row "internal validity adjustment" in the Cost Effectiveness sheets of this spreadsheet. For our cost-effectiveness estimates, see row "Bottom line cost-effectiveness (x cash)."
  • We use GiveDirectly's unconditional cash transfers as a benchmark for comparing the cost-effectiveness of different funding opportunities, which we describe in multiples of "cash." Thus, if we estimate that a funding opportunity is "10x cash," this means we estimate it to be ten times as cost-effective as unconditional cash transfers.
  • Note that a) our cost-effectiveness analyses are simplified models that are highly uncertain, and b) our cost-effectiveness threshold for directing funding to particular programs changes periodically. As of late 2022, our bar for directing funding is about 10x as cost-effective as GiveDirectly. See GiveWell’s Cost-Effectiveness Analyses webpage for more information about how we use cost-effectiveness estimates in our grantmaking.

• 23

  “Based on our current pipeline of spending opportunities and our projection of funds raised, we will likely increase our bar to 10x cash (up from 6x). At the end of the year, if we end up raising more funding than we expect, we’ll either roll over funding (which we’ll do if we expect that we’ll soon find additional opportunities that are greater than 10x cash) or allocate these funds to opportunities we’ve already identified that are less than 10x cash.” The GiveWell Blog: An update on GiveWell's funding projections, July 5, 2022.

• 24

  Removing the 50% internal validity adjustment we apply to mortality reduction benefits in our cost-effectiveness analysis causes cost-effectiveness to roughly double, since the vast majority of the total program benefits in our model comes from mortality reduction (versus morbidity and treatment costs averted).
  3.7 x 2 = 7.4

• 25

  "The achievable 10-year reduction in TB deaths more than doubled when the one-time intervention was coupled with health system strengthening that improved care delivery over the medium term." Shrestha et al. 2021, p. 11.
  7.4 x 2 = 14.8

• 26

  See our back-of-the-envelope calculation here.

• 27

  “Convene (virtually) a panel of TB experts and statisticians to obtain external review of our assumptions and methods. This would include one pre-analysis call (in November) to obtain initial thoughts on the approach, and one post-analysis call (in January) for review of calculations and estimates.” Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 2.

• 28
  • “TST/IGRA surveys are used to measure the prevalence of latent TB infection (LTBI), and TST/IGRA conversion is a common outcome in trials of TB vaccines. Since individuals exposed to TB will generally convert their TST/IGRA result before progressing to TB disease, measurement of TST/IGRA positivity can give an indication of the level of ongoing transmission in the population.” Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 1.
  • "This modeling study suggests that a focused and intensive intervention to halt TB transmission in a high-burden setting, leveraged to also strengthen subsequent TB care by the routine health system, can reduce TB incidence by over 40% (7800 cases averted per million population) and TB mortality by almost two-thirds (1710 lives saved per million population) over a 10-year period." Shrestha et al. 2021, p. 12.

• 29

  “Existing trials of interventions designed to reduce the population-level burden of TB have primarily used one of two approaches, namely population-based prevalence surveys and cross-sectional surveys of infection (e.g., TST or IGRA) in young children.

  Both of these approaches are problematic, for different reasons. Prevalence surveys give an accurate portrayal of TB prevalence, but (a) are resource-intensive to carry out [because the prevalence of TB is generally <1%]; and (b) cannot be performed without actually treating those people diagnosed as having TB (i.e., represent an intervention themselves). Thus, one must either perform an intervention –a pre-trial prevalence survey – in the control arm of the trial, or lack a pre-trial baseline against which the primary outcome can be assessed. Furthermore, a pre-trial prevalence survey is itself a case-finding intervention – one that may have a large but temporary impact on TB prevalence, making it difficult to disentangle the impact of the primary intervention from that of the pre-trial prevalence survey.

  An alternative approach that could be carried out on a sufficiently small scale so as not to “contaminate” the effect of the primary intervention is a TST/IGRA conversion study (i.e., measurement of TST/IGRA response, with longitudinal follow-up of those who are TST/IGRA-negative at baseline). TST/IGRA surveys are used to measure the prevalence of latent TB infection (LTBI), and TST/IGRA conversion is a common outcome in trials of TB vaccines. Since individuals exposed to TB will generally convert their TST/IGRA result before progressing to TB disease, measurement of TST/IGRA positivity can give an indication of the level of ongoing transmission in the population.”
  Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 1.

• 30

  "5. Provide the framework for a more detailed model of TB transmission (expanding on the simple model in step 2 above) that could use observed reductions in IGRA/skin-test conversion to project longer-term reductions in TB incidence and mortality.
  6. In consultation with IRD, Givewell, and other external consultants, identify secondary outcomes (e.g., changes in TB mortality, post-trial TB prevalence) that could bolster confidence in our primary outcomes and link with the model outlined in step 5 above."
  Johns Hopkins University, Analytic preparation for TB trial, 2022, p. 2.

• 31

  Aamir Khan, Executive Director, IRD, email to GiveWell, November 2, 2021 (unpublished).

• 32

  For our cost-effectiveness estimates for Karachi and two other unnamed cities, see the Cost Effectiveness sheets of this spreadsheet.

• 33

  $79,000,000 (Karachi) + $93,000,000 (City 1) + $44,000,000 (Area 1) + $9,000,000 (City 2) + $200,000,000 (other areas with similar levels of cost-effectiveness) = $425,000,000. See this spreadsheet.

• 34

  The Global Fund provides a limited budget for TB programs, which mostly supports the procurement of diagnostics, 2nd line drugs, and some preventive drugs. These programs are focused on TB control rather than TB elimination. Internal-only conversation with another TB organization, November 23, 2021 (unpublished).

• 35

  See the "Direct benefit of the trial" section of this spreadsheet.

• 36

  See row "Cost of the trial (millions)" in this spreadsheet.

• 37

  None of the three active case-finding RCTs (Table 4, Burke et al. 2021) resemble Zero TB’s screen-and-treat strategy. "Two cluster-randomised trials compared the effects of active case-finding versus no active case-finding on tuberculosis prevalence in general populations (table 4). One further cluster-randomised trial allocated urban clusters in Zimbabwe to one of two types of active case-finding, and also evaluated change in tuberculosis prevalence before and after implementation of active case-finding, a non-randomised comparison." Burke et al. 2021, p. e293. None of the three RCTs involved testing and treating latent TB.

• 38

  We heard from TB stakeholders that IRD runs a very specialized team that operates better than any other groups. In terms of TB case-finding, IRD's outcomes can improve over time relative to trials because its case-finding teams improve greatly over time. Internal-only conversation with another TB organization, November 23, 2021 (unpublished).

• 39

  See Slide 4 of this presentation.

• 40

  “I write because I have to make a decision soon at the end of this month. As you know we have a core Zero TB team in Karachi and our technical leads from IRD Global whom GW unrestricted funds have allowed us to support since last November. We continue to keep intact our other smaller TB project teams in other countries through whatever grants we can, in particular where we are able to integrate COVID and mental health with TB as we have been able to procure small to medium size grants for those. … If we are unable to proceed with a planning grant from GW, please let's decide sooner than later so we give them the runway to transition without panic.” Aamir Khan, Executive Director, IRD, email to GiveWell, August 17, 2022 (unpublished).

• 41

  See row "Value from deaths averted" in each of the Cost Effectiveness sheets of this spreadsheet.

• 42

  Every country reports a number of deaths from TB to the WHO, but many deaths go unreported. A lot of people who die from TB were never diagnosed as having TB or did not present at a health facility until such a late stage of illness that it was unclear whether TB was the initial cause of their illness and eventual death. Dr. David Dowdy, conversation with GiveWell, August 11, 2022 (unpublished).

• 43

  The Zero TB program entails actively screening people for TB, and diagnosing those who have TB. When some of those individuals die from TB, they will be reported as TB deaths when previously they may have gone undiagnosed and therefore unreported. So by strengthening the existing reporting system for TB deaths and reducing some of the underreporting that typically occurs, the program may end up underestimating its own impact. Dr. David Dowdy, conversation with GiveWell, August 11, 2022 (unpublished).

• 44

  A TB infection may not result in death until several years later, so an impact on TB mortality may not be evident in the same time frame that an impact on TB transmission would be. An ideal scenario would involve putting in place a reporting mechanism that could continue to track TB mortality even after the trial is completed. Dr. David Dowdy, conversation with GiveWell, August 11, 2022 (unpublished).

• 45

  Dr. David Dowdy, conversation with GiveWell, August 11, 2022 (unpublished).

• 46

  Note that this is not the cost-effectiveness of the initial planning grant we're recommending, but instead the cost-effectiveness of the full trial that we may decide to recommend funding after the planning period.