Guidance on Burden

Published: March 2025

Summary

Why did we look into our burden estimates?

To estimate disease burden, we have typically relied on data from the Institute for Health Metrics and Evaluation (IHME) Global Burden of Disease (GBD) study.

However, in our recent project to red-team our top charities, we noticed that these IHME estimates sometimes differ significantly from other sources:

• In Nigerian states where New Incentives operates, the United Nations Inter-agency Group for Child Mortality Estimation (UN IGME) estimates of under-5 mortality are 30-40% higher than IHME estimates.2
• In Chad, UN IGME estimates of under-5 malaria mortality are about 2.5 times higher than IHME estimates (about 1.6% vs 0.69% of children born alive dying from malaria before age 5). IHME estimates are significantly lower than in other countries where we fund malaria interventions.3
• In Nigeria, the United Nations Maternal Mortality Estimation Inter-agency Group (MMEIG) estimates of maternal mortality are about 3.5 times higher than IHME estimates (1,047 vs 299 deaths per 100,000 live births).

These differences, if incorporated into our cost-effectiveness analyses, could lead to substantial changes in which grants we make. As a result, we've decided to reassess the burden of disease data we use and investigate disease burden sources more closely in our grant investigations.

How we plan to assess burden going forward

• Consider multiple data sources. We think we should incorporate estimates from multiple sources beyond IHME's Global Burden of Disease study, as each source has its own limitations. For all-cause child mortality, we will consider IHME estimates alongside data from the UN IGME, recent national surveys such as the Multiple Indicator Cluster Survey (MICS), surveys from the Demographic and Health Surveys (DHS) program, and the Malaria Indicator Survey (MIS), and national vital registration systems where available. For cause-specific mortality, we intend to examine estimates from IHME, the UN IGME's Child and Adolescent Causes of Death project, the WHO Global Health Observatory, recent verbal autopsy studies, and disease-specific expert models. We’ll weight these sources based on data quality, methodology transparency, and potential biases. (more)
• Look into subnational data, but be wary of bias and noise. Sub-regions of a country can be very different from each other, but even large surveys often sample too few people to get precise subnational estimates of mortality, and routine surveillance data can be biased. When making subnational adjustments, we can make sure our story checks out by looking for broader patterns, speaking to people with local knowledge, and looking at whether differences persist over time and across different sources. (more)
• In general, don’t take noisy estimates at face value. If something looks anomalous (e.g., malaria spikes by 100% for a year, or neighboring states show radically different mortality rates), investigate it and consider regressing towards a regional mean, smoothing over multiple years, or disregarding the data source completely. (more)

Implications for our grants

This issue has led to some shifts in our grantmaking. For example, in Chad, incorporating UN IGME estimates led us to increase our estimate of malaria mortality by about 70%.4 This caused us to make a $3 million grant to seasonal malaria chemoprevention (SMC) in Chad and $25 million grant in insecticide-treated nets (ITNs) in Chad that we likely would not have made otherwise.5 We anticipate there will be additional examples as we continue applying this approach.

Key uncertainties and next steps

These are draft recommendations and may be updated as we learn more. Key uncertainties:

• How do we weigh different sources? We give similar weight to GBD, IGME, and high-quality surveys, since we think there are pros and cons to each, but may end up updating these weights if we learn specific sources are more or less reliable than we think.
• Should we incorporate local data sources? Global models (e.g., IGME/GBD) and standard surveys provide consistency, while local sources might have better, more recent data in areas we plan to support. However, these local sources have major drawbacks. Frequently, we find poor comparability across countries, and variable data quality in local health management information (HMIS) systems. We should explore whether these drawbacks outweigh potential benefits.
• There are likely technical nuances we haven't captured. We've found that comparisons between sources are more complex than they first appear. For example, we recently learned that IGME and IHME define diarrheal diseases differently. Similar technical differences likely exist elsewhere.

We may do more work to learn more about burden data. Possible next steps:

• Fund data collection. This includes potentially funding additional nationally representative surveys (DHS/MIS/MICS) or additional modules to these surveys, or supporting more autopsy data collection to better understand cause-specific mortality, particularly for malaria in sub-Saharan Africa. Our guess is that part of the reason different models disagree is that the data underlying these models is limited. We may look for cases where we could fund additional data collection to improve burden of disease estimates.
• Get a better understanding of what’s driving differences in models. This may come from bringing together modeling groups in regions with high disagreement to understand methodological differences.
• Look for ways to improve model transparency. We’ve found it difficult to engage with burden of disease models, and think that finding ways to see inside the black box of how they produce estimates may make it easier to understand which estimates to rely on and how to improve them.

Rest of this page

This page was adapted from our updated internal guidance on burden estimation. The sections below contain recommendations on and evaluations of specific sources for:

• Estimating all-cause mortality under age five (more), and in other age groups (more)
• Estimating cause-specific mortality under age five (more), and in other age groups (more)
• Making subnational adjustments (more) and dealing with noisy data (more)

Data sources for measuring all-cause and cause-specific mortality

All-cause mortality in children under 5

We should consider incorporating estimates of all-cause mortality from the UN IGME (more), IHME (more), and recent surveys (more). If we want to go deeper, we should consider looking for other sources such as randomized control trials or country-specific datasets (more).

When comparing estimates across these sources, we should compare apples-to-apples measurements as much as possible (more).

Sources

UN IGME Estimates

The UN IGME provides modeled estimates of all-cause child mortality. These estimates are referenced, sometimes with adjustments, by the WHO Global Health Estimates7 and the UN World Population Prospects reports.8 Our best guess is that these are about as reliable as IHME estimates, though this may vary by context.

We think the main strengths of IGME estimates are that they clearly label which data points are used for a given country, and their statistical model incorporates results from many surveys over time.9

Our biggest concern is that we can’t always discern why certain data points are excluded from their estimates.

Our understanding from speaking with experts and reading about the underlying model10 is that the process generating these estimates is:

• A group of researchers reviews data on all-cause child mortality from national surveys and other sources (e.g., DHS, MICS, MIS, Maternal Health Surveys, censuses when available).11 They make judgment calls about what to include based on how reliable they think the data is.12
• This data is put through a statistical process13 that combines estimates over time (and across countries) to generate an estimated trend of all-cause mortality projected through to approximately the present day.14
• At some point in this process, there is a country consultation15 where countries review estimates.

An example of what this looks like, in Ghana:

The dots are estimates from surveys. Hollow dots are excluded sources which do not factor into the model, while filled in dots are included sources. The blue line is the IGME estimate of mortality based on this data.

IHME Estimates

The IHME provides modeled estimates of all-cause child mortality. Our best guess is that these are about as reliable as UN IGME’s estimates, though this may vary by context.

We think strengths of IHME estimates are the large amount of data they include, and that they provide estimates at a variety of age splits.

Our biggest concern is that the methodology is somewhat complicated, and it’s hard for us to tell exactly how each data source they use influences their final estimate. We aren’t fully confident we understand the statistical framework.

Our best guess based on reading the GBD documentation and some brief conversations with people at IHME is that the process for estimating (e.g.) under-5 mortality is similar to the UN IGME model (above):

• The IHME collects estimates of all-cause mortality from national surveys and other sources,16 similar to IGME. They may decide to exclude outlier data, though we’re not sure exactly how these decisions are made.
• This data is put through a statistical process17 that combines estimates over time (and across countries) to generate an estimated trend of all-cause mortality projected through to approximately the present day.

They produce similar graphs to IGME:

The dots are estimates from surveys. Hollow dots are outliers, and our understanding is that they don’t factor into the model, though we haven’t explicitly confirmed this. The central green line is the IHME estimate of mortality from the surveys. The other lines in the IHME graph are various preliminary estimates generated by their statistical estimation process. There is some discussion of these steps at the following link.

"Raw Data": Recent national surveys of burden, vital registration, and sample registration

Both the IHME and IGME estimates are based on extrapolating from measurements of mortality produced by surveys or vital registration systems. In countries without complete vital registration, surveys like those produced by the DHS Program are especially important–they are the key input to the models we use.

In some cases, however, IHME and IGME don’t have access to these recent “raw data” sources at the time they produce their estimates, or they exclude them for unclear reasons. In these cases, we think we should give apparently reliable estimates from raw data sources weight as well. Roughly, we’d guess a recent, well-powered and unincorporated DHS survey should get at least as much weight as an IHME or IGME estimate.

The general strength of raw data is that it requires much less “modeling”. It’s a direct estimate of the thing we’re interested in at a particular point in time.

Weaknesses include potential for errors in surveys (e.g. failure to correctly weight the sample, poor administration of the survey), possible recall bias in surveys (respondents may forget a child died or how old they were),19 and statistical noise (more). Additionally, these estimates are just measures at a given point in time. IHME and IGME estimates may take into account other information, like historical trends in mortality, or patterns in other areas.

Key additional surveys and other sources of “raw data”:

DHS Surveys with full birth histories

These are generally 5,000-30,000 household20 nationally representative surveys. They collect full birth histories21 from women. A full birth history is a complete list of all children the woman has ever given birth to including their date of birth, sex, survival status, age (if alive), and age at death (if dead).22 This can be used to calculate the fraction of children born in a given year who die before age 5.

Mortality rates can often be pulled from STATcompiler, though not all DHS surveys are on STATcompiler. These are typically reported as averages for the 5 or 10 years preceding the survey (see footnote for discussion).23

MICS Surveys with full birth histories

These are similar to DHS surveys, but may tend to be slightly smaller (2,500 to 14,000 households).24 Our impression is that estimates from these surveys are slightly more likely to be excluded from IGME mortality estimates; we're unsure if that suggests lower reliability.

National vital registries/disease surveillance

In our experience, complete national vital registration data (e.g. full counts of births and deaths) is rare in places where we consider supporting programs, but sample registries which attempt to count all deaths in a subset of the population, like COMSA in Mozambique25 or the Sample Registration System in India, may provide additional data on mortality.

Additionally, charities sometimes provide data from health management information systems (HMIS), such as DHIS2, within countries (example from AMF in DRC). Our understanding is that these sources attempt to count every birth and death in the country, which allows for estimates of the mortality rate.

We don’t have a strong general opinion on sample registration systems and think we should consider giving them weight. We have not deeply investigated individual sources and our guess is that reliability may vary by context. We'd suggest (1) reviewing any available documentation on these methods and (2) talking to local experts on sample registration systems. Our default suggestion would be to give them somewhat lower weight because they seem somewhat less standardized across countries, and we could imagine fully evaluating these sources being difficult. But this is very uncertain and we find it plausible that sample registration is a very reliable source in particular areas.

Our understanding is that HMIS/DHIS2 surveillance data from within many sub-Saharan African countries can be relatively unreliable. That’s based on published studies reporting relatively weak data quality, even in countries like Tanzania26 which have lower mortality rates than the countries receiving most of our support,27 as well as some informal conversations. We generally wouldn’t recommend giving this data much weight unless we have reasons to believe that the data quality is high or these concerns don’t apply. That could be wrong–our perception is that there have been attempts to improve data quality, and it is possible the concerns we have heard from implementers and researchers are overstated.
More recent versions of any other source listed by IGME or IHME, including summary birth histories
We can see which data feed into these estimates by looking at graphs here (IHME) and graphs or source tables here or here (UN IGME). Sometimes a country-specific source is available and has been updated since the estimates were posted. Examples might include:

• Summary birth histories from a Malaria Indicator Survey (MIS) or other source. A summary birth history contains only the total number of children reported as born and having died for each mother. Under relatively strong assumptions about the timing of deaths, it’s possible to estimate child mortality from this data. If we find a reliable source providing recent estimates, it may be worth incorporating. However, this is significantly less reliable than full birth history data, and we don’t think we should try to produce these estimates ourselves as our understanding is that it’s fairly complicated. We would consider these lower quality than IHME, IGME, or full birth history survey estimates.
• Country-specific surveys. Occasionally we see country-specific surveys (e.g. in Niger, there was a survey called ENAFEME in 2021).28 We haven’t reviewed these in detail, but these might be worth checking for in other countries.

We should give high-quality, recent surveys weight roughly equal to or greater than IHME/IGME estimates, and adjust this weight up or down based on survey characteristics. Some reasons to consider giving a survey more weight:

• The data is recent. We expect that a survey from 2021 or later is going to be more informative about mortality in 2021 than earlier surveys.
• The sample size is large and confidence intervals are narrow. This means we can be less worried about statistical noise in our estimates.
• For child mortality, estimates are based on full birth history. Full birth histories include all the information required to estimate child mortality, while summary birth histories require more modeling assumptions which may be incorrect or inapplicable.
• The survey has historically been included by IGME or IHME in their estimates, but simply wasn’t available when estimates were constructed. If the survey has previously been included, we can have higher confidence in its quality.
• The survey appears to be complete (e.g., has low sample attrition or failure to reach households). Information on survey attrition or failure to reach households is usually included in survey reports. Low attrition, or low failure to reach households, means the survey is less likely to have missed some important sub-population or to be biased in its sample.

Reasons to give surveys less weight:

• It has been excluded by IGME and IHME as low-quality or as an outlier. This makes it somewhat more likely that there is something wrong with the survey, though we may still choose to give the survey some weight in these instances if we aren’t able to figure out what the issue is.
• There’s a clear risk of bias. This is somewhat general, but possibilities could include concerns such as a method of data collection that we think is likely to be more successful in cities, or a survey based on facility surveillance data that may miss deaths. We’d want to adjust for this bias if possible, but also consider downweighting the survey.
• It’s missing data, or has low response rates, especially for a particular part of the country. This could bias the results, but even if we don’t see a clear direction to the bias, the survey being incomplete should make us more uncertain about the survey’s value.
• It’s noisy or the sample size is small. This means we should be more worried about statistical noise in our estimates. For more on dealing with noise, see here.

Other possible sources of information

We think incorporating other sources of the kind listed above is a good first step, but in some situations we may want to go deeper; for instance, if all available sources seem to be relying on very limited data, or if there is significant conflict between sources.

Some additional steps we could take to further improve our all-cause mortality estimates include:

• Getting input from stakeholders, implementers, or other funders on all-cause mortality in this area and how it compares to other countries.
• Comparing all-cause mortality in the area of interest to nearby areas, or to areas that seem similar or predictably different: do they compare as we expect? If not, does the variation make sense?
• Looking at control groups from randomized trials. We think this provides an additional check on all-cause mortality rates, although we also believe we should be cautious about updating from this data alone, and be sure we understand the treatment and safety protocols for control group trial participants. We have heard that these protocols are sometimes better than the healthcare and health protocols that are generally available, which may make a control group estimate non-representative of expected mortality outside of the presence of the trial.

How to compare sources of under-5 mortality estimates

When comparing estimates, we should make sure estimates are all in the same units. Typically for all-cause mortality, this means they are all probabilities of death rather than age-specific death rates. (more)

Our desired estimates might be focused on an age group for which we don’t have estimates from the sources we wish to compare against. Our general recommendation is to find the closest matching age group where other data sources are available to generate an adjustment and to signpost any assumptions clearly.29

All-cause mortality in other age groups

For older age groups, we are less aware of the data landscape. Our impression is that there is less data available outside of the IHME estimates, but there is still some information:

• DHS/MICS surveys provide estimates of adult and maternal mortality for certain age groups, though we've not vetted reliability. Our understanding is that these underpin the IHME’s estimates. These estimates are based on somewhat more complicated methods than the childhood mortality estimates–they use a method called the sibling method, where people are asked about the mortality of their siblings.33 This has two implications for interpreting adult mortality:
  • Adult mortality is generally more uncertain. People must recall the ages and years of death for siblings, which may be more difficult than remembering ages of death of children, especially if the siblings move away, or if they are half-siblings or have weaker family ties.
  • We should be especially unsure about the geographic distribution within a country, or about the level of mortality in countries with high migration. If siblings move to a different region of the country, or to a different country, their death may be incorrectly factored into all-cause mortality in the area where their sibling lives. We're unsure how large of an issue this is.
• The UN Maternal Mortality Estimation Inter-agency Group (MMEIG) provides estimates of maternal mortality, though we've not evaluated these estimates for reliability. In Nigeria, these estimates diverge significantly from IHME’s and we aren’t sure why they differ.34
• National-level sample or complete vital registration systems may provide additional estimates of adult all-cause mortality. See caveats above.
• The IHME provides a “45q15” analysis – that is, the probability of living 45 more years conditional on reaching age 15 – which includes the sources they use for estimating adult mortality rates. Our sense is that these are generally sparser than their source lists for child mortality estimates but can provide additional leads in particular countries.
• The WHO Global Health Estimates also include a measure of adult mortality that is similar to the 45q15, but we know little about the methodology. It should be accessible via the Global Health Observatory.

Cause-specific mortality in children under 5

When looking into cause-specific mortality (either for direct burden estimates or external validity adjustments), similar principles to investigating all-cause mortality apply: we should consider incorporating other estimates of cause-specific mortality from the UN IGME (more), IHME (more), and any recent verbal autopsies or other local surveys (more). Compare measurements with the same units as much as possible (more).

For additional discussion of differences between two major sources of malaria estimates (UN IGME and IHME), see this report we commissioned from ReThink Priorities.35

UN IGME Estimates

The UN IGME provides estimates of cause-specific mortality via the Child and Adolescent Causes of Death Estimation (CA CODE) project. Our best guess is that these are similarly reliable to IHME, though this may vary by context, and we're not very confident in this assessment.

We think the main strengths of IGME estimates are that the methodology is fairly transparent and that the framework seems fairly cohesive across diseases, since shares of deaths due to each cause are mainly estimated in a single combined model,36 rather than spread across different models.

Our biggest concern is that the estimates appear to rely primarily on a limited number of verbal autopsies (0 in many countries we’re interested in).37

How are these estimates generated?

The IGME cause-of-death model is described on their website as follows:

"Estimates on causes of death are produced by the Child and Adolescent Causes of Death Estimation (CA CODE) project. Causes of death are estimated separately for countries with high versus low mortality, by data availability, and by age group. Empirical cause-of-death data were gathered through systematic review, known investigator tracing, and procurement of national and subnational studies. We adapted the Bayesian LASSO (Least Absolute Shrinkage and Selection Operator) approach to address data scarcity, enhance robustness, and assess the uncertainty of the estimates on a coherent framework. All-cause mortality estimates were harmonized with UN IGME, and estimates for a small number of low-burden causes (e.g., measles, collective violence, or HIV/AIDS) were systematically integrated from WHO and UNAIDS. The estimates have been reviewed and endorsed by the WHO Member States through country consultation."

A detailed description of the Bayesian LASSO approach can be found in Mulick et al 2022. Our high-level summary of this model is as follows:

1. They collect empirical cause-of-death (COD) data at the country level.38 In most African countries, this data is verbal autopsy data. The first figure below summarizes their data inputs as of 2015 (most recent complete map we could find). The second figure shows their updates for the 2019 version.
2. They also pull together some estimates of covariates that are expected to relate to common causes of death over time.39 These are factors like malaria prevalence, gross national income, measles vaccination, overall level of under-5 mortality, and prevalence of underweight status. A full list is available in the appendix of Perin et al 2021.40
3. Using the historical data that they have, they build a statistical model41 that predicts which causes of death are most common based on the cause-of-death data from verbal autopsies and the values of these covariates.42 They do this separately for mortality among children aged 0-1 months and 1-59 months.43
4. They incorporate estimates for certain less common causes of death separately (examples given are measles, HIV, and violence).44 We haven’t looked closely at how these are incorporated into the final estimates.

Note in the figures below that many countries have no verbal autopsy data points.
Figures

Data points in the UN IGME model as of 2000-2015:45

Additional data points added for the 2000-2019 version, noting that it’s possible that there were additional data points added in between 2015 and 2019:46

IHME Estimates

The IHME provides estimates of cause-specific mortality for all age groups, including children. Our best guess is that these are similarly reliable to IGME, though this may vary by context and we're not very confident in this assessment.

We think the main strengths of IHME estimates are that they try to pull together a broader range of data sources to produce their estimates and the data is generally easily accessible.

Our biggest concerns are that the model is very complex, and no individual team seems to have a fully cohesive view of the whole model. We worry that this could lead to errors, and unknown problems with the final estimates.48

How are these estimates produced?

Again, we're not fully certain about all steps, but at a high-level, our understanding from conversations with the IHME, a brief desk review of their methodology papers, and speaking with other experts is that:

• The IHME has processes to generate initial estimates for the number of deaths due to specific causes. The methods for generating initial estimates due to a particular cause vary. As an example, one can compare the methodology for measles here and pertussis here. They are similar, but not identical, e.g. pertussis references using “expert historical data”, but measles does not.
• Given estimates of disease-specific deaths from different processes (labeled “unadjusted deaths by location/age/year/sex” at the links above), the IHME harmonizes these unadjusted estimates to fit into the separately estimated all-cause mortality envelope.49 This process is called CODCorrect.50

This process somewhat differs from IGME CA CODE. As described above, IGME CA CODE primarily uses a large multinomial regression model to output shares of death due to all diseases in a single shot. The IHME uses a number of different disease-specific models to produce initial unadjusted estimates, which they then combine in the final (CODCorrect) process to generate the final adjusted estimates.

Verbal autopsy studies

Verbal autopsies are studies where caregivers or next of kin describe symptoms preceding a death, and physicians, experts, or algorithms analyze these descriptions to produce a best guess at the cause of death.51 These studies underpin estimates from IGME and IHME, and are somewhat close to ground-truth data. If recent verbal autopsy data is available, we believe it's worth incorporating into our estimates and giving reasonably high weight, especially if we don’t think it is otherwise factored into IHME or IGME’s estimates.

The strength of verbal autopsies is that they are based on actual deaths (rather than statistical models). In fact, they usually underpin the statistical models, so if a new verbal autopsy in a relevant geography becomes available, it’s worth looking into.

Its weakness is that verbal autopsies are themselves uncertain–we’re relying on recollections of symptoms, and guesses at what those symptoms imply, rather than tissue sampling or medical records. Verbal autopsies may also be unavailable for particular regions or age groups.

Minimally invasive tissue samples

Minimally invasive tissue sampling involves taking tissue samples from deceased people and using those samples to assess possible causes of death (see discussion here). Our understanding is that the idea is to find a relatively low-lift version of diagnostic autopsy that can be performed in low-income settings. An organization called CHAMPS collects some of this data.

The strength of this data is that it is as close to ground-truth data as we can get; for example, we could detect the presence of malaria parasites and other conditions in deceased people as a way to diagnose malaria.

The weakness is that it’s not yet widely available, and it’s unclear to us how much it is currently being used in community settings (as opposed to in facilities). We should make sure that any data we use is coming from representative populations, since causes of death could be different in facilities compared to communities.

Estimates from mechanistic disease burden models

For some diseases, there are modeling teams that have estimated disease burden using custom mathematical modelers (some examples here). We are still learning more about these models, and they may be a promising source of information given that they are typically built by experts in a particular disease.

One drawback of these models is that they may not adhere to the one-cause-per-death standard, as GBD and IGME models do. In these cases, the mechanistic models may “overcount” cause-specific deaths relative to cause-specific models, and we don’t think we should treat them as directly comparable to those estimates. We should look into how modelers address the issue before including these estimates in an average at face value.

How to compare across sources of cause-specific mortality

General principle is to compare estimates which are as analogous as possible. That means matching age groups, geographies, and disease groups. Three specific recommendations:

• Compare shares of deaths due to different diseases across sources when possible. Mortality rates or numbers of total deaths due to a disease can vary across sources because they disagree about the all-cause mortality envelope or population estimates. If we adjust for all-cause mortality and disease-specific mortality rates separately, we risk double-counting the all-cause mortality adjustment. By using shares of deaths due to different diseases across sources, we can be more confident the numbers are comparable and avoid double-counting.
• Check that disease hierarchies match across sources. The IHME disease hierarchy is here and the IGME disease hierarchy is here. For causes like measles and malaria, the two sources’ disease hierarchies map to the same disease codes, but we have seen some disagreements in defining diarrhea.52 We should check that causes are comparable across sources and make adjustments to make them more comparable if not.
• Look for sources that adhere to one-cause-per-death. IHME, IGME, and verbal autopsies reporting cause-specific mortality fractions either implicitly or explicitly ascribe one underlying cause to each death. Some disease-specific models may not adhere to this same standard, which could lead to higher estimates. Our recommendation is to focus on sources that use the one-cause-per-death framework, though other sources may provide useful triangulations/information about potential “indirect deaths”.53

Cause-specific mortality in older age groups

In addition to IHME estimates, the WHO has some estimates of cause-specific mortality for older people as well (e.g. malaria deaths in people over age 5; these are separate from IGME estimates). IGME also has estimates of cause-specific mortality in older children aged 5-19.

Our impression is that we should approach cause-specific adult mortality estimates with caution. We’ve heard concerns from officials at WHO about inferring mortality shares from verbal autopsies, and our best guess is that the adult mortality all-cause envelope is also very uncertain. We don’t have strong general recommendations for data on adult cause-specific mortality and think we’d need to investigate specific questions on a case-by-case basis.

Subnational burden data

Subnational adjustments are more likely to be necessary if:

• We have intuitive reason to believe the regions we're modelling might be different from the rest of the country.
• We are receiving a list of regions selected based on either:
  • Being the remaining regions after the highest priority areas have already received an intervention. This could mean these areas are somewhat likely to have a lower burden.
  • Being selected based on being the regions most urgently in need of some intervention. This would suggest these areas are likely to have a higher burden of the relevant disease.

In some cases, national estimates may be preferred over subnational estimates. For example, this would be the case if:

• We think it’s likely that our funding would funge across the country or region, so that the effect of the grant is a broad reduction in mortality rather than one concentrated in a specific marginally funded area.
• We think our grant will cover most of the country. In this case, it’s more likely that the update from incorporating subnational data will be small.

Subnational data sources vary by context, so we don’t have strong general recommendations, but some sources that may be useful include:

• The UN IGME produces subnational all-cause mortality estimates for many countries in sub-Saharan Africa.54
• DHS, MIS, and MICS survey data is often available by province or state.
• For malaria, the Malaria Atlas Project produces detailed subnational maps.
• Sometimes proxies of mortality are more available than mortality data at the subnational level. For example, we may have a recent survey of malaria prevalence, but no data on mortality. In the absence of better data, using proxies of mortality can help us make a best guess at subnational variation.
• Sometimes charities or ministries of health will provide granular subnational data from their health information system. We think we should generally approach this data with skepticism and watch for bias or noise, but this data can be useful and we shouldn’t disregard it either.

Advice for approaching subnational adjustments:

• Be more wary of noise. There’s less data available for subnational regions, and our impression in limited investigations is that it’s less reliable (e.g. sample sizes are smaller, estimates are often more uncertain). If we feel that national estimates are more reliable, then putting some weight on the national estimates may help us avoid selecting on noise.
• Be more wary of bias. Sometimes national DHIS data will be available through implementers or via other sources. This data can be useful for triangulation or understanding subnational patterns, but we should be wary of reporting bias; for example, more well-off areas may have better disease reporting, or districts with hospitals may report higher mortality because of better measurement.
• Try to understand the patterns. Because of the noise and bias issues, it’s especially important to have an intuitive understanding of why an area might be a high or low burden region. Look at a map, check population density or local GDP estimates, and think “does this make sense"?
  • For example, we’re more likely to trust anomalously low malaria burden estimates in an area if we know it’s a desert, or at high elevation, or near a large city (all of which are typically associated with lower malaria) than if it’s not.
• A conversation can be especially helpful. People in the relevant country may have immediate impressions about relative burden (e.g. “malaria is more of a problem in the south”). This can help us sense-check our estimates.

Dealing with noise

We have written about our approach to uncertainty previously; this section on pitfalls contains good general advice for this. We'll instead focus here on some ways to apply a skeptical eye to burden data.

• Distinguish whether what you’re looking at is “raw data” or a modeled output.
  • The IHME and UN IGME generally report modeled outputs in most countries where we make grants.55 These are based on various statistical processes (“models”) that take a bunch of data into account and spit out an estimated mortality, e.g. fraction of deaths in children aged 1-59 months from malaria in Chad in 2021.
  • MICS, MIS, and DHS surveys generally report something closer to “raw data”. For example, the “under-5 mortality” in these surveys generally comes from surveying a random sample of mothers on how many children they’ve had, when, how many are still alive, and how old children were when they died.
• For raw data, things to check include:
  • What’s the sample size and the confidence interval? A narrower confidence interval means an estimate is more precise, and we can be less worried about sampling noise.
  • If we have repeated measurements, how much does the statistic bounce around? If a statistic bounces around a lot when being measured consistently, we should consider that the most recent measurement may not be predictive of the future.
  • Does the geographic pattern make sense? Even if the sample is large and we have consistent measurements, it’s often good to check whether the geographic pattern makes sense. This can alert you to weaknesses in the data.
• For modeled outputs, things to check include:
  • If we have repeated estimates, how much does the estimate bounce around? Does the geographic pattern make sense? Both of these checks from above still apply.
  • Should the model in theory account for noise? Some models should be taking statistical noise, bias, etc into account. For example, the IHME uses a multistage process with spatial and temporal smoothing for their under-5 mortality estimates–our best guess is that this should in theory account for statistical noise, though we are not fully confident in how this model works.56
  • How much data is the model based on? More data gives us more reason to believe the estimates.
    • For example, our best guess is that there are no recent verbal autopsies measuring causes of death due to different diseases in Chad. So we think both the IHME and IGME are (at least partially) relying on data from other countries that “look like” Chad to estimate mortality in Chad. This makes us less confident in either estimate being “right” and more inclined to give both estimates some weight.
    • For all-cause under-5 mortality, IGME produces graphs with the data they use. In South Sudan, there is not much recent survey data; while in Ghana there is much more (see figures below). This makes us more uncertain about the South Sudan estimates, and more willing to update if other evidence conflicts with the IGME estimates.
• What to do if estimates seem noisy or untrustworthy:
  • Use a higher level of aggregation: Higher levels of aggregation bring more data into estimates, via either higher sample sizes or more data points for modeling. Examples of using a higher level of aggregation:
    • Taking an average across years if one year seems to have an anomalous spike.
    • Using data from a larger geographic area (e.g. in Nigeria, we start with zone-level MICS under-5 mortality estimates instead of state level).
    • Grouping a larger set of diseases together: Measurement of a single etiology (e.g. rotavirus) is likely to be noisier than measurement of a larger category (e.g. diarrhea). Looking at groups of causes may smooth out some of the noise in narrow cause estimates.
  • Give more weight to other sources: All else equal, if we think a given estimate is based on less data, or seems noisier, we should give more weight to other sources instead of relying on that estimate alone.

Figures

Under-5 mortality in South Sudan (few data points, high uncertainty)57

Under-5 mortality in Ghana (more data points, lower uncertainty)58

Examples of sense checks

Since our burden estimates often involve averaging across somewhat opaque source data and then making additional adjustments to those data, we should always make sure we step back and assess whether our final estimates are reasonable.

One way to do this is to write a simple mortality calculation, similar to a simple cost-effectiveness analysis, but limited to burden-related calculations (more). More targeted sense checks can include asking an expert’s opinion on our bottom line estimates (more), or doing more quantitative checks like examining differences across geographies (more) or comparing disease-specific mortality and all-cause mortality estimates (more).

Example of a simple mortality calculation

A simple mortality calculation should lay out all the steps/adjustments being made to burden, starting with the initial estimate(s) from primary data sources, and laying out the main adjustments made to this data to arrive at our estimated counterfactual burden. An example for malaria mortality in countries being considered for recent net distribution investigations is below.

As seen above, the GBD estimate for malaria burden in Nigeria (PMI States) was about twice as high as the estimate for Chad, but our bottom-line estimate is that their total 1-59 month malaria-attributable mortality is roughly equal burden in absence of nets. About half of the change comes from incorporating other sources of burden data, for a burden estimate increase of 72% in Chad vs. +14% in Nigeria (PMI states).

This helps focus in on the most pivotal adjustments and key uncertainties, as well as notice which adjustments have less impact59 on our bottom line, and can provide a roadmap for legibly describing our logic.

Examples of things we can ask experts/locals

Note: we're more unsure about the content about this section than others, but thought it was important to include this as a key way to sense-check ourselves.

People from a given country, or with significant experience living or working in a given country or region, may have ideas about disease burden across the country, particularly in a relative sense (e.g., “region X has more malaria than region Y”). It’s also worth understanding what sources of information inform their perspective. Some questions that might be worth asking include:

• Where in this region is the burden of this disease most severe?
• What data do you look at, and which people or conversations have informed your view of burden in these areas?
• We think that State X has twice as many deaths due to this disease as State Y. Does that align with your experience or perception?
• We saw this statistic in this data source we use. Does that seem credible and reasonable?
  

Example of comparisons across geographies

When looking into net distribution grants in Nigeria, we initially noticed our estimated malaria-attributable burden without nets in Lagos (0.67%) was around 3.5x as high as in Borno (0.19%).60

This was surprising because we typically hear about malaria (and public health more generally) being worse in the northern part of the country, where Borno is located.61 This prompted us to look into the estimates more closely, and when we did this, we found that there were a some things that might be inflating the estimates in Lagos relative to Borno:

• Taking an average that includes other sources (IGME, MICS) suggests around 20% higher all-cause under-five mortality than GBD in Borno and 7% lower in Lagos compared to IHME.
• We were reducing our estimate of malaria-attributable mortality by ~50% in Borno to account for seasonal malaria chemoprevention (SMC) not being reflected in GBD estimates. After more review, our best guess is that it’s partially reflected in the GBD’s estimates, so we only reduce our estimate by about 25%. There is no SMC in Lagos, so our Lagos estimates were unaffected by this change.
• We made some changes to how we estimate indirect deaths (more) which reduces total malaria-attributable mortality by about -20%.

After making some changes to address these and other issues and updating our cost-effectiveness analysis parameters, we now estimate that malaria burden without nets in Lagos (0.36%) is about 30% lower than in Borno (0.52%).

Example comparing malaria-attributable mortality to all-cause mortality

When looking at malaria-attributable mortality in Nigerian states, we noticed that our estimates were pretty close to the IHME GBD all-cause mortality estimates in some states; for example, they were within 10% in Lagos state. That is to say, our model implied that in those regions nearly all deaths were attributable to malaria.

Upon review, this stemmed from our adjustment for indirect deaths due to malaria.

1. The IHME GBD model estimates relatively high shares of deaths directly attributable to malaria in some states. For example, in Lagos we think they suggest that around half of 1-59-month deaths are directly attributable to malaria.62
2. We adjust direct malaria mortality upward by 75% across the board to account for indirect malaria deaths. Since .5*1.75 = ~90%, we effectively assume about 90% or more of deaths are attributable to malaria (directly or indirectly).

That seems very high. Moreover, in age groups or geographies where over 57% of deaths are directly attributed to malaria, using a blanket 75% adjustment would lead us to estimate that over 100% of deaths are attributable to malaria, which is impossible.

To address this, we modified our indirect deaths adjustment to be smaller in areas with higher direct malaria-attributable mortality. In Lagos, this means we only adjust upward by about 40% instead of 75%, so we now estimate that something closer to 75% of deaths are directly or indirectly malaria-attributable. This is around a -20% adjustment to total malaria-attributable burden in Lagos.

Sources

• 1

  Our updated estimates of malaria burden in Chad have led us to allocate $3.3 million in grantmaking for seasonal malaria chemoprevention (more), and $25.9m for insecticide-treated nets (not yet published).

• 2

  For example, in Yobe state, IGME estimates an under-five mortality rate of 132.92 deaths for every 1,000 live births (see here), while IHME estimates an under-five mortality rate of 9.97%, i.e. 99.7 deaths per 1,000 live births (see here). 132.92/99.7 = ~133%.

• 3

  See other estimates of national malaria mortality in our cost-effectiveness analysis for insecticide-treated nets here.

• 4

  Read more about this updated reasoning here.

• 5

  Read more about the SMC grant here: Malaria Consortium — SMC Renewals in Nigeria, Burkina Faso, Chad and Togo (January - October 2024) | GiveWell. The ITNs grant page has not yet been published.

• 6

  See countries with available subnational data here.

• 7

  See for example, the WHO estimates of under-5 deaths, which cite the papers underpinning IGME’s estimates. In our experience, estimates of deaths due to a particular disease are sometimes referred to as “WHO estimates” or “UN estimates”.

• 8

  “Adjust the under-five mortality rates for consistency with the estimates published by IGME in 2023 (United Nations. Interagency Group for Child Mortality Estimation, 2024)... In preparing estimates of child mortality for the 2024 revision, the Population Division coordinated closely with the United Nations Inter-agency Group for Child Mortality Estimation (IGME), which is led by UNICEF." United Nations, Methodology Report World Population Prospects 2024, p. 13-14.

• 9

  For example, see this source table of included and excluded sources for under-five mortality estimates in Nigeria.

• 10

  An explanation of the model can be found at UN IGME, Methods. See Alkema and New 2015 for additional detail.

• 11

  They also use vital registration data in countries where it is thought to be reliable and complete, which is mostly not the case in sub-Saharan Africa.

• 12

  "The first step is to compile all newly available data and add the data to the CME database. The UN IGME estimates are based on nationally representative data, and newly available data will include recently released vital statistics from civil registration systems, results from recent household surveys and censuses, and, occasionally, results from older censuses or surveys not previously available.
  Data quality is critical. The UN IGME assesses data quality and excludes data sources with substantial non-sampling errors or omissions as source data in its statistical model to derive the estimates. The UN IGME does not use covariates in the estimation process." UN IGME, Methods.

• 13

  The model used is called the Bayesian B-splines or “B3” model. For details, see Alkema and New 2015.

• 14

  "U5MR – The various data sources may suffer from different errors, for example random errors in sample surveys or systematic errors due to misreporting. Thus, different data sources can yield widely different estimates of U5MR for a given period. To reconcile these differences and take better account of the systematic biases associated with the several types of data inputs, an estimation method is used to fit a smoothed trend curve to a set of U5MR observations and to extrapolate that estimated U5MR trend to a defined time point." UN IGME, Methods.

• 15

  “UN IGME estimates were reviewed by countries through a country consultation process but are not necessarily the official statistics of United Nations Member States, which may use a single data source or alternative rigorous methods.” UN IGME, Levels and Trends in Child Mortality, 2023.

• 16

  "Private and public organizations around the world collect data through surveys, censuses, and other methods…Our data librarians conduct internet searches, review government and international agency websites, monitor known data sources, and consult with international partners to discover new datasets when they are published." Institute for Health Metrics and Evaluation, How we collect data.

• 17

  They describe the model as a three-stage process involving spatiotemporal and Gaussian process regression. We're not fully confident about what that means in practice.
  "We apply a non-linear mixed effects model that utilizes the relationship between key covariates such as income and education and the level of under-5 and adult mortality rates, spatio-temporal regression, and Gaussian Process regression." Institute for Health Metrics and Evaluation, Mortality Visualization FAQ.

• 18

  We have heard of errors in a MICS publication before (though not in the all-cause mortality tabulations), so approach any surprising results with skepticism. The raw data may be more reliable.

• 19

  Research from the DHS program suggests this is uncommon, but does occur (emphasis added): “The birth histories in most DHS surveys appear to be of excellent quality, although several surveys do show symptoms of omission, displacement, or both. With the selected indicators and criteria, omission of births is usually at the level of 2 percent or less, and only rarely exceeds 5 percent. Displacement of births is also usually 2 percent or less, and only rarely exceeds 3 percent of births in the past 10 years. The omission of deaths is a higher percentage; but usually less than 5 percent, although it may exceed 10 percent in some surveys. Displacement of deaths is usually less than 5 percent, and almost never more than 10 percent. With some unevenness across regions, DHS-6 appears to be the phase of DHS that had the lowest overall levels of incomplete birthdates, flagged dates of death, and omission and displacement of births and deaths." USAID, DHS Methodological Reports 11, 2014

• 20

  "Standard DHS Surveys have large sample sizes (usually between 5,000 and 30,000 households) and typically are conducted about every 5 years, to allow comparisons over time." The DHS Program, DHS Overview.

• 21

  NB: sometimes “direct” is used to describe estimates from full birth histories and “indirect” used for estimates from summary birth histories.

• 22

  Definition from The DHS Program, Guide to DHS Statistics.

• 23

  In general, for recent surveys, using 5-year averages without major adjustments should be okay. For example, if a survey is conducted in late 2023 and reports mortality from 2019-2023, that should be a reasonable proxy for mortality in 2021.

  If using a 10-year average, or an older DHS or MICS survey, we might want to adjust for the trend in mortality over time. We can consider doing this by looking at recent trends in disease estimates from IGME, IHME, or other surveys. For example, if we have an average mortality rate from 2008-2018, and our estimate is that mortality is decreasing by 2% per year, we might take 2013 as the midpoint of the years surveyed and adjust that downward by a factor of ~(1-2%)^(2021 - 2013).

  Often this won’t be a major problem, as the most recent surveys are the ones most likely to be omitted from IGME and IHME estimates, so we will not need to search for older DHS or MICS results.

• 24

  “The overall sample size should be in the range of 2,500 to 14,000 households." UNICEF, MICS3 Chapter 4 - Designing and Selecting the Sample, p. 24

• 25

  "The COMSA Strategy: JHU works with the National Institute of Statistics (INE) and the National Institute of Health (INS) in Mozambique to set up a representative nationwide sample of clusters for routine surveillance of pregnancies, birth outcomes and deaths." COMSA Mozambique, What We Do.

• 26

  “There are high variations in the tool utilisation and data accuracy at facility and district levels. The routine HMIS is weak and data at district level inaccurately reflects what is available at the source. These results highlight the need to design tailored and inter-service strategies for improving data quality.” Rumisha et al 2020

• 27

  Our general understanding and experience has been that countries with higher mortality tend to have weaker health data systems.

• 28

  Available here.

• 29

  For example, we might have an estimate of mortality in children aged 6-59 months, calculated based on IHME data, which we want to update to include other sources. We might then look at mortality estimates in children under age 5 from IGME and IHME. If taking a weighted average of those estimates yields a resulting mortality estimate that is 15% higher than the estimate from IHME alone, we could then adjust our IHME-based 6-59 month mortality estimate upward by 15%. We should highlight that this adjustment is based on the difference we found from the weighted average of 1-59 month mortality, and it’s possible that this difference is greater or smaller than what we would see if we were able to compare estimates for months 6-59 alone.

• 30

  For example, see here for the under-five mortality rate over time of Nigeria (UN IGME). This is labeled as "Deaths per 1,000 live births."

• 31

  "Because U5MR expresses risk over 5 years, whereas ASMR-U5 expresses risk per year, U5MR is often almost five times as large as ASMR-U5." World Food Programme, A Manual: Measuring and Interpreting Malnutrition and Mortality, p. 40

• 32

  "Because U5MR expresses risk over 5 years, whereas ASMR-U5 expresses risk per
  year, U5MR is often almost five times as large as ASMR-U5." World Food Programme (WFP) and Centers for Disease Control and Prevention (CDC), A Manual: Measuring and Interpreting Malnutrition and Mortality, p. 40. Intuitively, we think this discrepancy might be bigger in high mortality contexts, if the number of live births increases faster than the population.

• 33

  A discussion of this method is here: Tools for Demographic Estimation, Estimation of adult mortality from sibling histories.

• 34
  • "Nigeria and India had the highest numbers of maternal deaths, and accounted for approximately one third (35%) of all estimated global maternal deaths in 2017, with approximately 67000 (UI 48000 to 96000) and 35000 (UI 28000 to 43000) maternal deaths (23% and 12% of global maternal deaths), respectively." UN MMEIG, Trends in Maternal Mortality 2000-2017, p. 34-38.
  • IHME, conversely, estimates that approximately 26,000 maternal deaths occurred in Nigeria in 2017. See the graph here.

• 35

  While the work ostensibly compares the WHO and IHME estimates, the WHO estimates for children under 5 are derived from the IGME Child and Adolescent Causes of Death Estimation (CA CODE) project.
  "The number of malaria deaths among children aged under 5 years was calculated by applying the country-specific yearly malaria CoD fraction to the all-cause mortality envelope of 1–59 months estimated by the UN Inter-agency Group for Child Mortality Estimation." World Health Organization, World malaria report 2022, p. 135

• 36

  "The method proposed here is computationally complex, but it is executed in a single systematic framework that is used widely in other similar problems with compositional measures (Haan & Uhlendorff, 2006) and with attractive statistical properties that are well documented (Engel, 1988)." Mulick et al 2022, p. 2117

• 37

  See data point maps here.

• 38

  "For the high mortality model, we extracted relevant neonatal COD distributions from studies conducted in high mortality settings since 1980. Details of the literature review are found in (Oza et al., 2015). From the review, we identified 95 studies that reported causes for 100,119 neonatal deaths in 37 countries (range 1–18 studies per country) between 1980 and 2013." Mulick et al 2022, p. 2099

• 39

  "Data inputs:...Covariate data: We considered 14 explanatory variables for inclusion in our model." Mulick et al 2022, p. 2099

• 40

  See columns "Covariates not constrained by λ" and "Covariates constrained by λ" in the sections "Methods 0-1 Months" and "Methods 1-59 Months" of the appendix of Perin et al 2021.

• 41

  The details of the statistical model are fairly complicated. For more information on the model, see Mulick et al 2022, p. 2102, section 3.2, Derivation of Bayesian multinomial model with random effects. We haven’t thought through all the implications of this model in detail or had any expert review it. Our guess is that the limitations are driven by a lack of data – we think the model is going to be highly uncertain if there’s no historical verbal autopsy data in a given area, so we're not confident digging into the statistical modeling choices is likely to yield a lot of information. This could be wrong – there could be biases in the estimates that we’d understand by looking at the model more closely.

• 42
  • "We developed a new approach within a Bayesian framework for estimating the distribution of causes of death at national level based on COD data from national and subnational studies and data on covariates." Mulick et al 2022, p. 2114
  • "For the high mortality model, we extracted relevant neonatal COD distributions from studies conducted in high mortality settings since 1980…These studies typically used verbal autopsy (VA) methods to ascertain the cause of death." Mulick et al 2022, p. 2099

• 43

  See "Methods 0-1 Months" and "Methods 1-59 Months" in the appendix of Perin et al 2021.

• 44

  "Estimates for a small number of low-burden causes (e.g., measles, collective violence, or HIV/AIDS) were systematically integrated from WHO and UNAIDS." UN IGME, Methods for child and adolescent causes of death estimation.

• 45

  Liu et al. 2016, Supplementary Appendix, p. 4

• 46

  Perin et al 2021, Appendix 2, p. 30.

• 47

  Data access is account-limited, based on agreements between IHME and the organization requesting access.

• 48

  On this topic, in describing the UN methodology, Mulick et al 2022 write about GBD: “The proposed method is in contrast to those used by the Global Burden of Disease (GBD) consortium (Murray et al., 2020) for estimating the causes of mortality. This group looks at many separate causes for different age groups, including data from incomplete vital registration and registries for specific syndromes and aetiologies of disease. After all causes are estimated separately with Gaussian processes, they are then restricted to an age-specific envelope (total number of deaths due to all causes) in a separate process (Murray et al., 2020), although their methods and data sources are not publicly available in detail (Schwab, 2020). The method proposed here is not directly comparable to GBD methods because each are based on different source information. When approaching compositional data such as causes of death, addressing components individually as done by GBD is generally unbiased, but there are caveats for estimating the variance of separately estimated components which are subsequently fitted to an envelope (Begg & Gray, 1984; Fürnkranz, 2002; Hsu & Lin, 2002). The estimated variance for each component is used in the GBD method for harmonising the many causes to fit the mortality envelope (Murray et al., 2020) and so may introduce bias in the resulting estimates. The vast amount of input data used by GBD allows for the estimation of many different causes,which would be computationally difficult in the multinomial framework. However, the ‘squeezing’ process made necessary by the many different causes may lead to inaccuracies. The method proposed here is computationally complex, but it is executed in a single systematic framework that is used widely in other similar problems with compositional measures (Haan & Uhlendorff, 2006) and with attractive statistical properties that are well documented (Engel, 1988).”

• 49

  At a high-level, our understanding of this process is that it ensures that the total number of deaths due to each cause sums to the all-cause mortality. For example, if the only two causes of death were measles and pertussis, and there were 55 deaths from each cause estimated in the initial models, but the all-cause mortality estimate suggested a total of 100 death, then the number of deaths due to measles and pertussis would be adjusted so that they summed to 100 instead of 110 (=55 + 55).

  We are unsure about the exact details of this process, but that rough understanding is based on descriptions in published literature. For example, in describing estimates of the burden of prostate cancer, Safiri et al (2025) write: “The cause of death corrected (CoDCorrect) algorithm, which rescales the cause-specific mortalities, was utilised to align the overall number of single-cause deaths estimated for each sex-age-location-year group with the all-cause mortality estimates.”

• 50

  In addition to the published work in the previous footnote, the name “CODCorrect” is referenced in disease-specific methods descriptions. See, e.g., the description for measles here.

• 51

  “Verbal autopsy (VA) is a method used to determine the cause of death through interviews with the deceased person's next of kin or caregivers. These interviews involve a standardized questionnaire to gather details on symptoms, medical history, and the circumstances leading to death. Healthcare professionals or algorithms then analyze this information to identify the likely cause of death.” WHO, Verbal Autopsy Standard.

• 52

  For example, IGME includes the ICD-10 code A01 (typhoid and paratyphoid) in their diarrheal disease category, but IHME does not. See GBD 2019 Cause-ICD Codes Map – Causes of Death and UN IGME, Methods for child and adolescent causes of death estimation.

• 53

  For example, if there are many more deaths attributable to a given cause in a model that doesn’t make the one-cause restriction, we might think there is a larger scope for indirect deaths.

• 54

  For example, see subnational estimates for Nigeria here.

• 55

  They also rely significantly on vital registration data for some, mostly wealthier, countries. We think that IHME outputs are closer to “raw data” for major causes of death in these countries, though we haven’t looked into it closely.

• 56

  We think this because the IHME applies a number of adjustments to raw data to get final estimates. These adjustments take into account country characteristics and are smoothed across time and countries. We think both of these adjustments should in theory remove noise out of the raw data, particularly the “smoothing” process. Though, since we don’t understand the smoothing well, we're not sure if the process is likely to “over-smooth” (i.e. remove too many differences across time and space) or “under-smooth” (i.e. leave too much noise across time and space in the final estimates).

  “In stage 1, we use covariates to estimate what the mortality would look like given the levels of certain covariates including education level and income. Then we get the difference between stage 1 predictions and the observed raw data. These residuals are then smoothed over time and across countries within a GBD region. Finally, we add these smoothed residuals to the first stage prediction to generate second stage estimates…Stage 2 estimates are used as prior in a Bayesian model called Gaussian process regression. The end result from the Gaussian process regression is the final “GBD estimates”. IHME GBD, Frequently Asked Questions

• 57

  See this graph for South Sudan on the IGME site here.

• 58

  See this graph for Ghana on the IGME site here.

• 59

  For example, we can see in this adjustments table that the variation across sources is highly impactful in Chad. On the other hand, while there are a number of uncertainties in our vaccine adjustment, our best guess is that the adjustment is relatively small at this point and varies much less across states, so it’s unlikely to be a crux point for a grant decision.

• 60

  The grant page and cost-effectiveness analysis for this investigation are not yet published.

• 61

  See, e.g., this 2022 report on malaria in Nigeria from the WHO which notes, “the incidence of malaria is highest in the northern and north-eastern parts of the country." Borno is in the far northeast and Lagos is in the far southwest of the country.

• 62

  IHME estimates that all-cause deaths in Lagos in 2021 number 1.7k among children 1-12 months old, and 1.6k among children 1-4 years old. For malaria deaths, IHME estimates 753 among children 1-12 months old and 906 among children 1-4 years old. (753+906)/(1600+1700) = ~50%. Source: Institute for Health Metrics and Evaluation (IHME). Used with permission. All rights reserved.