Family Empowerment Media — Radio Transmitters for an RCT (March 2023)

Note: This page summarizes the rationale behind a GiveWell grant to Family Empowerment Media as of March 2023, when we made the grant. Family Empowerment Media staff reviewed this page prior to publication.

April 2024 update: One of the reasons we made this grant was to decrease the chance that Family Empowerment Media would need to delay its planned start date of November 2023 for its randomized controlled trial. However, the trial start date was ultimately delayed beyond November 2023 because Family Empowerment Media was still in the process of raising sufficient funds to cover the full costs of the trial at that time. The trial is now expected to begin in mid-2024.1

Published: April 2024

The organization

Family Empowerment Media (FEM) develops and broadcasts radio campaigns in Nigeria about contraception.2 FEM believes that providing accurate information about family planning through mass media campaigns can lead to increases in uptake of contraception, which in turn can reduce mortality and morbidity associated with unintended pregnancies, along with other potential benefits.3 FEM’s radio broadcasts include short advertisements, dramas, and Q&A shows.4

FEM launched in Nigeria in September 2020, after being founded through the Charity Entrepreneurship incubation program, inspired by a randomized controlled trial (RCT) of Development Media International's family planning radio campaigns in Burkina Faso.5 FEM estimates that it reached over 5 million listeners in its three-month pilot and subsequent nine-month campaign in Kano state in Nigeria beginning in May 2022, and had reached 20 million additional listeners through short test campaigns in other locations by July 2023.6 FEM works in partnership with local implementing organizations, radio stations, radio production firms, and state governments.7

FEM is planning to test the impact of its program with a cluster RCT in three regions of Nigeria: the South West, South East, and North.8 The RCT implementers plan to measure outcomes related to: (a) knowledge and attitudes about family planning, (b) use of contraception, and (c) number of pregnancies and spacing between pregnancies (based both on self-report and data from clinics).9 Outcomes will be measured at baseline, after 23 months of FEM's program, and one year after the completion of the trial.10

The grant

In March 2023, we recommended a $500,000 grant to FEM to close a time-sensitive funding gap for its planned RCT of its program, in order to prevent a delay to the study’s timeline and to help FEM get closer to closing its overall funding gap for the RCT.

The grant recommended by GiveWell funds the production of small-scale radio transmitters, to be used to randomize radio messaging during the trial.11 The transmitters replace family planning programming from radio stations with “control” content intended to be neutral (e.g., music) within a three- to six- kilometer radius of where the transmitters are placed.12

These transmitters should enable an RCT with a larger number of treatment and control clusters than the RCT of Development Media International's program in Burkina Faso, which was randomized at the level of community radio stations.13 FEM expects its study to include 165 broadcasting catchment areas to be randomized into treatment and control groups, whereas the Burkina Faso RCT included 16 clusters.14

The case for the grant

We are recommending this grant because:

• We think FEM’s program looks highly promising, and this RCT will allow us to learn more about it. Using tentative moral weights, our initial analysis of FEM’s program suggests that it could be significantly above our current cost-effectiveness bar, if FEM's program has a similar impact on contraceptive uptake as found in the RCT in Burkina Faso (mentioned above).15 We expect that we will be able to be more confident in our estimate of the impact of FEM's program in Nigeria after reviewing the results of FEM's RCT on contraceptive uptake.16
  • We also have a positive qualitative view of FEM as an organization, as we think it is well-aligned with GiveWell’s values, which gives us additional confidence in its potential for impact. Our impression is that FEM provided clear, transparent, and reasonable answers to our questions throughout our grant investigation.
  • We expect that the implementation of the RCT will also reveal valuable information about FEM’s ability to scale its program successfully. This information could affect our funding decisions regarding this and similar programs in future.
• A $500,000 grant narrows FEM's RCT funding gap. As of March 2023, our understanding is that other funders are also considering granting funds to FEM for its RCT. By closing part of FEM's RCT funding gap, we expect that we are increasing the likelihood that FEM is able to raise enough funding to cover the full costs of the RCT.
• Our understanding is that there are meaningful costs to delaying the timeline for the RCT. As of March 2023, FEM needs to acquire radio transmitters in the immediate future in order to proceed with its study timeline. It is our impression from correspondence with FEM that delaying the RCT could harm its organizational reputation with other funders and government partners.17 A period of organizational inactivity caused by a delay would also have led FEM to incur financial, time, and opportunity costs, including foregoing some potential impact of the program.18 Therefore, even compared to a scenario in which FEM subsequently closed this funding gap by other means, we felt our grant would still be beneficial, since we expect that an immediate decision from us would allow FEM to avoid delaying the RCT.

  Our understanding is that this funding gap is time-sensitive—absent our grant recommendation we expect FEM would need to push back its planned RCT start date of November 2023, a date chosen after consultation with local stakeholders.19 The funding gap for transmitters needs to be filled several months before the planned start date of the RCT in order to allow time for manufacturing and shipping.20 If the transmitters were delayed, FEM would need to renegotiate the trial’s start date, which could risk FEM's donor relationships and its credibility with its local partners in Nigeria.21

• Following up on the results of FEM’s RCT will lead us to learn more about family planning programs. GiveWell has not funded programs dedicated to family planning in the past. This RCT could lead us to learn more about family planning programs and potentially find impactful grantmaking opportunities that we have otherwise missed.

Risks and reservations

Our main reservations about this grant are:

• We have not conducted an in-depth review of either FEM or family planning interventions in general, so it’s possible our assessment would change with further research. Because of our limited prior work in this area, we are currently highly uncertain about the cost-effectiveness of family planning interventions such as this, and about the moral weights we should be using to evaluate their impact.

  • We expect that the results of the RCT will help us resolve some of our uncertainties about cost-effectiveness, but we are not sure how likely it is that the RCT will point to sufficiently strong cost-effectiveness for us to fund family planning programs in future. For example, our current assumption of the program’s impact on contraceptive uptake derives from the results of Development Media International's Burkina Faso study. Evidence from Nigeria will be considerably more relevant to FEM’s program, but at this stage we are unsure whether it will make FEM’s program look more or less cost-effective.
  • Our tentative moral weights for increased contraceptive uptake among people who would like to avoid pregnancy are based on estimates of averted maternal mortality and morbidity, along with some value placed on personal autonomy.22 We are highly uncertain about our estimates and we do not expect the RCT to resolve our uncertainty about the moral weights we should be using for this type of program.
• We have only reviewed FEM's planned methodology for its RCT at a high level. We have not carefully investigated specific survey questions FEM plans to use, its calculations of statistical power, or its plans to mitigate potential spillover effects.23
• Other donors might eventually have closed FEM’s RCT funding gap in full anyway. As of March 2023, our best guess is that there is roughly 40% chance that FEM could close its full $2.8 million RCT funding gap by July without our support. In this scenario, the main impact of our grant would have been to save another impact-focused funder $500,000.
• We did not do a full review of FEM's operating budget to determine if it could have used its own funds for these radio transmitters. We accepted FEM’s explanation that it could not, due to the risk that the remainder of its RCT funding gap would not be filled, and the fact that many of FEM’s existing funds were donated for specific other purposes.24
• When we made the grant, we could not be certain about whether FEM would receive the rest of the funding it required for the RCT, either from GiveWell or from other funders. Without this further funding, our grant would have had minimal impact, since the radio transmitters are only useful to FEM for the RCT.

Plans for follow-up

We plan to check in with Family Empowerment Media approximately every six months while the RCT is being prepared and conducted. The initial data collection is scheduled to occur 23 months after the beginning of the trial, with a second round of collection 12 months later. We will review this data once we receive it from FEM. This grant is non-renewable.

Our process

In May 2022, GiveWell published a short note stating that family planning radio campaigns did not appear likely to be highly cost-effective based on our initial analysis, and that we did not plan to prioritize additional work on the program.

We reopened our investigation into family planning radio campaigns in August 2022 when Anna Christina Thorsheim, Executive Director and Co-founder of Family Empowerment Media, reached out to us to let us know that she thought FEM was more cost-effective than our model suggested, due to FEM's lower costs per person reached and high maternal mortality rates in areas where FEM works.25

We approved this grant after a shallow investigation, in which we had several calls and exchanged emails with FEM in order to better understand its funding gaps. We did not resolve our wider uncertainties about the cost effectiveness of family planning interventions in general, nor of the moral weights we should be applying to programs in this area (as we mention above).

Sources

Document	Source
Anna Christina Thorsheim, conversation with GiveWell, August 15, 2022 (unpublished)	Unpublished
Anna Christina Thorsheim, conversation with GiveWell, March 24, 2023 (unpublished)	Unpublished
Anna Christina Thorsheim, email to GiveWell, August 18, 2022 (unpublished)	Unpublished
Anna Christina Thorsheim, email to GiveWell, February 22, 2023 (unpublished)	Unpublished
Anna Christina Thorsheim, email to GiveWell, March 16, 2023 (unpublished)	Unpublished
Charity Entrepreneurship, “Mass Media Campaign”, 2020	Source
Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.	Source (archive)
Family Empowerment Media, "Example of 60-second long ads"	Source
Family Empowerment Media, "Example of Stand Alone Drama Show"	Source
Family Empowerment Media, "Example of the running order for a Q&A show"	Source
Family Empowerment Media, "Funding application," 2023	Source
Family Empowerment Media, “Introducing Family Empowerment Media”, 2020.	Source (archive)
Family Empowerment Media, "RCT Budget CEA" 2023	Source
Family Empowerment Media, “RCT Questionnaire”	Source
Family Empowerment Media, "RCT Study Plan"	Source
Family Empowerment Media, "Results, reflections, and plans after 6 + months"	Source (archive)
Family Empowerment Media, diagram of transmitter technology	Source
GiveWell, "Development Media International - July 2021 Version"	Source
GiveWell, Approaches to moral weights: How GiveWell compares to other actors	Source
GiveWell, Family planning radio campaigns BOTEC with FEM Kano data	Source
GiveWell, GiveWell's cost-effectiveness analyses	Source
Glennerster et al. 2021	Source (archive)
Glennerster et al., "The Impact of a Family Planning Mass Media Campaign in Burkina Faso", 2023	Source (archive)
Innovations for Poverty Action 2019 "Augmenter la demande de contraception moderne au Burkina Faso grâce à une campagne médiatique"	Source

• 1

  Family Empowerment Media, comment on a draft of this page, March 18, 2024.

• 2
  • "We believe a well-run family planning charity focused on social and behavior change (SBC) campaigns on the radio could achieve outcomes in the realm of the most effective global health charities. Increasing access to and understanding of family planning services has been shown to have substantial positive effects on health, as well as education, income, and several other dimensions. SBC campaigns on family planning have generally been cost-effective, and a recent randomized control trial indicates that high-intensity radio-based SBC campaigns have the potential to be especially cost-effective." Family Empowerment Media, “Introducing Family Empowerment Media”, 2020.
  • "In general, family planning SBC interventions are typically effective in reducing unintended pregnancies and their associated maternal health burdens. SBC interventions seek to understand and facilitate change in behaviors and the social norms and environmental determinants that drive them.” Family Empowerment Media, “Introducing Family Empowerment Media”, 2020.
  • "In 2021, FEM launched our pilot: a 3-month family planning radio campaign, educating listeners about maternal health and effective contraception. Our ads and shows played up to 860 times in Kano state in northern Nigeria, reaching ~5.6 million." Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.

• 3
  • "Minimal or incorrect information about modern contraceptives is often a big barrier to greater use. Perhaps contrary to expectations, lack of access to contraceptive commodities is generally not a primary reason for low adoption. In Nigeria, just 5% of married women said physical access was a reason they did not use contraceptives. In contrast, 10% cited lack of knowledge on methods or sources of contraceptives; 20% cited health concerns. The latter is often connected with the former, as some women overestimate the side effects of contraceptives due to lack of knowledge. For instance, a study in two states in Nigeria found almost one in three women believed that contraceptives can make a woman permanently infertile. Unfortunately, there are countries and regions with major contraceptive stockout issues. However, by strategically picking areas where commodities are available, NGOs in this space can close information gaps and significantly increase contraception use." Family Empowerment Media, “Introducing Family Empowerment Media”, 2020.
  • "Across Nigeria, about half of women have an unmet need for effective contraception. Increasing access to contraception reduces unwanted pregnancies and increases space between births, making pregnancies much safer. It can save the lives of thousands of women and save others from devastating health conditions such as obstetric fistula, and postpartum anaemia and depression. When women and girls understand and access family planning services, they receive substantial positive effects: get more education, make more money, and enjoy the ability to take better care of their children." Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.

• 4

  For examples of FEM’s broadcasting, see here: drama, Q&A show, advertisement.

• 5
  • "FEM was conceived during the Charity Entrepreneurship Incubation Program in July and August and officially launched in September."
  • “After extensive research and helpful conversations with [Development Media International, the Johns Hopkins Center for Communications Programs, and BBC Media Action], we selected as FEM’s launch point a state in Northern Nigeria (Kano) where radio-SBC [social and behavior change] campaigns are underutilized.”
  • “From 2016-2018, Development Media International (DMI), a pioneer in conducting mass media SBC interventions, worked with Innovations for Poverty Action to test its campaign in Burkina Faso. DMI broadcasted ten family planning ads per day and three longer interactive programs per week on local radio stations. The results were dramatic. In treatment areas, use of modern contraceptives increased by 5 percentage points (pp), with belief that contraceptives make women infertile dropping 9 pp and belief they cause disease dropping 8 pp.”

  Family Empowerment Media, “Introducing Family Empowerment Media”, 2020.
  Our understanding is that FEM's program was started through Charity Entrepreneurship based on the research in Charity Entrepreneurship's report on family planning mass media campaigns: Charity Entrepreneurship, “Mass Media Campaign”, 2020.
  Charity Entrepreneurship's report discusses Development Media International's Burkina Faso RCT as a primary source of evidence for the effect of family planning radio campaigns. Charity Entrepreneurship, “Mass Media Campaign”, 2020:

  • "We found only one RCT on the effect of a radio campaign, in Burkina Faso. Although we take the rest of the literature into consideration as supporting evidence (see more detail in the ​weighted factor model’s strength of idea section​), we only used the point estimate from this RCT in our cost-effectiveness analysis. We believe it is the study that most rigorously estimated the causal impact of this kind of program." p. 26.
  • "Findings:​ The​ ​Burkina Faso RCT​ ​found a 5.9 percentage points increase in modern contraceptive use​.​ This translated into a reduction in contraceptive use gap of 8.4% (95% conf. interval: 2.5%-14.3%)." p. 26

  GiveWell has previously reviewed Development Media International's work on a different program, radio campaigns aimed at reducing child mortality. See our review here.

• 6
  • “Our ads and shows played up to 860 times in Kano state in northern Nigeria, reaching ~5.6 million. An independent survey showed that in 11 months, contraceptive use increased by ~75% among all women in the state. Since then, we launched a 9-month campaign, aired proofs of concept in three new locations and re-aired in 8 more states, reaching an estimated 20 million new listeners.” Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.
  • "In May 2022, we launched a nine-month campaign in Kano state, to an audience base of 5.6 million over 2600 times." Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.
  • Source for the July 2023 figure: FEM's “Empowering Numbers: FEM since 2021” blog post is dated July 2023.

• 7
  • "FEM collaborates with a wide range of stakeholders, from Nigerian state Ministries of Health and local religious leaders to engineering and tech institutions. We spotted impactful opportunities, and maintained these collaborations." Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.
  • "We will re-air our radio content with a different station partner in Kano, to create additional impact in northern Nigeria. We chose Arewa station, which reaches 4.6 million Nigerians, including over 1 million mostly rural women of reproductive age. We expect to avert ∼70 maternal deaths through this campaign." Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.

• 8

  “We are continuing to prepare for the randomized controlled trial of our intervention in three regions in Nigeria, with which we’ll reach an estimated 10-15 million listeners. We estimate this campaign will lead to ∼147,000 new contraceptive users, save ∼345 lives, and avert ∼5,000 unsafe abortions. It will also generate important, robust data about our intervention.” Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023.

  FEM identified the three regions as South West, South East, and North. Family Empowerment Media, comment on a draft of this page, March 18, 2024.
  
  Family Empowerment Media, "Funding application," 2023:

  • "In the RCT, we expect to reach 3.5 million listeners in Anambra, 3.5 million in Kaduna, and 4.2 million in Ondo State." p. 5.
  • "Research Question: The study will use a cluster randomized controlled trial to assess whether co-produced SBC campaigns (Activity 3 above) cause listeners to hear and understand FEM’s programs (Result 1, secondary outcome), increase their effective contraceptive use (Result 2, primary outcome), and increase interpregnancy spacing (Result 3, secondary outcome). We will also assess how contraceptive use and pregnancy spacing change a year after the campaign has ended.
  • Study Design: After a baseline survey, we will randomly assign health clinic catchment areas to receive FEM’s radio program for 23 months or a transmitter to replace FEM’s programs with music. The intervention period will complete with an end-line survey. Subsequently, we will not air campaign content in treatment and control regions for one year. We will complete the study with a follow-up survey that allows us to understand behavior change maintenance.
  • Sampling Frame and Randomization: We will sample 164 non-urban clinic catchment areas in three Nigerian states and randomly select 40 households with child-bearing-age women within each clinics’ catchment area. To build this sample frame, we will work with local government leaders and clinic officials to develop a list of eligible households.
  • Data Collection: We will collect both self-reported and clinic-level outcomes (see Section 8). For self-reported data, enumerators will conduct surveys assessing primary and secondary outcomes. These approximately one hour surveys will be conducted on encrypted tablets using SurveyCTO software. Self-reported survey data will complement routinely collected clinic-level data on contraceptive use, birth rates, and maternal health. Combining surveys with clinic reports will ensure data triangulation.
  • Analysis: We will estimate the effect of treatment on core outcomes using standard RCT evaluation methods; specifically, OLS regressions with standard errors clustered at the level of random assignment (i.e. clinic level). Following best practice, we will further increase statistical power by adjusting for baseline outcomes, randomization block fixed effects, and predetermined covariates (selected using machine learning with baseline outcomes), and calculate p-values using randomization inference.
  • Power Calculations: Our power calculations draw on the DMI’s Burkina Faso study. Assuming a 6% baseline rate and a 0.1 intra-cluster correlation, we estimate that 164 health clinic clusters (100 treatment, 64 control) and 40 respondents per cluster would detect a 3.1 pp increase in effective contraception usage with conventional standards for power (0.8) and statistical significance (0.05)." pp. 5-6.

• 9

  “Beyond this monitoring, we will measure the following:
  A change in knowledge, attitudes, and societal stigma
  Self-reported attitudes towards and approval of family planning, at the individual and (perceived) community levels
  Knowledge about modern contraception, based on an index of factual questions reflecting the content provided by FEM’s programs*
  An increased use of modern contraception
  Number of women who report currently using a modern method of contraception, as well as the type and frequency of the method used (including long-acting reversible contraception)
  Number of women who report intending to use contraceptives
  Clinic data on modern contraceptives received
  A reduction in pregnancies and an increased spacing of pregnancies
  Number of women reporting a new pregnancy after the intervention begins
  Clinic data on new pregnancies after the intervention begins
  Number of women reporting births spaced <21 months*
  Improved health and well-being

  • Clinic data on number of women receiving family planning counselling, antenatal care, or postnatal care
  • Clinic data on rates of anaemia (funding dependent)
  • Reported mental health*
  • Longitudinal: timely child vaccination*
  • Longitudinal: female labour force participation and household incomes*
  • Longitudinal: child education (e.g., graduation rates, passing grades)*

  Aspirational metrics are denoted by asterisks (*)"
  Family Empowerment Media, “RCT Study Plan”. p. 2.

• 10

  "Study Design: After a baseline survey, we will randomly assign health clinic catchment areas to receive FEM’s radio program for 23 months or a transmitter to replace FEM’s programs with music. The intervention period will complete with an end-line survey. Subsequently, we will not air campaign content in treatment and control regions for one year. We will complete the study with a follow-up survey that allows us to understand behavior change maintenance."Family Empowerment Media, "Funding application," 2023. p. 6.

• 11

  Anna Christina Thorsheim, email to GiveWell, February 22, 2023 (unpublished).

  The grant total of $500,000 is expected to cover a large majority of FEM's costs for procuring small-scale transmitters. The total for "Transmitter costs" in FEM's RCT budget for 2023 is $580,479. Family Empowerment Media, "RCT budget CEA" 2023, "RCT Full Budget" sheet (note that this sheet is unpublished).

• 12

  Anna Christina Thorsheim, conversation with GiveWell, August 15, 2022 (unpublished). For more detail, see this diagram, shared with us by FEM.

  • "After a baseline survey, we will randomly assign health clinic catchment areas to receive FEM’s radio program for 23 months or a transmitter to replace FEM’s programs with music." p. 6.
  • "Third, we developed a novel evaluation technology to enable randomized controlled trials (RCTs) on radio-based interventions in previously inoperable contexts. Most radio campaign evaluations use pre-post or geographic exposure-based designs as radio’s large reach challenges control group creation. With support from Grand Challenges Canada (GCC), FEM developed and piloted small-scale radio transmitters to assist control group creation. The transmitters temporarily override a station’s main signal in a selected four kilometer-radius area while campaigns air. This technology makes RCT with greater statistical power possible, laying the groundwork for radio-based public health program evaluation worldwide." p. 2.

  Family Empowerment Media, "Funding application," 2023

• 13
  • Anna Christina Thorsheim, email to GiveWell, August 18, 2022 (unpublished).
  • “To measure the impact of the radio campaign, researchers randomly allocated 16 community radio stations between an intervention group (eight stations) where the media campaign took place and a comparison group (eight stations) where regular programing continued.” Glennerster et al., “The Impact of a Family Planning Mass Media Campaign in Burkina Faso”, 2023.
  • "Most radio campaign evaluations use pre-post or geographic exposure-based designs as radio’s large reach challenges control group creation. With support from Grand Challenges Canada (GCC), FEM developed and piloted small-scale radio transmitters to assist control group creation. The transmitters temporarily override a station’s main signal in a selected four kilometer-radius area while campaigns air. This technology makes RCT with greater statistical power possible, laying the groundwork for radio-based public health program evaluation worldwide." Family Empowerment Media, "Funding application," 2023, p. 2.

• 14
  • “We will sample 165 non-urban clinic catchment areas in three Nigerian states.” Family Empowerment Media, “RCT Study Plan”.
  • “To measure the impact of the radio campaign, researchers randomly allocated 16 community radio stations between an intervention group (eight stations) where the media campaign took place and a comparison group (eight stations) where regular programing continued.” Glennerster et al., “The Impact of a Family Planning Mass Media Campaign in Burkina Faso”, 2023.

• 15

  See our back-of-the-envelope calculation here: GiveWell, Family planning radio campaigns BOTEC with FEM Kano data. Note that a) our cost-effectiveness analyses are simplified models that are highly uncertain, and b) our cost-effectiveness threshold for directing funding to particular programs changes periodically. As of late 2023, our bar for directing funding is programs that are about 10 times as cost-effective as unconditional cash transfers. See GiveWell’s Cost-Effectiveness Analyses webpage for more information about how we use cost-effectiveness estimates in our grantmaking.

• 16

  "Result 2) Increased use of effective contraception
  R2.in1: Number of women reporting that they have obtained effective contraceptives, and the regularity of obtaining contraceptives
  R2.in2: Number of women reporting effective contraceptive use, as well as the regularity of contraceptive use
  R2.in3: Number a of women reporting intention to use contraceptives
  R2.in4: Number of contraception requested at clinics associated with a given catchment area" Family Empowerment Media, "Funding application," 2023, p. 8.

  "Across Nigeria, about half of women have an unmet need for effective contraception. Increasing access to contraception reduces unwanted pregnancies and increases space between births, making pregnancies much safer. It can save the lives of thousands of women and save others from devastating health conditions such as obstetric fistula, and postpartum anaemia and depression. When women and girls understand and access family planning services, they receive substantial positive effects: get more education, make more money, and enjoy the ability to take better care of their children." Family Empowerment Media, “Empowering Numbers: FEM since 2021”, 2023."

• 17

  Anna Christina Thorsheim, email to GiveWell, March 16, 2023 (unpublished).

• 18

  Anna Christina Thorsheim, email to GiveWell, March 16, 2023 (unpublished).

• 19

  Anna Christina Thorsheim, email to GiveWell, February 22, 2023 (unpublished).

• 20

  Anna Christina Thorsheim, email to GiveWell, February 22, 2023 (unpublished).

• 21

  Anna Christina Thorsheim, email to GiveWell, March 16, 2023 (unpublished).

• 22

  See our tentative moral weight for a "couple-year of protection" (CYP) in our BOTEC for FEM here: GiveWell, Family planning radio campaigns BOTEC with FEM Kano data. We have not yet made the calculations underlying this moral weight estimate public.

• 23
  • "Power Calculations: Our power calculations draw on the DMI’s Burkina Faso study. Assuming a 6% baseline rate and a 0.1 intra-cluster correlation, we estimate that 164 health clinic clusters (100 treatment, 64 control) and 40 respondents per cluster would detect a 3.1 pp increase in effective contraception usage with conventional standards for power (0.8) and statistical significance (0.05)." Family Empowerment Media, "Funding application," 2023. p. 7.
  • "Sampling Frame. We will sample 165 non-urban clinic catchment areas in three Nigerian states. We will select clinics where (a) rural-to-urban migration is low (to mitigate risk of spillovers), (b) the distance from the radio station’s transmitter enables FEM’s transmitter to replace content for relatively large areas (to further mitigate risk of spillovers), (c) radio listenership is high, and (d) health clinics have a consistent supply of modern contraceptives and sufficient staffing to provide counselling. Within each experimental clinic catchment area, 40 households with women of child-bearing-age will be randomly sampled to generate a panel of survey respondents." Family Empowerment Media, “RCT Study Plan”. p. 1.
  • Family Empowerment Media, “RCT Questionnaire” Note that this questionnaire is an early draft version and the final questionnaire will be different (Source for this claim: Family Empowerment Media, comment on a draft of this page, March 18, 2024).

• 24

  Anna Christina Thorsheim, conversation with GiveWell, March 24, 2023 (unpublished).

• 25

  Anna Christina Thorsheim, email to GiveWell, August 18, 2022 (unpublished).