IRD Global — Breastfeeding Reminders Pilot in Sindh, Pakistan (March 2024)

Published: July 2024

1. Summary

1.1 What we think this grant will do

The World Health Organization (WHO) recommends initiation of breastfeeding at birth, exclusive breastfeeding for six months, and continued breastfeeding up to two years of age or beyond. The risks from not adhering to the recommended breastfeeding behaviors are likely to be higher in low-resource environments with poor sanitation and worse substitute foods.

This grant will fund IRD Global (IRD) to run and monitor a one-year pilot of SMS reminders for breastfeeding promotion in four districts in Sindh, Pakistan. The pilot would target caregivers of children up to 23 months of age with two-way SMS messages by leveraging IRD’s existing immunization registry.

We think that breastfeeding promotion could reduce the share of infants not being exclusively breastfed, which could lead to reductions in infant mortality.

1.2 Why we made this grant

Our best guess is that the direct benefits from SMS reminders are likely to be around 13 times as cost-effective as unconditional cash transfers (GiveWell’s benchmark for comparing different programs) in Sindh, Pakistan. At the time of writing this page, GiveWell’s funding bar is around 10 times as cost-effective as cash.

The intuition for this high cost-effectiveness is that:

• SMS reminders are inexpensive. The cost per child reached is low because the program is focused on digital reminders, and IRD already collects phone numbers for caregivers of infants as part of its existing programs. We estimate that the program would cost around $2.50 per child reached in early infancy (which is the age group who we think is most likely to benefit from the program). (more)
• Mortality during the first year of life is high in Sindh, Pakistan. We estimate that neonatal mortality rates are approximately 1.7% and that postneonatal mortality rates are approximately 1.5%. (more)
• We think that SMS reminders could increase the share of new parents exclusively breastfeeding. We estimate that exclusive breastfeeding (EBF) rates in the districts where IRD plans to implement reminders are low (we estimate around 34%), and SMS reminders could reduce the share of new parents not exclusively breastfeeding by roughly 8%.1 (more)
• We think that decreasing the share of new parents not exclusively breastfeeding is likely to reduce infant mortality by a modest amount. In Sindh, Pakistan we estimate that the decrease in non-exclusive breastfeeding rates from digital promotion is likely to reduce neonatal mortality by around 1% and postneonatal mortality by around 4% because breastfeeding reduces the number of children drinking water with harmful pathogens. (more)

A sketch of our cost-effectiveness analysis and level of uncertainty is in the table below. This analysis requires taking a stand on several uncertain parameters that could change cost-effectiveness substantially. These key parameters are in orange.

You can see our cost-effectiveness analysis for the program here and a simple version here. More on our cost-effectiveness estimate is here.

The other main factor informing our decision to make this grant is:

• Learning value. As part of the grant, IRD plans to contract with an external evaluator to design and conduct an evaluation of the pilot. We think this pilot could be a good opportunity to learn about the magnitude of the effect of SMS reminders on breastfeeding rates. If the evaluation finds effects on exclusive breastfeeding rates that are as large or larger than we estimate, then we could continue to fund or direct more funds to IRD’s platform. We might also consider similar programs in other contexts. (more)

1.3 Main reservations

• Limited evidence on key mechanisms. There is limited evidence on the effectiveness of two-way SMS reminders at increasing exclusive breastfeeding rates. We also don’t have direct and relevant experimental evidence on the link between breastfeeding reminders and mortality outcomes. (more)
• Potential downsides. The program will target all children up to two years of age with SMS reminders, and there could be downside risks of sending messages to caregivers who have stopped breastfeeding (such as shaming parents or annoying people with messages). We think this risk will be somewhat mitigated through IRD’s plans to ensure messages are designed and targeted for specific age cohorts prior to a pilot, and through allowing caregivers to opt out. (more)
• Limited resources on the initial investigation. Given the size of the grant and opportunity to learn more through piloting, we tried to spend a proportional amount of staff time on the initial investigation, but this might mean we have missed key considerations. (more)

2. Planned activities and budget

2.1 Background

According to WHO and UNICEF, increasing breastfeeding rates is critical to improving health, nutrition, and mortality outcomes.2 WHO reports that breastfeeding increases the nutrition and immune system of newborns.3 WHO and UNICEF recommend initiation of breastfeeding at birth, exclusive breastfeeding for six months, and continued breastfeeding up to two years of age or beyond.4

While these recommendations apply to all infants worldwide, the risks from not adhering to recommended breastfeeding behaviors are likely to be higher in low-resource environments with poor sanitation and worse substitute foods.5 According to UNICEF, around half of newborns in low- and middle-income countries (LMICs) are not breastfed at birth and around half of children younger than six months are not exclusively breastfed.6

For more information on our views on breastfeeding promotion, see our separate report.

We think Sindh province, Pakistan is likely to be a context where exclusive breastfeeding rates are low7 and water quality is poor.8 In Sindh, GiveWell currently funds IRD Global (IRD) to operate an electronic immunization registry (Zindagi Mehfooz, or ZM), provide SMS reminders for immunizations, and distribute mobile phone-based conditional cash transfers (mCCTs) . For more, see our report on IRD's mCCTs program. As part of our case for a previous grant, we hypothesized that these programs could be made more cost-effective by layering additional interventions such as SMS reminders for breastfeeding promotion.

2.2 What we think this grant will do

This grant of up to $800,0009 will fund IRD to run a one-year pilot which would test the impact of sending SMS reminders on breastfeeding rates. This program will leverage IRD's existing immunization registry platform to send two-way SMS messages (i.e., participants would be able to respond with questions) to pregnant people and caregivers of children aged under two.10 The platform is currently used to send two-way SMS reminders for immunization. We expect that the breastfeeding promotion messages will be tailored to caregivers based on the age of the child11 and that the messages will provide caregivers information about the benefits of breastfeeding and address challenges to breastfeeding. The specific message content will be developed through qualitative interviews and focus group testing,12 which we plan to review and provide feedback on prior to implementation. IRD plans to hire and train helpline operators to address questions from caregivers, which we think will allow the program to address specific concerns around or barriers to breastfeeding.13

Based on data from its ZM platform, IRD estimates that it will reach between 660,000 and 700,000 caregivers of children up to 23 months of age with phone numbers.14 This includes caregivers of children in IRD's existing ZM system, caregivers of children who will be newly enrolled in its system over the course of the pilot, and caregivers who are reached through the Pregnant Women and Birth Registry (PWBR).15

IRD will run focus groups to develop the messaging16 , and then pilot the program in four districts (Karachi Central, Korangi, Jacobabad and Dadu) while another four districts (Karachi East, Keamari, Kambar, and Khairpur) would be used as controls. As part of this grant, IRD also plans to contract with an external evaluator to conduct an evaluation of the pilot. The evaluation will involve baseline and endline household surveys, measuring outcomes such as exclusive breastfeeding rates, early initiation of breastfeeding, and continued breastfeeding.17 For more on what we plan to learn from this pilot, see the section on learning value.

The total budget includes:

• Around $459,360 for IRD to implement and run internal monitoring of the program.
• We have asked IRD to contract an external evaluator, and expect this to cost up to $350,000.

3. The case for the grant

3.1 Cost-effectiveness

Our best guess is that the grant will be around 13 times as cost-effective as unconditional cash transfers (GiveWell’s benchmark for comparing different programs).18 At the time of writing this page, GiveWell’s funding bar is to fund grants that we estimate to be around 10 or more times as cost-effective as cash transfers.19 The main benefit we expect from this grant is a reduction in infant mortality due to lower consumption of poor-quality water that has an elevated risk of causing diarrhea. Our best guess is that the grant will avert a death for around $3,000.

Intuitively, we think the program is highly cost effective because SMS reminders are relatively cheap per person targeted; the rate of exclusive breastfeeding is low in Sindh, Pakistan; and we expect that IRD’s program could increase exclusive breastfeeding rates, which we expect would reduce mortality.

To estimate the impact of breastfeeding promotion on mortality and other benefits including development benefits and other excluded effects, we relied largely on our existing work on breastfeeding promotion programs, and adapted our model to better reflect IRD’s program. We generally followed the same approach, but made adjustments for the context and program modeled, including:

• Our report on breastfeeding promotion models the effects of a 1-way SMS reminders program. Because IRD plans to pilot 2-way SMS messages, we use the treatment effect estimated from the 2-way reminders treatment arm in Unger et al. 201820 to estimate the effect of reminders on increasing exclusive breastfeeding rates. The treatment effect estimated for two-way digital reminders was a 36% reduction in children not exclusively breastfed at 6 months21 , which exceeded our estimate of the pooled effect size of in-person breastfeeding promotion interventions on the same outcome (a 27% reduction22 ). We find it unintuitive that a digital reminders program would have larger effects than in-person interventions23 , so we capped the treatment effects based on the in-person treatment effects estimate. This is a speculative adjustment, and we have not reviewed the studies for in-person interventions in-depth. It’s possible that digital reminders programs would have larger effects,24 and we plan to learn more about the program’s impact through the pilot. (For more, see here)
• We made speculative adjustments to include benefits for caregivers who are reached for breastfeeding promotion when their children are aged 1-6 months.25

A full sketch of the cost-effectiveness model is below:

• We assume that during the one-year pilot, IRD’s program will target almost 700,000 children, and that around 140,000 families will receive messages when the child is less than 1 month old26 . Including a multiplier for additional children27 , we think that around 180,000 children will be impacted by the program in early infancy.
• We estimate that a digital breastfeeding promotion program results in roughly a 27% reduction in the non-exclusive breastfeeding rate. After internal and external validity (IV/EV) adjustments, we expect IRD’s program would decrease the share of infants not being exclusively breastfed by roughly 8%.28
• We estimate that mortality rates in the postneonatal infancy period are 1.5% in the districts supported, and that reducing the rates of infants not being exclusively breastfed through breastfeeding promotion would reduce all-cause mortality by 3.5%.29
• We estimate that neonatal mortality is around 1.7% in the districts supported, and that reducing non-exclusive breastfeeding rates would reduce all-cause mortality by 1.0%.30
• As a result, we think this program would avert around 120 deaths.31
• We estimate that the program costs around $2.50 per child directly or indirectly reached in early infancy.32
• This implies a cost of around $3,700 per death averted33 , which is around 8 times as cost-effective as unconditional cash transfers.
• We then incorporate additional benefits, including development benefits, mortality benefits for children reached between 1 and 6 months of age, and other excluded effects (such as morbidity averted and deaths averted in the post infancy period).34 Our bottom line is that the program is around 13x.

IRD’s program is expected to reach caregivers of children up to 2 years of age; however, we do not currently explicitly model benefits for caregivers reached when their children are over 6 months of age. This is because our existing research on breastfeeding promotion focused on benefits for the youngest children.35 While there may be benefits for older children who are breastfed for longer periods of time, we are uncertain what the magnitude and type of benefits should be at the time of this grant and have decided to not conduct additional research to resolve our uncertainties because the program already appears highly cost-effective and additional research was unlikely to change our recommendations for this grant. This may mean we are underestimating the program’s benefits. For more, see our risks and reservations.

3.2 Learning Value

We think this pilot has potentially high learning value by allowing us to spend a little to reduce our uncertainties about the impact of SMS reminders on breastfeeding rates, and the costs of such a program. Because breastfeeding rates are low in many contexts with poor quality water, by reducing our uncertainty we could potentially fund future SMS reminders programs including in other locations if we learned the program was highly cost-effective. A positive update on breastfeeding promotion could also increase our confidence in the case for IRD’s ZM platform more generally.

As part of this grant, IRD plans to contract with an external evaluator to help design and run an evaluation of the pilot program. We think working with an external evaluator reduces the risk of bias and will increase our confidence in the reliability of any evaluation results. For more on what we hope to learn, see our plans for follow-up.

At a high-level, we expect the evaluation to include baseline and endline surveys in randomly selected households in the four intervention and four control districts. The primary outcome will be the exclusive breastfeeding rate for children under 6 months.36 The impact of SMS reminders on breastfeeding rates would likely be estimated by comparing the share of infants exclusively breastfed between the intervention and control districts at the baseline and endline surveys, and we expect the evaluation to be powered to detect a 5% difference in breastfeeding rates.37

Specific questions we hope to answer as a result of this grant include:

• What impact will SMS reminders have on exclusive breastfeeding rates? This is the most important outcome we hope to learn about. We are uncertain about the magnitude of breastfeeding rate increases we should expect from an SMS reminders program. For more about our uncertainties, see here.
• What is the cost per child of sending SMS reminders for breastfeeding? We expect to learn whether IRD is able to reach the anticipated number of caregivers over the course of the pilot. This would inform us about the effectiveness of the program at reaching children and therefore about the estimated cost per child for SMS reminders programs.

Note: As of the writing of this grant page (July 2024), the evaluation plan is not yet finished and details as described above are subject to change. We intend to publish the finalized plan on this page once it becomes available and note key differences if applicable.

3.3 We think IRD is well-placed to pilot and monitor this intervention

Through its existing ZM platform, IRD is well-placed to reach a large number of caregivers with infants at or near birth through SMS reminders. IRD already targets caregivers with SMS reminders for immunizations, so we think it’s likely that its program could effectively target caregivers with breastfeeding promotion reminders.
In addition to its routine monitoring of the ZM and mCCTs programs, IRD has experience conducting evaluations with external partners38 , including running RCTs.39 IRD has provided us with a detailed proposal for monitoring of the pilot, and we think it’s likely that we’ll be able to learn about the outcomes we’re most interested in through the planned activities.

4. Risks and reservations

4.1 Limited direct evidence of effectiveness

Uncertainty about benefits of digital reminders programs on breastfeeding rates and mortality. We have limited evidence on the effectiveness of two-way SMS reminders at increasing exclusive breastfeeding rates, particularly for contexts similar to present-day Sindh where IRD plans to pilot its program. We also don’t have any direct and high-quality experimental evidence linking breastfeeding promotion to mortality outcomes.

• Limited evidence on digital reminders on exclusive breastfeeding rates. We have based our estimate of the uptake in breastfeeding from digital reminders on one paper (Unger et al. 2018, which is a relatively small study). We have seen two (similarly small) additional studies released in 202340 which find similar or larger magnitudes of breastfeeding rate increases. Both of these studies tested only one-way reminders, so we have chosen not to average the effects for now.

• Adoption of other digital communications platforms may limit the effectiveness of SMS reminders. In addition to the limited evidence on digital reminders, we’re uncertain about the effectiveness of SMS messages, specifically, in present-day Pakistan as people rapidly adopt other communications platforms like WhatsApp.
  • One expert we spoke to noted that SMS reminders could be ineffective because people in similar areas increasingly favor WhatsApp over SMS.
  • IRD told us that WhatsApp was common and that it made a WhatsApp number available for responses within its program. However, IRD also told us that WhatsApp likely had lower penetration in rural areas.41 IRD mentioned that it is trialing WhatsApp to send reminders, and we plan to discuss this with IRD during follow-up.42

• No high-quality empirical evidence on the impact of breastfeeding promotion on mortality in low- and middle-income countries. Our model combines evidence on (a) SMS reminders on breastfeeding rates, (b) in-person breastfeeding promotion on diarrhea rates, and (c) the link between diarrhea morbidity reduction and all-cause mortality reductions as per our water-quality work. This pilot study would allow us to refine our estimate of (a), but not the link to morbidity or mortality outcomes. This seems like a reasonable first step, but it is possible we would want further evidence generation before directing too much funding to this program type. In particular, we think it’s plausible that reductions in the share of new parents not exclusively breastfeeding would result in lower diarrhea risk; however, the evidence we’ve reviewed does not show a strong correlation between changes in exclusive breastfeeding rates and changes in diarrhea morbidity. For more on how we estimate the impact of breastfeeding promotion on mortality, see our separate report.

Uncertainty about the benefits for older children. We also aren’t sure what the benefit is for sending SMS messages to caregivers of older children. Since the program already looks promising prior to incorporating these additional benefits, we’ve decided not to conduct any additional research to address our uncertainties. We may do more research if we decide to direct significantly more funding to breastfeeding promotion. We have two key uncertainties here:

• Caregivers who have ceased breastfeeding may not be able to start again. Many caregivers of older children will have stopped breastfeeding, and would be unable to restart. IRD told us that if the gap is short they may be able to restart,43 but we don’t have a good sense of what share of parents this might represent. It may also be the case that there are spillover benefits to other caregivers in someone’s network, but we think this is quite speculative and we have not accounted for this possibility in our model.

• Uncertainty about the type and magnitude of benefits. We’ve chosen not to model benefits for children over 6 months of age because we’re very uncertain what this benefit should be. Our existing work focuses on the benefits of increasing the share of people exclusively breastfeeding. It does not consider benefits from people breastfeeding for longer periods, or for predominantly breastfeeding instead of not breastfeeding.
  • IRD has told us that there is evidence for benefits in older infants.44 An external expert we spoke to also mentioned neurocognitive benefits for children who were breastfed for longer. We have not investigated the evidence for benefits to older children at the time of this grant because we estimate that the pilot program is cost-effective even without modeling these benefits. This could mean we are underestimating the benefits to an SMS reminders program, and we may do more research on the benefits to older children if we consider additional funding.

4.2 Potential for risk of harm

We think there could be some risks of harm from shaming or annoying caregivers if reminders are sent, in particular to caregivers of older children. There could also be some risk of harm if very malnourished mothers are pressured to breastfeed. We think that there are risks of sending messages to people who have stopped breastfeeding, such as shaming people about their caregiving or annoying people with messages that make them less inclined to read other messages (such as the immunization reminders also delivered through IRD's platform). We also think there are potential harms associated with breastfeeding promotion in general, including risks of malnourishing infants or mothers if advice is followed too strictly.45

• IRD has said that it plans to conduct qualitative in-depth interviews and focus groups to test messages prior to the pilot, as well as tailor messages to specific age cohorts and allow caregivers to opt out.46 We plan to review the messages IRD plans to send prior to the pilot. We think the risk of harm is likely to be low for a small pilot, but we plan to ask if IRD can learn about this in its pilot monitoring and would re-evaluate concerns if we decided to fund a larger rollout.

4.3 A limited grant investigation

We have tried to calibrate the amount of staff time spent on research to match the size of the grant, so it is possible we have missed key considerations. We spoke to one external expert and reviewed some additional research on SMS reminders for breastfeeding promotion, but otherwise conducted fairly minimal desk research for this grant investigation beyond our existing report on breastfeeding promotion. This is because the grant size was relatively small, we have already conducted a moderate level of research on key uncertainties through our existing work, and we think that we are better able to learn about our key uncertainties through piloting rather than additional desk research. We also did not consider whether to fund a more rigorous evaluation. This evaluation and program are both relatively low cost, and we believe the suggestive evidence provided by the planned evaluation will be sufficient to decide whether to continue funding the program. It is possible that this means we have missed important considerations. If we were to significantly scale up funding for breastfeeding promotion, or work with partners in other geographies, we may consider additional evidence generation.

5. Plans for follow up

We plan to follow-up on some uncertainties (such as what specific messages will be sent, whether this mitigates risks of annoying or shaming caregivers, and whether IRD has any plans to test sending WhatsApp message reminders) while the pilot design is being finalized.

IRD plans to contract an external evaluator to evaluate the pilot over the course of a year (see above). We will review the evaluation protocol, and provide any feedback before the protocol is finalized. We expect that the monitoring and evaluation results from the pilot would update us on a number of importants inputs to the CEA, which could help us resolve some uncertainty about the program’s impact and cost-effectiveness. In particular, we expect to learn:

• What is the effect of the program on exclusive breastfeeding rates for children under 6 months?
• What is the program cost per child reached?
• Did the expected number of people actually receive messages?
• How many people opt out of receiving messages?
• What is the baseline rate of exclusive breastfeeding in Sindh, Pakistan?

6. Internal forecasts

7. Our process

We identified this grant opportunity through conversations with IRD. We reviewed an initial proposal for breastfeeding reminders and spoke with IRD on several occasions to address our questions.

To generate a grant-specific cost-effectiveness estimate, we used our existing cost-effectiveness model for breastfeeding promotion and updated various parameters to match the specifics of this program.

We conducted a light review of additional studies on digital reminders that we were made aware of, and had one call with an external expert, Dr. Ankur Sooden, who at the time of the call was serving as a Project Lead for John Snow India Pvt. Ltd. (JSIPL), on breastfeeding promotion and immunizations programs. We did not conduct an in-depth review, or do a comprehensive search for additional studies.

For internal review, a Senior Program Officer gave feedback on various aspects of the grant investigation.

Sources

• 1

  We express the treatment effect this way because it captures the effect of the program independent of the baseline exclusive breastfeeding rate. We assume that an SMS reminders program’s effectiveness will differ across populations with different baseline rates of exclusive breastfeeding. In particular, we assume that a reminders program will have a smaller impact in areas where fewer children are not exclusively breastfed at baseline (i.e. higher levels of baseline exclusive breastfeeding rates). Expressing the effect size in terms of a reduction in the not exclusively breastfed population is a relatively simple way to capture this relationship.

  In the context where IRD will implement the program, an 8% reduction is equivalent to a 5 percentage point increase in the number of people exclusively breastfeeding (the baseline exclusive breastfeeding rate is 34%, and (1 - 34%) * 8% = 5 percentage points), and we would expect the program’s effect to be smaller if baseline exclusive breastfeeding rates were higher; however, we’re uncertain about the exact magnitude of the effect of IRD’s program.

• 2

  "Breastfeeding is critical for achieving global goals on nutrition, health and survival, economic growth and environmental sustainability. . . . Inadequate breastfeeding practices significantly impair the health, development and survival of infants, children and mothers. Improving these practices could save over 820 000 lives a year." The World Health Organization and United Nations Children's Fund, Implementation Guidance: Baby-Friendly Hospital Initiative, 2018, p. 8.

• 3

  “Breast milk contains all the nutrients that an infant needs in the first 6 months of life. . . . Breast milk also contains bioactive factors that augment the infant's immature immune system.” The World Health Organization, Infant and Young Child Feeding: Model Chapter for Textbooks for Medical Students and Allied Health Professionals, 2009.

• 4

  “WHO and UNICEF recommend: early initiation of breastfeeding within 1 hour of birth; exclusive breastfeeding for the first 6 months of life; and introduction of nutritionally-adequate and safe complementary (solid) foods at 6 months together with continued breastfeeding up to 2 years of age or beyond.” The World Health Organization, "Infant and young child feeding fact sheet," 2023.

• 5

  "Several mechanisms for a possible protective effect of breastfeeding against gastrointestinal infections have been proposed, including the presence in breastmilk of substances with antimicrobial or immunological properties, avoidance of contamination (as in non-human milk or baby bottles), and the general nutritional status of breastfed infants. . . .

  [N]on-breastfed infants are more exposed to pathogens that may cause diarrhea than breastfed subjects. Many studies attest to the presence of pathogens in foods offered to infants. For example, in The Gambia, Rowland et al observed that weaning foods traditionally given to children were contaminated with microorganisms that could cause gastrointestinal infections. Another study from Chile showed that most feeding bottles harbored large numbers of pathogens that could cause gastrointestinal infection.

  Last, it has been proposed that in low-income settings optimal breastfeeding practices can prevent undernutrition associated with repeated infections and with the use of over-diluted breastmilk substitutes. Good nutrition is essential for non-specific immunity that contributes to fighting infections in general." Horta and Victora 2013, p. 12.

• 6
  • Initiation at birth: UNICEF Global database on infant and young child feeding, “Early initiation of breastfeeding (birth)” spreadsheet, “Latest Regional Global” tab, “World Bank (FY22) income grouping aggregates” section, average of "Low Income," "Lower Middle Income," and "Upper Middle Income" rows. UNICEF, Global database on infant and young child feeding, “Early initiation of breastfeeding (birth)” spreadsheet, 2021.

  Average: (59 + 49 + 39)/3 = 49

  • Breastfeeding at six months: UNICEF, Global database on infant and young child feeding, “Exclusive breastfeeding (< 6 months)” spreadsheet, “Latest Regional Global” tab, “World Bank (FY22) income grouping aggregates” section, average of "Low Income," "Lower Middle Income" and "Upper Middle Income" rows. UNICEF, Global database on infant and young child feeding, “Exclusive breastfeeding ( 6 months)” spreadsheet, 2021.

  Average: (52 + 50 + 26)/3 = 42.67

• 7

  According to the 2018-19 Multiple Indicator Cluster Survey, 46.5% of children aged 0-5 months were exclusively breastfed in Sindh (see Table TC.7.3).

• 8

  Based on a light search of water quality estimates for Sindh, we believe that water quality is likely to be poor in many areas. We checked the first few pages of google and google scholar searches for “Water quality Sindh Pakistan” and reviewed abstracts but not full papers or methodologies:

  • Khan et al 2018 - “Our result shows that in some cities like Badin, Ghotki, Jacobabad, Khairpur, Mirpurkhas, Mithi, Tharparkar (without RO) Sangar, Thatta, water is unfit for drinking purpose as water quality parameters exceeding the prescribed standard values. Total viable count test was performed for microbial analysis and it was found that the sample from Badin, Ghotki, Jacobabad, Sanghar and Thatta were heavily loaded by the microbial growth of faecal coliforms and Escherichia coli. Whereas in other cities includes Karachi, Hyderabad, Shikarpur, Sukkur; water quality parameters fall within the prescribed standard values and no faecal contamination was found.”
  • Daud et al 2017 - “About 67%–93% of samples collected from different locations in three main cities, that is, Sukkur, Hyderabad, and Karachi, showed that water is unsafe for drinking purposes due to microbial and chemical contamination”
  • Memon et al 2011 [specific to southern Sindh] - “ All four water bodies (dug wells, shallow pumps canal water, and water supply schemes) exceeded WHO MPL for turbidity (24%, 28%, 96%, 69%), coliform (96%, 77%, 92%, 81%), and electrical conductivity (100%, 99%, 44%, 63%), respectively…Some common diseases found in the study area were gastroenteritis, diarrhea and vomiting, kidney, and skin problems.”
    • A more recent paper (Shahab et al 2016) finds the same: “Suitability of groundwater for drinking and agricultural purpose indicates that majority of the samples were very far from drinking water standards while few samples were considered unsuitable for agricultural purpose especially in the lower southern Sindh( Thatta, Badin and Tharparker).”
  • Jamali et al 2022 [Taluka Larkana, Sindh, Pakistan] - “Analysis based on the SPI categorized 49%, 30%, and 21% of the samples as moderately polluted, highly polluted, and unsuitable for drinking purposes, respectively, while the WQI categorized 53.53%, 18.6%, 16.27%, 11.6% of the samples as good, poor, very poor, and unsuitable for drinking, respectively. Whereas the PIG classified 55.18%, 13.95%, 5.95%, 11.63%, and 13.95% of the samples as insignificant, low polluted, moderately polluted, highly polluted, and unsuitable for drinking purposes, respectively. Overall, the study revealed that groundwater in most of the parts of the study area is not suitable for drinking purposes.”
    • Lanjwani et al 2020 also speaks about water quality inTaluka Ratodero (also in district Larkana): “results showed that Ratodero city groundwater was of concern for human health”
  • Kumar et al 2023 - “In Phuleli Canal, Sindh Province, out of eight water samples, most were found to contain fecal coliforms [56]. Another study in Khairpur found fecal coliform and coliform bacteria in 100% of water samples taken from the water channel, the distribution system, and household taps…The study found that coliform bacteria were present in 64% of the samples from Punjab Province, 67% of samples from Khyber Pakhtunkhwa Province, 83% of samples from Sindh Province”

• 9

  The grant amount depends on the total cost of the external evaluator that IRD plans to contract. The cost of the external evaluator will be up to $350,000, but has not yet been determined. We will update this page with the finalized budget once it is available.

• 10

  “Our key intervention comprises tailored 2-way SMS reminders sent to pregnant women and
  caregivers all 0-23 months children enrolled in the ZM-EIR in the selected (intervention) districts.” IRD, Breastfeeding pilot proposal, 2023 (unpublished).

• 11

  “We plan to tailor the SMS reminders based on the determinants highlighted in existing literature that affect breastfeeding[13] and the availability of demographic data in ZM-EIR. Specifically, messages will be tailored to:

  1. The age group of the child (based on the WHO breastfeeding guidelines categorized into pregnancy, 0–1 month, 2-6 months, >6 months, each of these categories corresponds to different stages of breastfeeding including initiation, exclusive breastfeeding and continued breastfeeding)." IRD, Breastfeeding pilot proposal, 2023 (unpublished).

• 12

  "To inform the SMS design, we will conduct formative qualitative in-depth interviews and focus groups with approximately 20 people per district (a total of 80 participants across the 4 intervention districts). Participants will include pregnant women, caregivers of 0-23 months children and accompanying family members as well as lady health workers, supervisors and clinic. [...] To inform the content and design of the SMS reminders, we will examine the participant’s views and information sources regarding infant feeding, their breastfeeding intentions, and their anticipated obstacles and support required for exclusive breastfeeding. We will also inquire about women's insights into the feeding practices within their social circles.” IRD, Breastfeeding pilot proposal, 2023 (unpublished).

• 13

  “To address caregiver and beneficiary queries, we will hire and train helpline operators. We will establish a dedicated webpage linked to the SEIR dashboard for logging and managing these queries. With the support of a breastfeeding expert and experienced nurse counselor, a comprehensive knowledge base will be developed to address anticipated caregiver questions and provide evidence-based answers. Helpline operators will utilize this knowledge base to respond to logged queries. A threaded message format will allow operators to track the conversation and resolution of each issue.” IRD, BF Draft Implementation Protocol, 2024 (unpublished)
  

• 14

  IRD, Target Population - Breastfeeding Promotion Study. See row "total", column "Phone # available (80%)" in Tables 2 and 3.

• 15
  • “The Pregnant Women and Birth Registry (PWBR) enrolls pregnant women and newborns from selected high-volume birthing facilities of the Sindh province in the PWBR linked to the SEIR.” IRD, mCCTs for Immunization Scale up, Annual Report, 2023, p. 26.
  • "We plan to include pregnant women in the study through the pregnant women and birth registry(PWBR) linked to ZM-EIR and operational in selected birthing facilities across the province." IRD, Breastfeeding pilot proposal, 2023 (unpublished).

• 16

  “To inform the SMS design, we will conduct formative qualitative in-depth interviews and focus groups with approximately 20 people per district (a total of 80 participants across the 4 intervention districts). Participants will include pregnant women, caregivers of 0-23 months children and accompanying family members as well as lady health workers, supervisors and clinic. The aim will be to gather feedback from participants to inform the design of the SMS reminders focusing on both urban and rural areas, recognizing the idiosyncrasies in perceptions and differing challenges that may exist across these different settings.” IRD, Breastfeeding pilot proposal, 2023 (unpublished).

• 17
  • “Baseline Household Survey: The survey will determine the status of our outcomes of interest in the intervention and control districts prior to the intervention roll-out”
  • “Endline Household Survey: The survey will determine the status of our outcomes of interest in the intervention and control districts after the intervention roll-out.”

  IRD, Breastfeeding pilot proposal, 2023 (unpublished).

• 18

  Note that a) our cost-effectiveness analyses are simplified models that are highly uncertain, and b) our cost-effectiveness threshold for directing funding to particular programs changes periodically. See GiveWell’s Cost-Effectiveness Analyses webpage for more information about how we use cost-effectiveness estimates in our grantmaking.

• 19

  This benchmark is based on our ‘moral weights’, a system we use to quantify the benefits of different impacts (e.g. increased income vs reduced deaths). We benchmark to a value of 1, which we define as the value of doubling someone’s consumption for one year. Our estimate of the value of direct cash transfers is 0.00335 per $. For more on how we use moral weights, see this document.

• 20

  "Population–Three hundred women attending antenatal care were randomised, 100 to each arm, and followed for 24 weeks postpartum. Pregnant women 14 years old with access to a phone and able to read SMS were eligible for participation.

  Methods–Women were randomised (1:1:1) to receive 1-way SMS versus 2-way SMS with a nurse versus control. Weekly SMS content was tailored for maternal characteristics and pregnancy or postpartum timing.

  Main Outcome Measures–Facility delivery, EBF and contraceptive use were compared separately between each intervention arm and the control arm by Kaplan-Meier analysis and X2 tests using intent-to-treat analysis." Unger et al. 2018, p. 2.

• 21

  See our calculations here.

• 22

  See our calculations here.

• 23

  Intuitively, this is because we think in-person interventions are generally more likely to offer higher levels of support and facilitate higher levels of engagement from caregivers.

• 24

  We spoke to one expert who told us that it could be plausible for two-way messages to be more effective than in-person promotion if they were well-tailored and responsive to caregivers’ needs. For example, if in-person communication is more lecture-style, or provides no privacy for patients to ask questions, it could be less impactful than expected.

• 25

  Our estimates for breastfeeding promotion are based on studies that estimate exclusive breastfeeding rates at 6 months. We think infants who are exclusively breastfed from birth until 6 months will benefit the most due to the reduced exposure to potentially low-quality water. We think other young children between 1 to 6 months of age are likely to benefit from breastfeeding for similar reasons; however, we’re uncertain what proportion of these infants will be reached after caregivers have already stopped exclusively breastfeeding and whether it will be possible to restart breastfeeding. Therefore, we subjectively make a large adjustment (-75%) to decrease the expected benefits from mortality averted (in the postneonatal infancy period) and long-term income effects for children who are reached at this age.

• 26

  Based on estimates provided by IRD, we estimate that roughly 130,000 children aged 0-6 months who are already enrolled in ZM-EIR and roughly an additional 240,000 newly enrolled children will be targeted for reminders (rough average of B43 & B61 and F43 and F61 in linked sheet, respectively). We make rough assumptions about the proportions of these populations that we expect IRD to reach at <1 months of age (roughly ⅙ for those already enrolled, and 80% for those newly enrolled). Finally we assume that 70% of caregivers enrolled in IRD’s platform would have a phone number (based on estimates provided by IRD), and we make a rough guess that 90% of caregivers with phone numbers would successfully receive at least one SMS reminder.

• 27

  See rationale in footnote 54 of our report on breastfeeding promotion programs.

  We assume a smaller multiplier (1.3 rather than 2) in this case because of Pakistan’s relatively lower fertility rate (https://data.worldbank.org/indicator/SP.DYN.TFRT.IN?locations=XM-PK). This multiplier should be lower if caregivers in this program would be re-enrolled during subsequent pregnancies (and therefore costs apply in future). However, it may also be the case that there are spillover effects to people not enrolled in the program. We haven’t accounted for these competing effects.

• 28

  27.4% * 0.55 * 0.5 = 7.5%

• 29

  We estimate the effect on all-cause mortality by:

  1. Conducting a meta-analysis on the impact of breastfeeding promotion programs on diarrhea risk in the postneonatal infancy period (28-365 days). We estimate that breastfeeding promotion programs result in a 16% decrease in diarrhea risk. Our meta-analysis is based on a subset of in-person breastfeeding promotion studies that measured the impact of breastfeeding promotion on diarrhea risk. See here.
  2. We estimate the impact of IRD’s program on diarrhea risk by scaling the effect size estimated from our meta-analysis by the ratio of the expected reduction in non-exclusive breastfeeding rates from IRD’s program (27%) and in-person breastfeeding reminders programs (27%; including all studies, even those which did not measure the impact of the program on diarrhea risk). See here. This is because we assume that a program which has a larger impact on the share of new parents not exclusively breastfeeding would have larger effects on diarrhea risk; however, we are uncertain about the exact magnitude because we do not observe a strong correlation between changes in breastfeeding rates and changes in diarrhea risk in the studies included in our meta-analysis. Furthermore, for this pilot, we cap the impact of IRD’s SMS reminders program to the impact of in-person reminders programs because we find it implausible that digital reminders would have larger effects than in-person interventions. Overall, we estimate the intervention will lead to a 4.4% reduction in postneonatal diarrheal risk, post IV/EV adjustments.
  3. We then calculate the all-cause mortality effects due to that reduction in diarrheal risk. This is based on our work on breastfeeding promotion mortality plausibility modeling. We estimate that a 4.4% reduction in postneonatal diarrheal risk results in a 3.5% reduction in all-cause mortality risk, post IV/EV adjustments.

  Our calculations are here. For more on this approach, see our report on breastfeeding promotion interventions.

• 30

  To calculate the mortality effects for neonates, we follow a similar approach to that outlined for postneonatal mortality reductions. Since most of the RCTs we reviewed in our custom meta-analysis assessed rates of diarrhea at 6 months (see our report on breastfeeding promotion programs for more information) we apply a scaling factor to estimate the effect on diarrhea rates for the neonatal age range.

• 31

  93 + 31

• 32

  $455,191 / 178,990

• 33

  $455,191 / (93 + 31)

• 34

  See our adjustments worksheet here.

• 35

  The meta-analyses we base our estimates on measure exclusive breastfeeding rates and diarrhea risk at 6 months, and we assume that the measure effects at 6 months are representative of the postneonatal (28 days to 1 year of age) period. We have not conducted a review of studies that measured the impact of exclusive breastfeeding for older children.

• 36

  Table 2: Key indicators and categories based on WHO criteria 5

  • Primary Outcome: Exclusive Breastfeeding (EBF) under 6 months
  • Definition: Percentage of infants 0-6 months who are breastfeeding exclusively. Exclusive breastfeeding is defined as breastfeeding with no other food or drink, not even water
  • Measurement in survey: Percentage of infants 0-6 months of age who were fed exclusively with breast milk during the previous day. Numerator: infants 0-6 months of age who were fed only breast milk during the previous day. Denominator: infants 0-6 months of age.
  • Age Group (at time of ‘enrolment’ into Program) for which outcome can be measured: Pregnancy, 0-1m, 2-6m

  IRD, Draft of BF evaluation protocol, 2024 (unpublished)

• 37

  “Keeping this in view a paired design is proposed for the baseline survey to test whether there is a difference in proportions across the two arms (intervention and non-intervention arms). To derive statistically significant results through two correlated proportions comparison formula, a sample size calculation was done, with the use of PASS 2022, version 22.0.5. The inputs used for sample size calculation were
  1. Paired sample McNemar Test (based on pairing/matching criteria of urban-rural and mCCT status) for comparing two matched proportions,
  2. Two-sided hypothesis testing methodology,
  3. Type I error rate (α) of 0.05,
  4. Detection of a difference of at least 5% in the intervention arm as compared to the non-intervention arm (the expected proportion of exclusive breastfeeding among the non-exposed is previously documented to be 45%)
  5. Power of 90%.”
  IRD, Draft of BF evaluation protocol, 2024 (unpublished)

• 38

  “Below is a brief overview of the surveys conducted as part of our previous projects, both in-house using IRD-hired survey teams as well as through third parties.

  In 2015, we engaged an external partner to conduct a household survey in Punjab province, Pakistan to determine immunization coverage across one selected district. The survey included 842 households, 881 women respondents of reproductive age who were inquired about TT vaccination and 1,056 under-5 children for routine EPI vaccinations.

  Between 2016-2018, we conducted a pre-post baseline-endline survey in-house in District Shikarpur, Sindh to determine district-wide immunization coverage indicators before and after a conditional cash transfer program. A total of 2,641 households were included in the survey from all four towns in district Shikarpur. As part of these surveys, a total of 2,641 respondents were successfully interviewed about their TT vaccination status and immunization coverage data of 3,398 children under three years of age was collected.

  In 2016 IRD trained a team of enumerators and field workers to conduct household survey for a total of 3,060 children between 12 and 23 months of age in Korangi town, Karachi through four types of survey sampling methods. Approximately 18,500 households were visited and one-third of the survey participants were invited to the nested antibody assessment as well.

  More recently, IRD engaged an external partner to conduct a household survey across two districts (Kashmore and Sujawal) in Sindh to determine the prevalence and risk factors for zero dose children. The survey was conducted in 2021 and collected data on 5,326 households and 4,521 under-3 children”
  IRD, Breastfeeding pilot proposal, 2023 (unpublished)

• 39

  In 2021 GiveWell funded an RCT through the University of Chicago to evaluate IRD’s mCCT program. For more, see our grant page here.

• 40

  See a summary of these studies here.

• 41

  IRD, comments during meeting on December 7, 2023 (unpublished).

• 42

  “We are also exploring the feasibility of leveraging WhatsApp to send out messages given the rapid popularity of the platform as the preferred choice of communication in the country regardless of education, socio-economic status, and gender,”
  IRD, Breastfeeding pilot proposal, 2023 (unpublished).

• 43

  IRD, comments during meeting on December 7, 2023 (unpublished).

• 44

  IRD, comments during meeting on January 12, 2024 (unpublished).

• 45

  See this section of our report on breastfeeding promotion programs.

• 46
  • “We plan to tailor the SMS reminders based on the determinants highlighted in existing literature that affect breastfeeding[13] and the availability of demographic data in ZM-EIR. Specifically, messages will be tailored to:

  1. The age group of the child (based on the WHO breastfeeding guidelines categorized into pregnancy, 0–1 month, 2-6 months, >6 months, each of these categories corresponds to different stages of breastfeeding including initiation, exclusive breastfeeding and continued breastfeeding)." IRD, Breastfeeding pilot proposal, 2023 (unpublished).

  • "To inform the SMS design, we will conduct formative qualitative in-depth interviews and focus groups with approximately 20 people per district (a total of 80 participants across the 4 intervention districts). Participants will include pregnant women, caregivers of 0-23 months children and accompanying family members as well as lady health workers, supervisors and clinic. [...] To inform the content and design of the SMS reminders, we will examine the participant’s views and information sources regarding infant feeding, their breastfeeding intentions, and their anticipated obstacles and support required for exclusive breastfeeding. We will also inquire about women's insights into the feeding practices within their social circles.” IRD, Breastfeeding pilot proposal, 2023 (unpublished).
  • "Caregivers will have the option to opt out of future SMS reminders either at the time of follow-up immunization visits or by responding to the 2-way SMS reminders and requesting cessation of the breastfeeding reminder service." IRD, Breastfeeding pilot proposal, 2023 (unpublished).