This is an interim intervention report. We have spent limited time to form an initial view of this program and, at this point, our views are preliminary. We plan to consider undertaking additional work on this program in the future.
Summary
- What is the program? Children in low-income countries may be at a higher risk of failing to meet milestones for development, relative to children in higher-income countries. It is possible that this has long-term consequences for these children as they grow into adults. Early childhood psychosocial stimulation programs aim to develop children's cognitive and language skills by training parents (primarily mothers) on how to support their child's development through play and other activities.
- What is the evidence of effectiveness? One small-scale study from Jamaica finds a psychosocial stimulation intervention, which had substantial short-term effects on cognitive development, led to a 35% increase in adult income 20 years after the intervention was completed. This is the only study we are aware of that measures the long-term effects of psychosocial stimulation programs on adult income. Our review of other studies of early childhood psychosocial stimulation programs, which measure shorter-term effects on cognitive skills but not effects on adult income, suggests the Jamaica program's effect on cognitive skills is higher than the typical program studied. Because the income effects found in this single, small-scale study substantially exceed our skeptical prior, and because the Jamaica program appears to have higher short-term effects on cognitive skills than the average program, our best guess is that the typical effects of psychosocial stimulation on adult income are likely to be much smaller than the effects found in the Jamaica study.
- How cost-effective is it? While it is possible that early childhood psychosocial stimulation programs have moderate impacts on adult income, these programs appear to be relatively expensive, compared to other interventions we have reviewed. As a result, we estimate that psychosocial stimulation's cost-effectiveness is below that of programs we would consider recommending funding in the future. However, we have high uncertainty around the model's key parameters, including the cost of psychosocial stimulation interventions and the appropriate effect size for long-term income effects.
- Is there room for more funding? We have not reviewed room for more funding in any depth.
- Bottom line: We do not plan to prioritize early childhood psychosocial stimulation programs as we consider programs to recommend funding in the future.
Published: January 2020
Table of Contents
What is the problem?
Our impression is that a commonly held view among those who study early childhood development is that early childhood is a critical time for brain development. This impression comes from Grantham-McGregor et al. 2007, which appears to be a reasonable reflection of a consensus view among academics who study this area, though we haven't reviewed the evidence ourselves or conducted a comprehensive scan of academics' views on this topic.1 Some have also hypothesized that low cognitive development in childhood may have long-term consequences for children as they grow into adults.2 (We review evidence for claims on long-term impacts in the discussion of evidence for effectiveness below.)
Our impression, based on Grantham-McGregor et al. 2007 and Black et al. 2017, which we believe capture the consensus view among experts in this field, is that poverty is associated with risk factors for low child development, including stunting and inadequate stimulation.3 This suggests that the problem may be especially large in developing countries, which have a large share of households in poverty.
Researchers and organizations like the World Health Organization (WHO) and UNICEF have recommended stimulation activities (e.g., singing, talking, reading and playing games)4 as one strategy to promote the development of cognitive, language and social-emotional skills.5
What is the program?
Early childhood psychosocial stimulation programs aim to develop childrens' cognitive and language skills by training parents (primarily mothers) on how to support their child's development through play and other activities.6 The programs may also provide training in other parenting practices, including secure attachment and positive parenting.7
The programs often involve home visits where trainers work directly with children as well. They vary in the amount of contact with parents involved, ranging from less than 10 hours to 120 hours according to one review,8 and may involve home visits, group sessions with parents only, visits to primary care clinics, or a combination of these.9 Most focus on children under three years old.10
Psychosocial stimulation programs are often targeted toward low-income households or children with other risk factors for low cognitive development, such as undernutrition,11 and are also frequently combined with nutrition programs.12
In recent years, psychosocial stimulation programs have been combined with or incorporated into larger government programs (e.g., cash transfer programs13 or large-scale health programs14 ).
What is the evidence of effectiveness?
In our cost-effectiveness analysis, we model the benefits of early childhood psychosocial stimulation programs as coming through their potential impact on adult income. While there are several studies that measure the effect of psychosocial stimulation programs on children's cognitive outcomes, we have a high degree of uncertainty about the extent to which cognitive gains in early childhood persist into adulthood. As a result, we focus our discussion of evidence for effectiveness on direct evidence for the effect of early childhood psychosocial stimulation programs on adult income.
We have found one study (Gertler et al. 2014) that measures long-term income effects of early childhood psychosocial stimulation programs. Gertler et al. 2014 find that a small-scale program in Jamaica in the 1980s led to large increases in adult income for children who received the program. Because this is the only study (to the best of our knowledge) to assess impacts on income, it forms the bulk of our review.
To understand how generalizable these effects are, we also reviewed evidence from additional studies on the impact of psychosocial stimulation programs. These do not measure long-term income effects but do report on short-term cognitive effects. We compare the short-term cognitive skill effect sizes from these other studies to the short-term cognitive skill effect sizes from the Jamaica study to adjust the effect size on income from Gertler et al. 2014. This is intended to better approximate effects on income from future implementations of psychosocial stimulation programs, though we view this as a rough approximation.
Our best guess is that the reported income effects from the Jamaica study by Gertler et al. 2014 are unlikely to generalize for two reasons. First, our skeptical prior is that early childhood psychosocial stimulation programs are likely to have relatively small effects on adult income. Because Gertler et al. 2014 is the only study we are aware of to measure long-term income effects of psychosocial stimulation programs, and because it has a small sample size, we estimate that the true effect of this type of program on income is smaller than that reported by Gertler et al. 2014 and closer to our skeptical prior. Second, when we compare short-term cognitive skill effect sizes from additional studies on psychosocial stimulation to those found in the Jamaica program, we find that the Jamaica program's effect on short-term cognitive skills is generally higher than the typical program. As a result, in our cost-effectiveness analysis, we estimate a much smaller effect on adult income than that found in Gertler et al. 2014.
The Jamaica study
Gertler et al. 2014 is a 20-year follow-up to the intervention described in Grantham-McGregor et al. 1991 and implemented 1986-1987. (We refer to studies of this program collectively as "the Jamaica study.")
At baseline, the Jamaica study included 129 growth-stunted children aged 9 months to 24 months. Approximately half of these children received weekly visits over the course of 2 years from community health workers designed to promote psychosocial stimulation.15 Previous studies found the intervention led to large impacts on cognitive development immediately following the intervention (0.91 SD) and at 15- and 20-year follow-ups (0.4-0.6 SD).16
Gertler et al. 2014 re-surveyed 105 of the original 129 individuals from the Grantham-McGregor et al. 1991 study.17 Gertler et al. 2014 compare earnings18 between a) those who received psychosocial stimulation alone or in combination with nutrition interventions and b) those in the control or nutrition alone group.19 They find that monthly earnings are 35% greater (effect size on log earnings of 0.30, p-value = 0.02) for those who received psychosocial stimulation alone or in combination with nutrition interventions than for those in the control or nutrition alone group.20
Overall, we do not have any serious concerns about the study but have not conducted a thorough vet of its quality. However, we do not view the study as sufficiently strong to update us up from our skeptical prior that early childhood psychosocial stimulation programs are likely to have more modest effects on adult income. This is because this is the only study we are aware of that estimates long-term income effects and because of the study's small sample size.
In particular, the study's small sample size raises concerns about "p-hacking" and publication bias, which increases our uncertainty about internal validity. Because studies with smaller sample sizes generally require a larger effect size to achieve statistical significance, a bias toward publication of only statistically significant findings (publication bias) or reporting only empirical specifications that produce statistically significant results (p-hacking) will lead to higher effect sizes.
We were unable to locate a pre-analysis plan for this paper. While we have no specific reasons to believe there were other empirical specifications tested but not reported, the lack of a pre-analysis plan raises concerns about p-hacking and lowers our confidence in the results. We also do not have any direct evidence of publication bias (i.e., other studies of early childhood psychosocial stimulation that included a long-term follow-up on income but the findings of which were not published), but we have not undertaken a thorough review of this issue.
Additionally, to the extent that the small sample reflects that this was a small-scale intervention that may not be effectively scaled, the small sample size in Gertler et al. 2014 (and in the original study by Grantham-McGregor et al. 1991) does increase our uncertainty about the external validity of these findings. We discuss the generalizability of the findings below as well.
Other findings from our preliminary quality assessment:
- The authors find baseline imbalance on 3 out of 23 variables.21 They discuss approaches to account for potential balance22 and show that results are similar with or without these controls.23
- The attrition rate of close to 20% raises some concerns, but the authors note there is no selective attrition for the main estimating sample of adults.24
- The authors do find that they were more likely to re-survey migrants in the treatment vs. control group, which could bias effects upward if migrants are likely to have higher earnings. They address this by imputing earnings for those control migrants they did not re-survey based on earnings observed for treatment migrants.25 The results are essentially the same.26
- The authors combine individuals who received stimulation alone and stimulation in combination with a nutritional intervention into the "treatment" group and individuals who received the nutrition intervention alone and no intervention into the "control" group for their main comparison, as discussed above. This could bias the effect of stimulation alone if the combination of nutrition enhances stimulation's effectiveness. In supplementary analyses, the authors report comparisons of stimulation alone vs. no treatment and find an effect size of 0.36 on log income (slightly larger than the main effect size of 0.30), which gives us some confidence that the results are not driven by these synergies. However, these regressions have an extremely small sample size.27
- The authors note concerns about the small sample size's impact on statistical tests and use tests designed for small samples, though we have not reviewed the validity of these tests.28
The results of Gertler et al. 2014 and Grantham-McGregor et al. 1991 suggest that the psychosocial stimulation program in the Jamaica study increased cognitive skills by 0.91 SD immediately after the intervention and led to an increase of 0.30 in log income 20 years later.
As a plausibility check, we briefly reviewed papers that use cross-sectional and time-series regressions to translate cognitive skills into earnings. While these approaches have limitations, both because a) cognitive skill is not randomly assigned and its effect must be isolated through other approaches and b) they apply to different settings, we view this as a rough test on the feasibility of the results from Gertler et al. 2014.29
The most closely-related estimate we could find was in Attanasio et al. 2014, who estimate, using data from the U.S. for children born in the 1970s, that a 1 SD increase in Peabody vocabulary score leads to a 30% increase in earnings at age 30 (or increase of 0.26 in log income).30 This estimate is slightly lower than the impact on income found by Gertler et al. 2014, and our best guess is that this estimate is biased upward because it does not control for other factors that are likely correlated with both cognitive skill measures and adult income.
Generalizability of the Jamaica study
To understand how generalizable the effects of the Jamaica study are, we also reviewed evidence from the impact of more recent psychosocial stimulation programs. These do not provide long-term follow-up on adult income but do measure shorter-term effects on measures of cognitive development. Comparing shorter-term impacts on cognitive development between the Jamaica study and other studies may give a better prediction of how likely effects of the size found by the Jamaica study are to occur in future settings.
To survey this literature, we reviewed a meta-analysis by Aboud and Yousafzai 2015, which reports on the cognitive development impacts at a one-year (or sooner) follow-up of psychosocial stimulation programs for children under age 2 in developing countries, published between 2000 and 2013.31 The included studies implement psychosocial stimulation through a variety of formats, including home visits, group sessions and clinic appointments.32 The average sample size is 220.33 17 out of the 21 included studies are randomized controlled trials (RCTs).34 We have not thoroughly vetted this meta-analysis or the included studies and have not explored how results might vary based on the features of the psychosocial stimulation programs, the quality of the study (e.g., RCT vs. non-RCT), or the measures of cognitive skill used.
We supplemented this with a review of RCTs reporting short-term effects on cognitive development published since 2013, as well as a review of studies assessing longer-term impacts on cognitive skills. We did not look for studies published before 2000.
Our goal was to get a rough sense of how the effects of the Jamaica study compare to those of other similar psychosocial stimulation programs studied in the literature. We did not aim to provide a completely comprehensive review, and it is possible we have missed studies within this literature. We did not thoroughly vet individual studies, specific details of psychosocial interventions used, or measures of cognitive skill development used. We have also not conducted a formal meta-analysis or undertaken any efforts to understand what is driving variation in effects of psychosocial stimulation across studies.
Our best guess is that if we reviewed these issues in further detail, we would revise our estimate of the effectiveness of psychosocial stimulation programs downward. As a result, our estimates of effectiveness are likely to be overly optimistic.
In our review of RCTs published since 2013, we found the following:
- Short-term effects (one-year follow-up or less). Aboud and Yousafzai 2015 find an effect size of 0.42 SD (95% CI 0.36, 0.48) for cognitive outcomes.35 Across 14 additional studies we identified that were published after 2013 and not included in Aboud and Yousafzai 2015, summarized here, 12 report an effect size in standard deviations.36 Among these, impacts on cognitive development range from 0.08 SD to 1.3 SD, with a median effect size across studies of roughly 0.30 SD, compared to an effect size of 0.91 SD in the Jamaica study immediately post intervention.37
- Longer-term effects (two-year follow-up or more). Among the five additional studies we found that report impacts on cognitive outcomes two or more years after the intervention was complete, summarized here, four find impacts that persist two or more years after the conclusion of the intervention. Impacts on measures of cognitive development range from -0.03 SD to 0.6 SD.38 Overall, studies find a significant degree of fade-out in effects (roughly 25%-35% of initial effect remaining at endline).39 The Jamaica experiment also finds fade out of cognitive development effects, though to a lesser degree (44%-66% of initial effect remaining at endline) and over a longer time horizon (15 years post-intervention, when children are 17-18 years old).40
This review suggests the typical psychosocial stimulation program reviewed in the literature has smaller effects on cognitive skills than the Jamaica study. In the cost-effectiveness analysis, we adjust our estimate of the effect of psychosocial stimulation programs on income downward based on these findings.
How cost-effective is it?
A preliminary cost-effectiveness model for this intervention is available here. We estimate that early childhood psychosocial stimulation programs are less cost-effective than programs we would consider recommending funding in the future. As a result, we do not plan to prioritize early childhood psychosocial stimulation programs as we consider programs to recommend funding in the future.
Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.
The cost-effectiveness analysis for early childhood psychosocial stimulation programs is driven by two key parameters, about which we are highly uncertain:
- Intervention cost. We estimate that the costs of psychosocial stimulation programs that achieve the effect sizes we model in our cost-effectiveness analysis are roughly $100 to $500 per child, based on a superficial scan of five programs where cost data were available. We have not vetted any of these estimates.
- Impact on adult income. Our cost-effectiveness also hinges critically on the impact of psychosocial interventions in early childhood on adult income. Because we rely heavily on one study, we are highly uncertain about impacts on adult income and rely on subjective assessments of internal and external validity to arrive at our final estimate.
In our preferred cost-effectiveness calculation, our best guess is that these programs cost $250 per child and increase adult income roughly 5 percent. The relatively high cost of this intervention, along with a more moderate effect on adult income, is a key driver of the relatively low cost-effectiveness of psychosocial stimulation programs in our model.
However, because we have high uncertainty about both cost and effect on income, our cost-effectiveness analysis explores how sensitive the results are to adjustments in key parameters. While these sensitivity checks still lead us to believe psychosocial stimulation programs are not within range of programs we would consider funding in the future, there are scenarios where our cost-effectiveness analysis suggests these programs could be closer to the range of programs we would consider recommending funding. We think it is unlikely early childhood psychosocial stimulation programs could be implemented at this cost and achieve the results necessary to show this level of cost-effectiveness. We would revise our assessment if we found high-quality evidence that programs could achieve similar or higher effects on cognitive skills or income at lower cost.
Is there room for more funding?
We have not reviewed room for more funding in any depth. Save the Children ran recent programs in Bangladesh41 and Rwanda,42 Catholic Relief Services implemented a program in Sierra Leone,43 and Pratham is the implementing partner for recent studies in India,44 so it is possible that these organizations may have room for more funding to implement psychosocial stimulation programs in the future. There are likely to be other organizations implementing these interventions that we have missed.
Key questions for further investigation
- What is the cost of this intervention in settings where it is likely to be implemented in the future, and are there models that have systematically stronger impacts at lower costs (e.g., group-based formats, scaled-up versions that achieve economies of scale)?
- Will subsequent follow-ups to the Jamaica study find larger impacts, once any individuals currently in school enter the job market?45
- Should we incorporate additional health effects (e.g., violence, mental health) into our model?46
- Do cash transfer programs have an impact on cognitive development, and if so, how much does this affect the impact of psychosocial stimulation programs compared to cash in our cost-effectiveness model?47
- How should we incorporate costs of additional schooling into cost-effectiveness?48
Sources
- 1
We are informed by the following excerpt from Grantham-McGregor et al. 2007: "Children's development is affected by psychosocial and biological factors and by genetic inheritance. Poverty and its attendant problems are major risk factors. The first few years of life are particularly important because vital development occurs in all domains. The brain develops rapidly through neurogenesis, axonal and dendritic growth, synaptogenesis, cell death, synaptic pruning, myelination, and gliogenesis. These ontogenetic events happen at different times and build on each other, such that small perturbations in these processes can have long-term effects on the brain's structural and functional capacity." Pp. 60-61.
This view is also reflected in the following sources:
- "The basic architecture of the brain is constructed through an ongoing process that begins before birth and continues into adulthood. Early experiences affect the quality of that architecture by establishing either a sturdy or a fragile foundation for all of the learning, health and behavior that follow. In the first few years of life, more than 1 million new neural connections are formed every second." Harvard University Center for the Developing Child, p. 1.
- "In the earliest years of life, especially from pregnancy to age three, babies need nutrition, protection and stimulation for healthy brain development. Recent advances in neuroscience provide new evidence about a baby's brain development during this time. As a result, we know that in their earliest years, babies' brains form new connections at an astounding rate – according to Harvard University's Center on the Developing Child more than 1 million every single second – a pace never repeated again." UNICEF, Early child development.
- 2
"Millions of children under 5 years of age in low and middle-income countries (LMIC) are failing to reach their potential in cognitive, language, and social emotional development, which has implications for their educational attainment and adult functioning." Grantham-McGregor et al. 2014, p. 11.
- 3
- "Poverty is associated with inadequate food, and poor sanitation and hygiene that lead to increased infections and stunting in children. Poverty is also associated with poor maternal education, increased maternal stress and depression, and inadequate stimulation in the home. All these factors detrimentally affect child development (figure 2)." Grantham-McGregor et al. 2007, p. 62.
- "New estimates, based on proxy measures of stunting and poverty, indicate that 250 million children (43%) younger than 5 years in low-income and middle-income countries are at risk of not reaching their developmental potential." Black et al. 2017, p. 77.
- "Between 2004 and 2010, the estimated number of children under 5 years in LMICs exposed to stunting or extreme poverty, and therefore at risk of not reaching their developmental potential, declined from 279·1 million (51% of children in 2004) to 249·4 million (43% of children in 2010) (table 1). South Asia experienced the largest decline in both the number and prevalence of children at risk (from 110·9 million to 88·8 million, and from 65% to 53%, between 2004 and 2010). Sub-Saharan Africa had the highest prevalence of children at risk of not reaching developmental potential (70% in 2004 and 66% in 2010).
"Population-level assessments measure the developmental status of populations and are used for monitoring global targets, such as UN Sustainable Development Goals. Stunting and extreme poverty serve as proxy measures because they are associated with children's development, are measured globally using uniform methods, and are responsive to environmental and economic changes. Direct population-level assessments are advantageous due to their sensitivity to variations in children's development and responsiveness to programmatic interventions. However, direct assessments are often costly and time-consuming to measure, and might require developmental and cultural adaptations." Black et al. 2017, p. 78.
- 4
"Stimulation practices are derived from the function of stimulating neurons (Shonkoff and Phillips, 2000). Examples include language interaction (e.g. singing, talking, reading); provision of learning materials and exposure to learning opportunities (e.g., books, magazines), physical interaction (e.g. sports, playing games) and parents' behavior, which serves as a model for children to imitate and emulate (Britto et al, 2002)." Britto et al. 2015, p. 17.
- 5
- "Beginning as newborns, children learn about their world and how to function in it from the adults in their lives. As children grow older, their experiences with adults and other children also occur in early childhood development programmes and preschools. During this phase of a child's life, it is teachers and early childhood professionals who continue the task of building critical foundations for a child's development and learning. In the early moments of a child's life, the stimulation that comes with love, play, talk, singing and reading books with a caring adult is not as simple as it seems. It serves an important neurological function. These interactions can boost cognitive, physical, social and emotional development. Brain scientists call this interaction 'serve and return'. And in these interactions between baby and adult, some of the simplest of moments can matter the most: eye contact, a hug, words, a song." UNICEF, Early Moments Matter for every child, p. 33.
- "In infancy through the first three years, important interventions are the promotion of infant-parent(s) bonding and attachment, responsive care and early stimulation, support optimal infant and young child feeding practices (including responsive feeding), prevention and timely management of illnesses, detection of developmental delays and early intervention, and support for maternal psychosocial well-being." WHO, Meeting report: Nurturing human capital along the life course: Investing in early child development, p. 8.
- "Although global attention to early childhood development has been established through its inclusion in the UN Sustainable Development Goals, 250 million children (43%) younger than 5 years in low-income and middle-income countries are at risk of not achieving their developmental potential, as discussed in Paper 1 of this Series. We suggest that this gap in human potential is partly due to two reasons: the failure to apply emerging scientific knowledge on nurturing care to shape young children's development; and the failure to take action at scale, using a multi-sector approach across key stages in the early life course.
"We define nurturing care as a stable environment that is sensitive to children's health and nutritional needs, with protection from threats, opportunities for early learning, and interactions that are responsive, emotionally supportive, and developmentally stimulating. As an overarching concept, nurturing care is supported by a large array of social contexts—from home to parental work, child care, schooling, the wider community, and policy influences. Nurturing care consists of a core set of inter-related components, including: behaviours, attitudes, and knowledge regarding caregiving (eg, health, hygiene care, and feeding care); stimulation (eg, talking, singing, and playing); responsiveness (eg, early bonding, secure attachment, trust, and sensitive communication); and safety (eg, routines and protection from harm)." Britto et al. 2017, p. 91. - "Poor levels of stimulation in the home, chronic undernutrition (stunting), and iron and iodine deficiencies are key risk factors; professionals have called for large-scale programs that integrate health, nutrition, and the promotion of child development." Grantham-McGregor et al. 2014, p. 11.
- "We already know that the brain develops most rapidly in the first few years of a child's life. During these critical years, neuroplasticity is at a peak—neurons form new connections at the astounding rate of up to 1000 per second. These synaptic connections are the foundation of a child's physical and mental health, affecting everything from longevity to the lifelong capacity to learn, from the ability to adapt to change to the capacity for resilience.
"New lines of research are expanding our understanding of the part environment plays in the formation of these neural connections. If children fail to get what they need—enough nutrition, nurturing, stimulation, and a sense of security—during the most critical years of early childhood, the impact on their lives and futures is enormous. For example, inadequate nutrition in the early years of childhood can result in stunting, which can cause diminished physical and cognitive development that undermine a child's ability to learn and earn later in life. Similarly, inadequate stimulation during the same critical period of earliest childhood can reduce learning capacity and ability to form social and emotional attachments." Lake and Chan 2015, p. 1816.
- 6
- "Instead, almost all of the parenting interventions we found in developing countries focused on teaching parents to stimulate children through play, often utilising homemade toys or other readily available household items." P. 25.
- "These interventions were typically implemented at home by parents (usually mothers) who may vary greatly in their parenting skills and educational backgrounds." P. 25.
"In 1986-1987, the Jamaican Study enrolled 129 stunted children age 9-24 months that lived in poor disadvantaged neighborhoods of Kingston, Jamaica . … The stimulation intervention (comprising groups 1 and 3) consisted of two years of weekly one-hour play sessions at home with trained community health aides designed to develop child cognitive, language and psychosocial skills. Activities included mediating the environment through labeling, describing objects and actions in the environment, responding to the child’s vocalizations and actions, playing educational games, and using picture books and songs that facilitated language acquisition." Gertler et al. 2014, supplementary material, p. 3.
- 7
"Psychosocial Stimulation … In this approach, primary caregivers are taught the importance of a range of behaviors and skills necessary to support children's non-health-related developmental outcomes. These include learning the importance of, and using skills that, promote positive parent-child interactions with children, providing positive attention and responsiveness to developmental milestones and cues, encouraging children's autonomy and exploration of their environment, and promoting attachment (Engle and Lhotska, 1999)." Britto et al. 2017, p. 49.
- 8
"Parenting programme implementation varied in relation to dose of intervention, setting, and curriculum. The total amount of contact with parents, which ranged from less than 10 h to 120 h, did not have a clear association with the size of effect." Britto et al. 2017, p. 94.
- 9
- "Three delivery formats were common: home visits, group sessions, and clinic appointments. Home visits were used by nine studies in which a trained paraprofessional visited homes weekly or monthly to talk to and play directly with the child while the mother watched. Often play materials or picture books were left in the home to be replaced at the next session. The curriculum specified age-appropriate activities to conduct with the child at each session. In the second model, group sessions allowed a village peer educator to address small groups of mothers who might be asked to bring their children to the session. The manual of activities might include demonstrations of playing/talking activities to do with the child followed by having mothers practice and be coached. The children might range in age, so the focus would be on showing the mother what to provide her child. The third model used well- or sick-baby clinic visits as an opportunity to inquire into what the mother knew about stimulating her child with toys and talk and to counsel her on improved practices; this was usually done by a professional." Aboud and Yousafzai 2015, p. 444.
- "Parent-focused interventions were often delivered through home visits, community groups, and a combination of home visits, group sessions, community activities and primary healthcare and nutritional services. Few interventions worked primarily with parents or caregivers. Most programmes worked with parents or caregivers and children together and focused on promoting development in infants and toddlers (children under three years). Only eight interventions targeted parents of children three years or above. Many of these interventions were designed as an integrated part of, or as an add-on to, the existing healthcare system, thereby utilising professional or paraprofessional community health workers as instructors." Rao et al. 2014, p. 25.
- 10
"Most programmes worked with parents or caregivers and children together and focused on promoting development in infants and toddlers (children under three years). Only eight interventions targeted parents of children three years or above." Rao et al. 2014, p. 25.
- 11
"These interventions targeted children in deprived environments, and attempted to reverse the negative effects associated with risk factors such as poverty, low birth weight, iron-deficiency, undernutrition and growth retardation." Rao et al. 2014, p. 25.
- 12
- "A growing body of randomized evaluations has rigorously evaluated stimulation and nutrition interventions, and tested their separate and combined impacts." J-PAL, Early childhood stimulation.
- "It is possible that undernourished children could benefit more from stimulation when simultaneously given improved diets. The World Health Organization already recommends incorporating child development activities into the management of malnourished and sick children. Some practical advantages to integrating interventions are that the health and nutrition sectors are often the only services reaching children under 3 years of age. Furthermore, children with poor health and nutrition are also at risk of poor development." Grantham-McGregor et al. 2014, p. 11.
- 13
"We identify four main models for combining cash transfer and parenting programs: integrated, convergence, alignment and piggy-backing. The full technical report reviews 10 cash transfer programs in lower- and middle-income countries that included accompanying measures aimed at promoting positive parenting behaviors for child development in the early years. Country cases include Bangladesh, Colombia, Indonesia, Madagascar, Mexico, Niger, Peru, Rwanda, Senegal, and Burkina Faso, while recognizing that other interesting cases exist such as Brazil's Crianza Feliz program." Arriagada et al. 2018, p. 5.
- 14
- "The National Programme for Family Planning and Primary Healthcare (often referred to as the Lady Health Worker [LHW] programme) provides an opportunity to integrate early child development services at scale (panel 1). The LHW programme, started in 1994, is a government-supported community health service providing care to families in rural, remote, and disadvantaged communities across Pakistan. The focus areas for health services are family planning and maternal and child health, which are delivered by LHWs. The programme does not encompass early stimulation at present, and recent assessments have emphasised the need to strengthen nutrition services. The aim of this study was to measure the effects of a responsive stimulation intervention delivered by LHWs to families with infants and young children from birth to 24 months of age living in rural Sindh, Pakistan, either alone or in combination with an enhanced care for nutrition intervention." Yousafzai et al. 2014, p. 1283.
- "We study the promotion of ECD in rural China through a home-based parent training intervention implemented by one of the world's largest bureaucracies, the China Family Planning Commission (FPC)." Sylvia et al. 2018, p. 3.
- 15
"In 1986-1987, the Jamaican Study enrolled 129 stunted children age 9-24 months that lived in poor disadvantaged neighborhoods of Kingston, Jamaica. Enrollment was conditioned on stunting because it is an easily and accurately observed indicator of malnutrition that is strongly associated with poor cognitive development. Stunting was defined using international standards as having a height less than two standard deviations of reference data by age and sex. The children were stratified by age (above and below 16 months) and sex. Within each stratum, children were sequentially assigned to one of four groups by random assignment. The four groups were (1) psychosocial stimulation (N=32), (2) nutritional supplementation (N=32), (3) both psychosocial stimulation and nutritional supplementation (N=32), and (4) a control group that received neither intervention (N=33). All children were given access to free health care regardless of the group to which they were assigned. The stimulation intervention (comprising groups 1 and 3) consisted of two years of weekly one-hour play sessions at home with trained community health aides designed to develop child cognitive, language and psychosocial skills. Activities included mediating the environment through labeling, describing objects and actions in the environment, responding to the child's vocalizations and actions, playing educational games, and using picture books and songs that facilitated language acquisition." Gertler et al. 2014, supplementary material, p. 3.
- 16
- Grantham-McGregor et al. 1991 report on impacts immediately after the intervention. They do not report impacts in terms of standard deviations, but subsequent authors report effect size of 0.91 SD: "While significant, these short-term effects were substantially smaller than the effects seen in the 2 Jamaican studies (0.91 SD [21] and 0.42 SD [44]), the Pakistan study (0.6 SD [18]), and an earlier Colombian study (0.90 SD [45])." Andrew et al. 2018, p. 13. We use this effect size in our cost-effectiveness analysis, but we have not vetted this 0.91 SD estimate.
- 15-year follow-up: "Our results indicate that stunted children who receive home-based stimulation in early childhood compared with those who do not have sustained cognitive and educational benefits at age 17–18 years with effect sizes of 0·4–0·6 SD." Walker et al. 2005, p. 1806.
- 20-year follow-up: "The benefits of stimulation to IQ were substantial, with an effect size of 0.6 SD, and benefits to educational outcomes were wide-ranging and should have implications for future earnings." Walker et al. 2011, p. 855.
- 17
"We resurveyed both the stunted and nonstunted samples in 2007–2008, some 20 years after the original intervention when the participants were ~22 years old. We found and interviewed 105 out of the original 129 stunted study participants." Gertler et al. 2014, p. 999.
- 18
We use "earnings" and "income" interchangeably in this report.
- 19
"The nutritional intervention (groups 2 and 3) consisted of giving 1 kg of formula containing 66% of daily-recommended energy (calories), protein, and micronutrients provided weekly for 24 months. The nutrition-only arm, however, had no long-term effect on any measured outcome (36, 38). In addition, there were no statistically significant differences in effects between the stimulation and stimulation-nutrition arms on any long-term outcome, although the arm with both interventions had somewhat stronger outcomes (see supplementary materials, section D). Hence, we combine the two psychosocial stimulation arms into a single 'stimulation' treatment group and combine the nutritional supplementation–only group with the pure control group into a single 'control' group, understating the benefits of the joint intervention." Gertler et al. 2014, p. 999.
- 20
- See Table 1, "All job types." Gertler et al. 2014, p. 999.
- The p-value is step-down p-value, designed to account for multiple hypothesis testing. "We correct for the danger of arbitrarily selecting statistically significant treatment effects in the presence of multiple outcomes by performing multiple hypothesis testing based on the step- down algorithm proposed in (40)." Gertler et al. 2014, p. 999.
- We use the effect size reported for "All job types" because this seemed to be the most appropriate measure of the program's impact on earnings. Effect sizes are smaller for "Full-time jobs" (0.22) and larger for "Nontemporary jobs" (0.36). See Table 1, Gertler et al. 2014, p. 999.
- We translate log income effect of 0.30 into percentages by taking exp(1.30) - 1 = 0.35.
- 21
"This sample was balanced. We only observe statistically significant differences in 3 out of 23 variables at baseline (table S.1)." Gertler et al. 2014, p. 999.
- 22
"We estimate the impact of the stimulation intervention on earnings by comparing the earnings of the stunted treatment group to those of the stunted-comparison group. We control for potential bias from baseline imbalances using inverse propensity weighting (IPW) (39)." Gertler et al. 2014, p. 999.
- 23
Effect size on log earnings for "All Job Types" is 0.34 (vs. 0.30 in main specification). Table S.15. Gertler et al. 2014, supplementary material, p. 28.
- 24
"In addition, there is no evidence of selective attrition." Gertler et al. 2014, p. 999.
- 25
"Another issue is the selective attrition of the migrants. We were able to locate and interview 14 out of the 23 migrants. Among those 14 migrants, we found a significantly larger share of the treatment migrants than of the control migrants. Overrepresentation of treatment migrants can be a source of bias as migrant workers earn substantially more than those who stay in Jamaica. We address potential bias by imputing earnings for the nine missing migrants. We replace missing values with predicted log earnings from an ordinary least-squares regression on treatment, gender, and migration status. Imputing the missing observations reweights the data so that the treatment and control groups of migrants are no longer under- or overrepresented in the sample. In a sensitivity analysis, we omit migrants and still find strong and statistically significant effects of the program on earnings (see section D.4 of the supplementary materials)." Gertler et al. 2014, p. 1000
- 26
Effect size is 0.28. Table 3.14. Gertler et al. 2014, supplementary material, p. 27.
- 27
See Table S.8, Gertler et al. 2014, supplementary material, p. 21.
- 28
"Statistical inference is complicated by small sample size and multiple outcomes. We address the problem of small sample size by using exact permutation tests as implemented in (21). We correct for the danger of arbitrarily selecting statistically significant treatment effects in the presence of multiple outcomes by performing multiple hypothesis testing based on the stepdown algorithm proposed in (40). In addition, we aggregate over outcomes using a nonparametric combining statistic. Section C of the supplementary methods gives details." Gertler et al. 2014, p. 999.
- 29
This approach assumes psychosocial stimulation operates through its impact on cognitive skills. If it operates through, for example, impacts on both cognitive and social-emotional skills, then isolating the effect of cognitive skills would underestimate effects, relative to a long-term randomized controlled trial (RCT) on psychosocial stimulation programs like the Jamaica study. If cognitive and social-emotional skills are correlated and social-emotional skills are not included separately in the regressions, then this regression-based approach may be closer to the impact found in the experimental ideal. Because it is not clear which of these situations hold, it is best to view this exercise as a very rough plausibility check.
- 30
"An important question is whether the size of the effects of the stimulation arm on cognition is relevant from a developmental and economic standpoint. An effect of 0.26 SDs is usually considered small to moderate. To examine the clinical importance of this finding, we used data on 203 young adults drawn from the national longitudinal survey of youth 1979 (NLSY79), a survey of the biological children of women in the NLSY79 obtained from the US Bureau of Labor Statistics. For this sample we have both an early childhood developmental test as well as an ability score for the mother. From this we have estimated that an increase in the Peabody picture vocabulary test scores at age 4 years of 0.25 of a standard deviation is associated with a 7.5% increase in income at age 30 (P=0.057), once we control for maternal ability scores. This gain in income is equivalent to the gain from one extra year of schooling. Therefore if sustained the cognitive improvement is likely to be important in the longer term." Attanasio et al. 2014, p. 5.
- 31
- "A search conducted on three databases for nutrition or stimulation interventions with children ages 24 months and younger from developing countries yielded 1,750 unique articles published between 2000 and 2013." P. 439.
- "With inclusion criteria that limited articles to those with children who were 0 to 24 months old at the time of the intervention and at follow-up within a year and who were not premature or did not have a serious illness (e.g., very-low birth weight, autism, cancer), and using a mental development outcome measure of cognition and/or language, we identified 21 articles with a stimulation intervention and 18 with a nutrition intervention (several had both)." P. 439.
- 32
"Three delivery formats were common: home visits, group sessions, and clinic appointments. Home visits were used by nine studies in which a trained paraprofessional visited homes weekly or monthly to talk to and play directly with the child while the mother watched. Often play materials or picture books were left in the home to be replaced at the next session. The curriculum specified age-appropriate activities to conduct with the child at each session. In the second model, group sessions allowed a village peer educator to address small groups of mothers who might be asked to bring their children to the session. The manual of activities might include demonstrations of playing/talking activities to do with the child followed by having mothers practice and be coached. The children might range in age, so the focus would be on showing the mother what to provide her child. The third model used well- or sick-baby clinic visits as an opportunity to inquire into what the mother knew about stimulating her child with toys and talk and to counsel her on improved practices; this was usually done by a professional." Aboud and Yousafzai 2015, p. 444.
- 33
Their total sample includes 21 studies and 4,610 total observations, implying an average sample size of 220. These numbers are calculated by counting the studies and summing observations in Supplemental Table 1a of Aboud and Yousafzai 2015, supplementary material, pp. 3-12.
- 34
This is based on review of the "Design" column in Supplemental Table 1a of Aboud and Yousafzai 2015, supplementary material, pp. 3-12.
- 35
- From Figure 2: "Forest plot for effect sizes (standard mean difference represented as a blue square and 95% confidence interval represented as blue lines) of stimulation on the cognitive development of children. Overall effect size was 0.420 (95% CI 0.36, 0.48). The studies were heterogeneous (homogeneity statistic Q = 112.81, df = 21, p < 0.01). The random effects model was therefore more appropriate." P. 441.
- From Figure 3: "Forest plot for effect sizes (standard mean difference represented as a purple square and 95% confidence interval represented as purple lines) of stimulation on the language development of children. Overall effect size was 0.468 (95% CI 0.37, 0.56). The studies were heterogeneous (homogeneity statistic Q = 46.66, df = 8, p < 0.05). The random effects model was therefore more appropriate." P. 442.
- 36
These two studies are World Bank 2017a/World Bank 2017b and Hartinger et al. 2017.
- 37
See the "Findings" column of the "RCTs of short-term impacts on cognitive development" table here.
- 38
See the "Findings" column of the "RCTs of longer-term impacts on cognitive development" table here, p. 9.
- 39
See the "Findings" column of the "RCTs of longer-term impacts on cognitive development" table here, p. 9.
- 40
This calculation comes from the following two sources:
- Grantham-McGregor et al. 1991 report on impacts immediately after the intervention. They do not report impacts in terms of standard deviations, but subsequent authors report effect size of 0.91 SD: "While significant, these short-term effects were substantially smaller than the effects seen in the 2 Jamaican studies (0.91 SD [21] and 0.42 SD [44]), the Pakistan study (0.6 SD [18]), and an earlier Colombian study (0.90 SD [45])." Andrew et al. 2018, p. 13. We use this effect size in our cost-effectiveness analysis, but we have not vetted this 0.91 SD estimate.
- "Our results indicate that stunted children who receive home-based stimulation in early childhood compared with those who do not have sustained cognitive and educational benefits at age 17–18 years with effect sizes of 0·4–0·6 SD." Walker et al. 2005, p. 1806.
We estimate fade out of 44% (0.4/0.91) to 66% (0.6/0.91). Because these effects on cognitive and related outcomes are based on different tests conducted at different ages, we view these as very rough approximations on fade out.
- 41
"Save the Children responded to this challenge by creating an early stimulation program in Bangladesh that uses the existing government infrastructure—community clinics and community health care and family planning providers—as its primary delivery mechanism. Rather than training and deploying separate ECS workers to implement the program, Save the Children supplemented the training of NNS workers to incorporate the program's early stimulation message and integrate it with the nutritional and family planning messages they already deliver to families. In addition, Save the Children provided specially designed ECS program materials and trained service providers to deliver early stimulation messages and program materials during their routine home visits, and in Expanded Program of Immunization (EPI) events, community clinics, and Growth Monitoring Campaigns. During the counseling, service providers were expected to show mothers and other caregivers how to use a child development card and books handed out to the families to provide children with a variety of early learning opportunities." Chinen and Bos 2016, p. 8.
- 42
"This paper investigates the impact of an early child development intervention that targeted parents of 0-3 years old children in rural Rwanda, called First Steps (Intera za Mbere). The program was developed and implemented by Save the Children in collaboration with Umuhuza (a Rwandan NGO) in the Ngororero district in the Western province of Rwanda." Justino et al. 2019, p. 2.
- 43
"This paper examines the impact of an Integrated early childhood development Parenting Intervention that was developed and implemented by Catholic Relief Services (CRS)-Sierra Leone, in partnership with the Koinadugu District Health Management Team during March 2016 through September 2017 in the four chiefdoms of Koinadugu district." Chandra et al. 2019, p. 5.
Note: We exclude this study from the studies reviewed here because it does not measure impact on cognitive outcomes.
- 44
"Pratham is the implementing partner for the studies in Odisha." GiveWell's non-verbatim summary of a conversation with Thomas Chupein and Ariella Park, May 1, 2017, p. 3.
- 45
- Gertler et al. 2014 note current estimates for 22-year-olds may be underestimating earnings impact: "One issue is that in the treatment group, there are more individuals who both work and attend school full-time than in the control group. Working, full-time students are likely to have lower earnings than nonstudents with the same education. Hence, observed average earnings likely understate the long-run earnings of the treatment group more than the control group, implying that we underestimate the long-run effects of treatment on earnings." Gertler et al. 2014, p. 1000.
- In a conversation with GiveWell, Thomas Chupein and Ariella Park from the Abdul Latif Jameel Poverty Action Lab (J-PAL) noted that there is a "30-year follow-up to the Jamaica study, led by Christel Vermeersch, which is ongoing and will conclude in 2018." GiveWell's non-verbatim summary of a conversation with Thomas Chupein and Ariella Park, May 1, 2017, p. 3.
- 46
"We found no significant benefits from supplementation. Participants who received stimulation reported less involvement in fights (odds ratio: 0.36 [95% confidence interval (CI) 0.12–1.06]) and in serious violent behavior (odds ratio: 0.33 [95% CI: 0.11–0.93]) than did participants with no stimulation. They also had higher adult IQ (coefficient: 6.3 [95% CI: 2.2–10.4]), higher educational attainment (achievement, grade level attained, and secondary examinations), better general knowledge, and fewer symptoms of depression and social inhibition." Walker et al. 2011, p. 849.
- 47
We found at least one study that finds an impact of a cash transfer program on cognitive development. "Cash transfer programs have become extremely popular in the developing world. A large literature analyzes their effects on schooling, health and nutrition, but relatively little is known about possible impacts on child development. This paper analyzes the impact of a cash transfer program on early childhood cognitive development. Children in households randomly assigned to receive benefits had significantly higher levels of development nine months after the program began. There is no fade-out of program effects two years after the program ended. Additional random variation shows that these impacts are unlikely to result from the cash component of the program alone." Macours et al. 2012, p. 247.
- 48
If increased schooling entails increased costs, these might partially offset impacts on income.
- 49
Chandra, A., Mani, S., Dolphin, H., Dyson, M. “Impact Evaluation of an Integrated Early Childhood Parenting Program in Sierra Leone.” Working Paper.
- 50
Justino, P., Leone, M., Rolla, P., Abimpaye, M., Dusabe, C., Uwamahoro, D., Germond, R. "Improving Parenting Practices for Early Child Development: Evidence from Rwanda." Working Paper. 5 Sept. 2019.