Investigation of alternative hypotheses for vaccination coverage increases during VillageReach’s pilot project

Published: May 2013

Background

In our July 2009 review of VillageReach, we attributed an increase in vaccination rates in Cabo Delgado to VillageReach’s program. VillageReach was our top-rated organization for 2009, 2010 and much of 2011 and it has received over $2 million due to GiveWell's recommendation. In 2012, we revisited this conclusion and laid out our reasoning in this blog post.

We sought to investigate possible alternative hypotheses for the increase in vaccination coverage in Cabo Delgado between 2003 and 2008. Our plan was to seek out conversations with people who might have knowledge of the context in which VillageReach carried out its pilot project, particularly funding sources for health programs, changes in government attention to vaccination, new programs, and involvement of other organizations in health programs in the province and surrounding provinces.

Our conclusion

Due to our difficulty finding and speaking with people who were well informed about the situation in Cabo Delgado during the period of VillageReach's pilot project (possibly because so much time had passed) or finding relevant data to address our questions, this project did not significantly increase our understanding of the drivers of the increase in immunization rates in Cabo Delgado between 2003 and 2008.

Research plan

We sought conversations with:

1. VillageReach staff who worked on the pilot project, particularly those who had been involved in selecting Cabo Delgado.
2. Government of Mozambique health department staff.
3. Staff of non-governmental organizations working on health in Mozambique during the pilot project. We sought conversations with Mozambique staff at UNICEF, the World Health Organization, USAID, Irish Aid, and the World Bank.

We were also interested in reviewing data on health spending and the work of other actors; we have not found data on this.

For each conversation we had, we asked (with some modifications depending on the context and the stage of our investigation):

• What factors led to the increase in vaccination rates in Cabo Delgado between 2003 and 2008?
  • What funding was available for vaccination? Was this a change from before?
  • What organizations were working on vaccination or health systems in Cabo Delgado during that time?
  • Were there changes in leadership related to health in the area?
  • Were the changes specific to Cabo Delgado, regional, national, or international?
• What factors led to the increase in vaccination rates in next door Niassa province?
  • Similar sub-questions
• Did something change in Cabo Delgado and Niassa specifically, that wasn't seen in the rest of the country?
• What additional factors could have affected vaccination rates in Cabo Delgado between 2008 and 2011?

Published conversations

• Manuel Novela (bio): Dr. Novela has been the Enlarged Program of Immunizations (EPI) Program Officer at the World Health Organization Mozambique office since 2007. We spoke to him in March 2012 and published notes from our conversation.
• Karin Turner: Deputy Director Health Systems for USAID Mozambique since 2010. We spoke with her and other USAID Mozambique staff members in April 2012 and published notes from our conversation.

Others that we sought out conversations with

We sought conversations with all of the people below.

In some cases, we got in touch with the individual and discussed our project with them; in some cases, we did not. We do not have permission from the people with whom we did speak to publish either the fact that we spoke with them or notes from the conversation.

We provide below the full list of individuals whom we hoped to speak with for the sake of transparency regarding our process.

• VillageReach staff members involved in the selection of Cabo Delgado for the pilot project.
• Mark Kane: Author of the evaluation of VillageReach’s pilot project and vaccination specialist.
• Dr. Mussa Hagy: Director of Health in Cabo Delgado.
• José Alexandre Chivale (bio): NPO–Logistician for Enlarged Program of Immunizations (EPI) since 2007. His past experience includes positions in the Mozambique Ministry of Health and in the Department of Health in Cabo Delgado.
• Nuno Gaspar: Mozambique Ministry of Health EPI manager.
• Terry Hart (bio): Immunization logistics advisor to UNICEF and other organizations.
• John Lebga: UNICEF Immunization Specialist.
• Onei Uetela: UNICEF Mozambique EPI Specialist.
• Julie Cliff: Professor of Medicine at Eduardo Mondlane University in Maputo, Mozambique.
• Jonas Chambule: Health Programme Adviser at Irish Aid.
• Laura Rose: World Bank Mozambique Senior Economist.

Further context provided by VillageReach

We spoke with Leah Hasselback from VillageReach about our questions in May 2012. Ms. Hasselback told us:

• In completing its evaluation of the pilot project, VillageReach had spoken with the World Health Organization (WHO) as well as with bilateral donors, and that no one had mentioned Cabo Delgado’s using common funds for immunization or additional immunization-specific funding for Cabo Delgado. Note that VillageReach’s assessment of other factors was completed after the fact.
• Several other factors occurred in Cabo Delgado between 2007 and 2010 that may have caused immunization rates to stay higher following the pilot project:
  • Mozambique introduced the pentavalent vaccine in November 2009. This vaccine, which includes 5 needed vaccines in one, was accompanied with significant vaccine-related promotion which also should have improved immunization rates.
  • Cabo Delgado added 20 additional health centers between the end of VillageReach’s pilot project and its beginning its scale up work. During the entire period of the pilot project, Cabo Delgado added only 1 health center.
  • There were immunization campaigns in 2008 that focused specifically on measles and polio.
  • FDC, the local NGO with which VillageReach partnered during the pilot project, ran a social mobilization campaign in 2008-09 in a single district of Cabo Delgado.