Dimagi — CommCare Connect Pilot (May 2024)

Note: This page summarizes the rationale behind a GiveWell grant to Dimagi. Dimagi staff reviewed this page prior to publication.

Published: October 2024

Background

Dimagi is a for-profit social enterprise that builds software to assist frontline workers (FLWs) – also known as community health workers (CHWs) – to visit households in low- and middle-income countries, primarily through its CommCare platform.1 In November 2023, we recommended a $49,722 grant to conduct scoping and develop a proposal for piloting its new healthcare platform, CommCare Connect (CCC), for use for child health campaigns (CHC) that consist of the paid door-to-door delivery of health commodities to children under five in communities with high child mortality and low prior health commodity coverage. (In contrast, the standard CommCare platform is intended for a wider set of use cases and doesn’t provide donors with the option of paying for outputs like verified child visits.2 )

CommCare Connect aims to increase commodity coverage by better incentivising FLWs to provide highly cost-effective health commodities to children who would otherwise not receive them, due to a lack of coverage by existing health systems.3 Our general understanding is that by default, it is common for FLWs to receive little, or no, remuneration for child visits they make. CommCare Connect incentivizes FLWs to distribute health commodities in these low coverage areas by paying them per verified child visit (and confirming they haven't already received the specific health commodity). FLWs are employed by locally-led organizations (LLOs), and Dimagi has historically paid LLOs around $2 per verified child visit.4

So far, CCC-CHC has operated at a relatively small scale: at the time we recommended this grant, Dimagi had supported 90,000 child visits in total across six pilots.5 Initially, Dimagi has focused on the distribution of vitamin A supplementation and deworming pills,6 but the platform is flexible, and Dimagi has been open to using it for additional highly cost-effective commodities, like oral rehydration solution (ORS) and zinc (which it incorporated into CCC-CHC’s standard package of commodities in March 2024), vaccine referrals, and malnutrition screenings.7

The grant

Grant purpose

The scoping grant we recommended in November 2023 sought to allow Dimagi to establish whether there is a sufficiently large total addressable market for CCC-CHC, and to develop a proposal for a potential pilot for the platform in Nigeria, which would focus on reducing uncertainties around its implementation. More information about our previous scoping grant is here.

The grant we are now recommending provides funding for an iteration of the pilot that Dimagi proposed. The goal of this pilot is to find out whether the program model is feasible as envisioned across multiple locations within Nigeria, with a variety of LLOs. We expect this pilot to result in around 300,000 child visits, or more than three times as many visits as the previous pilots of the program combined.8 We expect the pilot to allow Dimagi to resolve a number of key, open questions about the implementation of CCC-CHC (see below), which we consider a necessary step prior to any more thorough evaluation of the counterfactual impact of the program on commodity coverage.

Grant activities

We are recommending a grant of $1,000,186 for the pilot of CCC-CHC. With this funding, Dimagi will sign contracts with at least five LLOs to use CCC-CHC to distribute vitamin A supplementation, ORS and zinc, vaccine referrals, malnutrition screenings, and potentially deworming pills in at least three states in Nigeria.9

Dimagi’s pilot will:

• Work with five LLOs across at least two target states in Nigeria. Dimagi has received over 40 letters of support from LLOs operating in ten states in Nigeria with high health burdens.10 Following the recommendation of this grant, GiveWell and Dimagi will jointly decide which of these LLOs are best suited for participation in the pilot.
• Codify and test a set of standard operating practices for CCC-CHC. These procedures are designed to ensure CCC-CHC is being implemented consistently and with fidelity at scale. In the pilot, Dimagi will put particular emphasis on testing its procedures for monitoring and verification, to ameliorate the risk of fraud or collusion.11
• Gather data to reduce GiveWell’s uncertainties about the platform. In particular, Dimagi expects to be able to reduce our uncertainties around cost, the availability and performance of LLOs, and whether CCC-CHC can be implemented as intended.12
• Inform the design and preparation of an RCT. Following the conclusion of the operational pilot, Dimagi intends to run an RCT of the platform’s effect on counterfactual commodity coverage, which will be funded by USAID Development Innovation Ventures.13 The pilot will inform the research design and power calculations for this study.14

As of March 2024, Dimagi has executed smaller-scale pilots with six LLOs in five countries, conducting approximately 90,000 child visits in total.15 While these pilots were useful in informing CCC-CHC’s model and improving service delivery,16 they were not designed to target areas with high disease burdens, and focused on delivering a smaller range of commodities.17 This pilot will expand the number of health commodities distributed,18 and will allow us to gauge whether there are enough LLOs that are interested and able to effectively implement CCC-CHC in high-burden areas in Nigeria. At the larger scale of this pilot, Dimagi will also be able to work on standardizing its model through its standard operating procedures, which were not in place during its earlier pilots19 , and to improve and demonstrate its ability to manage many LLOs at once20 . Finally, there are some parts of CCC-CHC, such as child visit verification, where we would like to see Dimagi gain additional experience (which this larger pilot will provide) in order for us to be confident in the platform as a whole.

• Dimagi can successfully recruit and execute contracts with multiple LLOs in parts of Nigeria with high child mortality. (Prior to this trial, Dimagi had partnered with one LLO in Nigeria.22 ) We’ll learn about this through direct observation of whether Dimagi can execute these contracts. We’ll also assess Dimagi’s recruitment and contract execution process by learning whether Dimagi can reduce the time between contract signing and first child visit, and whether LLOs complete at least 1000 child visits within 120 days of contract signing, as its contracts currently stipulate.23
• Dimagi can convincingly verify whether child visits occurred. Dimagi is currently analyzing the data from its earlier pilots, which it believes will inform its strategies for verifying child visits in the future.24 We’ll learn about Dimagi’s ability to verify child visits as Dimagi develops a monitoring and verification standard operating procedure that lays out what data is captured via the CommCare Connect app, how its GPS verification process works, and to what extent that data will be spot checked via high-frequency checks.25 However, child visits will not be independently verified by an external organization, and while Dimagi anticipates being able to detect large-scale fraud that would lead to termination of contracts with LLOs, it’s unlikely that they’d see that in this small set of LLOs they’ll partner with for the pilot.26
• Dimagi’s standard operating procedures are comprehensive and complete. Because Dimagi’s procedures are newly developed or in the process of being finalized through this pilot, we do not yet know how well they will work. To learn about this, we’ll rely on Dimagi’s self-assessment of the procedures’ status, and on our own review of the procedures at the end of the grant period. We do not expect to feel confident that the procedures are fully comprehensive by the end of the pilot, as we expect that they will need to evolve over time as CCC-CHC scales.
• LLOs can implement CCC-CHC as intended (that is, with fidelity to the standard operating procedures). We’ll learn about this through data that Dimagi will collect on the outputs of the program (e.g. the number of child visits conducted, the number of FLWs employed per LLO, and the number of repeat visits),27 through demographic data for household visits (to confirm FLWs are targeting the right ages for specific commodities),28 and through qualitative information on FLW training and certification.29 The pilot will not include a formal process evaluation.30
• FLWs are paid what they should, when they should be. At present, Dimagi has a standard monthly payment invoicing procedure, but the terms of the payments vary from LLO to LLO.31 In this larger pilot, Dimagi plans to finalize a financial management and accountability standard operating procedure that sets out how its pay-for-performance model will work, and establishes guidelines for invoicing, payment processing, and audits.32 We anticipate that through this, Dimagi will be able to track metrics including the number of eligible child visits for payment, and the number of “on-time” payments (e.g. paid within 30 days).
• The five proposed bundled interventions are good fits for CCC-CHC. To learn about this, we’ll rely on Dimagi’s assessment of the particular interventions, and on feedback from LLOs to Dimagi. If any of the commodities are particularly bad fits for the program, we would expect to see this in the verification data. (For example, if there were regular stockouts of ORS, we would expect FLWs to have limited access to it, and therefore distribution to be low.)
• The cost per child visit on average is within the range that would make CommCare Connect look plausibly cost-effective to GiveWell. At present, our best guess is that the cost per child visit through CCC-CHC is around $2.20, a figure which, combined with our other assumptions about the effectiveness of the platform, is sufficient to put the use of CCC-CHC for distributing vitamin A supplementation and deworming pills at our cost-effectiveness bar of 10 times that of unconditional cash transfers.33 Dimagi will quantify the total costs associated with conducting household distribution of the prespecified health commodities for Dimagi and the LLOs, and potentially also for governments.34 This should allow us to gauge the accuracy of our current cost estimates.

Budget for grant activities

The $1,000,186 grant includes:35

• $645,000 in payment to LLOs. This includes:
  • $75,000 for start-up costs. The five new LLOs will receive $15,000 upon signing the contract;
  • $570,000 for verified child visits by FLWs, paid to the LLOs.
• $355,18636 for Dimagi’s project costs. This includes:
  • $144,219 for data collection, data analysis, and operational costs;
  • $70,750 for travel and security;
  • $48,882 for the development and refinement of the standard operating procedures;
  • $19,732 for development of onboarding materials for LLOs;
  • $71,602 for overhead.

Timeline

The grant activities will last for 18 months. This is broken down as follows37 :

• Months 1-4: set-up. During the first four months, Dimagi will codify its standard operating procedures for the platform, and sign contracts with LLOs.
• Months 5-15: implementation and monitoring. In this phase, Dimagi will oversee the use of the platform by the LLOs. This phase will resemble work Dimagi conducted in late 2023 and early 2024, where it oversaw 40,000 deployments of the program by two LLOs. It will include work to test and solidify the platform’s processes and standard operating procedures at greater scale, tracking data to reduce our uncertainties around cost, implementation, and the risk of fraud or collusion.
• Months 16-18: analysis and reporting. In this phase, Dimagi will analyze and report on the pilot and, if appropriate, develop proposals for further work.
• Throughout the project: preparation for an RCT. Dimagi will work with GiveWell and other organizations to iterate on the design of an RCT that would measure the impact of the platform on counterfactual commodity coverage.

The case for the grant

We are recommending this grant because:

• We expect to learn about a platform that could be a cost-effective way to increase coverage for a variety of health commodities GiveWell supports. Because CommCare Connect targets areas with low counterfactual healthcare commodity coverage and high mortality risk, we currently model the use of the platform for the delivery of vitamin A supplementation and deworming pills in Nigeria as at least 10 times as cost-effective as cash transfers.38 However, CommCare Connect could be used for delivering a wide range of other commodities that GiveWell believes are cost-effective and have coverage gaps. This implies there is potential for a large amount of cost-effective grantmaking with CommCare Connect in the future, across a number of GiveWell’s existing intervention areas (e.g. water, nutrition). As an indication of magnitude, we estimate that in nine states in Nigeria with large health burdens, there may be $20m in room for more funding annually for vitamin A supplementation and ORS and zinc alone.
• We think that learning whether CommCare Connect works at the scale of this pilot is a necessary precondition for any subsequent randomized evaluation or scale-up of the platform. The purpose of this grant is to assess whether the platform’s implementation ‘works’ as intended, prior to then assessing the impact of the platform on counterfactual coverage, which will be the key driver of its cost-effectiveness. Dimagi is currently working on the design of an RCT that would measure its counterfactual impact.39 If the pilot shows that the program model is viable, and the RCT provides evidence that CommCare Connect increases coverage relative to the counterfactual, GiveWell would then be interested in funding Dimagi at scale.
• We believe Dimagi is aligned with GiveWell’s values of transparency and impact, and it has been highly responsive to GiveWell’s interests to date. So far, Dimagi has been easy to work with, and has actively explored ways of making CommCare Connect more cost-effective by our standards, including through consideration of commodity choice, potential partnerships with other organizations, and mechanisms for increasing take-up. Dimagi has also secured over 40 letters of support from LLOs that are eager to participate in CCC-CHC, which we take as a signal that there is demand for this approach.40

Risks and reservations

Our main reservations about this grant are:

• We do not know whether Dimagi’s technology will be usable by a variety of LLOs. CommCare Connect has been piloted before, in Nigeria, India, Kenya, Zambia, and Tanzania, so we have some assurance that the platform is viable and in some level of demand.41 However, it is still possible that it will fail for other reasons, such as collusion or fraud, or because Dimagi is unable to address some of our key implementation questions, as described above. This could result in CommCare Connect not being viable at scale.
• Dimagi is a for-profit organization, and it’s possible that there are risks associated with granting them funds or that their orientation does not align with GiveWell’s values. We expect to structure the grant agreement and financial reporting requirements in a way that provides transparency in how the funds are spent. In this case, we’re less concerned about lack of alignment because we have a strong qualitative impression of Dimagi and its platform CommCare, and we understand they’ve received or are currently discussing grants from philanthropic donors like the Bill and Melinda Gates Foundation and USAID Development Innovation Ventures.42
• There may be a smaller total addressable market for the CommCare Connect than we currently expect. In Nigeria, there are already a number of organizations that supply commodities incentivised by CommCare Connect. For example, the Clinton Health Access initiative distributes ORS and zinc (and we have been discussing ORS and zinc distribution with New Incentives),43 while Malaria Consortium distributes vitamin A supplementation alongside seasonal malaria chemoprevention (SMC) in Bauchi and Niger states.44 We’re not sure about the future scale of these organizations’ operations for these commodities, or whether Dimagi could pivot to other commodities if ORS and zinc and vitamin A supplementation have relatively high coverage.
• The pilot will not answer all of our key questions about CommCare Connect. Since the pilot will only involve five LLOs, it will have limited generalizability. Furthermore, even if the pilot is successful, we will still be uncertain about the extent to which the platform results in increases in counterfactual commodity coverage. Dimagi is exploring an RCT to assess this.45
• It’s possible we should have provided a smaller grant for Dimagi to partner with fewer LLOs to mitigate the risk that there are major flaws with the CCC-CHC platform. However, it’s possible for GiveWell to recover funds for child health visits that do not happen, limiting the downside risk of this scenario. More than half ($570K of the grant) will go to compensating frontline workers for child health visits. If Dimagi isn’t able to find organizations that can complete training and visits, then those funds wouldn’t be disbursed and we’d have the opportunity to consider whether GiveWell wanted to “try again” with a different organization or take back funding.
• CommCare Connect may increase the workload of FLWs, who are already stretched thinly. It is possible that the financial incentives provided by CCC-CHC could result in FLWs becoming overworked, or in ‘crowding out’ other work that FLWs might have done in their communities instead. However, our general understanding is that many FLWs are both part-time and unpaid,46 so there is room for them to take on additional work, and CCC-CHC could lead to improvements in their welfare. Dimagi plans to study the issue of crowding out further as part of the RCT.47

Plans for follow up

• Immediately after recommending this grant, GiveWell will jointly decide with Dimagi which LLOs should be selected for participation in the pilot and in which Nigerian states.
• To follow the progress of the grant, we will check in with Dimagi every six months over the course of the 18 month pilot period.

Internal forecasts

For this grant, we are recording the following forecasts:

Our process

• Following the scoping grant we recommended in November 2023, Dimagi shared a pilot proposal that responded to our main questions, and forms the basis for this grant recommendation.
• Prior to recommending this grant, we had multiple calls with Dimagi to discuss options for the pilot. We also discussed the possibility of an RCT of Dimagi’s platform with another funder.
• Prior to the scoping grant, we discussed the potential of CommCare Connect with Chuck Slaughter, the founder and former CEO of Living Goods.
• Internally, we received feedback on the potential of CommCare Connect from our vaccines grantmaking team.

Sources

• 1
  • “Dimagi is a global tech for good social enterprise enabling impactful frontline work through scalable digital solutions and expert services … at least 50% of people lack access to essential health services. We help bridge this gap by empowering Frontline Workers to deliver critical services to communities in need.” Dimagi, “About Dimagi.”
  • CommCare Connect is a software-as-as-service mobile health platform that allows community health workers to track and maintain patient data. It is designed “to provide opportunities for Frontline Workers to learn, deliver, verify and be paid for high-impact interventions”. “In our long-term vision, individual workers will create free accounts on CommCare Connect and gain access to opportunities to deliver high-impact interventions, such as participating in vaccine campaigns, leading educational programs, or disseminating important information to households about early childhood development or family planning. Every opportunity on CommCare Connect will require the worker to use a provided mobile application to ensure accurate and verifiable delivery of the intervention, as well as payment to the worker.” Dimagi, “Delivering Child Health Campaigns with CommCare Connect”.

• 2
  • Use cases listed on the CommCare site include Monitoring & Evaluation (M&E), Information and Communications Technologies for Development (ICT4D), and research. Dimagi, CommCare.
  • “Dimagi’s new CommCare Connect initiative will enable Frontline Workers to provide additional services, further catalyzing their impact at scale. … Service delivery is verified - leveraging biometrics, GPS, and data algorithms.” Dimagi, CommCare Connect - 2023 in Review.

• 3

  For example, children in especially rural or remote areas are relatively less likely to be covered by these networks. “At least 50% of people lack access to essential health services. We help bridge this gap by empowering Frontline Workers to deliver critical services to communities in need.” Dimagi, "About Dimagi."

• 4
  • Locally-led organizations are Dimagi’s term for organizations (for-profit or non-profit) that organize and employ health workers.
  • “In the second mode of deployment (d2org), which is the one CCC-CHC utilizes, we partner with locally-led organizations (LLOs) to deploy high-impact interventions. Dimagi sets up pay-for-performance contracts with LLOs in which they get paid for digitally verified service delivery of FLWs using CCC apps. The FLWs can be government supported CHWs or field staff recruited and managed by the LLO. The LLOs manage many aspects of deploying interventions such as procurement, training, supervision, government engagement, and community mobilization.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 3.
  • “Current projects are on track to reach 100,000 children by Q1 2024 at $2 per verified visit, in Kenya, Nigeria, Tanzania, Zambia, and India.” Dimagi, Scaling Child Health Campaign through CommCare Connect, June 2023 (unpublished).
  • For a breakdown of the cost per child visit, see GiveWell, “Dimagi CommCare Connect BOTEC” (‘CCC Costs’ tab).

• 5
  • “Dimagi has signed pay-for-performance contracts with six LLOs in five countries that have delivered over 95,000 child visits.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 6.

• 6

  “We started work on CCC-CHC In Q2 2022. Our first pilot delivered only vitamin A and deworming medicine.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 6.

• 7
  • “Dimagi is excited that GiveWell sees CCC as a potential platform for other commodities and interventions. This aligns well with Dimagi’s view of CCC.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 13.
  • “ORS and zinc have not yet been distributed through CCC, but Dimagi is enthusiastic about adding it to the standard CCC-CHC program … Dimagi’s understanding from several communications with CHAI (and also confirmed by GiveWell) is that the intervention is a single visit that drops off two treatments of ORS per child. CCC-CHC hopes to deliver ORS at a comparable price point, along with Vit A, deworming, vaccine promotion, and malnutrition screening.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 11.
  • Dimagi informed us that they had begun incorporating zinc into the standard commodities package by March 2024. Neal Leash, Chief Strategy Officer and Co-President, Dimagi, comments on a draft of this page, September 16, 2024.

• 8
  • “Dimagi has signed pay-for-performance contracts with six LLOs in five countries that have delivered over 95,000 child visits.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p 6. 95,000x3 < 300,000.
  • For more exact figures on expected number of child visits funded, see GiveWell, “Dimagi CommCare Connect BOTEC” (‘Pilot options’ tab).

• 9
  • “We will work in 2-3 of the priority states listed above, in Nigeria, to be chosen with mutual agreement with GiveWell. We will further identify specific regions within states that are reachable by LLOs and have substantial need in terms of U5 mortality, suspected VA coverage, etc.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 21. The exact states will be determined by Dimagi and GiveWell following the recommendation of this grant.
  • Note that while the above quote states an intention to work in 2-3 states, we have been informed by Dimagi that they now intend to choose three or more states, as well as to sign contracts with at least five LLOs. Neal Leash, Chief Strategy Officer and Co-President, Dimagi, comments on a draft of this page, September 10, 2024.

• 10

  Dimagi, email to GiveWell, March 22, 2024 (unpublished).

• 11

  “Our implementation plan has the following elements … codify and test a set of SOPs for running CCC-CHC. In particular, expand and pressure test our monitoring and verification SOPs.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 23.

• 12

  “Our implementation plan has the following elements … Assess and gather data from these deployments in order to reduce GiveWell’s uncertainty around cost, adverse outcomes, availability of LLOs, etc.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 23.

• 13

  “Phase 3, Evaluation (starts in 2025): If Phase 2 results are encouraging, a natural next step would be a rigorous evaluation of the CCC-CHC implementation approach that was solidified in Phase 2. We expect the research would focus on the question of whether CCC-CHC increases counterfactual coverage.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 2.

• 14
  • “Preparing for RCT [throughout project]: Throughout the project we will work with GiveWell and IPA Nigeria to flesh out the RCT design for phase 3.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 24.
  • “Our implementation plan has the following elements … Design and prepare to implement an RCT after the conclusion of the operational pilot, including research design, power calculations, etc.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 23.

• 15

  Organization	Country	Category	Status	Number FLWs distributing Vit A and Deworming	Number of visits completed	Number additional visits expected by March 31, 2024
  Acclaim / EIM	India	Pre-pilot	Completed	10	1,592	0
  Cohesu	Kenya	1st round pilot (5k visits)	Completed	10	5,277	0
  LiveWell	Zambia	1st round pilot (5k visits)	Completed	10	5,095	0
  Sujukwa	Tanzania	1st round pilot (5k visits)	Launching	20	0	5,000
  C-WINS	Nigeria	2nd round pilot (40K visits)	Launching	41	0	40,000
  Sanmat	India	2nd round pilot (40K visits)	Launching	30	0	40,000

  Dimagi, Responses to GiveWell’s questions, August 17, 2023 (unpublished).

• 16

  Some specific learnings include:

  • “Real time vs. retrospective data: In the pre-pilot there was a lot of retrospective data collection despite our training to use the app in real-time. We have increased emphasis on the need to use the app in real-time, and do a better job of monitoring the data (timing, GPS points) to provide feedback early on in the project. We’ve also understood that there needs to be some flexibility in how FLWs can use the app while at a household, since it is often natural to engage with the household members and then enter data in small batches.
  • App and reporting improvements: Our mobile apps are much better than they were at the beginning of the first round pilots, and we have developed much more robust real-time reporting systems (which we are further upgrading). The app now contains various checkpoints to ensure that service delivery parameters are not missed and the reasons for non-delivery of services is documented. We have also included steps in the app to ensure that most optimal GPS coordinates are captured, an option to search and flag potential duplicate cases, and properties which can automatically categorize service delivery status to quickly flag pay/no-pay cases. This is done through a customized algorithm which has been developed based on verification rules we have defined for the implementation. For areas with lower smartphone and data coverage, we have also included short code-based SMSs to send key nutrition messages from our health and nutrition chatbot. In-built backend features like timestamps for each form combined with GPS and photograph capture wherever applicable will make it much harder to conduct incorrect data entry and reduce attempted frauds. Our supervisor app will be used to flag and verify potential duplicate cases and also flag suspicious cases of fake data entry as well as conduct monitoring visits for a representative sample of beneficiaries.
  • Contracts: We’ve improved our contract and contracting process substantially. The pre-pilot and pilots took weeks or months to close, with a lot of back and forth. We were able to sign the second round ones in about a week each. And we expect to soon have a clear and non-negotiable contract that we can share in advance for future pilots, before they apply to our program. The contracts are now much clearer in terms of how payment works, which we hadn’t fully worked out when we signed the 1st round ones.
  • Targeting areas of higher need: As discussed elsewhere, in the 1st round pilots we didn’t assess U5 mortality or target areas of higher malnutrition. This is reflected in our MUAC data. In the 1st round pilots we identified only a handful of children with “yellow” status. In the first week of work in Nigeria, already 32 out of 471 children were identified as yellow or red.
  • FLWs and pay: One learning from these projects and other CCC efforts is that many FLWs do not require or expect clarity on how their payment will work, and may accept being told they will be paid for work without knowing exactly how much. We’ve shifted to asking for more clarity for ourselves from the partner organizations on how payment to FLWs will work and will be requesting partner organizations to confirm details (how much and by what means) once each payment cycle is completed.” Dimagi, “Learnings from pre-pilot and 1st round pilots and improvements in 2nd round pilots” (unpublished).

• 17
  • “As discussed elsewhere, in the 1st round pilots we didn’t assess U5 mortality or target areas of higher malnutrition.” Dimagi, “Learnings from pre-pilot and 1st round pilots and improvements in 2nd round pilots” (unpublished).
  • “Our first pilot delivered only vitamin A and deworming medicine.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 6.

• 18

  “For Nigeria, CCC-CHC will bundle five intervention services: vitamin A, deworming medicine, vaccine check and promotion, malnutrition screening and referral, and free distribution of an ORS/zinc.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 15.

• 19

  “At a high level, our implementation plan has the following elements: … Codify and test a set of SOPs for running CCC-CHC. In particular, expand and pressure test our monitoring and verification SOPs.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 23.

• 20

  “First, we have an interest in working with several LLOs (vs just one or two) so that our findings are more generalizable. We also want to improve and demonstrate Dimagi’s ability to manage many LLOs at once.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 22.

• 21

  “If GiveWell decides to proceed, this phase would involve scale up of CCC-CHC in Nigeria in order to address uncertainty around costs, verification, and adverse outcomes, as well as validate some of the claims made about total addressable market and ability to recruit partner orgs. In this phase, we will codify and test our Standard Operating Procedures (SOPs) to run CCC-CHC at scale. We will design an RCT with our research partner, in order to be ready for phase 3.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 2.

• 22

  “For our second 40K pilot in Katsina, Nigeria: we’ve seen different estimates of the U5 rate, but were working with 110-135 per 1000 births. C-WINS identified a geography with ‘114,473 children 6-59 months eligible for Vit A and 101,754 eligible for deworming’”. Dimagi, Responses to GiveWell’s Questions, August 17, 2023 (unpublished).

• 23

  “The current contract stipulates that partners must complete at least 1000 child visits within 120 days of contract signing. Our current onboarding process (detailed in Appendix D) involves a detailed iteration on the campaign and microplan for implementation. The onboarding phase usually lasts about 2 months, and we have streamlined multiple processes, and continue to do so for reducing lead time between contract signing and initiating the implementation in the field” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 7.

• 24

  “An important area of focus throughout this project will be on further developing and assessing our methods of verification. We are currently analyzing the data from the pilots with C-WINS and Sanmat which will inform our improved strategies going forward. An example of our analysis is the age distribution of the 40,143 visits recorded by C-WINS … We suspect that some 5 year olds were being classified as 4 year olds due to the families and the FLWs wanting to give the services to children who just missed the age cutoff. This data is overall encouraging to us, in that it fits with what we would expect from actual service delivery, though also warrants a deeper investigation.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 24.

• 25

  Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 24-25.

• 26

  “We anticipate being able to identify large-scale fraud or systematic errors by LLOs would lead to cancellation of the pay-for-performance contract if necessary. One challenge will be that we do not anticipate getting any large-scale fraud from the LLOs we work with, so we will seek out alternative methods to test these methods.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 25.

• 27

  “During this phase, Dimagi will oversee the execution of CCC-CHC by the LLOs. We expect this phase to resemble our efforts in Q4 2023 and Q1 2024 when we were overseeing two LLOs to each run 40K deployments. Scaling to more and larger LLOs will help test and solidify our processes and SOPs.
  During this phase, Dimagi will carefully track and gather data necessary to reduce GiveWell’s uncertainty around cost, adverse outcomes, etc. During this phase we will also focus on our verification activities as described below.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 24.

• 28

  “An important area of focus throughout this project will be on further developing and assessing our methods of verification. … An example of our analysis is the age distribution of the 40,143 visits recorded by C-WINS.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 24.

• 29

  “The following is an initial list of SOPs that Dimagi will codify and test during the proposed Operational Pilot. … Guidelines for training Frontline Workers (FLWs) on the use of CCC apps, including modules on decision support, screening, dosing, and data capture. This SOP should also cover certification processes to ensure competency.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 26.

• 30

  See the list “Proposed Activities for Operational Pilot,” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 23.

• 31
  • “We have used a few variations of payment terms.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 25.
  • “We adopt a standard monthly payment invoicing process during the deployment phase which is oriented to the LLOs as well.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 29.

• 32

  “The following is an initial list of SOPs that Dimagi will codify and test during the proposed Operational Pilot. … Financial Management and Accountability: Financial procedures related to the pay-for-performance model, including detailed guidelines for invoicing by LLOs, payment processing, and audits to ensure accountability and transparency.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 26.

• 33

  See GiveWell, “Dimagi CommCare Connect BOTEC”.

• 34
  • We do not expect to learn much about the implications for the use of government services, but we’re not certain. Our understanding that Dimagi will quantify the total costs associated with conducting household distribution is based on the fact that we know Dimagi tracks both total cost, and cost per child visit, in its pilots.
  • “The LLOs have agreed to 1,000 Naira per visit, excluding the cost of the commodities. The total cost of the visit will depend on how much ORS we deliver per visit and currency exchange rates. A good working number is $2.10 per visit, although the currency exchange could make this even lower. Note: we also pay $15,000 USD to each LLO for setup.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 22.
  • “At a high level, our implementation plan has the following elements: … Assess and gather data from these deployments in order to reduce GiveWell’s uncertainty around cost.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 23.

• 35

  Dimagi, “GiveWell Budget May 2024” (unpublished).

• 36

  The total of the listed sub-figures is slightly different, due to rounding of sub-dollar amounts.

• 37

  Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 23-24.

• 38

  See GiveWell, “Dimagi CommCare Connect BOTEC” (‘CommCare Connect BOTEC’ tab). Note that this cost-effectiveness estimate does not incorporate ORS and zinc or vaccine promotion, which we expect would increase cost-effectiveness, so this 10x estimate should be considered a lower bound. Furthermore, note that a) our cost-effectiveness analyses are simplified models that are highly uncertain, and b) our cost-effectiveness threshold for directing funding to particular programs changes periodically. As of August 2024, our bar for directing funding is programs that are about 10 times as cost-effective as unconditional cash transfers. See GiveWell’s Cost-Effectiveness Analyses webpage for more information about how we use cost-effectiveness estimates in our grantmaking.

• 39

  “We will design an RCT with our research partner, in order to be ready for phase 3. … Phase 3, Evaluation (starts in 2025): If Phase 2 results are encouraging, a natural next step would be a rigorous evaluation of the CCC-CHC implementation approach that was solidified in Phase 2. We expect the research would focus on the question of whether CCC-CHC increases counterfactual coverage.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 2.

• 40

  Dimagi, email to GiveWell, March 22, 2024 (unpublished).

• 41

  Dimagi, Responses to GiveWell’s Questions, August 17, 2023 (unpublished).

• 42

  Dimagi, Responses to GiveWell’s Questions, August 22, 2023 (unpublished).

• 43
  • “In September 2023, GiveWell recommended a $6.6 million grant to CHAI for a 2-year diarrhea treatment program in Bauchi, Nigeria. The grant supports the distribution of oral rehydration solution (ORS) and zinc to households with children under the age of 5 and a randomized controlled trial (RCT) to measure the impact of this program on ORS and zinc usage.” GiveWell, Clinton Health Access Initiative — Oral Rehydration Solution and Zinc Distribution in Bauchi, Nigeria, 2023.
  • “We have been discussing with New Incentives (one of GiveWell's top charities) the idea of distributing ORS/zinc to caregivers when they bring their children to health clinics for routine vaccinations, and comparing ORS/zinc usage rates before and after rollout of the program.” GiveWell, Wageningen University — Add-ons to Sierra Leone RCT on Health Service Delivery to Remote Communities, 2024

• 44
  • “In October 2023, GiveWell recommended a $1.4m grant to Malaria Consortium to deliver vitamin A supplementation (VAS) alongside seasonal malaria chemoprevention (SMC) campaigns in 2 states in Nigeria.” GiveWell, Malaria Consortium — Co-Delivery of Vitamin A Supplementation and Seasonal Malaria Chemoprevention in Nigeria, 2023.
  • “This grant will fund Malaria Consortium to deliver VAS once per year in two states in Nigeria (Bauchi and Niger states) in 2024 (in both states) and 2025 (in Niger state only).” GiveWell, Malaria Consortium — Co-Delivery of Vitamin A Supplementation and Seasonal Malaria Chemoprevention in Nigeria, 2023.

• 45

  “Phase 3, Evaluation (starts in 2025): If Phase 2 results are encouraging, a natural next step would be a rigorous evaluation of the CCC-CHC implementation approach that was solidified in Phase 2. We expect the research would focus on the question of whether CCC-CHC increases counterfactual coverage.” Dimagi, Dimagi’s scoping work and proposed operational pilot in Nigeria, p. 2.

• 46

  This general understanding is based on conversations we’ve had with experts as well as our general impressions of the literature. For example, see Hanson, Kara et al., The Lancet Global Health Commission on financing primary health care: putting people at the centre, 2022, Fig. 8., which shows that approximately half of FLWs (also referred to as community health workers) in low-income countries are paid per day or activity, paid in-kind, or not paid. See also The New York Times, “Only God Can Thank You: Female Health Workers Fight to Be Paid,” 2023, which states “Community health workers put in long hours to protect people in developing countries from diseases such as malaria, Covid-19 and H.I.V. But most are compensated minimally, or not paid at all."

• 47

  “This is an important question for CommCare Connect broadly, and one we plan to study over the coming years. In all of our current pilots (launched and non-launched) we believe there is minimal displacement of FHW duties, though do want to confirm that empirically … In general, we see it as a positive secondary effect that CCC-CHC will create new, well paid, low-skilled, purposeful jobs in areas of high unemployment.” Dimagi, Responses to GiveWell’s questions, August 17, 2023 (unpublished).