Five dimensions of impact | Contribution Did the enterprise’s activities contribute to the outcome achieved? What would have likely been the outcome in the absence of the enterprise? The impact data categories under the ‘Contribution’ dimension help us answer these questions.

To estimate the outcome counterfactual, enterprises can use several approaches that vary in rigor and costs. Randomized control trials and quasi-experimental methods typically require significant resources but produce stronger evidence of contribution compared to market research and stakeholder feedback. However, carefully-designed market research and stakeholder feedback can yield valuable insights for (1) understanding what else may be driving the outcome, (2) building a ‘good enough’ counterfactual scenario, and (3) conducting the depth analysis. Moreover, these methods can often be combined to gain complementary findings.

• Stakeholder feedback: Stakeholder feedback requires consulting the individuals (or communities) affected by the enterprise’s activities to gain a nuanced understanding of the drivers behind the outcome (e.g., the enterprise’s activities, external factors, government interventions, cultural practices). If deployed well — covering a large and well-chosen sample and different points of view — stakeholder feedback can imply a likely counterfactual. This method should be combined with market research and/or evidence-based research, as they are mutually reinforcing.
• Market research: By taking a thorough look at an intervention’s context, market research can be used to build a ‘good enough’ counterfactual. This method requires a deep analysis of secondary resources (e.g., industry reports) to identify what else may be driving the outcome — from other organizations, to government interventions, to external factors (e.g., weather, economic conditions), to individuals’ unobservable characteristics (e.g., motivation, cultural practices). Market research should be paired with stakeholder feedback and/or experimental or quasi-experimental research to produce complementary insights, and to strengthen the credibility of the counterfactual estimate.
• Experimental and Quasi-experimental research: Enterprises can source depth contribution estimates through evidence-based (i.e., rigorous impact studies, conducted by independent researchers, of enterprises’ products, services, and other interventions). Because randomized control trials and quasi-experimental methods are specifically designed to assess counterfactuals, they provide particularly strong evidence of contribution.

Randomized control trials (RCTs) are the most common form of experimental research. RCTs measure the difference in outcomes over time among two randomly assigned groups:

• A treatment group (i.e., receives the intervention such as a product)
• A control group (i.e., one that did not receive the intervention, or received a placebo or another type of intervention).

The randomization ensures that the two groups are similar on observable (income, gender, health) and unobservable (self-motivation, energy) characteristics, creating a robust counterfactual. Although a popular method in international development, RCTs and quasi-experimental methods usually require significant resources.

When randomization is not cost-effective, ethical, or practical, quasi-experimental methods (e.g. regression discontinuity design, difference-in-difference) provide a range of statistical techniques to build experimental groups. Once these groups are created, practitioners compare the difference in outcomes over time between individuals who received the intervention and those who did not (the counterfactual). In contrast to RCTs, quasi-experimental methods require many more assumptions to develop a credible counterfactual.

Practitioners often find research done elsewhere and extrapolate the results to their own situation. When doing so, they should assess the generalizability of the original research by considering the study’s methodological rigor, population group, country setting, and type of intervention. For example, if an Indian enterprise relied on an estimate from an impact study that took place in Argentina – a country with significantly different socio-economic characteristics – then the generalizability of this estimate is likely to be low, rendering it unusable. Enterprises should exercise caution in extrapolating research results to other settings, and complement such extrapolation with stakeholder feedback and market research. Sources of evidence-based research include J-Pal’s evaluations, Innovations for Poverty Action’s research and 3ie’s systematic reviews.