Five dimensions of impact | Contribution Did the enterprise’s activities contribute to the outcome achieved? What would have likely been the outcome in the absence of the enterprise? The impact data categories under the ‘Contribution’ dimension help us answer these questions.
Introduction
To understand their own contribution to a social or environmental outcome, enterprises need to consider what likely would have happened in the absence of their activities. The data categories in the ‘Contribution’ dimension help enterprises and investors assess an enterprise’s contribution to the social (environmental) outcomes that people (planet) experience, relative to what likely would have occurred anyway (i.e., the outcome counterfactual).
Enterprises and investors operate in a dynamic social system, with various actors – from competing firms to government bodies to NGOs – seeking to contribute to the same set of outcomes.
Consider a solar energy company aiming to improve the health of Kibera slum residents by reducing kerosene use. This enterprise operates in an environment where other enterprises, government policies and NGOs are all striving to achieve the same outcome but through different mechanisms. Accordingly, if customers’ health improved by 20%, the solar energy company would need to consider how these other initiatives contributed to this percentage change, to understand its own contribution to the outcome.
The insights of the ‘Contribution’ dimension are essential.
If an enterprise finds that its contribution to a particular social or environmental challenge is minimal, then it may decide to (re)allocate resources elsewhere. Conversely, if the contribution is significant, then the enterprise may decide to increase its commitment.
More broadly, by considering their own contribution, enterprises deepen their awareness of the system they operate in – who the key actors are, how they interact, and what the levers for change may be. Equipped with this understanding, enterprises can work towards optimizing the entire system rather than a single intervention. This may mean developing a partnership with an NGO to reach last-mile consumers. It could also mean collaborating with government to upskill and increase the resilience of smallholder farmers.
In impact terminology, ‘Contribution’ overlaps with terms such as ‘additionality’, ‘deadweight’ and ‘attribution’. While they all seek to answer ‘what would have happened anyway?’ to derive an understanding of an enterprise’s contribution to an outcome, these terms often differ in methods and scope.
‘Contribution’ should not be mistaken with depth under the ‘How Much’ dimension, which covers the significance of the outcome by calculating the difference between the outcome in period and the baseline, without considering the influence of other factors (e.g., other organizations, economic conditions).
An enterprise’s depth contribution
=
outcome in period – outcome that would have been observed anyway
We have deliberately chosen the word ‘Contribution’ to cover all of the ways in which the counterfactual can be assessed — from market and evidence-based research to randomized control trials.
Case Study: Using market research and stakeholder feedback to understand an enterprise’s contribution
By 2011, One Acre Fund (OAF) had reached 78,000 smallholder farmers in Kenya and Rwanda — an impressive 145% year-over-year growth. The rapid scale and impact per farmer ($120 avg. increase in yearly income) convinced the East-African based organization to explore the Ghanaian market, with the view that it could serve as a potential launchpad for other West African countries.
After a few months of scouting, OAF launched a pilot program with 500 farmers, offering them a loan package with a half-acre of maize seed and fertilizer.
To understand the impact of the program and acquire lessons for scaling up in West Africa, OAF implemented a threefold approach:
- Collection of outcome data (i.e., annual income) before the program and at particular milestones
- Research on system dynamics
- Interviews with farmers and OAF field officers
One Acre Fund found that its existing intervention was not well-suited to the Ghanaian market, making its contribution to outcomes minimal.
First, farmer reliance on agriculture was low, with farmers also generating an income from other sources.
Second, the maize-based program was implemented in a region where there was a stronger focus on cash crops. While OAF quickly pivoted to another region with ideal farming conditions, the organization realized that the addressable population was too small to make the business model work.
Third, upon moving to the new area, a lack of rainfall plagued the farming season. OAF soon discovered that these droughts were not an anomaly, but a regional shift towards semi-arid growing conditions.
Based on these findings, OAF decided to shut down its Ghanaian operation, but not without gaining important lessons through the process, including the need to: (1) professionalize country scouting units; (2) constantly adjust and innovate during pilots and scale-ups; and (3) locate operations far from large urban cities when commercial agriculture and non-agricultural activities dominate.
Source: One Acre Fund (2014)
How can enterprises evaluate their contribution?
To estimate the outcome counterfactual, enterprises can use several approaches that vary in rigor and costs. Randomized control trials and quasi-experimental methods typically require significant resources but produce stronger evidence of contribution compared to market research and stakeholder feedback. However, carefully-designed market research and stakeholder feedback can yield valuable insights for (1) understanding what else may be driving the outcome, (2) building a ‘good enough’ counterfactual scenario, and (3) conducting the depth analysis. Moreover, these methods can often be combined to gain complementary findings.
- Stakeholder feedback: Stakeholder feedback requires consulting the individuals (or communities) affected by the enterprise’s activities to gain a nuanced understanding of the drivers behind the outcome (e.g., the enterprise’s activities, external factors, government interventions, cultural practices). If deployed well — covering a large and well-chosen sample and different points of view — stakeholder feedback can imply a likely counterfactual. This method should be combined with market research and/or evidence-based research, as they are mutually reinforcing.
- Market research: By taking a thorough look at an intervention’s context, market research can be used to build a ‘good enough’ counterfactual. This method requires a deep analysis of secondary resources (e.g., industry reports) to identify what else may be driving the outcome — from other organizations, to government interventions, to external factors (e.g., weather, economic conditions), to individuals’ unobservable characteristics (e.g., motivation, cultural practices). Market research should be paired with stakeholder feedback and/or experimental or quasi-experimental research to produce complementary insights, and to strengthen the credibility of the counterfactual estimate.
- Experimental and Quasi-experimental research: Enterprises can source depth contribution estimates through evidence-based (i.e., rigorous impact studies, conducted by independent researchers, of enterprises’ products, services, and other interventions). Because randomized control trials and quasi-experimental methods are specifically designed to assess counterfactuals, they provide particularly strong evidence of contribution.
Randomized control trials (RCTs) are the most common form of experimental research. RCTs measure the difference in outcomes over time among two randomly assigned groups:
- A treatment group (i.e., receives the intervention such as a product)
- A control group (i.e., one that did not receive the intervention, or received a placebo or another type of intervention).
The randomization ensures that the two groups are similar on observable (income, gender, health) and unobservable (self-motivation, energy) characteristics, creating a robust counterfactual. Although a popular method in international development, RCTs and quasi-experimental methods usually require significant resources.
When randomization is not cost-effective, ethical, or practical, quasi-experimental methods (e.g. regression discontinuity design, difference-in-difference) provide a range of statistical techniques to build experimental groups. Once these groups are created, practitioners compare the difference in outcomes over time between individuals who received the intervention and those who did not (the counterfactual). In contrast to RCTs, quasi-experimental methods require many more assumptions to develop a credible counterfactual.
Practitioners often find research done elsewhere and extrapolate the results to their own situation. When doing so, they should assess the generalizability of the original research by considering the study’s methodological rigor, population group, country setting, and type of intervention. For example, if an Indian enterprise relied on an estimate from an impact study that took place in Argentina – a country with significantly different socio-economic characteristics – then the generalizability of this estimate is likely to be low, rendering it unusable. Enterprises should exercise caution in extrapolating research results to other settings, and complement such extrapolation with stakeholder feedback and market research. Sources of evidence-based research include J-Pal’s evaluations, Innovations for Poverty Action’s research and 3ie’s systematic reviews.
ADDITIONAL Resources
- How do we know if we’re contributing to the SDGs?
- How people describe the impact they experienced – IMP report
- Impact Management Platform: Measure Sustainability Performance
- Impact Evaluation & Impact Management with Jane Reisman, Veronica Olazabal, and John Sherman
- Innovations for Poverty Action: The Mis-measure of Impact
