Five dimensions of impact | How much
Introduction
The “How Much” dimension covers the extent of the impact — across scale, depth, and duration.
Knowing the number of people reached (i.e., scale) is only one part of the equation. To fully understand the extent of the impact experienced by people and the planet, enterprises and investors need to consider all three data categories under the “How Much” impact dimension:
The three data categories within “How Much” provide complementary insights. Consider a solar energy company with operations in three East African countries. Reporting that 500,000 solar home systems have been distributed, reaching 450,000 households, gives some insight into the magnitude of the outcome; however, it does not reveal whether the solar home systems improved customers’ quality of life, or for how long those improvements lasted.
Conversely, knowing that customers’ productivity increased by 20% on average provides meaningful information about the degree of change that potential customers may experience. But without scale and duration data to back the depth results, the enterprise is missing two important points for understanding and improving its impact performance.
Enterprises and investors can also segment data in the “How Much” categories by behavioral characteristics or social identities and demographics to develop a more nuanced understanding of scale, depth, and duration and create tailored interventions. For example, in the case of the solar energy company above, different members of 450,000 households reached, such as men, women, and children, may have experienced markedly different outcomes, and outcomes may have differed for rural vs. urban households.
In an ideal scenario, enterprises would generate important positive outcomes that affect many people, bring deep change, and last for a long time. But none of these is necessarily more important than the other. Some enterprises and investors will set goals for a deep, enduring impact for relatively fewer people, over a more marginal impact for many – or vice versa. Others will see scale, depth, and duration as equally important.
In the sections that follow, we explore the three “How Much” categories, focusing on how each plays a role in understanding the significance of an outcome and improving impact performance.
What is scale?
The scale category captures the number of people — whether that is employees, customers or distributors — who experience the outcome. All else equal, an enterprise that affects 100,000 lives would be producing a more “significant” outcome than one that reaches 50,000 lives.
Additionally, data about scale can be used for strategic purposes including:
- (Re)setting scale targets: Comparing scale performance data against specific targets and the addressable population can yield useful information for (re)setting scale targets (either in total or by market segment).
- Understanding the value proposition: Comparing scale performance year-on-year can provide insights into whether customers (or other affected parties) are satisfied with the enterprise’s products (or policies). This is the same as calculating customer growth and churn rate.
The table below provides example scale indicators:
Segmenting data by gender to increase scale of women customers
Musika, a Zambian non-profit company that works to stimulate private sector investment in smallholder agriculture markets, partnered with Value for Women to offer gender lens technical assistance to smallholder agribusinesses serving farmers. The participating agribusinesses wanted to serve more women farmers and improve outcomes for women.
For each of the participating agribusinesses, collecting and analyzing gender-disaggregated data was the first step to understand how many women farmers they were serving. Though women are estimated to make up approximately 53% of the agricultural workforce in Zambia, the companies found that women farmers made up a far smaller portion of their customer base. Through segmenting customer data, they not only learned that the number of women farmers accessing their products and services was far below their target, but they also were able to explore the barriers to access that women farmers faced and implement changes in response. For example, after women and men reported farming with different types of livestock, one business expanded their offering of veterinary services and training to cover the livestock commonly owned by women, increasing women farmers’ enrollment in their programs.
Source: Value for Women (2021)
What is depth?
The depth of an outcome captures the degree of social or environmental change experienced by people and communities. As opposed to scale, available as raw data, depth is derived from comparing the level of outcome that people are currently experiencing against the baseline. Depth can be a 10-point improvement in test scores, a 30% increase in salary, or a 5,000-tonne reduction of CO2 emissions. The diagram below illustrates the concept behind the depth of an outcome.
Since depth is the difference between the outcome in the current period and the baseline, it can be calculated as relative change or absolute change. No one type of analysis is better the other. Both should be assessed as they provide complementary data points (see table below for a simple illustrative example). For guidance on selecting indicators for depth, visit the outcome and baseline indicator.
Along with scale and duration, the depth category is an essential data point for understanding the significance of an outcome, providing insight into the extent to which outcomes improved or deteriorated. Segmenting data on depth by behavioral characteristics or social identities can provide further insight on whether outcomes are improving for everyone, or only for specific subgroups.
Enterprises and investors can also leverage the depth category in combination with counterfactual data (i.e., what would have happened anyway) to estimate an enterprise’s contribution to the depth of an outcome. We explore this relationship in the “Contribution” impact dimension.
What is duration?
Duration refers to the time period for which people and communities experience the outcome.
The duration of an outcome is intuitively important: just as people experiencing positive outcomes want them to be long-lasting, they want negative outcomes to be short-lived. Society’s collective investment in education illustrates this point. Parents invest significant resources in educating their children under the assumption that the payoff will be positive and last for many years— a belief backed by World Bank research which shows that every additional year of schooling returns a 9% increase in hourly wages.
Duration is also relevant to assessments of the materiality and significance of enterprise impacts, as impacts that are not material in the short-term may become so in the long-term. For example, an enterprise may create jobs in a community in the immediate term and end up contributing to gentrification over time. Information about duration prompts enterprises and investors to think about sustainability, reflecting on how their activities can affect people and communities in the short-, medium-, and long-term.
Four facts shape the calculation of the duration of an outcome. First, outcomes have different durations. Some may last only a few months, whereas others may persist for years (as with the education example above). Second, outcomes may materialize immediately, or in the medium- and long-term. Third, many outcomes go beyond the end of the intervention and / or the measurement period. This means that those affected may still experience the outcome (and potentially experience new, unintended outcomes) even after the enterprise stops its activities and / or stops collecting outcome data. Fourth, communities may have different views about the time horizon that should be considered when making decisions. For example, publicly traded companies are incentivized to manage towards quarterly earnings, and investors in those companies may make decisions based on even shorter time horizons. In contrast, many First Nations, Inuit, and Métis communities in Canada consider effects on people seven generations into the future when making major decisions.
The diagram below illustrates the concept behind the duration of an outcome.
Duration can be estimated in several ways, with varying degrees of rigor. At one end of the spectrum, enterprises can survey affected people and communities on a recurring basis, from the start until the end of the intervention (or for even longer periods). While this method usually yields accurate duration estimates, it can be cost-prohibitive.
On the other end of the spectrum, enterprises can use existing research (e.g., impact evaluations) or market research (e.g., government or think tank reports) to estimate how long a particular outcome may last. When using either of these methods, enterprises should assess the validity of the estimate by considering how well the research study reflects their own intervention (e.g., Was the study conducted in the same country or in a completely different setting? Did the evaluated intervention have a similar set of inputs and focus on a similar population?) and by engaging with affected people and communities.
One way that investors and enterprises can account for negative or unintended outcomes that materialize or extend beyond the time period for which they actively collect impact data is by setting up a complaints registry or grievance redress mechanism (GRM) that provides a channel for affected people and communities to communicate any harm they experience. For example, private equity firm Aqua Capital set up an Integrity Channel for its investments in agriculture and food companies, providing a mechanism for anyone affected by these investments to report a grievance.
IN THIS LESSON
ADDITIONAL Resources
- Module 7: How Much
- How do we know if we’re contributing to the SDGs?
- How people describe the impact they experienced – IMP report
- Impact Management Platform: Measure Sustainability Performance
