Case StudyPortfolio-Level Impact Scoring: A Case Study of British International Investment
This case study was authored by Paddy Carter, Director, Development Impact, Research and Policy at British International Investment, and edited by Mike McCreless, Executive Director at Impact Frontiers.
Introduction
British International Investment (BII) is the UK’s development finance institution, with a history that goes back to 1948.1 BII has entered 2022 with a new name, a new strategy, and a new impact scoring tool.2 This case study describes and discusses our new impact scoring tool in the context of our 2022-2026 strategy and overall approach to impact management. We are sharing our methodology in the hope that other impact investors may find useful elements, and out of a desire to increase transparency around how impact investors manage impact.
An impact scoring tool must distinguish between higher- and lower- impact investments as accurately as possible, to serve its purpose of helping an investor to manage and maximise impact. But what “as possible” means requires careful thought.
We were particularly concerned with building a scoring system that our investment professionals will value — and that will not be rejected by the organization as unworkable. Other constraints on what’s possible include the need for flexibility to work with the varying and limited levels of information available prior to investment across many different types of transactions. Lastly, the scoring system needed to be objective enough to enable external assurance, so that it can be used in determining staff renumeration.
1 – BII’s sole shareholder is the Foreign, Commonwealth and Development Office, a UK government department.
2 – Until April 2022, BII was called CDC Group. It was founded as the Colonial Development Corporation in 1948, renamed the Commonwealth Development Corporation in 1963, and renamed CDC Group in 1999.
Strategic Impact Objectives
BII’s 2022-2026 strategy sets three strategic development impact objectives that respond to the opportunities and challenges we see in the countries that we serve. Our view is that to achieve the UN’s 2030 Sustainable Development Goals (SDGs) and meet commitments under the Paris Agreement, investment must deliver three things:
- Productive development – by raising the productivity of an economy so that it can support a decent standard of living for all;
- Sustainable development – helping transform the economy to reduce emissions, protect the environment and adapt to the changing climate; and
- Inclusive development – sharing the benefits of higher productivity and greater sustainability with poor and marginalized sections of society.
Our choice of these objectives, and our understanding of what it means for an investment to have a higher or lower impact when assessed against them, is informed by theory and evidence about the function of private investment in development, the role of publicly-owned development finance institutions in private markets, and hence where the largest social returns to investment can be found.3
Our impact management practices are designed to maximize our performance against these objectives throughout the investment lifecycle. This starts with how we approach sectors and regions, then in how we originate investment opportunities and take investment decisions, through to structuring investments and managing the portfolio, and finally to responsible exits and impact evaluations
3 – This thinking is described in a strategy background paper “The economics of development finance” available here: https://www.bii.co.uk/en/ news-insight/insight/articles/the-economics-of-development-finance
Managing impact across the investment cycle
Our approach to impact management is grounded in the Operating Principles for Impact Management (OPIM) (see Figure 1), of which we are a founding signatory. OPIM describes the processes that the best impact investors should have in place, but does not specify how those processes should be implemented.4
Figure 1: Operating Principles of Impact Management
4 – Our 2021 OPIM disclosure statement, which describes how we implemented these principles, is available here: https://assets.bii.co.uk/wp-content/uploads/2021/07/13115447/CDC-Operating-Principles-for-Impact-Management-2021.pdf
BII’s Impact Framework
At the heart of our approach sits an overarching Impact Framework, which draws on the five dimensions of impact from the consensus facilitated by the Impact Management Project and now hosted at Impact Frontiers (see Figure 2).5
This framework shapes how we assess the expected impact of each individual investment; how we manage, monitor, and evaluate impact after investment; and what we look for in responsible exits.
Figure 2: BII’s Impact Framework
5 – The IMP Framework: https://impactfrontiers.org/norms/
Investment-Level Impact Dashboard
We include a one-page summary sheet of impact, using this framework, in investment papers, which we call an Impact Dashboard. These Impact Dashboards remain the primary tool we use to make individual investment decisions, providing investment committee members with the most important information about anticipated impact in a consistent format.
Here is an example of an Impact Dashboard for a fictional impact-oriented venture capital fund:
Figure 3: Example Impact Dashboard
The first four dimensions of the Dashboard assess the expected ‘impact of the investment’ (and the sixth, the risks to it) which encompasses the entirety of the impact associated with the transaction we would be participating in, without trying to strip out only those elements we might attribute to ourselves. Once we have invested, we share responsibility for everything that happens subsequently, we would use our influence where possible to improve the total impact of the investment, and we would monitor impact by looking this bigger picture.6
6 – The “impact of the investment” differs from “enterprise impact” because, when we are investing in an existing enterprise and not a greenfield, we look at the change the investment will make to the impact of the enterprise, and not the overall impact including legacy operations. It is not always possible to cleanly distinguish the two.
Approach to Investor Contribution
The fifth dimension in this framework is our contribution as an investor. Because we are a publicly owned DFI, funded from Official Development Assistance spending, this is a crucial question. We exist to do something over and above what private investors are doing (a requirement often referred to by DFIs as ‘additionality’). We use a distinct investor contribution assessment framework to generate a low/medium/high rating to capture our view on the scale of the difference our involvement will make to the impact of the investment, and our degree of confidence that we are doing something private investors would not.7
The contribution rating is considered alongside our assessment of the anticipated impact of the investment in decision-making. For the sake of illustration, if we think of our contribution as a number between zero and one – where zero implies we contribute nothing that other investors would not, and one that the entirety of the impact of the investment would not occur without our participation, then our impact as an investor can be seen as the product of the two: the impact of the investment multiplied by our contribution. Hence, we may have a significant impact as an investor by making a low (but positive) contribution towards an investment with large impact, or conversely, by making a large investor contribution towards an enterprise or project with smaller impact. Either way, we want our impact as an investor to be commensurate with the resources being deployed.
One of the most important design decisions we took was to not incorporate contribution into the impact score. We felt it would be unhelpful to collapse the two into a contribution-weighted score, and that we would take better decisions by viewing the two separately. We also felt that is vital we can have an honest debate about our contribution and using it to weight the score would put too much pressure on what is a subjective judgment call. We describe below why we want to minimize subjectivity in the scoring. We can, of course, view the two together when we analyze our portfolio.
This approach to rating investor contribution could be adapted by private impact investors, for whom competing against each other to win investment opportunities is part of the job and where the question “would this investment have happened without us?” can be less relevant (depending on investment mandate). For example, different segments of the market could be rated for relative capital scarcity. An investor’s confidence that their actions will increase the overall quantity of investment in that segment of the market would be higher where capital is scarce or expensive.
7 – Our approach to rating contribution and its use in decision-making process is described here: https://assets.bii.co.uk/wp-content uploads/2022/05/19141040/Our-approach-to-investor-contribution.pdf
BII’s New Portfolio-Level Impact Score
BII developed an Impact Score as a means of ‘managing strategic impact on a portfolio basis’, which is the second principle of the OPIM. The Impact Score is designed to recognise and incentivise investments that are likely to contribute most to our three strategic impact objectives. It complements the more detailed assessments of the expected impact of each investment in our Impact Dashboards by providing a quantitative metric that can be aggregated and used to monitor and analyse impact performance across the portfolio. It is calculated using a subset of the information found in the Impact Dashboards and monitoring plans.
The Impact Score replaces our old Development Impact Grid, which we had used since 2012, as one of the key performance indicators that BII reported to the U.K. government.
The design of the Impact Score favours objectivity and simplicity over comprehensiveness and nuance. Qualitative aspects of the expected development impact of investments, are deliberately excluded from the Impact Score but remain central to investment decisions. These include the three elements of our impact framework ‘depth’, ‘risk’ and ‘contribution,’ because interpreting their relative importance across different investment contexts requires subjective judgement.
One of the challenges that has emerged during the first months of implementation has been achieving a shared understanding across the organization about how scoring works alongside our other impact management tools. The score must be trusted to do its job of shaping our portfolio and of measuring performance against strategic impact objectives, but it also deliberately excludes many decision-relevant considerations. There is balancing act of giving the Score enough weight in investment decisions, but not too much.
Design Features
Portfolio-level: Scores of individual investments are aggregated at the portfolio level into an ‘Aggregate Impact Score’ (a weighted average of individual impact scores for all investments committed to from 2022 onwards). There is no explicit target (higher is better) but we expect this aggregate score to range between four and eight during the 2022-26 period. We are accountable to our Board and FCDO for performance against the Aggregate Impact Score, and we will publicly report on it in our Annual Review. External scrutiny is an important part of the model.
The Impact Score is a portfolio performance measure and there is no minimum threshold score requirement for individual investments. This reflects the fact that it only provides a condensed picture of impact, so our complete view on an investment’s impact may sometimes differ from the ranking implied by its score. It provides us flexibility to select some investments with particularly strong expected financial performance in order to construct a portfolio that best achieves our overall impact and financial goals.
Predictable: The Impact Score is designed to be intuitive and easy to calculate from the early stages of investment origination, so that our investment teams can use it to prioritise their activities. The Impact Score would not effectively influence investment behaviour if it only emerged from a ‘black box’ at the final Investment Committee stage.
Pragmatic: The Impact Score is designed to adapt to the level of information we have available, which varies across products, sectors, and the stage of the investment process. Various ‘default’ assumptions can be used to calculate the Score when more granular information is not available.
Usage and interpretation
Every investment we make will have an associated Impact Score which is based on the expected development impact of the investment (ex-ante). The score for each investment will also be updated over the lifetime of the investment at regular intervals, based on the actual impact performance (ex-post). Because elements of our new score are based on predicted impact metrics, such as jobs created, updating is important to ward against optimism bias. We will update scores on a regular schedule, but the underlying timing of performance targets will vary according to each investment, according to our monitoring plans. A power investment will not start producing power until after the construction is completed, for example. Not every element of the score will change over time. For example, if the country of operation remains as expected and the Inclusive score is based on the default country method (see below) that element will not change.
The Aggregate Impact Score partially determines staff remuneration under our Long-Term Incentive Performance Plan – another reason that the score is designed as a simple and objective measure. We regard the construction of a high impact portfolio as a joint endeavour across the organization, so performance related payments are calculated on a company-wide basis, not on individual or team Scores. Although the Score will inform how we allocate teams to sectors and regions, some teams will inevitably find it easier to obtain higher scores than others. Theory and evidence also suggest that high powered individual financial incentives perform poorly when outcomes are the result of efforts by mission-oriented teams and performance is measured with error.8
The score is not an attempt to measure impact, in the way that monetizing impact (i.e., estimating a monetary value of impact created) would be. We would not be able to communicate how much impact our investments have had by telling people they scored an average of 6. The score is more like an ordinal scale – a ranking. It helps our shareholder monitor performance against our objectives (higher scores are better). But because the score is constructed from various sub-components and aggregated to compute a portfolio average score, it does embed relative judgements about when impact along one dimension is regarded as equivalent to impact along another dimension. All else equal, a score of 8 does count twice as much towards the portfolio average as a score of 4, so the system functions as a cardinal measure of impact in that respect. Through extensive testing on historical and pipeline transactions, and consultation across the organization and with the U.K. government, we arrived at relative scores that we felt did justice to our institutional view of the importance of different degrees of alignment with our three objectives.
8 – Ashraf, N., & Bandiera, O. (2018). Social incentives in organizations. Annual Review of Economics, 10, 439-463.
Calculating the Score
Every eligible investment will receive Productive, Sustainable, and Inclusive scores. The sum of the three scores will result in a Total Impact Score, which can range from -1 to 10. Figure 3 (above) summarises the elements of the score for each impact objective.
- Productive score: reflects how efficiently an investment addresses one of a set of developmental needs (see appendix A) and the extent to which the investment is expected to have positive spillovers onto the productivity of other firms. It ranges from 0 to 4.
- Sustainable score: reflects to what extent the investment will contribute to the net zero transition and to climate adaptation and resilience. The score depends on whether the investment qualifies as climate finance. If qualifying, an investment is scored according to its climate mitigation, adaptation and resilience attributes. If not, it is scored simply on greenhouse gas (GHG) emissions. It ranges from -1 to 4.
- Inclusive score: captures who is directly benefitting from the investment, using either known characteristics of workers and customers (initial income, gender and ethnicity), or a default country score. It ranges from 0 to 4.
Figure 4: Elements of BII’s Impact Score
Productive Score
The Productive Score is constructed from four elements. The first two elements are combined to generate a base score of 0 to 4; the second two award additional points to a maximum of 4.
- The relative degree of development need which the investment will address. The primary need selected for scoring purposes must be central to and consistent with the impact thesis presented in the Impact Dashboard. Eligible needs are linked to the Sustainable Development Goals and either improve people’s quality of life directly, such as through higher incomes, food security, healthcare, or directly by producing outputs that have proven large-scale positive economic spillovers, such as power, transport and logistics, or financial services for businesses. For each need, a score is assigned to countries based on a relative gap assessment: countries where the need is greater will have a higher score. Investments that operate in multiple countries are scored by a regional weighted average. See Annex A for the full list of needs and indicators.
- The intensity at which the investment delivers the impact. This captures how efficient the investment or company is in delivering impact compared to relevant benchmarks. Here efficiency is usually measured by the quantity of impact-relevant outputs, relative to the total quantity of capital required to produce them (i.e., outputs per dollar), although other performance indicators such as growth rates may be used in some cases. Benchmarks in different sectors are built from various data sources, including both data from our own historical investments and other information available for investments in our markets. Intensity is rated below, above or in line with benchmarks, and intensity will be considered immaterial where our investment does not result in additional impact. Default ‘in line with benchmark’ intensity scores will be used where no suitable benchmarks are available.
- Economic enablers. Investments will receive an additional point where they produce inputs that are required by many other firms; where there is evidence that reducing the price or increasing the quality of these inputs has a significant impact on the growth of firms; and where the impact case rests on these effects.
- Potential to catalyze markets. Additional scores will reward investments that have the potential to improve market structures and the behaviors of other market actors by significantly increasing competition, pioneering new business models that can lead to replication by others, reinforcing the demonstration of those business models, building skills and human capital, or improving the underlying enabling environment (such as the first public-private partnership in a country that involves the creation of new laws and regulations, for example). The impact thesis of such investments typically materializes only in the medium to long term – and relevant benchmarks may not yet exist – and we want to recognize the transformative potential.
Figure 5 below shows how first a ‘base’ score is calculated from degree of need and intensity, before economic enabler and catalyzing market points are added if warranted. The total Productive Score is capped at 4.
Figure 5: Productive Score Elements
Intensity is an area of the scoring system which we intend to build out over time, as we create better benchmarks and resolve issues around ensuring like-for-like comparisons between investments. At time of writing, we have a limited set of workable benchmarks, for job creation and power generation, with others being tested. A challenge here is to avoid omitting qualitative differences that would result in a misleading impression of which investments are generating impact more efficiently. For example, dispatchable power from renewable energy paired with battery storage is more expensive per MWh but qualitatively different to power that is only available when the sun shines or wind blows. Using “people reached” as an intensity indicator would also be misleading if greater scale is correlated with more shallow impact.
The needs score is agnostic about which needs are more important – for each need, the score is based on the degree of need in the country (or countries) of investment, relative to the set of countries that are eligible for Official Development Assistance. The economic enabler point is where we introduce a view, based on our review of the evidence from economics research, about which sectors are more important for raising productivity across an economy. The point is awarded using a two-part test: is the business producing something that can be expected to have spillovers on the productivity of many firms? and is that the primary impact thesis of the investment?
Electricity is an economic enabler, for example, but a grid scale renewables investment in India, a country where the supply of electricity is not really a constraint on growth, would not get an economic enabler point because the impact thesis there will be about decarbonising the grid, not about reducing the cost, increasing the quantity or reliability of electricity.
Scoring catalyzing markets calls for expert judgment, based on information about the market in question, supported by a catalyzing markets framework and guidance notes for specific markets. We ask ourselves: why is the market important for our development objectives; what is the preventing the market from functioning; how will our investment lead to changes in the behavior of other market actors; and how will those changes improve the market? Our confidence in a catalyzing markets thesis hinges on our understanding of the market failure and the likelihood our investment we address underlying causes. Pathways we consider especially powerful, such as investments that we expect to generate a significant response from competitors, are awarded two points. Others, such as proving out business models that have been pioneered elsewhere or creating necessary skills and capacities, receive one. Monitoring these investments usually requires the collection of market-level information, as opposed to the firm-level data which is more usual.
Sustainable Score
The Sustainable Score aims to incentivize investments that contribute the most towards the transition towards a net zero and climate-resilient economy. It measures the extent to which investments are creating impact by facilitating the transition to net zero through avoiding, reducing or sequestering GHG emissions (mitigation); through the introduction of circular economy business models; as well as strengthening the adaptive capacity and building the climate resilience of people, business, physical and natural assets, and economies to acute and chronic physical climate risks.
Figure 6: Sustainable Score Elements
The Sustainable Score differentiates investments that make a high contribution to climate action by qualifying as climate finance (to the right), and those that qualify partially or not at all (to the left). 9
On the right, investments that meet climate finance criteria are either already low-carbon, or they indirectly enable emission reductions in other activities. We also know that sustainability is about increasing the resilience and adaptive capacity of businesses, people, and nature to withstand the consequences of global heating. We can do this either through direct measures, or by specifically targeting businesses that provide solutions for others to adapt.
On the left, non-climate finance investments are always scored based on their GHG emissions, where investments with the highest emissions are awarded a negative score. Non-climate finance investments can obtain an additional point for pursuing measures that strengthen adaptation and resilience (A&R). Climate finance investments are scored on GHG emissions avoided (renewable energy), reduced (energy efficiency), or sequestered (forestry), general contribution to mitigation objectives, including circular economy (as set out in the list of eligible activities in our climate finance methodology), or on their A&R finance qualification.
9 – Climate finance is defined by the Common Principles for Climate Mitigation and Adaptation Finance Tracking agreed by the Multilateral Development Banks
SPOTLIGHT: BII’s Climate Strategy
In 2014, CDC (as BII was known then) committed to considering climate change in every investment; our first fully-fledged climate strategy was published in 2020.10 It is organized around the four Task Force on Climate Related Financial Disclosures (TCFD) pillars of strategy, governance, risk management and metrics and has two main objectives: to take responsibility for the climate impact of our entire portfolio by pursuing increased opportunities in climate sectors, and to future proof our dual mandate of financial return and development impact. It also sets our approach to Paris alignment both at portfolio and transaction-levels.
Our impact score was designed to reflect the same level of ambition as our 2020 climate strategy. This requires two things:
- First, to do more investments that make an active contribution to climate action, which implies making more investments that qualify as climate finance; and
- Second, to ensure that all our investments reduce emissions and improve resilience over time.
The challenge is that the design of the scoring system favors simplicity and objectivity, whereas climate change is tremendously complicated, and our climate strategy and related commitments are wide-ranging. We therefore faced difficult choices regarding which elements – just transition; adaptation and resilience; net zero by 2050 — to focus on, and what factors within these elements should generate the highest scores.
Our solution for the Sustainable Score puts the most weight on penalizing GHG emissions and awarding avoidance/reductions/sequestration, which accounts for most of the variation from -1 to 4, with some points available for investments that meet adaptation and resilience criteria. We decided our response to the challenges of a Just Transition will sit outside the scoring system.
The score requires investments to meet certain materiality thresholds (e.g., what proportion of the investment is targeted at A&R activities) which can be hard to determine and requires some expert judgment concerning whether the investee’s activities meet the relevant definitions (e.g., what activities quality as A&R). We simplified the initial scoring of non-climate finance investments, based on emissions, by using a list that defines lower and higher emitting sectors, but investments will be rescored after investment based on a comparison of actual emissions against the GHG Protocol and the Standard by the Partnership for Carbon Accounting Financials (PCAF). GHG avoidance is calculated by using average emissions intensity of power sources displaced from the grid by renewables, using country-specific data from the IFI Technical Working Group. Given the importance we ascribe to supporting economic transformation, we adjust scores after investment using official Paris-aligned sector emissions pathways, which show the rate at which a sector must decarbonize to meet the 1.5 degree temperature goal.
This is the only part of the system where an investment may receive a negative score. We are deeply cognizant of the possibility of other harmful consequences of investments and seek to manage those risks on every transaction. Other risks of negative impact are handled in the more detailed transaction-level assessments, outside the scoring system. But carbon emissions are often unavoidable with many investments, and high emissions detract from other positive impacts. We only invest in highly emissive businesses if they are assessed to be Paris-aligned.
10 – Available here: https://assets.cdcgroup.com/wp-content/uploads/2020/07/01181554/CDC-climate-change-strategy_FINAL-FOR-PUBLICATION-1.pdf
Inclusive Score
The Inclusive Score is based on the profile of the stakeholders that the investment is expected to benefit. We consider inclusion across three dimensions:
- Cross-country inequality;
- Within-country inequality (i.e., reach to lower-income members of society); and
- Reaching or empowering otherwise excluded groups, namely women and black African business owners and leaders.
Where stakeholder characteristics are known and reaching these stakeholders is central to the impact thesis, scores are determined by percentage of stakeholders reached living below $5.50/day (a poverty line based on consumption measured in Purchasing Power Parity dollars, maintained by the World Bank). These stakeholders could be customers, employees or suppliers, but the score will only consider the key stakeholder group corresponding to the intended impact of the investment. Evidence to support our assessment of the poverty level of expected stakeholders can be obtained through surveys or based on a set of approved proxies. The maximum direct reach score is 4.
Alternatively, default country scoring is used when data about the poverty level of key stakeholders is not available. This is typically the case when the investment thesis is based on the effects of “economic enablers” on productivity across a market or region. These scores are based on ranking countries according to the poverty gap, GDP per capita and fragility measures (see Appendix 2 for the full list). Where an investment is in multiple countries, the score will be determined by the average of the investment’s reach weighted by the most relevant metric (jobs, revenues, use of proceeds etc). The maximum default country score is 3. Although we do not use the poverty headcount rate for country scores, this means in effect that a country with 50% of the population below $5.50 is treated differently to a business with 50% of its workers below $5.50 – we do not assume the indirect impacts of an investment will be distributed evenly across a country. We also want to give deal teams and incentive to find out information rather than rely on the default.
Gender and diversity. Additional point(s) can be awarded for investments that meet these criteria. We use the 2X Challenge criteria to determine which investments enhance women’s economic participation and have developed similar criteria to recognize black African business ownership and leadership.11 These points are awarded in addition to the poverty level or default country score (one point for each qualification) to a maximum of 4.
Figure 7: Inclusive Score Elements
11 – Available here: https://www.2xchallenge.org/criteria
SPOTLIGHT: Black Ownership and Leadership for Development (BOLD)
In 2018, BII launched its first Gender Equality Position Statement, setting out our commitment to advance gender equality at all levels of the corporate value chain. As we gained experience applying a gender-lens to our investment decision making, we started to broaden our approach to consider other forms of diversity.
An initial review of our portfolio indicated that BII’s funding of black-owned and led businesses in sub-Saharan Africa is below market levels. If this proves to be correct, it needs to change. To make that happen, our Impact Score now awards points to black-owned and led businesses in sub-Saharan Africa, a new initiative we refer to as Black Ownership and Leadership for Development (BOLD).
We are in the early stages of developing our overall approach to BOLD, but our ultimate aim is to meaningfully increase the number of Black African-owned and led businesses within our sub-Saharan Africa portfolio.
For the BOLD component of the Impact Score, we took inspiration from the gender-lens investing approach developed for the 2X Challenge, an initiative founded by DFIs of the G7 nations. Under the 2X criteria, female representation is assessed in four areas: Ownership, Leadership, Employment, and Consumption. For the BOLD criteria, we assess black African representation at the Ownership and Leadership levels. At this stage, Employment and Consumption are not considered within our BOLD scoring, because, in sub-Saharan Africa, there is less need to incentivize investments in businesses with black customers or workers. The Ownership and Leadership thresholds for 2X and BOLD differ; this is based on our understanding of what the current market levels are for these two domains.
For investments in corporate structures with different layers, from holding companies to operating companies, we look at the entity that will receive our funds. For fund managers, we assess both the fund manager and the underlying portfolio of companies against the 2X /BOLD criteria. When both layers qualify, we award a full point. When only one layer qualifies, we award 0.3 of a point.
Our efforts to direct more of our capital towards female- and black African-owned and led businesses have thrown up important impact-related questions. As a DFI, funded by Official Development Assistance (which has a legislated mandate to reduce poverty), we are usually concerned with improving the lives of low-income people, rather than business leaders. Should our new focus on owners and leaders be grounded in the belief that female and black leadership will lead to better development outcomes in general? And better outcomes for female and black workers and customers in particular? Certainly, there is plenty of evidence that racist and sexist discrimination is a source of economic inefficiency12. Or is addressing female and black African underrepresentation a sufficient justification in its own right?
As with all elements of the impact score, choosing the appropriate weighting is challenging. We had to decide on the importance diverse leadership relative to other varieties of impact. Our decision was to award 1 point each for 2X and BOLD qualification, as contribution towards an Inclusion score that ranges from 0 to 4.
There are other difficult questions here, without neat answers. For example, why only positively score black owners, and why only sub-Saharan Africa? Yet we thought it was more important to start acting where the scale of inequity seems to be greatest, than to take more time to devise an approach that covers all contingencies, at this stage. In time, with the lessons of experience, we expect to refine and extend our approach.
12 – Examples are Hsieh et al. (2019 ) “The allocation of talent and US economic growth” Econometrica and Chiplunkar & Goldberg (2021) “Aggregate Implications of Barriers to Female Entrepreneurship” NBER.
Updating Scores Based on Performance
All investments will be re-scored at regular intervals to track whether they are performing against their original thesis. Scores can remain constant or move up or down depending on performance. A re-score will automatically be triggered at exit.
Each investment has a monitoring plan that is tailored to its impact thesis as outlined in the Impact Dashboard. For instance, the ‘intensity’ component of the Productive Score is based on a forecast impact metric at relevant time horizons, and this element of the score will be recalculated periodically, based on observed delivery. Even if no intensity benchmark was available at the time of investment, it will usually be possible to set quantitative targets so that the score can be updated in the light of performance against those targets. Other elements of the score that are amenable to rescoring based on new information include the geographic split for investments through fund managers, new information relevant to the Inclusive Score, or elements of the Sustainable Score that are tied to quantifiable thresholds.
We will update scores every two years, and upon exit. Once updated scores are available, they will feed into staff remuneration under our Long-Term Incentive Performance Plan. Underperforming investments will impact renumeration, which will help guard against optimism bias in scoring at the time of investment.
Example Transactions
This section presents a range of fictional but realistic examples of impact scores. More can be found in Appendix 3.
Our Experience So Far
At time of writing in mid-2022, we have only a few months’ experience of using this score in earnest (after having “shadow scored” transactions during the second half of 2021). There are already clear signs that the score has increased the alignment of origination efforts with our strategic priorities. This was already evident as a prototype of the score was used in 2021, when investment teams were devising their business plans.
Despite having tried to minimize the need for subjective judgments in many cases we found there was a need to discuss the choice of need to be scored against, the application of the guidelines governing economic enabler and catalyzing markets point, and discussion of the information we have to determine sustainability and inclusive scoring. We hold a weekly committee meeting to review decisions and to identify the need for more precise guidance in our scoring handbook and training materials, and we also hold less frequent steering committee meetings to adjudicate on questions such as the introduction of new benchmarks.
With the caveat that we should not read too much into a small sample, the average score for pipeline transactions is tracking moderately above the average we found in back-testing. Our perception is that this reflects more targeted origination and is also the result of engagement with project sponsors around issues such as inclusion. In other words it reflects a genuine increase in development impact, not that teams have learned to game the system.
Our overall impression is that the scoring system has been embraced by the organization because it generates a clear ranking of investments against our strategic impact objectives, despite the fact that nobody enjoys receiving a low score, and that we have landed in a sensible place on the tradeoff between accuracy and pragmatism. The score must be seen alongside the other tools in our impact management system, each with a distinct purpose. The most important accomplishment, we feel, is the creation of an impact management that starts from well-defined strategic objective and flows through every stage of the investment lifecycle. That, ultimately, is what we expect to really raise our impact as an investor.
Appendices
APPENDIX 1 — LIST OF NEEDS AND INDICATORS
APPENDIX 2 — LIST OF DEFAULT INCLUSION COUNTRY SCORES
APPENDIX 3 — EXAMPLE TRANSACTIONS
IN THIS LESSON
- Introduction
- Strategic Impact Objectives
- Managing Impact Across the Investment Cycle
- BII’s Impact Framework
- Investment-Level Impact Dashboard
- Approach to Investor Contribution
- BII’s New Portfolio-Level Impact Score
- Calculating, Productive, and Sustainable Score
- SPOTLIGHT: BII’s Climate Strategy
- Inclusive Score
- SPOTLIGHT: Black Ownership and Leadership for Development (BOLD)
- Updating Scores Based on Performance
- Our Experience So Far
- Appendices
