Faculty & Research
The School


Why we need to measure bad impacts as well as good

Impact measurement is a buzzword, but whose impact are we talking about, and how will it make a difference? Karim Harji explains. 

Businesses love measurement. It is often said that ‘you can’t manage what you can’t measure’. Unsurprisingly, therefore, a large part of any manager’s job is devoted to target-setting, reporting on Key Performance Indicators, assessing Return on Investment, looking at analytics, and conducting appraisals of one kind and another.
Non-profits and other social organisations normally tended to shy away from metrics like this. They placed a greater premium on good intentions and activities, and raising broader awareness of important social issues. For organisations with a social mission, as long as they did more, people and communities would be better. 
But with increasing demands for accountability and transparency around social sector funding, and growing interest in impact investing and business-originated philanthropic organisations, the need to measure ‘impact’ is rising up the agenda. Measuring impact is viewed as something all ‘good’ organisations do. For mission-driven entities, this includes both financial sustainability and social impact.
Done well, this is a good thing. It can improve decision-making in the organisation – both proving impact and improving performance – and ultimately increase the social value that is generated for marginalised individuals and communities. There are in fact a number of good examples of impact measurement design from the social and public sector. However, we hear much more about why measurement is not done well, than about when it is designed and used to make informed decisions. 
But because measurement has traditionally been the preserve of business, there is a tendency to look to the private sector for models. Unsurprisingly, this does not always work well when dealing with complex social and environmental issues. 
The most visible example of measurement in business is the Annual Report produced by public companies for their shareholders, which provides a snapshot of the financial health of the business alongside management outlook and commentary. Many charities and non-profits now produce a similar annual document featuring stories about the positive impact they have had as a result of their programmes. 
These reports are often not representative of actual impact measurement. They may fulfil a necessary communications function, but are typically presented with the intention of reassuring their donors or investors that their money has been used wisely, and that they are generating positive results. Inevitably, organisations choose to feature the most encouraging stories and maintain a typically upbeat tone. 
As a result, these reports (and their authors) seldom describe the full range of results, lessons, and implications – especially when things do not go according to plan. They focus on the intended, positive impacts, and can deliberately or inadvertently prioritise one interpretation of ‘what really happened’ over others. Unintended or negative impacts are often ignored, and this can be problematic when certain stakeholder groups – communities, governments, potential partners – require a clear understanding of the context within which social change happens. 
Why does this matter?  For a start, those whose perspectives may be most important to elevate – the ‘end beneficiaries’ of these social programmes and investments – may want to know that their lives are better, and that the people who are working on their behalf are also partly accountable to them. Of course, theirs is not the only point of view:  there are other power dynamics and organisational systems that also influence whose impact matters, and how it is measured. But good measurement practice should encourage the integration of multiple stakeholder perspectives, ideally on their terms. 
It is also important to realise that measurement is not simply a technical or communications exercise. People and organisations will act differently when they have access to more or better impact data. Impact reporting should therefore not be viewed as an annual, static process; rather, it can be one that encourages ongoing and timely use of impact data to influence how people and organisations behave, and how norms and rules are set and enforced. It can play a part in helping people continually make better decisions for the common good. 
You can explore these issues in more depth on our webinar Why measurement matters on 7 March 2018. Register your free place.


Karim Harji

Karim Harji View profile

Karim has over a decade of international experience in impact measurement and evaluation, particularly for impact investing, social finance, and social enterprise.
Executive Education

The Oxford Impact Measurement Programme