Esmeralda Bon shares her thoughts on measuring impact. In this blog post, she discusses key ideas behind non-academic impact. In her first post, she explored what measuring impact means. In her next post, she will look at specific considerations around impact for early career researchers.
What is non-academic impact?
Non-academic impact is impact that is not of an academic nature. Examples of non-academic impact are
- societal impact – such as the impact that the implementation of a new policy has on social circumstances
- actual physical impact, like that caused by a meteor hitting planet Earth.
The UK Data Service is expected to be interested in the former.
The measurement of this impact can be on a qualitative or quantitative dimension, as indicated with the following three non-academic examples.
A business may wish to measure the impact of a new product marketing campaign. The CEO may ask for spreadsheets showing the sales data from before and after the campaign, which is a quantitative approach, or collect the written or recorded opinions of (potential) customers about the product (a qualitative approach).
An activist could measure the impact of his speeches by counting how many people show up to his next demonstration (quantitative) or ask these people (his following) about their actual thoughts and beliefs regarding the matter at hand, to see whether these match with his (qualitative).
A father could measure the impact of his effort in bringing up his son and daughter by considering their success in ‘numbers’ (quantitative) – such as salary, awards and whether s/he has a permanent contract – or by looking at their norms and values, manners or belief systems (qualitative).
The measurement of impact – of any kind – can never be fully determined.
Both academic and societal impact can only ever be approximated because there is no experimental condition involved, but even numbers are not objective. That being said, the level of subjectivity, of involvement, required does differ depending on the approach taken.
Whereas simple statistic results indicating impact can be counted and analysed with the help of formulas and functions (low involvement), observations of impact require the observer to keep attention (medium involvement) and consultations with those impacted would require an even greater commitment, in terms of time and energy (high involvement).
The higher level of involvement, the more subjectivity we are dealing with.
However, this does not mean that a number-based, quantitative approach is necessarily preferable. The impact that we want to achieve is dependent on our goals and in some cases having a significant positive influence on just one person, policy or proposal outweighs being cited by a group of scholars.
About the author
Esmeralda Bon, @EsmeraldaVBon is one of our UK Data Service Data Impact Fellows for 2017. Esmeralda is an ESRC-funded PhD student in the School of Politics and International Relations at the University of Nottingham, in collaboration with the Committee on Standards in Public Life (CSPL), an advisory non-departmental public body sponsored by the Cabinet Office. Esmeralda is affiliated with the Centre for British Politics and the Nottingham Interdisciplinary Centre for Economic and Political Research (NICEP).