Rachel Oldroyd, UK Data Service Data Impact Fellow, explores a very real question facing researchers.
Measuring impact is essential to any research project and every academic will have pondered this question at some point in their career.
Research Councils UK define impact as
‘the demonstrable contribution that excellent research makes to society and the economy’
but quantifying impact is a complicated and often subjective task, making this question notoriously hard to answer.
Research-active academics are expected to use certain channels of research dissemination, such as presenting at international conferences and publishing high quality journal articles. Metrics which measure these activities are well-understood in academic circles.
Visit the ‘Measuring Impact’ webpages of any University library and you will presented with a list of quantitative measures of research impact, also known as bibliometrics. For example, citation-count quantifies the number of times a journal article is cited; scholarly output measures productivity, and journal impact factor provides an indication of the quality of the journal within which an article is published.
Whilst these measures are useful for understanding patterns of research dissemination in academia, they also have limitations.
Metrics measuring the frequency of citations are often not indicative of article quality, a journal article may be cited several times because other authors are refuting its findings. Similarly, citation count varies from field to field. In disciplines such as medicine and public health, systematic reviews are common and articles frequently cite other studies to validate their findings. Journal Impact Factor also varies largely from discipline to discipline, dependent mainly on the size of the field of study, and is therefore not always indicative of journal quality in a particular field.
Aside from these limitations, the use of bibliometrics to measure academic impact cannot be refuted. But impact is not only concerned with research dissemination in academia, but also the effect research has in the wider world.
This is something which these measures struggle to quantify. So how can the economic, societal and legislative effect of research and its impact on policy be measured?
Economic, societal and legislative impact outline not only how research can benefit individuals and organisations but also how research can contribute to understanding and improving policy issues.
Identifying people who may benefit from a research project at an early stage is vital in developing impact. Key stakeholders can provide useful feedback, ensuring the research outcomes are useful and timely. Key parties can also help to propagate the research among professional audiences who may be otherwise difficult to reach.
Presenting at a public seminar series, engaging with local schools through educational engagement events, targeting relevant professional publications and getting involved with public-outreach events are other ways in which impact can be increased above and beyond academia.
Although not strictly a quantitative measure of impact, platforms used to record such engagement activities, such as ResearchFish, are particularly useful for researchers.
Impact assessment platforms provide an ideal opportunity for researchers not only to reflect on engagement activities but also to think about their ongoing societal, economic and legislative impact. This may vary from a change in the audiences’ way of thinking or behaviour, to a change in policy or legislation. Many research organisations and research funders make use of ResearchFish to keep track of the outputs, outcomes and impact of research they fund.
Alongside the use of bibliometrics and recording engagement activities on impact assessment platforms, researchers can also use Altimetrics to measure online public engagement.
Altimetrics (alternative non-traditional metrics), can capture online activity not historically measured by bibliometrics, as unlike citation count and measures of journal quality, Altimetrics take into account the number of times an article is downloaded, shared or mentioned on social media or traditional media. Although they can be subject to the same problems as bibliometrics, Altimetrics can provide a clearer and timelier picture of how the research is being received and discussed online and also in wider-reaching media outlets. It can be argued that Altimetrics offer a more suitable alternative than bibliometrics for measuring public engagement.
Clearly, measuring impact is an imprecise science, but there are a number of ways in which researchers can keep track of academic impact and the impact their research generates in the wider world.
About the author
Rachel Oldroyd is one of our UK Data Service Data Impact Fellows. Rachel is a quantitative human geographer based at the Consumer Data Research Centre (CDRC) at the University of Leeds, researching how different types of data (including TripAdvisor reviews and social media) are used to detect illness caused by contaminated food or drink.