We took a look at the REF 2021 Impact Case Studies and how the UK Data Service collection has been used in the research submitted.
What is the REF?
The Research Excellence Framework (REF) is the system in England, Wales, Scotland and Northern Ireland for assessing the quality of research in higher education institutes. It was first conducted in 2014 and replaced a previous model. According to the REF website, there were three main aims of REF 2021:
- To inform the selective allocation of the four Higher Education funding bodies’ grants for research to the institutions which they fund, with effect from 2022-23.
- To provide accountability for public investment in research and produce evidence of the benefits of this investment.
- To provide benchmarking information and establish reputational yardsticks, for use within the Higher Education sector and for public information.
In total 1878 submissions were received from 157 universities across the UK which amounted to 185,594 research outputs and 6,781 impact case studies.
Impact and the REF
For REF 2021 impact was defined as
“the effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.”
Higher Education Institutions were required to submit impact case studies to show the impact that their work and research has beyond academia. Each submission was required to submit at least two case studies, with more for projects involving larger numbers of staff. Impact accounts for 25% of the overall outcome award for each submission and they were assigned against two criteria: reach and significance.
REF 2021
It is great to see the focus on evidence of impact in these submissions, with 50% of the studies receiving 4-star ratings (outstanding – the top rating) and a further 38% to be judged as very considerable (3-star).
We were interested to see how frequently and in what ways the data made available by the UK Data Service was used in these research projects and the submitted case studies. We did some basic analysis to see how often data in the Service collection was referenced in the different sections of the impact case studies. We also drilled down a bit deeper to look at a few projects that specifically mention the Service or mentioned using the data available in more detail.
It’s worth pointing out that this was not a scientifically rigorous analysis, but together they show how widely the data is used across disciplines and institutions.
Overall, we found over 450 case studies that mentioned at least one of data set that is available on the Service.
We shortlisted 12 studies to do a more focused analysis, these were:
- British Social Attitudes Survey
- Census Data
- Crime Survey for England and Wales
- English Housing Survey
- English Longitudinal Study of Ageing
- Family Resources Survey
- Health Survey for England
- Households Below Average Income
- Millennium Cohort Study
- Understanding Society
- World Development Indicators
- World Economic Outlook
These sets were mentioned in over 200 case studies. 49 studies mentioned one or more of the data sets in more than one of the impact sections, which we took broadly to mean that the data was perhaps more central or integral to their research.
Of these 49 studies the majority (40) mentioned census data. The rest were all mentioned at most four times. A further 11 case studies explicitly mentioned the UK Data Service in their case studies.
Areas the data is used
It is great to see the data available on the UK Data Service being used in so much high-quality research across the country. Having high quality data that is accessible and usable by researchers can help them to achieve impact through their work in lots of different ways.
To highlight just a few areas that the data was used in:
- The economics of aging
- Measuring and recording domestic violence
- Road safety
- Income inequality measurements
- Mental health
- Fitness and health
- Housing & homelessness
- Migrants
- Workers’ rights
Impact on the UK Data Service
It was also interesting to see that at least one case study referenced how they had impacted the UK Data Service!
In this case study, the Centre for Multilevel Modelling at University of Bristol discussed how they increased the number of non-academics and non-specialists who can analyse the complex datasets to help conduct research in a range of areas.
They did this by embedding their research into software, face-to-face training, and training materials. This, according to the Director of the UK Data Service User Support and Training team, the materials they produced will allow datasets in the Service collection to be more widely accessible. Quoting from Section 4 of the Case Study, the authors describe this impact:
A recent British Academy grant supported CMM in further developing our Stat-JR software to automate the creation of training materials. This encourages others to work with us. For example, recently the UK Data Service have used Stat-JR to develop online training using their own national data resources, with a significant effect on their internal training strategy. They expect the materials produced to benefit researchers through learning basic statistics techniques using key UK datasets. Their Director of User Support and Training stated: “The materials produced […] will make this dataset more visible to our wide community of existing researchers, help our training effort and attract new users to our resources.”
It is great to see that the impact goes both ways!
About the Author
James Lockwood is Research Impact and Engagement Manager at the UK Data Service.
You can find more about the UK Data Service training and events on our website.