The Smart Energy Research Lab has been working with the UK Data Service for nearly 7 years to provide researchers with access to high resolution domestic energy data. James O’Toole highlights some of the impacts the partnership has achieved and encourage researchers to explore the resources.
New releases
The recent release of the 6th Edition of the Smart Energy Lab Observatory Data (2019-2023: Secure Access) is the latest in a series of ground-breaking outputs in the field of UK energy research that have resulted from the collaboration between the SERL consortium and the UK Data Service since the project began in December 2017.
Other notable outputs include the SERL Stats Report (2nd Edition released in March 2024) and the SERL Statistical Dataset used to create it, as well as a growing list of academic papers and contributions to government reports and research projects.
(Researchers may find the SERL statistical dataset the most accessible first step in finding out about the SERL data. Access is quite simple for anyone with a UK Data Service account (which is simple to set up if you don’t already have one).
The SERL dataset comprises half-hourly electricity and gas smart meter data gathered from 13,000 consenting GB homes beginning in 2019. This data is linked with contextual data such as Energy Performance Certificates (EPC) and weather data; and demographic and building information collected from a participant survey. This level of granularity of smart meter data available to UK researchers is a quantum leap from the annual aggregated totals available to researchers before SERL.
The end goal of collecting and making this data available is to get a better, ‘real-world’ understanding of how energy is used in homes which can then inform policy, interventions, and innovative product design to allow us to tackle key energy challenges in the UK, making the energy supply more affordable, secure, and sustainable.
History
The SERL Consortium project was initially funded by the UKRI Energy and Physical Sciences Research Council (EPSRC) to commence in August 2017. The 5-year project was made up of 8 partners led by University College London and including the UK Data Service. The project was tasked with constructing the technical, governance and participant recruitment framework that would allow UK academic researchers to access smart meter data available via the Smart DCC national smart meter communications network. Working with technical partners CGI, SERL was one of the very first academic organisations approved to access smart meter data.
Over the next two years the complexities of the technical and governance structures needed to allow the data to flow were navigated with the UK Data Service team, a main partner in working out the technical processes needed to get data from the DCC into the SERL observatory. The Service also advised on the data governance procedures that needed to be in place to protect the consumers’ data under the Smart Energy Code and Genery Data Protection Regulation.
This was no mean feat in a complicated and fast-moving field and with a public and media that was understandably sceptical of smart meters as a recent technology being rapidly introduced to homes. Protecting the privacy of our consenting households and ensuring that their data was only used for the public interest research that they had consented to was always the paramount concern of SERL and this remains the case today.
Researcher access
One of the key reasons for the SERL project to exist was to build a resource via which all accredited UK academic researchers would to be able access smart meter data so that each individual academic or project would not have to attempt this process separately. Due to the time, cost, and complexity of setting up access to the DCC, this would be a massive duplication of effort and make access effectively impossible for many researchers. Although accessing data via SERL still involves several steps such as becoming an ONS DEA accredited researcher and being approved by an independent Data Governance Board, the process is a lot simpler than starting from scratch. With the production of the Stats Reports and the Statistical Dataset (beginning in 2022), outputs from the SERL dataset have been simplified and made more easily accessible for a wider audience.
As the host of the SERL dataset, the UK Data Service has also been a key player in managing researcher access to the SERL data, processing applications, and ensuring data governance rules were adhered to.
Participant recruitment
With the infrastructure in place, participant recruitment began in August 2019 and by March 2021 – after 3 waves of recruitment – SERL had recruited over 13,000 GB households.
The first SERL dataset was made available in August 2020 with data from ~1700 participants recruited in the pilot wave. The second edition including the next wave of recruits contained 4800 households in total. October 2021 saw the release of the third edition which comprised the full 13,300 cohort.
Impacts and current work
Fast forward 2.5 years and the impacts of the SERL project can be seen in a number of areas.
At the time of writing, 65 researchers from 34 research projects across 18 different organisations have accessed the Secure Access Dataset via UKDS. This does not include SERL’s involvement in recruitment, data processing and analysis for a number of government projects including evaluating the Smart Energy Savings Trials (SENS) and various Green Homes Grant intervention initiatives.
SERL has been cited as a resource used in a number of influential government reports such as the ‘Net Zero Review’ and the Digitalisation Taskforce Report
Funding was continued initially by a no-cost extension that took the project to August 2023 and the new Energy Demand Observatory and Laboratory project has now been funded which will continue and expand the work of SERL with more in-home sensors and data streams in selected cohorts of homes.
The SERL Statistical Dataset has been accessed over 200 times from over 65 organisations including the Bank of England, the Economist and the Prime Minister’s Office and SERL data has been referenced in Private Eye magazine’s Housing News and the Daily Telegraph.
The SERL Stats Report Vol I has been downloaded over 4.016 times across 72 countries and the second volume released in March 2024 has had 324 downloads across 10 countries. The SERL team have produced over 20 academic papers.
The project team have been able to offer real world practical advice to consumers based on the data in the form of a The Conversation article advising on the possible benefits of turning down thermostat temperatures. SERL’s research has also provided valuable information on the validity of EPC – a requirement for those buying and selling homes. SERL data suggested that higher EPC rated homes (bands C-G) use significantly less energy than EPC models predict. The work showed the importance of using ‘real-world’ data over models, as well as having implications as to the future of how EPCs will be calculated, potentially influencing the move towards more accurate measurements.
Covid-19 and the Energy Cost Crisis
The value of SERL’s rapid access to real-world energy data has been particularly demonstrated by projects carried out on the impacts of Covid-19 and the Energy Cost Crisis where SERL had the agility to be able to get data from households living through both of these events and analyse the effects they were having on real homes.
In the case of the energy cost crisis, SERL studies showed that consumers in the observatory sample paid more for their energy in Winter 2022-23 despite using 10% less energy. The study was carried out with analysis of the SERL data combined with the results of a survey sent out the SERL participants. The research also showed that those who reduced their energy consumption most were those who reported financial and well-being problems, suggesting that reducing energy consumption was more through necessity than choice. Identifying fuel poverty and looking at ways to alleviate it has been a key theme of numerous SERL research projects.
In the case of the Covid study the findings again illustrated the value of having near real-time data from real households to find out what is happening ‘on the ground’. Findings showed that average electricity consumption rose by 8% in the first year of lockdown and gas consumption rose by 6%. In the 2021 winter lockdown electricity rose by 12% and gas by 9%.
In addition to these impacts SERL is also working with DESNZ on several projects which look to extract the full value of SERL as a public funded resource by providing up to date energy consumption statistics and potentially linking SERL to other large datasets to add further value.
What’s next?
As mentioned above the Energy Demand Observatory and Laboratory (EDOL) programme funded by UKRI EPSRC and running from 2022-2027 will continue and expand upon the work of SERL. EDOL will have its own ‘observatory’ sample of 2000 homes with some basic sensors giving more in-home data. There will also be smaller ‘labs’ of around 200 homes looking at particular areas of interests, and some ‘forensic’ groups of 20 or so homes who will be asked to share additional data streams from their homes.
The future funding of the SERL Observatory Dataset, Statistical Dataset and Reports is under discussion but various key stakeholders recognise the value of the resource – and how it will grow over time – so we hope the data will continue to be of use in addressing key UK energy issues for many years to come.
For more information contact info@serl.ac.uk
About the author
James O’Toole is the Smart Energy Research Lab Consortium Manager and has worked on the project since 2017. He is responsible for the day-to-day administration of the consortium in collaboration with the SERL senior management team. James also leads on promoting SERL’s work as widely as possible across the field of energy research. He joined UCL Energy Institute in October 2015 and was previously End Use Energy Demand Centres Coordinator.