Jim Vine, a Senior Research Officer at Understanding Society, gives an overview of the new types of data being collected as part of the Innovation Panel, the UK Household Longitudinal Study’s test bed for survey methods.
As a large household study, Understanding Society gathers data to build up a long-term picture of life in the UK. As a longitudinal study, much of the data collected prioritises stability, so researchers can see how things change over time. Keeping the content of the study consistent helps us build a rich picture of life in the UK, and for many of our participants that history goes back over 30 years.
But Understanding Society has another important mission – to develop survey methods and experiment with collecting new forms of data, and for that we have a test bed: the Innovation Panel. So, how does it work?
The Innovation Panel is a sample of around 1,500 households, surveyed each year like the main Understanding Society panel, and most of the questions we ask are the same across the panels. The Innovation Panel, though, allows us to add on experiments in data collection and survey development. Some of this is intended to specifically investigate options for running Understanding Society. If there are things we can learn, such as techniques we can use to improve our own data collection, this is a good way to find them.
As a panel survey providing a rich dataset on respondents going back many years, any changes to the main panel need to be made very carefully. One way of taking that care is to use the Innovation Panel, where we can try out new ideas and get a sense of how they work.
For new types of data, we need to think about how the data might best be collected, how our participants might respond to the request, and what the final dataset would look like. We want our new data to be useful for research and to complement the data we already collect.
Testing on the Innovation Panel allows us to try out different data collection methods and see if they’re practical to roll out to the main survey. We can also consult with topic specialists so we can be confident that what we’re collecting is good for researchers.
As well as testing ideas for Understanding Society, the Innovation Panel has long been open to proposals from external researchers. Each year, we run an open competition to identify proposals for new experiments which might lead to valuable new insights. To hear when the next competition is launched, keep an eye on our news page or sign up for our newsletter.
Capturing life events as they happen
Most data in Understanding Society are collected through standardised questions in an annual interview. Contacting participants once a year keeps the survey burden lower for them but may mean participants forget some things that have happened during the year. Could we gather more regular feedback from participants without it feeling like we’re overloading them with requests?
Between Waves 11 and 13 of the Innovation Panel, we trialled event-triggered data collection. Participants were sent a monthly message asking if they’d experienced any important changes, such as a new job, a new relationship, or a pregnancy. If they had, we asked follow-up questions.
The results showed that participants were happy to respond to the short survey each month, and we had consistently good response rates across the months. This extra data — collected at the time of a life event — has the potential to enrich the data collected at the annual interview.
Researchers can use the Life Events Study data now, as it’s available from the UK Data Service – and we’re looking at the practicalities of collecting life event data in the main survey.
Testing new technologies
The Innovation Panel has also been used to test new ways of collecting data and exploring how these datasets work in practice for research. Using new technologies can be challenging for our participants and for us. Some experiments have been very successful, others less so.
We collected data on spatial navigation skills using an app-based game, which was very popular with participants, and allowed us to collect data on the cognition skills that lie behind one of the first signs of dementia.
Collecting information on body composition using an app that captured body measurements through a phone camera was less successful, as the app proved tricky to use and participants struggled to take pictures of the right quality.
Collecting new biological data has also been trialled in the Innovation Panel, with participants providing us with dried blood samples, hair samples, and finger measurements. Testing these in the Innovation Panel helped us prepare for collecting bio-measures in the main survey, which is taking place at the moment.
We currently have in the pipeline tests of air quality sensors in participants’ homes, interactive life history calendars, and wearable activity trackers, so there will be more new types of data for research in the years to come. All additional tasks go through ethical approvals processes, and when required we seek specific consent from participants, often built into the Innovation Panel survey.
Finding out what’s in the Innovation Panel
We’re keen for researchers to explore the findings from the experiments we carry out, and to use the new data generated by the Innovation Panel.
If you want to know what’s been collected, the annual Innovation Panel working paper is the place to look. In November, when the data from wave 16 of the Innovation Panel was released via the UK Data Service, we published a working paper describing the studies carried in the wave, featuring information on new survey questions as well as methodological experiments.
This wave of the Innovation Panel featured experimental work on:
- Measuring worries about climate change: The effect of a subtle wording change
- The prevalence of depression in Understanding Society
- Using e-vouchers for unconditional incentives
- Youth survey experiments
- Obtaining measurement data from the ‘red book’
- The effect of incentives on the willingness to participate in a gamified app study
- Use of a government logo
And the new survey questions were on:
- Establishing a databank of smallest effect sizes of interest (SESOI) for prominent psychological constructs
- Anticipated gender discrimination in the labour market
- Domestic workers and platforms
- Numeracy and the long-term future
- Flexibility stigma in the UK post-pandemic : intersectionality of stigma across gender and parental lines
- The relationship between employment status, passage of time judgements and wellbeing
- Cognitive reflection and politically motivated reasoning
Data description
The working paper about wave 16 of the Innovation Panel also included two chapters of a new type – data description chapters.
Some scholarly journals now accept articles providing descriptions of research datasets, sometimes termed data descriptor articles or data papers. Our data description chapters are intended fill a similar niche in the Understanding Society working paper series.
Data description chapters are intended to serve (at least) three purposes:
- improve the visibility of new data
- make it easier for data users to understand novel data that are being released
- provide a way for data users to acknowledge and cite specific aspects of the data, alongside their main data citation.
There are two of these in the working paper, one on Child development measures from the ‘red book’, and the other on Sea Hero Quest: Spatial navigation data.
Improving the visibility of new data
As we said at the beginning, Understanding Society is, at its heart, a data-generating endeavour. As a core investment in social sciences by the Economic and Social Research Council (ESRC) and the UK government, it produces a lot of data. The best thing for the data we produce is for them to be used – and they’re available for researchers via the UK Data Service (including Innovation Panel data).
As the data can be found in the UK Data Service catalogue, on some level they are already visible. By publishing data description chapters, we hope to enhance that visibility, giving prospective data users more opportunities to discover data that might be useful in their research.
Understanding novel data
For most variables produced by Understanding Society, the ultimate documentation about the data generating process is available in the form of the questionnaires. These are all published on the Understanding Society website, and data users can always refer to them if they want to dig into the nuance of how the information was obtained.
But in a few cases, the data variables we make available do not come from questionnaire responses. In those cases, having a data description paper can fulfil a similar function to published questionnaires, giving data users a way to understand what a variable is and where it comes from.
As well as making the data more understandable to direct users of the data, increasing the clarity of the variables and how they were generated can ultimately contribute to the transparency of any research which uses them. If data users cite the data description chapters – as we hope they will – readers of their publications will be able to follow up on the data source if they have any queries.
The future
Future waves of the Innovation Panel will see more datasets produced that might be well served by being the subject of data description chapters – there is at least one such study in the seventeenth wave of the Innovation Panel (IP17).
With more likely to come, we would welcome any comments you might have on these first two chapters. Do they do a good job of introducing you to the data? Did they spark any ideas about how you might use these datasets? Is there anything additional that you would like future data description chapters to cover?
What’s next?
The fieldwork for IP17 is complete, so the proposers of IP17 studies are working on their analysis. The questionnaires for IP18 are in the field, and earlier this year we ran the annual competition for IP19.
We have also produced a new style of user guide for the Innovation Panel. The PDF is still available, but we now also have a web-based guide, to make the different sections easier to navigate.
Find out more about the Innovation Panel.
If you have an idea for an idea for the Innovation Panel sign up to the Understanding Society mailing list to hear when the next competition is announced.
Please get in touch if you have any feedback on the IP16 Working Paper.
About the author
Jim Vine is a Senior Research Officer at Understanding Society, working in the study’s innovations directorate, with a research focus on the Innovation Panel survey and other novel developments within Understanding Society.
Comment or question about this blog post?
Please email us!