As part of our ongoing census series, Professor David Martin, Deputy Director of the UK Data Service, explores possible future directions for the UK censuses.
1961 could be deemed a turning point in census collection and processing.
Since then, we’ve basically been using the same questionnaire model, but with increasing use of technology. Innovations have included increasing automation in processing, changing the output geography design and, in 2021, designing a primarily online census, which has worked with great success .
Whether the UK censuses, in their current form, will continue is far from certain.
The quality of routine administrative data has greatly improved and there is growing pressure to use it to supplement or replace the traditional census. In 2023, the National Statistician will make a recommendation to Government on the future of the population statistics system beyond the 2021 Census.
The debate about whether we could replace the census with data from other sources is not new.
It was aired in 2001 but received significantly more attention after the 2011 census when there was growing excitement about the potential of administrative data.
Ten years on, we are much more knowledgeable and experienced about both the possibilities and pitfalls of working with these data. It is clear that there are some important topics covered by the census which are very hard to obtain from administrative data.
On the other hand, an important advantage would be avoiding the big census effort every ten years, both in terms of the increased workload and recruitment of temporary staff, but also in the currency of the data.
It seems certain that any recommendation will include greater timeliness of the core demographic and social information data.
Other countries’ approaches to the future of census
France
France has moved to a rolling census – instead of trying to enumerate the whole population in one go, a survey covers around 13% of the population each year as part of a 5-yearly cycle, and these data are pooled across years to give a census-equivalent picture of the French population.
Meanwhile, because the data are stratified, answers from the latest year can offer rapid insight into social change and modifications to the question set can be introduced progressively.
On the downside, this system can lead to some people in some places being potentially missed by the rolling sample, and the multi-year design makes the interpretation of the statistics more complex for users.
USA
The American Community Survey is another geographically stratified rolling survey model, although the annual sample size is only around 2.4%. The survey data can similarly be pooled over several years in order to provide estimates for at different scales.
The survey is held alongside a lightweight decennial census, which does not include the more detailed census-like questions covered in the survey.
The Scandinavian model
The Scandinavian countries rely wholly on the use of administrative data to assemble statistical information about many of the conventional census topics, drawing from healthcare, taxation, employment and education systems.
This has generally been made possible by the presence of population registers with a unique citizen identifiers and public acceptability of data linkage, in stark contrast to the UK context.
This approach makes it possible to extract census-like statistics on an annual basis, with no separate census enumeration required.
Belgium and the Netherlands
Other European countries such as Belgium and The Netherlands have moved away from conventional censuses over recent decades, and are now effectively conducting enumerations of temporarily-linked administrative data to produce census-like outputs.
These systems are being developed to make greater use of administrative sources and to produce outputs with greater frequency.
Australia, New Zealand, Canada
In general, many countries still (like the UK) conducting essentially traditional censuses such as Canada, Australia and New Zealand, have moved to using online data collection as the primary mechanism in the early 2020s, with support for those unable to complete the electronic questionnaire.
All are exploring the greater use of administrative data as part of their future population data systems.
Opportunities and challenges in the UK
There are many administrative data sources in the UK which could be developed or enhanced and securely linked to build statistical models of the population from which census-type data could be drawn.
The NHS central register, for instance, covers age, sex and internal migration status fairly well but includes few other attributes.
Linking these basic demographic characteristics for statistical purposes with benefits, tax, education and housing data, for example, could form the basis of a system to provide much of the core census data. The Office for National Statistics have been engaged in extensive experimentation of these approaches, alongside all the work on the 2021 census and have released multiple experimental datasets
There are, of course, some challenging issues with this approach which are still to be fully worked through.
Importantly, there are some census topics that simply aren’t covered by administrative data, for example journeys to work or household structure, for which the traditional census has been our principal source. This information is needed to inform many aspects of public policy, although some of the are interestingly also topics that will have been most affected by enumeration during pandemic restrictions.
Some of these concepts, such as travel to work, could potentially be measured very differently from aggregated big data sources such as tracking mobile devices, but would likely need to be augmented with survey data. These approaches offer exciting opportunities but still require substantial development.
We can anticipate a trajectory towards a system with much more timely outputs from administrative data, but which will not be so rich in coverage.
This is almost certainly going to need to be complemented by a more integrated survey system that will not provide all the detail for small areas, but which would effectively allow us to cover topics missing from the admin data, with maybe a periodic large survey or short census to re-establish the demographic baseline.
The Office for National Statistics are working on ambitious plans for a new multi-source population data system.
The future is going to be fascinating. Big questions which still need to be addressed include:
- As we change the system, to what extent will observed differences be due to changes in measurement and how much to real social change?
- If we do come to rely primarily on administrative sources, will the statistical agencies have sufficient powers to ensure continuity of measurement, or indeed to introduce additional topics over time?
Administrative data covering topics such as housing and income offer coverage of important domains which are absent from current censuses as the relevant questions aren’t asked. To take one example, in future we might expect to obtain far more accurate information on the vital issue of energy usage by observing smart meter data than by asking people about their behaviour.
These new approaches do not yet provide a complete solution to census users’ data needs, but are improving all the time, which means that some of the issues that we struggled with 10 years ago are now looking much more tractable.
While we don’t know yet what the future of the census will be, there will always be a need for high-quality, large scale population data to support research and policy making, and in the meantime the UK Data Service is gearing up to support users as they begin to access all the new census data products from 2021/22.
Further reading
US Census Bureau – Understanding and using American Community Survey data: what all data users need to know
Office for National Statistics – Census Transformation pages
Valente, P. (2019) Modernizing the census in Europe: traditional and new methods for the 2020 round N-IUSSP Newsletter of the International Union for the Scientific Study of Population 13
About the author
David Martin is a Professor of Geography at the University of Southampton and a Deputy Director of the UK Data Service.
He has been involved in several major data initiatives, including as Director of ESRC’s former Census Programme. His research in geographic information science led to development of the system of census output areas currently used in England, Wales and Northern Ireland. David was awarded an OBE in 2019 for services to Geography and Population Studies.
Were the differences between what the 2021 census found in England and Wales, and what official statistics thought was the case large? In other words, our current UK administrative systems at all accurate in terms of counting people, visitors, numbers of bedrooms in properties and so on? If not could they be made accurate enough before 2031?
We asked Dave Martin for his thoughts on your question Danny and here is his response:
This is an important question! The main series of official population counts between censuses are the Mid Year Estimates (MYEs). As the 2021 MYE series have not yet been produced, we cannot say exactly how close the 2021 MYEs will be to the census estimate of 59.6m (England and Wales) although we do know that is slightly lower than the 2020 MYEs of 59.7m. MYEs are rolled forward from the previous census using multiple data inputs and the series is then re-based to the census estimates once every 10 years: it looks as though the alternative sources have been tracking basic demographic characteristic well. ONS have published a blog about the 2021 reconciliation and we should know more in November 2022.
It will be longer before we can fully understand the differences between the detailed census topics (due for publication over the coming months) and ONS sources based on administrative data, but these are all being reviewed in detail in preparation for the National Statistician’s 2023 recommendations to Government on the future of the population, migration and social statistics system, including on the future of the census. ONS have been publishing and evaluating experimental data series on topics such as subnational income by ethnicity statistics and admin-based property floor area statistics but the published series do not all relate to census year. They report that the income and ethnicity data look promising, but the current version of the floorspace data is not of sufficient quality e.g., to provide an alternative measure of overcrowding. These are all active strands of work and the detailed census data are only just becoming available for comparison, so we expect to see a lot more on this in the near future.