Skip to content
GSS > News > Introducing the Administrative Data Methods Research Programme

Introducing the Administrative Data Methods Research Programme

Who we are: a new programme led by ONS Methodology Division.

Across the GSS we are using more and more administrative data within our official statistics. As part of ONS transformation, we have ambitious plans to put administrative data at the core of our statistical production and are considerably increasing their use. Unlike survey data, however, there is not yet a well-established and agreed statistical theory to underpin the use of administrative data in official statistics. The purpose of the Administrative Data Methods Research Programme is to to develop a statistical framework to answer the multidimensional challenges of using administrative data to produce official statistics.

Our aims are to: 

  • Address the key methodological and statistical challenges of using administrative and transactional data
  • Collaborate on cross cutting methodological challenges across Government Statistical Service (GSS), academia and internationally
  • Create a framework of methods to effectively use administrative data and maximising its potential
  • Catalogue and share best practice
  • Ensure the Administrative Data Methods Research Programme has a clear impact and application on official statistics within the ONS, GSS and beyond

This will help to ensure robustness of statistical outputs and put administrative data at the core of statistics within ONS.

To achieve these ambitious aims, we will work collaboratively across government, academia and other research organisations. We recognise that there is already existing research in these topics. Thorough research will be undertaken to ensure we are building on existing knowledge.

This is a long-term investment programme, with projects started in May 2019. The programme is in line with The UK Statistics Authority strategy which says:

“we must develop our capability to integrate administrative and commercial data sources, supported by appropriate methods and standards”

UK Statistics Authority (2015-2020) Better Statistics, Better Decisions

Administrative data can be defined as data which are initially collected for some administrative purpose—to run an organisation, such as a company, government, charity, school, hospital, and so on. The Administrative Data Methods Research Programme includes all sources of non-survey data including transactional data. The UK Statistics Authority has been strongly encouraging public bodies to fully utilise administrative data for statistical purposes. The 2012 Administrative Data Taskforce report stated that:

“Routinely collected administrative data are a rich and largely untapped source of information for research and policy evaluation in the UK, the value of which continues to appreciate over time”

Administrative Data Taskforce Report, December 2012

 The use of administrative data sources in the production of official statistics has grown substantially over recent years. This has brought many advantages such as reduced costs of production and reduced burden on respondents. It has also allowed for the potential for statistics to be produced more frequently and in greater detail. Increasing the use of administrative data also allows greater use of new data sources, providing valuable information for policy makers make better decisions and enabling the public to better hold organisations that spend public money to account. Rapid changes in technology or ‘the data revolution’ mean that there are more sources of data available than ever before. The ONS strategy for UK statistics states that;

“we must develop our capability to integrate administrative and commercial data sources, supported by appropriate methods and standards”

UK Statistics Authority (2015-2020) Better Statistics, Better Decisions

To make effective use of administrative and transactional data, we aim to develop a range of peer reviewed robust methods. These will be available to provide best practice guidance to be shared across the statistical community. This is key for maintaining the credibility of official statistics.

Although administrative data has been used in the production of official statistics for many years, the use and emphasis of these data have grown substantially as part of our ONS plan to put administrative data at the core of our statistical systems.

HM Revenue and Customs VAT turnover data has recently been introduced directly into the ONS National Accounts, and the next Census in 2021 will make increased use of administrative data to enhance statistics and improve statistics in between censuses (Annual assessment of ONS’s progress on the Administrative Data Census: July 2018 ).

David Hand in his 2018 paper explains that there are a range of statistical challenges associated with administrative data which need to be investigated including:

  • data quality issues,
  • changes in definitions between survey and administrative data,
  • determining accuracy and validity of research conclusions,
  • the development of principles and robust methods for linkage.

In response to David Hand’s paper the GSS Methodological Advisory Committee (GSS MAC) held an extraordinary general meeting (EGM) to discuss how to take the challenges presented forward. Following the EGM, ONS carried out an evaluation into current methodological work into non-survey data and completed an initial high-level review of published research into this topic. The Administrative Data Methods Research Programme was initiated to take this important work forward.

We want to approach this work in a collaborative way and plan to work with experts in the use of administrative data from academia, across the GSS and beyond.

Some of our key research projects include:

Quality frameworks and communicating uncertainty

 This project will investigate all potential causes of uncertainty in source data, including processing methods and linking as well as uncertainty caused by integrating different data sources. The project will focus on how to communicate and present uncertainty in non-survey sources, for example, what methods or visualisations are suitable to support and communicate uncertainty in Official Statistics to policy makers and researchers?

Devising a suite of exploratory analysis

The aim of this project is to produce a standard suite of exploratory analysis to be used by researchers as a check list when working with non-survey sources. This will help researchers carry out a full assessment of both aggregate and record level data. For example, a measure on how to compare a variable from an administrative source to a similar/equivalent variable from survey.  This suite of exploratory analysis will provide greater consistency and improve the efficiency of exploratory analysis.

Creation of a framework for dealing with under and over coverage error

 The aim of this project is to investigate over and under coverage error in a given dataset and devise a methodological approach to addressing these. This project will utilise an administrative data source to measure and show coverage error for different uses, and how this could change over time.

 Other projects/research areas include:

  •  linking multiple data sources
  • guidance on generating synthetic data
  • developing tools for data linkage
  • privacy and disclosure control

and many more…

Commissioned research projects will:

  • have high impact
  • inform the statistical design and methodology for incorporating administrative data into our statistical systems
  • help to build a sound methodological framework
  • have practical application
  • be applicable to a range of solutions
  • be peer reviewed

 Please contact: Admin.Data.Methods@ons.gov.uk if you:

  • would like any further information on the topics or projects we would like to commission or collaborate on;
  • would like to lead/bid for any of these work streams;
  • are already carrying out relevant research;
  • have any related knowledge of expertise you would like to share;

Look out for regular updates and progress reports which will be available on the GSS website.