Modernising a survey which has been running for over 50 years
The National Travel Survey (NTS) is considered a world-leading and gold-standard travel survey, with governments from many other countries using the NTS as a template for their own travel surveys. It has been running for over 50 years which is great as it captures long term trends, but modern travel trends are evolving fast (Uber and ebikes weren’t around when the NTS began for example). We have therefore been making some fundamental changes to the survey to ensure it can capture these modern trends and maintain relevance to meet the user needs of today.
As anybody working in Government statistics knows, it can be challenging to find the time to make big improvements on top of completing the usual business-as-usual tasks of managing a survey and producing statistical releases and analyses. We had lots of ideas, but to make it possible we needed to overhaul the way we usually do things. Our most time consuming task by far was processing our data for analysis and publication as this required a lot of manual work, so we found ways to automate large parts of the data production process using a combination of advanced Excel, SQL and R. Increasing levels of automation, whilst taking some time to set up, greatly freed up resources and allowed us to focus on making long term improvements. We also adapted agile project management strategies which allowed us to work much more flexibly and quickly.
Thinking creatively at how to better meet user needs has also played a key role. For example we have set up a NTS Panel (the “National Travel Attitudes Survey”), consisting of NTS participants who agree to be contacted for follow-up research. The panel gives us greater flexibility and granularity and can deliver evidence on a much more timely basis as opposed to just adding more questions to the standard survey. The panel can target a sub-section of the population (e.g. people who cycle) where it would be disproportionately burdensome to ask everyone in the full NTS. Panel responses can also be linked back to original NTS responses, to greatly enhance the utility of the data.
Making improvements has led to a snowball effect as with more efficiency we are able to make even more improvements. Other innovations and improvements we have made in the last year, which has been highlighted in a case study for the Code of Practice for Statistics, include:
- improving the NTS questionnaire following a feedback exercise, to check the relevance of NTS questions and the burden placed on respondent: questions undergo extensive cognitive and panel testing to ensure participants understand them and that they collect the data users want
- collaborating with other analysts, including those outside of Government, to produce NTS analytical reports, demonstrating the breadth of information available in the NTS.
- making the dataset accessible via the UK Data Service, and the ONS Secure Research Service, so that far more analysis can be undertaken than could be done by the NTS team alone
- conducting advance letter and incentive experiments investigating how to boost response rates
- Methodological improvements) to collect walking data more accurately
- conducting a Discovery to explore whether developing a digital NTS diary could reduce respondent burden and increase data quality
- designing interactive tables and revising the data table categories so that it is easier for users to find the data they are searching for on GOV.UK
- publishing ad-hoc analyses, so they are accessible to all and enable the reuse of NTS data
- using R Studio to provide regular standard errors and confidence intervals for NTS statistics and ad-hoc analyses
- producing a user-friendly quality report to inform users about the quality of the NTS data, including sampling, methodology, quality assurance procedures and confidentiality
NatCen, the survey contractor, have played a crucial role in making these improvements happen, particularly with conducting the incentive experiments and cognitive testing exercise. The interviewers conduct the survey to high standard and their feedback has also helped us identify where improvements can be made.
The exciting part is that this all feels like just the beginning as these changes have really increased engagement with our users and we have many more plans ahead to keep improving and make sure it maintains its status as a world leading travel survey.