Skip to content
GSS > Guidance > Working with users

Working with users

The Government Statistical Service must get input from and adapt to the changing needs of its users (such as government, business and the public) to be sure that our statistics are useful and valuable to society.

This page brings together resources about working with users of statistics, including guidance documents, case studies from GSS departments and links to useful external content.

The Good Practice team within the GSS Professional Support Team is working with UK statistics producers to improve how we work with users. Please get in touch with them for advice, or if you have any examples of good practice to share

StatsUserNet is the Royal Statistical Society’s interactive website for communication between users and producers of official statistics. It’s an ideal tool for reaching users of official statistics and finding out what they want from statistics. Please contact statsusernet@rss.org.uk for advice on how to make best use of the site.

As well as helping us improve how we work with the users of our statistics, we wrote some guidance on Statistics for Policy Professionals: Things you need to know. Take a look and tell us what you think.

 

Case Studies

MHCLG have a developed a dedicated webpage to keep users up to date with all the latest developments in MHCLG’s homelessness statistics.

Ministry for Housing Communities and Local Government (MHCLG) Homelessness Statistics User Forum

In November 2013 DCLG held a statistics user engagement day. It was an opportunity for us to get feedback and views on a wide variety of issues relating to our stats including outputs, methodology, our future work program and departmental statistical priorities. After an introduction session with a keynote from Will Moy of Full Fact the day was split into three lots of three parallel sessions on all areas the department produces statistics on. There were also stalls where users could talk to statisticians and gain further information about what we do within the department.

The small event that got a lot bigger

The idea of a user engagement day came to me out of slight frustration that we couldn’t answer some of the data requests that we were being sent. The questions that were being asked were perfectly reasonable but we just didn’t collect what they were asking for. In my area the forms that collect the data were produced over 16 years ago and although there have been some additions and removals, the form hasn’t changed much since then. Because of this, I thought it would be useful to ask users of our data if what we collect is still relevant and useful to them.

I decided to see if it would be possible to organise an event where we could get information and feedback from our users. Originally it was going to be small scale with a few different teams being involved, but after taking the idea to my Head of Profession, it quickly spiralled into a whole department event!

Organising the event

We managed to get at least one person from every area of the department that produces statistics to join a planning committee. This was where our first problem lay. We needed all these people to be involved but trying to organise meetings where everyone could attend and trying to get everyone to submit ideas for their sessions was not easy. We did get there though. Every person was in charge of organising their own session and feeding back to the organisation committee. We managed to meet every 3-4 weeks and had a shared area so that everyone had access to all documents. There was also a dedicated email address which was monitored by 3 people so that it was less likely that emails would be missed or affected by people on leave.

The other really big challenge we had was trying to predict how many people were going to turn up on the day. The first thing we did was to book all the conference rooms and large meeting rooms in the building but because we had nothing to base it on, we had no idea if on the day they were going to be full to the brim or nearly empty. We got around this by trying to advertise it as widely as possible but monitor numbers attending each session so that the most appropriately sized room could be allocated. We sent out invitations and summaries to all contacts we had that were interested in our stats and we also advertised on various statistics websites and forums including StatsUserNet and Knowledge Hub. Attendees signed up using a Survey Monkey link which allowed us to monitor attendees and stop taking bookings for individual sessions if numbers got too high.

The Big Day

Around 150 people showed up on the day representing local authorities, housing trusts academics charities as well as other government departments which was a lot more than I had ever imagined, and most were very enthusiastic and engaged. The day started with a very interesting and entertaining talk from our keynote speaker Will Moy from Full Fact who highlighted the importance and use of government statistics. Logistically the day was hard. We had to run exactly to schedule so that there was enough time to change rooms around during the break. There was a real buzz about the event, most people were fully engaged and really wanted to get involved in the sessions. The stalls were very popular too.

So what did attendees think?

Following the event we sent out feedback forms to attendees and we got a lot of positive feedback and there were a number of requests for the day to be repeated annually. Obviously, as it was the first time we had done an event like this, there were some suggestions for improvement but the overall response was very good.

“I thought it was a useful event where stats providers and users met in one place to share ideas, comments and provide feedback.” (Performance Analyst, Westminster Council).

“It would be good to have more of these events.” (Contracts, Performance and Partnerships Officer, Eastbourne Council).

“I found the day very engaging and informative.” (National Statistician’s Office).

What did DCLG statisticians think?

The response from DCLG statisticians was very positive. For most of them this had been the first time they had been able to get feedback, question and have discussions with users about their stats in this way. We got a lot of valuable feedback from users which we hope to use to help make improvements to our statistics. A number of statisticians also found it useful to just talk to the attendees to find out how their statistics were being used. We discovered from a social landlord that social lettings data is being used to inform local strategic decision making, for example to assess the impacts of changes to service provision and community investment decisions.

Next steps

We hope to continue to engage users in this way. We plan to either repeat the event as a whole or maybe break it down into a number of smaller themed sessions in future.

Lessons learned

For anyone who plans on doing a similar event these are a few things I would recommend:

  • Have a planning team. Someone needs to take overall charge but have people who can help with various aspects of organising the day.
  • Monitor numbers attending each session and make sure the room you have is big enough. No-one likes being packed into a small space.
  • Be detailed in what is going to be covered in each session. Make it clear who this session is aimed at and what outcomes you hope to achieve from it.
  • If you have budget constraints, be open and honest in advance about what you will be providing and what you won’t.

 

Google Alerts are emails sent to you when Google finds new results that match your search term. They are a simple way to monitor anything on the web. They automatically notify you when new content from news, web, blogs, video and/or discussion groups match your search term.

They can be used to monitor when your statistical publications or key phases linked to your statistics are being used in Google. They are a really good way to find out more information about how the statistics that you produce are being used.

How to set up Google Alerts

Setting up Google alerts is simple and the whole process takes under 5 minutes. Here’s a brief summary of how to do it and an example of how statisticians in NHS England currently use them.

Step by Step Guide

Visit the Google Alerts home page:

GA

Follow these guidelines:

GAInstruc2

Google alerts in practice: NHS England

A senior analytical lead in NHS England Anthony Harris has recently used Google Alerts to understand more about the use of his main Official Statistic publication, the NHS Friends and Family Test (FFT). This is a relatively new high profile release that receives regular media coverage.

Anthony identified the following benefits for his team of using Google Alerts:

  •  Access to local media coverage they’d never normally see. This enables them to spot common misunderstandings of the statistics and adjust their publication content / monthly media briefings accordingly.
  • Fast media monitoring. The traditional route for media monitoring is slow (external media monitoring alert to press team, who contact a policy team who then speak to an analyst) and this can result in “we need you to check the numbers in the next 5 minutes so we can send a rebuttal”. Google alerts can allow the analysts / statisticians to have an early warning and more time to give a considered response.
  •  It’s encouraging to see the full use of the data on which they’re working. Alerts help provide a way to show the true amount of use of their statistics.

Overall Anthony says…

“For a new statistic, it has been really interesting to see the full breadth of the online media coverage and be able to learn from it”.

Two words of warning:

1)    There are limitations to the Google algorithms as they are essentially a text search. For example, Anthony occasionally gets irrelevant articles flagged e.g. “Christmas is one of those times when friends and family test your patience”. Although these are quite rare, they will obviously vary depending on how many common alternative uses there are for the search phrase.

2)    Acronyms can be problematic. “FFT” is also used by “Fantasy Football Today”, an American football show on the US TV network CBS and this can cause havoc with Anthony’s monitoring! Although this is less of a problem for Google Alerts (where newspapers normally write out the title in full) it does affect Twitter monitoring. To get around the problem NHS England encourage the use of #NHSFFT rather than #FFT, and this has been successful in moving a lot of the online discussion to a hashtag which is about hospitals rather than touchdowns.

The Health and Safety Executive (HSE) has had a statistics e-bulletin service running since May 2006. The main aim is to keep statistics users up to date with the latest news, statistical releases and other useful information. Subscribers receive a bulletin approximately every two months by email. Typical bulletins will have three or four points of interest with links to further information.

We currently have over 30,000 subscribers and the bulletin has been a huge help to us in reaching our customer base and gaining a better understanding of customer needs. For example we used the bulletin community in spring 2011 when we ran a public consultation to establish how HSE’s data is used and the impact on users if we were to change the frequency or availability of our statistics. This consultation received 558 responses which formed a major part of the decision making process when determining funding of future data sources. The results were also presented to the HSE Board. More recently we have also used the bulletin to promote a statistics user conference which we held at our offices in Liverpool.

Producing the bulletin places a very small burden on the stats team. It is managed by an EO who co-ordinates individual content with our Grade 7 statistician leads.  Each bulletin takes an hour or two to develop. In setting the bulletin up, we did have the advantage that HSE was establishing e-bulletin services more generally.  Whenever anyone subscribes to a bulletin, they are given a list of other HSE bulletins they may be interested in. This was hugely beneficial to us in growing the original audience for the statistics e-bulletin.

HSE hosted a statistics user conference for external users at our Liverpool offices.  We used our e-bulletin community to promote the event and also asked policy colleagues to forward information to their key external stakeholders. The maximum number we could accommodate was 65 and we were over-subscribed. These were a mix of industry representatives, trade union reps, people from professional associations, academics, trainers and consultants.

Prior to the event we asked attendees for their key questions and also what their issues and difficulties were with our data. We used this both to pitch the agenda and also to prepare for any quiet periods if discussion was slow (which didn’t happen!).  The day was a mix of lecture sessions and discussion/Q&A opportunities. We opened up with an introductory session about our data sources and a tour around our website. In the afternoon, we had a theme of “benchmarking” as we know from previous consultations that a lot of our users want to benchmark their own health and safety data against the national statistics for their industry. We invited two external users to be part of that session and they gave 15 minute presentations.

At lunchtime we had a poster presentation so we could show some of the ad hoc analyses which have been done in-house and the relevant statistician was available for questions. We also had a couple of laptops set up so we could demonstrate how to navigate the stats website and how to use our online tabulation tool.

Overall, the event was a great success. Some 95% of attendees said they would attend a future event and 97% said they would recommend to a colleague. We also had some really positive comments, both on the feedback forms and face to face.  Although it was nerve-wracking beforehand not knowing what to expect, my advice to other stats teams would be give it a go and you may be very pleasantly surprised.

Can we make official health statistics better? This was the question posed to attendees at an event held by NHS National Services Scotland and the UK Statistics Authority in June 2014.

The aim of the day was to better understand user needs to help inform improvements in quality, value, accessibility and impact of information. The event was open to anybody with an interest in health statistics and anyone that couldn’t come along could follow the event on Twitter.

An interactive day with a variety of presentations and workshops:

Can we make official health statistics better?

10 Page Summary Report

NHS National Services Scotland published a 10 page summary report on the event which gives much more information about what was learned, how it will be used and advice for running future engagement events.

The day involved a mix of presentations and workshops, with attendees being asked for their views on how they use statistics and how they would like them communicated. An illustrator was on hand to capture the themes of the day – see these below!

The illustrations were used to spark discussion and to help identify the most important messages from the event.

One of the key messages was that one size does not fit all.

There are some users that want the data as soon as possible and others who would rather wait for the analysis. Some are happy to deal with provisional data, whilst others prefer to wait until it has been fully quality assured.

Some key messages from the day:test pic 3

  • One size does not fit all – Producers need to consider different ways of releasing information to suit a variety of user needs.
  • Users would value having indicators showing whether data were to be used with confidence (green) or caution (amber).
  • A need for more interpretation of the information we publish to help users understand what the statistics are telling them.
  • There was a clear appreciation of, and confidence in, the statistics produced by NHS National Services Scotland.

 

 An illustrators point of view of the key messages

pic4

 

The beginning…

The Office for National Statistics (ONS) has well established engagement frameworks with traditional stakeholders such as other government departments and businesses.

However, ‘younger users’ was one of the areas we wanted to improve our engagement in as well as improving our capability by encouraging more undergraduates leaving university to consider ONS as a potential employer.

So how do we do this?

In 2013 we launched the ONS Undergraduate Working Group on Economic Statistics. The planning for this group took place in the summer of 2013, with the first meeting in November 2013.

Sounds easy? Well it wasn’t as difficult as splitting the atom but there were a few challenges along the way!

Some key lessons learnt were:

CTR1         CTR2         CTR3         CTR4

Ok, it was a bit challenging but what does the group look like now?

Three universities are involved – University of the West of England, Cardiff University and Swansea University. 24 students are taking part, mainly 2nd and 3rd years but a couple of 1st years too either studying economics or maths.

uni pic2

The remit for the group is:

  • How to present statistics: commentary, products etc.
  • Measurement methods and approaches for key economic statistics: education on current approach and seeking ideas for improvements
  • How economic and statistical theory is turned into reality: assumptions that have to be made – are they sound or can they be improved?

There are three meetings per academic year, which are hosted at the ONS. The group are split into teams to take on projects between meetings and each team are assigned an ONS mentor for guidance and support. The teams look at some of the projects the ONS are working on and put forward recommendations to help improve work output. This approach has been successful by keeping undergraduates interested, help develop relationships, whilst gaining experience in delivering projects and for them to have tangible outcomes of their efforts and clear sight of how they have made a difference.

An important mission…

CTR5This is one of our case studies showing our involvement with the ONS.

The project involved getting the undergraduates together to examine a publication and offering feedback, whilst working alongside the Good Practice Team for guidance and support.

Check out our case study to the left to see how the project was deemed a success!

 

Is it all worth it? Absolutely!

We now have:

  • Regular access to an important segment of our user base with fresh perspective and ideas.
  • Raised the profile of ONS as a potential employer; already, some members of the group will joining ONS on 1 year placements as part of their university course
  • The undergraduates are receiving unique access, insight and involvement in official statistics

How to find out more…

If you are interested to find out more, or would like some support in creating your own undergraduate group then please contact the Good Practice Team.

The Welsh Government recently held a webinar about the Welsh Index of Multiple Deprivation (WIMD) consultation. WIMD is the Welsh Government’s official measure of relative multiple deprivation for small areas in Wales and in November 2013 we launched a consultation on proposed indicators for WIMD 2014. Prior to WIMD 2008, we held consultation events in north and south Wales. These events were an opportunity to explain to users what was being proposed and the reasons for selecting specific indicators. However, events can be costly and resource-intensive.

So on this occasion, we opted to hold a webinar instead!

What are webinars and how do they work?

  • A webinar is an online seminar. They are easy to set up and easy to join.
  • We used GoToWebinar. As many as 100 people can join to watch the webinar.
  • No specialist software is needed for the host or attendees – you just need a laptop/computer/tablet with internet access and a phone. There is also an option of using a webcam if you wish.
  • The attendees can see the slides of the presentation and hear the voice of the host live, without moving from their desk! They can also ask questions, either by typing them or asking them verbally.
  • The session can be recorded – this enables you to circulate the presentation with commentary to anyone who could not attend.

 How it went

We held two Webinars and both went smoothly with minimal technical issues. There were 22 attendees in total. In both webinars we found that people typed their questions rather than ask verbally – perhaps this was because most people had prepared their questions in advance. However, this did mean that the webinar didn’t feel as interactive as a meeting.

We found it helpful to sign in to the webinar as a guest on an additional tablet. This meant that we could see exactly what other webinar attendees saw and could check that everything worked as it should.

During the webinar you have the option of running interactive polls. To gauge how the attendees felt about the Webinar, we asked “Was the Webinar sufficient for your needs?” – 100% of those who answered selected ‘Yes’.

Lessons learned

Webinars are not a replacement for face-to-face user meetings and presentations – we still held presentations for some key users groups. However, for stakeholders who have an interest in learning more about a specific topic, within a limited amount of time and without having to leave their office, webinars are ideal.

We definitely recommend you give webinars a go – they are easy, informative and could save valuable time and resources.