GSS Logo

High quality statistics, analysis and advice to help Britain make better decisions.
 

Peer Review

What is peer review? 

Peer review is about asking others to identify improvements you could make to your publications. The feedback provided by participants can then be used to improve the content and presentation of future publications. A quick and easy option is to set up peer reviews in your business area or department. NISRA did this- read how.  Alternatively, you may value an external perspective from across the wider GSS.

Getting an external perspective – why not try a peer review ‘scrum’?

A scrum involves a team of around 6 GSS colleagues coming together to review a statistical publication put forward by a producer. The scrum approach is to discuss the report in detail around aspects such as first impressions, titles, main messages, context, interpretation, language and the use of tables and graphs. Participants provide suggestions how the publication could be changed and sometimes we finish off by mocking up a new front page!

A thorough write up of the session is completed by the scum facilitator and the clearly structured feedback is shared and discussed with the producer. The GSS Good Practice Team  will follow up with the producer at a later date with an expectation that at least some of the suggestions will have been implemented.

Why do it? 

It is quick and easy to set up; it encourages collaborative working across the GSS and can lead to real improvements in the presentation and impact of your statistics.

A scrum offers a fresh perspective on your publication as participants are usually from outside the producing department. They are a great opportunity to receive feedback from other statisticians, make improvements to your statistics release and work collaboratively with other government departments and learn from the best practice of others who may have gone through a similar process.

The scrum process also gives those taking part a great opportunity to spend some time thinking about what works in statistical publications, and how to provide constructive criticism. This can then apply in their own publications and helping others with theirs.

Here are some recent testimonies from colleagues in GSS departments:

“The scrum was really helpful. We knew that the Armed forces continuous attitude survey (AFCAS) report needed a refresh and wasn’t in the best format so having some external feedback was really useful”Nancy Singh, Armed forces continuous attitude survey, MOD

“The scrum provided a number of very practical suggestions on how we could improve our publication’s content and appeal. The majority of the feedback in terms of formatting, presentation, narrative structure etc has been implemented in the latest version” Josh Wallace, FOI Statistics, Cabinet Office

How can I get involved?

The GSS Good Practice Team (GPT) has an ongoing programme of scrum events advertised on this page so please…

  • Contact GPT and put forward a GSS publication that would benefit from a scrum review
  • Volunteer to participate in or even facilitate a scrum event – just email GPT
  • If you’d like a scrum in your department, book a room and we’ll be there (as long as we have enough GSS colleagues volunteering to participate). We’ve been to Bristol, London and Leeds, so will happily come to you.

 

Date
Time
Location
Thursday 11 January 2018
11am-3pm
ONS, 1 Drummond Gate, London
Thursday 1 February 2018
11am-3pm
ONS, 1 Drummond Gate, London
Tuesday 20 March 2018
11am-3pm
ONS, 1 Drummond Gate, London

 


Case Study

Health and Social Care Information Centre (HSCIC) ‘Summary Hospital-level Mortality Indicator (SHMI) – Deaths associated with hospitalisation’ report

One of the first publications to get the scrum treatment in April 2015 was the Health and Social Care Information Centre (HSCIC) ‘Summary Hospital-level Mortality Indicator (SHMI) – Deaths associated with hospitalisation’ report. From the extensive feedback, the key messages from the scrum that the production team picked up on were:

  • The title could be clearer
  • A user needs to get quite far into the report to find out what it’s about
  • What the data can and can’t be used for should be transparent

SHMI one pageThe production team used the mock up front page from the scrum to inspire a one-page summary ‘Deaths following time in hospital’ that was published at the end of July 2015 alongside the standard quarterly report. This is a departure from the standard HSCIC publication template and the producers were keen to get as much feedback as possible about whether this format is useful. The new summary is available at: http://www.hscic.gov.uk/shmi-onepagesummary.

The responsible statistician, Chris Dew, welcomed the feedback from the scrum: “Getting user feedback about publications can be quite difficult. The extensive comments from the scrum really helped us think about what the key information in the report is and how to quickly get this across to users. Hospital mortality is a really interesting topic and the ideas from the scrum will open this up to a wider audience. We also feel we can apply similar thinking to our other publications.”

It’s encouraging that the scrum helped HSCIC to put out a new product so quickly and we look forward to seeing their users’ reaction to this. We think this modified approach has given new energy to the review process and will lead to more rapid and wider change.

We have more scrum events in the future (see timetable above), so let us know if you would like to be part of the pack, or would like your publication to be the ball!