Reproducible Analytical Pipeline (RAP) champions
RAP champions support the implementation of reproducible analytical pipelines across government.
This means promoting reproducible analysis (i.e. analysis with a clear audit trail that explains how and why it was carried out) and the use of reproducible analytical pipelines (i.e. the software methods used to make analysis reproducible).
Champions are expected to share their knowledge and provide advice and support to members of the analytical community who want to learn about and implement reproducible analysis and reproducible analytical pipelines.
Want to be a RAP champion?
Each government department can nominate a RAP champion (or several champions) to work with the network of RAP champions across government. The network is not restricted to the production of official statistics, it is open to anyone in government who is working on, or considering, reproducible analytical pipelines.
If you would like to be a RAP champion please email: gsshelp@statistics.gov.uk
Purpose
The RAP champion network has been set up to support the effective implementation of reproducible analysis across government. By promoting reproducible analysis, champions support the Analysis Function and enable efficiencies.
Role of a champion
RAP champions are expected to:
- help others understand why and how they implemented RAP in their department and why it worked for them
- share code via platforms like GitHub so that others can learn from and adapt what has been done
- make time to respond to feedback
- allow others to shadow their team as they work through a RAP implementation, ideally supporting at least two shadowing opportunities a year
- quantify the benefits of their RAP implementation by keeping track of the time and resources saved by the implementation
- help others understand how to pick a suitable project for RAP development (the RAP Companion has some useful guidance on this)
- promote the work they have done by seeking out opportunities to present their RAP project at events (particularly those RAP projects that are actively delivering efficiencies)
- mentor people new to RAP by giving advice and being a critical friend – champions can get help setting this up by emailing gsshelp@statistics.gov.uk
- peer review at least one RAP project a year
- support the RAP champion community by attending RAP champion meetings, sharing findings and joining in the discussion on the #rap_collaboration channel on the gov-data-science slack workspace
- build their RAP champion role into their personal objectives
- plan for a replacement if they cannot continue their RAP champion role
- keep the contact list of RAP champions updated (champions will be a sent a link to an editable Google document)
- help to build and promote RAP guidance
Membership
The RAP champion network is open to anyone in government who is working on, or considering, reproducible analytical pipelines.
To be a RAP champion, you need to commit to the role outlined on this page. This may require you to get permission from your line manager or Head of Profession.
The network also accepts members who are interested in RAP work, but are not champions. However, wherever possible we would encourage full champion membership in order to grow the community and support reproducible analysis.
Benefits
The RAP champions network is a vibrant, cross-government community. It provides a space for analysts to share their experience, learn from others and support the wider analysis community.
On a personal level, being a RAP champion is a great corporate objective as it is an important “big picture” contribution to the civil service and a significant development opportunity.
Scope
All work on reproducible analysis in government is in scope. Lots of work to date has been done in the context of automating the production of official statistics, but other analysts using reproducible analysis are welcome to join and share their experience.
List of RAP champions (Google doc).
If you can’t access Google docs please let us know by emailing gsshelp@statistics.gov.uk
October 2019
The third RAP meetup took place on Thursday 10th October 2019 at the Office for National Statistics in Pimlico. Here are the presentations from the event.
Introduction and Recap
Martin Ralphs
Revisiting RAP Levels
Joshua Halls, Alexander Newton – Are there additional products we need to communicate the level of RAP?
In this discussion we considered what the core principles of RAP are and how these suggest certain types of tool use. The network agreed that they would comment further on the ideas presented, which propose some changes to the current thinking of the champions.
The DfE Quality Assurance Framework: Re-platforming Models into R
Nicky Brassington (DfE) – How do we perform and communicate quality assurance when re-platforming models?
QA of data science projects
Martin Ralphs – Results from the QA of Data Science survey
QA of code guidance
Joshua Halls presented his work on developing quality assurance guidance for coding, including a link to the GitHub repository for this work.
What do we do now workshop
In this discussion, we revisited the objectives of the network and discussed how we should work together and coordinate going forwards. We recommended that a steering group be set up, drawn from the main network, and that we use the model of Task and Finish groups to address specific requirements. Here are some draft terms of reference for discussion.
We also set out a draft work plan to develop the network and meet its objectives.
May 2019
The second RAP meetup took place on Tuesday 28th May 2019 at the Office for National Statistics in Pimlico, London. Here are the links to the presentations and topics.
RAP Perspectives from NHS Services Scotland
Anna Price, David Caldwell, Jack Hannah
Anna Price – Reproducible Analytical Pipelines in NHS Scotland
David Caldwell – NHS Scotland’s first RAP project
Jack Hannah – Scaling RAP in NHS Scotland
Networking and show and tell
Matt Kerlogue – Using govdown to make a map of civil servants
Towards a minimum viable product for RAP (workshop session)
In this workshop, we considered the minimum requirements for a reproducible pipeline. We looked at four themes: workflow, tools, standards and skills. For each, we listed elements that we thought were essential, nice to have and the pinnacle of good practice. The results of the discussion are summarised in these diagrams.
Champions have already thought a lot about what makes a RAP workflow, and what must be in it. These papers set out our thinking to date.
NHS Services Scotland paper on RAP levels
Duncan Garmonsway – Levels of RAP and getting started
Levels of RAP discussion on the RAP development website
ONS Data Access Platform team RAP Good Practices cheat sheet
Our RAP web presence and how to use it most effectively
Duncan Garmonsway (GDS)
Theodore Manassis (ONS)
Comparing civil services around the world: using RAP principles to process and model messy data
Matt Kerlogue (Cabinet Office)
October 2018
RAP champions got together for the first time on 16th October 2018 at the Office for National Statistics in Pimlico, London. This is what we talked about.
RAP blockers and solutions workshop session
Matt Dray and Matt Gregory, Government Digital Service
This paper summarises what we said in the workshop discussion. It formed the basis of a presentation to GSS Heads of Profession in early December.
RAP in DfE – discovery and how we scale
Laura Selby, Department for Education
and some additional background from DfE.
Seb Fox, Public Health England
Spreadsheet munging strategies
Duncan Garmonsway, Government Digital Service
Max Unsted, Department for Digital, Culture, Media and Sport
Blog posts
The Government Digital Service have published two very useful blog posts about RAP:
Reproducible Analytical Pipelines
Transforming the process of producing official statistics
The Office for Statistics Regulation have also blogged about RAP:
Alex Hayes has a nice blog about software testing in R.
Guides and courses
RAP Companion: a guide to building a reproducible analytical pipeline.
Reproducible Analytical Pipelines (RAP) using R: an online course that goes alongside the RAP Companion. It introduces the key ideas in reproducible analysis and provides links to other resources to support you in implementing RAP.
Choose tools and infrastructure to make better use of your data: Government Digital Service guidance about choosing tools and infrastructure that are flexible, scalable, sustainable and secure.
Communication across the network
Slack channel for RAP champions (#rap_collaboration). You will need to sign up to the gov-data-science workspace on Slack to join this channel. The best way to make use of this channel is to change the channel notification settings to receive notifications for “all new messages”.
RAP resources on GitHub
The network has a dedicated RAP resources site on GitHub and associated GitHub repository that you can contribute to directly. This is a place for the community to publish materials that it finds useful. In particular, resources that include live code can be published on the GitHub site.
If you want to look at some examples of where RAP has already been implemented in government take a look at our list of RAP examples (Google sheet).
If you want to add your own example to the RAP examples spreadsheet (Google form), please do!