Ever since the launch of the National Data Strategy a year ago, public sector organisations have been moving in a new direction. But building capabilities to unlock the power data has in improving services hasn’t been a small task.
Progress is varied across different civil service departments, and obstacles such as upgrading legacy technology systems, a culture of risk aversion towards data-sharing across departments, and the challenges of implementing a new model mean we’re still not making the best use of data.
To discuss this challenge, Civil Service World and Civica brought together government data specialists at an online round table. The discussion, considering both what has gone well with the strategy and what needs further effort, was structured around three ‘S’s which Civica has identified as key to making the best use of data: standards, skills and sharing.
The event was held under Chatham House terms with participants unnamed other than CSW editor Suzannah Brecknell, who began proceedings by noting that things were at “a really interesting point in the government’s data journey”.
Data specialists had, she suggested, won the argument about the importance and potential of data – the “why data?” – and now needed to focus on actually putting this into practice – the “how?”.
While responsibility for the National Data Strategy lies within the Department for Culture, Media and Sport, it is the Central Digital and Data Office – based in the Cabinet Office – which leads work to improve government use of data.
The meeting heard that there a five key dimensions to this work within the CDDO: quality, availability and access to data; standards and assurance; capability, leadership and culture; accountability and productivity; ethics and public trust.
Standards
The group began by considering where gaps in data standards remain, the extent to which common ones had been accepted and whether it was hard to engage interest in something basic like standards instead of the apparently sexier things like data visualisations and analytics.
Participants felt there were good standards around publication and presentation of statistics but also areas where standards were not yet comprehensively in place, notably around the underlying quality of data that enables interoperability. Sharing was “very much work in progress” and this impeded the delivery of good services.
Sharing
One contributor explained problems that can arise when data is brought together from multiple sources in government and the user must establish whether what appears to be the same person on different data streams is indeed one and the same and not someone else.
He explained: “Inevitably, when organisations collect their own primary data, when you smash those together this problem will surface.” There were ways to overcome this using matching algorithms, but their suitability would depend on whether data was used for research and operational purposes.
Errors in probabilistic matching for research made little difference but “if you mismatch in an operational action it follows you could take wrong action for one of the incorrectly matched people,” according to one participant.
Attendees heard that during the pandemic, there had been a positive move in healthcare towards consistent electronic patient records in hospitals, where old systems had used their own identifiers for patients. Databases were now shared within hospitals and with GPs in a consistent format with unique patient records.
Another speaker, though, said it had taken a year to get all the data for another pandemic-related exercise collected from different bodies, despite a clear imperative from senior levels. The problem was less technical standards than “expectations around behaviour and silo mentality”.
One contributor noted that it can be easier for a department to create a new database than use someone else’s – even where that would do the job – which also discouraged sharing, and there was “massive expertise in data, yet we find difficulty sharing it within government”.
Another participant reported considerable difficulties in data sharing between departments and said: “Some people have very limited experience of sharing data and legal advice can err on the side of caution.
"People don’t want to find themselves in prison, ultimately, by doing something that shouldn’t be allowed, so there is an element of fear and caution.”
In the case noted above, sharing had ultimately happened only by “pulling some heavy levers,” the participant said, “and it should not be the case that we have to go to the most senior people in government to get sharing to happen”.
The meeting heard that the CDDO work included considering non-technical barriers to data use, and this could also cover non-technical standards, such as common codes of behaviour and ethics around data use and sharing.
The understandable desire to protect privacy and act legally can lead to a siloed approach to data ownership, one participant said, with inconsistent decisions and proposals for sharing being unexpectedly turned down. It was suggested that a new model of data ownership – in which data was centrally owned and there were clear cross-government rules about its use – might help to increase sharing where appropriate.
Skills
Moving to discuss the question of skills, some felt there was a problem with the extent to which senior civil servants were comfortable with data.Senior leaders needed “just enough data skills and understanding to ensure resources are allocated sustainably”, one suggested, adding a 5-10-year pipeline of skills was vital.
Appreciation at senior levels of the role of data was essential so “those at the coal face know they will get back up” said one attendee, but another noted that“for anyone who doesn't have the word ‘data’ in job title this is an issue right up to ministerial level”. Senior leaders needed “just enough data skills and understanding to ensure resources are allocated sustainably”, one suggested, adding a 5-10-year pipeline of skills was vital.
Despite good progress at bringing data scientists into government, the group agreed that skills shortages existed, particularly data governance, management and quality, as well as data ‘crunching’, engineering and modelling. Another overlooked skill base was the public, said one attendee, as “there is nobody more motivated to make sure data is accurate than the individuals themselves”.
Discussing whether attitudes to data had shifted as a result of the pandemic, a contributor noted that the use of data in combatting Covid-19 had been “a high water mark, though a lot of people would disagree.
“Politicians and senior leaders, who would not think of themselves as data people, saw the potential but how do we capture these lessons and share that?”
Use of data in the pandemic had transformed public interest and attitudes, another speaker said. “Who would have thought a couple of years ago it would be common to have discussions at the dinner table about uncertainty in forecasting, logarithmic scales and false positives, yet these have entered normal conversations and hopefully we will not sink back.”
Another participant felt putting data in the public domain during the pandemic had demonstrated the future of sharing: “There was increased use of open data and we had the courage to put data in the public domain where it can be scrutinised and verified by experts, and often initial findings are enriched by that.”
There is a danger, though, of the government learning the wrong lessons from the pandemic, according to one participant who worried that the government would think saying a decision was data-based would be enough to secure public support.
The correct lesson, they suggested, was that public acceptance of decisions arose when ministers were open about the data underpinning them, “when they explained to the public why they are making decisions based on data made easily accessible to the public”.
They noted the contrast between the publication of data in the pandemic and the recent fuel shortages. Business secretary Kwasi Kwarteng had told the public “don’t worry, I’ve looked at data and there is enough fuel,” the participant recalled, but had not added “here is the data, you can see it too”.
As the discussion closed, Brecknell invited participants to state one change which would help move this agenda forward. Every participant offered a change to improve sharing – leading the chair to conclude that despite a range of problems, the opportunities of getting data sharing right meant it was the most fruitful avenue for near-term change: “You all jumped on sharing as something that would make a difference,” she said.