A lack of funding for departments could curtail their contribution of data to the new cross-government platform for government analysists, national statistician Prof Sir Ian Diamond has told MPs.
Diamond, who is also permanent secretary of the Office for National Statistics – which is leading on the £228.7m Integrated Data Service programme – said work was ongoing with chief data officers to “minimise frictions” related to the new “cloud native” platform.
His observations came in a letter to members of the Public Administration and Constitutional Affairs Committee dated 18 December 2023, but only published last week.
The IDS platform has been created for accredited people – including an estimated 14,000 government analysts – with a view to better informing effective policymaking. It is currently in beta mode, and due to launch fully by 2025.
In his letter to PACAC, Diamond said the platform has been designed address the lack of a central integration platform that can cater for the future needs of both data providers and analysts looking to make use of integrated data to develop cross-cutting analytical results.
He added that the vision for IDS was for it to be a “rich and diverse data catalogue” with “indexed and linkable data, with the latest provision of data science and generative AI potential”.
Diamond said that at the time of writing there were 81 datasets available in the IDS, with “high-value data assets” on policy areas such as levelling up, climate change and net zero, and work on further datasets was under way.
But he also noted that while departments have agreed to open up their essential shared data assets across government, in line with the Central Digital and Data Office’s roadmap for 2022-25, some officials fear political and financial pressures will affect departments’ behaviour.
“Discussions with government analysts have highlighted a range of concerns about how current incentives for departmental data sharing fit with the needs of ministerial-facing departments,” he said.
“There is also a wider financial risk regarding other departments’ ability to fund activity such as data cleansing, which may limit their ability to effectively share data.”
Diamond said that while the Treasury had set out the expectation that government departments would “support data sharing” in all 2021 Spending Review settlements, “no specific funding was provided, which may limit activity in some cases”.
The national statistician said the ONS was working with chief data officers across government to smooth data-sharing concerns related to IDS.
“One of the pilots in development is looking at data ownership and stewardship approaches to streamline the governance arrangements and make it quicker for departments to agree to share data via IDS, and for analysts to subsequently access that data for a broad range of analysis in the public good,” he said.
“I would welcome support from the committee to share and promote the benefits of data sharing across government for the public good.”
Diamond’s letter to PACAC followed an evidence session for the committee’s Transforming the UK’s Evidence Base inquiry in September last year.
The latest edition of CSW contains an interview with ONS chief data officer Fiona James, who talks about the Integrated Data Service against the backdrop of the organisation’s recently-published Data Strategy.