Over the coming months and years, “radical and ambitious principles” will be the bedrock of the work of the Office for National Statistics, according to the opening sentence of the organisation’s recently published Data Strategy.
This work will all support the ONS’s ambition of “moving to a new phase as both a data and technology leader”, adds the introduction, written by ONS chief data officer Fiona James.
Such grand terminology may come as something of a surprise for many citizens whose primary association with the statistics agency is the experience of filling out and posting back their census questionnaire once a decade.
But, as the strategy explains, a lot has changed at the ONS – and continues to do so.
“There is a trend of falling response rates to surveys, and people are more in control of their data and how they wish to share it,” the intro says. “This strategy sets out a vision for the ONS to pivot towards realising the full value of data as a strategic asset, while maintaining high levels of trust and transparency.”
Since the publication of the ONS’s previous data strategy in 2020, a lot has also changed in the world beyond the organisation, James tells CSW.
“Since then we’ve had a pandemic, the war in Ukraine, an energy crisis, [and] interest rate increases,” she says. “And more recently… we’ve had generative AI releases. This strategy is about recognising that data has become the number-one critical enabler to our operations.”
According James, the plan also recognises that, for an organisation that has always tended to “think a lot about our outputs… [such as] GDP figures, population statistics, or the census”, it is equally significant to reflect on “the essential input” that enables these insights: data.
From this starting point, the strategy sets out eight core missions:
- To enhance data in the ONS and across government by making it more findable, accessible, interoperable and reusable, as well as better linked
- To build a comprehensive understanding of data needs and opportunities
- To maintain high levels of trust in the ONS while raising awareness, understanding, and support for its new data approaches
- To deliver a smooth, commonly understood end-to-end data journey
- To treat data as an essential input to ONS ways of working and central to future operations
- To ensure that quality of data is known, communicated, and managed consistently throughout its full data lifecycle
- To develop secure options that provide business with expanded access to data, maintain public and stakeholder trust, and make available data for statistics and research purposes
- To ensure the ONS will have the skills to meet its current commitments as well as future challenges and opportunities
James explains that the strategy was originally meant to be purely an internal document, but it serves as such an effective statement of intent that the decision was taken to release the plan publicly.
A wider move towards being “outwardly focused” was accelerated by ONS work during the pandemic, which required the organisation to ensure it was “very joined up with a lot of partners [to deliver] the insights that we produced”, she adds.
“The strategy has a really big focus on strengthening our collaboration, particularly with the rest of government and those data communities and users. The Integrated Data Service is positioned front and centre,” James says.
The cloud-based platform, which is currently operating in public beta mode ahead of a full launch of a live service later this year, offers a single unified facility from which accredited analysts – largely within central government, but also including the devolved administrations and some external users – can access data to support research and decision-making.
The Integrated Data Service recently received an accreditation via the Digital Economy Act “which essentially means it’s open for business”, James says.
The IDS forms part of a “new set of data products that we’re delivering, which combine core reference data sources and matching services around people, businesses and locations”, James says, allowing for different data sets to be brought together, linked and correlated and, ultimately, enabling greater insight.
This is exemplified by the ECHILD database, in which the ONS has helped to bring together a wide range of government and NHS data sets to help public sector researchers link information from different sources to explore the relationship between health and education, and how the two impact one another during a child’s early years.
The database is comprised of 20m individual records of children – which have been pseudonymised, with all personally identifiable information stripped out – including details of NHS hospital treatment, maternity, mental health and medicine-dispensing data, as well as pupil databases, census information, and births and deaths records.
University College London, which leads the project, has already used the ECHILD database to study the impact of the pandemic and lockdown measures on children that require additional education support. ECHILD is now available via the ONS’s Secure Research Service, the access platform that is in the process of being replaced by the IDS.
2,000
Amount of data assets held in the ONS’s internal platform – a figure that has increased 50% in the past two years
3 years
Length of time covered by the Data Strategy
£6m
Approximate amount ONS spends on acquiring data each year
20 million
Number of records in the ECHILD database
Local, longitudinal, linked
The ‘three Ls of data’, as set out in the strategy
Costs and benefits
As well as collating public sector data, the strategy also recognises the growing role of external data sources, including commercial providers, and other suppliers of support services to assist the ONS’s work.
James says that the strategy’s ambitions to progress this engagement “are more like an evolution than a step change”.
“I think we recognise that we need to work with external providers to support where we want to get to, with the pace at which things are moving,” she adds.
Some data is provided to the ONS at no cost, on a “value-exchange” basis. This includes information derived from the CHAPS payment database operated by the Bank of England. Other data is acquired by the agency at a cumulative cost of about £6m a year – with further investment also often required to prepare data and put it to use.
The publication of the new Data Strategy comes as government agencies enter the final 12 months of the current three-year spending-review period. As departments work towards a submission for the next comprehensive review, due to take place in late 2024, the ONS is keen to use its growing collaboration with them to ensure plans and programmes involving the use of data are properly costed.
“Departments need to recognise that they don’t just produce data, but they consume data as well, and there are wider benefits,” James says. “For example, we’ve recently been doing something with mobility data – which is quite a high-demand analytical data set – and we’ve been negotiating with a particular commercial provider that could provide us with historical mobility data, as well as real-time data. And this is on behalf of government – not just for ONS use. So, we are contracting with them on a broad basis, rather than each department contracting with the supplier on an individual basis. As we run up to the next spending review, we need to bang the drum for sufficient investment in these data foundations: the data engineering and the data preparation costs need to be factored in properly, by departments and by programmes. If you’ve got a programme and you have a data element, you need to be costing in that data-standard work because, if you don’t, we’re never going to get to a point where it’s easy to interoperate between departments.”
Survey says
The statistical surveys that have long been the mainstay of the ONS’s work, meanwhile, will remain an important source of information, but James says their use will likely become more targeted, with the aim of complementing other data sources.
“The data strategy is aligned to the survey strategy that was produced a few years ago. And that points to surveys being used to address data gaps, where we need additional validation,” James says. “So, [for example], the vision for our business survey going forward – which this strategy supports – is that we have a much more integrated, more modularised design of a smaller number of business surveys. And we now have a business unit that’s supporting engagement with businesses, and we also have conceptual plans to put in place an interface – which would be built in partnership with businesses and accountancy software providers – so people can provide us with data via APIs much quicker than we are able to do now.”
However the data is provided, the strategy sets out the importance of building trust and transparency around it. Three of the eight missions reference the importance of public trust and these objectives will be supported by a public-facing programme of “transparency and listening”, according to James.
“We want to make sure we know what good looks like, and that everyone can find out what data ONS has access to, what we’re using that data for, and that we’re clearly setting out our intentions on data, how we comply with policies, and that we publish lists of all the data sources that we’re using, and how it tracks on to thematic analysis,” she says.
While citizens may feel a greater stake – and desire greater control – in how their data is used, James says that trust and support can be engendered if people feel the relationship with the data processors is mutually beneficial.
“We’ve been doing a lot on this recently around how do we create a much more connected relationship with citizens around how they use their data?,” she adds. “For example, individuals have a great relationship with their bank – because you get something back from the data that you have with them. So [we’re asking] what more can we do in this space, to create a much, much more personalised kind of experience – so you get something back from the data that, at the end of the day, is being used for the greater good.”