How can civil servants unlock the value of government data? Stuart Watson reports from a debate on the benefits and challenges of data sharing. Photos by Paul Heartfield
Government departments and agencies collect and hold a huge volume of data about citizens, organisations and their own operations. That information is used to shape policy and to deliver services – but too often it remains contained within organisational silos, its wider potential left unexploited. Meanwhile, with the civil service traditionally looking upwards to ministers and downwards to individual service delivery channels, opportunities to improve government’s operations by linking up horizontally are constantly being missed.
If departments were better able to analyse, cross-reference, exchange and aggregate their data, they could meet the needs of service users in the round – encouraging web-savvy citizens to receive more services digitally, for example, or spotting danger signals and intervening to prevent disaster rather than mopping up afterwards. A recent CSW round table, supported by analytics provider SAS, brought together leaders from a number of departments and agencies to discuss the obstacles to better use of data, and how to overcome them.
It’s good to share
At the outset, the assembled civil servants exchanged examples of the benefits of the using data more effectively, most of which revolved around the benefits of sharing data within or across organisations. Communicating success stories in this area is one of the principal tasks of the new Centre of Excellence for Information Sharing (CEIS). Its director Stephen Curtis highlighted the Troubled Families initiative, in which local and national public bodies are sharing data to identify and support families with multiple challenges: such data sharing facilitates “earlier intervention, which from a financial perspective takes people out of the high-cost services,” he said.
Sonia Dower, director of interventions and sanctions at the Home Office’s Immigration Enforcement Team, cited two recent initiatives designed to tackle illegal immigration. In one, the Home Office is sharing information with the private sector to deny illegal migrants access to bank accounts; in the other, it’s co-operating with the Driver and Vehicle Licensing Agency to revoke driving licences.
Meanwhile Heather Neate, from the Department for Work & Pensions’ Data Solutions Team, said that her department is sharing data with local authorities and other public sector bodies to help tackle fraud: “It is looking at those people who are not declaring everything they should be, reducing fraud, error and therefore debt across the welfare state,” she said.
Bruce Mann, the Cabinet Office finance director, gave a very practical example of how sharing data can not only improve but also save lives. He said that the Cabinet Office is helping to facilitate the swift sharing of information between social services and the emergency services about very vulnerable people in crisis situations, explaining: “It comes from a rather horrid example where a lady who was immobile drowned in her house during some flooding a few years ago because nobody knew she was there”.
There is value in operational as well as personal data, pointed out SAS UK’s public sector director John Tibble. “There is a vast array of data exploitation and exploration tools out there. Explore that [operational] data and there is a good chance you will find some interesting patterns and anomalies,” he said. As an example, he said SAS is working with a hospital to understand how people’s treatment on admittance affects their eventual clinical outcomes. “They are starting to use the techniques of ‘big data’ to transform patient outcomes, reduce costs and improve efficiency,” he explained.
Obstacles to data sharing
Yet despite the benefits of sharing data, a formidable array of legal and security impediments face organisations wanting to do so. Diana West, from the Ministry of Defence’s Corporate Services Transformation Team, said: “We find it very difficult to share data because of security constraints. It is not only about deciding what we can share, but we have the technical challenge around [deciding] where the IT is secure enough [to share] and where it is not.”
There can also be questions over who owns the information created when data is shared, particularly when non-government organisations are involved. Melissa Crawshay-Williams, a policy adviser from UK Trade and Investment, raised concerns about sharing data with private partners: “We get consultants coming to our office, working on their laptops with our data in our territory, and yet they say it is their document. There are some frictions there,” she said.
Data protection law presents a challenge, particularly in terms of an organisation’s ability to use information for something other than its original purpose. Neate said this is particularly challenging in the DWP’s efforts to tackle fraud, adding that she was hoping to see “data sharing legislation, at least across central government” mentioned in the Queen’s Speech last month. No such legislation appeared, however, and other participants suggested that political constraints mean it could be some time before such a law is enacted.
The Cabinet Office’s Bruce Mann agreed that between the Human Rights Act and domestic legislation on information security, there are “confused masses of guidance.” He suggested going to the Information Commissioner’s Office for a legal opinion on whether data can be shared in particular circumstances, but admitted there’s still a danger of legal challenges if people feel their data has been shared inappropriately.
Yet any awkwardness involved in sharing information must be balanced against the potentially dire consequences of not sharing it, said Andrew Goodman, a data analytics programme director in the Home Office. The establishment of the police national database, he said, was triggered by “the Soham murders of Holly Wells and Jessica Chapman. They died because we didn’t share data, and that’s something we mustn’t forget.”
Money problems
Sharing data involves budgetary challenges as well as legal and procedural ones – collecting, maintaining and sharing data sets costs money, and Tibble raised the issue of how these costs are allocated when more than one organisation benefits: “Although sharing data might be for a wider benefit, the IT and cost of furnishing that is borne by the department providing it and the benefit by the department accepting it, which is a problem,” he argued. He suggested that departments could charge one another for use of their datasets.
“Cross-charging could get very messy,” countered the CEIS’s Curtis: the organisations which receive and analyse data may not be the ones which ultimately benefit from any resultant improvements in decisions, costs or services. “You might have an interest in trying to work out the sort of change which needs to be made [in a service] and you might need some data to help with that,” he said. “You might not be the beneficiary of that change, yet you have paid for the data. Charging has to come into it, but it has to be broader than just the data exchange.”
Mann suggested that in most circumstances an agreement can be reached. “In a cross-departmental, multi-agency project, any finance officer like me would say: ‘Okay, who is getting the benefit? Let’s start to share it.’ It becomes an intelligent conversation about sharing the costs and benefits,” he said.
Data quality
Even if civil servants are confident that data can be shared legally and safely, and agree on who should pay, doubts over the provenance and quality of data, or simple incompatibility between datasets, can make it hard to share information.
“Sometimes organisations are not sufficiently confident in the quality of the data to be able to share it,” said Ofsted’s head of information, Alma Kucera. “In my previous organisation, HMRC, we had a lot of issues around data quality because of the way the systems were updated through various third parties, with a lot of data coming in from different sources – so you don’t know if it is the most recent update or not.”
This demonstrates the importance of holding information about the way data is collected and defined – known as “metadata” – argued Katherine Beard, a policy and engagement executive from Ordnance Survey: “It is important to have good quality metadata alongside the data, so the person using it can understand what assumptions you had to make [when producing it],” she said.
At the Office for National Statistics, the Beyond 2011 team is looking at how to make better use of public sector data in future censuses. Senior researcher Victoria Staples reiterated the importance of metadata in facilitating such re-use. “If we can increase the quality of metadata so everyone is very clear about where the data has come from and what verification and validation checks it has been through, then the onward use of that data will improve because there is an audit trail of quality throughout its use,” she said.
Mann added that data compatibility – or a lack of it – has been one of the main obstacles to government’s attempts to produce a quarterly data summary comparing departments’ performance. “Different data definitions and interpretations get in the way of good analysis and hence good decisions,” he said.
The spectre of Big Brother
Another challenge – illustrated by the delayed care.data scheme, which aims to share anonymised health data – is public concern about the ways in which personal data might be used or abused. The participants agreed that government’s attempts to gather and share information have an image problem, and that good communication will be key to overcoming this. If the public were better educated, said Amanda Gardiner, a public sector specialist from SAS, they would be more engaged and make sure that the information held about them is correct: “There is a general perspective that if the government is asking you for something it’s bad: they want tax off you, or something,” she said.
Participants also argued for transparency and greater public participation to address public fears. “Citizens should own their own data,” said Kucera. “The information should be public and citizens should be able to access it for themselves.” Curtis added that service providers must explain how data will be shared, and why. “People at the frontline should say: ‘I am going to pass this data across to those people, because if I don’t do that they can’t do this.’ We should build that into service design, so we are very transparent and clear with service users about how their data is going to be used.”
The key to persuading people that sharing information is a good thing is to convince them that by doing so they will receive a better service, argued the Home Office’s Goodman. In addition, greater public participation will improve data quality: “There are some real positive messages about why we should share data,” he said. “If we move into a future world where you have an application on your phone so that you put in all your passport details and that gets you through the border quicker, then people will want that service and the quality of that data will probably be very high.”
Driving change
Who will lead the drive toward better exploitation of data? Some favoured more central co-ordination. “When I talk to other government departments, everyone has a different level of willingness or risk aversion, and of resource they are prepared to put into this,” observed immigration specialist Dower. “It would be helpful to bring us all into the same starting position, and I feel that there is a real role for Cabinet Office in doing that. It would be helpful to have that central drive across government that says: ‘This is what you should all be doing’.”
Kucera, however, argued that leadership has to come from within departments: “You always get somebody who pushes and challenges to get the organisation itself to see the benefits of sharing and transparency, and it is that person who should be running it because I think it is driven culturally from the bottom up rather than top down,” she argued.
Curtis argued that data sharing should be built into training and development programmes in order to effect cultural change across the whole civil service, saying that his centre has been considering launching a campaign with a key message of “Information sharing is everyone’s responsibility”.
Even as departments come to realise the importance of using data effectively to improve services, most don’t have enough staff with top-class data analysis skills to do so, said Goodman. However, he said that this is no excuse for inertia: “Can we use our data now? We probably can. Can we get the most out of it now? Probably not, but that shouldn’t stop us. If we say we shouldn’t start this journey because we haven’t got the skills, we’ll never get there. With Cabinet Office, we need to build that cadre of people who have that experience.”
Whatever the complexion of the next government, the drive to reduce public spending looks set to continue into the next parliament, joining other perennial challenges such as serving an ageing population and fostering economic growth. As resources shrink and challenges grow, it will be ever more important that civil servants make better use of their data assets – and that they share them more widely. After all, there are few assets that become more valuable as they’re used by more people. In today’s climate, that characteristic alone makes government’s datasets an asset that must be worked as hard as possible.