Government data visualisation tools are open to gaming and need to be used responsibly by people with the right training, a report from the think-tank Demos has said.
The report, published today looks at ways to improve public services with big data, and focuses on data dashboards, which make large quantities of data, often from multiple sources, available to people.
Organisations use these tools to make everyday decisions, for instance on city planning, or to keep track of important information, such as voting information.
The UK’s Government Digital Service has built more than 800 such data dashboards for different departments, and the recent Government Transformation Strategy makes increased use of government data a priority, and pledged to appoint a cross-government chief data officer.
The Demos report said that dashboards provide a “remarkable opportunity to build more efficient and data-driven services and operations” in the public sector, but also signal a significant shift in the way government works.
Speeding up Whitehall’s digital dashboard
Civil servants still unaware of GDS's Government as a Platform plans – survey
Cabinet Office promises ‘deep transformation’ in long-awaited digital vision
Demos said that an increased use of dashboards will create demand for new skills, the report said, while dashboards by their nature simplify information for easy digestion, which has the “potential to mislead” and presents new challenges for those interpreting the data.
There is also a risk that, “by introducing a new emphasis on metrics, indicators and measures, it can create a greater focus on operational issues rather than longer-term strategic ones,” Demos said.
To make sure governments make the most of dashboards, Demos recommended that they focus on three main areas: defining the dashboard’s purpose, understanding its limitations and training staff.
The report said that it was crucial that dashboards are “carefully designed to match real organisational needs” - without this it will be hard to judge if it is a success or to justify it to citizens who are increasingly interested in data privacy and ownership.
“With the public increasingly concerned about the privacy of their personal data, having a clear purpose for any data dashboard should also guard against 'mission creep' and make sure these large, secure data sets are only used for their original, intended use,” the report said.
When the purpose has been identified, it must be clearly communicated to developers, designers and product managers as well as the intended users, it said, adding that the users need to be engaged from the start of the design process so they buy into the dashboard.
“Poorly conceived dashboards are likely to be rejected by users,” Demos said.
It is also crucial that anyone using the information not only knows the purpose of the dashboard, but also where the data was drawn from and what underlying biases is might have - these biases need to be considered before the data is used in decision-making, Demos said.
“Users can be blinded by large numbers or have insufficient understanding of the strengths and weaknesses of the data they are using,” the report said. “Like all metrics they can be ‘gamed’, and so users must be encouraged to have a critical eye for the dashboard’s limits.”
There is also a risk of performance bias in using dashboards, due to the emphasis they tend to put on performance indicators, and can “pull your data as well as the actions of a team in a specific direction”.
A focus on metrics can lead staff to “prioritise operational issues, rather than longer-term strategic issues, and may marginalise more reflexive approaches to a problem”, it said.
‘Off-the-shelf danger’
The report also emphasised the need for more training for civil servants in data science skills.
“Just because [dashboards] are designed to be user-friendly, it is dangerous to assume users will intuitively understand how to use them,” the report said.
Staff will need to develop a whole new set of skills, including data analytics, design, social science and public policy, to make best use of the data they will soon have access to, Demos said. This includes training in how to manage dashboards.
The report also said that governments will need to think carefully about whether to use an off the shelf programme, from providers like IBM, SAP or Geckoboard, or designing something bespoke.
Although the bespoke option will allow more flexibility and a more tailored design, and are likely to be easier to update and modify, it will be down to the organisation to train people in how to use it.
“Documentation and training information can be less comprehensive or entirely absent and it can be more challenging to maintain these dashboards over time,” the report said.
But off the shelf solutions bring their own issues, with Demos saying that there is a risk of software lock-in, high costs and limitations on the appearance of the dashboard.
The advantages of commercial products often come with training courses and it is generally easier to hand over work to the next person, which is useful for organisations with high staff turnover.
However, Demos also noted that even bespoke dashboards now incorporate pre-existing software, with open source software and free visualisation tools available for low-scale dashboard design.