A new resource from the Centre for Public Impact rates the effectiveness of policy initiatives from around the world. Mark Smulian reports
Henry Ford famously knew that half the advertising budget for his cars was wasted, but not which half. Civil servants might know that a policy worked – but not why – or that money spent on an initiative was wasted, but not the causes of failure.
Enter the Centre for Public Impact, not yet another think tank proposing initiatives, but something offering a different way to assess whether and why past actions worked. It is a global not-for-profit foundation, with a London-based team, whose main funder is management consultancy The Boston Consulting Group.
Last month it launched an “Observatory” – a free, online database of some 200 government initiatives from around the world that can be interrogated by geography or policy area.
Are the What Works Centres working – or is evidence-based policymaking still a pipe dream?
Public sector financial resilience: a webinar on doing more with less
The idea is to examine their consequences by reference to the CPI’s new framework for public impact – the Public Impact Fundamentals – which are a culmination of the CPI’s work with academics and practitioners such as Ngaire Woods of the Blavatnik School of Government and Mark Moore from the Harvard Kennedy School of Government.
When examining the case studies, three key things stood out as essential to achieving public impact: legitimacy, policy and action. In this model, “legitimacy” is the underlying support for any government initiative, namely, the level of public confidence, stakeholder engagement and political commitment it enjoyed. The “policy” criteria, meanwhile, concern the clarity of objectives, strength of evidence used and the idea’s feasibility, while “action” examines implementation and whether management, measurement and alignment with wider policies were satisfactory.
CPI executive director Adrian Brown – a former senior official who was once part of the Prime Minister’s Delivery Unit – says the Observatory was established because “we had done a lot of engagement with governments and stakeholders and they said they could not easily find real-world examples of how people had tackled policy areas around the world”.
"We wanted to show how things worked"
Civil servants may feel a self-congratulatory glow as they peruse the UK examples, where it would appear that Whitehall has scarcely ever put a foot wrong, with a profusion of “Strong” and “Good” ratings and a distinct lack of “Weak” ones. Not quite. According to Brown: “Our initial decision was to look for examples where there had been an impact. We wanted to show how things worked, so there are no complete disasters in there. But we will look at those in future. The UK examples are therefore not any comment on the civil service.”
“Civil servants often move around and there can be a loss of institutional memory in departments" – Adrian Brown, the Centre For Public Impact
A number of the case studies are distinctly historic – including, for example, utility privatisations from the Thatcher era – but Brown says this is intentional since “it takes time to judge the impact of a policy”. He adds: “Civil servants often move around and there can be a loss of institutional memory in departments; people are aware that a policy happened but the details of why it worked or didn’t may not be known – but they can find those here.”
The CPI also hopes that its work will help civil servants and politicians to judge evidence. There can sometimes be a rush to cite “evidence-based” as a justification in itself for some course of action, but Brown says the examples on the Observatory will show that while evidence is important, it is just one factor among several.
“We’ve worked with the Organisation for Economic Co-operation and Development on using evidence in policy, and evidence-based policy is not the sole factor to consider, as it is rare that evidence is black and white,” he says.
For the CPI, improving public impact is one of the greatest challenges of the 21st century. The foundation defines public impact as being “what governments achieve for their citizens”. This can vary, it says, and if results fail to match intentions “this wastes time and money, and damages public trust in leaders and democracy”.
When the Observatory was launched, senior research associate Joel Tito wrote that while no government sets out to fail, policies designed to make a positive difference often struggle to turn a good idea into real-life progress.
He concluded this was because of inevitably fruitless quests for “magic bullets”. In reality, the combined complexities of running departmental headquarters and organising service delivery on the frontline meant there were no “cut and paste” formulas, and governments had to adapt to different backgrounds, systems and cultures.
The case studies
Observatory case studies are grouped by world regions and by eight themes: city/urban; economic/finance;education; energy/environment; health; infrastructure; justice; technology. Each is rated “Strong”, “Good”, “Fair” or “Weak” against – in most cases – nine criteria: stakeholder engagement, political commitment, public confidence, clear objectives, evidence, feasibility, management, measurement and alignment.
Prime Minister’s Delivery Unit (UK)
This was created by Tony Blair in 2001, and led by Sir Michael Barber (pictured), who is now the CPI’s co-chair. The unit – abolished by the coalition government in 2010 – is described by the Observatory as having been set up after other measures to reform public services had proven ineffective and Blair “had become frustrated with the lack of progress made during his first term and wanted to use his second term to push through radical public service reform”.
The unit was part of the Cabinet Office and charged with bringing fundamental changes to delivery in the Home Office, and departments of health, education and transport. By 2005, as result of the unit’s work, most of Blair’s domestic policy priorities “had either been delivered or were heading firmly in the right direction”, the Observatory says.
In the CPI’s commentary on the unit’s work, Barber describes it as: “A safe space that kept all the forces at bay that can distract you from delivery.” The Observatory gives it “Strong” ratings for stakeholder engagement and political commitment and “Good” for all other criteria, except for evidence and alignment – both judged “Fair”.
Having Blair’s authority behind it gave the unit power and influence, even though the Treasury was sceptical and feared it would “impair the Treasury’s own implementation” of public service reforms. It became “an innovative and pioneering institution…instrumental in the ‘deliverology’ approach and is now seen as a successful major global innovation,” the Observatory concludes.
It was Blair, though, who later inadvertently weakened the unit. The Observatory notes that his support faded in 2003-04 when “the prime minister changed the focus of his public agenda and reduced his level of involvement, being distracted by the Iraq war”.
It says: “As soon as his involvement dropped off and there was less expectation of a prime ministerial stocktake, PMDU struggled at times to broker action with departments.”
Charging for Plastic Bags (England)
This recent initiative scored “Strong” or “Good” on every criteria. The charge was introduced following lengthy lobbying by environmental campaigners, who saw discarded bags a threat to wildlife and the ecosystem. Scotland, Wales and Northern Ireland introduced charges first, with England following in 2015. Data from waste-reduction body Wrap showed that 8.5 billion single-use bags were used in 2014, a 2.3% increase in the previous year.
Imposition of the 5p charge aimed to reduce usage by 80% in supermarkets and 50% in other shops. The charge is not a tax and retailers are encouraged to donate proceeds to charity, something found to increase the sense of cooperation between the consumers and retailers.
The Observatory says that in the charge’s first six months bag usage slumped by around 80% and remained static thereafter, with both consumers and retailers largely supportive. Relatively simple objectives – and clear evidence of a drastic reduction in use in other countries – helped to shape its implementation.
Work Programme (UK)
Launched in 2011 to reduce the amount paid in unemployment benefits by moving long-term unemployed people into sustained work, the controversial Work Programme is a mixed bag, the Observatory finds.
It introduced a payment-by-results approach and was expected to cater for 2.1 million people between June 2011 and March 2016, at a total cost of £2.8bn. But there was doubt over how well the contractors delivering the programme performed, and allegations that improper claims were made.
By July 2014, the National Audit Office said the programme had saved £21m “by extrapolating invalid job outcome payments and reducing payments to prime contractors accordingly”. The Observatory considers the programme “Strong” or “Good” on most grounds, but only “Fair” for evidence, feasibility and alignment.
As late as 2015 there was still a lack of evidence of the success of the payment-by-results approach, with the Observatory highlighting the NAO’s comment: “These contracts are hard to get right, which generates risk and cost for commissioners.” The NAO went on to say that payment-by-results offered the potential of innovative solutions to intractable problems but the associated extra risk “requires credible evidence [and] without such evidence, commissioners may be using this mechanism in circumstances to which it is ill-suited, to the detriment of value for money”.
Reforming the policy management system (Latvia)
After independence from the Soviet Union in 1991, Latvia needed to reform its civil service and policy design and implementation. Until 2000, the Latvian government functioned without a clearly organised policy planning process, the Observatory finds, and ministries produced policy papers lacking input from stakeholders and data on costs, leaving decision-makers “in the dark when trying to set a course for Latvia’s future”.
The Policy Coordination Department (PCD) was formed in 2000 to encourage strategic planning and coordination to ensure senior officials had the information and analysis needed to make informed decisions. This gained ministers’ approval because its proposed approach was “cooperative rather than confrontational”.
PCD set up pilot programmes to check the feasibility of rules proposed by the Latvian government for the processes of policymaking, coordinating activities across different ministries and linking policy planning to the budget process.
However these reforms fell foul of the 2008 financial crash and a new administration closed the PCD, which had been dependent on the support of specific politicians in an unstable political climate.
This was despite a World Bank study having ranked Latvia high among central and eastern European countries in a survey of the quality of policy analysis and coordination.
In its conclusions, the Observatory says while the PCD enjoyed good initial political commitment and fair stakeholder engagement, it had only weak public confidence.
Its objectives, evidence and feasibility are all ranked “Good”, but it is rated only “fair” for its actions.
The rise of electric vehicles (Norway)
The Norwegian government scored a series of “Strong” and “Good” ratings for its encouragement of electric vehicles. This is part of a programme to reduce emissions from new private vehicles, with a target that these should average 85g of carbon dioxide per kilometre by 2020.
A combination of research, financial incentives and public awareness campaigns has seen some 70,000 electric vehicles take to Norwegian roads, and 20% of new vehicles are fully electric. Norway now has the highest proportion of electric cars in its overall fleet of any country, although there is still a long way to go as these account for only 2% of the total.
Energy companies in 2009 presented an action plan for electrification of road transport, which assumed a 10% share for electric vehicles and part-electric hybrids in the passenger car fleet by 2020.
Implementing this plan, charging points were made available, taxes were cut for electric vehicles, drivers were offered test drives of these unfamiliar cars and city and regional governments exempted electric cars from fees and tolls, even though this cut their revenue. The Observatory says the rapid uptake of electric vehicles in Norway showed public faith in the objectives being pursued, though the incentives had been needed to spur acceptance.
Building a world-beating e-government (South Korea)
In 2001, South Korea passed the Promotion of Digitalisation of Administrative Work for E-Government Realisation Act and has since developed what the UN has judged the world’s most advanced e-government services and levels of public participation.
The Act gave legislative force to initiatives to improve the productivity, transparency, and social equality of administrative institutions. Plans were made to develop integrated portals and communication centres to encourage public participation so e-government would reduce digital divides and obstacles.
Services have increased in efficiency as the e-government plan has progressed, and the Observatory rates the programme “Strong” or “Good” on all applicable counts. Stakeholders were heavily engaged in the process both within and outside government, and, the Observatory notes, the programme benefitted from consistent and measurable objectives over a long period.
South Korea developed electronically-networked relationships for government-to-government work, government-to-business – including a single and transparent procurement system – and government-to-citizens. The Observatory hails good alignment between the government and the other stakeholders throughout these networks.
Learn more at www.centreforpublicimpact.org/observatory