The word agile is not usually associated with the civil service. Despite the fact that it adapts rapidly to overnight reshuffles and changing policy priorities, the civil service makes little use of agile project management, which adapts projects in response to changes in the environment and customer feedback. Universal Credit is a high-profile example of where agile management was tried, but was not successful – at least in its first phase.
An agile approach can be contrasted with one-off development of requirements and a plan, followed by implementation in sequence, known in management jargon as the "waterfall model". This model has the virtue of simplicity. Waterfalls, however, cascade in one direction. So although the environment and customer needs often change during a project, this model doesn’t encourage iterations to check that what customers thought they wanted at the start of a project is still relevant.
While experienced project managers can work with the formal systems that a waterfall model establishes, my own experience is that agile projects achieve a higher degree of commitment because people are not asked to ignore the uncertainties inherent in complex projects and fix too much too early – the interactions with customers can be challenging, but they keep the project relevant.
The agile model is well established in the technology industry, because it is relatively easy to rewrite code, and developments like the cloud and app stores allow prototypes to be rapidly and cheaply developed and tested with customers. The Institute for Government has long supported an agile approach in government information technology, and the Government Digital Service (GDS) uses and encourages it. For example, GDS and the Department for Work and Pensions (DWP) have redesigned the Carer’s Allowance, releasing a beta version more than year before the full launch, and adapting it based on feedback. The result is that user satisfaction rates are around 90%.
The Department for Work and Pensions’ Universal Credit programme, which aims to merge several benefits and tax credits, was initially badged as agile – a first for the department on this scale. The NAO, however, reported in 2013 that DWP “experienced problems incorporating the agile approach into existing contracts, governance and assurance structures". Universal Credit still has agile elements, and the Major Projects Authority has just rated it Amber/Red, meaning that “successful delivery of the project is in doubt".
The NAO points to a wider issue for government. If digital development is agile, but the processes that surround that development are based on the waterfall model, projects are more likely to fail. It’s my belief that, given that most government projects are complex, it makes sense to apply agile project management more widely outside the digital world. This has been recognised in some parts of government, including the Behavioural Insights and Open Policy Making teams. But government has not become agile by default. Why is this?
The first reason is that the waterfall approach is embedded in the practice, culture, and language of government. Projects use engineering terms – bottlenecks, levers, and gateways – which create an air of reassuring solidity, rather than reflecting the fluidity and unpredictability of most of government. While the Treasury’s guidance for evaluating digital projects has been amended to encourage a more agile approach, for other projects (including those covered by the Major Projects Authority), the guidance continues to reflect a waterfall model, as does PRINCE2, which remains the standard methodology in Whitehall.
There are particular constraints on government which limit the scope for an agile approach. One is the need for repeated authorisation of projects years in advance by parliament, the Treasury and other bodies. These authorisations often specify in detail how a project should be implemented. For example, the Crossrail Act of 2008 sets out exactly where works will be carried out; construction is projected to finish a decade later. High Speed 2 will be in a similar position.
Another difficulty is that government is not a start-up. Most government business builds upon expectations, legislation and systems which have been around for a long time. Even in the digital world, it relies upon decades-old legacy systems which are hard to adapt. What’s more, in politics there is a low tolerance for failure, whereas failing quickly and cheaply then moving on is essential to an agile approach.
These are challenges to Matt Hancock’s ambition of seeing the civil service become more agile. Of course, some projects need to be implemented sequentially: you’ve got to dig Crossrail tunnels before you lay the track, and a half-built railway doesn’t make a useful prototype, so it’s clearly not possible to take a purely agile approach in all areas. It’s also unrealistic to expect Parliament to pass legislation that leaves great discretion to government. But the principles of agile can be applied to some extent to all projects, even if doing so will disrupt hierarchies and take time to introduce.
Ministers should make a positive but realistic commitment to an agile approach. This will mean taking a fresh look at Whitehall’s project guidance, being transparent about progress on implementation so failure is learnt from rather than buried, and providing the resources and space for experimentation. It will also mean being selective about imposing central controls, instead encouraging departments to build management information systems that allow even large projects and processes to be adapted.
Agility is not a panacea. Any project needs resources and people with experience, authority, and commitment to see it through. Agility does not guarantee these things, but it does help managers to focus on producing things of value to their customers – and when money is short, that’s more important than ever.