The Public Accounts Committee has said it was “unacceptable” for an evaluation of government’s Troubled Families programme to be published over a year after the communities department received a draft copy.
The Troubled Families programme was launched by ministers in 2012 before being granted a £900m extension in 2013, and sees central government give councils up to £4,000 to identify and "turn around" families with entrenched social problems including unemployment, domestic abuse and truancy.
A evaluation of the programme, commissioned by the department in 2013, “was unable to find consistent evidence that it had any significant impact at this stage,” according to a Public Accounts Committee report published on Tuesday.
Troubled Families programme under fire after damning independent study
Louise Casey: I did not ask DCLG to sit on critical Troubled Families report
This evaluation was due to be published in late 2015, and the department received a draft version in October 2015. But DCLG did not publish a final version until October 2016, although part of the report was leaked to the BBC in August 2016.
The report describes this delay as unacceptable, and also criticises the department for choosing short-term measures to track a programme aimed at tackling deep-rooted social problems.
Committee chair Meg Hillier said of this focus on short-term measures: "A tick in a box to meet a prime ministerial target is no substitute for a lasting solution to difficulties that may take years to properly address."
Hillier was also clear that officials should not consider the MPs' comments on the delay in publishing the evaluation as a simple “slap on the wrist about Whitehall bureaucracy”.
“Let me assure them that given the ambitions for this programme, the implications for families and the significant sums of money invested, it is far more serious than that," she said.
"But it is particularly important with a new initiative that there is transparency so that the government can learn and adapt the programme.
“The department has undermined any achievements the government might legitimately claim for its overall work in this area.”
The report adds: “The department was evasive when explaining the reasons for this delay, furthering the impression that government is reluctant to be open and transparent about the Troubled Families programme."
A spokesperson for the government said the programme had "enabled local authorities to expand and transform the way local services work with families".
“But of course, there will always be lessons to learn and we have already made significant improvements to the second stage of the programme," they added.
“We will look carefully at the evidence to find out how we can improve the programme further to help some of the most vulnerable people in our society.”
"Deep-rooted"
DCLG perm sec Melanie Dawes told MPs at an evidence hearing earlier this year that the delay in finalising and publishing the report was due to concerns about data quality. An independent academic was commissioned to review the work before it was published.
The department also told MPs that the evaluation was an ambitious piece of statistical analysis that had not previously been attempted, and that evaluators had to work with poor quality data because the programme was still in its early stages.
The National Institute for Economic and Social Research, which led work on a part of the evaluation looking at the national impact of the programme, disputed both of those assertions.
The evaluation had six strands which looked either at financial benefits, national impact or changes in the delivery of services to troubled families.
Although it found no consistent evidence that the programme has had any significant impact so far, there was evidence of good practice stemming from the programme, including changing the way local authorities work with families and allowing them to support more people.
There was also evidence that families taking part in the programme were more confident, reporting that they felt better about their lives and the direction they were taking.
However, the evaluation could not directly link these improved outcomes to the Troubled Families programme, and the department was not able to assure the committee that it would provide evidence of a significant impact in the future.
The report also urges the department to review its payment-by-results mechanism, after the evaluation found evidence that the framework incentivised local authorities to move families quickly through the programme “without providing the support necessary to tackle deep rooted problems.”
There were also reports that some authorities had used national databases to identify families who had experienced positive outcomes and retrospectively define these as ‘troubled families’ in order to claim payments.
The committee also criticised DCLG for its use of the term “turned around” when discussing results of the programme, saying this was misleading “as the term was only indicative of achieving short-term outcomes under the programme rather than representing long-term, sustainable change in families’ lives".
MPs said that in future DCLG “should ensure that the terminology it uses to communicate the achievements of the Troubled Families programme gives an accurate depiction of how disadvantaged families make progress”.