Oral evidence: Troubled Families, HC 711

Oral evidence, 19 October 2016.

Witnesses

Dame Louise Casey CB, Melanie Dawes CB and Joe Tuke.

Chair

Good afternoon. Welcome to this afternoon’s meeting of the Public Accounts Committee. Before we start, let me give a special welcome to fellow members of Public Accounts Committees from around the world, who are here to watch our sitting today. You have an international audience today, Ms Dawes—your fame is spreading. We are doing a review, a call-back if you like, of the Troubled Families programme, which was a policy priority at the beginning of the last Parliament for the then Prime Minister, David Cameron. We hope that we can have a thoughtful discussion today, because we know that support for families with complex needs is not new, and it is also very complex. Policies in this area have spanned Governments of all political colours. With all the programmes that have happened, there has been some evidence of good work, including with this one, with some families, but the work of this scheme—as the NAO Report and the evaluation reports of the Department itself have highlighted—has been clouded and, frankly, let down by many systemic and evaluation failures. The lack of clear parameters and a fixed evaluation framework has undermined success. As we have seen in evidence from various organisations, a lot of the changes are behavioural, and those changes in attitude matter but are hard to measure. So we want to get into a bit of that at some point in the hearing. As a Committee, we are very clear that, yes, we are here to look at how taxpayers’ money is spent, but we are not just bean counters. We want to look at what is happening around long-term change, and at how effective and efficient this whole process has been and will be. But at the moment there appears to be little measurement or any plan to measure longitudinally, and that is something we are concerned about, so we will want to get into that. Before I get into the main subject area, I will introduce our witnesses. Then I want to talk to Melanie Dawes, in particular, about some of the information we have received. Our witnesses today are: Dame Louise Casey, who is now Director General of the Casey Review Team at the Department for Communities and Local Government, but who was head of the Troubled Families programme in the last Parliament; in the middle, Melanie Dawes, Permanent Secretary at the Department for Communities and Local Government, the sponsoring Department for the Troubled Families programme, which is a cross-Government initiative; and Joe Tuke—I think this is your first time in front of us, Mr Tuke, isn’t it?

Joe Tuke

It is, yes.

Chair

Welcome. You are the Director of the Troubled Families and Public Service Reform team at the Department for Communities and Local Government, which makes you the man now leading on the Troubled Families programme. Our hashtag, for anyone following, is #troubledfamilies. Before I get into the main issues in the Report, Ms Dawes, your Department initially expected evaluation in early 2014, as you know, but then it stretched to late 2015. In August there was a leak to the BBC of some of the evaluation work in the Department, so on 5 October our Committee requested the main report and the five feeder reports into it. Initially, your Department promised that we would receive two of those, but we did not get them. Who was suppressing the release of these documents?

Melanie Dawes

First, may I say thank you for the opportunity to talk about the programme today, but—

Chair

Could you answer the question?

Melanie Dawes

But I am keen to answer that question about the timing immediately. The evaluation was actually finalised on 4 October and you wrote, I think, on 5 October. My Secretary of State was very clear that he wanted us to publish the reports as quickly as we possibly could. We had the final proofed copy of the overall synthesis report, which was the remaining piece of the evaluation, on 10 October, which was a week ago, and then we published the reports on Monday. I appreciate that that has not left very much time for consideration—

Chair

Sorry, you just said that there were finalised reports on 4 October, but we requested them on 5 October.

Melanie Dawes

Yes, and that simply was the nature of the timing. We finalised the overall synthesis report, which is the one that sits across the top of all the different elements of the evaluation. We were then thinking about how to publish those documents, and it just so happened that your letter was sent on 5 October. We have moved as fast as we could to—

Chair

That is amazing timing. But the BBC had a copy of a report in August, and it had originally been promised early in 2014, so are you telling me that two and a half years later, by a miraculous coincidence, the report was finalised the day before this Committee asked for a copy of it?

Melanie Dawes

Well, I don’t know what copy the BBC had, of course—it was a leaked document—but what I can say is that the National Institute of Economic and Social Research bit of the evaluation was finalised at the end of August, and that forms part of the overall evaluation. You are right that we intended—certainly recently our intention had been—to publish the full evaluation at the end of 2015. What happened is that about a year ago, towards the late summer of last year, particularly with the National Institute element of the work—the very complicated and quite experimental national data evaluation—we began to get interim findings from the researchers and it became very clear that there were very many data issues that were causing problems with the work. We worked with the researchers through the autumn and the early part of this year, and we brought in an independent academic—Anna Vignoles from Cambridge University—to look at the data issues and confusions, and it therefore took longer.

Chair

This is all process, but that is extraordinary. So, let me repeat, it was expected in early 2014, yet two and a half years later it had not been published. That does not seem feasible. There was something out there—something had been produced—so who was stopping it from being released into the public domain? Was that you, Ms Dawes?

Melanie Dawes

There was nobody stopping it; it was just that it was not yet completed. I think we have learned along the way—

Chair

So was it not yet completed, or was it not yet completed to the satisfaction of what the Department wanted to release?

Melanie Dawes

The work just was not finished. These are independent evaluations—

Chair

I hope you get a discount from the evaluators. Are these documents that we have got now, which we have only had the chance to go over in the past 48 hours, or less, the same as the ones that were being discussed this summer? What has changed in them since August that they could not be published in August, once they had been on the BBC?

Melanie Dawes

The way the sequencing worked was that the individual elements of the evaluation were completed first. As I said, the critical National Institute of Economic and Social Research one was finished at the end of August, and during September ECORYS, the overall contractors, put together the synthesis report and discussed that with us. That took a month, and it was finalised at the beginning of October.

Chair

So you had other reports, which you could have released at any point once they were produced.

Melanie Dawes

We did in some areas. For example, on the cost savings calculator we published some elements of the evaluation earlier. I think it is important to see the evaluation as a whole. We decided it was best to do that and put the whole evaluation together with this political synthesis report.

Chair

You had various reports that you didn’t publish. What were you trying to hide in them that made you wait and sit on them?

Melanie Dawes

We weren’t hiding anything. In fact, I should say that I think the evaluation is very ambitious. The Department invested in a way that no other social programme has invested in an evaluation, but it was big and complex. It took longer than we expected and we have learned from that.

Chair

When did you see the first drafts of the reports? Did you alter, or request alteration to, any of the reports that were published that led into the main report?

Melanie Dawes

We always do quality assurance on research, but in the end they are independent research findings. The most difficult was the National Institute element, which was undoubtedly the most cutting-edge and experimental part of the evaluation and the most difficult to pin down and finalise, and where all the data issues really came into play. Because that was so difficult, we knew it would come under scrutiny. That was why we got Anna Vignoles to look at that and to quality assure for us not just the work, but how the conclusions were presented.

Chair

So did you personally look at how conclusions were presented in that report?

Melanie Dawes

I did not personally get involved in that process.

Chair

Which official in your Department looked at the presentation of that?

Melanie Dawes

Our analysis directorate oversees all of the Department’s evaluation work and the relationships with the contractors.

Chair

Did they go back and ask the researchers to change any of the recommendations?

Melanie Dawes

There were certainly some areas where we— In the end, as I said, we made sure that none of these judgments were made by us alone.

Chair

Just answer the question: did they go back and ask the researchers to change any of the recommendations in the report?

Melanie Dawes

Certainly not the recommendations, but in the case of, particularly with the National Institute—

Chair

But they did change something? You are indicating that they went back and asked for some changes, yes?

Melanie Dawes

We had a number of concerns about—

Chair

Yes or no—you went back and asked for some changes?

Melanie Dawes

We challenged the fundamental basis of some of how the data was treated by the National Institute in their work. It is not about the report or what was written in the report. It actually started with some quite big challenges that we felt we needed to bring out about how the data was being done.

Chair

So you weren’t happy with the report that they produced?

Melanie Dawes

We looked at the interim findings. That is not so much the report as the actual analysis. If I could perhaps give you a little bit more detail—

Chair

They are kind of the same thing, aren’t they? It’s a report by an independent academic body.

Melanie Dawes

No, I don’t think they are. Firstly, was the analysis done well, bearing in mind it was complex?

Chair

If you are paying an organisation that has a reputation for doing this work and it doesn’t do the analysis well, that is a pretty serious breach of contract. Are you saying they did bad analysis?

Melanie Dawes

It was very complicated. We think that, in some ways, they did not deal with the data as well as they could have done. For example, for three local authorities the data was not included, and in some areas, where data did not exist, and there were a number of reasons why that was the case, they assumed—

Chair

Did you pay these people to do this job?

Melanie Dawes

Yes, we did pay them.

Chair

As a result of your unhappiness did you ask them for any money back from the fee?

Melanie Dawes

We had long conversations about the nature—

Chair

Yes or no?

Melanie Dawes

No, we haven’t asked for any money back. In the end, the evaluation has been published and the work has been done.

Chair

So you are saying that you commissioned an independent report. When it came out you weren’t happy about the data analysis they had done, and you went back and asked them to change it as a result?

Melanie Dawes

It wasn’t about asking them to change it; it was about working with them and bringing in experts to look at that. As I said, it was a very complicated piece of work. No one has done this kind of thing before.

Chair

We will get on to data measurement in the other thing. You only released this at 6.15 pm on Monday. That was less than 48 hours before this hearing. It seems a long gap between even 10 October, when the final report was published—even if we are generous and allow you that—and releasing them to the Committee. Why was it so late?

Melanie Dawes

That was the quickest I have ever seen, to be honest. The final report was available and proofread, which is done independently by the researchers—that’s the way the process works—and then we had a week in which to get them out. Normally, the Government protocol is 12 weeks. We did that in a week because of today’s hearing.

Chair

Sorry, but it went online. These were documents that were there and available online that you could have given to us in confidence in the form in which you had them on 10 October.

Melanie Dawes

We did publish them as quickly as we could in that remaining week and we gave them to the National Audit Office on Friday.

Chair

We are different from the National Audit Office; we are the Public Accounts Committee.

Melanie Dawes

I appreciate that.

Chair

What was stopping you sending them to us before Monday?

Melanie Dawes

In the end, the decision was taken by our Ministers that they wanted to publish them. That then took a week of work to get all of that very complex material and the five evaluation reports, plus DCLG’s own work—

Chair

Ms Dawes, I don’t want to have long exchanges; short answers please. You had the final reports on 10 October, yet you couldn’t get us a copy of those reports before seven days later, even in confidence and for preparation for today’s hearing, knowing this was coming up.

Melanie Dawes

Well, in the end we did it as quickly as we could. I appreciate we haven’t left very long before today.

Chair

What was stopping them just being sent to us on 10 October? We have email.

Melanie Dawes

Well, in the end we were about to publish the report. What my Ministers wanted to do was to get them out in the public domain, and that is where we focused our efforts.

Chair

You have said that repeatedly, but why didn’t you get them out on 10 October to us? I don’t know why they couldn’t have gone public as well. Why couldn’t you publish them on 10 October? What happened between the 10th and the 17th?

Melanie Dawes

All I can say is that we did it as quickly as we could.

Chair

You have repeated that several times.

Melanie Dawes

It was partly about making sure all the information was there and available. These things are not as straightforward as just pressing a button and putting them on a website. It did take a bit longer.

Chair

We sometimes receive information like that in its early iteration, and then it can be published publicly, but you didn’t consider that you could send it to us even in draft form. Well, you are saying it was not draft form; you said it was finalised.

Melanie Dawes

It was finalised, yes.

Chair

So you couldn’t have sent them to us before. Why not?

Melanie Dawes

I’m sorry that we weren’t able to do that, but we decided to publish them and to get them out as quickly as we could.

Chair

Did your Ministers have to sign it off between the 10th and the 17th? Did they have oversight of it?

Melanie Dawes

They didn’t sign off, in any sense, the contents of the report. That is not what happened.

Chair

But were Ministers involved in the last week?

Melanie Dawes

They certainly weren’t involved at any stage in the content of the evaluation. It was an independent report.

Chair

Did they go through the Minister’s box in the last week to be agreed to be released?

Melanie Dawes

Certainly, decisions on when to publish are for Ministers, but they didn’t get involved in any of the content of the report.

Chair

So it was Ministers who decided to publish on Monday the 17th?

Melanie Dawes

That was their decision, yes, although it was as quickly as I feel the Department could have managed to get the documents out.

Chair

Okay. We’re not going to get very far. May I just remind you that it was 765 pages 48 hours before a hearing? Although we are all assiduous, there is a point about reasonableness there. Your answers have been, I have to say, a bit evasive, so I wonder who you are protecting in all this. There have not been good headlines for you on these reports. Are you protecting your Department or your Minister?

Melanie Dawes

No, I don’t think so. I’m sure Louise will want to come in, particularly on the content of the programme and the evaluation. To be honest, this is a big, complex evaluation, and the debate in the last 48 hours in the media has not brought out exactly what the evaluation shows, so I hope we can manage to do that and explain some of that today. I think this is a good programme that achieved some really good results in the last Parliament. We have adapted it, and particularly adapted our approach to the evaluation, so we can make it stronger for the future.

Chair

You have seen the headlines. We want to get beyond headlines in this Committee—we are obviously keen to look at things in a bit more depth—but headlines are there for a reason: they reflect some of the things that are in the report. You are not saying that the news coverage hasn’t been reflective of some of the outcomes of this evaluation.

Dame Louise Casey

Can I step in at this point? The first thing to say is that you are absolutely right that the timing does not help the Committee at all, in terms of when you got the information.

Chair

It was also slightly suspicious timing.

Dame Louise Casey

Indeed. Neither does it help people like me and others who believe we have nothing to hide and nothing to be worried about here. I would just say that what has been in the headlines for the last few days is the one element that we had problems with in the Department—I was not responsible for the Troubled Families programme at the time, but I am now fully aware of it—which is that when the draft report first came through, the organisation had put through data that was inaccurate and flawed. The process was—this was a while ago now—that they had to take the information in-house. That is why it has been delayed. I can understand your frustration and your line of questioning. I desire to be accountable here. The process was pretty challenging within the Department. Essentially, when they looked at the information—this is the analysts, not people like me or, indeed, Joe Tuke—they found things like they had put three local authorities’ flawed data through their system. They accepted it; the Department did not. They ended up having to go to the University of Cambridge to take this information to an entirely different person and say, “Is this okay? Is this not okay?” One of the other things that has been difficult, certainly for me, is that a lot of the comment and extraordinarily negative publicity in the last few days has been based on that one report. Appearing before you today is pretty challenging because we’re playing catch-up with one element of a 700-page report that essentially says that the Troubled Families programme has no impact. I, for one, do not believe that is true, and I think I can prove that to you, as can colleagues from the Department.

Chair

I am just saying that it was not difficult to predict, for those of you who have been around the block a few times on these things, that if something like that is in a report, you are going to get bad headlines. Were you trying to suppress that element of the report, Ms Dawes?

Melanie Dawes

No, we weren’t trying to suppress it.

Chair

I want to finish in a minute, because we could go round and round on this. There were extraordinary hoops to go through to get the right analysts to do the work they are being paid to do. It seems extraordinary that the Department could not even commission the right people and the right research, which is what you are effectively saying to us.

Melanie Dawes

Perhaps we can come on—not now, but perhaps later—to explain quite how complex that work is. I do appreciate, though, that it is somewhat extraordinary, and it was disappointing that the work effectively had to be redone.

Chair

It was disappointing—we will leave it at that for now. The Comptroller and Auditor General wants to come in. Can you be brief, please?

Sir Amyas Morse

Did you check all the research work and calculations in the report? Did you effectively audit all the findings in the report yourselves? You obviously went through all the numbers to check whether they were correctly calculated—is that right?

Melanie Dawes

No, it was not a question of going through a spreadsheet and checking numbers. It was a question, though—particularly for the national data study, which is a very unusual piece of research, we always have a conversation with the researchers to make sure we have understood the analytical basis of the work. Quite quickly it became apparent that there were real issues and judgment calls that we thought ought to be made differently. In the end, we worked through that with an independent, external evaluator as well.

Chair

I still find it extraordinary. This is a report commissioned by the Department. When you commission things—we look at procurement all the time—surely you set parameters, you talk about what data might be used and you have some rules about what is acceptable and what data is verified. Did you not do that when you commissioned this work?

Melanie Dawes

That is one of the reasons we did raise concerns: we thought that some of the data that had been accepted into the analysis was not as good as it needed to be.

Chair

This was a long way down the line. Let me repeat, this was going to be published in early 2014, so surely before the report was even in train there would have been some discussion between someone in the Department—I do not care who it is—to make sure that everyone was agreed and sighted on what sort of focus the report would have and what data it would use and make sure that the methodology had been agreed and understood on all sides. But you are saying that did not happen.

Melanie Dawes

I do not think this sort of evaluation was ever something where you could say, “We are going to do this bit exactly this way,” because it has not been done before. What is going on here is that we have taken data from 57 local authority areas, brought them up to form a national dataset and then linked them with national administrative datasets. That is completely cutting-edge. I am not surprised that the researchers found it difficult.

Chair

To be clear, did the researchers come to you and say, “We think we’ve got a problem here and we’d like to talk to you about how we are going to do this” or did they present something to you and you said, “Oh no, that’s not good enough. Go back and we’ll help you rethink it”? Which version of events was it?

Melanie Dawes

It was a more collaborative process than that. I think it was about talking about the findings of the evaluation, which at that stage were interim findings. This is last summer, not in 2014—

Chair

So they came to you and said, “We are not sure about this. We want to talk to you about it.”

Melanie Dawes

I was not part of those conversations; they were led by our analytical teams and overseen by our chief analyst.

Chair

We could get into how you procure research, but I have been a Minister and I would have been appalled if a piece of research had come back so out of kilter as you are suggesting. It is important to make sure that the clear parameters are set at day one.

Melanie Dawes

It was not completely out of kilter, but there were some judgment calls in a complex field that needed to be challenged and in the end the vast majority of the work that was done was absolutely fine. This is not a completely flawed piece of analysis. It is a very complex piece of analysis with genuine data problems where there were genuinely good questions about how to deal with those problems that needed to be discussed.

Bridget Phillipson

It just comes across that you are bashing the report because it was inconvenient.

Melanie Dawes

I can completely appreciate that that is how it may look, which is why we sought to have Anna Vignoles come in and look at it and make sure that we were not in any way prejudicing the findings. We did not write the final findings; she actually looked not just at the data but at what we could conclude from the data. The specific point that she drew out when she did her independent work was that because of the data limitations it was important not to say that the evaluation showed that there had been no impact. What was correct to say was that the evaluation could not prove that there had been an attributable impact from the programme. That is quite a nuanced point—I do not think it has come out in the press coverage in the last few days—but that was the critical point that she sought to draw out in the work she did.

Bridget Phillipson

Neither of those scenarios is great—to say that it has had either no impact or not a particularly good impact. Neither of those would demonstrate a great use of public money.

Melanie Dawes

What the evaluation does show is that the families in the programme did improve their outcomes. It does show that quite clearly, whether that is work or school attendance and so on. It also shows that there is a statistically significant improvement in how they feel about their lives, in particular feeling that the worst is behind them.

Chair

We don’t want to go into the detail.

Melanie Dawes

I think at some at level you have to bring out the detail a bit, if you will forgive me, with one more point. What we need to put in context is that this evaluation ran for only 12 to 18 months with individual bits of data. What proved not to be possible in the research was to form a control group that would have allowed you to say, “Here’s a group of families that did not have the Troubled Families intervention but are similar to the ones that did. Therefore, how can we measure that difference?” That was the thing that the research was not able to prove.

Bridget Phillipson

But the report does also say that they were unable to find consistent evidence that the programme had any significant or sustainable impact.

Melanie Dawes

That was attributable to the programme, although the outcomes did improve, and that is shown in the evaluation.

Chair

Ms Phillipson’s point rests. We are going to move on now from the actual publication of the report. I want to come to you, Dame Louise. Will you remind us how many years you headed the Troubled Families programme?

Dame Louise Casey

From 2011 until I went to Rotherham, which would be 2014, I think.

Chair

So it has been a big part of your life?

Dame Louise Casey

Very much so.

Chair

Does the media coverage and the evaluation disappoint you?

Dame Louise Casey

I am glad that is the starting point. I want to say that there is no way on God’s earth that it would be helpful to anybody to decide to suppress people, because obviously they have access to the press in a way that they have shown very clearly. Therefore, it is just not in our interests to do that—certainly not in my interests to do that. The frustration is that it is one part of a much bigger story. The other thing is that the way it has been presented in the media—if I am honest, quite deliberately—has not got the caveats across. Nowhere does the word “attributable” appear—

Chair

When you say “quite deliberately” who are you attributing that to? To the media or to the researchers?

Dame Louise Casey

No, not to the media. Sorry, I’ve got nothing to lose in a scenario like this. Lots of comment made by those closely involved with the evaluation, who have been leading on the press in the past few days, has been unedifying. They didn’t wait until the rest of the evaluation was out. I am sure they feel suppressed. That simply isn’t true. I am the first to say about Jonathan Portes and NIESR and their research that, after a lot of correction and sorting out, I accept the findings of the research. That is that over the timescale that they looked at the families, which was really early on in the programme, the changes in those families, which they do not dispute, cannot be directly attributed to the Troubled Families programme. You can, on the other hand, find a lot of information as to why. They had not, frankly, Chair, put any of the caveats in the public domain.

Chair

So you are unhappy with how the people who were funded by the Department to do this evaluation have conducted themselves.

Dame Louise Casey

I am. I will be honest about it: I am. I don’t want to make it a personal thing because I accept that, within the strictures of this one piece of research, it doesn’t prove what I hoped it would prove. Did I ask the Department to sit on it? No, I didn’t. I think it is better to have that stuff out and washed out in the public domain so that you can have a discourse about it. My frustration, if I am honest, is that we haven’t had a chance to set the record straight, that you are not only—

Chair

Well, you are here now and we can do that. In terms of the evaluation report itself, you say you agree with some findings. Are you happy now that the methodology was correct and that it is a proper and fair evaluation?

Dame Louise Casey

I am happy with ECORYS’ overall evaluation report, that you couldn’t publish separate ones, as you will find out when you finally get time.

Chair

We have seen how they link in.

Dame Louise Casey

You can see how they link together and you can’t take any one element on its own. In fairness to ECORYS—to which NIESR is a sub-contractor—what they have quite cleverly tried to do is knit together what the overall learning of the story is. I accept that. Within that, I wholly accept and understand why, at the beginning of this programme, when we compared these two things, they were not able to attribute changes in the families directly to the Troubled Families programme. I am not sitting before you here today disputing that fact. I am sitting before you today disputing how that fact has been used and how it has been used in isolation of the rest of the evaluation. No one disputes the fact that 116,000-plus families had problems and now have fewer problems. Nobody is disputing that; the stated aims of the programme. Were you to have read some of the publicity in the past few days and, indeed, what has been put out by this organisation, you would think, “Oh, the whole programme is useless.”

Chair

We are going to get on later to issues around the control group and so on, but I want to go back a little bit in history, to give us an idea of where the buck stops. First, Melanie Dawes, who is responsible for this programme? Is it you? Is it Dame Louise? Who is the main person who should be answering questions from us on this?

Melanie Dawes

Well, I am the permanent secretary of the Department that is responsible for this programme.

Chair

So the buck stops with you.

Melanie Dawes

Yes, I think it does. Clearly, Louise was in charge of the programme in the last Parliament.

Chair

Okay. I want to be clear. Louise Casey, you would have been involved in the days before the announcement of the programme. When were the first discussions about doing something with a group of families that have been dubbed “troubled”?

Dame Louise Casey

Prior to my or anybody’s involvement, in December 2010 the then Prime Minister David Cameron made a speech in which he outlined a public commitment to deal with troubled families. After the speech, the number came out as 120,000 families, based on the estimate they had at that point, which we were not involved with but we understand why they used that. It became clear to advisers in Downing Street and the permanent secretary at the Cabinet Office, Jeremy Heywood, that the Prime Minister’s commitment to tackle troubled families, as he had called them in his speech, was not necessarily going to happen in the way they wanted. Because of my personal experience doing things like family intervention and social policies for other Governments, they asked me to have a quick reconnoitre across Whitehall as to whether or not each Department was going to move on how they might meet the Prime Minister’s commitment. I reached a conclusion that they probably were not, depending on how we wanted to meet that commitment. I was appointed in October 2011. The Prime Minister then made another announcement in December 2011, where he said he wanted to turn around the lives of 120,000 families. I was involved in all of that process.

Chair

That is an incredibly helpful summary. It has often been publicised as something that happened after the riots, but you are telling us that there was a clear plan—or at least a plan, if not a clear plan—in place before that point.

Dame Louise Casey

In fairness, I had said to them in the summer during which the riots happened—at a very low level; I was not doing a major review—“There are different ways you could look at this. You could do it writ large. In other words, do you actually want to really transform the lives, spend money and have a massive Government programme? Or you could do a classic line to take, which is that there are lots of ways Government is helping all these families, such as through early help.” I said, “You’ve got two options here. Whitehall might try to get you to do the middle one. That will be a mistake.” It was left there. The riots happened, and then I think they decided around September—

Chair

“They” being No. 10?

Dame Louise Casey

The Prime Minister and the Quad at that point. They wanted to have a much greater focus on the Troubled Families programme.

Chair

So you had a bit of a perfect storm for any poor Government Department. The PM had made a pledge, and then the riots had happened, which made it imperative. There was this figure of 120,000, which you say the Department had no part in. It is a perfect political storm. Who then had to backfill a methodology to make sure the Prime Minister met this? There was not a programme in place when he made that announcement, was there?

Dame Louise Casey

No, there wasn’t a programme in place, but to be honest, I have now worked for an awful lot of Prime Ministers and Ministers, and they quite often make quite direct policy commitments and then it is the job of Whitehall to run behind them and make sure they commit them. I don’t think that is an unreasonable way sometimes for things to get done.

Chair

But he had made the pledge about 120,000 families being “turned around”. When you looked at how you were going to deliver, you knew there would be pressure for the next Prime Ministerial speech to point out how far people had been turned around. You had to develop criteria that allowed him to hit a numerical target and this slightly difficult-to-define target of “turned around”. Who was making decisions and setting the framework for that? Presumably it was you, Dame Louise.

Dame Louise Casey

A combination of people. Indeed, some of his Cabinet colleagues were closely involved in this. Can I explain? The 120,000 figure—which a number of critics, but not too many, have a problem with—was the starting point of some of the difficulties for people involved in criticising the programme. The fact of the matter is that no one—including us, in other lives and jobs—had ever actually tried to work out how you could tackle these particular sorts of families where you have a multiplicity of highly complex problems. In fairness to the Government, I would say the best proxy they had was the old Social Exclusion Unit data—the family survey data—which I think was then owned by the Department for Education, which gives you, more or less, this figure of 120,000 across a load of cohorts. I do not think it is unreasonable that when we were developing with them what they wanted to evidence by “turned around”, they took the 120,000 figure. We then went out to local authorities and said, “In terms of kids who aren’t in school, families that are committing antisocial behaviour, and worklessness, as well as another high-cost indicator should you need it, can you stack that number up?”—and they could. The criticism that has sometimes come in is that we were criminalising troubled families—

Chair

We haven’t used that word. You are digressing a bit.

Dame Louise Casey

Sorry; I am jumping ahead. I am digressing. I am in therapeutic mode, because it has been a rough few days.

Chair

The Public Accounts Committee is great therapy.

Dame Louise Casey

I have never thought of it that way before, Chairman.

Chair

You can tell us all.

Dame Louise Casey

Essentially, it was a reasonable position for them to take. If you look at people who are poor, deprived and in overcrowded conditions—if you are not in school, for example, that is one of the likely answers.

Chair

I am aware that you know an awful lot about this. As constituency MPs, we all do too. We see this. We really need to cut through narrative and focus on what happened in this whole programme. Obviously, you have read what some of the evaluators have said. One of the evaluators said that the whole thing became “an exercise in self-justification”. There is a feeling that you had the number, you had the Prime Ministerial pledge—we all know, especially those of us who have been in government, how you have to run to make sure that a Prime Minister’s pledge is met, otherwise it is a big embarrassment for No. 10. The Prime Minister possibly doesn’t even realise the amount of work that is being done to make sure that pledge is met, even if it is artificially met. What do you say to that comment—“an exercise in self-justification”? It comes across from what you have just described that the numbers were there—the numerical driver was 120,000; the driver was this “turned around” evaluation.

Dame Louise Casey

I am quite comfortable—I was at the time and I am now—that we could find and we would find a cohort of at least 120,000 families in this country who would meet the stated objectives of the programme as laid out. I did not—

Chair

But what about the “turned around” bit?

Dame Louise Casey

I have worked for one Prime Minister who used the word “respect” when he talked about antisocial behaviour; I have worked for another Prime Minister who decided to call parents—

Chair

How did you decide what was “turned around”?

Dame Louise Casey

This Prime Minister used the term “turned around”. For him, that was an expression about turning around a family; I do not think he necessarily thought in terms of indices.

Chair

Exactly. So why did you then make sure that was included in evaluation in a very narrow way?

Dame Louise Casey

Because it was incredibly important to make sure that this time around, the programme had a real focus and had real outcomes attached to each one of these families. In the good days, we would put plenty of services into these families. The question is: did anything change? I was well up for an outcome-based programme—to use the lingo—and very keen to make sure we reduced crime and anti-social behaviour in those families. You know yourself that the silver bullet in social policy is getting children into school all day, every day and fed, if we can possibly make it, so of course I was comfortable with those two as key indicators. Again, we all know—whichever party you represent here today—that work is the most profound thing we can do for any household. I was not backfilling anything. Yes, there was a fight and a debate at Cabinet about how many indicators they wanted to put in—I know I am about to be told to shut up, even by the Permanent Secretary—but of course there was a trade-off.

Chair

I want to ask you a question, Dame Louise. That is why—

Dame Louise Casey

I am sorry; it is just annoying to see a programme so shot down.

Chair

I know you know a lot about this and you want to tell us about it. While I joke that it is therapy, it is actually a serious Committee trying to get serious answers. Can we just go back to the evaluation? You had this pledge of 120,000. You had the phrase “turned around”, which has become this quite mechanistic approach with quite tight criteria attached. You then had to draw up a cost calculator, because one of the other measures of success was the proof that intervention would save costs in the long run. What evaluation did you do to draw up that cost calculator? That is a huge political prize. I should think every Prime Minister anywhere would want to prove the value for money of this, but it is very difficult to do. What evaluation did you and your team do to work up that cost calculator model?

Melanie Dawes

Could I come in here and tell you my observation of this, having come in just under two years ago? Yes, the Department had to move fast to put this programme in place, and yes, the rest of Whitehall had to move fast to work with them, but the payment by results mechanism that was put in place is quite clear. Yes, people can call it mechanistic, and some have, but it did actually get local government and its local partner agencies moving around some clear outcomes for the first time in a cross-agency way. So, from my perspective as accounting officer, the payment by results gave us some clarity about whether or not the programme’s outcomes were being achieved, and whether we were paying money for something we could be sure was happening.

Chair

You had a nearly 100% success rate.

Melanie Dawes

In fact, we have never said that. Local authorities worked with more families than that. That is something that the evaluation report does draw out. We can be confident from the payment by results mechanism that we know we were paying for outcomes; the question is a wider one from the evaluation. When it then comes to the cost-benefit calculator, that piece of work was done—again, most other programmes have not attempted to do this—to try to get at the true value for money, if you like, of whether this was really going to do what local government believed it was and save money.

Chair

So, what evaluation did you do to draw up the cost calculator? You have given me a lot of words there, but what evaluation did the Department do to draw up the cost calculator? How did you work out the methodology for it?

Melanie Dawes

Joe might want to come in on the details of this. We worked with local government to work out how we were going to capture reductions in things like police call-out rates, the need for people to attend family homes, and so on. Data systems were set up to try to capture some of that data. Over time that became slightly more sophisticated and we were able to put costs on it.

Chair

Do you think the cost calculator is now a good calculator of the cash savings?

Melanie Dawes

I think it is a really good first effort to do something we have not done before.

Chair

We are trying to have a discussion about honesty. So you are saying it is not perfect—

Melanie Dawes

It is not perfect, and in particular—

Chair

What do you think needs to be done to get it better?

Joe Tuke

When we go out to talk to local authorities, the reason they find the cost savings calculator really useful to them is that it is about, how do they drive the integration of services? How do they show the benefits that working in this way brings, not just for them but for their partner agencies? In order to do that though, you need to gain information, but that is a really difficult thing to do, not just because of things like data protection but also because systems don’t talk to each other. So an awful lot of work had to be done to develop that—

Chair

That is the mechanistic part. I am not saying that it is not difficult but how did you work out what should be measured? You were talking to local authorities. What needs to be done to make it better, because Ms Dawes has acknowledged that it is not perfect?

Joe Tuke

The big thing is being able to get the data from other agencies about the reductions in particular volumes of indicators that they are responsible for. So, for instance, how do you get the police to tell you the reduction in the number of police call-outs? How do you get them to give you information about the convictions for the cohort of troubled families you have? How do you find out from the schools about the reduction in truancy? How do you get the reduction in the number of ambulance call-outs or A&E episodes?

Chair

So where are you at now? What do you need to be doing to make it better?

Joe Tuke

At the start we worked really hard with seven local authorities. This isn’t something that is easy for local areas. We managed to produce a report showing some of the ways in which they had done it. The best local authority at the time was Wandsworth. We stole their person leading on it, brought him into the team and got him to work with all our other areas. We did about 30 tutorials around the country; we did a helpline. Eventually, we got about 67 local authorities that could use the cost savings calculator to a reasonable degree. So they could use it with a random sample of families.

Chair

So it is getting better all the time?

Joe Tuke

It is getting better all the time. In the new programme, we have designed it so that we pre-populate a lot of the information for them from the impact study.

Chair

I am going back to Dame Louise. I am going to ask some quick-fire questions if I can, because I need to move on to colleagues. Who decided on the success criteria? Did No. 10 give you a free rein, or did you decide?

Dame Louise Casey

No, the three emblematic payment by results of education, work and anti-social behaviour and crime were signed off by the Cabinet.

Chair

Signed off by the Cabinet, yes, but who decided in the Cabinet? Who decided to recommend it?

Dame Louise Casey

To be honest, there was a fairly robust debate over several meetings both of the quad and the Cabinet about what they wanted in it. I was very clear that I wanted education to be in there because I genuinely think that is—[Interruption.] Also, it was very clear that we wanted to do anti-social behaviour and crime, and we wanted to do worklessness because they are the ones that actually show change.

Chair

So, you are saying the decision was really clearly the quad and Cabinet?

Dame Louise Casey

Yes, very much so.

Chair

Can I ask some more quick-fire questions about how you, Dame Louise, fitted in the system? You were in this odd role, in a way, because in lots of the roles you have had in Government you have not been quite in a departmental structure in the same way as you are now as a director general. Who did you report to on a regular daily basis and periodically? The Prime Minister? No. 10?

Dame Louise Casey

I did not have to report to many people on a daily basis. I was based in the Department for Communities and Local Government. My reporting line was to Sir Bob Kerslake on civil service and to Jeremy Heywood in No. 10. Politically the programme ran into the Prime Minister of the day and the lead Cabinet member on it was Eric Pickles.

Chair

Who did you provide updates to? If someone said, “How is it going?” did they just chat to you in the corridor or did you have to go to a meeting with somebody and say, “This is what’s good, what’s going well, what’s not going well”?

Dame Louise Casey

I think it was mainly the Secretary of State. It would have been Eric Pickles.

Chair

It would have been the Secretary of State. That bit would have been through the Department. Did you report to any other Departments apart from the Cabinet Office and the DCLG?

Dame Louise Casey

No, not at all. There was a Social Justice Committee that used to take Troubled Families as one of the things they discussed, I think every quarter.

Chair

Was that at Cabinet level?

Dame Louise Casey

That was a Cabinet-level Social Justice Committee serviced by the Cabinet Office. We went to the quad, as was, on a fairly regular basis. It felt at the time like we saw a lot of people.

Melanie Dawes

Could I add that I was in the Cabinet Office at the time, as you may remember? What I observed is exactly as Louise just said. The primary responsibility was within the Department but there was a lot of interest in the programme.

Chair

Exactly. That is why I’m asking.

Melanie Dawes

It came to discussion in a number of different Cabinet Committees, including Cabinet itself every so often.

Chair

As far as going and talking to people about things, Dame Louise, you have demonstrated today that you are very good at coming to talk to people about things that you are passionate about. That is nothing other than a compliment. You talk passionately about something but, out of all those people you were reporting to, were any of them looking at the data and asking about the metrics and the calculator and those sort of detailed things?

Dame Louise Casey

Relentlessly. We had things like the implementation unit in the Cabinet Office looking at what we were doing. I need to be clear. The reporting line and the accounting officer for the Troubled Families programme was Bob Kerslake, Melanie’s predecessor. In terms of that, I would report to Bob. The money and everything ran through the Department for Communities and Local Government. At a policy level, we were obviously trying to work across Departments. DCLG was not in charge of work. That was Iain Duncan Smith’s area. So we had a framework by which we had to talk to and enlist help from other Government Departments. One of the things we may get on to later is that half way through the programme we managed to score a deal with the Department for Work and Pensions through Iain Duncan Smith in order to get people out of Jobcentres and into our Troubled Families programme. So it was a very cross-departmental programme.

Chair

It was 500, wasn’t it, people from Jobcentres? I need to pass on to Chris Evans in a moment. There was an implementation taskforce set up in 2015 when I think Greg Clark was Secretary of State. That was under you, Joe Tuke.

Joe Tuke

That was, yes.

Chair

Why was that set up? How was that different from some of the things that went before?

Melanie Dawes

That was essentially a Cabinet Committee. Last year there were a number of those Committees on a number of different topics.

Chair

So it was a different Cabinet Committee from the one Dame Louise Casey talked about.

Melanie Dawes

It was designed to help our Department and Secretary of State to get the support that was needed across other Departments, and also for those other Departments to ensure that they understood what the programme was delivering. It was a Committee, which is quite common.

Chair

Back to Dame Louise, among all those things, which was the most important? I do not doubt that you have a passion for this subject area but, in the end, there is a political and day-to-day reality. What was more important, keeping No. 10 happy or getting long-term change for these families?

Dame Louise Casey

Without a shadow of a doubt, I would not have left being the Victims Commissioner, which was a difficult job but one I thought worthy, and done this unless I thought there was a cat in hell’s chance of actually trying to change the system for these families, as well as help the families. It was unfinished business for me. We had set up family intervention projects around 2006, 2007. We set up 53 of those projects throughout the United Kingdom. In every single social policy job I have done, systemically, we fail multiple-need families to a—

Chair

So you are convinced that, even if you had not met short-term targets, No. 10 would have been happy.

Dame Louise Casey

They would have to be happy. The bottom line here is that we wanted to help the most difficult families. I hope we come on to this at one point. Some of the coverage and misrepresentations—

Chair

We don’t want to go back over that please.

Dame Louise Casey

The idea that we were trying to go for low-hanging fruit and easy families just isn’t true. It has now been backed up in the research that has only been published this week, which shows whom we were getting to through the programme. No, it did not necessarily fit easily with everybody that we were reaching out to these extremely difficult families, but we did and—

Chair

We are slightly digressing from the point I was making, which was the long-term aims versus the short-term aims. However, I will pass straight over to Chris Evans and we may come back to that.

Chris Evans

I just want to start—120,000 families. I appreciate that you were not involved in setting that target. It was 400,000 families for phase two. Do you think, in all honesty, the bar has been set very, very high here, and you were always set up to fail anyway? I am trying to be helpful with that question.

Dame Louise Casey

If I did the 120,000—I mean, what is really interesting about this is that what we have to try to do in public policy terms is to work out both what is costing us the most money and who we are allowing to lead the most difficult lives. We did not have to work very hard once we had the 120,000 to—basically, we went out to local authorities and said, “Do you think you have families that fit this criteria?” They did not have to work very hard at it. So, on the original 120,000, the short-term thing would have been to go for easy families. We didn’t; we went for families with significant levels. There were days, and that was why I was pleased that we were able to get the extended programme, where I did not think 120,000 was enough, and I thought that we were not helping enough families through that programme with the right outcomes. I have also thought in retrospect—we wanted the whole system to change, to move from being just about children to being just about families, and we could not mandate that from the Troubled Families programme. We had to inspire people to do that, we had to entice people to do that and we had to work collaboratively with people to do it. So, on the 120,000, we did not have to work very hard to find those families, and I stand by it. I think that Joe and colleagues have led the process on the 400,000, but again I do not think that over time we will find that too difficult—

Chris Evans

Looking at figure 2 on page 7 of the Report, I find that the definition is very vague. I come from the south Wales valleys, which would be somewhere where there is a lot of troubled families, and I would know a number of families who would qualify for this. The other situation that I worry about as well is that, when you have the interventions, a measure of success is someone going to school regularly, or someone getting involved in the Work programme or being back in work for six months. Do you think this evaluation happened far too early in the programme, because this type of work needs a number of years for it to play out? Would you say that was fair criticism?

Dame Louise Casey

Yes.

Melanie Dawes

Perhaps I could answer that question. I do think it happened somewhat early and it did not run for long enough. It was a snapshot, run over 12 to 18 months, and that is one of the reasons why there were so many problems with missing data and so on. So one of the things we have done for the new programme is to use that same data infrastructure, if I can call it that slightly technical thing, that same infrastructure of a national-level dataset built from locally reported—local authority-reported—outcomes for families. We are now running that data every six months, so we are going to be able to look at that longer-term impact in the way the Chair was suggesting earlier is important—that more longitudinal look. So we have made an investment here, through the first programme, but we have broadened it and are going to be able to use it much more systematically for the future.

Chris Evans

Okay. Before I give the figures, Dame Louise, you have already spoken about your fight to get education involved. What I’m concerned with more than anything—this is looking from a purely policy point of view—is that once those children are back in school, how are we monitoring their development in school? For example, are we looking at whether they have been disruptive in class? What is their educational achievement like? Are they subject to some sort of soft exclusion—standing outside the door if they’ve been disruptive, or whatever? When they’re in school, how are they being monitored, because that seems to be a very difficult thing to monitor?

Dame Louise Casey

That is an incredibly difficult thing to monitor. Did we attempt to do that in the first programme? No, we didn’t. That is not part of what the evaluation was, so I think that is a fair thing to say. As you go deeper and deeper into social policy, you could look at all sorts of things, of which that would be one. I think that is fair enough. I suppose the starting point was that it was very clear that we really didn’t have to work very hard to find families where children were not in school at all, where truancy and exclusion were really high. So the starting point was, “Let’s deal with that first.” We asked them to deal with that alongside issues such as domestic violence, drug and alcohol abuse, unemployment, depression—all of those things. So the answer to your question is: no, we didn’t, partly because we were prioritising the starting point. We were prioritising where we were starting from in a programme that was pioneering and ambitious on an incredible level, because we were not just an education programme, just an unemployment programme, just an ASB programme or indeed a programme that just looked at those sorts of families. We were trying to get to families that had multiple needs and were multiply complicated. In a way, that was our starting point. Getting the system to do that was sometimes really difficult indeed.

Chris Evans

I want to turn the question on its head. According to figure 8 on page 16, of the 1,048 families for which full data were available, 40% had three or more children, compared with 16% nationally; 49% were lone-parent households, compared with 16% nationally; and one in five had been at risk of eviction in the previous six months. Do you think we are intervening too late? Do you think they should have been identified much earlier?

Dame Louise Casey

Chris, as the sun comes up and goes down, we always intervene too late.

Chair

Please call the Member Mr Evans.

Dame Louise Casey

Mr Evans, as the sun comes up and goes down, we are intervening too late. That is the difficulty with all of this policy. At which point do you move into the system and work out where you are able to get the most traction and the most changes? Having said all that, when you look at the impact of the programme against the families with those multiple problems, having a 77% reduction in police call-outs, a 61% reduction in the risk of eviction and—dare I say it?—a trickier and more controversial one, a 46% reduction in under-18 conceptions, you can see that with those families you cannot give up on them. That would be really wrong.

Chair

Sorry, you just rattled off a number of figures. Where were you quoting those from?

Dame Louise Casey

Those figures are from the new ECORYS evaluate—the long evaluation. They are in the synthesis report. In a way, that is what is quite interesting because—

Chair

We have a lot of documents.

Chris Evans

In the evaluation, is there any further breakdown? I noticed that 71% of families have a health problem. Is there anything on disabilities that could be shared with the Committee?

Joe Tuke

In the new programme we will have a much better idea of that, because we are looking at a broader range of issues and collecting more information. We have information on a number of health problems—for instance, around people’s mental health issues. We know that over half of the households in the cohort and in the new Troubled Families programme so far have reported mental health problems. We will have a much richer amount of data to play back to local authorities, as Melanie said, every six months, and will report that publicly as well.

Chris Evans

Before I move on to Phase Two—sorry, did you want to come in?

Melanie Dawes

Sorry to interrupt. I think you were asking whether we should have intervened earlier. One of the things that local authorities certainly said about the first programme was how many more families there were who they felt needed help and who might not be quite at the stage that was needed for the first programme. That is one of the reasons why we broadened the criteria for the second programme. The overall case load is expected to be a lot bigger because we are dealing with a broader range of families slightly earlier in the process.

Chris Evans

Is this programme dependent on how well the caseworker does their job? It seems to be very much dependent on the relationship between the caseworker and the troubled families. As we have seen in the probation service, if the probation officer is overworked there is a tendency for things to go wrong. Does the programme win or lose based on that relationship?

Joe Tuke

That is the most important relationship: that between the key worker and the family. What is different about this is that it is not just a probation officer who is working with an individual member of the family who perhaps has an offending history. Here we have got a dedicated family key worker who deals with all the family and can see the interconnectedness of the problems and can see that the offending may have a link to the domestic violence that the parent is experiencing, or may have a link to mental health problems or whatever it might be. It is that key worker who has the most important relationship. There is quite a lot in the evaluation. I am sure that Members talk to their own local authorities and get to meet some of these key workers; there are some great examples of public servants who have got a great attitude of persistent, assertive working with families who have shunned services in the past. They create a trusting relationship with the family—a sort of “tough love” approach. The evaluation shows how that practice, which we encouraged through the first programme, really showed improvements over the course of the three years of that programme. I think we are seeing that again. We put out there, in the first programme, about these are the key characteristics—

Chair

You talk about some very good public servants, but are you sure, and do you do any evaluation, or do you require local authorities to look at the quality of key workers, casework leaders, in those situations?

Joe Tuke

We do not mandate the approach to them. We encourage that, but we pass and share good practice across the piece, and we know that, as they do in social care, they will have supervision in place around that. Ofsted, of course, comes in and looks at children’s services departments.

Chair

Ofsted would evaluate these examples?

Joe Tuke

Ofsted will come in and look at children’s services departments, which would include early help, which generally is where family workers are.

Chair

I am sure Chris Evans is going to pick up on this, but it seems like there is a different set of professionals doing the same role, so they would be monitored and evaluated differently.

Melanie Dawes

One of the things that the evaluation brings out is that one of the positive impacts of the programme is that there has been an investment in a new workforce here. There were some family workers there before, but this has been a very big increase. As for many other areas of local government services, we do not specify; we inspect. We were relying here on the payment by results outcomes framework to make sure they were getting results.

Chair

Before I bring Chris Evans back in, the Comptroller and Auditor General wants to come in.

Sir Amyas Morse

I just wanted to ask something, looking forward. These reports could not show a link through the national data sets between the programme and the improvements that they had observed in troubled families. I am trying to say that carefully, to try to get it right.

Melanie Dawes

That is correct.

Sir Amyas Morse

Thank you. Do you expect that it will be possible to demonstrate such a link in future? It is quite important to get that straight. If you cannot answer it now I sense you are going to need to answer it at some point quite soon; otherwise we could be off on a set of inappropriate expectations again.

Melanie Dawes

I think for me, to be honest, that is the six million dollar question. We have—and perhaps we will get into this—so many difficulties with the data, and with this question of what is the counterfactual that you are assuming would otherwise have happened with these families, that it is always going to be very hard to prove beyond statistical doubt, which is what the evaluation was seeking to do, that it is the programme itself that has led to the improvement. Personally, I think we do need to be realistic about whether this quite complex national data set will show that, or when it will show it, but we are committed to carrying on trying. I think one of the big legacies of this programme, which the LGA have brought out in their submission to you, is the investment that has been made in data, which is unprecedented—joined-up local service data at a local and national level.

Dame Louise Casey

In terms of your question, Mr Evans—about family intervention workers and whether you can guarantee that if those workers are superb they can help those families transform—first of all, one of the learnings for me would be that I think we need to institutionalise what we think family intervention actually is at a much more formal level. It has been my belief ever since I went to the Dundee homeless families project in Scotland, when I worked at Shelter, so at least 25 years ago, that if you look at what those workers do and you try to codify and explain it, you would have—we have a legion of them now, but we have done it through inspiration. We wrote something called “Working with Troubled Families” where we said there are five factors of family intervention. I can list them now, but I won’t. They will be in endless bits of paper. They essentially said it is all about the relationship between the worker and the family. They cannot just deal with one individual in that family; they have to deal with them all. They are dedicated to those families. They do odd things; rather than assessing they might order a skip, so you get the rubbish cleared out of the house. It is a very interesting way, I think, of working with families that we started in the Respect action plan with the family intervention project, and that we have been trying to promote through this programme.

Chair

And which organisations such as Family Action, based in Hackney, have done for a while.

Dame Louise Casey

Family Action in your constituency, Action for Children in Ms Flint’s constituency, there is another version in Bristol—it is quite interesting.

Chair

We are in danger of getting—we are not a policy committee, so I am sure our sister Committee will be interested in this.

Dame Louise Casey

But it is value for money writ large, Chairman.

Chair

Fantastic. We will take that on board. I think you have made the point.

Chris Evans

This is my last question before I move on to Phase Two. Figure 10 shows the cost and saving per family. These figures are presented as gross, and there is the claim that the Government have estimated that they will have saved £1.2 billion by March 2015. There was an assumption—the average gross saving was £11,200 across seven exemplar authorities. Can you explain how the seven exemplar authorities were chosen? Do you think the sample was too small—I know it has gone higher now? Why were they not just presented as nets? That is quite a simple question really.

Melanie Dawes

I will ask Joe to comment on this perhaps. This is the same work that Joe was describing earlier that we did with local authorities and that was so new for them. That was partly why we had to build it up, with those that were able to, in a more incremental way. I want to emphasise that these figures are indeed gross. They are not net figures and we are not claiming that they are net savings. We are simply presenting them as they are. In order to say that they are net, we would need to be much clearer than we are able to be about the counterfactual. That point actually lies behind a lot of the issues we had with the evaluation and it is intrinsic to the nature of what we are trying to do here. I do not think it is a failure of the evaluation; it just makes it difficult actually being able to say what would otherwise have happened to families. That is why we cannot say for sure that this a net saving to the Exchequer. If we had them in the room, local authorities would say that they thought there were net savings, but it is very hard for us to prove that.

Chris Evans

So you are saying that the figures should never have been presented anyway, because they are meaningless?

Melanie Dawes

We would rather give you the data that we have, but I absolutely do not want to over-claim for it; I want to be absolutely clear about that.

Chris Evans

We are all politicians around this table, and if there is an estimated net saving of £1.2 billion, the Government are going to be okay with that, so really that figure is just worthless.

Melanie Dawes

It is not worthless; it is what it is. Some of the data that lies behind it is in this table, and some is in the cost-benefit report that we published on Monday. As that report says in its summary—as I just described—it is a gross estimate drawn from local authorities that are working with this data for the first time. It is quite new. We think it is valuable, but I do not want to over-claim for it.

Chris Evans

It is not the first time that has been used. Even the then Prime Minister was saying that you had helped 116,654 families and that therefore the programme was a success. Where did you arrive at that? I am looking at paragraph 1.21, which says, “The data showed that a total of 116,654 families eligible for the programme had met the criterion” of reducing crime, and the Prime Minister hailed this as a success. That seems a massive figure. You have helped 116,654 out of the 117,910 families you identified. Where does that figure come from? To be honest, if that is what you have turned around, you should be running every single Government Department.

Dame Louise Casey

Let us be clear that the figure you are referring to is the figure for the number of families who met the stated objectives of the programme, and for which the local authority received not only an attachment fee—to use payment-by-results language—but the outcome fee. It was suggested in the financial framework that local authorities reached more families, although we would not pay them for them, in order both to reach that and as part of a system change. I don’t think they did masses more, but our sense around the country is that people did more in order to meet that. We need to be clear: we know that 116,654 families basically had their lives changed by the programme according to the payment-by-results system. That is a 99% success rate. You could ask, given where we are, why it was not a full 100%, because they could actually reach out to more families in order to meet that. But I would have probably lived with that as the person responsible for the programme. If I may just comment on the money at the moment; I was responsible for the Troubled Families programme at the point we published that report. This programme was managed a lot by hearts and minds. We wanted the system to change. I wanted people to feel part of something that was pioneering—doing things differently—and was about implementation. I was meeting families on a daily basis. I did two or three visits a week, and I would meet families that I knew had police call-outs up to their eyeballs. Frankly, some of their neighbours would talk to me on the way in and say, “Are you going to bloody get rid of them?” And yet, during the course of the programme, it was clear that their children were in school, they weren’t down the doctors, they weren’t doing police call-outs. Somewhere in there, there has to be a cost saving. Scientifically, you don’t have counterfactuals, but it was part of saying, “Come on, everyone. We can do this.” Part of it is saying, “There must be cost savings in this.” It is also about saying to the public sector, “You have to shine a light on your money.” You can’t just say, “Let’s spend money on families.” You have to say, “So what are you saving?” I think it is the most difficult part of the whole evaluation, and as the ECORYS report shows, we have been unable in the final furlong of this particular evaluation of the first Troubled Families programme to sit before this Committee with the type of information we would want. I think that will be very difficult for the future, because I don’t see a country where I put some people with absolutely nothing in a corner, where nobody touches them at all and they are not involved in any—

Chair

Okay. We are not suggesting—

Dame Louise Casey

It is quite important that that is a lesson for the future—we cannot work our way through that easily.

Chair

We are going to talk further about evaluation.

Chris Evans

I have one short question before we move on to Phase Two. Paragraph 1.19 of the Report says: “The Department checked 2,367 claims in total and found 115 of these claims to be invalid.” There was a further review process that led to a further 214 invalid claims being identified. This is quite simple. What do you mean by invalid claims?

Joe Tuke

That could be something like an administrative error. If you are claiming for a result, you have to make sure, for instance, that all the children in the family are attending school above a certain threshold. It may be that when we looked at that claim, three of the children were and one of them was just below that threshold, and it was a mistake in the claim they had made. We might then say, “Withdraw that claim.”

Chris Evans

So just to clear things up, there are no instances of fraud or anything like that.

Joe Tuke

No instances of fraud.

Bridget Phillipson

To go back to figure 10 about the cost saving and the way the data have been used, Ms Dawes, you seem quite relaxed about extrapolating from seven local authorities much wider savings, yet your attitude towards the independent evaluation and the data issues was perhaps a bit harder.

Melanie Dawes

I wouldn’t want to overstate the issues we had with the National Institute element of the research. It is really important for me to emphasise that. That piece of the research was cutting edge. It was experimental. It was, by definition, a statistical research programme, and those programmes have to run to certain standards, so it was looking for quite hard-edged results. To do that kind of work with the data uncertainty that this programme inevitably was subject to, given that it was so new even to be collecting the data in the first place, means that the job of the researchers was really difficult. That was the issue we had about how you then interpret the results. What I am saying about our own piece of analysis is that I don’t want to claim too much from it. It is what it is. I think it is useful, but we are not resting big numbers on it.

Bridget Phillipson

The Government did claim that £1.2 billion was saved as a result, on the basis of data from seven local authorities. On the one hand, the Government were quite relaxed when the data backed up a pre-existing belief, but not so relaxed when independent research called into question some assumptions about the effectiveness of the programme.

Melanie Dawes

We are certainly not claiming any definitive cost savings from the programme, because we have not been able to do the counterfactual analysis that would put those figures—

Chair

I think Ministers have.

Bridget Phillipson

The Report we have says: “The government used this data to claim that the programme had already saved taxpayers an estimated £1.2 billion by March 2015. This assumed that the average gross saving…in the seven ‘exemplar’ local authorities would be the same” and so on. The Government have claimed that they saved more than £1 billion.

Joe Tuke

That Report was early on in the programme. It makes it really clear that these are gross savings, and there are whole annexes about the caveats, saying that that is the case and we were not able to have a proper counterfactual for it. So the Report itself is very clear that these are gross savings. The extrapolation was done from the seven areas as an indicative amount. The big thing about that piece of work was, as Dame Louise was trying to suggest, that we were trying to encourage local authorities to go through a more rigorous process of identifying the costs and benefits of their programme, partly so that they could engage partners in this and we could get more integrated services. It was really hard—we were only able to get seven authorities, at that very early stage in the programme, to complete the cost savings calculated to a reasonable standard. Since then, we have done more work, as we have talked about, and the report that we published on Monday does that for 67 local authorities and still talks about gross savings because of the difficulties with the counterfactual.

Bridget Phillipson

Doesn’t making grandiose claims about the effectiveness of programmes undermine the whole concept of some of this work? We all want to see families turned around and help and support given where there are difficulties in people’s lives, but if Ministers go out there claiming that it saves £1.2 billion and clearly that cannot be evidenced, that undermines the whole basis of that social policy intervention.

Melanie Dawes

I am confident that the department’s report was as Joe described—the caveats were in the report. Then there is just the question of how those results are expressed. But the results in the report were clear, and I think the caveats were there.

Chris Evans

Mr Tuke, I want to move on to you and focus on Phase Two of the programme. When you took over this programme, what was your impression of it? What had been done well and what had gone wrong? I want you to be totally honest.

Joe Tuke

I should explain first that I was working on the first Troubled Families programme with Louise as well, so I cannot claim I came in new to this. When the new Troubled Families programme was announced, we had time to go and talk to local authorities about their experiences of it and how they would like to improve or expand on it if we had a second phase. The big thing they wanted to do was to get more overtly to a broader set of problems. So although we know that in the first programme families had on average seven significant problems, the actual eligibility criteria were more restrictive, as we have learnt. In this case, we are saying that actually the big problems are around violence and health and getting to younger children, because in the first programme mainly it was families with school-age children because of the criteria. So it was getting to those children. And, more overtly, getting to those children who were considered to be needing help in some way: perhaps children in need etc. So it was a broader set of problems that we were able to look at, and that gets us into a bigger cohort as well. I think the most important thing was allowing local areas to identify the families they were really confident were of most concern and cost them most in their local area.

Dame Louise Casey

Can I add something to that? The other thing that I felt we wanted the next programme to include was longer follow-up. Part of the problem is that once people claimed the results, we knew that most of them were continuing to work with those families. We have individual examples of that around the country. In the second programme—the roll-out programme or expanded programme—they are going to track those families for a much longer period of time. That is the right thing to do, getting to the under-fives, getting to domestic violence and tracking families longer—

Chair

You have anticipated some of our questioning on that.

Chris Evans

Is it a learning outcome of Phase One that there has been too much of a drive towards short-termism rather than long-termism and that really what will show whether the programme fails or succeeds is what is happening in 10 years’ time rather than in 10 months’ time?

Dame Louise Casey

I personally feel that that is an unfair characterisation. It would be unfair to say that we were chasing short-termism over long-termism. If we were doing that, we would have concentrated on the PBR and we would not have gone for wholesale system reform, we would not have pushed reform of family intervention and we would not have done lots of things that we did during the course of that programme.

Chair

Dame Louise, it is not unfair for Mr Evans to raise that when all the measurements of a turned-around family are all short-term measures. That is what I think Mr Evans was trying to say. You cannot disagree with that.

Dame Louise Casey

Indeed, I do not dispute that. I would just caution that in order to meet the criterion on education a child had to be in school for a full school year. When we use the phrase “short-termism”, people think weeks and months. In order to claim that you have succeeded with that child, it is over a year. I am just being honest and saying I wish we could track them for longer, that’s all.

Chris Evans

That is the point I was raising. If you look at the target criteria, they are being in a work programme; being in work for six months and being in school for an entire school year. If we look at examples such as my earlier question on educational attainment, how many GCSEs does a student pass at the age of 16 when you have intervened at the age of six? Those are the types of things I was talking about. Are those learning outcomes from Phase One?

Joe Tuke

They certainly are. We will be getting a much broader set of outcome information. Both at local level, the plans which key workers have with their families are across a broader range, so they have outcomes that they set against each of those six problems that the programme is tackling. We are tracking that at national level through our evaluation as well. What is interesting about the first programme is that, although the outcomes we were measuring have a more limited timescale through the payment by results, when you are out and talking to local areas, for them, yes, there is a time when they can claim a payment, but that is an artificial point in their work with a family. They carry on working with those families if they need to and then they step them down. The evaluation talks about excellent stepping down and exit arrangements to ensure that the support is sustained—sometimes by voluntary organisations, et cetera. One of the districts, Warrington, for instance, has tracked its families who claimed payment by results from the first programme. Only 3% have had to be referred back into services for some significant levels of support. Although the outcomes set in the first programme may, in your terms, Mr Evans, be more short term, I think the actual practice was that they continued to be supported and the improvements sustained. We will be measuring that more in the new programme.

Chris Evans

So there is an exit from the programme at some point?

Joe Tuke

Certainly, local authorities want to be able to step people down to a lower level; they want to reduce dependence on the service. That is the whole point.

Chris Evans

My question on all this section of the report—I remember when I was a candidate in Cheltenham in 2005 and knocking doors and people were going on about—

Chair

No anecdotal evidence.

Chris Evans

It is not quite anecdotal. When I was knocking on the doors and people were talking about antisocial behaviour, I was talking about Tony Blair’s respect agenda. Then I remember a speech by Gordon Brown when he was talking about individual intervention. This is not something new; it is not a new concept, as Dame Louise said earlier. Is there not a concern that you are giving money to local authorities to do something that they are already doing anyway? Is there a danger there?

Melanie Dawes

There is a perfectly fair question here about what some people call “deadweight”. Was all this going on already? I think the evaluation shows that what was really different and what was achieved through Troubled Families was a very different way of working. It may well be that some of the agencies were working with the families already, but they were not doing that in a joined-up way that was around the family rather than the individuals in the family. The bit of the evaluation that looks at process is very clear that this is quite a transformational new set of services that is now going on in local government with its partner agencies. The other thing that is very new that was not happening before is the focus on outcomes. Again, I think it is a very question for us as to whether a PBR framework focuses you too much on the short term.

Chair

PBR is payment by results.

Melanie Dawes

Yes, whether payment by results focuses you too much on the short term. In the end, the programme is trying to strike a balance. If you do not have any kind of outcome focus, it is really hard to galvanise this new way of working. Equally, you always have to make sure that it is not too narrow.

Chair

We should just reflect that we received evidence from the Local Government Association and others that talked about joined-up working. We also received evidence that suggested that it was not universal and that the work was a bit patchy in places.

Melanie Dawes

Yes, I think you see again the ECORYS synthesis report brings out that some local authorities are great at this—

Chair

We will perhaps touch on this a bit more in good practice sharing.

Melanie Dawes

Yes.

Chris Evans

That question was generated from looking at sections 1.6 and 1.7 of the report. The Troubled Families programme was based on the Dundee Families Project, as Dame Louise mentioned; 53 family intervention projects were set up in England from 2007, I assume as part of the Labour Government’s Respect agenda; and the community budget programme was from the coalition Government. So all this work has been going on for a very long time. That is where that question comes from. What makes this different?

Dame Louise Casey

As, unfortunately, the person who has been involved with all the ones that you just mentioned prior to this one, I think that the difference with this one is that it is part of every single local authority in England, so it is not about specific projects in specific areas. Yes, we set up 53 family intervention projects under the Respect campaign. They were lovely projects but they were boutique projects, and when they lost favour, they more or less disappeared. People like me and Joe—because we had lived through that era—were determined to ensure that this programme was much more about changing the mainstream system, and trying to come at things such as family intervention and whole family in a very different way. You are right to ask, Mr Evans, whether—even having delivered the Troubled Families programme and being pretty happy with the fact that we changed the lives of 116,000 families—I think we have completely sorted out the way you might deal with these families. No, I do not. Do I think we got every single family that you referred to in your constituency? No, I do not. This is a learning process. It is really difficult when you are running social policy and a financial framework, when you are spending taxpayers’ money, which is entirely different from spending your own, and when you have to work through a system in a different way.

Chris Evans

The crux of the whole thing is, are you really saying that a family is turned around simply because they manage to qualify for a payment?

Melanie Dawes

What we are saying is that those outcomes were achieved in order for the payment to be made. That is very clear. From the Department’s perspective, it is a question of what you mean by “turned around”. I point to one element of the evaluation, which is in the overall synthesis report—

Chair

I am just going to bring Caroline Flint in on this.

Caroline Flint

Would you find it helpful if the phrase “turn around” was dropped?

Melanie Dawes

It is a question of the overall presentation of the programme.

Dame Louise Casey

As the person no longer responsible, I think it was right for the time, Ms Flint. As the programme goes forward, it will be better, particularly because the new programme has a much broader range of indices where you are measuring—

Chair

Are you saying yes or no, Dame Louise?

Dame Louise Casey

I am saying, yes. It would probably be helpful to think about what you called it next. It worked at the time. I would not use it in the future.

Chair

Melanie Dawes?

Melanie Dawes

I just want to be a bureaucrat and give you some statistics, if you don’t mind.

Chair

I’ll indulge you.

Melanie Dawes

One of the things that I find most striking about the evaluation is that families report a 17 percentage point improvement in answer to the question that they are confident that their worst problems are behind them. That is quite interesting. I am not saying that means “turned around”. I am just saying that families—

Caroline Flint

The phrase “turned around” has a certain connotation. We need to be careful that, despite very good work that is happening on the ground, it does not lead to complacency about what families will still need.