Oral evidence: The Government’s Management of Major Projects, HC 1631

Oral evidence, 26 November 2018.

Chair

It is a great pleasure to welcome our witness to this oral evidence session in our inquiry into the management of major projects. Could I invite our witness to identify himself for the record, please?

Professor Flyvbjerg

My name is Bent Flyvbjerg. I am the BT Professor of major programme management at the University of Oxford’s Saïd Business School.

Chair

We are extremely grateful for your being with us. I apologise, on behalf of myself, as well as the Committee, for this late start to our proceedings, since there is other business in the Chamber that has been preoccupying us. We have 18 questions to get through. I am sure you are capable of talking at length for 55 minutes on each one of them, but we would like to get through as quickly as possible. We will ask short questions and if we can have to the point answers that would help us enormously. Thank you very much. To start with, you refer to what you call an “iron law” that megaprojects are “over budget, over time, under benefits, over and over again”. What is the evidence that these properties are really universal to major projects?

Professor Flyvbjerg

The evidence is there from thousands of projects around the world, from 103 countries on six continents. The only continent we don’t have data for is Antarctica. Those data show, with an unusually high level of statistical significance, that that law applies. It is not a deterministic law in the sense that each and every major project follows this law, but the majority of projects do and, as I said, with a very high level of statistical significance. It is a probabilistic law: if you have to make a prediction and you are going to do a major project—all you know about what you are doing is that you are doing a major project—you should assume that it will be over budget, over time and under benefits. That is what the data show.

Kelvin Hopkins

I am very sympathetic to your thesis, but could you say if any projects have proved notable exceptions to your iron law?

Professor Flyvbjerg

Yes, there are notable exceptions. Like I said, it is not a deterministic law so there are successes. One out of 10 major projects, for instance, is on budget. Of course, we are very interested in those projects and understanding whether they were just lucky or whether they have a methodology that makes it possible for them to reproduce that result of being on budget over and over again. We find that there are projects and there are project directors and organisations that are able to do this over and over. We try to learn from them so that we can reproduce their skills and knowledge on other projects.

Kelvin Hopkins

You may have read the recent analysis comparing the electrification of East Coast Main Line by British Rail 30 years ago and the cost of electrification now, which is seven times higher in real terms. What are the characteristics that have made that massive difference?

Professor Flyvbjerg

Construction is very expensive in the UK and in other countries. The UK is comparatively more expensive than most countries, so today it is just more expensive to build things than it used to be for all sorts of reasons. One reason is that the rules and regulations have become much more developed so there are many more hoops that you have to jump through. Another thing is that productivity has not gone up in the construction sector as it has in other sectors of the economy, so manufacturing, even agriculture, services and so on, has had large productivity increases whereas construction has not. Of course, that again will mean that it is more expensive to build things because productivity is low.

Kelvin Hopkins

A seven times escalation in real terms over 30 years is still extraordinary.

Professor Flyvbjerg

It is extraordinary. I agree with that.

Mr David Jones

Professor Flyvbjerg, you mention that you have a large amount of data from around the world. How would you say that the UK’s management of major projects ranks among similar countries?

Professor Flyvbjerg

This is something that we have studied. It seems that all the countries that we work with and visit have the feeling that they are the worst. They are bad and other countries are better. I can reassure you it is not the case for Britain and not the case for most other countries. It is astonishing to us to see that countries are pretty much the same. There is one difference and that is between emerging economies and developed economies. We find that budget overruns and so on are larger in emerging economies and we take that to be because of more difficult logistics. It is more difficult to build things in emerging economies because of more difficult logistics and there is also more corruption. That is an additional cost and an additional risk. Apart from that we find very few geographical differences, so my answer is that the UK is just as good or just as bad as other countries however you wish to formulate it.

Mr David Jones

Are there any other countries that demonstrate better management of major projects than the UK on a consistent basis?

Professor Flyvbjerg

We did a special study for the Netherlands and we found that the Netherlands is better at delivering major projects in the rail sector than the UK and other countries, not just the UK but countries other than the Netherlands. We have found other examples like that. We found that Hong Kong is better at doing roads than other geographies. There are small differences like that.

Mr David Jones

Is there anything that we can learn from the Netherlands and Hong Kong?

Professor Flyvbjerg

Yes, I think there is. All countries can learn from other countries. One of the things that is happening is learning from other countries. Even learning from other projects within your own country is something that would be useful, especially learning from projects that do exist—like we talked about earlier—that are successful and do not submit to the iron law. Those are the projects that we can learn from. We can find them in other countries but we can also find them here in the UK.

Mr David Jones

For example, why are the Netherlands so good at rail projects?

Professor Flyvbjerg

Our explanation of that is that the Netherlands has a very long planning tradition because of its geographical situation on the North Sea. It has had to protect the country from flooding for ages. This has developed a very rigorous planning culture because it was like a survival thing. It has had disasters where lots of people drowned because the dykes were breached and large parts of the Netherlands flooded. That has made the Netherlands a little more rigorous in its infrastructure planning than other countries. That is how we explain it.

Mr David Jones

Are there similar reasons in Hong Kong, which again has geographical and topographical challenges?

Professor Flyvbjerg

Hong Kong has built so many major projects in quick succession that it has been able to do what in most places you are not able to do, which is to learn very quickly from one project to the next. Major projects happen fairly rarely in most geographical areas. There are many years in between so it is difficult to get a really effective learning process, but Hong Kong has had an effective learning process.

Kelvin Hopkins

Did you look specifically in the Netherlands at the Betuweroute, the dedicated rail freight line from Rotterdam to the Ruhr?

Professor Flyvbjerg

Yes. That is one of the projects we studied in great detail. I can refer you to publications that we have done about that.

Chair

Can I ask about the example of the Channel Tunnel, which you refer to in your evidence? The construction costs were 80% over budget and the financing costs were 140% over budget. Then you say, “However convenient for users—who are heavily subsidised—the Channel Tunnel detracts from the economy rather than adding to it”. How do you measure that? How do you establish that?

Professor Flyvbjerg

You measure it by doing a cost benefit analysis where you include all the costs and all the benefits for the British economy. This is a study for the British economy not including the French economy where you would find the same if you did it. That is how you do it. It is very difficult to understand that result when you actually use the service, as I am sure many, if not all, in this room have done. It is very convenient and effective and to be on it and think of it as a failure is very difficult. That just goes to show you can have major projects that technologically and functionally are very successful, as I would say the Channel Tunnel or Eurotunnel is. However, financially and economically it is a disaster. Ask the original investors what they think about the Channel Tunnel—okay so you know what I am talking about.

Chair

I should declare an interest from a very long time ago. I was one of the initial subscribers to the start-up share capital of that in a very small way and I think I got one free trip. In the popular imagination, my adage for major projects is: can you think of one piece of major infrastructure that should not have been built, apart from the Humber Bridge? The Jubilee Line extension, the M25, the Channel Tunnel are very good examples where people think, “No, we could not possibly have done without the Channel Tunnel, the Jubilee Line” and so on, or a very expensive piece of road, the Limehouse Link, the most expensive piece of road in British history. These suddenly become indispensable to people’s lives. How do you measure that in your cost benefit analysis?

Professor Flyvbjerg

That is easy. You just measure how much it is worth for people. The right way to look at it, for something like the Channel Tunnel or any other piece of infrastructure is: is this the best way to spend the money? I would say that if you did an analysis like that and asked yourself what would the best way have been for the UK, hypothetically speaking, to spend £8 billion to £9 billion at that stage in history, which would have made the biggest contribution to the economy, it would not have been the Channel Tunnel. No matter how happy we all are about it being here today, that would not have been the most effective way to use the money for the British economy at that stage. That is what the numbers show.

Chair

Then one concludes that the numbers must be wrong.

Professor Flyvbjerg

This is something that we call hindsight bias. Also, there is a survivorship bias. When you say, “Is there any piece of infrastructure that should not have been built?” you only look at the pieces of infrastructure that are still here. There are lots of pieces of infrastructure that should not have been built, which nobody is thinking about today because they are not in use or they have fallen out of use a long time ago, but people invested in them and did not make a return. There are lots of examples of that.

Chair

The psychology is different from the economic analysis?

Professor Flyvbjerg

Yes, very much so.

Chair

Because infrastructure investment is politically quite popular.

Professor Flyvbjerg

Yes, it is.

Chair

Are you suggesting that that is a mistake?

Professor Flyvbjerg

No, I am not suggesting it is a mistake. I am suggesting that if you want to focus on building the infrastructure that gives the biggest return for the economy, you need to be very careful about what you build and give due consideration. Political popularity is not enough. Creating jobs while you are building the piece of infrastructure is not enough. You need to look at how it affects the whole economy over the whole lifetime of the piece of infrastructure.

Ronnie Cowan

I have to declare an interest here of 35 years working in IT. You said, “If a major project is not already messed up, injecting a good dose of IT will do the job”. I am not disagreeing with you. Why? What happens?

Professor Flyvbjerg

We find that IT is the most difficult type of major project to do. It has the worst performance measured as cost overruns, delays, benefits and so forth. We figure that one reason is that it is the youngest type of infrastructure, so there has been less time to learn how to do it well. That is one reason. There is another reason that might be even more important. That is that IT is intangible. It is very difficult for people to have a spatial image in their head of what it is that we are building. It turns out that, when people cannot picture in their heads what it is that they are building and they do not have a clear image of what it is that they are building, it is more difficult and more risky. Take a house or bridge, physical infrastructure, as an example. You can walk around the house; you can walk into the house; you can view it from different angles. You can walk on the bridge; you can drive across the bridge. We have a clear idea of what it is that we are doing when we are building a bridge, a house or a similar piece of physical infrastructure. That is not the case when we are building IT systems, so it is much more difficult to picture what it is. We find that that makes it more difficult to succeed with IT projects. People simply cannot picture what it is that they are doing and so that creates risks in and of itself for budget overruns and delays.

Ronnie Cowan

I would agree it is still new compared to the other construction measures you are talking about. It is also changing all the time. There are two sets of changing: people are learning and relearning constantly, but we have been building major IT projects for at least 50 or 60 years. Have we learned anything?

Professor Flyvbjerg

Yes. It has got better. It used to be the wild, wild west and now it is only the wild west.

Ronnie Cowan

Are you saying it is a cowboy?

Professor Flyvbjerg

It has got better but you are right, there is an additional thing. The technological side is so fast and it is accelerating. Technology changes every two or three years, so if you have a longer planning horizon than two or three years you can be pretty sure that the technology is going to be different at the end of the project compared to the start and, of course, that creates major risk in its own right.

Ronnie Cowan

What do we do about it?

Professor Flyvbjerg

What do we do about it?

Ronnie Cowan

We cannot stop the advance of technology.

Professor Flyvbjerg

No. The problem is that IT is unstoppable right now and it is being inserted in every other type of project. You have a larger and larger element of IT in all projects. Even something that sounds as physical as tunnelling has a large element of IT now, with safety systems, signalling systems and so on, and it is growing. That means that it is getting riskier in terms of budgets and schedules. When you ask what do we do about it, one thing is that we need to learn from our mistakes. We need to understand how to do IT projects in smaller bits and be very reluctant to do something like the national NHS IT system, which is £8 billion to £12 billion. That is not the right size to do IT. We need to chop it up into smaller chunks to be able to do them faster so we have smaller technological risk in the technology changing but also so that the losses are smaller in case something goes wrong. There is a whole range of different things that you can do with IT in order to make them less risky and those are just some of them.

Ronnie Cowan

You say to make them smaller and, if you are looking at the NHS IT system, I agree, it is an absolute mess. But people want access to their records, wherever they are in the country at any given moment. It has to reach everybody and everybody has to be able to access the service, so it cannot be that small.

Professor Flyvbjerg

No. When I said smaller, I did not mean the final outcome. I meant the increments that you build it in. Instead of having one big bang project—I am sure you are familiar with big bang projects—I would recommend doing incremental projects so that you end up with the same size system but you just build it in much smaller and less risky increments.

Ronnie Cowan

Small chunks.

Professor Flyvbjerg

Yes. Exactly.

Ronnie Cowan

We touched on the Channel Tunnel earlier. Prior to any major infrastructure project, are there models available to us so that we can guesstimate the benefits and the downfalls of it?

Professor Flyvbjerg

Are you talking specifically about the benefits now?

Ronnie Cowan

Yes. Can we estimate them before a project even starts?

Professor Flyvbjerg

You can get a pretty good estimate but I would say that the benefits are the hardest part of any major project to estimate. The costs are difficult, schedules are difficult, benefits are way more difficult. The most important part you are talking about now is the best benefits estimate. That does not mean that it is impossible to get a sense of what we are going to get but there will be a lot of uncertainty on any number that you estimate regarding benefits. That is what our data show.

Ronnie Cowan

You have mentioned the four sublimes: political, technological, economics and aesthetics. Which drives the most mistakes?

Professor Flyvbjerg

Probably politics, I am sorry to say.

Ronnie Cowan

You use a phrase “rapture politicians”.

Professor Flyvbjerg

Yes. I once knew a politician, and it wasn’t in the UK I have to say. He said to me, “Bent, when I am telling my grandchildren about what I did while I was in Parliament, do you think that I would be most proud of some law that is sitting on the shelf in the Parliament library or to be able to drive across this beautiful bridge, one of the longest in the world, and throw out my arms, ‘This is what your granddad did while he was in Parliament’?” He thought the answer was self-evident. There is an attraction to doing something tangible and monumental and manifest. As it is not from your own money, sometimes accountability suffers. That is why we have committees like this one of course.

Ronnie Cowan

Do you have any particular project in mind that you think would fall into that category, a project currently going on in the UK?

Professor Flyvbjerg

I would not want to identify specific projects but there is a lot so it would not be difficult.

Dame Cheryl Gillan

HS2?

Professor Flyvbjerg

There is also the opposite. There are projects that are very utilitarian, very useful and delivered very well—including in the UK—so we shouldn’t go overboard.

Ronnie Cowan

The Scottish Parliament would probably be an example of a massive overspend, but it did take 300 years to build.

Professor Flyvbjerg

It had a 1,600% cost overrun.

Ronnie Cowan

It is worth every penny.

Chair

Would you make a distinction between rail and road projects in cost benefit analysis?

Professor Flyvbjerg

There is a distinction. The cost overruns are higher in rail projects than on road projects so I would say that rail projects are more risky than road projects, simply in cost and schedule and benefits. Benefit shortfalls are bigger on rail projects than on road projects.

Chair

Benefits are higher?

Professor Flyvbjerg

No. I am just talking about the risks now and not the outcomes.

Chair

Benefits tend to be overstated on rail projects.

Professor Flyvbjerg

Exactly. That is the point I was making.

Chair

Are they understated on road projects?

Professor Flyvbjerg

On average, yes, but just a little, about 10%.

Kelvin Hopkins

A quick point on HS2 and forecasting benefits. Some two years ago apparently, PricewaterhouseCoopers prepared a report for the Government saying that HS2 would be a horrendously expensive white elephant. The report was suppressed and has recently come to light. What do you make of politicians who want to push these projects even when they are told they are a complete waste of money?

Professor Flyvbjerg

That certainly happens. I would not be able to comment specifically on that report and HS2, but that definitely happens. We have seen lots of projects that have been proposed and built that could not be justified in economic terms but politicians felt they could be justified in political terms. That is fair enough. It is both the privilege and right to Parliament of make decisions like that. They do not always have to align with economic concerns, so there is not necessarily anything dubious about it but that is the way it is.

Dame Cheryl Gillan

I was on the floor of the House waiting to be called by the Speaker, so I apologise for not being here at the beginning of the Committee session. Can I ask you about the benefits of a major project over time? Is it reasonable, when it has been nine years since a project was initiated, that the Government should have checked on a regular occasion throughout that nine or 10-year period whether the cost benefit analysis is still indicating good value for the taxpayer? Would you expect a Government to do that long before we got to the point of no return and the final pressing of the button on the project?

Professor Flyvbjerg

I would like the Government to do that but I would not expect it, in the sense that if I had to make a prediction of will they do it, they wouldn’t because it is very uncommon for the Government to do that. There is a strong tendency to look forward and not look back, in both government and business. There is not much difference between business and government in that sense. Very rarely do people stop and look back and assess, “Did we actually get the benefits that we set out to get from this project?” and then study whether it is the case or not. It would be lovely to have. I think it would be good decision making. It would also be wonderful for us scholars to have data like that but I have to say that, unfortunately, it does not happen very often.

Dame Cheryl Gillan

That would be the best case scenario to confirm that the Government are spending their money in the right place and learning, as time passes, from other projects or from their mistakes.

Professor Flyvbjerg

Yes, I agree.

Mr David Jones

As a matter of fact, how well are the benefits of projects evaluated once they are complete?

Professor Flyvbjerg

Rarely, but there are instances and we study everything we can find to see what the benefits are. We do have data on that in addition to the cost and schedule data. There are projects that come in greatly over benefits. They generate benefits way above what was predicted, and we have examples of that. It is not the typical situation but it does happen. There are projects that are on benefits and there are projects that are under benefits. There is a lot of variation. That is one thing. The predictions are very uncertain regarding benefits. There is also the thing called wider benefits, not just the narrow benefits of immediate cost and benefits of a project but wider benefits to employment, that generate employment, development in the region where it is built and so on. That has also been studied. One of the leading authorities in the world on that is right here in the UK at the University of Kent, Professor Roger Vickerman. He found that on average the wider benefits are 10% of the benefits of major projects and, again, it can vary a lot. Wider benefits are often exaggerated, according to Vickerman, in the sense that people use them to justify projects that cannot be justified on the narrow benefits.

Mr David Jones

That is even after the event, even after they have been completed?

Professor Flyvbjerg

This is all about after the event. Everything I am talking about now is after the event. Often people appeal to the wider benefits but, as a rule of thumb, you can count on about 10% and that is it.

Mr David Jones

How readily available is the data to academics to enable you to study the benefits, if any, of such projects?

Professor Flyvbjerg

They are not readily available. It is hard work to get to those data. I am very often envious of my colleagues who are studying unemployment, the GDP. They just write to the Office for National Statistics in whichever country they work and they get the statistics. We have to go out and mine every datapoint that we use in our study, so the answer is that it is very difficult. The data are not readily available but it does not mean that we cannot find some.

Mr David Jones

Are the Government willing and happy to release that sort of data to enable you to carry out these studies?

Professor Flyvbjerg

That varies a lot from project to project and Government Department to Government Department.

Mr David Jones

You mentioned the wider benefits. What sort of data would you rely upon to assess the wider benefits of a project?

Professor Flyvbjerg

For wider benefits you would look at, for instance, employment figures. You might look at the development property prices. If we are talking about a rail project or a road project or another piece of infrastructure, did it affect the property prices in the corridor it was built in? Those are the types of things you look at.

Mr David Jones

Given the difficulties you have already mentioned, is it the case that it may be very hard to properly capture all the benefits of individual projects?

Professor Flyvbjerg

Yes, that is the case. It is.

Kelvin Hopkins

The Government forecast that there will be £330 billion of benefits from investments of £83 billion in its Government Transformation and Service Delivery projects in the Government’s Major Projects Portfolio. In the light of previous experience, how realistic does this seem?

Professor Flyvbjerg

As I said earlier, benefits estimates are always highly uncertain. The first thing I would say is that that would be an uncertain number. In the best of cases it sounds like a really good return, but again, as a scholar, unfortunately I have to say very few people stop and turn around and look at the actual outcomes of projects like that. These are what we call change management projects, where you try to change the way things are done in government or, indeed, in business. There is a lot of that happening in government and business in the UK and around the world in these years. Like I said, unfortunately, very rarely do people turn around and study this and give us solid data to document that you actually get this type of return. Therefore, my answer is I would consider that number uncertain.

Kelvin Hopkins

Given that the figure of the benefits is four times the investment, it is a massive advance and anybody looking at that and believing it would think, “Well, let’s go ahead with all of them”, but in fact history proves that perhaps that figure is overblown.

Professor Flyvbjerg

There is definitely a risk of a benefit shortfall. The higher the estimated benefits are on paper, the higher the risk is of a benefit shortfall in reality of course. That is like physics, like the law of gravity. That is the way those two things are related. If I was presented with a number like that, I would ask: what is your evidence? Can you show me examples where you actually had four times the return on the investment benefits-wise? Then I would let the evidence convince me or the opposite.

Mr David Jones

To what extent would you say the Government are taking on board your iron law, if at all? It seems to me that if they were aware of the likelihood of projects turning out to be grossly over budget, they might be put off embarking upon them in the first place.

Professor Flyvbjerg

That is correct. I would say that the UK Government and several other Governments around the world are taking the iron law quite seriously. I am very encouraged to see that it is being taken seriously and that there are measures implemented in the UK and elsewhere to change things for the better. But you are right that there is this argument that if we knew the real costs and benefits nothing would ever get built. We hear this over and over. I disagree with that. I think there are lots of projects that if you knew the real cost and benefits you would want to build them. Those are exactly the projects we should be building. We should not be building projects just because they look good on paper. That is not a good reason to do projects.

Mr David Jones

You mention that the UK Government are taking this on board. What examples can you give of the Government deciding not to proceed with a project or amending a project because of their concern about the likely cost overruns and so on?

Professor Flyvbjerg

I am from Denmark originally. I moved to the UK in 2009 to take up this professorship that I am now in at the University of Oxford, but I worked with the UK Government before that, in the early 2000s, with the Department for Transport and the Treasury. In that period, it was a shock that, after we started studying cost overruns and so on, the UK Government stopped a number of tram projects. It is very rare to stop infrastructure projects. Once they are on the public agenda they go through. This was the first instance when I saw this in the UK, where we looked at the cost benefit analysis that had been done on paper—like the example that we just heard about, the change management projects—and then we ask: is that realistic? What do the numbers say? Then we recalculate. When the numbers show a lower benefit cost ratio, projects actually have been stopped. There are examples of specific tram projects I remember, from the first round here in the UK, that were stopped. It generated shock waves through the major projects community that that could actually happen because it wasn’t the case until then.

Kelvin Hopkins

Your preferred approach to assessing project costs and benefits involves drawing on comparisons with previous similar examples. I did quote one. How robust is the data to support such an approach?

Professor Flyvbjerg

It depends on the type of project you are looking at, but we have very robust data from everything in transportation infrastructure and very good data in energy and in IT, as we talked about earlier. We are expanding and refining the data all the time, so I would say that we have very valid and reliable data from most areas. If we come across a new area, which happens several times a year, where somebody asks us, “Can you help us with generating data in this new area?” we now know how to go about it, what the procedures are and where to look and so on to get relevant, reliable data.

Kelvin Hopkins

On the previous point about projects being cancelled, we have just had a nuclear power station cancelled. Personally, I agree with it but sometimes they do get cancelled even at that stage.

Professor Flyvbjerg

Yes. Nuclear power is a case in point right now, internationally.

Chair

How good are we at sharing information about major projects internationally? How comparable is the data?

Professor Flyvbjerg

The data we have are comparable. We make sure that we use the same standards for data collection as anywhere in the world so our data can be compared across geographies, including nations. I would say most countries are not that interested in what is going on in other countries. They will focus on what is happening in their own country, but as scholars, of course, we have to look at a lot of different countries and compare across nations.

Chair

Does it just tend to be universities or individual academics or are there international bodies that collate this information?

Professor Flyvbjerg

There is no international body that has this as its purpose, but an organisation like the World Bank has data on projects for many different countries where they finance projects. The United Nations would also have some data. The World Bank and similar development banks are probably your best bet if you wanted to look at international organisations with data.

Chair

Can we move on to the governance of major projects? The governance of major projects always seems to be very problematic. Why do you think this is?

Professor Flyvbjerg

One reason is lack of accountability. You need to have very strong accountability for very expensive projects like this. You also need to have accountability that is robust over time. It is a problem that many of the projects that we talk about here can take eight to 10 years or even longer to build whereas, as you know, the political cycle is four or five years in most countries. Most decision makers in politics will not think about the time horizon for themselves beyond four or five years. It can be very difficult to maintain and enforce accountability for something that has a life that is several times longer than the political lifecycle. That is one reason. Another reason is lack of talent. There is simply not enough talent around to get good people in every major project, so that also creates a problem. Turnover is also high on major projects. People don’t stay in the same position for long, so there is a loss of experience and knowledge on the individual project. I don't know if you would call that a lack of governance. In a way it is because the talent is not being managed in the way where you have the talent that you need to accomplish the project. These are some of the main reasons that you have difficult governance. The third reason is often that the business case for projects has not been developed to a very high standard. The quality of the business case is low, so you start the projects on the basis of not enough knowledge, not precise enough knowledge about what it is that you are going to do. That is another area where you could improve governance and where governance is indeed being improved in these years. I would say these are the main reasons for problems with governance.

Chair

What do you think the top three things are that a Minister or a senior official in a Government Department should be thinking of doing in order to establish good governance of a major project?

Professor Flyvbjerg

First, the Minister should say, “I will only start a project if I have a high quality business case, with and everything that goes into that”. That is a very good focus point, the business case. We also call it the front end. If you don’t have a properly planned project, no matter how good a team you get, they cannot deliver it. We have a very good example of that. World class teams really know and they prove, over and over, that they can deliver projects to time and to budget and so on, but they are given a very bad business case and they fail because it is impossible. That is my first thing, a high quality, realistic business case. The second thing is to hire a team that has tried to deliver the type of project that you want to do before. Get a team that know what they are doing and have a track record where they have proved that they can do this. No matter how good your business case is, if you don’t have a good enough team, it will not be implemented. Those two things are equally important because either of them not being fulfilled will derail the project. The third thing is the governance structure. You need to have an accountability structure around the business case and the team whereby they are being held accountable, whereby they are incentivised, whereby they are rewarded if they do what the Minister wants them to do and punished if they do not do what the Minister wants them to do.

Chair

So often we see people rewarded when they join and then they are rewarded to leave, and there does not seem to be much accountability in between. When you say having a good governance structure around it, what does a good governance structure look like?

Professor Flyvbjerg

A good governance structure, first of all, measures what is going on. You need to measure performance, so there is performance measurement and there are decisions about: what is going to happen if performance falls below a certain level? What is happening if they perform well? What is happening if they perform not so well? When do we escalate problems? That is often missing in the setup of major projects that—

Chair

How do you escalate problems?

Professor Flyvbjerg

How do you escalate and where do you escalate them? Who is responsible if this situation happens and how quickly are we going to accept that this problem exists before we escalate it? This is what we call a monitoring structure, where you monitor the project and you know what to do depending on what the performance data tell you. That is one of the things that has to be in place.

Dame Cheryl Gillan

How much are the problems with governance related to the structures that Governments set up to deliver these projects? I am thinking, for example, of the arm’s length body that is being contemplated for restoration and renewal of the Houses of Parliament and the arm’s length body, HS2 Limited, that has been set up to run HS2. Is that a good vehicle for good governance in your opinion?

Professor Flyvbjerg

I do think that the arm’s length principle is a good principle for projects like that. It is always good to have somebody who is in a position where they can give an independent review and independent assessment of what is going on. I don’t think it is enough but it is one important element.

Dame Cheryl Gillan

How are they able to give an independent review of what is going on when their very existence depends on turning it into reports?

Professor Flyvbjerg

Sorry, could you say the last part again?

Dame Cheryl Gillan

You say independent assessment is vital, but an arm’s length body is not going to turn in independent assessments on its performance, so how can that satisfy your test of good governance?

Professor Flyvbjerg

Well maybe I don’t understand the arm’s length principle in this case, because I would understand arm’s length to mean that you are independent and you would be able to give an independent report.

Dame Cheryl Gillan

You are given the money to run the project and you are told to get on with it, basically, so the accountability starts to break down then between the arm’s length body and the Department.

Professor Flyvbjerg

From the way you describe it it doesn’t sound like an effective setup, so what I would like to see there would be a truly independent body that could review what is going on and publish an independent report. I think that is one important element but, like I said, it is not enough. You need other measures of accountability to make things work, in addition to independent bodies like that.

Dame Cheryl Gillan

Have you done any studies as to what qualities and capabilities Government should have when delivering these major projects? What would be the ongoing skillset that you would expect to find within Government, bearing in mind that they have an arm’s length body over there supposedly doing the project for them?

Professor Flyvbjerg

We found that one of the important skills for Government to have and maintain in this case is contract negotiating skills. Many of these projects come down to the writing of the contract. Sooner or later, that is what is going to happen. It is all going to be set down in contracts. If the Government do not have the skills to sit at the table and negotiate the contract, and the type of knowledge on their side when they are negotiating their side of the contract, they are going to lose out in the design of that contract. Unfortunately, we see that happening often. With all the outsourcing that was happening during a certain period, the Government were hollowed out, so you did not have that kind of expertise in-house with Government. That meant that when you sat down to negotiate the contract your counterparts from the outside would be much stronger on their side of the table than you would be on your side of the table. That would come back to haunt you later when you delivered the project, because your side of the contract would not be as beneficial as their side of the contract to them. I would say that the contracting skill is a key skill. There are also skills of procurement. There is a lot of buying going on when you do projects that cost several hundred million pounds or more. Procurement skills, the commercial skills involved in organising the procurements, economically and in time so that it is effective, are additional skills that you need. Those are the two main things.

Dame Cheryl Gillan

Can I press you? You said that one of the problems was accountability, lack of talent and turnover. Would you agree that turnover at the board level for a major project should be minimised and also turnover at ministerial level? When you are in a project that has had five or six junior Ministers and four or five Secretaries of State, when you have had a part-time chairman who then abandons the project and directors come and go willy-nilly, would you see that as being a great weakness in this area?

Professor Flyvbjerg

That is a weakness, and I would agree with you that there needs to be stability. I would start at the level of the project director; stability at the level of the project director, stability at the level of the SRO—the senior responsible officer above the project director—stability at the permanent secretary level, and if there is a board of directors, some stability there. You can see how difficult that is but, if we look at the success of the project, any instability in that setup will produce risks for the project.

Dame Cheryl Gillan

That is a recipe for failure in other words?

Professor Flyvbjerg

The more instability you have in the project.

Chair

You mention the importance of getting the contract and the procurement right. What is your experience of looking at how Ministers and officials think if they get the contracts right everything will be okay. They don’t think about who is going to lead the people or how they will be led. On both sides of this contract you need to trust each other and understand each other, so contracts become a substitute for leadership. How do you prevent that?

Professor Flyvbjerg

You prevent it by not thinking the way you just described. It is not enough to get the contract right. The contract is just a starting point for delivery. If it is not right, you just create a problem all the way down the line of delivery. It is not enough to get the contract right, so it is a necessary but not sufficient condition. Once it is in place and it is good, you also need to have the talent that is able to deliver that contract. That is a completely different skill from being able to negotiate and sign the contract.

Chair

What skill is that?

Professor Flyvbjerg

That is the product delivery skill that we talked about before. That is hiring the team that actually knows how to deliver the project that is described in their contract. That is the second condition that I mentioned earlier as part of—

Chair

You have been very clear. Thank you.

Ronnie Cowan

We have touched on this. We talked about building teams, delivering projects, retaining staff, sharing data. Is it not the case that a senior politician who has gone from the Home Office to Health to Transport is in no way qualified to manage a multi-billion pound contract?

Professor Flyvbjerg

I would say yes in a society like the UK. In a democracy like the UK, that is the way things are set up and nobody is expecting the elected politician to actually run the project. An elected politician is expected to represent people but they will need professionals to do the professional stuff.

Ronnie Cowan

To retain staff, to retain the skills, to learn from previous mistakes, to build great teams, would we not be better off if we had a national infrastructure company that could handle these major projects?

Professor Flyvbjerg

That is a good question and my honest answer is that I don’t know. There are examples like this around the world and it is the way things used to be. There used to be departments for public works. There would be one department that would take care of that. We have left that somehow. There are some countries that still have corporations. The pro is that you would actually have an organisation that has the job of doing this, so it does it over and over and it would learn and gain experience and supposedly get better and better. The problem is: would you be able to hold that organisation to account? Those are the things that need to be considered.

Ronnie Cowan

Under the current system, these companies are coming and going. If you look at a major bridge construction, we will bring in electricians and people in charge of pouring the concrete, and there are structures of management all the way through that. The project may take four or five years to complete. When it is finished, they disband and disappear and all that knowledge falls away. That is why people move from company to company. In the networks that are built up during that project, people get to trust other people. They saw you working on the job and you were very good. They get poached and then you get the disruption of staff. Under a national infrastructure company—not perfect by any manner of means—a lot of these problems could be addressed.

Professor Flyvbjerg

I think you are right. If the company would work according to the ideals that you set out, if we could make it work like that, yes, it would be a good idea. The question is: what are the things that would get in the way of the company delivering that? That also needs to be considered. I think that is an analysis that would be needed to be made first before you could make a decision like that.

Kelvin Hopkins

A question about public and private. To what extent does it make a difference to the success of a major project if it is delivered by the public or private sector?

Professor Flyvbjerg

That is one of the things that we are studying right now to see whether the private sector is more effective, as is often assumed. So far, there is no indication that that is the case. This is difficult, whether it is the public sector or it is the private sector. That is what our data show. From the data we have, it is not like the private sector is better than the public sector in delivering major projects. They just don’t get in the news as much, so it seems to us that we do not hear as much about the private sector. We do from time to time, but that is only when the Airbus A380 goes wrong and the whole company is at risk then it gets out. If it is smaller things it doesn’t get out and we don’t hear about it in the news. I think that is the main difference between the public and private sector. There is more transparency for public projects and that is why we hear more about them and, therefore, we think they are doing worse than the private sector.

Kelvin Hopkins

I could ask a dozen questions about the trials. I shall avoid that. Look at the Betuweroute, which I have mentioned. When I was over there looking at the Betuweroute I said, “Who did this?” “The state.” It was publicly done, very efficiently done. Over here, Crossrail has gone horribly wrong at the last moment. It is going to be delayed at least a year. The electrical systems inside them don’t work. Are there problems in contracting, instability of staff, the fact that, as you say—remember your words—with not enough talent and loss of experience and knowledge, with constant changing around, with contracting and subcontracting, you lose that corporate memory, the long-term concern and also the highly skilled, long experienced engineers running things rather than young project managers who do not necessarily have any qualification in engineering? Are there differences?

Professor Flyvbjerg

There is a problem and if you look to Holland again, at the same time as Betuweroute was built they would be looking at HSL South, which is the high speed rail line south from Amsterdam to the border, to Belgium and to Brussels and on to Paris. That was done as a public-private partnership and then it went very wrong. If you just look at these two projects, in Holland you have two similar multi-billion euro projects, one in the public sector that was successful and one that was a private project that was not successful. It had large cost overruns and delays. However, I would not generalise on the basis of two projects. When we look at many more projects what we see is that both the public sector and the private sector have difficulty doing this.

Kelvin Hopkins

At a meeting in this building, one of the people in the railway industry said that British Rail works miracles on a pittance. Since then we have seen fragmentation, privatisation and horrendous rises in costs. Could we not make some serious comparisons between public sector and private sector, even in Britain?

Professor Flyvbjerg

I think so, yes. Like I said, we are very interested in this and we are studying it right now. I think it would be very useful to do something like that.

Kelvin Hopkins

I could pursue the theme but I will spare the Chairman.

Mr David Jones

We have touched on this already but I am sure you took a great interest in the circumstances surrounding the collapse of Carillion, and you may have possibly read the report that this Committee prepared on the collapse of Carillion. We identified in that report the need for Government to improve their contracting with third parties. How significant are those initial contracts in ensuring that a major project is delivered successfully?

Professor Flyvbjerg

They are crucial. If the contract is flawed the delivery will be flawed. That is how important it is. That is why I emphasise that as one of the two most important things to get right. It is the contract, and to have the contract writing skills in-house in Government is immensely important.

Mr David Jones

I think you mentioned in reply to an earlier question that you felt that skills had been hollowed out to a certain extent. Is that right?

Professor Flyvbjerg

Yes, that is correct. During a period, that happened, and the pendulum has swung back somewhat now. I think that everybody realised that you can’t have the Government as a hollow shell that does not have the talent in-house. It is impossible to do good governance and good government if you don’t have the talented people to write the right contracts and to do the right procurement and so on.

Ronnie Cowan

We touched earlier on the fact that some nuclear power plants are not going to get built. Are we locked in? Should we still be cancelling major projects after they get to a certain stage?

Professor Flyvbjerg

Like I mentioned earlier, it is very uncommon to cancel major projects, even in the early stages when they only exist on paper. Even when they exist only on paper, early in the decision-making process it seems like, once a major project has gone on some kind of agenda somewhere, it is very difficult to stop it. I would say that it would be better if more projects were stopped, especially at that stage. It is more difficult once construction has started. There are examples of projects that have been stopped after they had been started but it is rare, even rarer than projects on paper.

Ronnie Cowan

There is a lot of danger that at some point people will panic and think, “We have to cancel this project”. In actual fact, they have to see it through to the end.

Professor Flyvbjerg

Yes. There is something called sub-cost and this applies to major projects that, once you have done some investment of money and time, there is a point of no return and after that people find it too embarrassing to cancel things. Even if the cost benefit ratio indicates clearly this should be cancelled, it doesn’t happen. That is very common. It is not a British thing.

Ronnie Cowan

The Sydney Opera House?

Professor Flyvbjerg

The Sydney Opera House had a 1,400% cost overrun, so a little smaller than the Scottish Parliament.

Ronnie Cowan

Both beautiful buildings. What price do you put on art?

Professor Flyvbjerg

You talk to anybody in Australia or anywhere and they will say, “Hey, who cares about the cost overrun?” It is a huge success and, anyway, it is making tons of money for Sydney, for New South Wales and for Australia and bringing lots of joy to people, including myself. I have gone there several times to see it. It is an amazing building.

Ronnie Cowan

A Danish architect?

Professor Flyvbjerg

Danish architect, but can you mention one other building that that Danish architect has done?

Ronnie Cowan

Not much after that. That nearly killed him.

Professor Flyvbjerg

Exactly. Is that really an acceptable cost that—

Ronnie Cowan

That is my next question. It is those artists we are throwing on the fire.

Professor Flyvbjerg

Poor guy, but that is one of the costs that we need to take into consideration. I would actually prefer to have the Sydney Opera House and other similar buildings by the same architect. I don’t see why we should have such mismanagement that we need to kill the architects in the process.

Kelvin Hopkins

Is there a difference between culture, which has to be subsidised—and historically has always been subsidised and we pay for it—and something that is absolutely vital to our lives, which is transport, health service and whatever?

Professor Flyvbjerg

Yes, of course but I would say in both cases there is good management and bad management.

Chair

I am just thinking of projects of that nature that one could talk about. I am thinking about this building because it is going to be very expensive to refurbish this building, and how you put that into a cost benefit analysis. You cannot.

Professor Flyvbjerg

Of course you can.

Chair

It is a work of art.

Professor Flyvbjerg

You might decide, “No matter what the cost benefit analysis shows, we are going to do it”. Of course, that is the case with a building like this. There is no way you are going to say, “We are going to bulldoze it because the cost benefit analysis shows that it is not good economics”. That is what I mentioned earlier. It is the privilege of parliamentarians that they make those kinds of decisions. It is not all about economics.

Chair

Yes, it is not all about economics.

Dame Cheryl Gillan

I think when it comes to railways it is all about economics. As you referred to major projects that have been cancelled that you have studied, could you provide a written list for the Committee with the reasons for their cancellation and at what stage they were cancelled? That would be very helpful because obviously you have a vast resource of research. Can I move on to communications? To what extent does early stakeholder consultation and engagement impact on the success of major projects?

Professor Flyvbjerg

I think that historically there have been two different philosophies mainly on this point. One is let’s try to avoid consultation as much as we can and just get the project done, get on with it, and that has backfired historically. It turned out not to be a good idea. It turned out that you would have the consultation whether you wanted it or not, because the consultation would force itself on the project if you did not take it into account upfront. Therefore, we had the second model, which is to do it upfront, to make extensive consultation part and parcel of the project from early on. I would say that today good projects do this. They have extensive consultation from early on and all the way through. I think Crossrail is a good example of how to do that. There are many other projects following the same model. That is the model subscribed to by the project directors and CEOs and SROs who I talk to around the world, who I consider successful in delivering projects. They know there is no way around public consultation. If you go in and spend several billion pounds and something is going to affect a lot of people and the type of society we live in, they will fully expect to be heard and to be able to come up with grievances if they have any; not necessarily expecting that they get what they want but certainly that they will be heard. That is the main model in our type of society.

Dame Cheryl Gillan

Can I ask about the legislative process for major projects, for example on transport projects, where you have a hybrid Bill process where petitioners can come and explain how they are affected? Do you have any experience in looking at how effective that legislative process is in giving a voice to people who are directly affected by a major project?

Professor Flyvbjerg

We have not done any studies at Oxford of this, and I am not aware of other studies that have studied that in a systematic fashion where you can actually get an answer to whether it works or not or how well it works.

Dame Cheryl Gillan

It certainly does play into the communication process and it can—as with HS2—actually aggravate the communication process. What about ongoing communications during a major project? What we find is, even if the communications and the posting of communications on Government websites are up to date, often they contain heavy redactions, which are possibly unnecessary. One can understand commercial-in-confidence redactions but, in your experience, do you see other redactions? Is this not obfuscating some of the issues and problems that are arising with a project as it goes through?

Professor Flyvbjerg

Yes that happens and in some cases it is unavoidable because it might be commercial secrets. There are lots of things that can be put out there. In the ongoing process I find that it is really important that the project delivery organisation has people employed, and it is their job to make sure that everything is communicated out and people are heard during the delivery process too. That makes for smoother delivery than if you don’t do these things. Good project directors and CEOs of major projects know that and they do that.

Dame Cheryl Gillan

That should be built into the original contracts that Government let. This goes back to the expertise in drafting contracts and procurement. If you fail to do that you have already built in some issues with the communications programme around projects.

Professor Flyvbjerg

Yes.

Dame Cheryl Gillan

That is all from me.

Chair

A very interesting session. Will you tell us if you think there are other people like you who we should be taking evidence from?

Professor Flyvbjerg

I will. Let me think about it.

Chair

We are finding it quite hard to find people with the breadth of expertise and breadth of knowledge that you have, but we are very grateful to you for coming today. Are there any other questions?

Ronnie Cowan

Just to pick your brain while you are here. We talked about the Opera House. Did Frank Gehry ever have any major overruns?

Professor Flyvbjerg

Frank Gehry had a major cost overrun on the Walt Disney Concert Hall in Los Angeles. He compares it to Utzon, which is the name of the architect of the Sydney Opera House, and his experience in Sydney. Gehry felt that it almost killed his career, just like Utzon, so it was close to an Utzon experience and Frank Gehry decided that he never wanted that to happen again. It was just too risky. Many of the people who know how to deliver projects have had experiences like that. It is like entrepreneurs who failed and then they become successful. Some people who work with major projects have failed or been close to failure with one or more major projects and then they decide, “I can’t afford this because it might kill me” and they find out how to do it right. That is what happened with Gehry.

Chair

If you were writing our report and you could make three recommendations to our Government about how to improve major projects, what would be the top three?

Professor Flyvbjerg

It would be the three that I mentioned earlier.

Chair

With that, thank you very much indeed.

Professor Flyvbjerg

Thank you for having me. I really appreciate it.