The cost of public works

[Prince George]

A recent editorial ovor here about the plans to build a road tunnel under the waterfront (at a couple of billion dollar price-tag) mentioned the very interesting work of a Danish political/social scientist Bent Flyvberg. He's spent 20 years studying the outcome of public works projects and the incidence of significant cost blowouts. In the vast majority of cases, the costs are out by more than 30% and the resulting income (eg ridership on transit, tickets through the stadium) is far short of the projections. This article has most of the lowdown, choice quotes:

In a worldwide study of 258 rail, bridge and road projects over 70 years, Flyvbjerg found that nine of 10 went over budget. In another analysis, he showed that the average transit system carries less than half as many riders as preconstruction forecasts predicted. ... A decade later, by collecting data from 20 nations on five continents, Flyvbjerg had produced the first statistically significant analysis to show what [an economist at the US DOT] had argued with case studies: The vast majority of public works projects go drastically over budget and aren’t as well patronized as proponents claim. He also found that modelers didn’t seem to be improving their estimates over time; the scale of overruns remained relatively constant. Rail and highway projects are often the worst boondoggles, and they form the bulk of Flyvbjerg’s research. But other researchers have found dangerous overoptimism in all kinds of megaprojects.

The point there with the deviation not improving is particularly important to him, with 70 years worth of data available how are we continuing to be so consistently low?

Few in the field question the existence of faulty cost and need estimates for public works projects. But the experts fight bitterly over the causes for the consistently inaccurate projections and how to address them. Flyvbjerg’s opinions on the subject can be distilled to a single theme: The politicians are lying to us. ... Flyvbjerg suggests that two dynamics underpin the horribly inaccurate estimates for most megaprojects: Either politicians, planners and consultants are lying outright to get the projects funded, or they’re guilty of “optimism bias,” succumbing to the psychological tendency to overestimate the likelihood of positive events. It may seem a pessimistic assessment, but Flybjerg stands by what he’s read, seen and heard.

He calls it "strategic misrepresentation" - the crucial thing for these people is to get the project in motion, you can ask for more money later and by the time the full cost can be counted it's likely that you're out of office anyway.

In a sense, what he's saying isn't surprising; what is interesting is that he's got the data to back up what we would ordinarily just state as opinion or suspicion - when you see the plans for a large public project, expect to add another 50% onto that quoted cost. And his interest in public vs private projects seems to be because the public project is spending our money rather than a private corporation's money; the private developments still go off the rails and in similar patterns.

[Wayno]

He calls it "strategic misrepresentation" - the crucial thing for these people is to get the project in motion, you can ask for more money later and by the time the full cost can be counted it's likely that you're out of office anyway.

This is a classic project management technique seen regularly in the IT industry. Spend the budgeted $1m, tell management you are 80% complete, ask for $100k more to finish the project. Spend the $100k, tell management you are now 90% complete and need another $100k - asymptote to infinite...

[Aidan]

Having read his article, I regard his approach as rather lazy, and not the solution at all. The best it can hope to achieve is the cancellation of projects where the cost is out of control - but that would mean that the infrastructure remains inadequate.

Examining cost overruns in comparable projects is a very good strategy, but the objective should not be to predict the cost overruns - it should be to avoid them. Cost blowouts don't just happen, they are caused. By understanding what can go wrong,they can be prevented relatively cheaply. The example of the Channel Tunnel is quite a good one - the initial geological surveys had indicated that the ground was well suited to tunnelling on the British side, but more difficult on the French side due to jointing in the chalk. As the Tunnel Boring Machines were only intended to go half way each, the French TBMs were designed to withstand difficult conditions, but the British ones weren't. They should have been.

More difficult geology than anticipated was the main cause of cost overruns, but IIRC the tunnel ventilation / pressure relief ducts were also changed to a more expensive (but more effective) arrangement. And the demand forecasts failed to take into account the rapid growth of low cost airlines, which not only increased competition but reduced the significance of Paris as a destination.

[Prince George]

Having read his article, I regard his approach as rather lazy, and not the solution at all. The best it can hope to achieve is the cancellation of projects where the cost is out of control - but that would mean that the infrastructure remains inadequate.

Bear in mind that just at the start of April Bent Flyvbjerg was made not only a professor at Oxford's Said School of Business, but became the chair of Major Programme Management, and all this at one of the schools whose graduates are sought after by firms like McKinsey; so he has rather serious credentials in exactly this field.

His studies demonstrate that after 70+ years of projects from which to learn "what can go wrong" and prevent them, recent projects are not doing significantly better than past ones. The results consistently show that nothing of substance has changed. His programme is to improve the honesty of cost/revenue forecasts so that we can make more meaningful decisions in how limited funds will be spent. Money will no doubt continue to be spent on infrastructure (or, indeed, on stadia, museums, and opera houses) but the money has to come from somewhere and it's hard to make reasonable plans in the absence of reliable information.

[Aidan]

Having read his article, I regard his approach as rather lazy, and not the solution at all. The best it can hope to achieve is the cancellation of projects where the cost is out of control - but that would mean that the infrastructure remains inadequate.

Bear in mind that just at the start of April Bent Flyvbjerg was made not only a professor at Oxford's Said School of Business, but became the chair of Major Programme Management, and all this at one of the schools whose graduates are sought after by firms like McKinsey; so he has rather serious credentials in exactly this field.

Even if he were regarded as the world's foremost expert, it wouldn't change my view. I base my opinions on logic, not consensus.

And it's hardly surprising he's a professor of business, as business is typically limited to the whether to do it type of questions. The how to do it better type of questions are in the realm of engineering.

His studies demonstrate that after 70+ years of projects from which to learn "what can go wrong" and prevent them, recent projects are not doing significantly better than past ones.

Yes, and that's a big problem. 70+ years of projects from which to learn has not translated into 70+ years of attempting to learn. The article's example of using an inexperienced tunnelling crew is a case in point. With insufficient understanding of what went wrong, cost overruns and severe technical problems were easily predictable.

To be fair, there are signs he may know this, as he has cautioned against overreliance on his approach.

The results consistently show that nothing of substance has changed. His programme is to improve the honesty of cost/revenue forecasts so that we can make more meaningful decisions in how limited funds will be spent. Money will no doubt continue to be spent on infrastructure (or, indeed, on stadia, museums, and opera houses) but the money has to come from somewhere and it's hard to make reasonable plans in the absence of reliable information.

But when we ignore the possibility of making it cheaper, we will always have too little infrastructure getting built - and when it does get built we will continue to pay too much for it.

[Prince George]

Aidan, I think that you are looking for an engineering solution for what is a people problem. Do you honestly believe that there has not been repeated and intense efforts to understand the engineering, scheduling, and budgeting of these projects? Considering there are people whose entire career is doing just that, I can hardly believe that to be the case. As you yourself say, in the example of the Green Belt tunnel, "cost overruns and severe technical problems were easily predictable", so why weren't they; we who are all so removed from that environment can say "wow, that sounds crazy", why didn't they do the same?

Every field seems to have this pattern: software has the excellent COCOMO models from Barry Boehm, reams and reams of books or research papers on building software, and a list of methodologies so long that they risk running out of funky names, yet the overdue-overbudget software project is almost the only dependable thing in the industry (about 90% of all projects); the defence industry is rife with the same problem, inspite of high-profile successes like the Polaris program that introduced PERT and made the GANT chart universal, and governments forming whole colleges and organisations for system engineering and defence acquisitions. Likewise, these large-scale public projects have their own field of budgetting and estimating, with whole companies dedicated to that business.

It seems to me that to sustain your position, we must suppose that the people involved in these projects have either been stupid or incompetent. I'm always tempted to think that way, myself, but whenever I do I learn that actually the people involved are frequently people much like me. The lesson seems to be that people just like me, armed with the same knowledge that I am, are capable of making (or being implicated with) lousy decisions more often than not.

[Aidan]

Aidan, I think that you are looking for an engineering solution for what is a people problem.

Cost overruns in engineering projects are an engineering problem that demands an engineering solution. What you seem to regard as a people solution (building less infrastructure) is not really a solution at all - it just prolongs other problems, and doesn't make infrastructure any cheaper.

Do you honestly believe that there has not been repeated and intense efforts to understand the engineering, scheduling, and budgeting of these projects? Considering there are people whose entire career is doing just that, I can hardly believe that to be the case.

It's not that there haven't been any, it's that there haven't been anywhere near enough. This sort of analysis should be done at the end of every public sector infrastructure project, and the results published so that the problems can easily be anticipated before the next similar project.

As you yourself say, in the example of the Green Belt tunnel, "cost overruns and severe technical problems were easily predictable", so why weren't they; we who are all so removed from that environment can say "wow, that sounds crazy", why didn't they do the same?

I don't know enough about the politics and public sector management process of Scandinavia to be able to tell you. Remember, you found the report - not me.

Every field seems to have this pattern: software has the excellent COCOMO models from Barry Boehm, reams and reams of books or research papers on building software, and a list of methodologies so long that they risk running out of funky names, yet the overdue-overbudget software project is almost the only dependable thing in the industry (about 90% of all projects); the defence industry is rife with the same problem, inspite of high-profile successes like the Polaris program that introduced PERT and made the GANT chart universal, and governments forming whole colleges and organisations for system engineering and defence acquisitions. Likewise, these large-scale public projects have their own field of budgetting and estimating, with whole companies dedicated to that business.

And yet they keep making the same mistakes, so clearly something is wrong.

It seems to me that to sustain your position, we must suppose that the people involved in these projects have either been stupid or incompetent. I'm always tempted to think that way, myself, but whenever I do I learn that actually the people involved are frequently people much like me. The lesson seems to be that people just like me, armed with the same knowledge that I am, are capable of making (or being implicated with) lousy decisions more often than not.

You're making the mistake of assuming competence is digital. But that's far from the case. Not only is competence analogue, it is often very specialized. You can be more competent than 99.9% of the population but still not be sufficiently competent to do the job well. And competence in designing projects does not always equate to competence in constructing them or vice versa.

[Wayno]

Aidan, I think that you are looking for an engineering solution for what is a people problem.

Cost overruns in engineering projects are an engineering problem that demands an engineering solution.

You are both right, but PG is perhaps a tad closer to the mark. People are inherently poor at learning from others. Applying "ever more smart and diligent" engineering solutions does work to reduce some people-related issues, but then again people always find new and unique ways to screw things up...

[Aidan]

Aidan, I think that you are looking for an engineering solution for what is a people problem.

Cost overruns in engineering projects are an engineering problem that demands an engineering solution.

You are both right, but PG is perhaps a tad closer to the mark. People are inherently poor at learning from others. Applying "ever more smart and diligent" engineering solutions does work to reduce some people-related issues, but then again people always find new and unique ways to screw things up...

But the cost overruns aren't caused by new and unique ways to screw things up - they're being caused by things being screwed up again in the old and predictable ways.