Currently, research at the Centre focuses on the following questions:
1. What are the main problems in major programme management?
2. What are the main causes of these problems?
3. What are possible cures to the problems?
4. For programme successes, how is success defined, explained and replicated?
5. How and why does programme performance vary across nations, time and sectors?
6. Do ICT programmes perform worse than other programme types?
7. Is private or public delivery of programmes more effective and more democratic?
8. What are the main risks in major programmes and how are they best mitigated?
9. What is quality and quality control in decision making for major programmes?
The research agenda is constantly evolving, and other research questions are being developed. We welcome suggestions for developing the agenda further.
A database on performance in major programmes is at the core of research. Each research project in the Centre, including single-case studies, is expected to contribute to and draw on the database for comparisons, benchmarkings, etc.
The research projects listed below are currently in progress at the Centre:
Performance and decision-making in major ICT programmes: problems, causes, cures, Alexander Budzier and Bent Flyvbjerg
Large-scale transport infrastructure management in the Netherlands: problems, causes and cures, Chantal Cantarelli and Bent Flyvbjerg
How to conduct due diligence of business cases, cost-benefit analyses, and social and environmental impact assessments for major programmes, Bent Flyvbjerg
The planning fallacy revisited: systematic bias and recognized ignorance in forecasting and decision making, Bent Flyvbjerg
Private versus public finance in major programme management, Bent Flyvbjerg
The sector project: how and why major programme performance varies across sectors, Bent Flyvbjerg
The geography project: how and why major programme performance varies with geography, Bent Flyvbjerg
Theories of success: explaining success stories in major infrastructure programmes, Mig de Jong and Bent Flyvbjerg
Successful Programs, Failed Projects. Eamonn Molloy and Allison Stewart
Engaging the Islamic World through Major Sports Events, Kasim Randeree
Reputational capital and mega project management: Exploring the reputational threat of hosting the Olympic Games, Kasim Randeree
Investigation of Project Management and Systems Engineering at the CERN ATLAS Experiment Janet Smart
Sectoral differences in systems engineering for major programmes, Janet Smart
Programme management in big science, Janet Smart
Systems Engineering – modern methods. Janet Smart and Tomomi Kito
The impact of major programme failure on corporate reputation, Kasim Randeree
Persistent Ignorance: How uncomfortable knowledge affects project planning and delivery in the Olympic and Commonwealth Games. Allison Stewart, Bent Flyvbjerg and Steve Rayner
Performance and decision-making in major ICT programmes: problems, causes, cures
Contact persons: Alexander Budzier and Bent Flyvbjerg
Pilot studies indicate that performance in major ICT programmes is even more problematic than in other types of major programmes, resulting in waste of taxpayers' and shareholders' money for public and private projects, respectively. Project sponsors are key decision makers on large-scale ICT projects. Sponsors decide, for instance, whether or not to do a project, how to prioritise between projects, define scope and size. These decisions influence greatly the success of a project. This research aims to understand project sponsor decision-making, its limitations, and areas for improvement. The core questions of this research are (a) How do stakeholders/project sponsors decide upon their ICT projects? (b) What are the shortcomings in ICT decision-making, if any? (c) What are their root causes? (d) How can inefficiencies in sponsors' management be overcome? (e) Is it possible to develop quantitative tools to support decision-making? (f) If so, how can such a quantitative tool help? Methodologically the research is based on a survey of 50-100 large-scale ICT projects that form part of major ICT programmes. The survey is aimed at analyses of costs, effort, schedule escalations, and benefit shortfalls; project and environmental factors; project and portfolio management prac¬tices; and the role of project sponsors. The survey is followed up 3-5 in-depth case studies aimed at understanding decision- and sense-making of stakeholders as well as practices to overcome shortcomings in decision-making. The theoretical contribution of the research is expected to be in theories of quality in decision making.
Large-scale transport infrastructure management in the Netherlands: problems, causes and cures
Contact persons: Chantal Cantarelli and Bent Flyvbjerg
Infrastructure investments per square kilometer is probably higher in the Netherlands than in any other European nation. Nevertheless, little systematic knowledge exists about the actual costs of this infrastructure, to what degree it was built on budget and schedule, whether it delivers the promised benefits and what the social and environmental impacts are. This research develops a metric and collects data that helps begin answer such questions, especially as regards costs and schedule. It further investigates how current practices of managing major transport infrastructure programmes may be improved in the Netherlands. The key questions of the research are (a) What are the main problems in major transport infrastructure management in the Netherlands, (b) What are the causes of the problems, (c) What cures may be developed to solve the problems, and (d) How do problems, causes and cures in the Netherlands compare to other nations? Thus the research involves benchmarking the Netherlands against other nations. Methodologically, the research is based on metrics development and statistical analyses. Theoretically, the research will contribute to theories of 'lock in', optimism bias and strategic misrepresentation. The research is carried out in collaboration with Chantal Cantarelli, Delft University of Technology.
How to conduct due diligence of business cases, cost-benefit analyses, and social and environmental impact assessments for major programmes
Contact person: Bent Flyvbjerg
The research question for this project is: how may theories of the planning fallacy and the outside view be used to conduct due diligence of business cases, cost-benefit analyses, and social and environmental impact assessments for major programmes? In previous research we have documented a much-neglected topic in economics, management, and planning, namely that ex ante estimates of costs and benefits are often very different from actual ex post costs and benefits. This means that, in practice, business cases, ex ante cost-benefit analyses, and social and environmental impact assessments are typically poor predictors of the actual value and viability of investments. The idea is to use due diligence to establish just how reliable or unreliable a given business case or cost-benefit analysis is and what this means as to whether or not a given investment should be made. Due diligence is generally an evaluation of the assets of a company, an investment, or a person. In this research project, due diligence is specifically understood as an evaluation of the costs and benefits of a given major investment, and especially whether the estimated costs and benefits for that investment are likely to materialize. Due diligence will thus be developed to be used as a type of “quality control” of business cases, cost-benefit analyses, and the go-decision in major programmes. The theoretical and methodological foundations for the research will be sought in Daniel Kahneman's and Amos Tversky's work on the planning fallacy and the outside view. Kahneman and Tversky did not develop their theories with this purpose in mind. The hunch is, however, that their theories will be highly effective in due diligence and quality control of decision making. Data will be drawn from a large database on estimated and actual costs and benefits in major projects, which is at the core of research in the BT Centre. Theory and methodology will be developed at a level of generalization that is sufficiently high to allow their use, not only in business cases and cost-benefit analyses, but also in social and environmental impact assessments, which are typically also key in decision making on major programmes, and which also suffer from the problem of predicted impacts being very different from actual ones.
The planning fallacy revisited: systematic bias and recognized ignorance in forecasting and decision making
Contact person: Bent Flyvbjerg
How come that planners, forecasters and decision makers keep repeating the planning fallacy over and over, with no link to the outside view (systematic use of knowledge of past outcomes), even when everyone knows about bias and optimism? This is the puzzle at the centre of an empirical-theoretical study designed to test and further develop theoretical models that explain bias in forecasting and decision making in terms of optimism (Daniel Kahneman) and strategic misrepresentation (Bent Flyvbjerg). New data will be collected from forecasting and decision making in major transport programmes. The research is carried out in collaboration with Daniel Kahneman, Princeton University; Petter Næss, Aalborg University and Otto Anker Nielsen, Technical University of Denmark.
Private versus public finance in major programme management
Contact person: Bent Flyvbjerg
A main assumption behind privatisations, public-private partnerships and private finance initiatives in major programme delivery is that the involvement of private risk capital in programme delivery, for instance in infrastructure provision, will bring much-needed discipline to the planning and delivery of programmes. The research in this project is designed to test this assumption. So far the assumption has been tested only with small samples of programmes and the evidence is mixed. Statistically valid conclusions do not exist. This study will attempt to change this state of affairs. The primary research question is whether programme performance is significantly affected by type of financing regime, and especially whether private or public finance results in better performance in major programme management. Systematic and comparable data on major programme performance is notoriously hard to come by. This is the case for both public and private sector programmes, but including private programmes in data collection adds the further complication that performance data for such programmes are often considered business secrets. Data collection is therefore especially difficult and time consuming. Methodologically, the research is based on statistical analyses combined with case studies. Theoretically, the study will contribute to theories of privatization and theories of where best to draw the borders between government and markets in major programme management.
The sector project: how and why major programme performance varies across sectors
Contact person: Bent Flyvbjerg
Risk in major programme management is generally poorly understood and is typically grossly underestimated. Sector risk – i.e., risks arising from planning and delivering programmes in specific sector contexts, for instance in ICT as opposed to in construction – is even less well understood. This research project is set up to measure and explain sector risk in major programme management and to test whether and how sector matters to project performance. The research will also investigate 'sector convergence', i.e. the phenomenon that sectors merge into one another, as in the case of, for instance, the banking and transport sectors incorporating ever larger elements of ICT, with the risks and possibilities this entails. Data will focus on risk and programme performance. Methods will be metrics, benchmarking and statistical analysis, combined with case studies. Theories of risk and performance will be tested and developed.
The geography project: how and why major programme performance varies with geography
Contact person: Bent Flyvbjerg
Risk in major programme management is generally poorly understood and is typically grossly underestimated. Geographical risk – i.e., risks arising from planning and delivering programmes in specific geographical contexts, for instance emerging economies – is even less well understood. This is a problem, because currently a major geographical shift in investments in major programmes is taking place, with investments proportionally moving away from developed to emerging economies. For example, in the past five years, China has spent more on infrastructure in real terms than in the whole of the 20th century; for high-speed rail alone, China has built as many kilometers rail lines in four years as Europe has in two decades. Over half of infrastructure investments are now taking place in emerging economies, a development that is expected to accelerate further in the coming decade. Similarly, geographical shifts are taking place in oil and gas extraction, mining, etc. Systematic and comparable data to document and understand the risks arising from these shifts are largely absent from the research literature. This research is set up to generate data on geographical risk in major programme management and to test whether and how geography matters to project performance, including whether and to what extent the striking geographical shift in investments is placing increased pressure on project delivery. Theories of risk – including political risk, risk of corruption and logistical risk – will be tested.
Theories of success: explaining success stories in major infrastructure programmes
Contact persons: Mig de Jong and Bent Flyvbjerg
Major infrastructure programmes generally have a history of cost overruns, delays and demand shortfalls. Costs and construction periods are underestimated and benefits are overestimated. This leads to economic inefficiencies and decreased faith in governments, consultants, contractors, programmes and programme delivery. Exceptions to the rule exist, however. A few programmes are built on time and budget and deliver the promised benefits and may be termed programme successes in this sense. The research asks the question of whether such programmes succeed due to happenstance or whether more systematic factors are at play that may be explained and replicated. Data is collected from successful major transport infrastructure programmes in roads, rail, bridges and tunnels. The main research methods used are case studies and benchmarkings. The purpose is to develop a "theory of success" for major infrastructure programmes. The research will use and contribute to theories of decision making and success. The research is carried out in collaboration with Mig de Jong, Delft University of Technology.
Successful Programs, Failed Projects
Contact persons: Eamonn Molloy and Allison Stewart
This research investigates the question of why project- and program-based organizing appears to be increasingly popular when evidence suggests that most projects fail. We start from the assumption that studying language offers insights into cognition (Weick, 1979; Whorf, 1956; Sapir, 1944) and that language plays a significant role in influencing managers’ ability to conceptualize modes of operating (Thiry & Deguire, 2007; Keegan & Turner, 2001). To explore the question, we therefore analyze over sixteen hundred occurrences of the terms ‘project’ and ‘program’ in conjunction with ‘success’ and ‘failure’, to explore the linguistic association of the terms in a representative sample of English language texts. Data are collected from the Oxford English Corpus (OEC), a structured and coded database of two billion words of naturally occurring English collected from the World Wide Web. The analysis shows that the general estimate that two-thirds of projects fail is not reflected in the way that projects and programs are represented in the English language at large. Within the OEC, the terms project and program are twice as likely to be associated with the term success as failure. Furthermore, the associations vary in relation to non-linguistic factors such as mode, register, domain of usage and temporal orientation.Additionally, the data show that although the terms project and program are linguistically similar (Molloy, 2010), programs are significantly more associated with success than expected, and projects are more associated with failure. Several explanatory options are proposed and implications for future research and for practice are discussed.
Engaging the Islamic World through Major Sports Events
Contact person: Kasim Randeree
Understanding the Islamic world has been regarded from a western perspective as necessary for varying reasons throughout the last millennium up to the present day. Conquest, governance, exploitation and most recently engagement have been primary motivators. In modern times, sport, and in particular mega events such as the Olympic Games and World Cup Football, have been used as an equalising force for promoting engagement with nations of differing political and ideological persuasions. A Muslim city or nation has to date, never hosted an Olympic Games or FIFA World Cup Finals, though Muslims account for nearly one quarter of the world’s population and have a strong record of active participation in the two events. Today, in a post 9/11 world, with an emphasis by the West on promoting moderation within Muslim societies across the globe, it is important that the world soon witnesses an Olympic Games and FIFA World Cup finals hosted by a city and nation in the Muslim world.
This research thus examines the feasibility and likelihood of a Muslim host city for the Summer Olympics and host nation for the FIFA World Cup Finals, for two reasons. Firstly, examining the potential of hosting a global mega-event provides insights on developmental characteristics of a city and nation. An examination into political, historic, cultural, economic and other issues is useful in gauging the progress of the Muslim world in comparison to other developed nations. Secondly, the awarding of a major sporting spectacle in the Muslim world sends a clear indication of a desire to engage with it and, in the absence of any published study on the subject, this research would open a discourse that would be of value to scholars and interested parties in diverse fields such as Islam, international politics, sport, economics, international development and beyond.
Reputational capital and mega project management: Exploring the reputational threat of hosting the Olympic Games
Contact person: Kasim Randeree
Hosting the Olympic Games is not merely a sporting endeavour, but is an undertaking that is fraught with political, socio-cultural and economic pressures. Aligning these pressures with reputational issues for the multitude of event stakeholders, most notably the bidding city, if not properly managed as a mega project, can create a ‘perfect storm’, which could result in poor decision-making, destructive loss of reputational capital for stakeholders (particularly the host city) and potentially the delivery of a failed Olympic Games.
Mega project management, and particularly the management of mega events, deserves special attention in the context of reputation. This is, in part, because they come under considerable public scrutiny. For example, the Olympic Games require public sector support and fiscal involvement, without which a city could not bid to host the games. In fact, due to rising costs of hosting the games, coupled with the growth in interest at an international level, the International Olympic Committee (IOC) cannot, more than ever, risk failure in the delivery of an Olympic Games. Consequently, the IOC decided that governments must act as financial guarantor when a city bids to host the Olympics.
Factors which have historically impacted negatively on the reputation of Olympic stakeholders, the IOC and host cities in particular, include bribery and inequity in the bidding process, human rights issues, international terrorism, poor fiscal forecasting, deficiencies in infrastructure development, optimism-bias, the promise-performance gap and games boycotts. On the positive side have been a limited number of games that have been successfully delivered, with profitability and medium to long-term regional sustainable infrastructure development, with corresponding gain in reputational capital to the host city.
This research thus examines reputational issues, which have encompassed the Olympics of the past, by examining lessons from a longitudinal survey of documented Summer Olympic Games since 1960. The outcomes of the research carry significant lessons for future Olympic host cities, most immediately London 2012 and Rio de Janeiro 2016 and have broader implications for all mega-sporting events.
Investigation of Project Management and Systems Engineering at the CERN ATLAS Experiment
Contact person: Janet Smart
The ATLAS Experiment is one of the detectors positioned on the beam of the Large Hadron Collider at CERN, Geneva. ATLAS is an example of a major project that has seen the specification, design and construction of one of the largest machines ever built, consisting of 20 million components, involving over 130 scientific institutions and 37 nations. Despite the complexity of the design, engineering and construction of the project, it is functioning and is analyzing data from the LHC. In many ways, the ATLAS community seems to defy the standard models of project management and systems engineering, so they offer a possibly unique opportunity to study the extent to which they do or do not follow recommended practices, and to identify some of the reasons for, and consequences of, their non-standard project management behaviour. Representatives of the community of particle physicists, including graduate students, engineers and technicians, will be interviewed to uncover the formal and informal methods that the community uses to manage progress towards achieving the vision of understanding the constituent particles of matter. Through a variety of pedagogic and publication routes, the outcomes of this project will contribute to the understanding of the impact of culture on project and programme management, and suggest novel methods for coping with project crises and re-alignments.
This project gratefully acknowledges funding from EPSRC grant number EP/I07488/1.
Sectoral differences in systems engineering for major programmes
Contact person: Janet Smart
Somehow, major programmes have to be designed, built and delivered. This amazingly complex task is managed by breaking the programme vision into tasks and tranches, with bounded interactions and connections between them. Work therefore can focus on the tasks within the constraints of the agreed interactions. The process of translating systems requirements into designed systems is called systems engineering. This process is closely connected to project management and management of the contracts. – At first thoughts, this should seem like a straightforward, linear, engineering process, but in fact it is much more complex, since the final architecture of the system may evolve over time, through a process of negotiation between designers, builders and clients. The negotiation process may be managed in different ways according to the cultural practices of the participating organizations and the individual team members. These differences can also be due to coping with different rates of technological progress across sectors, which may contribute to cultural and procedural differences. Programmes may be impossible to define accurately at the outset, and are subject to a rapidly evolving technological, legislative and social environment, which may impact the vision and actuality of the programme. – This research project is currently being scoped by carrying out interviews with the managers of major programmes in ICT, construction, defence and so-called “Big Science”, leading to case studies and further research. The outcomes of this research will be narratives on contrasting methods of approaching systems design and the definition of architecture, with analyses of the cultural and social differences that require or enable these differences to give rise to effective problem-solving and negotiation strategies. The contributions will be to the understanding of systems engineering and design for complex systems and major programmes. This project will integrate linear models of systems engineering, with socio-technical approaches to complex systems engineering, leading to better technical design of complex systems, and better management of the complex process of developing and delivering large-scale technical systems.
Programme management in big science
Contact person: Janet Smart
Many large science infrastructure projects, such as ground-based or orbiting telescopes, or particle physics projects such as CERN, involve international consortia of universities, high technology companies and government-funded organizations. These projects may take up to twenty years from concept to delivery, and have an operating lifetime of twenty or thirty years or more.
In many ways, managing scientific infrastructure projects is likely to be similar to managing any major international programme. This project aims to investigate whether the patterns of over-estimating benefits and underestimating time-scales and costs that have been seen in other kinds of major programmes carry over into Big Science projects too. One of the big differences between science and commercial projects is that time is the contingency of the science project. If the scientists are seeking the Higgs Boson, for example, it will still be there even if the instrument is 10 years late.
A dataset of about 20 major, international science projects will be gathered, contributing to the BT Centre’s database of major programmes. This will cover carefully sourced data on planned costs and schedules, and the delivered project outcomes. The data will be enriched with interviews with the managers and directors of these major programmes.
This project will contribute to the developing work on the planning of major programmes, while investigating differences between the management of big science and commercial or infrastructure projects of a similar size and complexity. This project will be of use to national and international organizations that fund major science projects, helping them to identify the factors that are likely to lead to cost over-runs and failure to deliver the intended outcomes.
Systems Engineering – modern methods
Contact persons: Janet Smart and Tomomi KitoThe textbook methods of systems engineering are often tailored to suit the project or programme and the organization that is responsible for supporting and constructing it. Furthermore, modern ideas on lean and agile manufacturing are being drawn into the project management and systems engineering community of practice. This project is looking at the ways in which systems engineering is adapting to meet the requirements of large, complex projects. The effects of these changes will be felt on the modeling of the systems engineering process, as well as the methodologies for assessing progress, dealing with project crises and managing project or programme governance. This project is interviewing a sample of large engineering organizations, both commercial and non-commercial, to uncover their understanding of the future in systems engineering. The output of the project will be a number of case studies of modern systems engineering in practice.
The impact of major programme failure on corporate reputation
Contact person: Kasim Randeree
Many major programmes fail to achieve the intended outcome within budget and according to the planned deadline. The organizations that are involved in the delivery of such programmes are often large and well-known, so they have to ensure their continuation after programme, which means ensuring that consumers and corporations continue to engage with them. The research questions that will be addressed in this project include: (a) Within the context of major programmes, to what extent do public bodies and corporations anticipate the likely onset of programme set-backs and failures? (b) How do clients and other customers of a corporate body involved in a failing programme take action to protect themselves from reputational damage? (c) How do stakeholders and community bodies respond to programme failure and setbacks, and how are their reactions managed by corporate organizations? – This preliminary 10-month project will investigate the impact of major programmes on two or three major corporations. The corporations will be selected from more than one sector, e.g. finance, IT, construction, facilities management. The research methods will involve a thorough search of the archive of articles and comment pieces in the press in order to assess the public perception of the programme and the companies involved. Interviews will be carried out with personnel involved in the delivery of the programme, those involved in management of the stakeholders and those personally affected by the actions that were taken to protect the corporate reputation. This project will be conducted jointly within the BT Centre for Major Programme Management and Oxford's Centre for Corporate Reputation. The research will draw on current research on corporate reputation, and work on stakeholder management and engagement within the fields of project and programme management.
Persistent Ignorance: How uncomfortable knowledge affects project planning and delivery in the Olympic and Commonwealth Games
Contact persons: Allison Stewart, Bent Flyvbjerg and Steve Rayner
This research seeks to explore the concept of ‘uncomfortable knowledge’ in major event programmes, with a focus on the Olympic and Commonwealth Games. In a major programme environment, we suggest that uncomfortable knowledge refers to information that is either strategically or inadvertently avoided due to the potential for organisational conflict if it is addressed, for example, with respect to budget overruns or scope disagreements.
As programmes, the Games are staggeringly complex, with multi-billion dollar price tags, a fixed and immovable timeframe, and the human resource challenges of building and dissolving a major organisation in a few months. In this environment, uncomfortable knowledge can therefore pose a significant risk. To explore this risk, this research takes a two-pronged approach: first, we are conducting a historical analysis of budget, scope, and demand data for the Games since 1960. Second, we are working with the following committees that are currently planning and delivering the Games to understand how information is used and shared: Vancouver 2010 Winter Olympics, Delhi 2010 Commonwealth Games, London 2012 Summer Olympics, Sochi 2014 Winter Olympics, Glasgow 2014 Commonwealth Games, and Rio 2016 Summer Olympics. Ultimately, this research seeks to expand the current understanding of uncomfortable knowledge to the major programmes context, and to provide actionable suggestions for major programmes to deal with this risk.