Some IT project efforts succeed, and some fail. Do warning signs typically arise en route to failure? What might be done to heed these warnings? Can lessons be learned from failing early in a project? And perhaps most importantly, how are "success" and "failure" defined by stakeholders?
I addressed these subjects as part of a media query by CIO magazine, and ended up being quoted by the publication on September 14, 2017 alongside other technology leaders on the subject of project failure. What follows are the original responses I provided alongside some additional sidebars I wrote following publication.
(Extent of quote highlighted in orange, paraphrased quote highlighted in gray, and the CIO magazine article can be found here.)
CIO: Why are organizations so reluctant to kill IT projects that are clearly doomed?
Gfesser: From my experiences as a consultant, the most common reason organizations can be so reluctant to kill doomed IT projects is because of incurred costs that cannot be recovered. The idea is that a significant cost has already been paid, and one might as well make furthered attempts to succeed so that this initial cost is not all for naught. But the problem with this reasoning is that if the direction of the project is unchanged it will continue to fail, leading to even greater incurred costs.
The second most common reason I have seen is that reputations are at stake, and leadership would really like to get the project back on track to some extent so that they don't look incompetent. In all of my years in consulting, I've never seen a project which was significantly behind ever get back on track, but partial success can be met by scaling back the original scope, or moving in a different direction than was originally planned, both avenues of which are enabled my agile methodologies. At the same time, it is important to remember that some leaders define "success" so rigidly as to prevent these alternatives from ever taking place.
Two common reasons doomed IT projects are not killed:
- Incurred costs that cannot be recovered
- Leadership reputation and competency
CIO: How can project leaders determine whether it's the project concept or the project execution that's fundamentally at fault in a failing project?
Gfesser: Additional questions to ask here involve inquiring as to how "concept" and "failure" are actually defined, and the stage at which the project started to fail. From the context, "concept" likely refers to either a conceptual description or representation of what the product or output might look like, or what is often called a "proof of concept", although this latter possibility typically takes place after an initial inception phase. Either way, the distinction between these two is important, because a proof of concept might not be viewed by stakeholders as being part of the execution. However, construction of a proof of concept is often a needed step between the inception and the execution, because taking this additional measure can help mitigate risks that might not otherwise materialize until additional costs are incurred.
In other words, the question as to whether the project concept or project execution is at fault becomes much less necessary if the team actually takes the time to go through the often needed steps of building one or more proof of concepts followed by a prototype which will serve as the actual foundation for the end product that is to be created. Proof of concepts are often built to make sure that adopted technologies will actually do what is claimed, and once it is determined that this is the case, it makes sense to take the next step of building a prototype, and later an MVP ("minimal viable product") in order to start getting the product in front of stakeholders for the purposes of feedback. Project execution is typically the most difficult part of a project, but taking these steps will help ensure success, and if failure takes place in these early stages it can also be seen as success because you found out what you can't or don't want to build! Going back to my earlier comment, stakeholders need to realize that even a flawed idea can be carried out successfully - but is the end product or output really what stakeholders want?
Questions to ask on project concept versus project execution:
- How are "project concept" and "project execution" being defined?
- Viewing a project as just two stages is likely a gross oversimplification. At what stage did the project actually start to fail?
- What do stakeholders actually need or want? Are users actually going to use what is being built?
CIO: Is there a stage at which projects are most likely to fail?
Gfesser: Signs of failure typically appear early in a project if project execution was performed in an agile manner. Failure is not necessarily a bad thing, because it can lead to a more successful product by understanding what doesn't work. One of the benefits of using agile methodologies is to help ensure that the product being built is what the users of the product will actually want to use. If after a series of one or more agile sprints it is determined that the product is going in a wrong direction, the course can be corrected. And in such a scenario, the product owner should actually want project execution to fail so that it can succeed down the road. I've actually never seen a project that failed solely due to choice of product or technology. Failure is almost always due to people or processes, and processes are often comprised of people.
As I think about this topic, I still recall a particular project from early in my career that I have used many a time as an example of how not to execute project management. The consulting firm for which we were working at the time had instituted an executive dashboard - every project being executed at the company was displayed on this dashboard so that executives could see projects at risk. My project manager at the time admittedly spent most of his time tweaking the formulas of the spreadsheets being used under the covers until every cell in the dashboard visualizations turned green. Such a project is simply in the wrong hands from the start! Since this particular manager didn't run projects in an agile manner, visibility into accurate status really was not made possible until a later phase when the product started becoming available for stakeholders to see. If these projects had been run in an agile manner, stakeholders would not only have seen from an early phase that failure was taking place, but they could have had an active part in making a course correction!
Do not hide failure. Hiding failure will likely lead to a product that will not be used, will be used grudgingly, or will be targeted for heavy modification or replacement by another product. If used properly, agile practices will bring failure to light at an early stage, enabling opportunities to change. The later the course correction, the greater the cost.
CIO: Are there any specific types of IT projects that are more prone to failure?
Gfesser: Essentially, any projects involving some degree of innovation, or a relatively high level of uncertainty. All projects face some level of uncertainty, and it is projects with high uncertainty that are more prone to failure because not all uncertainty can be addressed. Projects which have been carried out in a similar way in the past, or involve significant repetitive work are less prone to failure, because precedent exists. Projects which are the result of some level of innovation are more prone to failure, because risks are being taken to build something new or in a new manner, and this latter category of projects frequently involve new software development.
CIO: What are the warning signs that indicate a project is in trouble?
Gfesser: The warning signs that indicate a given project is "in trouble", rather than "failing" typically include not meeting end user stakeholder needs, and missed deadlines. Again, if a project is being carried out in an agile manner, these signs will be seen early because at the end of every sprint the team evaluates how it executed its work in comparison to what it decided to take on at the beginning of the sprint. In cases where not as much was accomplished, the team will adjust its velocity expectations for future sprints. And in cases where users were not pleased with the incremental delivery at the end of the sprint, the team will incorporate the feedback they receive into the next iteration of work.
CIO: What lessons can one learn from a failed project?
Gfesser: The lessons learned from participating on a failed project are innumerable. But in my opinion, the additional thing to keep in mind here is that there are many lessons to be learned from failing early in a project, and then shifting direction in order to succeed. And what makes this redirection possible is adoption of agile principles which enable changes in requirements throughout the project so that failure is not postponed until a late stage at which point accumulated losses have already become too great.
Media Query Source: Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7 , Part 9