The One Reason AI Projects Fail and How to Fix It

Play Audio Version

One of the recurring problems with artificial intelligence (AI) is the challenge of implementation. The media view on AI is often focused on how transformative the technology can be and business journals will skate quickly over the effort required to make it work.

The Harvard Business Review published a feature in 2023 suggesting that ‘most AI projects go off-course.’ In 2024, the industry analyst Gartner suggested that around 85% of AI models or projects fail. This is enormous. Which CFO would sign off on a new AI project if they knew this?

But there is a common theme. Most of these AI projects fail because the managers expected the system to work straight out of the box, or they were just not prepared to put in the effort required to effectively train the model. There are always clear failures in the preparation for these projects. The real story is not that most AI projects fail, it is that most managers don’t prepare for how to use AI in their business.

To prepare for most AI projects you will always need to build some solid foundations. This needs review, thought, and change. You need to consider the data that is in your organization. How is it stored? How is it arranged? Can it be used to train your new AI model? If not, then what needs to be done before you attempt to start the planned AI pilot?

Andrei Barysionak, the Global Delivery Director for Data Management at IBA Group, wrote a good article in June this year explaining why the quality of your data is the first thing you need to check before launching any AI projects.

Andrei makes the situation entirely clear: “Everything is simple here — unlike traditional app development, which relies on coding, AI projects are all about data. AI and ML systems use it to learn and make recommendations: they reveal patterns in the information provided for their training, teach themselves those characteristics, and then compare them with new observations to generate a solution. If there is not enough data or if the data quality for AI training is dubious, the project will be doomed to failure.”

Many senior executives are reading about the transformative power of AI in the Financial Times or the Economist and asking their CIO to make some “AI magic” happen in their business.

But as Andrei describes, you can’t just “install” an AI model like you would install a new app. You can’t try one version and then upgrade to a new one if doesn’t work. Everything about your AI model depends on the quality of the data in your business. All the value that your AI system can create will emerge from this training data.

Statistics like those published by Gartner appear frightening. Many senior executives seeing these failure rates might be tempted to avoid funding any new AI projects, but at the same time they keep on hearing about all the AI success stories in other sources of business media.

AI has the power to transform your business and you can avoid the situation where your project fails disastrously because the data used to train the model is inadequate or flawed. It just requires planning and a focus on the data within your business long before the desired AI project is attempted.

AI projects don’t follow the typical coding rules where code can be improved as you go long – the ‘move fast and break things’ culture. If you don’t prepare your data then the AI simply will not work as expected. Your project will fail. And if you signed off on the budget then your lack of preparation may result in the need for some new career options.

IBA Group has an experienced practice focused on data management, data analytics, and AI. Check the website here for case studies and examples of projects that have been designed and delivered in addition to ideas, insights, and project suggestions.

Follow IBA Group on LinkedIn for regular updates and comment. For more information on technology strategy and how tech connects to real business solutions please click here