Moving From Data Chaos To Finding Insight In Information

IBA Group recently hosted a webinar titled From Data Chaos to Business Insights. This was focused on designing cloud strategies and how to manage modern predictive data analytics and featured four of the experts from the IBA Group team.

This reminded me of the recent sad death of Mike Lynch when his yacht was caught in a storm off the coast of Sicily. Lynch was an expert in machine learning who conducted research at the University of Cambridge and then launched a business called Autonomy. He sold this company to Hewlett Packard in 2011 and then endured years of legal action because HP said the deal had been overvalued.

Lynch was found not guilty of all charges in March this year. His yacht trip around Italy was planned to be a celebration with his family and friends, but it ended in tragedy.

Autonomy was an interesting business because it was engaged in machine learning and AI to explore data and to find insight and patterns. It was doing all this more than a decade ago.

Not many of the news reports about the death of Mike Lynch actually explained what Autonomy was doing and why HP was so interested in buying the company so it’s worth considering once again.

There was a time when data always had to be stored in a very formal and structured format. A database featured a series of tables with rows and columns – think of it like a series of grids.

One table might contain columns such as customer number, date, time, item code, item value. This table is storing purchases. The customer number can be called a key because it works for other tables. If you are browsing the purchases table then you can use the customer number to search the table containing customer delivery data. Each table is related to others using these keys. It’s known as a relational database.

But this isn’t the modern reality of how a company uses and stores data. There may still be some relational databases around, such as customer or transaction records, but the most common data format is unstructured. This is all the documents, manuals, texts, emails, and other internal messages sent within the organization.

All this data contains the real value, the real knowledge of how everything in the business works. The problem has always been how do you point a machine learning tool at a whole load of random data and then expect to make sense of it all, let alone finding connections and patterns in all this unstructured data?

That is what Autonomy was doing over a decade ago and that’s why HP was interested. Most people knew HP as a hardware company then, mostly printers. They wanted to pivot into IT services in the same way that IBM did much earlier.

Machine learning tools are now far more powerful and this presents some amazing opportunities for companies to build and train systems that can help their customers. If you have a comprehensive library of Frequently Asked Questions (FAQ) and product manuals then you can now easily train an AI chatbot with this data and then ask it questions about the products – it will be able to answer using all that knowledge.

The bot can be available to your employees or even your customers directly and will be able to answer any question – provided the training data had the answers.

The webinar talks about moving from data chaos to business insights. This has been possible for some time now, but has become exponentially easier to achieve in the past two years. There are many different opportunities out there right now.

Data analytics is becoming an essential tool for all businesses, both for predicting what your customers want and need and also for exploring the chaotic unstructured data inside a business – what insights can be discovered in there?

For examples of IBA expertise on data analytics and AI, please click here. Follow IBA Group on LinkedIn for regular updates and comment.