Making Sense of All That Data

IBA Group
Mark Hillary

In my last blog I mentioned that Big Data has progressed far beyond just being a business buzzword. There are entire industries being shaken to their core because a leading player finds a way to analyse their customers better than their rivals. Far from being a management trend, this is a strategy that will fundamentally change many industries.

But have you ever stopped to appreciate just how much data we are creating today?

Some excellent analysis in Business Insider recently explored this question. The problem is that people and companies are just creating so much data – it is increasing at an exponential rate. At the present rate we are doubling the amount of digital data that exists every two years.

But even though we are creating and storing these enormous amounts of data, only 0.5% of it is being analysed. There is so much data out there that companies, governments, and individuals feel swamped, unable to gain insights from it.

The Business Insider article features a comment that cuts to the heart of the Big Data issue: “You have to start with a question and not with the data,” says Andreas Weigend, former Chief Scientist of Amazon, now director of the Social Data Lab and lecturer at UC Berkeley.
Businesses need to start thinking about the insights they could get from their customers, to ask more ‘what-if’ questions. There are solutions out there in the data, but it is impossible to analyse every byte of data.

A typical analogy for the average person might be the difference between email and Twitter. You check every email, even if it is only long enough to decide that it should be deleted. However, you only check Twitter messages that are arriving as you are looking at the news stream, or you use intelligent filters and analysis to ensure that interesting messages are made visible.

Businesses need to start thinking of their Big Data strategy in the same way. How can insights be drawn out from the data they already have?

Big Data Means Big Business

IBA Group
Mark Hillary

Big Data is still just a buzzword for many people. Magazines and newspapers that do not cater strictly to a business audience continually need to explain what they mean when talking about the subject and the strong association with technology means that even some business leaders are still unaware of the true benefits.

But as with all technology projects and ideas, if they can be associated with actions that can improve a business, make it more efficient, deliver services faster, or create new products before competitors, then the leaders can see the advantage.

Forbes magazine recently documents a few examples that demonstrate some of the advantages. Carnival Cruises needs to plan the best way to serve passengers in much the same way as an airline does. However a cruise is a much longer journey than a flight and across all their ships and passengers, Carnival has 80 million cruise days per year. If they could just earn $1 per day extra from each passenger then that’s an immediate $80m boost to revenue.

That’s an example of how to analyse customer behaviour so products and services can be targeted more effectively. Retailers still do this with loyalty cards, although the idea of a loyalty card has been falling from fashion in recent years – customers are tired of giving away their personal data in return for very small benefits.

But data can also help to save money and improve service too. The Australian telco Telstra uses Big Data analysis on their entire network with predictive analysis so potential faults on lines, and in specific areas, can be identified before they happen. Outage time is reduced, engineers can be moved into position faster, and not only does the company save on maintenance, but the customer is happier too.

Every big business uses data today. Every business has the opportunity to analyse this data in a more effective way. There is always information available if you know how to dig deep into the data you have.

Business Intelligence is Being Led by Data Intelligence

IBA Group
Mark Hillary

The analyst firm Gartner recently published some fascinating data trends in Forbes magazine. They summarised how important Big Data is becoming for business intelligence in three clear trends:

1. By 2020, information will be used to reinvent, digitalize or eliminate 80% of business processes and products from a decade earlier.

2. By 2017, more than 30% of enterprise access to broadly based big data will be via intermediary data broker services, serving context to business decisions.

3. By 2017, more than 20% of customer-facing analytic deployments will provide product tracking information leveraging the IoT.

These trends are exciting because what they point to is how communication is changing between individuals and how this is now affecting the way that companies do business.

Mobile, social, cloud, and shared information are all forces that have really only grown in importance over the past 5-6 years. Many company leaders have not realised how all these factors will change the way that companies do business and how decisions from new products to choosing a partner company will all be data-led.

The Gartner predictions are point at corporate behaviours just 18 months in the future. Have you explored how your own organisation is using data today and if not then can you be sure that your competitors will not be making better business decisions a year from now?

Outsourcing Trends Becoming Important in 2015

IBA Group
Mark Hillary

I was recently asked about the classic price vs service argument by a consultant who advises on IT outsourcing. I replied that I am surprised there is still a debate over this. You can compare IT supplier based on the quality of what they do and then compare equally competent suppliers on price, but price is not a primary variable that should be used to compare companies.

After all, if the service delivered does not work then how much have you saved? The price debate reminds me of where IT outsourcing was a decade ago – it was surprising to be asked about this in 2015 when most organisations have a far more mature approach to finding expert partners.

I looked at CIO magazine to see what they considered the key trends in IT outsourcing would be this year. They published a good summary at the beginning of the year and never once mentioned that price would be an important comparison point.

Several of the trends they identified are very important though and I don’t feel that they are being given enough focus in the business and technology media:

1. A focus on outcomes: outcome based pricing has been around for years, but is often focused on BPO outsourcing where specific business processes can be priced. A focus on the outcome rather than process of delivering IT will be how many projects are charged in future.

2. The business ordering direct. The CIO used to manage all information systems, but now the business units are doing far more ordering direct because many solutions can be delivered using apps or the cloud, therefore not impacting on the infrastructure managed by the CIO. This means that suppliers need to develop new relationships and change their sales strategy.

3. Analytics taking over. In areas such as CRM and customer service technology systems data is all that matters now. This approach to data-led decision-making is affecting many business functions including the more creative ones such as sales and marketing.

The IT outsourcing trends are changing and developing as the IT services space develops, but sometimes it seems that the advisors cannot escape some of the old debates.

Crowdsourcing and Big Data Can Come Together

IBA Group
Mark Hillary

Could Big Data help the buses in New York run on time? That’s what one city politician is hoping for. New York City Council Member Ben Kallos has campaigned for the Metropolitan Transit Authority (MTA) to openly release all their data on bus arrival and departure times.

Kallos is convinced that the buses never run on time. When he has complained in the past to the MTA they have always suggested that he is wrong. He managed to obtain three months worth of bus data and with the help of some data analysts he proved that the buses only ran on time 58% of the time.

The MTA has pleaded that the data is in a difficult format for it to be quickly released to the public so his project has not moved further than this pilot stage, but this demonstrates the power of citizens and politicians understanding how much data is available from public bodies.

London has released live position data for their buses for several years now. Anyone can create a Google map like this, showing the real-time position of all the buses in the city.

Some of these online maps may appear to have no purpose other than to astonish the viewer – look at how much data is available! However, when politicians start using Big Data projects to help citizens then the value of Big Data is clearly going mainstream and being understood by the general public.

Could Big Data help to prevent the spread of Ebola?

IBA Group
Mark Hillary

I’ve written about Big Data in several blogs here. What it is. How it can be defined. And even how it can be used, but there are two additional factors that really help with any understanding of how Big Data fits into the modern organisation.
How do you make it work – what tools are needed to move from just having a large amount of data available to being able to gain insights from that data?

Do I have the kind of data that I can get insights from? What kind of insights am I expecting from the information that I have?
I have added two links to recent news stories that can elaborate further on these questions.

It is no use to anyone if you just have an enormous amount of data, but no tools to analyse it with. You can’t engage with Big Data using an Excel sheet. The volumes involved are often enormous and far more than what you could load into a computer to be studied as a single block of information.

So this is one of the first issues to address, and it may be where an expert partner can help the most. Is your data available? What tools can you use to collect together all that structured and unstructured data, and how can you even start to analyse it?
Now, what are the kinds of insights you can find? It depends on your business and the type of data available, but with Big Data you can uncover all kinds of relationships between variables that were not visible without the analysis.

This example of how Big Data is helping to predict where Ebola will strike next is a great example. It is just taking information we already have such as infections, locations of hospitals, number of doctors and so on, but using past knowledge and these factors to make predictions. Now imagine if you could start applying these insights in your business?

Making Big Data work with the right tools and determining the type of insight you need are two important factors in planning how you can make it work for you.

Will a Skills Shortage Threaten the Future of Big Data?

IBA Group
Mark Hillary

As the economy develops, old jobs vanish and new ones are created. This process has always taken place as technology creates new needs and old skills are replaced. Just consider how important the blacksmith used to be before cars were commonly used and if someone described themselves as a blogger or flash developer in 1985 it would have made no sense – times change.

Big Data is another of these major changes. Not just in the sense that we are becoming able to analyse larger sets of data thanks to the technology becoming faster and more powerful – especially with more memory being available, but because the ability to do this work is now a skill itself. Understanding Big Data and being able to manipulate and analyse large sets of data is a popular skill to be exploring now as every analyst predicts that Big Data use is set to explode in the coming years.

The analyst firm IDC wrote a study in 2012 that predicted the amount of data available to analyse would increase 50 times by 2020. This prediction remains true – if anything it may be even more by 2020 as new applications that create data are launched all the time – from smart watches to other wearable technologies.

All this has led to a fear that there will not be enough people available to work on Big Data projects. McKinsey believes that the US will need almost 200,000 new data scientists by 2018 and the British Royal Academy of Engineering has predicted a need for 1.25 million graduates in science and technology subjects by 2020.

A recent column in the British ‘Daily Telegraph’ claimed that this shortfall in data scientists might even be a threat to the future of business. But what all these concerns in the UK and US often fail to acknowledge is that there are many other countries with an abundance of data scientists.
Offshore outsourcing has been long proven as a strategy for software development and other IT requirements. It will be exactly the same once data science becomes a mainstream part of every business. And companies like IBA Group are already doing it today.

Is Jargon Preventing an Acceptance of Big Data?

IBA Group
Mark Hillary

The Smart Data Collective blog recently published a view that there is too much jargon circulating in the industry related to Big Data. In fact, as I mentioned in my last blog, Big Data is itself a term that is often misunderstood and needs more clarity.

The blog is interesting because the author takes a good example of an over-used business term, ‘digital’, and explores what we mean when we read and use this term. Many of the definitions from the dictionary have nothing at all to do with the definition of digital business you might expect – modern, hi-tech, and connected.

In fact, many more terms are taken from the dictionary and bent and shaped into something new by technology companies. Innovate, disrupt, and thought leadership are all terms that mean something different if you are not working for a technology company, but how can we improve the use of Big Data as a term?

The advantage we have is that Big Data is a genuine and meaningful area of data science. It’s not just jargon created for use by MBA students as they discuss their plans for ‘wealth-generation’.

Big Data needs to be understood by the general public and by the company leaders that have never really felt that they had to understand technology before. But almost everyone has now used Facebook, or contacted a customer service centre, so it is becoming easier to connect the theory of how Big Data can be used to the ways in which people see it every day.

Understanding Big Data

IBA Group
Mark Hillary

Big Data is a subject we have explored often on this blog because it’s an area where IBA has extensive experience and knowledge, but it is often difficult to explain. How big does a database need to be before it can be considered ‘Big Data’ and why do we need this separate terminology to refer to manipulating and analysing this data when relational databases have been in use for decades?

One example that goes a long way to answering these questions is the way that customer service is changing – especially for retailers. Products used to have a phone number and email address that customers could use to reach the manufacturer or retailer – usually to complain after a purchase.

Now, customers use online forums, review sites, Twitter, Facebook as well as the more traditional and direct channels such as online chat, emails, or a voice call. Customers are not always directly contacting a brand when they comment on a product, yet they usually expect a response.

Exploring this mass of information is a classic Big Data example. The retailers want to connect together all these communication channels into an ‘omnichannel’, yet this is impossible when they are considered to be regular databases.

If a customer emails a complaint, then tweets about the problem because their email is not answered and then finally calls because neither tweet nor email has been answered then the ideal situation for a retailer is that the agent on the phone knows about the earlier email and tweet.

But to make this work is not easy. The company has no control over Facebook or Twitter – it’s not internal data. And how can comments on a tweet be connected to a customer on the telephone?

All this is feasible, if you have enough information from various sources and you can analyse it quickly enough. Every company that interacts with their customers is now exploring this problem so maybe Big Data is about to hit the headlines again.

Big Data and Cloud in decision-making

IBA Group
Aleš Hojka, CEO of IBA CZ
Vitězslav Košina, Business Consultant at IBA CZ

It is a reality today that organizations have to deal with a multitude of unstructured documents and other data. These data have true value, if they are properly and timely processed and extracted, and also are supplied with really useful links.

Almost any business has in some way implemented a data management system (DMS), a content management system (CMS) or a business intelligence (BI) solution. Unfortunately, a new system often provides a very low added value, especially when a large amount of data is involved. Getting reasonable output from unstructured data is problematic. Why is it so?

DMS, CMS, and BI systems can be really effective, if they meet two essential requirements. The first is a sufficiently flexible environment and quick access to information complemented by strong information security. A really flexible environment shoul be able to respond to resource and computing requests in a very short, practically in real time. This can be easily achieved by using a cloud-based solution. As these requests are dynamic and cannot be easily predicted at the time when the system is designed, cloud can be very instrumental. In addition, keeping resources for a «rainy day» is not an effective allocation of resources.

Therefore, cloud is a prerequisite for dealing with big data, but it is not the only one. In some cases, data and documents are not available anytime and anywhere without limitations, though we have an environment designed for large amounts of data. This is true.

It should be noted that effective decision-making involves an increasingly growing amount of information. As the information should be available on mobile phones and tablets, an information management system cannot transfer huge amounts of data and should have short response time. If we meet these two basic requirements, then we are poised for truly efficient extraction and processing of large amounts of data, which can be further aggregated and analyzed to make a grounded decision, deal, and etc.

For many companies, using Software as a Service (SaaS0 is a problem, because they have to do with sensitive commercial information or client data. From the perspective of a cloud-based solution, a dedicated cloud is needed as apposed to shared capacities.

Many organizations outsource only the infrastructure, while we recommend to outsource the entire solution. With a cloud solution, the issue of security is solved to the extent that is normal for corporate clients. In this case, only the needed capacity is commissioned and there is no need to reserve resources for the team that takes care of the infrastructure and applications. The organization can thus focus on business and management objectives, as well as provide added value to the clients.

With mobile devices, security is a very thorny issue because attacks on their security are more likely than on the computers that are located in the company’s premises, where they are protected physically. The good news is that these devices are so powerful that allow for implementing the strongest encryption and other elements of modern security.

Taking into account the links between documents and their associated metadata, as well as other data sources, cloud computing is the best solution one can choose. It is however possible, if needed, to move from SaaS to the model, in which the solution is managed by the customer and yet not loses the flexibility that cloud offers us in terms of resources. Using cloud is much cheaper than building one‘s own infrastructure. Moreover, it enables organizations to concentrate on their requirements and business needs instead of specific software or infrastructure.

Internet of Things

IBA Group
Mark Hillary

I have written recently about Business Analytics (BA). What is BA? How does it affect your IT strategy and your business in general? I have also observed that there is a relationship between BA and Big Data (BD) – they are related concepts.

To clear up any confusion, I would say that BA is related to taking a set of data, performing a modelling operation, and using the model to predict some kind of future state – what-if calculations. BD is more of a continuous analysis of very large-scale business information.

But the business concept that is driving forward the importance of both Business Analytics and Big Data is really The Internet of Things (IoT).

Even for a fairly short blog post, this is already starting to fill up with three-letter-acronyms so let’s define what the IoT really is. More and more devices are capable of communication using the existing Internet infrastructure. It used to be computers that we would connect to the Internet, then laptops, then smart phones. Now it is tablets, ebooks, televisions, and every corporate electronic system you can think of – from security systems to electricity meters to photocopiers.

This revolution in making almost every device connect to the Internet is the starting point for the IoT. The classic consumer example is usually the connected fridge that can recommend a dinner based on what is inside, though a more useful example might be your car diagnosing a problem and communicating with the service centre without your own interaction.

In 1999, about 250mb of data per person was created each year. By 2012 ten times this amount of data per person was being generated. Data creation is increasing and the speed of increase is accelerating. Every day people are generating data with their smart phones without doing anything – just by switching it on, connecting to the Internet and allowing applications to work in the background.

This change in both the consumer and corporate environment is driving both the need for continuous Big Data analysis and also the ability to predict what may happen next based on Business Analytic tools.

How is Business Analytics Used in the Real World?

IBA Group
Mark Hillary

As I have mentioned in the last couple of blogs Business Analytics (BA) is about using data and models to make better decisions. This can help governments to improve policymaking and companies to develop better strategies. The aim is to use information to improve the day-to-day performance of the organisation.

But what are the real business decisions being taken with BA tools and how does the data analysis process improve the decisions?

    – In the airline business, employees and aircraft need to be in the right place at the right time to maximise efficiency. This is a complex planning procedure when things are going right, but when a flight is delayed or a key staff member is ill or a plane requires unscheduled maintenance then the knock-on effect can be huge. Analyzing the possible options is a great example of how BA can influence real decisions.
    – Hotels want guests in their rooms as often as possible. Empty rooms in a hotel mean lost profit and a hotel that quickly sells out a particular night may have been able to charge more for the rooms on that evening. Analyzing past customer behaviour, the competition, and other influencing factors such as big sport or music events in the neighbourhood is another classic example of how data can feed into a specific business decision – how much to charge for a room.
    – Banks and finance companies need to make quick decisions about whether to lend money to a particular customer based on limited information – such as the salary. However there are many other factors that could determine whether the individual is good for the loan or not and these can all be quickly factored together to provide a decision.
    – Supermarkets always seem to have full shelves these days. Can you remember Saturday afternoons where many big supermarkets would run out of important products? With the entire history of each product and how it sells in each store it is now far easier for supermarket managers to ensure they are ordering the correct stock levels.

These are just four entirely different industries, but in each one it is clear that Business Analytic tools are changing how managers make decisions in those companies.

Better decision-making leads to increased efficiency, a better use of existing resources, and the opportunity to perform better – to earn more by delivering a better service. For this reason, managers in all industries should be thinking about what Business Analytics can do for them.

Selecting the Right Business Analytics Partner

IBA Group
Mark Hillary

IDC is one of the leading global industry analysts, so it’s always interesting to see their own guidance on choosing a supplier. Their recent report ‘IDC MarketScape Excerpt: Worldwide Business Analytics Consulting and Systems Integration Services 2014 Vendor Assessment’ focused exclusively on the questions you need to ask when searching for a Business Analytics partner.

1. Pay attention to domain knowledge. Over the years, some service providers have built deep industry expertise across certain business needs.

2. Create a culture of analytics. It is not enough for you to have access to the right data, you need to create the processes that can make use of this data across your entire organization.

3. Don’t neglect the basics. Companies often find they have data issues once the migration has commenced, which will then delay the entire migration project. Stop and avoid all these roadblocks by taking the data cleansing stage seriously so the migration can run smoothly.

4. Align the strength of your supplier with project success. If you find a good match then ensure that your partner gets stronger as the project succeeds.

Of course, point 5 from IDC was to utilize their own research when selecting a supplier. Most of these points are what any manager with a good experience of outsourcing would be planning anyway, but it cannot be stressed enough that good planning for the migration and creating a culture of data analytics are essential for success.

Business Analytics and Big Data

IBA Group
Mark Hillary

Business Analytics is a topic that is often confused with Big Data. While the analysis of Big Data is related to the continuous analysis of business information through an analytical process, they are related concepts rather than exactly the same thing.

There are various kinds of analytics to start with:

Descriptive Analytics: how to gain insight from historical data, creating reports and scorecards that give a better vision of some existing data.

Predictive analytics: modeling through the use of predictive models and machine learning – allowing the system to learn what might happen next based on the data that is being studied usually in real-time.

Prescriptive analytics: taking a large data-set and attempting to create decisions, choosing possible paths, simulating what might happen if certain decisions are taken.

Decisive analytics: this supports human decision-making with very visual analytic information that helps the user.

So the field of Business Analytics is more related to the process of taking data and either modeling outcomes or predicting what may happen next, rather than just attempting to spot trends in a large data set.

Business Analytics is really a tool that can support executives to make better decisions by supporting their decisions with data, rather than just estimates or guesses. By using actual data from the business and modeling potential outcomes based on decisions that could be taken, the data can help to support the direction a business leader should take.

This type of process has existed for a long time, but it has been the creation of vast pools of business data – the move towards a Big Data environment – that has really stimulated the need for improved Business Analytics. The increased amount of data has provided more information that can be analysed and yet has also made it more difficult to reach a conclusion on the right decisions – without better analysis.