The data blackhole – and how you can avoid falling in it

If anyone asked you to do a complete audit of all of your organisation’s data, you’d probably break out in a cold sweat. And for good reason – trying to keep track of data has become a mammoth task – especially with the falling costs of storage and the demand for data-hungry digital technologies like the Internet of Things.

Yet, now more than ever, getting to grips with all this data and turning it into meaningful insights is the key to your organisation’s success. Data used well can give you a clear competitive edge – in everything from product improvements, to new revenue streams, to new efficiencies and productivity. And let’s face it, it’s really hard to justify not doing this when you’re storing it like really expensive junk in your attic.

The question then is, how do you do this, and moreover, how do you compare to your competitors? Here at Exasol, we conducted an in-depth study of two major European markets, the UK and Germany, and asked IT and business decision makers how they made of use of their data. The results are fascinating and, along with our supporting series of whitepapers, should arm you with everything you need to get out of your data blackhole.

Unseen, fragmented data

There are few problems more significant in any data project than an inability to find, access or otherwise fully exploit the data sources that hold the value a business should be unlocking. But that’s exactly what our latest research Moving the Enterprise to Data Analytics found. 

82% of organisations admitted to not knowing where all their critical data was located. For backup and archived data, that figure was 86% and for miscellaneous data was as high as 92%. 

Compounded with this lack of data visibility, our study revealed a growing problem with data storage. Corporate data has become so vast and fragmented that it’s now routinely difficult to extract fully-formed insights. More than half (55%) agreed that the fragmentation of data across multiple databases, local storage and disparate systems is preventing their organisations from fully extracting value from it. 

Organisations may be getting their hands on better data, and they may even see the possible insights that can be gleaned, but the data analytics they seek are often out of reach because that data is residing in multiple places. For example, they are often in proprietary applications, departmental Business Intelligence databases, or mounting up in data lakes. 

The knock-on effect - unused data 

This proliferation of ‘dark’, fragmented data has had predictable effects. Quite simply, many organisations simply aren’t using their data as much as they could. 

The study found that on average 54% of business decisions are data-driven. To break that down, some 15% of organisations said that less than 30% of business decisions are data-driven. A further 15% said that 50-60% of decisions are based on data, with broadly the same amount also saying that 60-70% of decisions were based on data. 

Forrester research suggests insights-driven businesses are growing at an average of more than 30% annually and are on track to earn $1.8 trillion by 2021.  Our research revealed that only 1% of organisations currently regard themselves as a data-driven enterprise. 

Given that data-driven business is widely acclaimed in the business community, it may come as surprise that so few have fully realised a completely data-driven model. But of course, it's not that companies are intentionally not using their data. They are simply trying to understand what it is that they have, which is understandable considering the high volumes of complex data that their technologies are pumping out. 

So what can be done? 

Better education and efforts to trigger cultural change in favour of using and trusting data insights to inform decision will bring this number up. As will putting the right tools in place to make it easy for users to extract insights from data. Success means that insights to inform decisions can be extracted quickly and effectively, be that in real-time or near real-time, and that means having the right database and analytics platform at the heart of your business. 

Find out how your progress compares to that of your peers, and the common challenges they are facing in our new white paper Moving the Enterprise to Data Analytics

My other articles can be found on our Exasol blog