Losing the big data adoption race? Or just following from a safe distance?


The speed of Big Data adoption is a much hyped element of our online world, and where data driven organizations seem to be thriving.

As a concept, Big Data is nothing new. It’s origins stem from the earliest days of computing when societies (large businesses and governments) started massing vast amounts of data for a range of purposes such as tax collection, research, surveillance or census analysis. Over the last few decades, the combined influences of the internet, collapsing cost of compute and storage services and widespread use of all things cloud-based (from smartphones to social media) have seen an explosion of data being generated, stored and repurposed – globally.

Widespread Big Data adoption is a reality and a now a seemingly irreversible fact of life. The volume of data being collected, aggregated and analysed by major corporations, governments and the scientific community alone is big, to say the least, and increasing exponentially.

Given that it’s been a number of years since the term Big Data has been in widespread use, I thought it would probably a good time to take stock of where we are at down the Big Data adoption evolutionary time-line, and more specifically, what it means for your organisation.

For the moment, setting aside the ongoing debate over the collection, ownership and protection of our personal information globally, through sources such as our mobile device usage patterns or internet usage, it is probably timely to ask:

Question: What does the reality of Big Data adoption for my organisation look like?

Context is king in the Big Data adoption

The combined influences of the IT industry itself and technology evangelists are very effective in fanning the flames of hype and enthusiasm for the latest big thing.  Making sense of this hype is not always a straightforward exercise when attempting to identify its relevance to your organisation. Given that Big Data initiatives appear to be surfacing across the board – question is:

When it comes to Big Data adoption, how will you go about testing the assumption that ‘if it’s worked over there, then it’ll work here’ in the context of your organisation?

The fact that the analyst firm Gartner positioned Big Data in late 2014 as passing the peak of the hype cycle and starting down the Trough of Disillusionment, lends weight that appropriate due diligence should be exercised when it comes to tackling your Big Data initiative.

Big Data can be a game changer and deliver real value if it supported by the right business case and adopted at the appropriate point in time for your business. Whilst Big Data offers the potential for realizing exceptional value in the right hands and for the right reasons, it also has the potential to expose your organisation to additional risk if poorly done.

Any important investment should be sent through the rigour of an appropriate business case assessment, and Big Data is no exception.

Not doing so could be akin to gambling – the only difference is that, get it really wrong, and the risks may extend to more than just losing your money on a dud project.

Afraid of missing the ‘Big Data’ roadshow?  Fear not – you’re not alone.

Analysts and the media are awash with impressive projected (and actual reported) compound annual growth rates (CAGR) of spending in Big Data. Despite these compelling numbers, the reality is that the majority of organisations are still coming to terms with Big Data.

According to Capgemini’s January 2015 report, only 13% of organizations have achieved full-scale production for their Big Data implementation.  More telling is the fact that ‘only 27% of respondents described their Big Data initiatives as “successful” and only 8% described them as “very successful”.’  These trends are also echoed elsewhere.

Keeping a finger on the pulse

Whilst it is important that you and your organisation keep a finger on the pulse of Big Data trends, your competitor’s initiatives, business opportunities, pitfalls as well as challenges,  it is important to test the rationale for your investment in any Big Data initiatives.

The same applies, of course, in the adoption of any of the latest IT innovations, trends and technologies in your organisation. Fortunately, there are a wide range of approaches at your disposal when it comes to adopting new, disruptive or innovative technologies in your organisation.  This can range from a limited scope proof-of-concept project based on lean startup principles, through to tackling a major enterprise-wide initiative.

The key is to ensure that you and your organisation take on any Big Data initiatives expecting that changes need to be made. These changes will often have less to do with the technology itself and more to do with the leadership capabilities, organizational structures, culture and collective willingness to change.