Why do so many big data projects fail? (And how can we make them succeed?)
The value of data analysis has been proven time and time again – information-centric businesses are already 20% more profitable and have twice the market value of their peers. Despite the success stories however, it seems that all too often we’re seeing big data projects fail or not managing to last beyond the pilot phase. In fact, nearly four in five (78%) executives confess that they are struggling to understand how to get real value from big data, and most organizations continue to use just 12% of their available data. What therefore, are the barriers preventing organizations from making them viable and what’s the secret to planning and implementing a successful big data project?
Frequent causes of failure
Big data projects often fail because of vendor pressure driving senior executives to “do big data” before they’ve identified the problems that need to be solved. In this scenario, one-size-fits-all solutions are purchased and deployed quickly, but remain underutilized, if utilized at all, by real business users. At the other end of the scale, some businesses are yet to recognize the importance of big data projects, leaving them to fall behind the curve.
Failure to deploy a company-wide data strategy also leads to projects being unsuccessful. When separate departments each undertake their own data experimentation, senior management is often unaware of what is going on across the business, leaving projects in siloes, and shadow IT becoming a major security risk.
There are however, seven key principles that can help improve the success of business analytics and big data projects, which I’ve outlined below.
- Focus on business insights
Many executives are under pressure to deploy analytics technologies before they know what needs addressing. To get the right software and infrastructure in place however, it’s essential to identify the areas of the business that will most benefit from big data projects first and only deploy systems that will help address these.
- Start small
IT projects are often associated with long and complex processes, but when technology evolves at such a rapid pace, organizations would be better off experimenting with small proof of concepts before making any major investment.
- Learn from best practices
Big data has already proven its potential across many different industry sectors, and to take advantage in their own organizations, senior management should identify existing use cases, replicate best practices and develop new specific analytic applications.
- Emphasize both standards and flexibility
Many businesses tend to run into problems by deploying multiple solutions that offer quick fixes. While this approach may bring some benefits, it often leads to security or integration nightmares when it comes to upscaling or integrating between siloed data projects. Taking a standardized approach that is open to integration and personalization is key to success here.
- Speed and performance make all the difference
In a digitally accelerated world, success will lie in being able to process big data faster than any other player in the market. In financial services, microseconds can make all the difference between millions gained or lost on high-speed digital trading networks, and the same is true of big data. Applying High Performance Computing (HPC) technologies to next-generation analytics can help organizations process billions of operations per second, vastly improving the performance of certain platforms.
- Stay open to innovation
Technology is advancing at such a pace and to keep up with these developments, organizations must practice continuous innovation and encourage open standards and interoperability between different partners. This co-innovation approach will enable businesses to take learning from others in the market and adopt best of breed practices.
- Keep security front of mind
If data is the new oil of tomorrow’s economy, it’s not difficult to ascertain where tomorrow’s hackers will devote their efforts. Data is now a primary target, and when it typically takes more than 200 days before an intrusion is discovered, millions of customers are at risk of having their personal information exposed to unwanted eyes. With reputation, IP, customer trust and regulatory compliance at sake, organizations must make security a core foundation for all business analytics projects.
The third digital revolution takes us into a world where data will be the ‘new black gold’ and advanced analytics capabilities will be a key business differentiator. To succeed, new approaches in data acquisition, processing and analysis will be needed; and organizations that adopt these seven core principles will be those that get ahead.