How do you get started with Big Data and Analytics
From retail and financial services to healthcare and agriculture, businesses across every industry are turning to Big Data and analytics, recognizing it as a largely untapped source of value. According to the commissioned study conducted by Forrester Consulting on behalf of Atos, “The Future Belongs To Those Who Monetize And Maximize Their Data”, 40% of businesses who responded are already using data analytics across key business functions. Yet, despite the clear business benefits of Big Data, many don’t know where to start in terms of making sense of the wealth of information that stands before them. The same survey revealed that 63% of organizations are just maturing in this space. Here, I offer my advice on how to take that first step.
- Identify use cases
When starting a Big Data project, it is important that the reasons for doing so are identified and defined first. These include factors such as getting to know your customers better, personalising your promotions and reducing operational costs. To do this, start by informing individuals from different backgrounds of the principles of Big Data and analytical technologies so that everyone can be quickly brought to an advanced level of understanding on the subject. Next, run group workshops to enable everyone to exchange their ideas. From these brainstorming sessions, one or more priority use cases should emerge.
I would also recommend bringing in an internal sponsor to defend the interests of the project to the management of the company and to give the project the attention and support that a Big Data initiative needs.
- Qualify your data
The next step is to identify and qualify the available information (internal or external to the company) and to determine the relevant data for the selected use case(s). The sources of data should then be highlighted, as well as their origin (e.g. internal, partner-based, social or a public body) the data type, format, availability and finally the quality.
- Prepare the project
To move quickly and avoid bottlenecks during the implementation stage of a Big Data project, deploy an agile approach based on a multidisciplinary team that brings together all the necessary skills. This includes Big Data developers, data analysts, expert technicians, data scientists, Big Data architects and a project manager.
It is also wise to carry out a Proof of Concept (POC) first, which can help to validate a use case through the setting up of a Data Lab (a testing platform) and initiate the transition to the industrialization phase.
- Set up the platform
Of course, not all Big Data platforms are the same; and the architecture and components will depend on the customer’s needs, use cases and available data. The tools to be used should be assessed on a case-by-case basis, according to their needs:
- Collection of data
- In order to set up the appropriate collection channels, make sure you’re clear on the volume and depth of historical data required, as well as the nature and complexity of the flow of data
- Next, look at the preparation and the quality of the collected data: does it need to be sorted, cleaned, enhanced? If so, which processes and tools should be used?
- Determine the choice of infrastructure and type of storage desired (such as a relational database, NoSQL and/or Hadoop cluster) and the expected performance
- Processing and analysis
- Depending on the use cases, you will have to select a suitable method of analysis. Ask yourself whether it should be in real-time with a delay? Are you looking for correlations, anomalies or trends; or are you looking to discover, explain or predict? Each case requires data correlations to see the implementation of specific predictive algorithms
- Data visualization
- Provide users with information that is easy to understand and access using the appropriate visualization tools
Big Data projects are becoming increasingly more widespread in business and are being scaled as their benefits are proven. To ensure projects can be deployed successfully, it is critical to implement rules of good data governance so that all security requirements (authentication and accessibility), confidentiality, traceability, backup and access can be respected.
For more information about Atos Codex, the Atos brand for advanced big data analytics, IoT and cognitive services, see our previous articles on how agile analytics is impacting various industries and how organizations can deploy the best data analytics approaches to monetize and maximize value from their data.