No Compromise: Securing Big Data
Big Data platforms have become increasingly important to an organization’s daily operations. IT departments must quickly adapt, ensuring that the platforms are optimised for users as well as managed efficiently and in line with the continuously changing demands of IT infrastructure. One aspect, however, must remain constant during all the change: there can be no compromises when it comes to security.
Whether customer marketing data, test results in R&D, technical data for predictive maintenance, information in Hadoop database or in Data Lake, big data projects are unique in their ability to provide both granular details as well as a view on the macro-trends. The insights these projects provide frequently go to the heart of the issue and are invaluable in the strategic planning of modern organizations.
As with any critical application, a Big Data solution must be provided with flawless security. To meet this challenge, security must be part of the intrinsic properties of the Big Data platform, and be interwoven with it from its inception. Best described as “Security by Design", putting data security at the heart of any project, right from the start, is the only way to sustainably meet the dual requirement of robustness and scalability.
A risk analysis should be undertaken at the design stage, taking into account the incoming data and results that will be produced, as well as the nature and extent of potential threats. This makes it possible to anticipate and plan for the requirements to ensure data security, while at the same time, not limiting user access or data flow.
Critical to Big Data security is privacy, especially when it comes to personal data. Guaranteeing this will take several additional measures, including data encryption, to ensure that information is accessible and readable only to persons with the correct permissions. Using anonymised data, or separating the individual information from the identity of individuals via pseudonyms can work very well. Indeed, pseudonymising is a requirement of the impending recent General Data Protection Regulation (GDPR) set out by the European Parliament in April 2016. To strengthen citizens’ control of their personal data and user confidence in digital services, GDPR also imposes good practices in operating data, thus establishing a necessary level of compliance for Big Data platforms.
Another security aspect to be factoring into any Big Data project is data integrity. To ensure the accuracy and consistency of the results of the analysis, it is essential to work on data that has not been altered between copies and processing in IT systems, allowing for certification signature and traceability systems.
The overall safety of the Big Data platform can be achieved via secure access, applying a fine management of access rights – such as the Zero Trust model – to manage and track access rights at the individual level, and data leak prevention to avoid copying of the data on to external storage devices.
Finally, everything will be under global supervision which will identify in real time any cyber-attack or attempted unauthorised access and to intervene before the intruders reach the data.
With these multiple controls in place, Big Data is provided with a custom defence platform that is both flexible and robust. What it protects, as with the threats it protects against, is constantly changing, and for it to remain permanently effective organisations must conduct regular security audits, including penetration testing.
Ultimately, this approach must be considered from the project’s inception – and built into its core. This is the only way to ensure that security and functionality evolve hand in hand throughout the life cycle of the solution.