At the confluence of HPC and Big Data
With ever increasing volumes of data created by new sources of information in our connected world, managing Petabytes of data has proven to be a real challenge. Leveraging our long-standing and successful experience in implementing massive data management solutions based on multiple technologies, Atos now proposes not only tailored solutions, but also a set of Bull tools dedicated to storage for high performance computing, such as Smart Data Management Suite, Hierarchical Storage Management and Bull Director for HPSS.
Smart Data Management Suite
Customers are often facing lower than expected efficiency for their application workloads. Tools to investigate runtime behavior of applications are precious to understand where the bottlenecks are.
As I/O is one of the least known part of job executions, Atos has developed a comprehensive set of tools to understand and improve efficiency of running applications in complex workloads.
This collection of software and hardware tools already available are :
- Bull IO Instrumentation as an application I/O profiler
- Bull IO Pattern Analyzer as a multiWjobs comparison tool and accelerators usage automation
- Bull Fast IO Libraries as software accelerator modules
- Bull Smart Burst Buffer as a versatile Flash storage accelerator
A smart and focused use of these tools will provide valuable information on running applications and facilitate investigations on unoptimized jobs among complex workloads.
Hierarchical Storage Management
As data sets with hundreds of terabytes or many petabytes of data become increasingly common-place, Hierarchical Storage Management (HSM) becomes a cornerstone of successful intensive data management systems. HSM is a data storage technique that automatically moves data between high-performance / high-cost storage devices and low-end / low-cost but more capacitive storage devices. To optimize costs, HSM systems store the bulk of data on slow, economical devices such as robotic tape libraries, and copy data to faster disk drives whenever they are needed for processing.
HPSS (High Performance Storage System) is an HSM system born from a Collaboration between IBM Global Business Services and 5 American DOE laboratories, augmented lately by the French CEA/DAM. Atos has a long experience of implementing HPSS in challenging environments where long-term data preservation and re-use of massive data sets are key.
Bull Director for HPSS: improve HPSS performance the smarter way
Atos developed Bull Director for HPSS to remove some HPSS bottlenecks.
Bull Director for HPSS is an add-on to this HSM. It improves the global performance of HPSS for tape libraries, in an environment with many concurrent access requests, by grouping simultaneous requests from different users in real time.