Henry Golas, Director of Technology, Cloudian
Historically, Object Storage has been viewed as a cheap and cheerful archive for large amounts of data. These archives were the ‘last stop for data’, data that needed to be kept for long periods of time, but not accessed very frequently. The primary focus was storing data as cost effectively as possible, without much regard for other requirements.
As applications transition to a cloud native world, the requirements for data storage have changed. Data needs to be available, accessible via S3 API, and performant. This shift has moved Object Storage from a “Dollars per GB” to a “Dollars per IOPs” model as today’s requirements now justify flash media for Object Storage platforms.
Three use cases pushing the performance requirements of object storage:
1) By leveraging the power of custom metadata, Object Storage is gaining momentum with applications that store transactional data in object format. As an example, imagine a financial application that bundles all the relevant supporting documentation for a transaction, like a mortgage, with the object and adds searchable metadata. Downstream applications would be searching and reading the same objects. Flash based Object Storage is required to keep up with the transactional throughput of the application.
2) Data streaming and Internet of Things (IOT) now generate Terabytes and Petabytes of data per hour. Using Autonomous Vehicles as an example, daily data generation estimates are from 3TB-15TB per day, per vehicle. Testing and training a fleet of AVs now requires flash based Object Storage to keep up with the data volume.
3) Data analytics and big data players are extending their data lakes with Object Storage. This allows the separation of compute and storage resources. Why buy compute when storage is required? Hadoop supports S3A requests for S3 compatible object storage, allowing for direct read and writes of S3 objects. Companies like Vertica (https://www.vertica.com/) and their Vertical Analytics Platform support S3 for communal storage, allowing compute resources to be scaled independently. Data analytics and big data need performant object storage to keep up with modern demands.
The net here is that modern applications aren’t just storing data and forgetting about it. These applications are actively using Object Storage as a performant storage platform.
Cloudian HyperStore is able to meet the demands of modern applications with a flash based Object Storage offering. Cloudian delivers this software defined storage solution as a turnkey Cloudian appliance or on standard x86 hardware.
To further increase performance and drive cost efficiencies, Cloudian HyperStore’s data tiering functionality allows a smaller subset of hot data to reside on flash object storage, while the remaining cold data set stays on traditional HDD technology.
This tiered solution blends the traditional cost centric “Dollars per GB” approach, with the “Dollars per IOPs” requirements of modern applications, allowing up to 40% cost reductions vs an all flash-based solution.
Check out more details at cloudian.com/flash
