Storage Challenge: Longer Data Retention Drove New Capacity Demands
To manage log data, the firm employed Splunk, a powerful and scalable data management solution. But when their log data retention requirement soared from one month of retention to 13 months, they saw a dramatic increase in storage cost and complexity. A simpler, more scalable, and less costly storage solution was essential.
Object Storage Provides a Simple, Scalable Answer
The organization found a strategic partner in Cloudian, a leading provider of object storage solutions. Cloudian’s scalability and compatibility with Splunk Enterprise platform made it the ideal choice for addressing the pharmaceutical firm’s growing storage challenge.
Moreover, Cloudian seamlessly integrated with Splunk SmartStore, a next-gen architecture, designed to make Splunk implementation more scalable and cost effective by offloading less-frequently used data to limitlessly scalable, always-on, external object storage.
Cloudian Replicated Clusters Provide Data Availability
To meet their new storage needs, the firm deployed two Cloudian clusters, each with one petabyte (PB) of usable capacity. The clusters were deployed at the company’s two data centers and were configured as one primary and one replicated, ensuring data availability. This robust setup not only addressed the immediate capacity requirements but also laid the foundation for future scalability.
With Cloudian in place, the pharmaceutical firm successfully mitigated the challenges posed by the extended data retention period. The deployment not only met the increased storage demands but also delivered additional benefits. The organization leveraged Cloudian’s compatibility with Splunk SmartStore to create a dedicated repository for the “warm” tier of Splunk data. This strategic use of Cloudian significantly reduced Splunk Server costs, optimizing the overall data management infrastructure.
Cribl Observability Platform Further Improves Cost Efficiency
To further enhance their data management capabilities, the pharmaceutical firm also integrated the Cribl Observability Platform. This platform seamlessly connects to and ingests data from a wide variety of log data sources. By storing data with full fidelity in a Cloudian data lake — before it gets fed into higher level tools like Splunk for analysis — the company not only achieved comprehensive log data management but also realized significant cost savings in data ingest.
Conclusion
The pharmaceutical firm’s journey to optimize data management illustrates the success achieved by leveraging Cloudian’s scalable and cost-effective storage solutions. The combination of Cloudian’s compatibility with Splunk infrastructure and seamless integration with the Cribl Observability Platform has empowered the organization to navigate the complex landscape of electronic prescription transmissions and health information exchange efficiently. As the pharmaceutical industry continues to evolve, the company remains well-positioned to scale its data infrastructure while maintaining control over costs and ensuring data integrity.