FinOps for Private and Public Cloud Consumption to Avoid Budget Overruns
Cloudera, the hybrid data company, has announced the general availability of Cloudera Observability for all customers using Cloudera Data Platform (CDP) on public or private cloud environments. This solution enhances the open data lakehouse implemented with CDP by providing actionable insights for data, applications, and infrastructure components. Its goal is to optimize costs, automatically resolve issues, and improve overall performance. The announcement was made at The Six Five Summit, an analyst event taking place from June 6-8, 2023.
According to Rob Bearden, CEO of Cloudera, one of the major challenges faced by companies today when managing cloud workloads is gaining a comprehensive view of infrastructure and service expenditures. With Cloudera Observability, customers gain unprecedented visibility into workload and resource utilization, enabling better control over budget, automatic management of cost overruns, and performance improvements.
In the current landscape, hybrid and multi-cloud environments have become the standard. As enterprise data is stored both on-premises and potentially across multiple public clouds, it becomes challenging to track and manage cloud consumption across different departments, cost centers, and infrastructures. Cloudera Observability tackles these challenges by leveraging the company’s experience in hybrid data solutions. It empowers customers to monitor, understand, and optimize their CDP deployments, offering customizable automatic actions and pre-built actions to raise alerts, proactively prevent issues, and optimize workloads.
Nitish Mittal, a Partner at Everest Group, notes that many companies struggle with cloud costs and maximizing the continuous value from their cloud investments. Cloudera’s metadata-based approach in CDP allows users to effectively manage their cloud costs. It includes identifying rogue users, coordinating workloads for optimal cost, and flagging potential resource overruns. This becomes increasingly crucial for enterprises seeking effective management of their data on cloud platforms.
An example of Cloudera Observability’s effectiveness comes from a customer who had a rogue user generating millions of unnecessary queries that severely impacted essential workloads’ performance. The support team, equipped with all the necessary insights into the customer environment, quickly identified the root cause of the issue without requiring multiple interactions. This expedited support approach reduces operational overhead, allowing for faster issue resolution.
Cloudera Observability is provided at no additional cost as part of applicable subscriptions to CDP. It focuses on optimizing frequently used data engines such as Hive, Impala, and Spark for data engineering workloads. For advanced capabilities, customers can opt for Cloudera Observability Premium, which includes custom auto-actions, deeper insights, and richer automated troubleshooting. Support for new data engines and platform components will be added over time.
Cloudera Observability is interoperable with Apache Iceberg, a key component of Cloudera’s open data lakehouse delivered through CDP. Apache Iceberg is a high-performance open table format for large analytic tables that brings reliability to big data while supporting concurrent work from multiple compute engines.
Cloudera’s open data lakehouse enables organizations to perform quick analytics on both structured and unstructured data at a massive scale. It eliminates data silos and allows data teams to collaborate using their preferred tools on any public or private cloud. Cloudera Observability contributes to a more cost-effective outcome across all of CDP’s functions, delivering an enhanced experience for enterprise users. This becomes increasingly important as companies advance their data management efforts to support large language models and other AI initiatives in hybrid and multi-cloud environments.