Most Kubernetes teams track pod CPU and memory. Almost none track what the cluster actually costs to run.

We decided to change that. For the past 6 months, we have been running KubeLedger on our own OpenShift cluster — the one we use daily to manage hosted control planes, multi-cluster operations, and security workloads.

The results were eye-opening. Here is what we found, with real numbers and zero embellishment.

1. The Hidden 30% Tax

Every Kubernetes node reserves resources for the operating system, kubelet, and system daemons. These resources are called non-allocatable — they are consumed but can never be scheduled to pods.

24–30% of total CPU was non-allocatable — consistent across every month for 6 months, never dropping below 24%.

On our cluster, non-allocatable overhead consumed between 24% and 30% of total CPU every single month. That means for every 10 cores you pay for, 2.5 to 3 cores are invisible to kubectl top nodes.

Most monitoring tools report namespace-level usage and call it a day. They show you what pods consumed, but not the gap between what the node provides and what can actually be scheduled. This gap is your hidden tax.

KubeLedger tracks both sides: allocatable and non-allocatable. Because you cannot optimize what you cannot see.

2. Where CPU Actually Goes

When we broke down CPU consumption by functional category, the picture became clear:

Only ~14% of CPU was available for hosted workloads. The remaining 86% was infrastructure tax — the cost of running the platform itself.

This is not unusual. Most enterprise OpenShift clusters with ACM, ACS, and ODF will see similar ratios. The difference is whether you can see it or not.

3. Growth Nobody Noticed: 840 to 3,708 Cores

KubeLedger tracked total cluster CPU consumption month by month:

+341% CPU growth from September 2025 to January 2026 — driven by HCP rollout, ACM scaling, and ACS deployment.

The jump from September to January was driven by three events: the rollout of two hosted control planes (December), ACM scaling to manage additional clusters, and the deployment of the full ACS security stack.

Without historical tracking, this growth would have been invisible until the infrastructure bill arrived. KubeLedger stores 12 months of history with zero database management, powered by RRDtool.

Capacity planning starts with data. Not guesswork.

4. Top 5 Consumers Own 72% of CPU

In February 2026, just five namespaces consumed 72% of all allocatable CPU:

This is per-namespace accounting — the foundation for chargeback and showback. When a platform team can show exactly which components consume which resources, conversations about infrastructure cost become data-driven instead of political.

KubeLedger is the System of Record that makes this data available without Prometheus queries or custom dashboards.

5. Memory: The Silent Budget Killer

CPU gets all the attention in Kubernetes capacity discussions. Meanwhile, memory quietly drains budgets.

In January 2026, our cluster consumed 25 TiB of cumulative memory:

Memory is often the first resource to hit limits — and the hardest to debug when OOM kills start cascading across pods. Unlike CPU, which can be throttled, memory pressure causes immediate evictions.

KubeLedger tracks CPU and memory side by side, per namespace, over time. Because optimizing one while ignoring the other is half a strategy.

6. 88 Namespaces, Zero Configuration

Over the 6-month tracking period, our cluster grew from 77 to 88 namespaces:

KubeLedger auto-discovers every namespace on the cluster and starts tracking immediately. No labels to add. No annotations to configure. No Prometheus rules to write.

Every new namespace was automatically picked up — including the HCP namespaces that appeared in December when we rolled out hosted control planes.

7. Hourly Trends Reveal What Daily Averages Hide

KubeLedger captures hourly granularity, revealing patterns that daily averages smooth away:

• API Server: peaks at 0.7 cores during business hours, drops to 0.4 at night

• ACM: bursts up to 1.0 core during reconciliation cycles, with a baseline of 0.66

• Monitoring: fluctuates 0.3–0.7 cores depending on scrape intervals and alert evaluation

• HCP #2: shows the widest variance (0.45–1.25 cores) due to workload scheduling patterns

These patterns matter for right-sizing. If you provision based on daily averages, you will either overprovision (wasting money) or underprovision (risking throttling during peaks).

Daily averages hide the peaks. KubeLedger does not.

8. A Cost Monitoring Tool That Costs Almost Nothing

We believe a cost monitoring tool should not cost you a fortune to run. Here is what KubeLedger consumes on the cluster it monitors:

0.016 cores / 12 MiB — that is 0.002% of total cluster CPU, tracking 88 namespaces with 12 months of history.

For context, KubeLedger uses less CPU than a single sidecar container. The tool tracking 88 namespaces with hourly, daily, and monthly granularity runs on 16 millicores.

How? No Prometheus dependency. No external database. No storage backend to scale. KubeLedger uses RRDtool for fixed-size, append-only storage that never grows beyond its initial allocation.

Less than 100 MB of storage for 1 year of retention. Deploy and get answers.

Conclusion: You Cannot Optimize What You Cannot See

Six months of tracking our own cluster taught us three things:

1. The hidden tax is real. 24–30% of CPU is non-allocatable and invisible to most tools. If you are not tracking it, you are underestimating your actual infrastructure cost.

2. Growth is gradual until it is not. Our cluster went from 840 to 3,708 cores in 5 months. Without historical tracking, this is invisible until the bill arrives.

3. Per-namespace accounting changes conversations. When you can show exactly which components consume which resources, cost discussions become evidence-based.

4. And this is just CPU and memory. KubeLedger also tracks NVIDIA GPU utilization and memory consumption per pod — with the same per-namespace accounting, powered by DCGM Exporter metrics. As AI workloads drive GPU costs through the roof, the same visibility gap exists: teams pay for expensive GPUs without knowing how they are actually consumed.

KubeLedger is the open-source System of Record that tracks the full picture of Kubernetes costs — revealing the 30% hidden in non-allocatable overhead for precise, per-namespace accounting.

Try KubeLedger

Deploy on your cluster in under 5 minutes. Zero configuration. Zero database. Full visibility.

• Website: kubeledger.io

• GitHub: github.com/realopslabs/kubeledger

• Follow RealOps Labs on LinkedIn for more production insights.

Appendix: Methodology

• Cluster: OpenShift 4.x bare-metal, multi-node, running ACM, ACS, CNV, and ODF.

• Tracking period: September 2025 – February 2026 (6 months).

• Tool: KubeLedger (formerly kube-opex-analytics), deployed as a single pod with read-only RBAC access.

• Data source: Kubernetes Metrics API (resource.metrics.k8s.io) polled at 5-minute intervals.

• Non-allocatable calculation: Total node capacity minus allocatable capacity, as reported by the Kubernetes API.

• Storage: RRDtool round-robin databases. Fixed-size, no external database required.

All data in this article comes from real CSV exports of KubeLedger running on our own OpenShift lab cluster. No data was simulated or adjusted.

This is more than just a new logo. It represents a maturation of the project and a sharpened focus on what we do best: Kubernetes Resource Accounting.

Why the change?

When Rodrigue Chakode first architected this tool, the goal was to bring High-Performance Computing (HPC) rigor to Kubernetes efficiency. The mission was to track resources with extreme precision without the "heaviness" of standard observability stacks.

Over time, we realized that our users weren't just using us for "analytics" (looking at graphs). They were using us as a Ledger—a System of Record to allocate costs, charge back to namespaces, and hold teams accountable for their usage.

The name KubeLedger reflects this reality. We are the accountant for your cluster, designed to reveal the 30-50% of costs often hidden in unused resources and non-allocatable overhead.

What is changing?

1. The Name & Home

The project has moved to a dedicated GitHub organization backed by RealOps Labs: github.com/realopslabs/kubeledger. This ensures long-term stability, enterprise-grade focus, and continuous innovation for the community.

2. The Development Model: Ensuring Sustainability

To ensure the long-term sustainability of KubeLedger in an era where open source projects are often commoditized by large platforms without contribution, we are adopting the Business Source License (BSL 1.1).

What does this mean for you?

• For Users & Companies: Nothing changes. You can use, modify, and deploy KubeLedger internally for free, just as before.

• For Contributors: The source code remains fully available. Pull requests and community contributions are welcome.

• For Cloud Vendors: You cannot sell KubeLedger as a managed service (e.g. SaaS, PaaS).

Our Commitment to Openness: We believe in the open ecosystem. That is why our license includes a strict Change Date. After 4 years, every version of KubeLedger automatically converts to the permissive Apache 2.0 License, guaranteed. This model protects our ability to innovate today while ensuring the software eventually belongs to the public domain.

3. The Terminology

We are standardizing our environment variables from KOA_ to KL_. (Don't worry, the old variables will continue to work for the next 6 months to ensure a smooth transition).

What remains the same?

The Efficiency. We still believe that a cost monitoring tool shouldn't cost you a fortune to run. KubeLedger remains:

• Lightweight: < 100MB memory footprint.

• Maintenance-Free: No database management required (thanks to RRDtool).

• Long-Term: 12 months of history out of the box.

How to Migrate

Moving to KubeLedger is straightforward. We have prepared a detailed Migration Guide to help you switch your Docker images and update your manifests.

A Note from RealOps Labs

We are cloud specialists passionate about efficiency. By bringing HPC strategies to Cloud Native environments, we aim to eliminate waste and "guesswork" in capacity planning.

Thank you for trusting us with your cluster metrics. We are excited to build the future of Kubernetes Accounting with you.

— The RealOps Labs Team

The Hidden Cost of Kubernetes

If you're running Kubernetes, you know the drill: Setting requests and limits is an art, and paying the cloud bill is a painful reality.

We've all been there—over-provisioning "just in case," or wondering why the cluster autoscaler is spinning up nodes when half the pods are idle. Efficiency is hard to track without expensive enterprise tools.

That's why I built kube-opex-analytics.

What is kube-opex-analytics?

kube-opex-analytics is an open-source tool designed to give you instant visibility into your Kubernetes cluster's resource consumption and cost allocation.

It focuses on what matters most for cost optimization:

• Actual Usage vs. Requests: See exactly how much CPU and Memory your namespaces are using compared to what they reserved.

• Cost Allocation: Break down costs per namespace based on flexible billing models.

• Trends Over Time: visualize hourly, daily, and monthly trends to spot patterns.

• GPU Support: First-class citizen support for tracking NVIDIA GPU utilization (because those resources are very expensive!).

Key Features

1. 📊 Real-time & Historical Analytics

Unlike some tools that only show "now," kube-opex-analytics consolidates metrics into hourly, daily, and monthly views. This is crucial for spotting that 3 AM spike or the weekend idle time.

2. 💰 Cost Chargeback & Showback

Want to show the Data Science team how much their experiments are costing? The tool allows you to set hourly rates or use a ratio-based cost model to generate accurate cost reports per namespace.

3. 📉 Efficiency Analysis

The "Usage Efficiency" view overlays your actual resource usage against your requested capacity. This is your "aha!" moment for right-sizing pods.

4. 🧠 GPU Metrics

With the rise of AI/ML workloads, GPU visibility is non-negotiable. We integrate with the NVIDIA DCGM exporter to show you who is actually saturating those A100s.

Quick Start 🚀

You can have it running in your cluster in less than 2 minutes.

Using Kustomize:

kubectl create namespace kube-opex-analytics
kubectl apply -k https://github.com/rchakode/kube-opex-analytics/manifests/kustomize -n kube-opex-analytics

Using Helm:

# Clone the repos
git clone https://github.com/rchakode/kube-opex-analytics.git --depth=1
cd kube-opex-analytics

# Create namespace
kubectl create namespace kube-opex-analytics

# Install with Helm
helm upgrade --install kube-opex-analytics ./manifests/helm -n kube-opex-analytics

# Watch pod status
kubectl get pods -n kube-opex-analytics -w

Once deployed, just port-forward to see the dashboard:

kubectl port-forward svc/kube-opex-analytics 5483:80 -n kube-opex-analytics

Open http://localhost:5483 and start optimizing!

Why Open Source?

Cloud cost management shouldn't be a luxury. By making this tool open source, I hope to help DevOps engineers and platform teams get the visibility they need without the enterprise price tag.

We are actively looking for contributors! Whether it's a new dashboard feature, a helm chart tweak, or just a bug report—your input is welcome.

• GitHub: rchakode/kube-opex-analytics

• License: Apache 2.0

Give it a spin and let me know what you think in the comments! 👇

Sidenote: Readers may also be interested in this related story

Introduction

A couple of months ago Kubernetes Opex Analytics has been introduced as an original open source resource usage analytics tool to make cost sharing and capacity planning decisions easy on Kubernetes clusters. Released under the terms of Apache 2.0 License the tool comes with built-in analytics charts covering various use cases. It received a lot of user feedback, some requesting the ability to expose its metrics to existing Prometheus environments.

A Prometheus exporter has been introduced in version 0.3.0, so in this story I’ll be pleased to introduce and describe it. This comes along with an integrated Grafana dashboard to let users easily getting started. All that comes in complement of the native Kubernetes Opex Analytics features and dashboard.

What will be covered next

The story will give details on the exposed metrics and provide steps to setup the Exporter and the provided Grafana dashboard. Before that we’ll review the core features of Kubernetes Opex Analytics for readers not familiar with it.

If you’re already familiar with the concepts of Kubernetes Opex Analytics, you can skip the next section and move forward.

Kubernetes Opex Analytics in a Nutshell

Kubernetes Opex Analytics is designed on the following core concepts:

• Namespace-focused: Means that consolidated resource usage metrics consider individual namespaces as fundamental units for resource sharing. A special care is taken to also account and highlight non-allocatable resources .

• Hourly Usage & Trends: Like on public clouds, resource use for each namespace is consolidated on a hourly-basic. This actually corresponds to the ratio (%) of resource used per namespace during each hour. It’s the foundation for cost calculation and also allows to get over time trends about resources being consuming per namespace and also at the Kubernetes cluster scale.

• Daily and Monthly Usage Costs: Provide for each period (daily/monthly), namespace, and resource type (CPU/memory), consolidated cost computed given one of the following ways: (i) accumulated hourly usage over the period; (ii) actual costs computed based on resource usage and a given hourly billing rate; (iii) normalized ratio of usage per namespace compared against the global cluster usage.

• Efficient Visualization: For metrics it generates, Kubernetes Opex Analytics provides dashboards with relevant charts covering as well the last couple of hours than the last 12 months (i.e. year) as shown below.

Kubernetes Opex Analytics — Screenshot of the Built-in Dashboard

Installing Kubernetes Opex Analytics

Next on we assume that the installation will be done on a Kubernetes cluster, as you can also install it on Docker as described here.

There is a Helm chart to ease the deployment on Kubernetes, either by using Helm Tiller or kubectl.

In both cases check the values.yaml file to modify the configuration options according to your needs (e.g. to have persistent volume for data storage).

Using Helm Tiller:

helm upgrade \
  --install kube-opex-analytics \
  helm/kube-opex-analytics/

Using kubectl:

helm template \
  --name kube-opex-analytics \
  helm/kube-opex-analytics/ | kubectl apply -f -

This will enable the built-in dashboard via a HTTP service named kube-opex-analytics on port 80.

Prometheus Exporter

Metrics are exposed for Prometheus via the /metrics endpoint.

Exposed Metrics

As shown on the sample on Figure below the exposed metrics are:

• koa_namespace_hourly_usage : exposes for each namespace its current hourly resource usage for both CPU and memory.

• koa_namespace_daily_usage : exposes for each namespace and for the ongoing day, its current resource usage for both CPU and memory.

• koa_namespace_monthly_usage : exposes for each namespace and for the ongoing month, its current resource usage for both CPU and memory.

Press enter or click to view image in full size

Kubernetes Opex Analytics — Sample of Metrics Exposed for Prometheus

Prometheus Scrape Job

The job can be configured like below. An interval less than 5 minutes (i.e. 300s) is useless as no new metrics will be generated within the meantime.

scrape_configs:
  - job_name: 'kube-opex-analytics'
    scrape_interval: 300s
    static_configs:
      - targets: ['kube-opex-analytics:80']

Recall that Kubernetes Opex Analytics works with hourly-consolidated metrics, so you may need to wait at least an hour to have all metrics available.

Grafana Dashboard

Once metrics available in Prometheus, get this Grafana Dashboard and import it into Grafana. The dashboard relies on a variable KOA_DS_PROMETHEUS that shall point to your Prometheus data source.

Once configured properly the dashboard shall just work out of the box to display charts as described hereafter.

Hourly Usage

There are two panels displaying respectively usage charts for CPU (left) and memory (right) over the selected interval (7 days by default). Series for the different namespaces are stacked. This makes usage comparison easy and also helps show how loaded the cluster is. On the below example we can see that during the last 5 days, global CPU and memory usage reached more than 90%.

Kubernetes Opex Analytics — Hourly Resource Usage

Current Day’s Usage

The two panels display respectively cost charts for CPU (left) and memory (right) over the ongoing day. Values are computed using cost algorithms described early in this story.

Kubernetes Opex Analytics — Current’s Day Resource Usage

Current Month’s Usage

The two panels below display respectively cost charts for CPU (left) and memory (right) over the ongoing month. Values computed using cost algorithms described early in this story.

Press enter or click to view image in full size

Kubernetes Opex Analytics — Current’s Month Resource Usage

Move forward

In short we’ve introduced in this story a Prometheus exporter as well as a Grafana dashboard for Kubernetes Opex Analytics.

As you may have noticed charts of the described Grafana dashboard are less rich than the ones of the built-in dashboard of Kubernetes Opex Analytics. For instance daily and monthly usage are limited to the current day, respectively current month. This makes difficult to compare current usage with previous ones. These are limitations inherent in how Grafana handles bar charts based on series names. The current implementation hence leaves room for further improvements, any contributions will be really appreciated.

I want to recall that Kubernetes Opex Analytics is open source, it’s an open-to-contribution project. We’re always pleased to receive feedback and contributions on Github: submit an issue if you encounter problems or have some ideas of improvement; make a pull request; or give a star.

Enjoy!

Usage accounting and cost allocation are two of mains operational problems for production Kubernetes clusters. This article is not just another discussion about the problem, but a pragmatic contribution that goes further and introduces Kubernetes Opex Analytics — an open source project (Apache License 2.0) that provides, short-, mid- and long-term CPU and memory resource usage analytics for cost allocation and capacity planning on Kubernetes clusters. The provided analytics are computed per namespace basis with different time aggregation perspectives that cover up to a year. Beyond namespaces, the analytics also highlight part of resource dedicated to the infrastructure (i.e. non-allocatable capacity) as illustrated by the following screenshot.

Kubernetes Opex Analytics — A Sample of Monthly Resource Usage per Namespace.

Before the Tool, a Story

A few months ago I decided to give a try to Google Kubernetes Engine (GKE). Going on my way I found that Google offers Free Tier accounts with $300 that you can spend at will for any Google Cloud Platform products you want. I found that really interesting to have sufficient money to try out GKE with no limited capacity as it can be the case elsewhere. Then I asked myself how to make these free $300 more beneficial than start and stop GKE clusters? Okay, I would have learnt GKE things at no cost, but…

But I decided to spend my free $300 in a more productive way, so I started working on Kubernetes Opex Analytics while trying out GKE. Today I’m very happy for my journey through GKE and really proud to contribute back Kubernetes Opex Analytics to the cloud native and Kubernetes communities.

How it Works

Kubernetes Opex Analytics periodically collects CPU and memory usage metrics from Kubernetes’s APIs, stores, processes and consolidates them over time to produce its target analytics.

What Kubernetes APIs

Kubernetes Opex Analytics needs read-only access to the following APIs:

• /apis/metrics.k8s.io/v1beta1

• /api/v1

Current Analytics

The following analytics are currently implemented, but they are expected to be extended in future versions:

• One-week CPU and Memory Usage Trends as consolidated hourly usage per namespace and globally for a cluster over the last 7 days.

• Two-weeks Daily CPU and Memory Usage per namespace as cumulative hourly usage for each namespace during each day of the last 14 ones.

• One-year Monthly CPU and Memory Usage per namespace as cumulative daily usage for each namespace during each month of the last 12 ones.

• Last Nodes’ Occupation by Pods providing for each node the share of resources used by active pods on the node.

Edit: May 12, 2019. Starting from version 0.2.0, Kubernetes Opex Analytics introduces other cost allocation models. Refer to the README to learn how to configure cost model.

Getting Started

This section will show you how to install and get started with Kubernetes Opex Analytics in less than 5 minutes.

Requirements

We assume that you use a proxied access to Kubernetes APIs as follows:

$ kubetcl proxy

This command will make the proxy access available at http://127.0.0.1:8001 .

Start Kubernetes Opex Analytics

Kubernetes Opex Analytics is released as a Docker image. So you can quickly start an instance as follows:

$ docker run -d \
        --net="host" \
        --name 'kube-opex-analytics' \
        -v /var/lib/kube-opex-analytics:/data \
        -e KOA_DB_LOCATION=/data/db \
        -e KOA_K8S_API_ENDPOINT=http://127.0.0.1:8001 \
        rchakode/kube-opex-analytics

This will make the web access available at http://127.0.0.1:5483/ .

This command provides:

• A local path /var/lib/kube-opex-analytics as data volume for the container. That's where Kubernetes Opex Analytics will store its internal analytics data. You can change the local path to another location, but you MUST take care to adapt the KOA_DB_LOCATION environment variable accordingly.

• An environment variable KOA_DB_LOCATION pointing to the path to use by Kubernetes Opex Analytics to store its internal data. You can remark that this directory belongs to the data volume atached to the container.

• An environment variable KOA_K8S_API_ENDPOINT setting the address of the Kubernetes API endpoint.

Watch Analytics Charts

Due to the time needed to have sufficient data to consolidate, you may need to wait almost a hour to have all charts filled. This is a normal operations of Kubernetes Opex Analytics.

When everything will be ready you should be able to watch charts as below.

Each chart provides tooltips (mouse hover action) giving details on resource usage for each individual namespace. Furthermore, you can also export each chart as PNG image and its associated dataset in JSON and CSV formats.

Last Week Hourly Resource Usage Trends

Two-weeks Daily CPU and Memory Usage

One-year Monthly CPU and Memory Usage

Last Nodes’ Occupation by Pods

Summary & next steps

I’m very happy to share this open source project with cloud-native and Kubernetes community. I hope that you’ll enjoy it. Kubernetes Opex Analytics is expected to be a fast moving project, so we welcome contributions in any forms (feedback, documentation, code…). Please submit any issues or enhancement requests here. If you like this work, consider to give it a star on Github. You can also fork it, and make a pull request if you wan to share back improvements you made on your side!

Edit — June 30, 2019: Kuberneres Opex Analytics now enables a Prometheus Exporter. Read the related story.