Investigating a Cloud Cost Spike

How to investigate

The order matters more than the individual checks — each step is chosen to narrow the search before the next one.

1. 1

   Break the increase down before touching anything

   A forty percent increase is a single number that hides which of several hundred line items moved. Acting before attribution means optimising whatever is most familiar rather than whatever changed, which is how teams spend a week saving a small fraction of the problem.

   • Compare this month against last at the level of service, account or subscription, region, and tag — not on the total.
   • Sort by absolute increase rather than by size, because the line that grew most is the question, and a large stable line is not.
   • Check whether the change is a step on a specific date or a gradual ramp. A step points at a deploy or a configuration change; a ramp points at accumulation or organic growth.
   • Establish whether the month had the same number of billing days and whether any pricing or discount term changed.

   What this narrows: This normally identifies one or two services and one account carrying most of the increase, which reduces the rest of the investigation to a specific question. A step change also gives you a date to correlate against.

2. 2

   Correlate the date with what changed

   Most step increases have a direct cause in the change log, and finding it is far quicker than reasoning about the architecture.

   • Check deployments, infrastructure changes and configuration changes around the step date.
   • Look for instance type or size changes, replica count changes, and autoscaling parameter changes.
   • Check whether a new environment, cluster or service was created and never removed.
   • Check whether a committed-use discount or reserved capacity term expired, which raises the bill with no change on your side at all.

   What this narrows: A change on the step date is very likely the cause. Nothing on the step date moves attention toward accumulation, toward a workload that grew on its own, or toward a pricing term.

3. 3

   Ask whether the increase is actually a problem

   Cost rising with business is a success, not an incident, and treating it as one wastes effort and credibility. Getting this straight early also tells you what a good outcome looks like.

   • Compute cost per unit of business value — per request, per active user, per transaction — for both months.
   • If unit cost is flat and volume rose, the platform is behaving correctly and the conversation is about margin rather than about waste.
   • If unit cost rose, something became less efficient, and that is the thing to investigate.

   What this narrows: Separates growth from regression. It also reframes the discussion with finance from a bill increase to an efficiency measure, which is the more useful conversation.

4. 4

   Check the categories that surprise people

   Compute is what teams look at first because it is visible. The increases that are unexplained after that check are usually in the categories nobody has a dashboard for.

   • Data transfer: traffic between regions or availability zones, egress to the internet, and traffic crossing a link to on-premises. All were effectively free in a data centre and are billed here.
   • Storage accumulation: snapshots and backups with no expiry, old object versions, and logs retained indefinitely.
   • Non-production environments running around the clock for a team that works eight hours a day.
   • Managed service tiers: a database or cache resized during an incident and never resized back.
   • Observability volume: log ingestion and metric cardinality, which grow with a single verbose deployment and are charged by volume.

   What this narrows: Each category has a distinct signature. Data transfer growing without compute growing suggests a traffic pattern change such as a chatty new call path or a cache that stopped working, which is a performance finding as well as a cost one.

5. 5

   Look for something running that should not be

   A meaningful share of sudden increases are a mistake rather than a design decision, and these are the cheapest to fix once identified.

   • Look for compute with low utilisation but high cost — provisioned and idle.
   • Check for a job that is retrying in a loop, or a queue consumer scaled up to drain a backlog and never scaled back.
   • Check autoscaling minimums, which are sometimes raised during an incident and left.
   • Check for resources with no owner tag, which correlate strongly with being forgotten.
   • Check for an unintended recursive pattern — a function triggered by writes that itself writes to the same place is the classic runaway.

   What this narrows: Finds the outright waste, which is usually a small number of items carrying a disproportionate share of the increase.

6. 6

   Fix in order of size, then make it visible

   The investigation is only half the answer. Without attribution and alerting, the same conversation recurs next quarter with a different cause.

   • Take the largest, most reversible actions first: remove what should not be running, resize what is oversized, apply lifecycle rules to storage and snapshots, and schedule non-production environments to stop outside working hours.
   • Only then consider committed-use discounts, which are a genuine large saving and lock in a shape — so commit after utilisation has settled, not during an investigation.
   • Enforce tagging so every resource has an owning team and environment, since an untagged estate cannot be attributed.
   • Set budget alerts per team and per service at a threshold that fires in days rather than at invoice time.
   • Report cost per team back to the teams, which changes behaviour more reliably than any central optimisation effort.

   What this narrows: Converts a recurring surprise into a monitored metric with an owner, which is the actual goal.

Likely causes, and the evidence for each

Naming a cause is worth little without saying what would confirm it. Each of these is paired with the observation that rules it in or out.

Drilldowns to expect