Serverless cold starts and the real cost of “pay per use”

“Pay per use” sounds ideal until the function spends a noticeable fraction of its time starting up instead of running useful code. Cold starts are not free: they consume billed duration, they affect user-visible latency, and at high concurrency they can force you into provisioned capacity that changes the cost model entirely. I have seen teams optimise memory and code for months only to discover that cold-start frequency was the dominant driver of both latency and spend.

The real cost of serverless is therefore not just the per-ms price. It is the combination of execution time, cold-start overhead, concurrency behaviour and the occasional need to pay for idle capacity to keep starts warm.

One service I reviewed had a p95 latency of 1.8 s while the actual handler work was under 80 ms. Almost the entire difference was cold starts during traffic valleys followed by bursts. After we measured the cold fraction and moved the hottest path to provisioned concurrency, p95 dropped below 200 ms and the monthly bill fell because we stopped paying for thousands of long init periods. The code itself barely changed.

What actually constitutes a cold start

A cold start occurs when the platform has to create a new execution environment: pull the code package, initialise the runtime, run global initialisation code, and only then invoke the handler. The duration varies by runtime, package size, memory setting and the amount of work done outside the handler.

Warm starts reuse an existing environment. They are dramatically faster and usually cheaper because the billed duration starts closer to the actual handler work.

The platform decides when to keep environments warm and when to reclaim them. That decision is opaque and changes with load, time of day and the provider’s current heuristics. Treating cold starts as “rare” is only safe if you have measured them under realistic traffic.

Simple ways to measure cold-start impact

I use three complementary signals:

1. Platform metrics (if available): cold-start count, init duration, concurrent executions.
2. Application-level timing: a high-resolution timestamp at the very beginning of the handler and another after any heavy initialisation. The difference approximates the cold-start penalty when the environment is new.
3. Synthetic probes: a low-frequency invoker that calls the function from a cold state (after a long idle period) and from a warm state, recording latency percentiles.

A minimal measurement snippet inside a Node.js or Python handler looks like this:

// Node.js example – place at the absolute top of the file
const coldStartBegin = process.hrtime.bigint();
let isCold = true;

exports.handler = async (event) => {
  const handlerBegin = process.hrtime.bigint();
  if (isCold) {
    const initMs = Number(handlerBegin - coldStartBegin) / 1e6;
    console.log(JSON.stringify({ type: "cold_start", initMs }));
    isCold = false;
  }
  // … real work …
};

# Python equivalent
import time
_cold_start_begin = time.perf_counter()
_is_cold = True

def handler(event, context):
    global _is_cold
    handler_begin = time.perf_counter()
    if _is_cold:
        init_ms = (handler_begin - _cold_start_begin) * 1000
        print(json.dumps({"type": "cold_start", "initMs": init_ms}))
        _is_cold = False
    # … real work …

After a few days of production traffic the logs give a clear distribution of init times and the fraction of invocations that paid the cold-start tax. That fraction is the number I care about most.

Memory, package size and init cost

Higher memory settings usually reduce cold-start duration (more CPU is allocated proportionally on most platforms) but increase the per-ms price. There is a sweet spot that minimises total cost for a given workload; it is rarely the lowest memory tier.

Large deployment packages and heavy global imports push init time up. I treat package size as a first-class performance and cost metric: keep dependencies lean, avoid importing large libraries at module level if they are only needed on certain paths, and prefer lazy loading where the language allows it.

A practical experiment I run:

# Deploy the same function at 128 MB, 512 MB, 1024 MB, 2048 MB
# Invoke from cold (after 15–30 min idle) and from warm, 50 times each
# Record p50 / p95 latency and estimated cost per 1 M invocations

The results almost always show that the cheapest overall configuration is not the one with the lowest memory price. On one Java function the 1024 MB setting produced the lowest total cost because the reduction in cold-start duration more than offset the higher per-ms rate. On a lean Python function the opposite was true. The only reliable way to know is to measure both latency and estimated cost under the real package and runtime.

Concurrency and the point where provisioned capacity wins

On pure on-demand billing you pay only for the duration of each invocation. When concurrency spikes, the platform creates many new environments at once; each of them incurs a cold start. The latency tail lengthens and the billed duration includes all those init times.

Provisioned concurrency (or the equivalent reserved / min-instances feature on other platforms) keeps a configured number of environments warm. You pay for those environments even when they are idle, but you eliminate cold starts for the traffic that stays within the provisioned limit. The break-even is straightforward:

• Calculate the monthly cost of the cold-start duration under observed traffic.
• Calculate the monthly cost of the provisioned capacity needed to absorb the 95th-percentile concurrency.
• Choose the lower number, then re-measure after a week because traffic patterns change.

I keep a small spreadsheet with those two columns for every latency-sensitive function. When the cold-start tax exceeds the provisioned cost for more than two consecutive weeks, I switch.

When serverless stops being the cheaper option

There are clear regimes where a small always-on instance (or a container with a min-replica of 1) becomes both cheaper and more predictable:

• Steady traffic with low variance: the idle capacity of a tiny VM is cheaper than repeated cold starts plus provisioned concurrency.
• Very large packages or long init times that cannot be reduced further.
• Workloads that need persistent connections, local caches or in-memory state that is expensive to rebuild on every cold start.
• Functions that are invoked so frequently that they are effectively warm all the time yet still pay the per-invocation overhead.

In those cases I move the hot path to a small reserved instance or a container service with a minimum of one replica and leave the spiky or infrequent paths on pure serverless. The hybrid is usually the cheapest overall design.

Operational habits that keep the bill honest

• Log cold-start events with their duration; alert if the cold fraction exceeds a threshold that matters for the service’s latency SLO.
• Review the top functions by cold-start contribution every two weeks for the first two months of a new service.
• Treat package size and global initialisation work as performance regressions that block merges.
• Revisit the provisioned-concurrency decision whenever traffic volume or pattern changes significantly.
• Keep a canary or shadow invocation path that deliberately forces a cold start once per hour so the init duration stays visible even when production traffic keeps environments warm.

These habits turn cold starts from an invisible tax into a metric the team can act on. Once the metric exists, the cost conversation becomes concrete: we can calculate the monthly init tax, compare it with provisioned capacity, and decide with numbers instead of anecdotes.

Serverless is still an excellent default for spiky, event-driven and low-duty-cycle work. The “pay per use” slogan remains true only while cold starts are rare or cheap. Once they become frequent, the real cost includes the init tax and, eventually, the decision to pay for warmth. Measuring that tax early turns it from a surprise into a controllable design parameter.

When the numbers show that cold starts dominate, I plug the observed invocation rate, average duration and cold fraction into the Serverless calculator and compare pure on-demand against provisioned and against a small always-on alternative. The calculator forces the comparison to be numerical; the measurement data makes the comparison honest.

I also keep a short decision record for every latency-sensitive function: measured cold fraction, chosen memory, whether provisioned concurrency is enabled and at what level, and the date of the last review. When traffic doubles or the package grows, the record makes the re-evaluation mechanical instead of political.

Serverless remains the right default for many workloads. The moment cold starts become a visible fraction of either latency or cost, the design has to acknowledge them. Measuring early, logging the init tax, and comparing the three options (on-demand, provisioned, always-on) keeps the “pay per use” promise from turning into an unexpected fixed cost.