Horizontal Pod Autoscaler: why it scales late, or not at all

An HPA that reports <unknown>/70% is not scaling and never will. An HPA that reports a number but adds pods five minutes after the traffic arrived is working exactly as configured. These are different problems and they get treated as one.

The dependency that breaks it silently

The HPA’s CPU target is a percentage of the pod’s CPU request. No request, no denominator, no metric.

kubectl get hpa
NAME   REFERENCE        TARGETS         MINPODS   MAXPODS   REPLICAS
api    Deployment/api   <unknown>/70%   2         10        2

<unknown> almost always means one of two things: metrics-server is not running, or a container in the pod has no resources.requests.cpu. Check the second first, because it is more common and the error message never says so.

kubectl get deploy api -o jsonpath=\
'{.spec.template.spec.containers[*].resources.requests.cpu}{"\n"}'

An empty result, or a value for the app container but nothing for a sidecar, is your answer. Every container in the pod needs a request, including the log shipper and the service mesh proxy nobody thinks about.

Then confirm the metrics pipeline:

kubectl top pods           # should print numbers, not an error
kubectl get apiservice v1beta1.metrics.k8s.io

We covered why requests and limits behave differently elsewhere; this is one more reason requests are not optional.

The formula, which explains most surprises

desiredReplicas = ceil( currentReplicas × ( currentMetric / targetMetric ) )

Four consequences fall out of it.

It is a ratio, not a step. At 4 replicas averaging 140% of a 70% target, the HPA jumps straight to 8. It does not add one and reassess.

It rounds up. Any excess over target adds at least one replica.

A 10% tolerance suppresses small moves. Ratios between 0.9 and 1.1 produce no change, which stops the replica count oscillating around the target.

The average includes pods that are still starting. A pod that is Ready but still warming a cache reports low CPU and drags the average down, which delays the next scale-up. Slow-starting applications should carry a readinessProbe that stays false until they are genuinely ready, not merely alive.

Scale-up is instant, scale-down is deliberately not

The asymmetry is the default and it confuses people who watch a load test finish.

• Metrics sync interval15show often it reconsiders
• Scale-up stabilisation window0sreacts immediately
• Scale-down stabilisation window300sholds the peak

Scale-down waits five minutes and uses the highest recommendation seen in that window, so repeated spikes keep the replica count at the high-water mark on purpose.

Scaling up happens on the next sync, roughly every 15 seconds. Scaling down waits out a 300-second stabilisation window, during which the HPA uses the highest recommendation it has seen. Traffic that spikes and drops repeatedly keeps the replica count at the high-water mark, on purpose.

Both are configurable:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 2
  maxReplicas: 20
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100          # at most double
          periodSeconds: 30
        - type: Pods
          value: 4            # or +4 pods
          periodSeconds: 30
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 600
      policies:
        - type: Percent
          value: 25           # shed at most a quarter per minute
          periodSeconds: 60
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Shortening the scale-down window because a load test looked slow is the wrong lever. Lengthen it if your traffic is spiky and pod startup is expensive; the cost of a few extra pods is almost always lower than the cost of thrashing.

CPU is the wrong signal more often than people expect

CPU utilisation is a proxy for load. It is a good proxy for CPU-bound work and a poor one for everything else.

A worker pulling from a queue may sit at 20% CPU with a backlog of fifty thousand messages, blocked on I/O. CPU says scale down. Reality says scale up hard. The same applies to a service whose latency comes from a slow downstream, or one whose concurrency limit is connections rather than cycles.

Scale on the thing that actually represents pressure:

  metrics:
    - type: External
      external:
        metric:
          name: rabbitmq_queue_messages_ready
          selector:
            matchLabels:
              queue: orders
        target:
          type: AverageValue
          averageValue: "30"       # ~30 messages per replica

External and Pods metrics need an adapter — Prometheus Adapter, or KEDA if you want queue-length scaling with less wiring. KEDA also does scale-to-zero, which the standard HPA will not do.

Where a request-rate signal is available, it usually beats both:

    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

When the HPA is not the problem

Three failure modes look like autoscaling and are not.

Pods are Pending, not missing. The HPA scaled; the scheduler could not place the pods. kubectl describe pod shows Insufficient cpu. You need Cluster Autoscaler or more nodes; adding replicas to a full cluster achieves nothing.

A ResourceQuota is in the way. The namespace hit its CPU or pod ceiling and creation is rejected. The HPA reports the desired count and the ReplicaSet reports the failure — look at ReplicaSet events, not HPA events.

Something else is also writing replicas. An HPA and a GitOps controller that both manage the same Deployment will fight, with Argo CD reverting the count every sync. Add spec.replicas to the ignore-differences list and let the HPA own it.

A default worth starting from

For a stateless HTTP service with no better signal available:

• minReplicas: 2 — one replica has no headroom and no availability
• maxReplicas at roughly 4× your steady state, so a bad day cannot exhaust the cluster
• CPU target of 60–70%, leaving room for the ratio to react before saturation
• Default scale-down window, until you have evidence it is wrong

Then measure. An autoscaler tuned against a load test that does not resemble production traffic is tuned against fiction.