Horizontal autoscaling is a feedback controller. It observes delayed measurements, computes a desired replica count, and waits for new capacity to schedule, start, and become ready. That delay must fit the workload’s overload strategy.
Quick answer
HPA changes replica count from resource, custom, or external metrics under configured bounds and behavior. CPU utilization commonly depends on CPU requests, so missing or unrealistic requests distort the signal. Autoscaling is reactive and lagged; it is not infinite capacity or an overload-control substitute. Admission, queues, deadlines, and downstream budgets must protect the system while new replicas become useful.
Learning objectives
- Connect metric semantics, target values, requests, startup delay, stabilization, and replica bounds to a user-facing capacity objective.
- Prevent HPA from scaling an application beyond database, broker, or remote-service limits.
Prerequisites
Complete Scheduling, Disruption, and High Availability, Load Shedding, and the course. The full cluster is at Cloud-Native Backend Operations.
Choose a signal with an owner
CPU may work for CPU-correlated stateless work, but it can miss connection wait, queue delay, downstream throttling, or low-CPU blocking. Queue depth without arrival rate and processing age can cause a controller to chase expired work. Custom metrics need bounded labels, reliable collection, and a clear relationship to useful capacity.
Set minimum replicas for expected availability and warm capacity. Set maximum replicas from cluster and dependency budgets. Configure scale-up and scale-down behavior so startup, readiness, cache warming, and traffic redistribution are observable before the next decision. A conservative scale-down window can prevent oscillation after a short drop.
Production failure scenario
Checkout latency rises because the database is saturated. Pod CPU falls while threads wait for connections, so an HPA targeting CPU scales down. The remaining Pods own fewer total worker slots but the database is still saturated, increasing queue age and timeouts.
Stop the harmful scale-down, bound admitted work, inspect pool and database evidence, and choose a signal related to the constrained outcome. Adding replicas might worsen the incident if each adds connections.
Common misconceptions
- HPA cannot provide capacity before metrics, scheduling, and startup complete.
- CPU utilization without accurate requests is not a trustworthy scaling contract.
- Queue depth alone does not distinguish useful backlog from expired work.
- More replicas can increase pressure on a shared dependency.
- Scale-to-zero and event-driven activation are not assumed by the stable HPA lesson.
Decision checklist
- Define the user objective and bottleneck the scaling signal represents.
- Measure observation, decision, scheduling, startup, readiness, and traffic-shift delay.
- Derive maximum replicas from cluster, connection, quota, and downstream budgets.
- Protect the lag window with bounded queues, admission, deadlines, and shedding.
- Configure stabilization and rate policies against measured workload behavior.
- Verify useful completions and user SLIs, not replica count alone.
Continue with Kubernetes Rolling Updates and Safe Backend Releases.
Related reading
- Kubernetes Scheduling, Disruption, and High Availability
- Kubernetes Resource Requests, Limits, and JVM Containers
- Backpressure and Overload Protection
Official sources
- Kubernetes Horizontal Pod Autoscaling, accessed August 18, 2026.
- Kubernetes autoscaling workloads, accessed August 18, 2026.
- Kubernetes resource metrics pipeline, accessed August 18, 2026.