Overview
Autoscaling is one of Kubernetes’ most attractive capabilities—scale up when traffic arrives, scale down when it leaves, ensuring service quality while controlling costs. But “automatic” doesn’t mean “mindless.” Poorly configured autoscaling can cause: delayed scaling leading to service degradation, aggressive scale-down disrupting long connections, and flapping scaling wasting resources.
K8s autoscaling consists of three layers plus an event-driven extension:
| Layer | Component | Scaling Dimension | Trigger |
|---|---|---|---|
| Pod horizontal | HPA | Pod replica count | CPU/memory/custom metrics |
| Pod vertical | VPA | Pod resource quotas | CPU/memory historical usage |
| Node | Cluster Autoscaler | Node count | Pending Pods |
| Event-driven | KEDA | Pod replica count | Event sources (Kafka/Redis/…) |
Based on Kubernetes v1.30. Reference: Kubernetes Autoscaling Documentation
HPA: Horizontal Pod Autoscaler
How It Works
HPA is a control loop that executes every 15 seconds by default:
1. Get current metric values from the metrics API (CPU/memory/custom)
2. Calculate desired replicas = ceil(current replicas * (current metric / target metric))
3. Compare with current replicas, decide to scale up or down
4. Call Deployment/ReplicaSet Scale API to change replica count
Core formula:
Desired replicas = ceil(current replicas × (current metric value ÷ target metric value))
For example: currently 4 replicas, CPU utilization 80%, target 50%, then desired replicas = ceil(4 × 80/50) = ceil(6.4) = 7.
CPU/Memory-Based HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 3 # Minimum replicas
maxReplicas: 50 # Maximum replicas
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization # Utilization percentage
averageUtilization: 60 # Target CPU utilization 60%
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 70 # Target memory utilization 70%
Prerequisite: Pods must have
resources.requests.cpuandresources.requests.memoryconfigured, otherwise HPA cannot calculate utilization. This is the most common reason HPA doesn’t work.
Custom Metrics-Based HPA
CPU/memory are generic metrics, but often you need to scale based on application-specific metrics: QPS, message queue depth, active connection count.
Deploy Metrics Pipeline
Custom metrics require a complete metrics pipeline:
Pod metrics → cAdvisor → kubelet → Metrics Server (resource metrics)
↓
App metrics → Prometheus → Prometheus Adapter → K8s API (custom metrics)
↓
HPA queries
Install Prometheus Adapter:
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus-adapter prometheus-community/prometheus-adapter \
--namespace monitoring \
--create-namespace \
--set prometheus.url=http://prometheus-server.monitoring.svc.cluster.local \
--set prometheus.port=80
Configure custom metric rules:
# Prometheus Adapter configuration
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-adapter
namespace: monitoring
data:
config.yaml: |
rules:
- seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
resources:
overrides:
namespace: {resource: "namespace"}
pod: {resource: "pod"}
name:
matches: "^(.*)_total"
as: "${1}_per_second"
metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)'
HPA using custom metrics:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa-custom
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 3
maxReplicas: 50
metrics:
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "1000" # Target 1000 QPS per Pod
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
Multi-Metric Combination
HPA can configure multiple metrics simultaneously, taking the maximum desired replica count from each:
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 70
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "1000"
# HPA calculates desired replicas for each metric, takes the maximum
Scaling Behavior Control
K8s v1.23+ supports fine-grained control of scaling behavior via the behavior field:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 3
maxReplicas: 50
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
behavior:
# Scale-up policy
scaleUp:
stabilizationWindowSeconds: 0 # No stabilization window for scale-up, execute immediately
policies:
- type: Percent
value: 100 # Max 100% scale-up per step (double)
periodSeconds: 30 # At most one scale-up per 30 seconds
- type: Pods
value: 4 # Or max 4 Pods per step
periodSeconds: 30
selectPolicy: Max # Take the maximum of both policies
# Scale-down policy
scaleDown:
stabilizationWindowSeconds: 300 # 5-minute stabilization window for scale-down
policies:
- type: Percent
value: 10 # Max 10% scale-down per step
periodSeconds: 60 # At most one scale-down per 60 seconds
Behavior Control Strategy Reference
| Strategy | Description | Recommendation |
|---|---|---|
stabilizationWindowSeconds | Stabilization window, metrics must persist before acting | 300s for scale-down, 0s for scale-up |
type: Percent | Scale by percentage | General use |
type: Pods | Scale by absolute count | Precise control |
selectPolicy: Max | Take max of multiple policies | Scale-up priority |
selectPolicy: Min | Take min of multiple policies | Conservative scale-down |
selectPolicy: Disabled | Disable that direction | Scale-up only |
HPA Production Practices
Common issues and solutions:
| Issue | Cause | Solution |
|---|---|---|
| HPA not working | Pod missing resources.requests | Add requests config |
| Metric fetch failure | Metrics Server not deployed | Deploy metrics-server |
| Custom metrics not working | Prometheus Adapter misconfigured | Check rules config |
| Scale-up too slow | stabilizationWindow too long | Set scale-up window to 0 |
| Scale-down flapping | Scale-down window too short | Set to 300s+ |
| Cold start anomaly | New Pod metrics not yet collected | Not fully solvable, use warmup |
Cold start problem: When HPA scales up new Pods, they need time to start and report metrics. During this period, HPA can’t get the new Pod’s metrics and may continue scaling, leading to over-scaling. Solutions:
- Configure
startupProbeso Pods only participate in load when fully ready - Set reasonable
scaleUppolicies to limit scaling speed - Use KEDA’s warmup mechanism
VPA: Vertical Pod Autoscaler
How It Works
VPA (Vertical Pod Autoscaler) analyzes Pod historical resource usage and automatically adjusts resources.requests and resources.limits. Unlike HPA, VPA changes individual Pod resource quotas, not replica count.
Three Modes
| Mode | Behavior | Use Case |
|---|---|---|
Off | Only recommend, don’t apply | Evaluation phase |
Initial | Apply recommendation at Pod creation, don’t modify running Pods | Production recommended |
Auto | Auto-apply recommendations, requires Pod restart | Use with caution |
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: myapp-vpa
namespace: production
spec:
targetRef:
apiVersion: "apps/v1"
kind: Deployment
name: myapp
updatePolicy:
updateMode: "Initial" # Only apply recommendation at Pod creation
resourcePolicy:
containerPolicies:
- containerName: '*'
minAllowed:
cpu: 100m
memory: 128Mi
maxAllowed:
cpu: 4
memory: 8Gi
controlledResources: ["cpu", "memory"]
VPA Limitations
VPA’s biggest issue: In
Automode, modifying resource quotas requires restarting Pods because K8s doesn’t support modifying Pod resource requests at runtime. This means VPA’s Auto mode causes service disruption.
| Limitation | Description |
|---|---|
| Requires Pod restart | Resource changes require Pod recreation |
| Cannot coexist with HPA on same dimension | HPA scales replicas by CPU, VPA changes CPU quota—conflicts |
| Minimum resource limits | Need minAllowed/maxAllowed to prevent anomalous values |
| Admission webhook | Requires VPA Admission Controller, adds complexity |
Viewing VPA Recommendations
# View VPA recommendations
kubectl describe vpa myapp-vpa -n production
# Example output
# Recommendation:
# Target:
# CPU: 250m
# Memory: 500Mi
# Lower Bound:
# CPU: 100m
# Memory: 200Mi
# Upper Bound:
# CPU: 500m
# Memory: 1Gi
# Uncapped Target:
# CPU: 250m
# Memory: 500Mi
VPA and HPA Cooperation
VPA and HPA can cooperate, but cannot manage the same resource dimension simultaneously:
| Approach | HPA | VPA | Notes |
|---|---|---|---|
| Option 1 | CPU scaling | Memory adjustment | HPA manages CPU, VPA manages memory |
| Option 2 | Custom metric scaling | CPU+memory adjustment | HPA manages business metrics, VPA manages resources |
| Option 3 | CPU scaling | Off mode | VPA only recommends, manual adjustment |
VPA Production Recommendations
- Use Initial mode in production: Only applies recommendations at new Pod creation, doesn’t affect running Pods
- Evaluate with Off mode first: Run for a while to observe if recommendations are reasonable
- Set minAllowed/maxAllowed: Prevent anomalous recommendations
- Don’t manage CPU with both HPA and VPA: This is the most common configuration conflict
Cluster Autoscaler
How It Works
Cluster Autoscaler (CA) focuses on Pending Pods. When HPA scaling causes cluster resource shortage and Pods are in Pending state, CA automatically adds nodes:
HPA scales up → Pod Pending (insufficient resources) → CA adds node → Pod scheduled
Traffic drops → HPA scales down → Node utilization low → CA removes node
CA decision logic:
- Periodically scan for Pending Pods
- If Pending is due to resource shortage, simulate scheduling to calculate needed nodes
- Call cloud provider API to create new nodes
- After new nodes join cluster, Pending Pods are scheduled
Scale-Down Logic
CA is more cautious about scaling down:
- Find low-utilization nodes (total Pod CPU/memory requests < threshold)
- Simulate migrating that node’s Pods to other nodes
- If migration is possible, drain the node and delete it
# Cluster Autoscaler configuration example (AWS EKS)
apiVersion: apps/v1
kind: Deployment
metadata:
name: cluster-autoscaler
namespace: kube-system
spec:
template:
spec:
containers:
- image: registry.k8s.io/autoscaling/cluster-autoscaler:v1.30.0
name: cluster-autoscaler
command:
- ./cluster-autoscaler
- --scale-down-unneeded-time=10m # Scale down after 10min low utilization
- --scale-down-delay-after-add=10m # No scale-down within 10min after scale-up
- --scale-down-unempty-time=30m # Wait 30min after drain before deleting
- --max-node-provision-time=15m # Max wait for node creation
- --balance-similar-node-groups=true # Balance similar node groups
- --expander=least-waste # Expansion strategy: least waste
- --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
env:
- name: AWS_REGION
value: us-east-1
Expansion Strategies (Expander)
| Strategy | Description | Use Case |
|---|---|---|
random | Random selection (default) | Simple scenarios |
most-pods | Select node group that schedules most Pods | Prioritize Pod scheduling |
least-waste | Select node group with least resource waste | Cost optimization |
price | Select cheapest node group | Cloud cost optimization |
priority | Select by priority | Mixed node groups |
Cloud Provider CA Implementations
| Cloud | Implementation | Characteristics |
|---|---|---|
| AWS EKS | Auto Scaling Group | Mature and stable |
| GCP GKE | Native integration | Out of the box |
| Azure AKS | VM Scale Set | Mature and stable |
| Self-hosted | Cluster API | Need to manage own infrastructure |
CA Limitations
| Limitation | Description |
|---|---|
| Not real-time scaling | Node creation takes 1-5 minutes |
| No cross-AZ scheduling | Node groups are AZ-bound |
| Pod eviction limited by PDB | PDB may prevent scale-down |
| Spot instance reclamation | Need Node Termination Handler |
| No node type selection | Only scales within node group |
KEDA: Event-Driven Autoscaling
Why KEDA
HPA scales based on CPU/memory/custom metrics, but many scenarios’ scaling signals are not resource metrics but events:
- Kafka queue backlog → scale up consumers
- Redis queue length increasing → scale up workers
- Cron schedule → scheduled scaling
- PostgreSQL high connection count → scale up
KEDA (Kubernetes Event-Driven Autoscaling) is a CNCF project specifically for event-driven autoscaling.
KEDA Architecture
Event source (Kafka/Redis/...) → KEDA Scaler → KEDA Operator → HPA → Deployment
↓
ScaledObject (CRD)
ScaledJob (CRD)
KEDA’s workflow:
- Deploy
ScaledObjectCRD, defining event source and scaling rules - KEDA Operator watches ScaledObject, creates corresponding HPA
- KEDA External Scaler gets metrics from event source
- HPA scales based on metrics
Install KEDA
helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda \
--namespace keda-system \
--create-namespace
Kafka Consumer Scaling Example
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: kafka-consumer-scaledobject
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: kafka-consumer
minReplicaCount: 0 # Scale to 0 when no messages
maxReplicaCount: 50 # Max 50 consumers
pollingInterval: 30 # Check every 30 seconds
cooldownPeriod: 300 # Wait 300s before scaling to 0
triggers:
- type: kafka
metadata:
bootstrapServers: kafka-broker:9092
consumerGroup: my-consumer-group
topic: orders
lagThreshold: "100" # Scale up when lag exceeds 100 per partition
offsetResetPolicy: latest
partitionLimitation: "0,1,2,3" # Only monitor specific partitions
Supported Event Sources
KEDA supports 60+ event sources:
| Category | Event Sources |
|---|---|
| Message queues | Kafka, RabbitMQ, AWS SQS, Azure Service Bus, NATS |
| Databases | PostgreSQL, MySQL, MongoDB, Redis |
| Monitoring | Prometheus, Datadog |
| Cloud services | AWS CloudWatch, Azure Monitor, GCP Pub/Sub |
| Scheduling | Cron |
| Custom | External Scaler |
ScaledJob: Batch Processing Scaling
For batch processing tasks, KEDA provides ScaledJob, which creates Jobs directly instead of scaling Deployments:
apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
name: image-processor
namespace: production
spec:
jobTargetRef:
template:
spec:
containers:
- name: processor
image: myapp/processor:v1
command: ["./process"]
restartPolicy: Never
maxReplicaCount: 10
pollingInterval: 30
triggers:
- type: aws-sqs
metadata:
queueURL: https://sqs.us-east-1.amazonaws.com/123456789012/image-queue
queueLength: "5" # Create one Job per 5 messages
awsRegion: us-east-1
KEDA Advantages
| Advantage | Description |
|---|---|
| Scale to Zero | Scale to 0 when no events, extreme cost savings |
| Rich event sources | 60+ event sources |
| HPA compatible | Generates HPA underneath, can coexist |
| Simple to use | One ScaledObject CRD does it all |
Scaling Strategy Combinations
Recommended Production Combinations
| Scenario | HPA | VPA | CA | KEDA |
|---|---|---|---|---|
| Web API | CPU + QPS | Initial | Yes | No |
| Message consumer | No | Initial | Yes | Yes |
| Batch processing | No | Off | Yes | ScaledJob |
| Database | No | Off | No | No |
| WebSocket | Connection count | Initial | Yes | No |
Complete Autoscaling Configuration Example
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: api-server
template:
metadata:
labels:
app: api-server
spec:
containers:
- name: api
image: myapp/api:v1
resources:
requests:
cpu: 200m
memory: 256Mi
limits:
cpu: 1000m
memory: 1Gi
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
---
# HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api-server-hpa
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api-server
minReplicas: 3
maxReplicas: 30
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "500"
behavior:
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 30
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 10
periodSeconds: 60
---
# PDB (graceful eviction during HPA scale-down)
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: api-server-pdb
namespace: production
spec:
minAvailable: 2 # Keep at least 2 Pods available
selector:
matchLabels:
app: api-server
PodDisruptionBudget Cooperation
Both HPA scale-down and CA node scale-down evict Pods, requiring PDB protection:
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: api-server-pdb
spec:
minAvailable: 50% # Keep at least 50% available
# or maxUnavailable: 1 # At most 1 unavailable
selector:
matchLabels:
app: api-server
| PDB Parameter | Description | Use Case |
|---|---|---|
minAvailable: N | At least N available | Fixed replica count |
minAvailable: 50% | At least 50% available | Elastic replica count |
maxUnavailable: 1 | At most 1 unavailable | Conservative strategy |
maxUnavailable: 25% | At most 25% unavailable | Aggressive strategy |
Production Practices
Autoscaling Monitoring
# Prometheus alert rules
groups:
- name: hpa-alerts
rules:
# HPA at max replicas
- alert: HPAAtMaxReplicas
expr: kube_hpa_status_condition{condition="ScalingLimited",status="true"} == 1
for: 10m
annotations:
summary: "HPA {{ $labels.hpa }} reached scaling limit"
# HPA cannot get metrics
- alert: HPAMetricsUnavailable
expr: kube_hpa_status_condition{condition="ScalingActive",status="false"} == 1
for: 5m
annotations:
summary: "HPA {{ $labels.hpa }} cannot get metrics"
# CA cannot scale
- alert: ClusterAutoscalerUnschedulable
expr: cluster_autoscaler_unschedulable_pods_count > 0
for: 10m
annotations:
summary: "{{ $value }} Pods cannot be scheduled"
Autoscaling Testing
# Load test to trigger HPA scale-up
kubectl run load-generator --image=busybox:latest --restart=Never \
-- /bin/sh -c "while true; do wget -q -O- http://api-server.production.svc.cluster.local:8080/; done"
# Watch HPA status
kubectl get hpa -n production -w
# Watch Pod scaling process
kubectl get pods -n production -l app=api-server -w
Autoscaling Troubleshooting
# View HPA detailed status
kubectl describe hpa myapp-hpa -n production
# View HPA metrics
kubectl get --raw "/apis/autoscaling/v2/namespaces/production/horizontalpodautoscalers/myapp-hpa" | jq .
# View custom metrics
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/production/pods/*/http_requests_per_second" | jq .
# View CA logs
kubectl logs -n kube-system -l app=cluster-autoscaler --tail=50
# View node scaling events
kubectl get events --field-selector reason=TriggeredScaleUp
Common Issues
| Issue | Cause | Solution |
|---|---|---|
HPA shows unknown | Pod missing requests or metrics-server abnormal | Check requests config and metrics-server |
| HPA not scaling | Metric value below target | Check actual metrics |
| HPA at max but still insufficient | maxReplicas too small or insufficient nodes | Increase max or check CA |
| Scale-down too slow | stabilizationWindow too long | Reduce window |
| Scale-down flapping | Large traffic fluctuations | Increase window or use min replicas as floor |
| CA not scaling | Node group at limit | Increase ASG max |
| VPA not working | updateMode is Off | Change to Initial or Auto |
Summary
K8s autoscaling is a multi-layered system. Key takeaways:
- HPA is the workhorse: CPU/memory-based HPA is the most fundamental and practical autoscaling mechanism. Always configure
behaviorto control scaling speed and prevent flapping. - Custom metrics are more precise: CPU doesn’t reflect real load—QPS, queue depth, and other business metrics are closer to reality. Deploy Prometheus Adapter for custom metric HPA.
- Use VPA Auto mode with caution: VPA’s Auto mode requires Pod restarts. Use Initial or Off mode for recommendations in production.
- CA covers the node layer: HPA only manages Pod replica count. When nodes are insufficient, CA automatically adds nodes. Note that CA scale-down has latency—don’t expect real-time.
- KEDA covers event-driven: Use KEDA for message queue and batch processing scenarios, with Scale-to-Zero support for extreme cost savings.
- PDB is mandatory: Scaling without PDB can cause service unavailability, especially during CA node eviction.
- Monitoring and alerting are essential: HPA hitting limits and CA unable to scale both need alerts—otherwise you won’t know when autoscaling fails.
Autoscaling isn’t “configure and forget.” It requires continuous observation and tuning. Review autoscaling history weekly and adjust parameters based on actual traffic patterns.
References & Acknowledgments
This article referenced the following materials during writing. We thank the original authors for their contributions:
- Kubernetes Autoscaling Documentation — Kubernetes Official, referenced for Kubernetes Autoscaling Documentation