Overview

Autoscaling is one of Kubernetes’ most attractive capabilities—scale up when traffic arrives, scale down when it leaves, ensuring service quality while controlling costs. But “automatic” doesn’t mean “mindless.” Poorly configured autoscaling can cause: delayed scaling leading to service degradation, aggressive scale-down disrupting long connections, and flapping scaling wasting resources.

K8s autoscaling consists of three layers plus an event-driven extension:

LayerComponentScaling DimensionTrigger
Pod horizontalHPAPod replica countCPU/memory/custom metrics
Pod verticalVPAPod resource quotasCPU/memory historical usage
NodeCluster AutoscalerNode countPending Pods
Event-drivenKEDAPod replica countEvent sources (Kafka/Redis/…)

Based on Kubernetes v1.30. Reference: Kubernetes Autoscaling Documentation

HPA: Horizontal Pod Autoscaler

How It Works

HPA is a control loop that executes every 15 seconds by default:

1. Get current metric values from the metrics API (CPU/memory/custom)
2. Calculate desired replicas = ceil(current replicas * (current metric / target metric))
3. Compare with current replicas, decide to scale up or down
4. Call Deployment/ReplicaSet Scale API to change replica count

Core formula:

Desired replicas = ceil(current replicas × (current metric value ÷ target metric value))

For example: currently 4 replicas, CPU utilization 80%, target 50%, then desired replicas = ceil(4 × 80/50) = ceil(6.4) = 7.

CPU/Memory-Based HPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp
  minReplicas: 3                    # Minimum replicas
  maxReplicas: 50                   # Maximum replicas
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization           # Utilization percentage
        averageUtilization: 60      # Target CPU utilization 60%
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 70      # Target memory utilization 70%

Prerequisite: Pods must have resources.requests.cpu and resources.requests.memory configured, otherwise HPA cannot calculate utilization. This is the most common reason HPA doesn’t work.

Custom Metrics-Based HPA

CPU/memory are generic metrics, but often you need to scale based on application-specific metrics: QPS, message queue depth, active connection count.

Deploy Metrics Pipeline

Custom metrics require a complete metrics pipeline:

Pod metrics → cAdvisor → kubelet → Metrics Server (resource metrics)
App metrics → Prometheus → Prometheus Adapter → K8s API (custom metrics)
                                 HPA queries

Install Prometheus Adapter:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus-adapter prometheus-community/prometheus-adapter \
  --namespace monitoring \
  --create-namespace \
  --set prometheus.url=http://prometheus-server.monitoring.svc.cluster.local \
  --set prometheus.port=80

Configure custom metric rules:

# Prometheus Adapter configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-adapter
  namespace: monitoring
data:
  config.yaml: |
    rules:
    - seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
      resources:
        overrides:
          namespace: {resource: "namespace"}
          pod: {resource: "pod"}
      name:
        matches: "^(.*)_total"
        as: "${1}_per_second"
      metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)'    

HPA using custom metrics:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa-custom
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp
  minReplicas: 3
  maxReplicas: 50
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "1000"       # Target 1000 QPS per Pod
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Multi-Metric Combination

HPA can configure multiple metrics simultaneously, taking the maximum desired replica count from each:

metrics:
- type: Resource
  resource:
    name: cpu
    target:
      type: Utilization
      averageUtilization: 60
- type: Resource
  resource:
    name: memory
    target:
      type: Utilization
      averageUtilization: 70
- type: Pods
  pods:
    metric:
      name: http_requests_per_second
    target:
      type: AverageValue
      averageValue: "1000"
# HPA calculates desired replicas for each metric, takes the maximum

Scaling Behavior Control

K8s v1.23+ supports fine-grained control of scaling behavior via the behavior field:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp
  minReplicas: 3
  maxReplicas: 50
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60
  behavior:
    # Scale-up policy
    scaleUp:
      stabilizationWindowSeconds: 0    # No stabilization window for scale-up, execute immediately
      policies:
      - type: Percent
        value: 100                      # Max 100% scale-up per step (double)
        periodSeconds: 30               # At most one scale-up per 30 seconds
      - type: Pods
        value: 4                        # Or max 4 Pods per step
        periodSeconds: 30
      selectPolicy: Max                 # Take the maximum of both policies
    # Scale-down policy
    scaleDown:
      stabilizationWindowSeconds: 300   # 5-minute stabilization window for scale-down
      policies:
      - type: Percent
        value: 10                       # Max 10% scale-down per step
        periodSeconds: 60               # At most one scale-down per 60 seconds

Behavior Control Strategy Reference

StrategyDescriptionRecommendation
stabilizationWindowSecondsStabilization window, metrics must persist before acting300s for scale-down, 0s for scale-up
type: PercentScale by percentageGeneral use
type: PodsScale by absolute countPrecise control
selectPolicy: MaxTake max of multiple policiesScale-up priority
selectPolicy: MinTake min of multiple policiesConservative scale-down
selectPolicy: DisabledDisable that directionScale-up only

HPA Production Practices

Common issues and solutions:

IssueCauseSolution
HPA not workingPod missing resources.requestsAdd requests config
Metric fetch failureMetrics Server not deployedDeploy metrics-server
Custom metrics not workingPrometheus Adapter misconfiguredCheck rules config
Scale-up too slowstabilizationWindow too longSet scale-up window to 0
Scale-down flappingScale-down window too shortSet to 300s+
Cold start anomalyNew Pod metrics not yet collectedNot fully solvable, use warmup

Cold start problem: When HPA scales up new Pods, they need time to start and report metrics. During this period, HPA can’t get the new Pod’s metrics and may continue scaling, leading to over-scaling. Solutions:

  1. Configure startupProbe so Pods only participate in load when fully ready
  2. Set reasonable scaleUp policies to limit scaling speed
  3. Use KEDA’s warmup mechanism

VPA: Vertical Pod Autoscaler

How It Works

VPA (Vertical Pod Autoscaler) analyzes Pod historical resource usage and automatically adjusts resources.requests and resources.limits. Unlike HPA, VPA changes individual Pod resource quotas, not replica count.

Three Modes

ModeBehaviorUse Case
OffOnly recommend, don’t applyEvaluation phase
InitialApply recommendation at Pod creation, don’t modify running PodsProduction recommended
AutoAuto-apply recommendations, requires Pod restartUse with caution
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: myapp-vpa
  namespace: production
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: myapp
  updatePolicy:
    updateMode: "Initial"            # Only apply recommendation at Pod creation
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 100m
        memory: 128Mi
      maxAllowed:
        cpu: 4
        memory: 8Gi
      controlledResources: ["cpu", "memory"]

VPA Limitations

VPA’s biggest issue: In Auto mode, modifying resource quotas requires restarting Pods because K8s doesn’t support modifying Pod resource requests at runtime. This means VPA’s Auto mode causes service disruption.

LimitationDescription
Requires Pod restartResource changes require Pod recreation
Cannot coexist with HPA on same dimensionHPA scales replicas by CPU, VPA changes CPU quota—conflicts
Minimum resource limitsNeed minAllowed/maxAllowed to prevent anomalous values
Admission webhookRequires VPA Admission Controller, adds complexity

Viewing VPA Recommendations

# View VPA recommendations
kubectl describe vpa myapp-vpa -n production

# Example output
# Recommendation:
#   Target:
#     CPU: 250m
#     Memory: 500Mi
#   Lower Bound:
#     CPU: 100m
#     Memory: 200Mi
#   Upper Bound:
#     CPU: 500m
#     Memory: 1Gi
#   Uncapped Target:
#     CPU: 250m
#     Memory: 500Mi

VPA and HPA Cooperation

VPA and HPA can cooperate, but cannot manage the same resource dimension simultaneously:

ApproachHPAVPANotes
Option 1CPU scalingMemory adjustmentHPA manages CPU, VPA manages memory
Option 2Custom metric scalingCPU+memory adjustmentHPA manages business metrics, VPA manages resources
Option 3CPU scalingOff modeVPA only recommends, manual adjustment

VPA Production Recommendations

  1. Use Initial mode in production: Only applies recommendations at new Pod creation, doesn’t affect running Pods
  2. Evaluate with Off mode first: Run for a while to observe if recommendations are reasonable
  3. Set minAllowed/maxAllowed: Prevent anomalous recommendations
  4. Don’t manage CPU with both HPA and VPA: This is the most common configuration conflict

Cluster Autoscaler

How It Works

Cluster Autoscaler (CA) focuses on Pending Pods. When HPA scaling causes cluster resource shortage and Pods are in Pending state, CA automatically adds nodes:

HPA scales up → Pod Pending (insufficient resources) → CA adds node → Pod scheduled
Traffic drops → HPA scales down → Node utilization low → CA removes node

CA decision logic:

  1. Periodically scan for Pending Pods
  2. If Pending is due to resource shortage, simulate scheduling to calculate needed nodes
  3. Call cloud provider API to create new nodes
  4. After new nodes join cluster, Pending Pods are scheduled

Scale-Down Logic

CA is more cautious about scaling down:

  1. Find low-utilization nodes (total Pod CPU/memory requests < threshold)
  2. Simulate migrating that node’s Pods to other nodes
  3. If migration is possible, drain the node and delete it
# Cluster Autoscaler configuration example (AWS EKS)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
spec:
  template:
    spec:
      containers:
      - image: registry.k8s.io/autoscaling/cluster-autoscaler:v1.30.0
        name: cluster-autoscaler
        command:
        - ./cluster-autoscaler
        - --scale-down-unneeded-time=10m        # Scale down after 10min low utilization
        - --scale-down-delay-after-add=10m      # No scale-down within 10min after scale-up
        - --scale-down-unempty-time=30m         # Wait 30min after drain before deleting
        - --max-node-provision-time=15m         # Max wait for node creation
        - --balance-similar-node-groups=true    # Balance similar node groups
        - --expander=least-waste               # Expansion strategy: least waste
        - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
        env:
        - name: AWS_REGION
          value: us-east-1

Expansion Strategies (Expander)

StrategyDescriptionUse Case
randomRandom selection (default)Simple scenarios
most-podsSelect node group that schedules most PodsPrioritize Pod scheduling
least-wasteSelect node group with least resource wasteCost optimization
priceSelect cheapest node groupCloud cost optimization
prioritySelect by priorityMixed node groups

Cloud Provider CA Implementations

CloudImplementationCharacteristics
AWS EKSAuto Scaling GroupMature and stable
GCP GKENative integrationOut of the box
Azure AKSVM Scale SetMature and stable
Self-hostedCluster APINeed to manage own infrastructure

CA Limitations

LimitationDescription
Not real-time scalingNode creation takes 1-5 minutes
No cross-AZ schedulingNode groups are AZ-bound
Pod eviction limited by PDBPDB may prevent scale-down
Spot instance reclamationNeed Node Termination Handler
No node type selectionOnly scales within node group

KEDA: Event-Driven Autoscaling

Why KEDA

HPA scales based on CPU/memory/custom metrics, but many scenarios’ scaling signals are not resource metrics but events:

  • Kafka queue backlog → scale up consumers
  • Redis queue length increasing → scale up workers
  • Cron schedule → scheduled scaling
  • PostgreSQL high connection count → scale up

KEDA (Kubernetes Event-Driven Autoscaling) is a CNCF project specifically for event-driven autoscaling.

KEDA Architecture

Event source (Kafka/Redis/...) → KEDA Scaler → KEDA Operator → HPA → Deployment
                                       ScaledObject (CRD)
                                       ScaledJob (CRD)

KEDA’s workflow:

  1. Deploy ScaledObject CRD, defining event source and scaling rules
  2. KEDA Operator watches ScaledObject, creates corresponding HPA
  3. KEDA External Scaler gets metrics from event source
  4. HPA scales based on metrics

Install KEDA

helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda \
  --namespace keda-system \
  --create-namespace

Kafka Consumer Scaling Example

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: kafka-consumer-scaledobject
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: kafka-consumer
  minReplicaCount: 0                  # Scale to 0 when no messages
  maxReplicaCount: 50                 # Max 50 consumers
  pollingInterval: 30                 # Check every 30 seconds
  cooldownPeriod: 300                 # Wait 300s before scaling to 0
  triggers:
  - type: kafka
    metadata:
      bootstrapServers: kafka-broker:9092
      consumerGroup: my-consumer-group
      topic: orders
      lagThreshold: "100"             # Scale up when lag exceeds 100 per partition
      offsetResetPolicy: latest
      partitionLimitation: "0,1,2,3"  # Only monitor specific partitions

Supported Event Sources

KEDA supports 60+ event sources:

CategoryEvent Sources
Message queuesKafka, RabbitMQ, AWS SQS, Azure Service Bus, NATS
DatabasesPostgreSQL, MySQL, MongoDB, Redis
MonitoringPrometheus, Datadog
Cloud servicesAWS CloudWatch, Azure Monitor, GCP Pub/Sub
SchedulingCron
CustomExternal Scaler

ScaledJob: Batch Processing Scaling

For batch processing tasks, KEDA provides ScaledJob, which creates Jobs directly instead of scaling Deployments:

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: image-processor
  namespace: production
spec:
  jobTargetRef:
    template:
      spec:
        containers:
        - name: processor
          image: myapp/processor:v1
          command: ["./process"]
        restartPolicy: Never
  maxReplicaCount: 10
  pollingInterval: 30
  triggers:
  - type: aws-sqs
    metadata:
      queueURL: https://sqs.us-east-1.amazonaws.com/123456789012/image-queue
      queueLength: "5"              # Create one Job per 5 messages
      awsRegion: us-east-1

KEDA Advantages

AdvantageDescription
Scale to ZeroScale to 0 when no events, extreme cost savings
Rich event sources60+ event sources
HPA compatibleGenerates HPA underneath, can coexist
Simple to useOne ScaledObject CRD does it all

Scaling Strategy Combinations

ScenarioHPAVPACAKEDA
Web APICPU + QPSInitialYesNo
Message consumerNoInitialYesYes
Batch processingNoOffYesScaledJob
DatabaseNoOffNoNo
WebSocketConnection countInitialYesNo

Complete Autoscaling Configuration Example

# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-server
  template:
    metadata:
      labels:
        app: api-server
    spec:
      containers:
      - name: api
        image: myapp/api:v1
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 1000m
            memory: 1Gi
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5

---
# HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 30
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "500"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 30
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 10
        periodSeconds: 60

---
# PDB (graceful eviction during HPA scale-down)
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-server-pdb
  namespace: production
spec:
  minAvailable: 2            # Keep at least 2 Pods available
  selector:
    matchLabels:
      app: api-server

PodDisruptionBudget Cooperation

Both HPA scale-down and CA node scale-down evict Pods, requiring PDB protection:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-server-pdb
spec:
  minAvailable: 50%          # Keep at least 50% available
  # or maxUnavailable: 1     # At most 1 unavailable
  selector:
    matchLabels:
      app: api-server
PDB ParameterDescriptionUse Case
minAvailable: NAt least N availableFixed replica count
minAvailable: 50%At least 50% availableElastic replica count
maxUnavailable: 1At most 1 unavailableConservative strategy
maxUnavailable: 25%At most 25% unavailableAggressive strategy

Production Practices

Autoscaling Monitoring

# Prometheus alert rules
groups:
- name: hpa-alerts
  rules:
  # HPA at max replicas
  - alert: HPAAtMaxReplicas
    expr: kube_hpa_status_condition{condition="ScalingLimited",status="true"} == 1
    for: 10m
    annotations:
      summary: "HPA {{ $labels.hpa }} reached scaling limit"

  # HPA cannot get metrics
  - alert: HPAMetricsUnavailable
    expr: kube_hpa_status_condition{condition="ScalingActive",status="false"} == 1
    for: 5m
    annotations:
      summary: "HPA {{ $labels.hpa }} cannot get metrics"

  # CA cannot scale
  - alert: ClusterAutoscalerUnschedulable
    expr: cluster_autoscaler_unschedulable_pods_count > 0
    for: 10m
    annotations:
      summary: "{{ $value }} Pods cannot be scheduled"

Autoscaling Testing

# Load test to trigger HPA scale-up
kubectl run load-generator --image=busybox:latest --restart=Never \
  -- /bin/sh -c "while true; do wget -q -O- http://api-server.production.svc.cluster.local:8080/; done"

# Watch HPA status
kubectl get hpa -n production -w

# Watch Pod scaling process
kubectl get pods -n production -l app=api-server -w

Autoscaling Troubleshooting

# View HPA detailed status
kubectl describe hpa myapp-hpa -n production

# View HPA metrics
kubectl get --raw "/apis/autoscaling/v2/namespaces/production/horizontalpodautoscalers/myapp-hpa" | jq .

# View custom metrics
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/production/pods/*/http_requests_per_second" | jq .

# View CA logs
kubectl logs -n kube-system -l app=cluster-autoscaler --tail=50

# View node scaling events
kubectl get events --field-selector reason=TriggeredScaleUp

Common Issues

IssueCauseSolution
HPA shows unknownPod missing requests or metrics-server abnormalCheck requests config and metrics-server
HPA not scalingMetric value below targetCheck actual metrics
HPA at max but still insufficientmaxReplicas too small or insufficient nodesIncrease max or check CA
Scale-down too slowstabilizationWindow too longReduce window
Scale-down flappingLarge traffic fluctuationsIncrease window or use min replicas as floor
CA not scalingNode group at limitIncrease ASG max
VPA not workingupdateMode is OffChange to Initial or Auto

Summary

K8s autoscaling is a multi-layered system. Key takeaways:

  1. HPA is the workhorse: CPU/memory-based HPA is the most fundamental and practical autoscaling mechanism. Always configure behavior to control scaling speed and prevent flapping.
  2. Custom metrics are more precise: CPU doesn’t reflect real load—QPS, queue depth, and other business metrics are closer to reality. Deploy Prometheus Adapter for custom metric HPA.
  3. Use VPA Auto mode with caution: VPA’s Auto mode requires Pod restarts. Use Initial or Off mode for recommendations in production.
  4. CA covers the node layer: HPA only manages Pod replica count. When nodes are insufficient, CA automatically adds nodes. Note that CA scale-down has latency—don’t expect real-time.
  5. KEDA covers event-driven: Use KEDA for message queue and batch processing scenarios, with Scale-to-Zero support for extreme cost savings.
  6. PDB is mandatory: Scaling without PDB can cause service unavailability, especially during CA node eviction.
  7. Monitoring and alerting are essential: HPA hitting limits and CA unable to scale both need alerts—otherwise you won’t know when autoscaling fails.

Autoscaling isn’t “configure and forget.” It requires continuous observation and tuning. Review autoscaling history weekly and adjust parameters based on actual traffic patterns.

References & Acknowledgments

This article referenced the following materials during writing. We thank the original authors for their contributions:

  1. Kubernetes Autoscaling Documentation — Kubernetes Official, referenced for Kubernetes Autoscaling Documentation