Managed Clusters Aren't Set-and-Forget: 5 Production Pitfalls After Migrating from Self-Hosted K8s to ACK

Overview

“Fully managed control plane” — this phrase makes many teams think migrating to an ACK managed cluster means they can kick back and relax. In reality, managed only takes care of etcd, kube-apiserver, kube-controller-manager, and kube-scheduler. Data plane problems? Still all yours.

I led a migration of 120+ microservices from a self-hosted K8s cluster to ACK Managed Cluster Pro. The migration took 3 months with zero-downtime cutover — but the first month after switching, 3 AM alerts never stopped. Not because ACK has problems, but because managed and self-hosted clusters behave differently in places you can’t see. These differences were masked by your own operational habits in the self-hosted world, and only surface after moving to managed.

This article breaks down the 5 most painful pitfalls I hit, each with root cause analysis and fix. If you’re planning a migration from self-hosted K8s to ACK, or just finished migrating and are drowning in alerts, this will save you some pain.

Bottom line: these 5 pitfalls aren’t ACK bugs — they’re cognitive gaps from moving between self-hosted and managed paradigms. When you self-host, you control every component and can debug from scratch. When managed, the control plane becomes a black box, and you need a different operational mindset.

Pitfall 1: Network Plugin Selection — Flannel vs Terway, Wrong Choice Means Performance Disaster

Background

The first decision when migrating to ACK is the network plugin. ACK supports two: Flannel and Terway. Choose wrong, and every network performance issue downstream traces back to it. And you can’t switch after cluster creation — wrong choice means rebuilding the cluster.

I chose Flannel for a simple reason: our self-hosted cluster used Flannel, so Pod CIDRs wouldn’t change, and application configs needed zero adjustment. Two weeks after go-live, problems surfaced.

First, P99 latency for one microservice jumped from 170ms (self-hosted) to 230ms. No application changes, no code changes — only the underlying network changed. Then came the MLPS 2.0 audit, where the auditor required Pod traffic to be auditable. Flannel’s Pod IPs live in a separate CIDR that VPC Flow Logs can’t capture — instant medium-risk finding.

Root Cause

Flannel on Alibaba Cloud uses VPC custom routes for cross-node Pod communication. Each node gets a Pod CIDR subnet, and Pod traffic is forwarded via VPC route tables. This works, but has three fundamental problems:

First, performance overhead. Flannel’s VXLAN mode requires encapsulation/decapsulation, adding 50 bytes overhead per packet. In microservice-heavy call patterns, this compounds. I ran a benchmark:

Test environment: 2 nodes ecs.g7ne.4xlarge, netperf TCP_CRR for Pod-to-Pod, wrk against Nginx Service with 100-byte small page. Data references Alibaba Cloud’s Terway and Cilium integration performance report.

Flannel’s data path: Client Pod → cni0 bridge → flannel.1 interface → VXLAN encapsulation → remote flannel.1 → cni0 → Server Pod. Four hops with two encapsulation operations — latency and throughput both suffer.

Terway Shared ENI’s data path: Client Pod → ENI → VPC direct → remote ENI → Server Pod. No encapsulation, no tunneling, pure VPC Layer 2. In the 100-byte small page test, Terway Shared ENI’s ClusterIP throughput was 277% higher than Flannel, with 50% lower latency.

Second, Pod IPs outside VPC subnet. Flannel’s Pod IPs use an independent CIDR (e.g., 172.20.0.0/16), separate from the VPC subnet (e.g., 192.168.0.0/16). You can’t use security groups to control Pod traffic, and VPC Flow Logs can’t audit it. During our MLPS 2.0 audit, the auditor flagged this as a medium-risk finding.

With Terway, Pod IPs share the VPC subnet with ECS instances. Security groups, Flow Logs, and Network ACLs all work. This difference is huge for compliance.

Third, no NetworkPolicy support. Flannel doesn’t support Kubernetes native NetworkPolicy. If your microservices need network isolation (e.g., database Pods only accessible from application Pods), Flannel can’t do it. You’d need to install Calico — but Flannel and Calico coexistence on Alibaba Cloud has more pitfalls: route table conflicts, ARP broadcast storms, and painful debugging.

Terway Exclusive ENI mode supports Kubernetes native NetworkPolicy. Shared ENI mode supports equivalent capabilities via security groups. Both modes meet network isolation requirements.

Fix

Rebuild the cluster with Terway Shared ENI mode. Why shared ENI instead of exclusive? In exclusive mode, each Pod occupies one ENI, and ECS instances have ENI limits (e.g., ecs.g7.large supports 3 ENIs, minus the primary leaves only 2 Pods). Pod density is too low. Shared ENI mode allows multiple Pods to share one ENI via IP allocation — density isn’t ENI-limited.

ECS instance specs and ENI limits directly determine Pod density in exclusive mode:

Shared ENI mode’s Pod limit depends on auxiliary IPs per ENI. For example, ecs.g7.2xlarge has 10 auxiliary IPs per ENI, 5 ENIs = 50 IPs, minus primary = ~45 Pods available.

Key Terway Shared ENI configuration:

# Terway Shared ENI eni-conf ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: eni-conf
  namespace: kube-system
data:
  eni_conf: |
    {
      "version": "1",
      "enable_eni_shared": true,      # Enable shared ENI mode
      "eni_subnet_id": "vsw-xxx",     # Pod vSwitch (can differ from nodes)
      "eni_security_group": "sg-xxx",  # Pod-level security group
      "max_pool_size": 20,            # ENI IP pool size (pre-allocated)
      "min_pool_size": 5              # Minimum reserved IPs (avoid Pod creation wait)
    }    

Configuration notes:

• eni_subnet_id can be different from the node’s vSwitch. I recommend using separate subnets for Pods and nodes for easier network policy and traffic isolation
• Don’t set max_pool_size too high — it pre-allocates IPs and can exhaust the vSwitch IP pool. Set it to 1.5x expected Pod count per node
• min_pool_size ensures reserved IPs are available, reducing Pod creation latency from seconds to milliseconds

Migration steps (if already on Flannel):

1. Create a new Terway cluster
2. Gradually migrate applications via canary deployment
3. Use the DaemonSet logging approach from K8s Log Collection Strategy to ensure log continuity
4. Decommission the old Flannel cluster

My recommendation:

• Under 50 nodes, non-performance-sensitive test clusters: Flannel is fine, simpler config
• 50+ nodes, production with high microservice call frequency: Terway Shared ENI + DataPathv2 acceleration mode
• Extreme network performance (e.g., AI training): Terway Exclusive ENI or IPvlan mode

Don’t be fooled by “Flannel is simpler.” On Alibaba Cloud, Terway is the native son. Flannel is kept for compatibility only.

Pitfall 2: CSI Cross-Zone Storage Scheduling — Why Pods Stay Pending Forever

Background

After migrating to ACK, we configured multi-AZ deployment for high availability. 3 availability zones with 10 Worker nodes each, StatefulSet running 30 Pods, each with a cloud disk for persistent storage.

In our self-hosted cluster, we used Ceph RBD — cross-node shared storage with no AZ limitations. After migrating to ACK, we switched to cloud disks (Block Storage), which are AZ-level resources — a disk created in AZ-A can only be attached to ECS instances in AZ-A.

When the StatefulSet scaled up, new Pods stayed Pending. Event logs showed had volume node affinity conflict — the PV was created in AZ-A, but the Pod was scheduled to AZ-B. Or PV creation failed with The specified AZone inventory is insufficient — the specified AZ had no disk inventory.

Root Cause

ACK’s default StorageClass uses the csi-disk provisioner with Immediate binding mode — PVC triggers PV and cloud disk creation immediately. But the Pod hasn’t been scheduled yet, so CSI doesn’t know which AZ the Pod will land in. The disk gets created in the default AZ.

When the Pod gets scheduled to a node in a different AZ, it finds the disk isn’t in the same AZ — mount fails. Pod status stays at ContainerCreating, with events showing:

Warning  FailedAttachVolume  2m (x10 over 5m)  attachdetach-controller  AttachVolume.Attach failed for volume "pvc-xxx" : rpc error: code = Internal desc = "had volume node affinity conflict"

This problem doesn’t exist in self-hosted clusters because Ceph RBD is distributed storage accessible from any node. Cloud disks are AZ-bound resources — this is an IaaS-layer constraint, not a K8s issue.

Common cloud disk CSI errors and root causes:

Fix

Change StorageClass volumeBindingMode from Immediate to WaitForFirstConsumer. With delayed binding, PVC creation doesn’t immediately create the disk — it waits until the Pod is scheduled to a node, then creates the disk in that node’s AZ.

# Fixed StorageClass — delayed binding + multi-AZ
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: alicloud-disk-ssd-topology
provisioner: diskplugin.csi.alibabacloud.com
parameters:
  type: cloud_ssd          # Disk type: cloud_ssd/cloud_essd/cloud_efficiency
  regionId: cn-shenzhen    # Region
  zoneId: cn-shenzhen-a,cn-shenzhen-b,cn-shenzhen-c  # Multiple AZs
  encrypted: "true"        # Encrypted disk (compliance requirement)
  performanceLevel: PL2   # ESSD performance level (PL0/PL1/PL2/PL3)
reclaimPolicy: Retain      # Use Retain in production, not Delete
volumeBindingMode: WaitForFirstConsumer  # Key: delayed binding
allowVolumeExpansion: true # Allow online expansion
mountOptions:
  - noatime                # Mount option: don't update access time
  - nodiratime

After the fix: Pod gets scheduled to node → CSI reads node topology → creates disk in node’s AZ → mount succeeds. had volume node affinity conflict errors disappear.

Cloud disk type selection guide:

I recommend ESSD PL1 as the baseline for production. Efficiency cloud disk’s IOPS is too low (5,000) — it becomes a bottleneck for MySQL-like high-IOPS workloads. SSD cloud disk and ESSD PL1 are similar in price but ESSD performs better, so there’s no reason to use SSD cloud disk.

Bonus pitfall: If you use NAS (file storage), NAS doesn’t have cross-AZ mount limitations, but it has a different problem — permission groups. If Pods run as non-root users, NAS mounts will fail with chown: Operation not permitted. The fix is to configure the NAS permission group to no_squash mode, or configure securityContext.fsGroup in the PV:

# NAS mount permission fix
spec:
  securityContext:
    fsGroup: 1000      # Container runs as GID 1000
    fsGroupChangePolicy: "OnRootMismatch"  # Only chown when root owner doesn't match
  containers:
  - name: app
    securityContext:
      runAsUser: 1000  # Non-root user
      runAsGroup: 1000
    volumeMounts:
    - name: nas-data
      mountPath: /data

Pitfall 3: Node Pool Scale-Down Delay — Cluster Autoscaler’s 10-Minute Window

Background

After migrating to ACK, we enabled Cluster Autoscaler (CA) for node auto-scaling. Scale up during daytime peak, scale down at night. Scale-up worked fine, but scale-down was always slow — node utilization was below 10%, and CA still waited nearly 20 minutes before scaling down.

In our self-hosted cluster, we used ESS (Elastic Scaling Service) with custom scripts. Nodes below 30% utilization for 5 minutes triggered scale-down. CA’s defaults are much more conservative.

The most painful part: no errors, no alerts — it just silently burns your money for an extra 20 minutes per scale-down event. Over a month, that’s thousands of RMB wasted. When the boss asks why cloud costs went up 15%, you discover CA’s defaults are too conservative.

Root Cause

Cluster Autoscaler’s scale-down logic is controlled by three time parameters:

Default scale-down-unneeded-time is 10 minutes, plus the scale-down-delay-after-add 10-minute protection period — worst case, a node sits idle for 20 minutes before being reclaimed. For pay-as-you-go ECS nodes, those 20 minutes are pure waste.

Another pitfall: When CA scales down, it needs to drain Pods from the node. If Pods use hostPath or local volumes, CA won’t drain them by default (to protect data), leaving the node permanently un-schedulable for scale-down.

In self-hosted clusters, our script forcefully drained all Pods. CA’s design is more conservative — it prioritizes data safety over cost savings. This is correct design, but the defaults are too conservative.

CA’s scaling decision flow:

1. Scan interval (--scan-interval, default 10s)
2. Check for Pending Pods → if yes, scale up
3. Check if node utilization < 50% (--scale-down-utilization-threshold)
4. Check if node idle time exceeds scale-down-unneeded-time (default 10m)
5. Check if post-scale-up protection period has passed (scale-down-delay-after-add, default 10m)
6. Attempt to drain all Pods from node (respecting PDB)
7. If drain succeeds → call cloud API to delete node
8. If drain fails → wait scale-down-delay-after-failure and retry

Fix

Tune CA parameters based on your workload. On ACK, modify cluster-autoscaler startup parameters via component management:

# Cluster Autoscaler parameter tuning
spec:
  containers:
  - name: cluster-autoscaler
    command:
    - ./cluster-autoscaler
    - --scale-down-unneeded-time=5m        # From 10m to 5m
    - --scale-down-delay-after-add=5m     # From 10m to 5m
    - --scale-down-delay-after-failure=1m # From 3m to 1m
    - --scan-interval=10s                  # Scan interval (default unchanged)
    - --max-node-provision-time=15m        # Node creation timeout
    - --balance-similar-node-groups=true   # Multi-AZ node pool auto-balancing
    - --expendable-pods-priority-cutoff=-10  # Low-priority Pods don't trigger scale-up
    - --scale-down-utilization-threshold=0.5  # Trigger scale-down below 50% utilization
    - --max-graceful-termination-sec=120   # Max graceful termination wait

Also annotate all Pods using hostPath to explicitly tell CA they’re safe to evict:

metadata:
  annotations:
    cluster-autoscaler.kubernetes.io/safe-to-evict: "true"

Configure PDB for critical applications to prevent unwanted eviction:

# PodDisruptionBudget protecting critical service
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-gateway-pdb
  namespace: production
spec:
  minAvailable: 2           # Keep at least 2 Pods available
  selector:
    matchLabels:
      app: api-gateway

My benchmark results:

Don’t set scale-down-unneeded-time too low. I tried 2 minutes — traffic fluctuations caused constant scale up/down cycles, and ECS creation fees actually increased. 5 minutes is a proven balance — no false scale-downs during traffic spikes, but idle nodes don’t run too long.

Node pool design recommendations:

ACK node pools are the foundation of layered elastic architecture. My recommended configuration:

Baseline uses annual subscription, spot and on-demand use pay-as-you-go. During traffic peaks, scale up the spot pool first (cheaper). When spot instances are reclaimed, automatically switch to on-demand pool. This combination saves 30%+ compared to a single node pool.

Pitfall 4: Image Pull Rate Limiting — The Hidden Cost of Free ACR Accelerators

Background

After migrating to ACK, we used Alibaba Cloud Container Registry (ACR) Personal Edition (free) as our image registry, with Alibaba Cloud’s free image accelerator. Most of the time it worked fine, but one morning during CI/CD batch deployment of 30 microservices, half the Pods got stuck in ImagePullBackOff.

Pod event logs showed:

Warning  Failed   3m (x5 over 4m)  kubelet  Error: ErrImagePull
Warning  Failed   3m                kubelet  Failed to pull image "registry.cn-shenzhen.aliyuncs.com/xxx/app:latest":
  rpc error: code = Unknown desc = failed to pull and unpack image "registry.cn-shenzhen.aliyuncs.com/xxx/app:latest":
  failed to extract layer sha256:xxx: process "/bin/sh -c apt-get install -y ..." did not complete successfully: exit code: 1
Normal   BackOff    2m (x8 over 4m)  kubelet  Back-off pulling image

Looked like a build issue, but it was actually accelerator sync delay — the latest tag pulled was from two days ago, and the old version had a bug causing apt-get to fail.

Root Cause

Alibaba Cloud made changes to the free image accelerator in July 2024:

1. Restricted to Alibaba Cloud ECS/ACK intranet — non-Alibaba Cloud machines get 403 Forbidden
2. Stopped real-time sync of latest Docker Hub images — latest tag frequently returns stale versions, sync delay can be days
3. Mass batch pulls trigger rate limiting — free accelerator has no dedicated SLA, 30 Pods pulling simultaneously triggers 429 Too Many Requests

We used image: latest in CI/CD, and accelerator sync delay meant pulling a two-day-old version. During batch deployment, 30 Pods pulling simultaneously exhausted the free accelerator’s concurrent connection limit, returning 429.

Fix

Two-step approach:

Step 1: Use exact version tags in all CI/CD, disable latest. Enforce Git commit SHA or semantic versioning in the build pipeline:

#!/bin/bash
# CI/CD pipeline image build script
set -euo pipefail

IMAGE_REGISTRY="registry.cn-shenzhen.aliyuncs.com"
NAMESPACE="my-namespace"
APP_NAME="app"
IMAGE_TAG=$(git rev-parse --short HEAD)

# Build and push exact version image
docker build -t ${IMAGE_REGISTRY}/${NAMESPACE}/${APP_NAME}:${IMAGE_TAG} .
docker push ${IMAGE_REGISTRY}/${NAMESPACE}/${APP_NAME}:${IMAGE_TAG}

# Also push a stable tag (for rollback)
docker tag ${IMAGE_REGISTRY}/${NAMESPACE}/${APP_NAME}:${IMAGE_TAG} \
           ${IMAGE_REGISTRY}/${NAMESPACE}/${APP_NAME}:stable-${CI_PIPELINE_ID}
docker push ${IMAGE_REGISTRY}/${NAMESPACE}/${APP_NAME}:stable-${CI_PIPELINE_ID}

# Output image address for deployment step
echo "IMAGE=${IMAGE_REGISTRY}/${NAMESPACE}/${APP_NAME}:${IMAGE_TAG}" > deploy.env

Step 2: Upgrade to ACR Enterprise Edition, configure artifact sync. ACR Enterprise supports automatic image sync from Docker Hub, GCR, k8s.io to your private registry, with minute-level latency. Also configure Pod imagePullPolicy: IfNotPresent to avoid redundant pulls:

# Pod image pull policy
spec:
  containers:
  - name: app
    image: registry.cn-shenzhen.aliyuncs.com/my-namespace/app:a1b2c3d
    imagePullPolicy: IfNotPresent  # Use local cache if available
  imagePullSecrets:
  - name: acr-credential          # ACR Enterprise auth Secret

Node containerd image acceleration config:

# /etc/containerd/config.toml image acceleration
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
    endpoint = ["https://registry.cn-shenzhen.aliyuncs.com"]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.k8s.io"]
    endpoint = ["https://registry.cn-shenzhen.aliyuncs.com"]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."gcr.io"]
    endpoint = ["https://registry.cn-shenzhen.aliyuncs.com"]

Cost comparison: ACR Enterprise costs ~300 RMB/month, but the debugging time and downtime costs saved far exceed this. I calculated that batch deployment failure: 30 microservices × average 2 min debugging × 3 people = 3 hours of wasted effort. One incident pays for half a year of ACR Enterprise.

Image pre-warming strategy (recommended for large clusters):

When a cluster has 50+ nodes, batch deployment causes all nodes to pull images simultaneously, overwhelming the registry. Pre-warm images on nodes before deployment:

#!/bin/bash
# Image pre-warming script — pull images on all nodes before deployment
IMAGE="registry.cn-shenzhen.aliyuncs.com/my-namespace/app:a1b2c3d"
NODES=$(kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="InternalIP")].address}')

for NODE in $NODES; do
  ssh root@$NODE "crictl pull $IMAGE" &
done
wait
echo "All nodes image pre-warming complete"

Deploy after pre-warming — Pods use cached images directly, no pulling needed. 100-node cluster deployment time drops from 8 minutes to 2 minutes.

Pitfall 5: Monitoring Blind Spots — Managed Control Plane Observability Gap

Background

In our self-hosted cluster, Prometheus directly scraped kube-apiserver, etcd, and kube-controller-manager metrics. After migrating to ACK managed cluster, the control plane is managed by Alibaba Cloud — kube-apiserver’s address is an internal domain, and etcd is completely unreachable. Our entire monitoring and alerting setup became useless.

In the first week, we hit a mysterious issue: API requests occasionally timed out, but Worker node resource levels were normal and Pods weren’t restarting. After hours of debugging, we discovered the control plane’s kube-apiserver was responding slowly — but we couldn’t see any control plane metrics in the managed cluster. It was like driving blindfolded.

This issue is a classic “blind spot alert” — you know something is wrong but can’t see the root cause. It connects to the alert noise治理 framework mentioned in Alerting Strategy Design: From Noise to Signal.

Root Cause

ACK managed cluster’s control plane is invisible to users. You can’t access kube-apiserver’s /metrics endpoint, and you can’t see etcd’s state. ACK Pro provides control plane observability enhancements, but they’re not enabled by default — you need to manually install ack-kubernetes-dashboard or enable control plane monitoring in component management.

Metrics you took for granted in self-hosted clusters either aren’t visible or require extra setup in managed clusters:

Key difference: in self-hosted clusters, you can directly curl http://kube-apiserver:6443/metrics to get raw Prometheus format data. In managed clusters, you can only indirectly get aggregated metrics via CloudMonitor API — less granular and less flexible.

Fix

Step 1: Enable control plane monitoring in ACK Console. Go to Cluster Details → Component Management → install ack-kubernetes-dashboard and metrics-server. Pro clusters can also enable etcd monitoring metrics in CloudMonitor.

Step 2: Configure Prometheus to scrape control plane metrics. ACK managed cluster’s control plane monitoring is exposed via CloudMonitor, requiring configuration to scrape CloudMonitor metrics:

# Prometheus scraping ACK control plane metrics (via CloudMonitor exporter)
scrape_configs:
- job_name: 'ack-control-plane'
  cloudmon_configs:
  - region: 'cn-shenzhen'
    metrics:
    - 'acs_k8s_controlplane_ApiServerQPS'       # API QPS
    - 'acs_k8s_controlplane_ApiServerLatency'   # API latency
    - 'acs_k8s_controlplane_EtcdRequestLatency' # etcd read/write latency
    - 'acs_k8s_controlplane_SchedulerLatency'  # Scheduling latency
    - 'acs_k8s_controlplane_ControllerManagerQueue' # Controller queue depth

- job_name: 'ack-worker-nodes'
  kubernetes_sd_configs:
  - role: node
  relabel_configs:
  - source_labels: [__address__]
    regex: '(.*):10250'
    target_label: __address__
    replacement: '${1}:9100'  # node_exporter port

Step 3: Add alert rules. Key control plane alert thresholds:

# Control plane alert rules
groups:
- name: ack-control-plane-alerts
  rules:
  - alert: ApiServerHighLatency
    expr: histogram_quantile(0.99, rate(acs_k8s_controlplane_ApiServerLatency_bucket[5m])) > 1
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "kube-apiserver P99 latency exceeds 1 second"
      description: "Cluster {{ $labels.cluster }} API Server P99 latency is {{ $value }}s"
      
  - alert: EtcdHighLatency
    expr: histogram_quantile(0.99, rate(acs_k8s_controlplane_EtcdRequestLatency_bucket[5m])) > 0.1
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "etcd P99 latency exceeds 100ms"
      description: "etcd read/write latency too high, may cause cluster instability"
      
  - alert: SchedulerHighLatency
    expr: histogram_quantile(0.99, rate(acs_k8s_controlplane_SchedulerLatency_bucket[5m])) > 5
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "kube-scheduler P99 latency exceeds 5 seconds"
      description: "Scheduling latency too high, new Pods may stay Pending for extended periods"

Step 4: Configure Grafana Dashboard. Control plane monitoring dashboard should include these panels:

Key lesson: Managed doesn’t mean don’t monitor. The control plane isn’t your responsibility to maintain, but when it has issues, the first person getting paged is still the SRE. Configure control plane monitoring before migration completes — don’t wait for problems to fill the gaps. This pitfall is also covered in Reliability Measurement Framework — observability gaps are an SRE’s biggest blind spot.

Migration Decision Framework: When to Choose ACK Managed Cluster

Based on this migration experience, I’ve summarized a decision framework. Managed clusters aren’t right for every scenario:

My clear recommendation: Use ACK Pro Managed for production. Reasons:

1. etcd operations are the biggest risk in self-hosted K8s — disk full, network partition, write spike — any one can take down the entire cluster. Managed etcd is operated by Alibaba Cloud with hot/cold backup and disaster recovery. You only need to focus on the application layer.
2. Control plane SLA comes with compensation standards. Self-hosted kube-apiserver outage means you’re on your own. Managed at least has contractual backing.
3. But you must accept one premise: control plane observability is worse than self-hosted. You monitor via CloudMonitor metrics instead of direct scraping — a mindset shift many people find uncomfortable.

Pre-migration Checklist:

Cost Comparison: Self-Hosted vs ACK Managed

Migration decisions ultimately come down to cost. Here’s my detailed comparison:

Cost savings come from two areas: no Master node hardware costs, and no etcd ops labor. The 467 RMB/month management fee is far less than the 2,700 RMB/month self-hosted Master hardware. Labor savings are even bigger — etcd operations are the most labor-intensive part of self-hosted K8s.

But one hidden cost is easily overlooked: the migration itself. 3-month migration period, 2 full-time engineers, extra resources during canary cutover — these one-time costs run about 50,000-80,000 RMB. At 10,000 RMB/month savings, payback period is 6-8 months.

Summary

After migrating to ACK managed cluster, the control plane is indeed managed for you, but data plane pitfalls are as numerous as ever. The common thread across these 5 pitfalls: they all stem from behavioral differences between self-hosted and managed clusters in places you can’t see. Your operational habits masked these differences in self-hosted — they only surface after migration.

Core lessons from 5 pitfalls:

1. Choose Terway over Flannel — 50% performance difference, no NetworkPolicy support, Pod IPs outside VPC. For 50+ node production clusters, don’t hesitate.
2. StorageClass must use WaitForFirstConsumer — Cloud disks are AZ-level resources. Delayed binding ensures Pod and disk land in the same AZ. Can’t fix after the fact — prevent it.
3. Tune Cluster Autoscaler parameters — Default 10-minute idle threshold is too conservative. Tuning to 5 minutes saves 15% on elastic node costs. But don’t go below 3 minutes or you’ll get flapping.
4. Use exact image tags + ACR Enterprise — Free accelerator has rate limiting and sync delay. CI/CD batch deployment will break. One incident’s labor cost pays for half a year of Enterprise.
5. Configure control plane monitoring before migration — Managed doesn’t mean don’t monitor. kube-apiserver latency and etcd read/write latency are exposed via CloudMonitor in ACK Pro, but not enabled by default.

One bigger lesson: migrating to managed isn’t the endpoint — it’s a new starting point. Managed solves the “control plane availability” problem but introduces a new “observability degradation” problem. A good SRE doesn’t avoid mistakes — they think through potential pitfalls before migration and prepare contingencies. My biggest mistake in that migration was assuming managed means one less monitoring setup to configure. In reality, the monitoring configuration method changed, but monitoring itself can’t be reduced.

If you’re doing a similar migration, I suggest checking each of these 5 pitfalls. Not that you’ll hit every one, but knowing about them means when problems arise, at least you know which direction to investigate — instead of Googling “ACK Pod Pending volume node affinity conflict” at 3 AM.

References & Acknowledgments

This article referenced the following materials during writing. Thanks to the original authors:

1. Alibaba Cloud ACK Network Plugin Documentation — Alibaba Cloud official, Terway vs Flannel feature comparison and configuration
2. 2026 In-Depth Analysis of Alibaba Cloud ACK Managed Cluster — Zhihu, ACK Pro vs Basic feature differences and network/storage practices
3. Cloud Disk Storage Volume FAQ — Alibaba Cloud official, CSI disk cross-AZ scheduling failure causes and solutions
4. Cilium Integrates Alibaba Cloud ENI in Latest Release — CSDN, Terway IPvlan mode and Cilium eBPF performance comparison
5. Cluster Autoscaler Community Case: Large-Scale Production Experience — CSDN, CA multi-AZ node group balancing and scale-down parameter tuning
6. Alibaba Cloud Cloud-Native Elastic Solution — Zhihu, ACK elastic scaling architecture and HPA/CronHPA configuration guide
7. Alibaba Cloud Docker Image Accelerator Complete Review — CSDN, 2026 image accelerator usage limit changes and ACR Enterprise artifact sync
8. Pod Troubleshooting SOP — Alibaba Cloud official, ACK cluster Pod abnormal state troubleshooting methodology
9. Multi-Cloud Architecture Design: From Vendor Lock-in to Cross-Cloud Disaster Recovery — Multi-cloud architecture decisions and vendor lock-in avoidance
10. K8s Log Collection Strategy: DaemonSet, Sidecar, and Agentless — K8s log collection method selection on ACK
11. Reliability Measurement Framework: Four-Layer Model — Observability system building and monitoring blind spot identification