K3s Edge Computing in Practice: Lightweight Kubernetes Deployment and Operations for Resource-Constrained Environments

Overview

You’re an ops engineer at a smart manufacturing company. The factory floor has 200 edge gateways, each running data collection and real-time quality inspection services. Previously deployed with bare Docker, every update meant writing scripts to SSH into each machine, pull images, and restart containers. Running through 200 machines took half an hour, with a few always failing due to network jitter.

You think: isn’t this exactly what Kubernetes solves? Orchestration, scheduling, rolling updates, self-healing — all there. But when you actually try to install K8s, reality hits hard: the factory gateways use ARM-based industrial PCs with 2-core CPUs and 2GB RAM. etcd alone eats 500MB, and running kube-apiserver, kube-scheduler, kube-controller-manager leaves almost nothing for actual workloads.

Enter K3s. It’s a lightweight Kubernetes distribution developed by Rancher, packaging all control plane components into a single 50MB binary. Memory footprint is under 512MB, supporting ARM64/x86_64, with built-in containerd runtime, Flannel networking, CoreDNS, and Traefik Ingress — ready out of the box. It runs on a Raspberry Pi, let alone an industrial PC.

This article covers K3s in edge computing scenarios: from architecture to cluster setup, from networking to edge autonomy, from monitoring to troubleshooting. Not a beginner tutorial rehash, but lessons learned from production deployments.

K3s Architecture: Why It Runs on 512MB

Key Differences from K8s

K3s isn’t a “stripped-down K8s” — that characterization is too crude. It’s a Kubernetes distribution redesigned for resource-constrained environments. The core differences:

Feature	K8s	K3s
Binary size	~300MB (multiple components)	~50MB (single binary)
Minimum memory	2GB	512MB
Storage backend	etcd (required)	SQLite (default) / etcd / MySQL / PostgreSQL
Container runtime	Requires separate containerd/Docker	Built-in containerd
CNI networking	Manual installation	Built-in Flannel
Ingress	Manual installation	Built-in Traefik
DNS	Manual installation	Built-in CoreDNS
Alpha/Beta features	All included	Removed
Cloud provider code	All included	Removed
Architecture support	x86_64 / ARM64	x86_64 / ARM64 / ARMv7

K3s’s core design philosophy: in edge scenarios, you need K8s’s orchestration capability, not its full complexity. After removing alpha/beta features and cloud provider-specific code, K3s retains K8s’s core API and functionality — Pod, Deployment, Service, ConfigMap, HPA, CronJob are all there, and kubectl commands are fully compatible.

Single Binary Architecture

K3s packages the following components into one binary:

k3s binary
├── kube-apiserver       # API server
├── kube-scheduler       # Scheduler
├── kube-controller-manager  # Controller manager
├── kubelet              # Node agent
├── kube-proxy           # Network proxy
├── containerd           # Container runtime
├── flannel              # CNI network plugin
├── coredns              # DNS service
├── traefik              # Ingress controller
├── servicelb            # Service load balancer
└── local-path-provisioner  # Local storage volume provisioner

This means you don’t need to separately install and configure a dozen components like in K8s. One binary, one command, done.

Storage Backend Selection

K3s defaults to SQLite for storage — perfect for single-node clusters with zero extra dependencies and zero resource overhead. But SQLite doesn’t support multi-replica writes, so multi-node Server clusters need a different backend:

Storage Backend	Use Case	Resource Overhead	HA Support
SQLite (default)	Single-node Server	Very low	No
etcd	Multi-node Server, production	Medium	Yes
MySQL	Existing MySQL infrastructure	Medium	Yes
PostgreSQL	Existing PG infrastructure	Medium	Yes

My recommendation: for single-node Server in edge scenarios, the default SQLite is sufficient. For HA multi-Server, use embedded etcd (K3s supports starting built-in etcd with --cluster-init mode).

Single-Node Cluster Setup: Up and Running in 5 Minutes

Environment Preparation

Using Ubuntu 22.04 LTS as an example (both ARM64 and x86_64):

# Basic environment setup
sudo apt update && sudo apt upgrade -y

# Install time sync (K8s components are time-sensitive)
sudo apt install chrony -y
sudo systemctl enable --now chrony

# Disable swap (Kubelet doesn't support swap)
sudo swapoff -a
sudo sed -i '/swap/d' /etc/fstab

# Load required kernel modules
sudo modprobe overlay
sudo modprobe br_netfilter

# Persist kernel modules
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

# Kernel parameter tuning
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

One-Command Installation

# Install K3s Server (use mirror acceleration for China)
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-installation.sh | \
  INSTALL_K3S_MIRROR=cn sh -s - \
  --write-kubeconfig-mode 644 \
  --disable traefik \
  --node-label "node-type=edge"

# Check installation status
sudo k3s kubectl get nodes
sudo k3s kubectl get pods -A

Key parameter explanations:

--write-kubeconfig-mode 644: Allows non-root users to read kubeconfig
--disable traefik: Disable Traefik if you don’t use Ingress, saves resources
--node-label: Label the node for scheduling purposes

After installation, the K3s kubeconfig is at /etc/rancher/k3s/k3s.yaml. Copy it to ~/.kube/config to use standard kubectl:

mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/config

China Mirror Acceleration

Edge devices in China often pull Docker Hub images painfully slowly. K3s uses containerd, not Docker, so the configuration is different:

# Create containerd mirror configuration
sudo mkdir -p /etc/rancher/k3s

cat <<EOF | sudo tee /etc/rancher/k3s/registries.yaml
mirrors:
  docker.io:
    endpoint:
      - "https://registry.cn-hangzhou.aliyuncs.com"
  gcr.io:
    endpoint:
      - "https://registry.cn-hangzhou.aliyuncs.com"
  quay.io:
    endpoint:
      - "https://quay.mirrors.ustc.edu.cn"
configs:
  "registry.cn-hangzhou.aliyuncs.com":
    auth:
      username: "your-aliyun-account"
      password: "your-password"
EOF

# Restart K3s to apply
sudo systemctl restart k3s

Multi-Node Cluster: Server + Agent Architecture

Architecture Design

In edge scenarios, multi-node clusters are typically designed like this:

┌─────────────────────────────────────┐
│       Edge Room / Factory Floor      │
│                                     │
│  ┌──────────┐    ┌──────────┐      │
│  │ K3s Server│    │ K3s Agent│      │
│  │ (Manager) │    │ (Worker) │      │
│  │ 2C/4G    │    │ 2C/2G    │      │
│  └─────┬────┘    └─────┬────┘      │
│        │                │           │
│        │   ┌──────────┐│           │
│        └───┤ K3s Agent ├┘           │
│            │ (Worker) │            │
│            │ 2C/2G    │            │
│            └──────────┘            │
│                                     │
└─────────────────────────────────────┘

The Server node runs the control plane and can also run workloads. Agent nodes only run workloads. One Server managing a dozen Agents is perfectly adequate for edge scenarios.

Deploying the Server Node

# On the Server node
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-installation.sh | \
  INSTALL_K3S_MIRROR=cn sh -s - \
  --write-kubeconfig-mode 644 \
  --node-label "node-type=server" \
  --node-label "location=edge-room-1"

# Get Agent join token
cat /var/lib/rancher/k3s/server/node-token
# Output like: K10xxxxxxxxxxxx::server:xxxxxxxxxxxx

Deploying Agent Nodes

# On the Agent node
# Replace <SERVER_IP> with the Server node's IP
# Replace <NODE_TOKEN> with the token from the previous step
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-installation.sh | \
  INSTALL_K3S_MIRROR=cn sh -s - agent \
  --server https://<SERVER_IP>:6443 \
  --token <NODE_TOKEN> \
  --node-label "node-type=agent" \
  --node-label "location=edge-room-1"

# Verify Agent joined
sudo k3s kubectl get nodes

Air-Gapped Installation: An Edge Scenario Necessity

Many edge devices have no internet at all — factory networks are closed, images can only be pulled from internal registries. This requires air-gapped K3s installation.

# Step 1: Download K3s binary on a machine with internet
# Download URL: https://github.com/k3s-io/k3s/releases
# Select the binary for your architecture (amd64 or arm64)

wget https://github.com/k3s-io/k3s/releases/download/v1.30.0%2Bk3s1/k3s-arm64
# Note: Use k3s-arm64 for ARM64 devices, k3s for x86_64

# Step 2: Download K3s system images package
wget https://github.com/k3s-io/k3s/releases/download/v1.30.0%2Bk3s1/k3s-airgap-images-arm64.tar.zst

# Step 3: Transfer files to the edge device
scp k3s-arm64 edge-device:/usr/local/bin/k3s
scp k3s-airgap-images-arm64.tar.zst edge-device:/tmp/

# Step 4: Install on the edge device
ssh edge-device << 'EOF'
sudo chmod +x /usr/local/bin/k3s

# Import system images
sudo mkdir -p /var/lib/rancher/k3s/agent/images/
sudo cp /tmp/k3s-airgap-images-arm64.tar.zst /var/lib/rancher/k3s/agent/images/

# Download the K3s install script (also needs to be transferred from a networked machine)
# Assuming the script is at /tmp/k3s-install.sh
sudo chmod +x /tmp/k3s-install.sh
sudo /tmp/k3s-install.sh
EOF

The key to air-gapped installation is placing the K3s binary and airgap image package in the correct directories beforehand. The install script detects locally available images and skips downloading from the internet.

Network Solution Selection: Edge Scenario Trade-offs

K3s Built-in Networking

K3s defaults to Flannel (VXLAN mode) as the CNI. For most edge scenarios, this is sufficient, but there are a few things to note:

Network Mode	Use Case	Performance	Complexity
Flannel VXLAN (default)	General purpose	Medium	Low
Flannel Host-GW	Same-subnet nodes	High	Low
Calico	Network policies needed	High	Medium
Cilium	Advanced networking	High	High

For edge scenarios, I recommend the default Flannel VXLAN. The reason is straightforward: edge network topologies change frequently, and VXLAN has the lowest requirements for the underlying network — as long as nodes can reach each other by IP, it works. Host-GW mode offers better performance but requires all nodes to be on the same Layer 2 network, which is rarely the case across factory floors.

Custom Flannel CIDR

By default, K3s uses 10.42.0.0/16 for Pod CIDR and 10.43.0.0/16 for Service CIDR. If your edge network happens to use these ranges, you need to change them:

# Specify custom CIDRs during installation
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-installation.sh | \
  INSTALL_K3S_MIRROR=cn sh -s - \
  --cluster-cidr 172.20.0.0/16 \
  --service-cidr 172.21.0.0/16 \
  --cluster-dns 172.21.0.10

Network Instability Issues

The biggest pain point in edge scenarios is network instability — the connection between factory gateways and the Server may be intermittent. What happens when a K3s Agent goes offline?

Pods keep running: Agent going offline doesn’t affect already-running Pods; containers keep running in containerd
No new Pod scheduling: The Server considers the Agent unreachable and won’t schedule new Pods to it
Status reporting interrupted: Pod status and node resource usage can’t be reported to the Server

K3s has a key feature for edge scenarios — Edge Autonomy. When the Agent loses connection to the Server, the local kubelet continues managing existing Pods. It doesn’t kill Pods just because it can’t reach the Server. When the network recovers, the Agent reconnects automatically and syncs state.

But the default Pod eviction policy needs tuning, otherwise reconnection might trigger mass Pod rebuilds:

# Adjust kubelet eviction policy to tolerate longer disconnections
# Create configuration on Agent nodes
cat <<EOF | sudo tee /etc/rancher/k3s/agent.yaml
kubelet-arg:
  - "node-status-update-frequency=30s"
  - "node-monitor-period=30s"
  - "node-monitor-grace-period=5m"
  - "pod-eviction-timeout=5m"
EOF

# Restart Agent
sudo systemctl restart k3s-agent

Parameter meanings:

node-status-update-frequency: Agent reports status every 30s (default 10s; can reduce frequency to save bandwidth in edge scenarios)
node-monitor-grace-period: Server allows 5 minutes without reports (default 40s; needs to be extended for edge)
pod-eviction-timeout: Only mark Pods for eviction after 5 minutes (default 5m)

Edge Application Deployment in Practice

Deploying a Data Collection Service

apiVersion: apps/v1
kind: Deployment
metadata:
  name: data-collector
  namespace: edge-apps
  labels:
    app: data-collector
spec:
  replicas: 3
  selector:
    matchLabels:
      app: data-collector
  template:
    metadata:
      labels:
        app: data-collector
    spec:
      # Schedule only to Agent nodes
      nodeSelector:
        node-type: agent
      containers:
      - name: collector
        image: registry.cn-hangzhou.aliyuncs.com/edge/data-collector:v1.2.0
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 256Mi
        # Liveness probe is essential in edge scenarios to auto-restart during network jitter
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 30
          failureThreshold: 5  # Allow 5 failures (150s), tolerate network jitter
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 15
        # Edge scenario configuration via ConfigMap injection
        env:
        - name: COLLECT_INTERVAL
          valueFrom:
            configMapKeyRef:
              name: collector-config
              key: interval
        - name: MQTT_BROKER
          valueFrom:
            configMapKeyRef:
              name: collector-config
              key: mqtt-broker
        volumeMounts:
        - name: data-volume
          mountPath: /data
      volumes:
      - name: data-volume
        hostPath:
          path: /var/edge-data
          type: DirectoryOrCreate
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: collector-config
  namespace: edge-apps
data:
  interval: "5s"
  mqtt-broker: "tcp://mqtt.internal:1883"
---
apiVersion: v1
kind: Service
metadata:
  name: data-collector
  namespace: edge-apps
spec:
  selector:
    app: data-collector
  ports:
  - port: 8080
    targetPort: 8080
  type: ClusterIP

Rolling Update Strategy

Edge devices have limited bandwidth, and pulling images can be slow. Adjust rolling update parameters to avoid saturating bandwidth with concurrent image pulls:

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1    # Update only 1 Pod at a time
      maxSurge: 0          # Don't create extra Pods; reduce then add

Image Pre-distribution

200 edge devices pulling images simultaneously will overwhelm any registry bandwidth. A better approach is image pre-distribution — pulling images to each device during off-peak hours:

#!/bin/bash
# Image pre-distribution script
# Run on each Agent node to pre-pull the next version's images

IMAGES=(
  "registry.cn-hangzhou.aliyuncs.com/edge/data-collector:v1.3.0"
  "registry.cn-hangzhou.aliyuncs.com/edge/data-processor:v2.0.0"
)

for image in "${IMAGES[@]}"; do
  echo "[$(date)] Pulling image: $image"
  # K3s uses containerd; use crictl to pull
  sudo k3s crictl pull "$image"
  if [ $? -eq 0 ]; then
    echo "[$(date)] ✓ Pull successful"
  else
    echo "[$(date)] ✗ Pull failed, will retry later"
  fi
done

echo "Image pre-distribution complete. Next deployment will use local cache."

Then batch-execute via Ansible:

# Ansible playbook: image pre-distribution
- name: Pre-distribute images to edge nodes
  hosts: edge-agents
  become: yes
  tasks:
    - name: Copy pre-pull script
      copy:
        src: pre-pull-images.sh
        dest: /tmp/pre-pull-images.sh
        mode: '0755'

    - name: Run pre-pull script
      shell: /tmp/pre-pull-images.sh
      async: 600          # Max wait 10 minutes
      poll: 0             # Async execution, non-blocking

    - name: Check pre-pull status
      async_status:
        jid: "{{ ansible_job_id }}"
      register: job_result
      until: job_result.finished
      retries: 30
      delay: 20

Monitoring and Alerting: Observability for Edge Clusters

Monitoring for Resource-Constrained Environments

The standard Prometheus + Grafana monitoring stack is too heavy for edge devices — Prometheus alone consumes 500MB+ of memory. Edge scenarios need a lighter approach.

Recommended: K3s built-in metrics-server + lightweight Prometheus Agent + remote storage

Edge Cluster                        Cloud
┌─────────────────┐         ┌──────────────────┐
│ K3s Cluster     │         │ Prometheus +     │
│                 │         │ Grafana          │
│ metrics-server  │         │                  │
│ (resource metrs)│         │ (centralized     │
│                 │         │  query+alerting) │
│ Prometheus Agent├────────→│ Remote Write     │
│ (collect+forward)│        │                  │
└─────────────────┘         └──────────────────┘

Prometheus Agent mode only collects and forwards — no local storage — with very low resource usage (~50MB memory). All metrics are sent to the cloud Prometheus via remote_write for centralized storage and querying.

# Prometheus Agent configuration (edge node)
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-agent-config
  namespace: monitoring
data:
  prometheus.yml: |
    global:
      scrape_interval: 30s      # Lower collection frequency for edge
      evaluation_interval: 30s

    # Agent mode: collect and forward only, no local storage
    remote_write:
      - url: "https://cloud-prometheus.example.com/api/v1/write"
        # Edge networks are unstable; configure retries
        queue_config:
          capacity: 500
          max_shards: 2
          min_shards: 1
          max_samples_per_send: 100
          batch_send_deadline: 30s

    scrape_configs:
      # Collect K3s node metrics
      - job_name: 'k3s-node'
        static_configs:
          - targets: ['localhost:10250']
        scheme: https
        tls_config:
          ca_file: /var/lib/rancher/k3s/agent/client-ca.crt
          cert_file: /var/lib/rancher/k3s/agent/client-kubelet.crt
          key_file: /var/lib/rancher/k3s/agent/client-kubelet.key
        bearer_token_file: /var/lib/rancher/k3s/agent/client-kubelet.token

      # Collect container metrics
      - job_name: 'k3s-containers'
        static_configs:
          - targets: ['localhost:10250']
        scheme: https
        tls_config:
          ca_file: /var/lib/rancher/k3s/agent/client-ca.crt
          cert_file: /var/lib/rancher/k3s/agent/client-kubelet.crt
          key_file: /var/lib/rancher/k3s/agent/client-kubelet.key
        bearer_token_file: /var/lib/rancher/k3s/agent/client-kubelet.token
        metrics_path: /metrics/cadvisor

Key Alert Rules

Edge scenario alert rules differ from cloud — special attention to network disconnection and resource exhaustion:

# Edge cluster key alert rules
groups:
  - name: edge-cluster-alerts
    rules:
      # Agent node offline
      - alert: EdgeNodeOffline
        expr: up{job="k3s-node"} == 0
        for: 5m
        labels:
          severity: P1
        annotations:
          summary: "Edge node {{ $labels.instance }} offline for over 5 minutes"
          description: "Check node network connectivity and K3s Agent process status"

      # Node memory usage too high
      - alert: EdgeNodeMemoryHigh
        expr: |
          (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100 > 85          
        for: 10m
        labels:
          severity: P2
        annotations:
          summary: "Node {{ $labels.instance }} memory usage above 85%"
          description: "Edge devices have limited memory; check for abnormal Pod memory usage"

      # Disk space low
      - alert: EdgeNodeDiskSpaceLow
        expr: |
          (1 - node_filesystem_avail_bytes{fstype!~"tmpfs|overlay"} /
                node_filesystem_size_bytes{fstype!~"tmpfs|overlay"}) * 100 > 80          
        for: 15m
        labels:
          severity: P2
        annotations:
          summary: "Node {{ $labels.instance }} disk usage above 80%"
          description: "Check if container logs and images need cleanup"

      # Pod restart count too high
      - alert: EdgePodRestartLoop
        expr: increase(kube_pod_container_status_restarts_total[1h]) > 5
        for: 5m
        labels:
          severity: P2
        annotations:
          summary: "Pod {{ $labels.pod }} restarted more than 5 times in 1 hour"
          description: "Possible app crash or OOM; check container logs"

      # K3s Server unreachable
      - alert: K3sServerDown
        expr: up{job="k3s-apiserver"} == 0
        for: 2m
        labels:
          severity: P0
        annotations:
          summary: "K3s API Server unreachable"
          description: "Check K3s Server process and etcd/SQLite storage"

Troubleshooting: Common Edge Scenario Issues

Issue 1: Agent Node Join Failure

Symptom: After Agent installation, kubectl get nodes doesn’t show the node.

Troubleshooting steps:

# 1. Check Agent logs
sudo journalctl -u k3s-agent -f --no-pager | tail -50

# 2. Common cause: wrong token
# Verify the token on Server matches what Agent uses
cat /var/lib/rancher/k3s/server/node-token

# 3. Common cause: network unreachable
# Test connectivity from Agent to Server
curl -k https://<SERVER_IP>:6443/readyz

# 4. Common cause: firewall blocking
# K3s Server requires these ports:
# 6443/tcp - K8s API
# 8472/udp - Flannel VXLAN
# 51820/udp - Flannel WireGuard (if enabled)
# 10250/tcp - Kubelet

# 5. Common cause: time out of sync
# K8s components are time-sensitive; >30s drift causes TLS cert validation failure
timedatectl status

Issue 2: Pod Stuck in ContainerCreating

Symptom: Pod stays in ContainerCreating after deployment.

# View Pod events
kubectl describe pod <pod-name> -n <namespace>

# Common cause 1: image pull failure
# Check containerd image pull logs
sudo crictl logs $(sudo crictl ps -a --name <container-name> -q)

# Common cause 2: CNI plugin not ready
# Check Flannel Pod
kubectl get pods -n kube-system | grep flannel

# Common cause 3: disk full
df -h
# Clean unused images
sudo k3s crictl rmi --prune

Issue 3: Edge Node Frequent Disconnection

Symptom: Node frequently switches between Ready and NotReady.

# 1. Check network stability
ping -c 100 <SERVER_IP>
# If packet loss > 5%, network quality is poor

# 2. Adjust K3s Agent reconnection parameters
# Edit /etc/rancher/k3s/agent.yaml
cat <<EOF | sudo tee /etc/rancher/k3s/agent.yaml
server: https://<SERVER_IP>:6443
token: <NODE_TOKEN>
kubelet-arg:
  - "node-status-update-frequency=30s"
EOF

sudo systemctl restart k3s-agent

# 3. Check if node resources are exhausted
# If memory is insufficient, kubelet may not report properly
free -h

Issue 4: SQLite Database Lock

Single-node K3s using SQLite may occasionally experience database lock causing API Server unresponsiveness:

# Check K3s Server status
sudo systemctl status k3s

# If stuck, check logs
sudo journalctl -u k3s --no-pager | tail -100

# Typical lock logs:
# "database is locked" or "SQLITE_BUSY"

# Emergency recovery: restart K3s
sudo systemctl restart k3s

# Long-term solution: migrate to embedded etcd
# Stop current K3s
sudo /usr/local/bin/k3s-uninstall.sh

# Reinstall with etcd mode
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-installation.sh | \
  INSTALL_K3S_MIRROR=cn sh -s - \
  --cluster-init \
  --cluster-cidr 10.42.0.0/16 \
  --service-cidr 10.43.0.0/16

Multi-Cluster Management: Centrally Managing Edge Clusters from the Cloud

Edge scenarios typically involve dozens or hundreds of K3s clusters distributed across different rooms and floors. Manually managing each cluster’s kubectl context is impractical. Multi-cluster management tools are needed.

Solution Comparison

Tool	Maintainer	Scale	Resource Overhead	Features
Rancher	SUSE	Medium-Large	Higher	Full multi-cluster management
kubecm	Open Source	Small	Zero	Context switching only
Cluster API	CNCF	Large	Medium	Cluster lifecycle management
Fleet	SUSE	Large	Medium	GitOps + multi-cluster

For small deployments, kubecm is sufficient — it’s a CLI tool that manages kubeconfig contexts:

# Install kubecm
curl -sLo kubecm.tar.gz https://github.com/sunny0826/kubecm/releases/latest/download/kubecm_Linux_x86_64.tar.gz
tar -zxvf kubecm.tar.gz kubecm
sudo mv kubecm /usr/local/bin/

# Add cluster context
kubecm add edge-factory-1 --kubeconfig /path/to/factory-1-kubeconfig

# Switch cluster
kubecm switch edge-factory-1

# List all clusters
kubecm list

If you have more than 20 clusters, consider Rancher. Rancher manages both K3s and K8s clusters with a web interface and RBAC:

# Install Rancher on the cloud (quick deploy with Docker)
docker run -d --restart=unless-stopped \
  -p 80:80 -p 443:443 \
  --name rancher \
  rancher/rancher:v2.9.0

# Then register an Agent in each K3s cluster
# Rancher generates a kubectl apply command
# Execute it on the K3s Server

Performance Tuning: Squeezing Every Drop from Edge Devices

Resource Allocation Strategy

Edge devices have limited resources that must be carefully budgeted. Here’s a typical allocation for a 2C/4G industrial PC:

Total resources: 2 vCPU / 4GB RAM / 32GB Disk
─────────────────────────────────────
System reserved:     0.3 vCPU / 512MB
K3s components:      0.3 vCPU / 512MB
  - kubelet
  - containerd
  - Flannel
  - CoreDNS
  - metrics-server
Available for apps:  1.4 vCPU / 3GB
─────────────────────────────────────

K3s Server Tuning

# K3s Server startup parameter tuning
# Edit /etc/systemd/system/k3s.service

# Add these parameters to the ExecStart line:
# --kube-apiserver-arg="default-watch-cache-size=100"  # Reduce watch cache memory
# --kube-apiserver-arg="max-requests-inflight=200"     # Limit concurrent requests
# --kube-controller-manager-arg="node-sync-period=30s" # Reduce sync frequency
# --kube-controller-manager-arg="concurrent-deployments=2"  # Reduce concurrency
# --etcd-arg="quota-backend-bytes=17179869184"         # Limit etcd storage to 16GB

# Example configuration
ExecStart=/usr/local/bin/k3s server \
  --kube-apiserver-arg="default-watch-cache-size=100" \
  --kube-apiserver-arg="max-requests-inflight=200" \
  --kube-controller-manager-arg="node-sync-period=30s" \
  --kube-controller-manager-arg="concurrent-deployments=2" \
  --write-kubeconfig-mode 644

Container Runtime Tuning

# containerd configuration tuning
# Edit /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl

# Key tuning items:
# - Limit log size
# - Reduce GC frequency
# - Limit concurrent downloads

[plugins."io.containerd.grpc.v1.cri"]
  # Limit container log size
  max_container_log_line_size = 16384
  # Reduce image pull concurrency
  [plugins."io.containerd.grpc.v1.cri".image_decryption]
    key_model = "node"

# Log limits
[plugins."io.containerd.runtime.v1.linux"]
  # Keep max 3 log files per container
  # Max 10MB per log file
  log_rotation = true
  log_max_size = "10MB"
  log_max_files = 3

Automated Disk Cleanup

Edge devices have limited disk space. Container logs and images gradually fill the disk. Regular cleanup is needed:

#!/bin/bash
# K3s edge node disk cleanup script
# Recommended to run daily via cron

set -euo pipefail

LOG_FILE="/var/log/k3s-disk-cleanup.log"
THRESHOLD=75  # Trigger cleanup when disk usage exceeds 75%

echo "[$(date)] Starting disk cleanup..." >> "$LOG_FILE"

# Check disk usage
DISK_USAGE=$(df / | awk 'NR==2 {print $5}' | tr -d '%')
echo "[$(date)] Current disk usage: ${DISK_USAGE}%" >> "$LOG_FILE"

if [ "$DISK_USAGE" -lt "$THRESHOLD" ]; then
  echo "[$(date)] Disk usage below threshold, skipping cleanup" >> "$LOG_FILE"
  exit 0
fi

# 1. Clean unused container images
echo "[$(date)] Cleaning unused images..." >> "$LOG_FILE"
sudo k3s crictl rmi --prune >> "$LOG_FILE" 2>&1

# 2. Clean container logs (keep last 3 days)
echo "[$(date)] Cleaning container logs..." >> "$LOG_FILE"
find /var/lib/rancher/k3s/agent/containerd/logs -name "*.log" -mtime +3 -delete >> "$LOG_FILE" 2>&1

# 3. Clean K3s logs
echo "[$(date)] Cleaning K3s logs..." >> "$LOG_FILE"
sudo journalctl --vacuum-time=3d >> "$LOG_FILE" 2>&1

# 4. Clean temp files
echo "[$(date)] Cleaning temp files..." >> "$LOG_FILE"
find /tmp -type f -mtime +7 -delete >> "$LOG_FILE" 2>&1

# Check post-cleanup disk usage
DISK_USAGE_AFTER=$(df / | awk 'NR==2 {print $5}' | tr -d '%')
echo "[$(date)] Post-cleanup disk usage: ${DISK_USAGE_AFTER}%" >> "$LOG_FILE"
echo "[$(date)] Disk cleanup complete" >> "$LOG_FILE"

# Add cron job
# echo "0 3 * * * /usr/local/bin/k3s-disk-cleanup.sh" | sudo tee /etc/cron.d/k3s-cleanup

Security Hardening: Special Risks for Edge Devices

Edge devices are deployed in physically uncontrollable environments (factory floors, outdoor cabinets), with higher security risks than data center servers. Pay special attention to:

1. K3s API Server Authentication

# Don't use --write-kubeconfig-mode 644 in production
# Use stricter permissions
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-installation.sh | \
  INSTALL_K3S_MIRROR=cn sh -s - \
  --write-kubeconfig-mode 600

2. Network Isolation

# Use iptables to restrict K3s API to management network only
sudo iptables -A INPUT -p tcp --dport 6443 -s 10.0.0.0/8 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 6443 -j DROP

# Persist iptables rules
sudo apt install iptables-persistent -y
sudo netfilter-persistent save

3. Image Signature Verification

Edge devices can be physically accessed; attackers might replace images. Enable image signature verification:

# K3s containerd image verification configuration
# /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl

[plugins."io.containerd.grpc.v1.cri".containerd]
  # Enable image pull verification
  default_runtime_name = "runc"
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
    runtime_type = "io.containerd.runc.v2"

# Configure to trust only private registry
[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/etc/containerd/certs.d"

Summary

K3s’s value in edge computing isn’t just that it’s “light” — it’s that it reduces deployment and operations complexity by an order of magnitude while maintaining full K8s API compatibility. You manage edge clusters with the same kubectl operations as cloud K8s, but with a quarter of the resource footprint.

Key points for production:

Architecture: Single-node Server + SQLite works for small edge deployments (≤20 Agents). For large-scale, use embedded etcd multi-Server clusters. Existing MySQL/PG infrastructure can be reused.

Networking: The default Flannel VXLAN is most edge-friendly — low requirements for underlying network, tolerates cross-subnet. But tune kubelet’s disconnection tolerance parameters, otherwise network jitter causes frequent Pod rebuilds.

Application deployment: The core constraints in edge are bandwidth and resources. Image pre-distribution is more reliable than real-time pulling. Resource limits are mandatory — a Pod without limits on a 2GB device will OOM sooner or later.

Monitoring: Don’t run full Prometheus on edge devices. Use Agent mode for collection and forwarding only; centralized storage goes to the cloud. Alert rules should focus on node offline, disk space, and Pod restarts.

Security: Edge devices have uncontrollable physical security. Network isolation and image verification are essential. Don’t expose the K3s API port to the public internet.

One practical insight: K3s installation is indeed a one-command affair, but production edge cluster operations go far beyond installation. Network instability, resource constraints, device dispersion, image distribution — these are the real challenges. Only after solving all of these can your edge K3s cluster truly go to production.

References & Acknowledgments

The following resources were referenced during the writing of this article. Thanks to the original authors for their contributions:

K3s - Lightweight Kubernetes (Official Docs) — Rancher official, provided authoritative documentation on K3s architecture, installation, and features
Edge Computing in Cloud-Native Environments: From K3s to Edge Node Full-Stack Deployment — CSDN, provided comparison of K3s, KubeEdge, and OpenYurt with edge deployment practices
How to Configure and Tune K3s on Ubuntu 22.04 LTS for Edge Computing — Tencent Cloud Developer Community, provided hardware selection, system optimization, and K3s tuning parameters for edge
Lightweight K8s: K3s for Edge Computing Scenarios — CSDN, provided detailed K3s vs K8s feature comparison and resource usage data
2026 Edge AI Explosion: Edge-Side Model Inference + K3s Edge Deployment — 51CTO Blog, provided K3s deployment practices in Edge AI scenarios and containerized model deployment

Overview#

K3s Architecture: Why It Runs on 512MB#

Key Differences from K8s#

Single Binary Architecture#

Storage Backend Selection#

Single-Node Cluster Setup: Up and Running in 5 Minutes#

Environment Preparation#

One-Command Installation#

China Mirror Acceleration#

Multi-Node Cluster: Server + Agent Architecture#

Architecture Design#

Deploying the Server Node#

Deploying Agent Nodes#

Air-Gapped Installation: An Edge Scenario Necessity#

Network Solution Selection: Edge Scenario Trade-offs#

K3s Built-in Networking#

Custom Flannel CIDR#

Network Instability Issues#

Edge Application Deployment in Practice#

Deploying a Data Collection Service#

Rolling Update Strategy#

Image Pre-distribution#

Monitoring and Alerting: Observability for Edge Clusters#

Monitoring for Resource-Constrained Environments#

Key Alert Rules#

Troubleshooting: Common Edge Scenario Issues#

Issue 1: Agent Node Join Failure#

Issue 2: Pod Stuck in ContainerCreating#

Issue 3: Edge Node Frequent Disconnection#

Issue 4: SQLite Database Lock#

Multi-Cluster Management: Centrally Managing Edge Clusters from the Cloud#

Solution Comparison#

Performance Tuning: Squeezing Every Drop from Edge Devices#

Resource Allocation Strategy#

K3s Server Tuning#

Container Runtime Tuning#

Automated Disk Cleanup#

Security Hardening: Special Risks for Edge Devices#

1. K3s API Server Authentication#

2. Network Isolation#

3. Image Signature Verification#

Summary#

References & Acknowledgments#