K8s Networking Model: CNI and Service Networking

Four Core Requirements of the Kubernetes Networking Model

The Kubernetes networking model is built on four core requirements. Understanding them is the foundation for mastering K8s networking. According to the official Kubernetes networking model documentation, these four requirements form the cornerstone of cluster network communication.

1. Pod-to-Pod Communication

K8s requires that all Pods can communicate directly via IP without NAT (Network Address Translation). This means:

Each Pod has its own IP address
Pod-to-Pod communication uses real Pod IPs, without NAT translation
Regardless of which Node a Pod is scheduled on, the Pod-to-Pod network remains flat and reachable

This is the most fundamental design decision in the K8s networking model. In traditional data center networking, cross-host container communication typically relies on port mapping or NAT, whereas K8s chose a flat network model, making each Pod an equal first-class citizen in the network.

2. Node-to-Pod Communication

Processes on a Node (including kubelet and kube-proxy) must be able to communicate directly with any Pod on that Node, again without NAT. This requirement ensures:

kubelet can perform health checks (liveness/readiness probes)
Monitoring agents on the node can directly scrape Pod metrics
Host network processes can interoperate with the Pod network

3. Service Network

A Service provides a stable virtual IP (ClusterIP) that load-balances traffic to backend Pods. The Service network is a virtual address space separate from the Pod network (default 10.96.0.0/12), implemented through iptables/ipvs rules maintained by kube-proxy.

4. NetworkPolicy

K8s has a built-in network policy mechanism that allows administrators to define network access rules between Pods, enabling microsegmentation. By default, K8s allows all inter-Pod traffic; NetworkPolicy provides fine-grained whitelist-based control.

CNI Plugin Comparison

CNI (Container Network Interface) is a container networking standard maintained by the CNCF. K8s implements Pod networking through CNI plugins. Below is a comparison of three mainstream CNI plugins.

Architecture Comparison

Feature	Calico	Flannel	Cilium
Data plane	iptables / eBPF	VXLAN / Host-GW	eBPF
Network policy	Full support	Not supported	Full support + L7
Performance	High	Medium	Very high
BGP support	Native	Not supported	Supported
Observability	Moderate	Weak	Strong (Hubble)
Use case	Mid-to-large production	Small scale / getting started	Large scale / high performance

Flannel: The Simplest Option

Flannel, developed by CoreOS, is one of the earliest K8s CNI plugins. Its design philosophy is simplicity and reliability:

# flannel configuration example (kube-flannel.yaml)
apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-flannel-cfg
  namespace: kube-flannel
data:
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan",
        "DirectRouting": true
      }
    }

Flannel supports two backend modes: VXLAN and Host-GW. VXLAN encapsulates cross-host traffic via UDP, offering good compatibility but slightly lower performance. Host-GW directly modifies host routing tables, providing better performance but requiring L2 network connectivity.

Calico: The Production Favorite

Calico is one of the most widely used CNI plugins in production environments. Its core strengths lie in BGP routing + powerful network policies:

# Calico IPPool configuration
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: default-ipv4-ippool
spec:
  cidr: 10.244.0.0/16
  ipipMode: CrossSubnet    # Use IPIP tunnel for cross-subnet traffic
  vxlanMode: Never          # Do not use VXLAN
  natOutgoing: true
  nodeSelector: all()

Calico uses BGP to exchange routing information between Nodes, with each Node acting as a BGP router. This design eliminates encapsulation overhead, delivering performance close to native networking. Calico also offers richer policy capabilities than K8s native NetworkPolicy (GlobalNetworkPolicy, namespace isolation, etc.).

Cilium: The eBPF-Powered Future

Cilium leverages eBPF technology to process network packets in kernel space, avoiding the overhead of iptables rule traversal:

# Install Cilium with kube-proxy replacement enabled
helm install cilium cilium/cilium \
  --namespace kube-system \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=10.0.0.1 \
  --set k8sServicePort=6443 \
  --set hubble.enabled=true \
  --set hubble.relay.enabled=true

Cilium’s eBPF data plane offers two key advantages:

Performance: eBPF handles load balancing at the kernel socket layer, bypassing iptables rule chains entirely
Observability: The Hubble component provides real-time flow maps and L7 protocol visibility

Selection Recommendations

Small clusters / learning environments: Flannel—simple configuration, low resource footprint
Mid-to-large production: Calico—BGP routing + mature policy ecosystem
High performance / large scale: Cilium—eBPF data plane + Hubble observability

Service Types Explained

A Service is K8s’ abstraction for exposing applications, routing traffic to backend Pods via Label Selectors. According to the K8s Service documentation, Services come in four types.

ClusterIP (Default)

ClusterIP exposes the service within the cluster, assigning a virtual IP accessible only from inside the cluster:

apiVersion: v1
kind: Service
metadata:
  name: web-app
spec:
  type: ClusterIP          # Default type
  selector:
    app: web-app
  ports:
  - port: 80               # Service port
    targetPort: 8080       # Pod port
    protocol: TCP

NodePort

NodePort opens a fixed port on every Node (default range 30000-32767), making the service accessible externally via NodeIP:NodePort:

apiVersion: v1
kind: Service
metadata:
  name: web-app-nodeport
spec:
  type: NodePort
  selector:
    app: web-app
  ports:
  - port: 80
    targetPort: 8080
    nodePort: 30080        # Specify port, or leave unset for random assignment

LoadBalancer

The LoadBalancer type relies on cloud provider LB services to automatically provision an external load balancer:

apiVersion: v1
kind: Service
metadata:
  name: web-app-lb
spec:
  type: LoadBalancer
  selector:
    app: web-app
  ports:
  - port: 80
    targetPort: 8080
  externalTrafficPolicy: Local  # Preserve client source IP

In non-cloud environments, MetalLB can be used to provide bare-metal LoadBalancer support.

Headless Service

A Headless Service doesn’t allocate a ClusterIP. DNS queries return Pod IP lists directly, which is useful for StatefulSets:

apiVersion: v1
kind: Service
metadata:
  name: mysql-headless
spec:
  clusterIP: None          # Headless marker
  selector:
    app: mysql
  ports:
  - port: 3306

The core value of a Headless Service is direct Pod addressing—each Pod gets its own DNS name (pod-name.service-name.namespace.svc.cluster.local), which is the foundation of StatefulSet stable network identity.

Ingress Controllers

Ingress provides HTTP/HTTPS L7 routing capabilities and is the core component for external traffic entry into the cluster.

NGINX Ingress Controller

NGINX Ingress is the most widely used Ingress controller, built on the NGINX reverse proxy:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "100m"
    nginx.ingress.kubernetes.io/rate-limit: "100"
spec:
  ingressClassName: nginx
  tls:
  - hosts: ["api.sre.wang"]
    secretName: tls-secret
  rules:
  - host: api.sre.wang
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-app
            port:
              number: 80

Traefik

Traefik is a cloud-native Ingress controller that supports automatic service discovery and dynamic configuration:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: web-ingress
spec:
  entryPoints:
  - websecure
  routes:
  - match: Host(`api.sre.wang`)
    kind: Rule
    services:
    - name: web-app
      port: 80
  tls:
    certResolver: letsencrypt

Selection Comparison

Feature	NGINX Ingress	Traefik
Performance	High (NGINX in C)	Medium-high (Go)
Configuration	Annotations	CRD (more flexible)
Dynamic reload	Requires reload	Hot reload
Middleware	Annotation config	CRD definitions
Community	Largest	Active
Use case	Traditional HTTP services	Cloud-native / microservices

NetworkPolicy

By default, all inter-Pod network traffic in K8s is open. NetworkPolicy provides Label-based fine-grained access control.

Namespace Isolation Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production
spec:
  podSelector: {}            # Match all Pods in the namespace
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: ingress-nginx  # Only allow access from ingress-nginx namespace

Application-Level Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-access-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend            # Policy target: backend Pods
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend       # Only allow frontend Pods to access
    - podSelector:
        matchLabels:
          app: monitoring      # Allow monitoring components to access
    ports:
    - protocol: TCP
      port: 8080

Key Considerations

NetworkPolicy uses a whitelist model: once a Pod is selected by a policy, only explicitly allowed traffic can pass
Policies are additive: when multiple policies select the same Pod, their allowed traffic is unioned
NetworkPolicy requires a CNI plugin that supports it—Flannel does not support it by default

Summary

K8s networking is a layered system: CNI handles Pod network connectivity, Service provides service discovery and load balancing, Ingress manages L7 traffic entry, and NetworkPolicy implements security isolation. Understanding each layer’s responsibilities and selection criteria is the foundation for building reliable container networks.

Recommended production combination: Calico or Cilium (CNI) + NGINX Ingress (L7 entry) + NetworkPolicy (security isolation). For teams pursuing ultimate performance and observability, Cilium + Hubble is currently the best choice.

References & Acknowledgments

This article referenced the following materials during writing. We thank the original authors for their contributions:

official Kubernetes networking model documentation — Kubernetes Official, referenced for official Kubernetes networking model documentation
Calico — Docs, referenced for Calico
Cilium — Docs, referenced for Cilium
K8s Service documentation — Kubernetes Official, referenced for K8s Service documentation
MetalLB — Metallb, referenced for MetalLB
NGINX Ingress — Kubernetes SIGs, referenced for NGINX Ingress
Traefik — Doc, referenced for Traefik

Four Core Requirements of the Kubernetes Networking Model#

1. Pod-to-Pod Communication#

2. Node-to-Pod Communication#

3. Service Network#

4. NetworkPolicy#

CNI Plugin Comparison#

Architecture Comparison#

Flannel: The Simplest Option#

Calico: The Production Favorite#

Cilium: The eBPF-Powered Future#

Selection Recommendations#

Service Types Explained#

ClusterIP (Default)#

NodePort#

LoadBalancer#

Headless Service#

Ingress Controllers#

NGINX Ingress Controller#

Traefik#

Selection Comparison#

NetworkPolicy#

Namespace Isolation Policy#

Application-Level Policy#

Key Considerations#

Summary#

References & Acknowledgments#