Troubleshooting Path
kubectl get pods → check status
kubectl describe pod → check Events
kubectl logs → check logs
Common Pod States
| State | Meaning | Common Cause |
|---|---|---|
Pending | Not scheduled | Insufficient resources, scheduling constraints |
CrashLoopBackOff | Crash loop | App error, config issue |
ImagePullBackOff | Image pull failed | Image not found, auth failure |
OOMKilled | Out of memory | Memory limit too low |
CrashLoopBackOff
Most common issue. Troubleshooting steps:
# Check previous crash logs
kubectl logs <pod> --previous
# Check exit code
kubectl get pod <pod> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}'
Exit code meanings:
- 137: OOMKilled → increase
resources.limits.memory - 1: Application error → check app logs
- 126/127: Command not found or permission issue
ImagePullBackOff
kubectl describe pod <pod> | grep -A5 Events
Common causes: image name typo, registry auth needed, network issue.
Configure image pull secret:
spec:
imagePullSecrets:
- name: registry-secret
Pod Stuck in Pending
# Check scheduling failure reason
kubectl get pod <pod> -o jsonpath='{.status.conditions[?(@.type=="PodScheduled")].message}'
Common output:
Insufficient cpu→ CPU resources insufficienthad untolerated taint→ node taint not tolerateddidn't match node affinity→ node affinity mismatch
Network Issues
# Check if Service has Endpoints
kubectl get endpoints <svc>
# Check label matching
kubectl get pods --show-labels
kubectl get svc <svc> -o jsonpath='{.spec.selector}'
# DNS resolution test
kubectl exec -it <pod> -- nslookup kubernetes.default
Quick Reference
Pod won't start → describe pod → check Events
Pod crashing → logs --previous → check exit code
Service down → get endpoints → check labels
DNS failing → check CoreDNS
Scheduling fail → describe pod → check conditions
OOMKilled → describe pod → increase memory limit
Summary
Core of K8s troubleshooting: use kubectl describe effectively, understand what each abnormal state means, and narrow down the scope layer by layer.