Kubernetes Disaster Recovery: From etcd Backup to Velero

Overview

“Hope for the best, prepare for the worst.” In Kubernetes production environments, disaster recovery is the last line of defense. Whether it’s etcd corruption, accidental deletion, node failure, or even entire cluster loss, having a solid backup and recovery strategy is critical.

This article systematically covers K8s disaster recovery—from etcd backup and restore, Velero full-cluster backup, PV data backup, to cross-cluster recovery and disaster recovery architecture design.

Based on Kubernetes v1.30 and Velero v1.14. Reference: Kubernetes etcd Backup, Velero Documentation

Disaster Recovery Fundamentals

RPO and RTO

Metric	Full Name	Description	Example
RPO	Recovery Point Objective	Maximum acceptable data loss	RPO=15min means at most 15min of data loss
RTO	Recovery Time Objective	Maximum acceptable downtime	RTO=4h means recovery within 4 hours
RTO/RPO	-	DR strategy targets	Low values = high cost

DR Strategy Levels

Level	RPO	RTO	Description
Level 1	24h	24h	Daily backup, manual recovery
Level 2	1h	4h	Hourly backup, semi-automated recovery
Level 3	15min	1h	Continuous backup, automated recovery
Level 4	0	0	Active-active, no data loss, instant switch

Recovery Scope

Scope	Description	Method
Single resource	Accidental deletion	etcd rollback or Velero restore
Multiple resources	Bulk misconfiguration	Velero restore
Node failure	Node unavailable	Reschedule + data recovery
etcd corruption	Cluster state lost	etcd restore
Cluster loss	Entire cluster down	Cross-cluster recovery

etcd Backup and Restore

Why Back Up etcd

etcd is the sole persistent store for K8s cluster state—everything (Pods, Services, Deployments, Secrets, ConfigMaps) lives in etcd. If etcd is lost and there’s no backup, the entire cluster is gone.

etcd Backup

# Method 1: etcdctl snapshot (recommended)
export ETCDCTL_API=3
etcdctl snapshot save /backup/etcd-snapshot-$(date +%Y%m%d-%H%M%S).db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

# Verify snapshot
etcdctl snapshot status /backup/etcd-snapshot-20260101-120000.db \
  --write-out=table

# Method 2: Copy etcd data directory
# Stop etcd first
systemctl stop etcd

# Copy data directory
cp -r /var/lib/etcd /backup/etcd-data-$(date +%Y%m%d)

# Restart etcd
systemctl start etcd

# Method 3: Automated backup script
cat > /usr/local/bin/etcd-backup.sh << 'EOF'
#!/bin/bash
set -euo pipefail

BACKUP_DIR="/backup/etcd"
RETENTION_DAYS=7
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
BACKUP_FILE="${BACKUP_DIR}/etcd-snapshot-${TIMESTAMP}.db"

export ETCDCTL_API=3

# Create backup directory
mkdir -p "${BACKUP_DIR}"

# Take snapshot
etcdctl snapshot save "${BACKUP_FILE}" \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

# Verify
etcdctl snapshot status "${BACKUP_FILE}" --write-out=table

# Clean up old backups
find "${BACKUP_DIR}" -name "etcd-snapshot-*.db" -mtime +${RETENTION_DAYS} -delete

# Upload to remote storage (e.g., S3)
# aws s3 cp "${BACKUP_FILE}" "s3://k8s-etcd-backup/$(date +%Y/%m/%d)/"

echo "Backup completed: ${BACKUP_FILE}"
EOF

chmod +x /usr/local/bin/etcd-backup.sh

# Schedule with cron
echo "0 */6 * * * root /usr/local/bin/etcd-backup.sh >> /var/log/etcd-backup.log 2>&1" > /etc/cron.d/etcd-backup

etcd Restore

# 1. Stop all control plane components
systemctl stop kube-apiserver
systemctl stop kube-controller-manager
systemctl stop kube-scheduler
systemctl stop etcd

# 2. Backup current etcd data (safety measure)
mv /var/lib/etcd /var/lib/etcd.broken

# 3. Restore from snapshot
export ETCDCTL_API=3
etcdctl snapshot restore /backup/etcd-snapshot-20260101-120000.db \
  --data-dir=/var/lib/etcd \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

# 4. Fix permissions
chown -R etcd:etcd /var/lib/etcd

# 5. Restart etcd
systemctl start etcd

# 6. Verify etcd is healthy
etcdctl endpoint health \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

# 7. Restart control plane components
systemctl start kube-apiserver
systemctl start kube-controller-manager
systemctl start kube-scheduler

# 8. Verify cluster
kubectl get nodes
kubectl get pods -A

HA etcd Cluster Recovery

For multi-node etcd clusters, recovery depends on how many nodes failed:

Scenario	Data Status	Recovery Method
1 node failed	Quorum intact	Remove failed member, rejoin
Multiple nodes, quorum lost	May be inconsistent	Restore from backup on all members
All nodes failed	Restore from backup	Full restore from backup

# Recover single failed member
# On healthy member:
etcdctl member list \
  --endpoints=https://healthy-node:2379 \
  --cacert=... --cert=... --key=...

# Remove failed member
etcdctl member remove <failed-member-id> \
  --endpoints=https://healthy-node:2379 \
  --cacert=... --cert=... --key=...

# Rejoin fixed node
systemctl stop etcd
rm -rf /var/lib/etcd
etcdctl member add node-3 \
  --peer-urls=https://node-3:2380 \
  --endpoints=https://healthy-node:2379 \
  --cacert=... --cert=... --key=...

# Update etcd config to use initial-existing flag
# Start etcd with --initial-cluster-state=existing
systemctl start etcd

Velero Overview

What Is Velero

Velero (formerly Heptio Ark) is an open-source K8s backup and migration tool. Unlike etcd snapshots (which back up everything), Velero provides granular resource-level backup and restore:

Feature	etcd Snapshot	Velero
Backup scope	Entire etcd	Selected namespaces/resources
Resource filtering	No	Yes (labels, selectors)
PV data backup	No	Yes (via snapshots or restic)
Cross-cluster migration	No	Yes
Scheduled backups	Manual	Yes (Schedule CRD)
Recovery granularity	All or nothing	Per-resource
Version compatibility	Strict	Flexible

Velero Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Velero Architecture                      │
│                                                              │
│   ┌──────────────┐    ┌──────────────┐    ┌──────────────┐  │
│   │   Backup     │    │   Schedule   │    │   Restore    │  │
│   │   (CRD)      │    │   (CRD)      │    │   (CRD)      │  │
│   └──────┬───────┘    └──────┬───────┘    └──────┬───────┘  │
│          │                    │                    │          │
│          ▼                    ▼                    ▼          │
│   ┌─────────────────────────────────────────────────────┐    │
│   │                   Velero Controller                   │    │
│   │   ┌─────────────────┐  ┌─────────────────────────┐  │    │
│   │   │  Backup/Restore │  │  Storage Location        │  │    │
│   │   │  Controller     │  │  Controller              │  │    │
│   │   └─────────────────┘  └─────────────────────────┘  │    │
│   └─────────────────────────────────────────────────────┘    │
│          │                    │                               │
│          ▼                    ▼                               │
│   ┌──────────────┐    ┌──────────────┐                       │
│   │   Object     │    │   Volume     │                       │
│   │   Storage    │    │   Snapshot   │                       │
│   │   (S3/MinIO) │    │   (CSI/     │                       │
│   │               │    │    restic)   │                       │
│   └──────────────┘    └──────────────┘                       │
└─────────────────────────────────────────────────────────────┘

Installation

# Install Velero CLI
wget https://github.com/vmware-tanzu/velero/releases/download/v1.14.0/velero-v1.14.0-linux-amd64.tar.gz
tar -xzf velero-v1.14.0-linux-amd64.tar.gz
sudo mv velero-v1.14.0-linux-amd64/velero /usr/local/bin/

# Create credentials file
cat > /root/velero-credentials << 'EOF'
[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
EOF

# Install Velero with S3 backend
velero install \
  --provider aws \
  --bucket k8s-velero-backup \
  --backup-location-config region=us-east-1,s3ForcePathStyle=true,s3Url=https://minio.example.com \
  --snapshot-location-config region=us-east-1 \
  --secret-file /root/velero-credentials \
  --use-volume-snapshots=true \
  --use-restic \
  --default-volumes-to-restic \
  --namespace velero

# Verify installation
kubectl get pods -n velero
kubectl get backupstoragelocation -n velero

Velero Backup

On-demand Backup

# Full cluster backup
velero backup create full-cluster-backup-20260101

# Backup specific namespace
velero backup create production-backup --include-namespaces production

# Backup with label selector
velero backup create frontend-backup \
  --selector app=frontend

# Backup specific resource types
velero backup create deployments-only \
  --include-resources deployments,services

# Backup with exclude rules
velero backup create exclude-system \
  --exclude-namespaces kube-system,velero \
  --exclude-resources secrets,configmaps

Scheduled Backups

# Daily full cluster backup, retain 7 copies
velero schedule create daily-full-backup \
  --schedule="0 2 * * *" \
  --include-namespaces production,staging \
  --ttl 168h

# Hourly namespace backup
velero schedule create hourly-prod-backup \
  --schedule="@every 1h" \
  --include-namespaces production \
  --ttl 24h

# List schedules
velero schedule get

# Pause/resume schedule
velero schedule pause daily-full-backup
velero schedule unpause daily-full-backup

Backup Hooks

# Pre/post backup hooks (e.g., flush database)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: production
  annotations:
    pre.hook.backup.velero.io/container: postgres
    pre.hook.backup.velero.io/command: '["/bin/bash", "-c", "pg_dump -U postgres mydb > /tmp/dump.sql"]'
    post.hook.backup.velero.io/container: postgres
    post.hook.backup.velero.io/command: '["/bin/bash", "-c", "rm /tmp/dump.sql"]'
spec:
  template:
    spec:
      containers:
      - name: postgres
        image: postgres:16

View Backups

# List all backups
velero backup get

# View backup details
velero backup describe daily-full-backup-20260101 --details

# View backup logs
velero backup logs daily-full-backup-20260101

Velero Restore

On-demand Restore

# Full restore from backup
velero restore create --from-backup daily-full-backup-20260101

# Restore specific namespaces
velero restore create --from-backup daily-full-backup-20260101 \
  --include-namespaces production

# Restore with new namespace name
velero restore create --from-backup daily-full-backup-20260101 \
  --namespace-mappings production:production-restored

# Restore specific resources
velero restore create --from-backup daily-full-backup-20260101 \
  --include-resources deployments,services,configmaps \
  --include-namespaces production

# Restore with label selector
velero restore create --from-backup daily-full-backup-20260101 \
  --selector app=critical

Restore Strategy

# Dry run (don't actually restore, just show what would be restored)
velero restore create --from-backup daily-full-backup-20260101 \
  --dry-run \
  -o yaml > restore-plan.yaml

# Review restore plan
cat restore-plan.yaml

# Actual restore
velero restore create --from-backup daily-full-backup-20260101

# Verify
velero restore get
velero restore describe daily-full-backup-20260101-202601011500 --details

Restore to Different Cluster

# On target cluster, configure the same backup storage
velero install \
  --provider aws \
  --bucket k8s-velero-backup \
  --backup-location-config region=us-east-1,s3Url=https://minio.example.com \
  --secret-file /root/velero-credentials \
  --use-volume-snapshots=false \
  --namespace velero

# Set source backup location
velero backup-location create shared-backup \
  --provider aws \
  --bucket k8s-velero-backup \
  --config region=us-east-1,s3Url=https://minio.example.com \
  --default

# List backups from source cluster
velero backup get

# Restore
velero restore create --from-backup daily-full-backup-20260101

PV Data Backup

Volume Snapshot Method

# Requires CSI driver with snapshot support
# Check if VolumeSnapshotClass is available
kubectl get volumesnapshotclass

# Create snapshot during backup
velero backup create snapshot-backup \
  --include-namespaces production \
  --snapshot-volumes=true

# VolumeSnapshotClass example
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: csi-hostpath-snapclass
driver: hostpath.csi.k8s.io
deletionPolicy: Retain

Restic Backup

For CSI drivers that don’t support snapshots, Velero uses restic for file-level backup:

# Install Velero with restic support
velero install \
  --provider aws \
  --bucket k8s-velero-backup \
  --backup-location-config region=us-east-1 \
  --secret-file /root/velero-credentials \
  --use-restic \
  --default-volumes-to-restic    # Default all volumes to restic backup

# Annotate Pod to specify restic backup volumes
apiVersion: v1
kind: Pod
metadata:
  name: app-with-data
  namespace: production
  annotations:
    backup.velero.io/backup-volumes: data,config    # Comma-separated volume names
spec:
  containers:
  - name: app
    image: myapp:v1
    volumeMounts:
    - name: data
      mountPath: /data
    - name: config
      mountPath: /config
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: app-data
  - name: config
    persistentVolumeClaim:
      claimName: app-config

Backup Method Comparison

Feature	Volume Snapshot	Restic
Backup method	Block-level snapshot	File-level copy
CSI dependency	Requires snapshot support	None
Backup speed	Fast (seconds)	Slow (depends on data size)
Cross-provider	No	Yes
Deduplication	No	Yes
Incremental	Depends on CSI	Yes
Resource overhead	Low	Medium

Cross-Cluster Recovery

Scenario: Cluster Migration

Source cluster: Cluster-A (us-east-1)
Target cluster: Cluster-B (eu-west-1)

1. Backup on Cluster-A → Upload to shared S3
2. Configure Cluster-B to read from same S3
3. Restore on Cluster-B
4. Verify applications
5. Switch traffic (DNS or load balancer)

Scenario: Disaster Failover

Primary cluster: Cluster-A (us-east-1)
DR cluster: Cluster-B (us-west-2)

Normal operation:
  - Application runs on Cluster-A
  - Velero scheduled backup → S3 (cross-region replication)
  - DR cluster standby

Disaster failover:
  1. Cluster-A fails
  2. Cluster-B restores latest backup from S3
  3. Traffic switches to Cluster-B
  4. Cluster-A rebuilt (new cluster, restore from S3)

Cross-Cluster Recovery Steps

# On DR cluster (Cluster-B):

# 1. Install Velero, configure same backup storage
velero install \
  --provider aws \
  --bucket k8s-velero-backup \
  --backup-location-config region=us-west-2 \
  --secret-file /root/velero-credentials \
  --use-restic

# 2. Sync backups from source
velero backup-location create primary-backup \
  --provider aws \
  --bucket k8s-velero-backup \
  --config region=us-east-1 \
  --default

# 3. List backups
velero backup get

# 4. Restore
velero restore create --from-backup daily-full-backup-20260101

# 5. Verify
kubectl get pods -A
kubectl get svc -A

Recovery Drills

Why Drill

A backup that hasn’t been tested in recovery is not a backup—it’s a hope. Regular recovery drills:

Verify backup integrity
Practice recovery procedures
Measure actual RTO
Identify gaps
Build team confidence

Drill Plan

Drill Type	Frequency	Scope	Success Criteria
etcd restore	Quarterly	Test cluster	Cluster recovers, all Pods running
Velero restore	Monthly	Test cluster	Resources restored, data intact
Cross-cluster failover	Semi-annually	DR cluster	Application accessible on DR
Full disaster simulation	Annually	All clusters	RTO/RPO met

Drill Execution

# etcd Recovery Drill

# 1. Record current state
kubectl get pods -A -o wide > /tmp/before-drill-pods.txt
kubectl get deployments -A > /tmp/before-drill-deployments.txt

# 2. Simulate disaster (on test cluster only!)
# Delete a namespace
kubectl delete namespace test-app

# 3. Restore from backup
velero restore create --from-backup latest-backup \
  --include-namespaces test-app

# 4. Verify recovery
kubectl get pods -n test-app
diff <(kubectl get pods -A -o wide) /tmp/before-drill-pods.txt

# 5. Record results
echo "Drill completed at $(date)" >> /tmp/drill-log.txt
echo "RTO: <measured time>" >> /tmp/drill-log.txt

Drill Checklist

□ Backup files exist and are accessible
□ Recovery procedure documented and up to date
□ Recovery team knows the procedure
□ Test environment available
□ Recovery completed within RTO
□ All critical resources restored
□ Application functional post-recovery
□ Data loss within RPO
□ Recovery logs captured
□ Improvement actions identified

DR Architecture Design

Single Cluster with Backup

┌──────────────────────────────────────────┐
│              Single Cluster               │
│                                          │
│  ┌────────────┐  ┌────────────────────┐ │
│  │ K8s Cluster │  │ Velero Backup      │ │
│  │             │──│ → S3 Storage       │ │
│  │ Applications │  │ → Daily snapshots  │ │
│  │             │  │                   │ │
│  └────────────┘  └────────────────────┘ │
└──────────────────────────────────────────┘

RPO: 24h (daily backup)
RTO: 4-8h (manual recovery)
Cost: Low
Suitable: Development, testing

Active-Passive (Primary-DR)

┌─────────────────────────────────────────────────────────────┐
│                  Active-Passive Architecture                   │
│                                                              │
│  ┌──────────────────┐          ┌──────────────────┐        │
│  │  Primary Cluster  │          │   DR Cluster      │        │
│  │  (us-east-1)      │          │   (us-west-2)     │        │
│  │                   │          │                   │        │
│  │  ┌─────────────┐  │          │  ┌─────────────┐  │        │
│  │  │ Applications│  │          │  │  Standby     │  │        │
│  │  │ (Running)   │  │          │  │  (Idle)      │  │        │
│  │  └─────────────┘  │          │  └─────────────┘  │        │
│  └────────┬──────────┘          └──────────────────┘        │
│           │                    ▲                              │
│           ▼                    │                              │
│  ┌──────────────────────────────────────────┐               │
│  │      Cross-region S3 Replication         │               │
│  │      (Velero backup sync)                │               │
│  └──────────────────────────────────────────┘               │
└─────────────────────────────────────────────────────────────┘

RPO: 1h (hourly backup + replication)
RTO: 1-2h (automated restore)
Cost: Medium (DR cluster runs minimal)
Suitable: Production

Active-Active

┌─────────────────────────────────────────────────────────────┐
│                  Active-Active Architecture                    │
│                                                              │
│  ┌──────────────────┐          ┌──────────────────┐        │
│  │  Cluster A       │          │   Cluster B       │        │
│  │  (us-east-1)      │          │   (eu-west-1)      │        │
│  │                   │          │                   │        │
│  │  ┌─────────────┐  │          │  ┌─────────────┐  │        │
│  │  │ Applications│  │          │  │ Applications│  │        │
│  │  │ (Running)   │  │          │  │ (Running)   │  │        │
│  │  └─────────────┘  │          │  └─────────────┘  │        │
│  └────────┬──────────┘          └────────┬──────────┘        │
│           │                              │                    │
│           ▼                              ▼                    │
│  ┌─────────────────────────────────────────────────────┐    │
│  │            Global Load Balancer                     │    │
│  │    (DNS / Traffic Manager / Service Mesh)          │    │
│  └─────────────────────────────────────────────────────┘    │
│                                                              │
│  ┌─────────────────────────────────────────────────────┐    │
│  │         Shared State Replication                    │    │
│  │    (Database replication / Object storage sync)     │    │
│  └─────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────┘

RPO: 0 (real-time replication)
RTO: 0 (instant switch)
Cost: High (both clusters fully active)
Suitable: Mission-critical

Best Practices

Backup Strategy

Multiple backup types: etcd snapshots (full cluster) + Velero (granular) + application-level backups (database dumps).
Backup frequency matches RPO: If RPO=15min, schedule 15min backups.
Backup retention: Keep at least 7 daily, 4 weekly, 12 monthly backups.
Offsite backup: Store backups in a different region or cloud.
Backup verification: Regularly verify backup integrity.

Recovery Strategy

Documented runbooks: Every recovery procedure should have a step-by-step runbook.
Regular drills: Test recovery quarterly at minimum.
Automated recovery: Use scripts or tools to reduce human error.
Team training: Ensure multiple team members can perform recovery.
Post-incident review: After any real recovery, document lessons learned.

etcd Specific Best Practices

Practice	Description
Regular snapshots	At least every 6 hours for production
Multiple backup locations	Local + remote (S3)
Verify snapshots	Use `etcdctl snapshot status`
Test restore	Quarterly on test environment
Monitor etcd health	Disk space, memory, latency
Compact and defrag	Periodically compact and defrag etcd

Velero Best Practices

Practice	Description
Use schedules	Automate backups with Schedule CRD
Filter resources	Exclude kube-system if not needed
Use hooks	Pre/post hooks for database dumps
Test restores	Monthly restore drills
Monitor backups	Check backup status daily
Restic for non-CSI	Use restic for volumes without snapshot support

Common Issues

Backup Failure

# View backup errors
velero backup describe <backup-name> --details

# Common issues:
# 1. Object storage unreachable
#    Check credentials and endpoint
# 2. Volume snapshot timeout
#    Check CSI driver and storage class
# 3. Restic repository lock
#    Run: velero restic repo unlock
# 4. Resource too large
#    Split backup by namespace

Restore Failure

# View restore errors
velero restore describe <restore-name> --details

# Common issues:
# 1. Resource already exists
#    Use --existing-resource-policy=update
# 2. Namespace doesn't exist
#    Create namespace first or use --namespace-mappings
# 3. PV binding fails
#    Check storage class and PV availability
# 4. Restic restore slow
#    Check network and storage performance

etcd Restore Issues

# etcd won't start after restore
# Check:
# 1. File permissions
chown -R etcd:etcd /var/lib/etcd
# 2. Certificate validity
openssl x509 -in /etc/kubernetes/pki/etcd/server.crt -text -noout | grep -A2 Validity
# 3. Data directory conflicts
rm -rf /var/lib/etcd.broken
# 4. etcd config file
# Check --data-dir and --initial-cluster flags

Summary

K8s disaster recovery is the last line of defense—hope for the best, prepare for the worst. Key takeaways:

etcd is the most critical backup: All cluster state lives in etcd; without etcd backup, there’s no recovery. Schedule regular snapshots and store them offsite.
Velero for granular recovery: Velero enables resource-level backup and restore, more suitable for daily ops than etcd snapshots.
PV data needs special attention: etcd and Velero back up K8s resource objects; PV data requires volume snapshots or restic.
Cross-cluster recovery requires planning: Backup storage must be accessible from both clusters; test cross-cluster restore.
Untested backups are not backups: Regular recovery drills are critical—verify backup integrity and practice recovery procedures.
Match DR architecture to requirements: Choose single-cluster with backup, active-passive, or active-active based on RPO/RTO requirements and budget.
Document and automate: Every recovery procedure should be documented, automated where possible, and team-trained.

Disaster recovery is not a one-time project—it’s an ongoing practice. Backups, drills, and documentation must be kept up to date as the cluster evolves.

References & Acknowledgments

This article referenced the following materials during writing. We thank the original authors for their contributions:

Kubernetes etcd Backup — Kubernetes Official, referenced for Kubernetes etcd Backup
Kubernetes 灾难恢复文档 — Kubernetes Official, referenced for Kubernetes 灾难恢复文档
Velero Documentation — Velero Project, referenced for Velero Documentation

Overview#

Disaster Recovery Fundamentals#

RPO and RTO#

DR Strategy Levels#

Recovery Scope#

etcd Backup and Restore#

Why Back Up etcd#

etcd Backup#

etcd Restore#

HA etcd Cluster Recovery#

Velero Overview#

What Is Velero#

Velero Architecture#

Installation#

Velero Backup#

On-demand Backup#

Scheduled Backups#

Backup Hooks#

View Backups#

Velero Restore#

On-demand Restore#

Restore Strategy#

Restore to Different Cluster#

PV Data Backup#

Volume Snapshot Method#

Restic Backup#

Backup Method Comparison#

Cross-Cluster Recovery#

Scenario: Cluster Migration#

Scenario: Disaster Failover#

Cross-Cluster Recovery Steps#

Recovery Drills#

Why Drill#

Drill Plan#

Drill Execution#

Drill Checklist#

DR Architecture Design#

Single Cluster with Backup#

Active-Passive (Primary-DR)#

Active-Active#

Best Practices#

Backup Strategy#

Recovery Strategy#

etcd Specific Best Practices#

Velero Best Practices#

Common Issues#

Backup Failure#

Restore Failure#

etcd Restore Issues#

Summary#

References & Acknowledgments#