Posts

Overview Scheduled tasks are a foundational component of operations automation — log rotation, data backups, certificate renewal, health checks, report generation — nearly every ops scenario relies on scheduled execution. Most people’s understanding of scheduled tasks stops at crontab -e plus a line like 0 2 * * * /path/to/script.sh, but this is far from sufficient in production: who gets notified when a task fails? Who handles execution timeouts? How do you coordinate tasks across multiple machines?...

The Hidden Costs of Oversized Images Many teams overlook image size during the early stages of containerization. A Spring Boot image often weighs 800MB–1.2GB, and even a Go application image can easily exceed 700MB. The problems caused by oversized images go far beyond “wasting a little disk space”: Slow pulls and deployment latency: In CI/CD pipelines or elastic scaling scenarios, nodes must pull the image before starting the container. A 1GB image takes over 80 seconds to download on a 100 Mbps internal network, while a 50MB image takes just 4 seconds....

Overview kubectl is the most commonly used tool for Kubernetes administrators, yet most people only use 20% of its capabilities. You type kubectl get pods -n production dozens of times a day without realizing a single alias can cut the keystrokes in half. When problems arise, you only know kubectl describe and kubectl logs, unaware that krew plugins can troubleshoot network, resource, and certificate issues with a single command. This article systematically covers the kubectl productivity toolchain — from aliases to plugins to interactive tools — to double your daily K8s operation efficiency....

CI/CD is the lifeblood of modern software delivery. Manual builds and deployments are not only inefficient but also breeding grounds for incidents — the “works on my machine” tragedy almost always stems from a lack of automated pipelines. As GitHub’s native CI/CD platform, GitHub Actions integrates seamlessly with code repositories, offers generous free tiers for open-source projects, and has become one of the most popular CI/CD tools. This article starts from core concepts and walks through two practical scenarios — a Go project and a Hugo site — to comprehensively explain pipeline design....

Why Choose Loki The traditional ELK (Elasticsearch + Logstash + Kibana) stack is powerful, but has two core pain points: High storage costs: Elasticsearch fully indexes log content, with index bloat reaching 3-5x the raw data Operational complexity: ES cluster scaling, shard rebalancing, and index lifecycle management are complex, making production clusters costly to maintain Loki, open-sourced by Grafana Labs, is designed around the philosophy of “doing for logs what Prometheus did for metrics....

In the Prometheus ecosystem, Prometheus generates alerts based on alerting rules, while Alertmanager manages the entire alert lifecycle: grouping, routing, inhibition, deduplication, and notification delivery. A poorly configured Alertmanager can drown on-call engineers in a flood of duplicate alerts at 3 AM, whereas a well-designed routing and inhibition strategy ensures that “the right person receives the right alert at the right time.” Reference: Prometheus Official Documentation — Alertmanager I. Alertmanager Architecture Alertmanager’s processing pipeline consists of five stages:...

Architecture Exporter → Prometheus (storage) → Grafana (visualization) ↓ Alertmanager (alert routing) Docker Compose Deployment version: '3.8' services: prometheus: image: prom/prometheus:v2.52.0 ports: ["9090:9090"] volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml - ./rules:/etc/prometheus/rules - prom_data:/prometheus command: - '--storage.tsdb.retention.time=30d' restart: unless-stopped grafana: image: grafana/grafana:10.4.2 ports: ["3000:3000"] volumes: [grafana_data:/var/lib/grafana] restart: unless-stopped alertmanager: image: prom/alertmanager:v0.27.0 ports: ["9093:9093"] volumes: [./alertmanager.yml:/etc/alertmanager/config.yml] restart: unless-stopped node-exporter: image: prom/node-exporter:v1.8.1 ports: ["9100:9100"] restart: unless-stopped volumes: prom_data: grafana_data: Core Configuration # prometheus.yml global: scrape_interval: 15s evaluation_interval: 15s rule_files: - "rules/*....

Overview Logs are the eyes of system operations. From kernel messages to application logs, from security auditing to performance analysis, logs run through every aspect of troubleshooting. Modern Linux uses journald as the system logging daemon, combined with logrotate for log rotation, forming a complete log management infrastructure. This article dives deep into journald internals and configuration, advanced journalctl query techniques, log rotation strategies, remote log collection solutions, and practical analysis cases....

systemd Architecture Overview systemd is the de facto standard init system for modern Linux distributions, having replaced SysVinit as the default init in most mainstream distributions since 2015. It is not merely a “service starter” but a complete system and service manager. Core Unit Types systemd manages system resources through units, with each unit type corresponding to a specific resource type: Unit Type Extension Purpose service .service System services (daemons) socket ....

Overview Many teams perceive Docker Compose as merely a “local development orchestration tool.” In reality, for small to medium production scenarios (single node or a handful of nodes), Compose remains a highly cost-effective solution. Its syntax is simple, the learning curve is low, and it doesn’t require a full K8s cluster operations team—yet it handles multi-service orchestration, dependency management, health checks, and resource limits effectively. This article skips basic Compose syntax and focuses on real production pain points: how to manage service dependencies without cascading startup failures, how to write reliable health checks, how to avoid hardcoding secrets in compose files, how to prevent logs from filling up disk space, and when to migrate from Compose to K8s....