Grafana Dashboard Best Practices

Grafana is the most popular visualization platform in the cloud-native era, but there’s a world of difference between “functional” and “effective.” A cluttered dashboard leaves on-call engineers lost in a sea of panels, while a well-designed one conveys system health in 5 seconds. This article starts from design principles, covers the variable system, panel selection, and alerting integration, and ties everything together with a complete SLO dashboard. Reference: Grafana Official Documentation...

April 17, 2024 · 8 mins · 1646 words · XuBaojin

Prometheus PromQL: From Beginner to Expert

PromQL (Prometheus Query Language) is the query language of the Prometheus monitoring system and the core of the cloud-native monitoring stack. Whether you’re building Grafana dashboards, writing alerting rules, or running ad-hoc queries during incident troubleshooting, PromQL is indispensable. This article starts from the data model and progressively covers aggregation operations, common functions, practical queries, and advanced techniques like subqueries. Reference: Prometheus Official Documentation — Querying basics I. PromQL Data Model PromQL has four fundamental data types....

April 4, 2024 · 7 mins · 1294 words · XuBaojin

Alertmanager Alert Routing and Silencing Strategies

In the Prometheus ecosystem, Prometheus generates alerts based on alerting rules, while Alertmanager manages the entire alert lifecycle: grouping, routing, inhibition, deduplication, and notification delivery. A poorly configured Alertmanager can drown on-call engineers in a flood of duplicate alerts at 3 AM, whereas a well-designed routing and inhibition strategy ensures that “the right person receives the right alert at the right time.” Reference: Prometheus Official Documentation — Alertmanager I. Alertmanager Architecture Alertmanager’s processing pipeline consists of five stages:...

February 27, 2024 · 9 mins · 1867 words · XuBaojin

Zabbix vs Prometheus: Monitoring System Selection Guide

Overview In the monitoring system space, Zabbix and Prometheus are the two giants. Zabbix comes from the traditional operations era, dominating physical machine/VM environments for nearly 25 years; Prometheus rose in the cloud-native era, becoming the de facto standard for the Kubernetes ecosystem. Many teams face a question when choosing a monitoring system: Zabbix or Prometheus? The answer isn’t either/or. Many mature teams run both systems simultaneously in production — Zabbix handles the infrastructure layer (network, hardware, OS), while Prometheus handles the application and cloud-native layers....

February 5, 2024 · 15 mins · 3041 words · XuBaojin