SRE Fundamentals

Overview Many teams treat SRE as just “operations with a new name” — hire a few people who can write scripts, change their titles, and call it a transformation. This mindset ignores a fundamental truth: SRE is an engineering methodology, not a toolchain. When Google created the SRE function in 2003, the core idea was “treat operations problems with software engineering methods,” which fundamentally changed the positioning, workflow, and culture of operations....

Overview Performance issues are among the most common scenarios every SRE encounters: users report “it’s slow,” alerts say “P99 latency exceeds threshold,” monitoring shows “CPU is almost maxed.” But many teams handle performance issues with a “tune wherever it’s high” approach — add machines when CPU is high, add indexes when SQL is slow, add cache when latency is high. This symptomatic treatment may work short-term, but over time it makes the system increasingly complex, costs keep rising, and problems become harder to troubleshoot....

Overview The Google SRE Book contains a frequently cited principle: SRE teams should spend no more than 50% of their total work time on toil. This principle seems simple, but in practice, many SRE teams’ toil ratio far exceeds 50% — some even reach 80% or more. Why should SRE take “toil” so seriously? Because toil is the invisible killer of reliability: Toil consumes enormous amounts of time, leaving engineers no energy for work that genuinely improves reliability Toil typically involves manual operations that are error-prone, actually introducing new incidents Toil leads to burnout and attrition of talented engineers Toil doesn’t scale — when the system grows 10x, toil grows 10x too This article systematically covers how to manage operational work across toil definition and identification, source analysis, automation elimination paths, the 50% cap principle, measurement and tracking methods, and team practices....

Overview Woken up by an alert at 3 AM, facing an unfamiliar service — how fast can you recover? If you need to scroll through chat logs, ask colleagues, and dig through code to figure out what to do, your team is missing one thing — a Runbook. A Runbook is the most fundamental yet most easily overlooked engineering practice in the SRE framework. It bridges the gap between “alert” and “action” — an alert tells you “something is wrong,” and a Runbook tells you “what to do....

Overview Incidents are inevitable, but the quality of your incident response determines the impact scope and duration. A mature incident response framework can bring order to chaos — ensuring the right people do the right things, information flows where it needs to go, and recovery happens as fast as possible. The reality many teams face during incidents: alert bombardment, chat channel flooding, unclear ownership, duplicate investigations, inconsistent external messaging, and inability to explain what happened after recovery....

Overview The Error Budget is the most ingeniously designed mechanism in the SRE framework. It transforms the long-standing “stability vs. iteration speed” debate — previously settled by opinion and politics — into a quantifiable engineering decision framework: your system has an “unavailability allowance,” and when it’s spent, you stop and fix things. In practice, however, many teams define SLOs and error budgets but stop at displaying a percentage number on a dashboard....

Overview The first dilemma many teams face when practicing SRE is: they know what an SLO is, but they don’t know how to set one. They either copy Google’s 99.99% or pick an arbitrary 99.9% — only to find that the number neither reflects user experience nor drives engineering decisions. A good SLO isn’t plucked from thin air. It’s derived from business goals through a series of engineering methods: user journey analysis, metric selection, value calibration, multi-tier design, and regular review....