Overview
Your company uses a cloud service with an SLA that clearly states 99.95% monthly availability. One night, the service goes down for 30 minutes. You file a compensation claim with the cloud provider. The calculation goes roughly: downtime as a fraction of total monthly minutes, multiplied by your monthly fee. Assuming you pay 1000 yuan/month, that’s 30 minutes divided by 43,200 minutes times 1000, giving you about 0.7 yuan.
That’s right. Seven tenths of a yuan. Your service was down for half an hour, customer support lines were flooded, refund requests piled up, and the cloud provider compensated you with 0.7 yuan.
This isn’t a joke. It’s the actual calculation logic used in a 2025 CSDN article breaking down cloud provider SLAs. Of course, real cloud providers use tiered compensation (which we’ll cover later), but the core issue remains: the SLA number looks impressive, but the compensation clauses are packed with exclusions and narrow definitions. When something actually breaks, the payout barely covers a fraction of your real losses.
This article isn’t about how to “read” an SLA—there are plenty of those online. I’m going to talk about how, as an SRE, you design your own service’s SLA, how to avoid the pitfalls in cloud provider SLAs, and how to turn a contract clause into an enforceable engineering constraint.
SLA, SLO, SLI: Stop Mixing Them Up
Many teams treat these three terms as synonyms. In meetings, someone says “our SLA is 99.9%” when they actually mean SLO. The distinction isn’t wordplay—confusing them causes real problems.
Here’s a comparison table, with details following:
| Dimension | SLI (Indicator) | SLO (Objective) | SLA (Agreement) |
|---|---|---|---|
| What it is | Quantified measurement of service quality | Internal target based on SLI | External contractual commitment |
| Who it’s for | Engineering team | Engineering + Product | Customer / Business |
| Breach consequence | None | Freeze deployments, invest in stability | Compensation, contract violation |
| Strictness | Objective fact, no strict/loose | Must be stricter than SLA | Looser than SLO, provides buffer |
| Typical example | P99 latency = 120ms | P99 < 200ms, success rate > 99.9% | Monthly availability > 99.5%, else 10% credit |
SLI: What Are You Actually Measuring
SLI is purely objective data. It answers the question “what did we actually observe.” For example:
- Request success rate: successful requests / total requests
- Latency percentile: P99 response time
- Availability: normal service time / total time
SLI definitions seem simple, but the pitfalls are deep. Take “success rate”—how do you define “success”? Does HTTP 200 count? What about HTTP 200 with incorrect data in the response body? What about HTTP 500 that succeeds on retry within 50ms? SLI definition precision directly determines whether the downstream SLO and SLA are meaningful.
Google’s SRE Book makes a classic point: SLI should be as close to user experience as possible. Not measuring your server’s CPU utilization, but measuring the end-to-end latency from when a user sends a request to when they receive a response. This principle comes up repeatedly when designing SLA monitoring later.
To be specific: a payment API. If you define SLI as “minutes the API process is alive / total minutes,” then a 45-minute period where the process is running but returning 500 doesn’t count as downtime. But users genuinely couldn’t pay during those 45 minutes. Users don’t care if your process is alive—they care if their request succeeds. So the correct SLI should be “successful requests / total requests”—measured from the dimension of actual user requests, not from the server process dimension.
Going further: if the API responds but latency jumps from 50ms to 8 seconds, users will likely time out and give up. The success rate SLI might not catch this (the request eventually succeeded, just slowly), but user experience is already degraded. So latency should also be an SLI dimension. A complete SLI for a service should cover at least three dimensions: availability (success rate), latency (P99/P999 response time), and throughput (processing capacity ceiling). The weight of each dimension is allocated by business characteristics, not one-size-fits-all.
Two contrasting examples: an e-commerce homepage—latency should have the highest weight (users leave after 3 seconds, but the homepage rarely fails). A payment endpoint—success rate should have the highest weight (being slow is tolerable, but failing to pay is fatal). A log ingestion service—throughput matters most (a single log entry delayed 2 seconds is fine, but not finishing 100K entries per second is a problem). SLI weighting isn’t a technical decision—it’s a business decision. The product manager and SRE should sit together and derive SLI weights from business scenarios, not have the SRE decide alone.
SLO: Your Internal Target
SLO is the benchmark a team sets for itself. For example, “over 30 days, 99.9% of requests have P99 latency below 200ms.” SLO isn’t published externally and doesn’t involve compensation, but it’s the core tool for engineering governance—the error budget is derived from SLO.
For detailed SLO design, refer to my previous articles SLO Design in Practice and SRE Core Concepts: SLI, SLO and Error Budgets. Here I’ll emphasize one core principle:
SLO must be stricter than SLA.
This is Google SRE’s iron rule. If you promise 99.9% externally, your internal SLO should be at least 99.95% or even 99.99%. Why? Because you need buffer. From fault occurrence to detection, confirmation, and recovery, there’s a large time window. If SLO and SLA are equal, any slight SLI fluctuation triggers an external breach, and you don’t even have reaction time.
To feel this in numbers: SLA promises 99.9%, allowing 43 minutes of monthly unavailability. If SLO is also 99.9%, you only have 43 minutes of margin. A single 30-minute incident consumes 70% of the budget, leaving only 13 minutes for the rest of the month. Any minor fluctuation could trigger SLA breach. But if SLO is 99.95%, your internal budget is only 21.5 minutes. A 30-minute incident first triggers SLO breach (freezing deployments), but there’s still 21.5 minutes of buffer before SLA breach. Those 21 minutes are your emergency window—enough for a rollback or failover.
Chris Jones and Niall Murphy said at SREcon 2016: SLA isn’t the right tool for SREs to manage a service. SREs manage services with SLOs and error budgets; SLA is just legal and business-level fallback. This sounded heretical at the time; now it’s consensus.
One more thing: in many companies, SLA is negotiated by the business team, SLO is set by the engineering team, and nobody connects the two. The business team raises the SLA commitment to 99.99% to close a deal, while the engineering team has no idea what that number means and is still capacity planning for 99.9% SLO. When something breaks, they discover the gap between promise and capability is an order of magnitude. SLA and SLO must be managed by the same mechanism—before the business team makes a commitment, they must use the engineering team’s SLO data as the basis; when the engineering team changes SLO, they must evaluate the impact on SLA commitments.
SLA: The Contract Layer
SLA is a formal agreement between the service provider and the customer. It specifies: what metrics are promised, what happens when targets aren’t met (usually service credit), and what situations don’t count as violations (exclusion clauses).
SLA terms are typically less strict than SLO. This isn’t laziness—it’s a pragmatic buffer strategy. Internal SLO is the bar you strive for; external SLA is the floor you fall back on. The gap between them is the engineering team’s emergency space.
The key distinction is consequences. When SLO isn’t met, the engineering team stops and fixes bugs. When SLA isn’t met, the legal team starts calculating compensation. One is an engineering action, the other is a contract action.
Azure’s official documentation summarizes this relationship well: SLA describes the minimum guarantee from the service provider, SLO reflects the reliability the user actually needs. Don’t simply adopt the SLA percentage as your service’s reliability target—consider your own code, dependencies, and user tolerance.
Cloud Provider SLA Compensation: Can the Payout Cover Your Losses
Most cloud providers don’t pay cash compensation—it’s “service credit” that offsets your future bills. Get this straight first: you’re not getting money back, you’re getting a voucher.
Tiered Compensation: Less Generous Than You Think
Major cloud providers generally use tiered compensation. Here’s a typical structure (specific values vary by service and provider, for illustration only):
| Monthly Availability | Credit % | Description |
|---|---|---|
| < 99.9% | 10% | Below commitment by 0.1% |
| < 99.0% | 25-30% | Severely below commitment |
| < 95.0% | 50-100% | Catastrophic failure |
Seems reasonable at first glance? Think carefully: suppose you pay 5000 yuan/month for a cloud service with a 99.9% SLA. One month, actual availability is 99.8%—below 99.9% but well above 99.0%. You get 10% credit = 500 yuan in vouchers.
But what does 99.8% mean? Out of 43,200 monthly minutes, 86 minutes of service unavailability. 86 minutes of production downtime—for e-commerce, that could be tens of thousands of lost orders; for finance, potentially millions in interrupted transactions. You get 500 yuan in vouchers that can only be applied to next month’s bill.
The Fine Print in Compensation Formulas
A more subtle issue is the calculation method. Some cloud providers don’t calculate by “how far below the commitment,” but by “downtime as a fraction of total time.” This is the 0.7 yuan formula from the beginning: downtime / total monthly minutes × monthly fee = compensation. This formula is particularly unfriendly to short, frequent outages—each outage lasting 5 minutes, 10 times, totaling 50 minutes of downtime, but each individual incident doesn’t meet the “minimum claim threshold” (covered later), potentially resulting in zero compensation.
AWS states in its official blog: SLA is a commitment to customers, including remedial actions when targets aren’t met. But the remedial action is “service credit” not “loss compensation.” Microsoft’s reliability architecture guide is more blunt: Be Redundant, design to overcome failure—don’t expect SLA compensation to cover your business losses; design failure into your system.
My Recommendation
Don’t treat cloud provider SLAs as your business insurance. The payout barely covers a fraction of your losses. The correct approach:
- Clarify your business’s actual tolerance for downtime (calculate losses per minute, not as a percentage)
- Derive the service reliability level you need from business tolerance
- If the cloud provider’s SLA is below your needs, compensate with architecture—multi-AZ, cross-region disaster recovery, service degradation plans
- Treat cloud provider SLA only as “selection reference” and “post-incident legal fallback,” not as the sole dependency for business reliability
To add a real-world data point: AWS’s us-east-1 suffered a major outage in 2017 that lasted several hours, affecting Netflix, Pinterest, and many other customers. According to reports, some customers chose to sue AWS for SLA violations. AWS ultimately agreed to pay affected customers a certain amount in compensation. But that “certain amount” was a drop in the bucket compared to the actual losses these customers suffered from service disruption (user churn, revenue decline, brand damage). This is why Microsoft’s reliability guide says “embrace failure, design to overcome it”—don’t count on SLA compensation as your backstop.
Composite SLA Math: How Failure Probability Multiplies on Dependency Chains
If your service depends on three cloud services, each with a decent individual SLA, what happens when you combine them? There’s a mathematical trap here.
Serial Dependencies: The Multiplication Rule
Assume your service architecture is: API Gateway → Compute Service → Database, with serial dependencies. If any one fails, your service fails. Composite SLA is calculated by multiplying the individual SLAs:
Composite SLA = SLA_gateway × SLA_compute × SLA_database
Plugging in real numbers:
| Component | Individual SLA | Annual Allowed Downtime |
|---|---|---|
| API Gateway | 99.95% | 4.38 hours |
| Compute Service | 99.95% | 4.38 hours |
| Database | 99.99% | 0.88 hours |
| Composite SLA | 99.89% | 9.59 hours |
Three services at the 99.95% level in series yield a composite SLA of only 99.89%. If you promise 99.9% externally, you’ll find your dependency chain itself doesn’t meet the bar.
Microsoft Azure’s Cloud Adoption Framework documentation provides a formula to estimate annual outage time:
Est. Outage = (1 - Composite_SLA) × 8760 hours
A 99.89% composite SLA corresponds to approximately 9.59 hours of annual downtime. This means even when each component individually looks “fine,” the combined reliability may be significantly worse than you expected.
Parallel Redundancy: The Addition Rule
If you’ve designed redundancy (e.g., multi-AZ deployment), the failure probability calculation changes. Two independent 99.9% services in parallel: only when both fail simultaneously does the service fail:
Composite SLA = 1 - (1 - SLA_1) × (1 - SLA_2)
= 1 - 0.001 × 0.001
= 1 - 0.000001
= 99.9999%
This is why multi-AZ deployment significantly improves reliability. But the prerequisite is that both instances are truly independent—no shared control plane, no shared network ingress, no shared authentication service. If they share any of these, the independence assumption breaks down, and so does the composite SLA math.
Each “9” Costs Exponentially More
From 99% to 99.9%, allowable downtime drops from 87.6 hours/year to 8.76 hours/year—a 10x improvement. From 99.9% to 99.99%, down to 0.876 hours/year—another 10x. But the engineering cost of each improvement isn’t linear.
| Availability | Annual Downtime | Monthly Downtime | Relative Cost | Typical Architecture |
|---|---|---|---|---|
| 99% | 87.6 hours | 438 min | 1× | Single instance + auto-restart |
| 99.9% | 8.76 hours | 43.8 min | 3-5× | Multi-instance + load balancer |
| 99.99% | 52.6 min | 4.38 min | 10-20× | Multi-AZ + auto-failover |
| 99.999% | 5.26 min | 0.44 min | 50-100× | Multi-region + active-active + global traffic steering |
Each additional “9” roughly costs 3-5x more engineering resources. Going from 99.99% to 99.999% isn’t “trying a bit harder”—it’s upgrading from multi-AZ to multi-region global traffic steering, an order-of-magnitude architecture change.
When making SLA decisions, ask yourself: is one more “9” worth the cost? 99.99% means only 4.38 minutes of allowed monthly downtime. A single deployment rollback might exceed that. Does your business really need this level of availability? For most businesses, 99.9% is sufficient.
A Real Lesson
I saw a team use AWS API Gateway + Lambda + DynamoDB. The three services had SLAs of 99.95%, 99.95%, and 99.99% respectively. They calculated the composite SLA as 99.89% and thought “close enough to 99.9%, good enough.” Then AWS us-east-1 had a regional outage—all three services went down simultaneously because they were all deployed in the same region. The “independent failures” they assumed were actually the same underlying infrastructure.
Lesson: composite SLA math only holds when components are truly independent. Cross-region deployment costs more than single-region, but what you buy is the validity of the independence assumption.
True Redundancy: Independence Verification Checklist
Deploying across multiple AZs or regions doesn’t equal true independence. This checklist helps you verify:
# Independence verification checklist
independence_checklist:
control_plane:
- question: "Are control planes independent across regions?"
risk: "Multi-region with shared control plane fails entirely when control plane goes down"
check: "Confirm each region has independent API Server / control nodes"
data_replication:
- question: "Is data replication synchronous or asynchronous?"
risk: "Synchronous replication may cause write timeouts during network partitions"
check: "Core data async replication + eventual consistency; non-core data can be synchronous"
failover_mechanism:
- question: "Is failover automatic or manual?"
risk: "Manual failover RTO may exceed SLA commitment"
check: "DNS health check + automatic failover, RTO < 5 minutes"
capacity:
- question: "Can a single region handle full traffic?"
risk: "Post-failover overload in single region causes cascading failures"
check: "Each region capacity-planned for 150% of peak traffic"
shared_dependencies:
- question: "Are there shared auth, DNS, CDN, or other foundational services?"
risk: "Shared service failure takes down all regions"
check: "Auth service deployed multi-region; DNS uses multiple providers"
This checklist can’t be filled once and forgotten. Re-check after every architecture change—added a new microservice? Does its dependency introduce a new shared point? Added read-write splitting to your database? Are the write and read instances in the same AZ?
6 Commonly Overlooked Pitfalls in SLA Clauses
Cloud provider SLA documents are typically dozens of pages. Most people only read the availability percentage on the first page and sign. The real traps are buried in the back.
Pitfall 1: Exclusion Clauses—These Don’t Count as Downtime
Nearly every SLA has exclusion clauses listing situations not counted as unavailability:
- Planned maintenance: Maintenance windows the cloud provider notifies in advance don’t count as unavailable time. The problem is “how much advance notice,” “how many times per month,” and “how long is each window”—some SLAs don’t specify. In practice, the provider announces “maintenance tonight for 2 hours” and those 2 hours don’t count against SLA, and you have no recourse.
- Broad force majeure interpretation: Some SLAs classify “third-party network provider issues” as force majeure. Backbone network jitter causing user access problems—is that force majeure? Ambiguous clauses leave room for interpretation.
- Customer-side issues: You deleted an instance, misconfigured a security group, or your app has a bug causing resource exhaustion—these obviously aren’t the provider’s fault. But where’s the boundary of “customer-side”? If a provider component has a bug that forces you to make incorrect configurations, whose fault is that?
Pitfall 2: “Unavailable” Is Defined More Narrowly Than You Think
Most SLAs define “unavailable” as: all requests failing continuously for X minutes. Note three keywords: all, continuously, and failing.
- “All” means partial user inaccessibility doesn’t count—only when all users can’t access does it count.
- “Continuously” means intermittent failures don’t count—down 3 minutes, up 2 minutes, down 3 minutes again. If no single continuous period exceeds X minutes, it may not qualify for claims.
- “Failing” usually means HTTP 5xx or connection timeout. Your API returns 503 but not 500? Might not count. Your API response time jumps from 50ms to 10 seconds but doesn’t completely fail? In most SLAs, this doesn’t count as “unavailable.”
Pitfall 3: Minimum Claim Threshold
Many SLAs require a single outage to exceed a certain duration to qualify for claims. For example, “a single outage lasting more than 15 minutes is eligible for compensation.” If the outage lasts 14 minutes, even if it affects the availability percentage, you can’t claim.
This threshold seems reasonable (avoiding massive claim volume from micro-fluctuations), but the practical effect is: high-frequency short-duration outages become an SLA blind spot. Down 10 minutes at a time, 5 times, totaling 50 minutes of downtime—none individually meets the 15-minute threshold, so zero compensation.
Pitfall 4: Monitoring Measurement Discrepancy
SLA availability is typically measured “by the cloud provider’s monitoring system.” The problem is your monitoring and the provider’s monitoring may be completely different.
Your monitoring measures from the user perspective—simulating real user requests, probing from multiple regions, tracking success rates. The provider’s monitoring measures from the server side—checking if the service process is alive and the port is reachable. The process is alive but returning errors for all requests? Your monitoring shows a fault; the provider’s monitoring shows “available.”
This discrepancy has actually happened. A team using AWS API Gateway experienced a configuration update that caused 30% of requests to return 503. From AWS’s monitoring, the API Gateway instance was alive, the port was reachable, so the “service was available.” From the team’s monitoring, 30% of user requests failed for 45 minutes. Compensation claim? Denied, because AWS’s SLA definition didn’t recognize this as “unavailable.”
Pitfall 5: Compensation Caps
Most SLAs have a compensation cap—typically not exceeding the monthly service fee. You pay 5000 yuan/month, maximum compensation is 5000 yuan (in vouchers). Your business lost 500,000 yuan due to the outage? SLA compensation has nothing to do with that.
This cap means the provider’s risk is closed-ended, while your risk is open-ended. Understanding this, you know why you can’t rely on SLA as business insurance.
Pitfall 6: Notification and Claim Deadlines
SLAs typically require customers to submit claims within a certain period (e.g., 30 or 60 days) after the outage, with expired claims voided. Some also require you to provide “outage evidence”—your own monitoring logs, impact scope descriptions, etc.
If your team is busy firefighting after an outage and nobody thinks to compile evidence and submit a claim within 30 days, by the time things settle down and you go back to the provider, the deadline may have passed.
My suggestion: Make “SLA claim” a fixed step in the incident recovery process. Not every outage triggers a claim—but add “Did this outage affect SLA? If so, submit claim within X days” to the postmortem checklist. Use an automated script to generate claim materials (timeline, impact scope, monitoring screenshots) at recovery time, store them in the ticket system for submission during postmortem.
Another common issue: cloud providers require claims through their specific channels (ticket system, support case), not accepting emails or verbal notifications. If you use the wrong channel, you may be deemed “not formally submitted within the deadline.” Confirm the specific claim process steps and channels when signing the SLA—don’t discover you took a detour after the fact.
One more easily overlooked point: SLA calculation in multi-tenant scenarios. If your service is multi-tenant (one instance serving multiple customers), and one customer’s abnormal behavior (e.g., unthrottled large queries) affects other customers, how does this “partial degradation” count in the SLA? Some contracts specify that only “all tenants simultaneously unavailable” counts as SLA breach, partial tenant impact doesn’t. This is the same logic as the cloud provider’s “all requests must fail to count as unavailable”—and it works against you. Multi-tenant services should define SLA at the tenant level—“each tenant’s monthly availability不低于 X%"—not a blanket “service availability不低于 X%.”
SRE Perspective on Internal SLA Design: 6 Engineering Decisions
We’ve covered the pitfalls in cloud provider SLAs. Now let’s talk about designing your own service’s SLA. Not the contract-signing level—the engineering level.
Decision 1: How to Set the SLA Number—Reference User Tolerance, Don’t Copy
The most common mistake: seeing the cloud provider’s SLA is 99.9%, so writing your own SLA as 99.9%. That’s copying, not designing.
SLA numbers should be derived from business tolerance. Ask three questions:
- How long an outage can users tolerate? E-commerce users might tolerate 5 minutes; financial trading users might only tolerate 30 seconds.
- What’s the direct economic loss per minute of downtime? Calculate loss per minute, not “probably a lot.”
- How to quantify indirect losses (reputation, user churn)? Reference historical outage user churn data.
After calculating, convert tolerance to an SLA percentage. For example, if users can tolerate 43 minutes of monthly downtime (~0.03% unavailability), SLA is 99.9%. But don’t forget the buffer principle—your SLO should be stricter than SLA. So SLO at 99.95%, SLA at 99.9%.
# SLA number derivation tool (illustrative)
def calculate_sla(tolerable_downtime_minutes_per_month, buffer_ratio=0.5):
"""
Derive SLA number from business tolerance
Args:
tolerable_downtime_minutes_per_month: Minutes of monthly downtime the business can tolerate
buffer_ratio: How much stricter SLO is vs SLA, 0.5 means SLO error budget is half of SLA's
Returns:
(sla_percentage, slo_percentage, monthly_error_budget_minutes)
"""
total_minutes = 30 * 24 * 60 # 43200
sla_downtime = tolerable_downtime_minutes_per_month
sla_percentage = (1 - sla_downtime / total_minutes) * 100
# SLO is buffer_ratio times the error budget of SLA
slo_downtime = sla_downtime * buffer_ratio
slo_percentage = (1 - slo_downtime / total_minutes) * 100
error_budget = sla_downtime - slo_downtime # Buffer between SLA and SLO
return sla_percentage, slo_percentage, error_budget
# Example: business tolerates 43 minutes of monthly downtime
sla, slo, buffer = calculate_sla(43)
print(f"SLA: {sla:.2f}% | SLO: {slo:.2f}% | Buffer: {buffer:.1f} minutes")
# Output: SLA: 99.90% | SLO: 99.95% | Buffer: 21.5 minutes
Decision 2: SLA Breach “Compensation” Mechanism—Internals Need One Too
External SLA breach costs money; internal SLA breach costs what? Engineering resources.
Specific approach: when SLO (internal target) is violated but SLA (external commitment) hasn’t been breached yet, trigger a “deployment freeze” mechanism—all non-urgent changes are paused, and the engineering team focuses on stability issues until the error budget recovers. This is the mechanism I detailed in Error Budget Depleted, PM Still Pushing Releases.
If the SLA itself is breached (external violation), it’s no longer just about deployment freezes—you need to trigger postmortem, impact assessment, and customer communication processes, potentially requiring legal involvement.
Key point: SLA breach must have engineering consequences. If a breach results only in a report being written, SLA is just a number that drives no behavior change.
Decision 3: SLA Monitoring Metrics—Define from User Perspective
This is the flip side of “Pitfall 4” above. When you design your own SLA, the “unavailable” definition must come from the user perspective, not just the server side.
Google SRE’s principle: SLI should be as close to user experience as possible. How specifically?
# SLA monitoring definition example (Prometheus + Sloth style)
version: "prometheus/v1"
service: "payment-api"
slos:
- name: "payment-api-availability"
objective: 99.9
description: "Payment API monthly availability, measured from user perspective"
sli:
events:
total: "sum(rate(http_requests_total{job='payment-api'}[5m]))"
errors: "sum(rate(http_requests_total{job='payment-api', status=~'5..'}[5m]))"
# Note: using http_requests_total, not process_up
# process_up only checks if process is alive, not if user requests succeed
# http_requests_total measures how many requests users actually made and how many failed
- name: "payment-api-latency"
objective: 99.0
description: "P99 latency < 500ms, covering 99% of requests"
sli:
events:
total: "sum(rate(http_request_duration_seconds_count{job='payment-api'}[5m]))"
errors: "sum(rate(http_request_duration_seconds_bucket{job='payment-api', le='0.5'}[5m]))"
# Latency SLI: requests over 500ms count as "not meeting standard"
# This definition is closer to user experience than "process alive"
# Because users don't care if your process is alive—they care if requests succeed and are fast
Two key points:
- Use request success rate, not process alive rate. The process is alive but all requests return 500—from the user’s perspective, the service is down.
- Probe from multiple regions. Network jitter in one region only affects that region’s users. From the server side, it might look like a “local issue”; from the user’s perspective, it’s “service unavailable.”
I’ve found in practice that user-perspective SLIs detect about 30% more faults than server-side SLIs. Because a large class of faults are “service alive but user experience degraded” gray-zone faults that server-side monitoring simply can’t see.
Decision 4: SLA Review Cycle—Set by Rate of Business Change
SLA isn’t set-and-forget. Services change, dependencies change, user bases change, and SLA must be reviewed regularly.
The review cycle depends on your business change rate:
| Business Type | Recommended Review Cycle | Reason |
|---|---|---|
| Fast iteration (SaaS, internet products) | Quarterly | Architecture and dependencies change frequently, SLA may be outdated |
| Stable (internal systems, traditional enterprise apps) | Semi-annually | Less change, but needs periodic calibration |
| Contract-bound (finance, healthcare) | Annually | Stable within contract period, but needs industry benchmarking |
During review, examine three data points:
- SLI actual performance vs SLO target: If SLI consistently far exceeds SLO for 3 consecutive months (e.g., SLO is 99.9% but actual is 99.99%), SLO may be set too low and can be raised. If consistently near the breach line, SLO is set too high—either lower the target or invest in capability.
- Error budget consumption trend: Is it steady consumption or concentrated consumption? Steady indicates normal fluctuation; concentrated indicates systemic issues.
- User feedback: Have users complained when SLI showed normal? This indicates a blind spot in SLI definition.
One review signal that’s easily missed: the relationship between deployment frequency and error budget. If you notice error budget jumps slightly after every deployment—not a big jump, just small consumption—it indicates deployment quality instability. These small consumptions don’t trigger alerts individually, but accumulate and might exhaust the budget by month-end. Record the error budget change for each deployment, aggregate weekly—if the trend line goes down, deployment quality is improving; if flat or going up, you need to add testing gates.
Decision 5: SLA Documentation—Manage with Code, Not Word
SLA documentation shouldn’t be a Word file in some corner; it should be executable code. Define SLA in YAML or TOML, with tools automatically generating monitoring rules and alert policies.
# sla-definition.yaml — Managing SLA as code
apiVersion: sre.example.com/v1
kind: ServiceLevelAgreement
metadata:
name: payment-api-sla
effectiveDate: "2026-09-01"
reviewCycle: quarterly
spec:
service: payment-api
# External commitments
externalCommitments:
availability: 99.9
measurementWindow: 30d
creditPolicy:
tiers:
- below: 99.9
creditPercent: 10
- below: 99.0
creditPercent: 30
- below: 95.0
creditPercent: 100
maxCredit: monthlyFee
currency: serviceCredit # Not cash
exclusions:
- plannedMaintenance # Requires 48h advance notice
- customerMisconfiguration
- forceMajeure
claimDeadline: 30d
measurementSource: customerFacingProbe # Measure from user perspective, not server side
# Internal targets (must be stricter than external commitments)
internalSLO:
availability: 99.95
latencyP99: 500ms
latencyP99Objective: 99.0
errorBudgetPolicy:
freezeDeployment: true # Auto-freeze deployments when error budget exhausted
burnRateAlerts:
- threshold: 14.4 # 1 day's budget in 1 hour
severity: critical
- threshold: 3.6 # 1 day's budget in 6 hours
severity: warning
The benefit of this approach: SLA definitions and monitoring systems are unified—change the YAML and monitoring rules take effect automatically. No need for manual sync between two systems, reducing oversights.
But having a definition file isn’t enough; you also need an audit process. Run an SLA compliance audit every quarter: check if the SLA definition file is consistent with the actual monitoring system (any cases of YAML changed but not deployed), whether exclusion clauses are being abused (planned maintenance used as a shield too many times), whether compensation records are complete (any breaches that didn’t go through the compensation process). Audit results go into the quarterly SRE report, signed off by the SRE lead. This isn’t a formality—the value of auditing is discovering the cracks between definition and execution. I’ve seen a team whose SLA file stated “maintenance windows require 48-hour advance notice,” but in practice, maintenance notices were frequently given only 4 hours in advance. When the audit caught this crack, they either fixed the process (actually give 48 hours) or changed the SLA definition (acknowledge inability and adjust the commitment). Both directions work; pretending not to see it doesn’t.
Decision 6: Buffer Design Between SLA and SLO
I’ve repeatedly mentioned “SLO is stricter than SLA,” but by how much? How to set the buffer?
This is a real engineering decision. Too large a buffer puts excessive pressure on the engineering team—the error budget is quickly exhausted, frequent deployment freezes, and business teams are unhappy. Too small a buffer raises SLA breach risk—legal and business teams are unhappy.
My recommendation: SLO error budget at 50% of SLA error budget. That is, if SLA allows 43 minutes of monthly downtime (99.9%), SLO allows only 21.5 minutes (99.95%). The 21.5 minutes in between is the buffer—when SLO is breached, there are still 21.5 minutes before SLA breach, giving the engineering team time to recover.
This 50% isn’t arbitrary. Datadog’s error budget analysis article mentions that error budget is defined as 1 minus the SLO target value. If SLO is 99.9%, error budget is 0.1%. When SLA is also 99.9%, they’re equal with no buffer. Raising SLO to 99.95% makes the error budget 0.05%—exactly half the SLA error budget.
Of course, the specific ratio should be adjusted by business scenario. For latency-sensitive businesses (e.g., real-time trading), the buffer can be larger (SLO error budget at 30% of SLA); for more tolerant businesses (e.g., internal tools), the buffer can be smaller (70%).
What happens if the buffer is too large? SLO becomes very lenient, error budget is ample, and the team never triggers “deployment freeze.” The result is SLO becomes a formality—everyone thinks “it won’t be exceeded anyway,” and willingness to invest in stability drops. When a major outage finally hits, SLI crashes through the SLA line, and the buffer was never used. Buffer isn’t “bigger is better”—the key is whether you do the right things in the buffer zone: detect SLO breach and immediately invest in stability engineering, not thinking “haven’t breached SLA yet, no rush.”
What if the buffer is too small? Every minor fluctuation triggers deployment freezes, business teams complain, and SRE is forced to loosen SLO standards. This is more painful than setting it right initially. The correct approach: start at 50%, run for a quarter with data, then adjust based on actual consumption patterns. Don’t set it in stone—SLA is a living constraint, not carved on stone tablets.
One counterintuitive point: sometimes deliberately letting SLO be breached is the right decision. If your SLO is 99.95% and the error budget is exhausted one month, the rule says freeze deployments. But if there’s a critical security patch that month—it may cause short-term fluctuation but long-term improve stability—you should deploy it. SLO is a tool, not a shackle. Error budget exhaustion isn’t “no changes allowed,” it’s “only changes that reduce risk.” A security patch is precisely reducing risk. The key is that the decision process is transparent: why make changes when the budget is exhausted, what’s the basis, who approves. These decisions should be documented, not done quietly.
Engineering Response After SLA Breach: From Detection to Postmortem
SLA breach isn’t just a legal issue—it’s first an engineering issue. The response chain from fault occurrence to postmortem:
Detection (SLI drops below SLO threshold)
↓
Confirmation (rule out false positive, confirm impact scope)
↓
Notification (notify relevant teams and stakeholders)
↓
Recovery (execute Runbook, prioritize service restoration)
↓
Stabilization (confirm root cause fixed, no secondary faults)
↓
Postmortem (output improvement items)
↓
SLA Assessment (did it trigger external breach, is compensation needed)
Each step should have corresponding automation tooling. Detection through monitoring alerts, confirmation through SOP checklists, notification through on-call systems, recovery through Runbook automation. I covered these in detail in Incident Response Framework and Runbook Writing Guide.
Here I’ll focus on the SLA-specific part—SLA Assessment.
After service recovery, calculate the impact of this outage on SLA:
#!/bin/bash
# SLA impact assessment script
# Input: outage start time, end time, service name
# Output: impact of this outage on monthly SLA
SERVICE=$1
START_TIME=$2 # Format: "2026-09-15T00:30:00+08:00"
END_TIME=$3 # Format: "2026-09-15T01:15:00+08:00"
# Calculate outage duration (minutes)
DOWNTIME_MINUTES=$(python3 -c "
from datetime import datetime
fmt = '%Y-%m-%dT%H:%M:%S+08:00'
start = datetime.strptime('$START_TIME', fmt)
end = datetime.strptime('$END_TIME', fmt)
print(int((end - start).total_seconds() / 60))
")
# Total minutes in current month
TOTAL_MINUTES=$(python3 -c "
import calendar
from datetime import datetime
now = datetime.now()
_, days = calendar.monthrange(now.year, now.month)
print(days * 24 * 60)
")
# Calculate monthly availability
AVAILABILITY=$(python3 -c "
downtime = $DOWNTIME_MINUTES
total = $TOTAL_MINUTES
uptime = total - downtime
print(f'{uptime / total * 100:.4f}')
")
echo "Service: $SERVICE"
echo "Outage duration: ${DOWNTIME_MINUTES} minutes"
echo "Total monthly minutes: ${TOTAL_MINUTES}"
echo "Current availability: ${AVAILABILITY}%"
echo ""
# Determine SLA status
SLA_TARGET=99.9
python3 -c "
availability = float('$AVAILABILITY')
target = $SLA_TARGET
if availability < target:
print(f'⚠ SLA BREACH! Current {availability}% < Target {target}%')
# Calculate compensation ratio (illustrative)
if availability < 95.0:
credit = 100
elif availability < 99.0:
credit = 30
else:
credit = 10
print(f'Estimated compensation ratio: {credit}%')
else:
buffer = availability - target
print(f'✓ SLA normal. Current {availability}%, buffer {buffer:.2f}%')
"
This script isn’t just about numbers. Its value: right after recovery, when the team is still tense, you can immediately tell the business side “whether this outage triggered external SLA breach” and “whether to start the compensation process.” No need to wait for manual calculation or legal confirmation—the engineering layer gives the judgment directly.
SLA Dashboard: Turning Numbers into Actionable Information
The SLA assessment script handles “single outage impact,” but daily operations need continuously visible SLA status. Teams need to know at any time: is the current monthly SLA status green or red? How much error budget remains? Is the burn rate in the alert zone?
Build an SLA status dashboard in Grafana with three core metric blocks:
Block 1: Current SLA Status. Display current month’s availability percentage, SLO target, SLA commitment, color-coded (green = SLO met, yellow = SLO breached but SLA not breached, red = SLA breached). This answers “are we safe right now.”
Block 2: Error Budget Consumption. Display remaining error budget (minutes), consumption rate (fast/slow), burn rate. This answers “how far are we from breach.” Burn rate exceeding 14.4 (consuming 1 day’s budget in 1 hour) triggers alerting.
Block 3: Historical Trends. Past 12 months’ SLA achievement rate, incident count, MTTR trend. This answers “are we getting better or worse.”
# Error budget burn rate alert rules (Prometheus)
# 1 hour consuming more than 1 day's budget → critical
- alert: ErrorBudgetBurnRateCritical
expr: |
(
sum(rate(http_requests_total{job="payment-api", status=~"5.."}[1h]))
/
sum(rate(http_requests_total{job="payment-api"}[1h]))
) > (1 - 0.999) * 14.4
for: 2m
labels:
severity: critical
team: sre
annotations:
summary: "Error budget burn rate exceeds 14.4 (1 day's budget in 1 hour)"
description: "Payment API error rate too high, estimated 1 day's budget consumed in 1 hour"
# 6 hours consuming more than 1 day's budget → warning
- alert: ErrorBudgetBurnRateWarning
expr: |
(
sum(rate(http_requests_total{job="payment-api", status=~"5.."}[6h]))
/
sum(rate(http_requests_total{job="payment-api"}[6h]))
) > (1 - 0.999) * 3.6
for: 5m
labels:
severity: warning
team: sre
annotations:
summary: "Error budget burn rate exceeds 3.6 (1 day's budget in 6 hours)"
description: "Payment API error rate consistently high, please monitor trends"
The burn rate alerts have two thresholds: 14.4 corresponds to 1 day’s budget consumed in 1 hour, and 3.6 corresponds to 1 day’s budget consumed in 6 hours. The former is an acute failure requiring immediate intervention; the latter is chronic degradation requiring sustained attention. This multi-window, multi-burn-rate alerting strategy comes from the Google SRE Workbook, which I also covered in Error Budget Consumption Strategies.
A Production Story
In late 2025, a SaaS company built a payment service on AWS. The architecture was simple: API Gateway → Lambda → DynamoDB, all deployed in us-east-1. They promised customers 99.9% monthly availability.
One day, AWS performed an API Gateway configuration update that introduced a regression bug. The team’s monitoring showed: 23% of payment requests returned 503, lasting 45 minutes. User-side payment failure rates spiked, and the customer support system was overwhelmed with complaints.
The team immediately sought compensation from AWS. AWS’s response: from AWS’s monitoring, the API Gateway instance was running normally, the port was reachable. The SLA definition of “unavailable” means “all requests failing continuously,” and since 77% of requests were succeeding, it didn’t meet the “unavailable” definition. The claim was denied.
Worse, this 45-minute outage dropped the company’s monthly availability to 99.87%, below the external commitment of 99.9%. They had to pay customer compensation out of their own pocket while the cloud provider paid nothing.
After this incident, the team made three changes:
- Redefining “unavailable”: Changed the internal SLA’s “unavailable” to “error rate exceeding 5% for more than 2 minutes.” No longer “all requests failing” to qualify as unavailable.
- Cross-region deployment: Deployed hot standby in us-west-2. Although costs increased 40%, next time us-east-1 goes down, traffic can automatically failover.
- Raised SLO to 99.95%: Gave themselves 21.5 minutes of buffer. If a similar outage occurs again, the error budget would be exhausted and deployment freeze triggered before SLA breach, allowing the team to intervene earlier.
They later added one more: independent probing. The team deployed a continuous probing script outside AWS (on a VPS), measuring API success rates from the user perspective. Not relying on AWS monitoring data, not relying on in-app monitoring—independently measured by a third party. The probe runs every 10 seconds, recording response codes and latency. This data is more convincing in claims than “our Grafana screenshots,” because it’s collected by an independent third party, making it harder for the cloud provider to question its impartiality.
The lesson from this story: don’t wait until SLA is breached to discover the definition has problems. Every SLA clause—what’s “unavailable,” how it’s measured, what counts as “excluded”—should be thought through at design time, not realized after a denied claim.
Summary
SLA isn’t a single number; it’s a set of engineering constraints. Key takeaways:
SLA, SLO, SLI must not be mixed. SLI is the measurement, SLO is the internal target, SLA is the external contract. SLO must be stricter than SLA; the difference is the buffer. Google SRE’s experience: SLO error budget at 50% of SLA error budget.
Cloud provider SLA compensation isn’t worth relying on. Tiered compensation seems reasonable, but “service credit” isn’t cash, the cap is the monthly fee, and it doesn’t cover business losses. Exclusion clauses, narrow definitions, minimum claim thresholds, and monitoring measurement discrepancies—any one can turn the payout into an empty promise.
Composite SLA math is worse than intuition. Serial dependency SLAs multiply—three 99.95% in series yield only 99.89%. Parallel redundancy SLAs use “1 minus failure probability product,” but only if components are truly independent. Cross-region deployment costs aren’t wasted—what you buy is the validity of the independence assumption.
Six decisions for internal SLA design: derive numbers from user tolerance not copying, breaches need engineering consequences not just reports, monitor from user perspective, review cycle by change rate, manage documentation as code not Word, leave 50% buffer between SLO and SLA.
Independent probing is your last line of defense. Don’t rely solely on cloud provider monitoring data or in-app monitoring. Deploy independent probing scripts in third-party environments, continuously measuring from the user perspective. Third-party data is most convincing in claims. Probing is cheap—one VPS and a shell script does the job.
SLA is a living constraint. It’s not something you sign and forget. Review quarterly, adjust the SLO-SLA buffer ratio based on actual data. Watch the trend line of deployment frequency vs error budget. Use audit processes to find cracks between definition and execution. SLA management isn’t a one-time legal action—it’s continuous engineering practice.
One final reminder: SLA isn’t a document, it’s an engineering process. From SLI definition, SLO setting, monitoring deployment, alert policies, incident response, postmortem, to quarterly audit—every link must exist. Missing any link, SLA degenerates into numbers on paper. Numbers on paper protect neither your users nor your business.
References & Acknowledgments
The following materials were referenced during the writing of this article. Thanks to the original authors for their contributions:
- How to read a Service Level Agreement (SLA) — Microsoft Azure official documentation, SLA vs SLO relationship definitions and SLA clause interpretation methodology
- Improve application reliability with effective SLOs — AWS Cloud Operations Blog, SLI/SLO/SLA definitions and error budget concepts
- Service Levels and Error Budgets — Chris Jones & Niall Murphy (Google), SREcon16 presentation, SLA isn’t the right tool for SREs to manage services, SLOs and error budgets are
- Business commitment in cloud management — Microsoft Azure Cloud Adoption Framework, composite SLA calculation formula and annual outage time estimation
- Failure is Always an Option — Microsoft Learn, SLA compensation doesn’t equal business loss coverage, design to overcome failure from the architecture level
- Burn rate is a better error rate — Datadog technical blog, error budget definition, calculation methods, and burn rate alerting
- Cloud Service SLA Revealed: Behind 99.99%, Promise or Word Game? — CSDN, cloud provider SLA compensation calculation examples, exclusion clause breakdown, and “unavailable” definition analysis
- SLA Service Level Agreement In-Depth Analysis — Tencent Cloud Developer Community, SLA common misconceptions, availability percentage conversion, and planned downtime handling