3 Days of Approval, Still Crashed: Replacing Manual CAB with Error Budget Gates Cut Change-Related Incidents by 60%
Overview 1:47 AM. Phone buzzes. The alert group explodes with 200+ messages. Order success rate drops from 99.8% to 92%. The on-call engineer restarts the service, rolls back the version, and fights for 40 minutes to restore service. Next day’s postmortem: root cause was a one-line database connection pool config change pushed during the afternoon release. This change went through the full approval process — developer submission, test verification, manager sign-off, SRE review....