Demo: API error-rate alert with context
A useful API alert combines an expression with enough context for the responder. The annotation should say what is happening and where to start, not just repeat the metric name.
- Use a ratio when possible so traffic volume changes do not distort the alert.
- Add a for duration to avoid paging on short deploy blips.
- Include service, severity, and runbook labels consistently.
groups:
- name: api.rules
rules:
- alert: ApiHighErrorRate
expr: sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) > 0.05
for: 10m
labels:
severity: page
service: checkout
annotations:
summary: "Checkout API 5xx error ratio is above 5%"
runbook: "Check recent deploys, dependency errors, and upstream payment latency."The generator should produce YAML that is readable enough to review before promtool validation.