Reactive and read-only; covers every new or still-unresolved alert in
{{sentry_project}} found in a sweep, nothing carried over between sweeps.
Handle each alert as an independent unit: a failure or inconclusive diagnosis
on one alert never blocks triage of the others found in the same sweep.
Step 1 — Pull the alert from Sentry (read-only)
Via the Sentry connector, for every alert in {{sentry_project}} that is new or
still unresolved since the last sweep:
The full stack trace and exception type.
Event frequency (spike vs. steady trickle) and the first-seen timestamp.
Whether Sentry has already grouped it under an existing, previously-seen
issue fingerprint.
Step 2 — Correlate the logs (read-only)
Via the Datadog connector, pull the log lines in a window around the
first-seen timestamp (default: 10 minutes before to 5 minutes after), scoped
to the service(s) named in the stack trace. Line the log entries up against
the trace to confirm the failure mode — a matching upstream timeout, a bad
payload, a dependency erroring — rather than trusting the exception message
alone.
Step 3 — Check recent deploys on GitHub (read-only)
Rank candidate deploys by how closely they precede the first-seen timestamp
and whether the changed files touch the failing service/module. Treat the
closest matching deploy as the suspect, not a certainty — say so in the
diagnosis.
Step 4 — Classify severity and benign vs. real
Signal
Classification
Matches a known flaky/noisy pattern (repeated transient timeout, third-party rate limit, a fingerprint seen before with no user impact)
Benign — close with reasoning, no page
New error type, affects one user or a narrow code path, no matching deploy
Real, low severity — post diagnosis, no page unless it recurs
Spike in frequency, matches a recent deploy, or affects a shared/critical path
Real, elevated severity
Marked fatal/critical in Sentry, or severity at or above {{page_severity_threshold}}
Page immediately regardless of diagnosis confidence
Trace, logs, or deploy history are inconclusive after Steps 1–3
Unresolved — page, diagnosis attached as partial context
Step 5 — Post the first-pass diagnosis
Post one message per alert to {{incident_channel}}: the error and where it's
from, the first-seen time and frequency, the correlated log lines, the
suspect deploy (PR/commit + author, or "no matching deploy found"), and the
severity classification from Step 4.
Step 6 — Page or close
Page — at or above {{page_severity_threshold}}, or unresolved per Step
4: @-mention the on-call rotation in {{incident_channel}} on the same
message, with the diagnosis attached as context, not a substitute for
looking at it.
Close — benign or low severity with a confident diagnosis: mark it
resolved in the post with the reasoning, no page.
Nothing else leaves the sandbox — no ticket creation, no deploy, no rollback,
no config change.