Automate investigations

Set up

1. Get a free API key and sign in to the console. If we email you a link, open it to confirm your address; the key appears then. The key is your account: it owns your workspaces and sets your plan.
2. Create a workspace in the console, or with POST /api/sre/workspaces and your key. You get a workspace token, shown once, and an alert URL.
3. Test the connections. BoxAI checks each connector and shows the service map it found.
4. Send alerts from your alerting tool to the alert URL, or turn on watch mode.
5. Optionally, post deploys and config changes from CI/CD to the change events URL.

Each tool must be reachable over public HTTPS. For private tools, put them behind Cloudflare Tunnel or an authenticating proxy and give BoxAI a read-only token.

Connectors

Every workspace also accepts change events (below). No service map available? Send one as map when you create the workspace: [{ "name": "checkout", "dependsOn": ["payments-api"], "aliases": ["checkout.prod.svc"] }].

Alerts in

POST to your alert URL with Authorization: Bearer <token>. BoxAI recognises the format by its shape:

An alert seen in the last hour (same fingerprint) is skipped, and each workspace runs at most 12 investigations per clock hour (UTC), so a flapping alert can't flood anyone. Both are enforced atomically: a burst of alerts arriving at once can't slip past either. An alert skipped because a limit was reached isn't marked as seen, so a resend after the limit resets is investigated.

Change events

curl -X POST https://boxai.dev/api/sre/events/<workspace> \
  -H "Authorization: Bearer <token>" -H "content-type: application/json" \
  -d '{"service": "checkout", "kind": "deploy", "message": "checkout v1.4.2",
       "version": "v1.4.2", "previousVersion": "v1.4.1"}'

kind is deploy or config. ts is optional (default now, within 7 days). Up to 50 events per request, as an array. Kept for 7 days. Use it from Jenkins, GitLab, Azure DevOps, Argo CD notifications, feature-flag tools or a person.

Watch mode

With a Prometheus connector and "watch": true, BoxAI raises its own alerts. Every 10 minutes it reads the entry points of your service map (services nothing else calls), and when 5xx rate, p95 latency, throughput or restarts move past twice or half their baseline for two points in a row, it alerts on the highest affected service and investigates. You can run it alongside your own alerting.

Each watched workspace is checked on its own schedule, so one slow Prometheus never delays another. The console shows when it was last checked. If a check can't read your metrics (an expired token, a firewall change), the console says so in red; it never reports all clear when it couldn't look.

What the agent does

1. Map. Finds the service map. Only services the alerting service depends on can be the cause.
2. Check. Pulls metrics, error logs and deploys for the alerting service and what it calls directly, for the hour before the alert.
3. Decide. Nothing anomalous yet: look one level deeper. A leading suspect has unchecked dependencies: check them before blaming it. No change explains the suspect: look back 6 hours for changes. Otherwise: conclude.
4. Conclude. Scores every suspect with the five checks, writes the next step and what it ruled out, and the report.

At most 5 evidence rounds and 25 services per investigation. Each investigation runs on its own, one round every 1.5 seconds plus the time your tools take to answer, so most finish well within a minute of the alert. Every step, with the reason for it and what it found, is in the report.

Reports out

The report page (/incidents/run/?id=...) has an unguessable link and shows the map, every step the agent took, the scores, the timeline and what was ruled out.

API

Calls marked API key take Authorization: Bearer <your BoxAI key>. Workspace calls take the workspace token or the key of the account that owns the workspace. Give alerting tools and CI the workspace token, never your key. Tools that can't set headers can use ?token= instead, at the cost of the token appearing in their logs.

Data and limits

• Connector tokens are encrypted (AES-GCM) before they're stored and never returned by the API.
• Investigations keep the alert, the events the agent used and the result, for your plan's history period (below) and up to 500 per workspace, then they're deleted automatically. Deleting a workspace deletes them at once. Change events are kept 7 days.
• Requests to your tools are bounded: at most 60 points per metric series, 300 log lines per query, 8 GitHub repos per round, 6-second timeouts.
• The incident update can be written by a model on Cloudflare Workers AI, within a daily budget. The findings are sent to it; your tokens never are.
• BoxAI never writes to your tools.

Pro and Team limits apply once paid plans are on sale; until then every account has the Free limits. Every workspace also runs at most 12 investigations per clock hour, and each account can create at most 3 workspaces a minute.