Status page at /status, with incidents site administrators post (#166)
* Status page: incidents, and component statuses from them and the kill switches Migration 0033 adds status_incidents and status_incident_updates. lib/status.ts derives each component's status as the worst unresolved incident naming it (Statuspage's names), puts what an instance-wide kill switch stops under maintenance, and reports the website as down when D1 can't be read. Opening, updating and resolving incidents is validated and audited. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo * Status page at /status, Settings → Status, and its REST API /status is public, linked from a new site footer. GET /api/v3/diffinery/status returns the summary shaped like Statuspage's (cached for 30 seconds). Site administrators open incidents and post updates under Settings → Status or through /api/v3/diffinery/admin/status/incidents, which is 404 to anyone else. "status" becomes a reserved login. The e2e test runs in the security shard. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo * Docs: the status page, its API, and incidents in site administration A new Administration page explains the components, how statuses are derived, the API, and why an external uptime monitor of /healthz is still needed. The API reference gets the endpoints' notes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo --------- Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent 5e425b5commit 8fb3f5537c5a2dc32a254f1000ea8146ccf50a8dBrowse files
22 files changed+1507−10
Files (22)
- .github/workflows/e2e.yml+1−1
- apps/diffinery/docs/accounts/invitations.md+1−1
- apps/diffinery/docs/admin/site-administration.md+17−2
- apps/diffinery/docs/nav.json+2−1
- apps/diffinery/docs/reference/rest-api.md+3−0
- apps/diffinery/docs/security/audit-log.md+2−0
- apps/diffinery/docs/self-hosting/status-page.md+137−0
- apps/diffinery/migrations/0033_status_incidents.sql+27−0
- apps/diffinery/src/app.ts+2−0
- apps/diffinery/src/docs/api-meta.ts+35−0
- apps/diffinery/src/docs/reference.ts+3−1
- apps/diffinery/src/http/status.ts+59−0
- apps/diffinery/src/lib/status.ts+356−0
- apps/diffinery/src/lib/util.ts+1−1
- apps/diffinery/src/web/account.tsx+5−2
- apps/diffinery/src/web/assets.ts+2−0
- apps/diffinery/src/web/routes.tsx+3−0
- apps/diffinery/src/web/status.tsx+289−0
- apps/diffinery/src/web/ui.tsx+5−0
- apps/diffinery/test/e2e/status.test.mjs+316−0
- apps/diffinery/test/unit/docs.test.ts+1−1
- apps/diffinery/test/unit/status.test.ts+240−0
modified.github/workflows/e2e.yml+1−1View file
| @@ -59,7 +59,7 @@ | ||
| 59 | 59 | # Shards of similar length; a new test file runs in "rest" unless it's added to another. |
| 60 | 60 | core='/e2e\.test\.mjs$' |
| 61 | 61 | agents='/(agentic|review|tasks)\.test\.mjs$' |
| 62 | - security='/security\.test\.mjs$' | |
| 62 | + security='/(security|status)\.test\.mjs$' | |
| 63 | 63 | case "${{ matrix.shard }}" in |
| 64 | 64 | core) files=$(ls test/e2e/*.test.mjs | grep -E "$core") ;; |
| 65 | 65 | agents) files=$(ls test/e2e/*.test.mjs | grep -E "$agents") ;; |
modifiedapps/diffinery/docs/accounts/invitations.md+1−1View file
| @@ -233,7 +233,7 @@ | ||
| 233 | 233 | The **Join Diffinery** page suggests a username made from the part of the email address before the `@`. Rules: |
| 234 | 234 | |
| 235 | 235 | - Starts with a letter or digit, then letters, digits, `-` and `_`, up to 39 characters. |
| 236 | -- Not already taken (case-insensitive) and not reserved. Reserved names include `api`, `mcp`, `settings`, `new`, `notifications`, `invite`, `login`, `admin`, `explore`, `docs`, `auth`, `signin`, `signup`, `security`, `diffinery`, `ghost`, `github-actions`, `noreply`, `support`, `abuse`, `postmaster`, `root` and `staff`. | |
| 236 | +- Not already taken (case-insensitive) and not reserved. Reserved names include `api`, `mcp`, `settings`, `new`, `notifications`, `invite`, `login`, `admin`, `explore`, `docs`, `auth`, `signin`, `signup`, `security`, `status`, `diffinery`, `ghost`, `github-actions`, `noreply`, `support`, `abuse`, `postmaster`, `root` and `staff`. | |
| 237 | 237 | |
| 238 | 238 | Your username is part of your repository URLs (`https://diffinery.com/<username>/<repo>`). You can change it later under **Settings → Profile**, which changes those URLs; existing clones then need a new remote URL (see [Profile and preferences](/docs/accounts/profile-and-preferences#change-your-username)). |
| 239 | 239 |
modifiedapps/diffinery/docs/admin/site-administration.md+17−2View file
| @@ -1,9 +1,9 @@ | ||
| 1 | 1 | --- |
| 2 | 2 | title: Site administration |
| 3 | -description: What site administrators do: manage people and admins, handle abuse and security reports, suspend and delete accounts, stop Actions, oversee imports, read the audit log and check email. | |
| 3 | +description: Site administrators manage people and admins, handle abuse and security reports, suspend and delete accounts, stop Actions, post status incidents, oversee imports, and check the audit and email logs. | |
| 4 | 4 | --- |
| 5 | 5 | |
| 6 | -Site administrators run a Diffinery instance. They decide who gets in, help people who lost their passkeys, handle private security reports, and check the audit log and email delivery. This page covers who the site administrators are and each page in the **Site administration** group of **Settings**. | |
| 6 | +Site administrators run a Diffinery instance. They decide who gets in, help people who lost their passkeys, handle private security reports, post incidents on the status page, and check the audit log and email delivery. This page covers who the site administrators are and each page in the **Site administration** group of **Settings**. | |
| 7 | 7 | |
| 8 | 8 | ## Site administrators |
| 9 | 9 | |
| @@ -16,6 +16,7 @@ | ||
| 16 | 16 | | **Email** | `/settings/admin/email` | Email delivery status, the suppression list, template previews and the log of every email | |
| 17 | 17 | | **Kill switches** | `/settings/admin/kill-switches` | Stop Actions and agent sandboxes for a repository, an account or the whole instance | |
| 18 | 18 | | **Security reports** | `/settings/admin/security-reports` | Private vulnerability reports from the report form, to read, triage and resolve | |
| 19 | +| **Status** | `/settings/admin/status` | Incidents on the public status page, to open, update and resolve | | |
| 19 | 20 | | **Audit log** | `/settings/audit` | The most recent security-relevant events | |
| 20 | 21 | |
| 21 | 22 | Their **Settings → Profile** page shows **Site admin** as yes, and they have an **admin** badge in the users list. |
| @@ -387,6 +388,20 @@ | ||
| 387 | 388 | echo '{"state": "resolved", "note": "Fixed in 2026.10.15; reporter credited."}' | diffinery api PATCH /diffinery/admin/security-reports/<id> --input - |
| 388 | 389 | ``` |
| 389 | 390 | |
| 391 | +## Status | |
| 392 | + | |
| 393 | +Every instance has a public [status page](/docs/self-hosting/status-page) at `/status`, with JSON at `GET /api/v3/diffinery/status`: each component's status, incidents in progress and those resolved in the last 14 days. Anyone can read it without signing in. | |
| 394 | + | |
| 395 | +**Settings → Status** (`/settings/admin/status`) shows what the page says now, a form to open an incident (a title, the impact, the affected components, a state and the first update), each open incident with a form to post an update or resolve it, and the recent incidents. While an incident isn't resolved, its impact sets the status of each component it names; pausing hosted compute (see [Kill switches](#kill-switches)) puts **Actions** and **Agents and sandboxes** under maintenance without one. Each change is in the audit log (`status_incident.create`, `status_incident.update` or `status_incident.resolve`). Incidents and their updates are public, so keep internal details out of them. | |
| 396 | + | |
| 397 | +| Method and path | Description | Needs | | |
| 398 | +| --- | --- | --- | | |
| 399 | +| `GET /api/v3/diffinery/admin/status/incidents` | Incidents, newest first, with their updates | site administrator, a `read` token not limited to repositories, or a browser session | | |
| 400 | +| `POST /api/v3/diffinery/admin/status/incidents` | `{"title": "...", "impact": "major", "components": ["git"], "state": "investigating", "body": "..."}`: opens an incident | site administrator, with an `admin` token not limited to repositories, or a browser session | | |
| 401 | +| `POST /api/v3/diffinery/admin/status/incidents/{id}/updates` | `{"state": "resolved", "body": "..."}`: posts an update, which resolves the incident when its state is `resolved` | site administrator, with an `admin` token not limited to repositories, or a browser session | | |
| 402 | + | |
| 403 | +To anyone else these endpoints answer `404`. The status page runs in the instance's own Worker, so it is down whenever the whole Worker is: watch `/healthz` with an external uptime monitor too (see [Monitoring from outside](/docs/self-hosting/status-page#monitoring-from-outside)). | |
| 404 | + | |
| 390 | 405 | ## Workflow scheduler |
| 391 | 406 | |
| 392 | 407 | Scheduled workflows are started by Diffinery's own timer, every 5 minutes, which also catches up checks that came late (see [Scheduled workflows](/docs/actions/events#schedule)). When a scheduled run doesn't start, read the scheduler's state to tell a timer that stopped from a run that failed to start: |
modifiedapps/diffinery/docs/reference/rest-api.md+3−0View file
| @@ -37,6 +37,7 @@ | ||
| 37 | 37 | | MCP server for agents | `https://diffinery.com/mcp` | [MCP](/docs/agents/mcp) | |
| 38 | 38 | | Container registry (OCI distribution API) | `https://diffinery.com/v2/` | [Container registry](/docs/packages/container-registry) | |
| 39 | 39 | | OIDC discovery for workflow tokens | `/.well-known/openid-configuration`, `/.well-known/jwks` | [OIDC](/docs/actions/oidc) | |
| 40 | +| Status page | `/status`, and `GET /api/v3/diffinery/status` as JSON | [Status page](/docs/self-hosting/status-page) | | |
| 40 | 41 | | Health check | `/healthz` returns `{"ok": true}` | | |
| 41 | 42 | |
| 42 | 43 | ## Compatibility with GitHub |
| @@ -671,6 +672,8 @@ | ||
| 671 | 672 | | `DELETE /diffinery/admin/linked-accounts/{id}` | Site administrators: clear anyone's GitHub link. | |
| 672 | 673 | | `POST /diffinery/reports` | Report a person, repository, issue, pull request or comment to the site administrators. See [Moderation](#moderation). | |
| 673 | 674 | | `PUT`, `DELETE /repos/{owner}/{repo}/diffinery/comments/{id}/minimized` | Hide (minimize) a comment with a reason, or show it again. | |
| 675 | +| `GET /diffinery/status` | The [status page](/docs/self-hosting/status-page) as JSON, shaped like Statuspage's summary: `status`, `components`, unresolved `incidents` and `updated_at`. No authentication needed. | | |
| 676 | +| `GET`, `POST /diffinery/admin/status/incidents`, `POST /diffinery/admin/status/incidents/{id}/updates` | Site administrators: list and open status incidents, and post updates that resolve them. See [Site administration](/docs/admin/site-administration#status). | | |
| 674 | 677 | |
| 675 | 678 | Other Diffinery additions sit next to the GitHub-shaped endpoints: |
| 676 | 679 |
modifiedapps/diffinery/docs/security/audit-log.md+2−0View file
| @@ -128,6 +128,8 @@ | ||
| 128 | 128 | | `email.suppression.lift` | Someone lifts a bounce or complaint on their own address by following the verification link from **Settings → Email** | the email address | `reason` of the entry lifted | |
| 129 | 129 | | `security_report.create` | Someone sends a private [security report](/docs/admin/site-administration#security-reports) through `/security/report`. The actor is the reporter when signed in, otherwise `system`. Nothing of the report's content is recorded. | the report's id | `signed_in` | |
| 130 | 130 | | `security_report.triage`, `security_report.resolve`, `security_report.reopen` | A site administrator marks a security report triaged or resolved, or new again (**Save** on the report's page, or `PATCH /api/v3/diffinery/admin/security-reports/{id}`) | the report's id | `from` and `to` (the states), `note`, `via` (`web` or `api`) | |
| 131 | +| `status_incident.create` | A site administrator opens an incident on the [status page](/docs/self-hosting/status-page) (**Open the incident** on **Settings → Status**, or `POST /api/v3/diffinery/admin/status/incidents`) | the incident's id | `title`, `impact`, `components`, `state`, `via` (`web` or `api`); not the update's text | | |
| 132 | +| `status_incident.update`, `status_incident.resolve` | A site administrator posts an update to an incident (**Post the update**, or `POST /api/v3/diffinery/admin/status/incidents/{id}/updates`); `resolve` when it resolves the incident | the incident's id | `from` and `to` (the states), `via` | | |
| 131 | 133 | |
| 132 | 134 | ### Moderation |
| 133 | 135 |
addedapps/diffinery/docs/self-hosting/status-page.md+137−0View file
| @@ -1,0 +1,137 @@ | ||
| 1 | +--- | |
| 2 | +title: Status page | |
| 3 | +description: The public status page at /status and its JSON: the components, how their statuses are derived, posting incidents, and why it needs an external uptime monitor beside it. | |
| 4 | +--- | |
| 5 | + | |
| 6 | +Every Diffinery instance has a public status page at `/status`, like GitHub's githubstatus.com. It says whether each part of the instance works, what the site administrators are doing about an incident in progress, and what went wrong in the last 14 days. Anyone can open it without signing in, and it is linked as **Status** at the bottom of every page. The same information is available as JSON for monitors and scripts. | |
| 7 | + | |
| 8 | +The page is served by the instance itself, so it can't report that the whole instance is down. See [Monitoring from outside](#monitoring-from-outside). | |
| 9 | + | |
| 10 | +## What the page shows | |
| 11 | + | |
| 12 | +From the top: | |
| 13 | + | |
| 14 | +1. **The overall status**: a banner with the worst status of any component, such as "All systems operational" or "Partial outage". | |
| 15 | +2. **Active incidents**: each unresolved incident with its title, impact, the components it affects and its updates, newest first. | |
| 16 | +3. **Components**: each component with its status, and a note when something other than an incident set it. | |
| 17 | +4. **Past incidents**: incidents resolved in the last 14 days, with all their updates. | |
| 18 | + | |
| 19 | +Updates are plain text: Markdown and HTML show as written. | |
| 20 | + | |
| 21 | +## Components | |
| 22 | + | |
| 23 | +| Id | Name | What it covers | | |
| 24 | +| --- | --- | --- | | |
| 25 | +| `web` | Website and API | The web UI, the REST API and the MCP server | | |
| 26 | +| `git` | Git operations | Clone, fetch and push over HTTPS, and Git LFS | | |
| 27 | +| `actions` | Actions | Workflow runs and jobs on hosted and self-hosted runners | | |
| 28 | +| `agents` | Agents and sandboxes | Agent tasks, reviews, web terminals and sandboxes | | |
| 29 | +| `packages` | Packages | The container registry | | |
| 30 | +| `email` | Email | Notification and security emails | | |
| 31 | + | |
| 32 | +## How statuses are derived | |
| 33 | + | |
| 34 | +A component's status uses the names of Atlassian Statuspage, which GitHub's status page uses too. It is the worst of: | |
| 35 | + | |
| 36 | +- the impact of each unresolved incident that names the component: | |
| 37 | + | |
| 38 | + | Incident impact | Component status | | |
| 39 | + | --- | --- | | |
| 40 | + | `critical` | `major_outage` (**Major outage**) | | |
| 41 | + | `major` | `partial_outage` (**Partial outage**) | | |
| 42 | + | `minor` | `degraded_performance` (**Degraded performance**) | | |
| 43 | + | `maintenance` | `under_maintenance` (**Under maintenance**) | | |
| 44 | + | |
| 45 | +- the instance-wide [kill switches](/docs/admin/site-administration#kill-switches): while hosted compute is paused, **Actions** and **Agents and sandboxes** are `under_maintenance`, with the notes "Hosted jobs are paused by the site administrators. Jobs on self-hosted runners still run." and "Agent sandboxes are paused by the site administrators." The pause's reason isn't shown. An incident with a worse impact wins, and the note stays. | |
| 46 | + | |
| 47 | +With neither, a component is `operational`. From best to worst: operational, under maintenance, degraded performance, partial outage, major outage. An incident counts from when it is opened until it is resolved, whatever its state (investigating, identified or monitoring) in between. | |
| 48 | + | |
| 49 | +The overall status follows the worst component: | |
| 50 | + | |
| 51 | +| Worst component | `indicator` | Description | | |
| 52 | +| --- | --- | --- | | |
| 53 | +| `operational` | `none` | All systems operational | | |
| 54 | +| `under_maintenance` | `maintenance` | Service under maintenance | | |
| 55 | +| `degraded_performance` | `minor` | Degraded performance | | |
| 56 | +| `partial_outage` | `major` | Partial outage | | |
| 57 | +| `major_outage` | `critical` | Major outage | | |
| 58 | + | |
| 59 | +When the page can't read the D1 database, it still answers: **Website and API** shows as a major outage with the note "This instance can't read its database right now, so most pages and API calls fail, and incidents can't be shown." and no incidents are listed. | |
| 60 | + | |
| 61 | +## Posting incidents | |
| 62 | + | |
| 63 | +Site administrators manage incidents under **Settings → Status** (`/settings/admin/status`). | |
| 64 | + | |
| 65 | +To open an incident, fill in **Open an incident** and click **Open the incident**: | |
| 66 | + | |
| 67 | +- **Title**: one line, up to 200 characters. | |
| 68 | +- **Impact**: minor, major, critical or maintenance (see the table above). | |
| 69 | +- **Affected components**: at least one. | |
| 70 | +- **State**: investigating, identified or monitoring. | |
| 71 | +- **First update**: what is happening, up to 5,000 characters of plain text. | |
| 72 | + | |
| 73 | +The incident is on the status page at once. Under **Open incidents**, each incident has a form to post an update: pick its new state, write the update and click **Post the update**. Choose **Resolved** to resolve it: its components go back to operational (unless something else affects them) and it moves to past incidents for 14 days. Posting an update with another state to a resolved incident reopens it. | |
| 74 | + | |
| 75 | +Opening, updating and resolving are recorded in the [audit log](/docs/security/audit-log) as `status_incident.create`, `status_incident.update` and `status_incident.resolve`, without the text of the updates. Everything typed into an incident is public: don't put internal details or people's data in it. | |
| 76 | + | |
| 77 | +Diffinery doesn't open incidents by itself, except for the kill switch and database signals above, and it doesn't email anyone about them. | |
| 78 | + | |
| 79 | +## JSON | |
| 80 | + | |
| 81 | +`GET /api/v3/diffinery/status` needs no authentication and is shaped like Statuspage's summary. Shared caches may keep it for 30 seconds (`cache-control: public, max-age=30`). | |
| 82 | + | |
| 83 | +```json | |
| 84 | +{ | |
| 85 | + "status": { "indicator": "major", "description": "Partial outage" }, | |
| 86 | + "components": [ | |
| 87 | + { "id": "web", "name": "Website and API", "status": "operational" }, | |
| 88 | + { "id": "git", "name": "Git operations", "status": "partial_outage" }, | |
| 89 | + { "id": "actions", "name": "Actions", "status": "under_maintenance", "note": "Hosted jobs are paused by the site administrators. Jobs on self-hosted runners still run." } | |
| 90 | + ], | |
| 91 | + "incidents": [ | |
| 92 | + { | |
| 93 | + "id": "4f1c...", | |
| 94 | + "title": "Pushes are failing", | |
| 95 | + "impact": "major", | |
| 96 | + "state": "identified", | |
| 97 | + "components": ["git"], | |
| 98 | + "created_at": "2026-10-14T09:12:00.000Z", | |
| 99 | + "updated_at": "2026-10-14T09:40:00.000Z", | |
| 100 | + "resolved_at": null, | |
| 101 | + "updates": [ | |
| 102 | + { "id": "9a0b...", "state": "identified", "body": "We found the cause and are rolling out a fix.", "created_at": "2026-10-14T09:40:00.000Z" }, | |
| 103 | + { "id": "77de...", "state": "investigating", "body": "We are investigating failed pushes.", "created_at": "2026-10-14T09:12:00.000Z" } | |
| 104 | + ], | |
| 105 | + "html_url": "https://diffinery.com/status#incident-4f1c..." | |
| 106 | + } | |
| 107 | + ], | |
| 108 | + "updated_at": "2026-10-14T09:41:05.120Z" | |
| 109 | +} | |
| 110 | +``` | |
| 111 | + | |
| 112 | +(The example lists three of the six components.) `incidents` holds only unresolved incidents; `updated_at` is when the summary was made. | |
| 113 | + | |
| 114 | +Site administrators manage incidents through the API too. To anyone else these endpoints answer `404`, as if they didn't exist. | |
| 115 | + | |
| 116 | +| Method and path | Description | Needs | | |
| 117 | +| --- | --- | --- | | |
| 118 | +| `GET /api/v3/diffinery/admin/status/incidents` | Incidents, newest first, resolved or not, as in the summary plus `created_by` and each update's `author`. Takes `per_page` (default 50, up to 200). | site administrator, a `read` token not limited to repositories, or a browser session | | |
| 119 | +| `POST /api/v3/diffinery/admin/status/incidents` | `{"title": "...", "impact": "major", "components": ["git"], "state": "investigating", "body": "..."}`: opens an incident with its first update. Answers `201` with the incident. | site administrator, with an `admin` token not limited to repositories, or a browser session | | |
| 120 | +| `POST /api/v3/diffinery/admin/status/incidents/{id}/updates` | `{"state": "resolved", "body": "..."}`: posts an update and moves the incident to its state. Answers `201` with the incident. | site administrator, with an `admin` token not limited to repositories, or a browser session | | |
| 121 | + | |
| 122 | +What is missing or unknown is refused with `422` and a sentence saying what to fix, such as "Choose at least one affected component." | |
| 123 | + | |
| 124 | +```sh | |
| 125 | +curl https://diffinery.com/api/v3/diffinery/status | |
| 126 | +echo '{"title": "Pushes are failing", "impact": "major", "components": ["git"], "state": "investigating", "body": "We are investigating failed pushes."}' | diffinery api POST /diffinery/admin/status/incidents --input - | |
| 127 | +echo '{"state": "resolved", "body": "Pushes work again."}' | diffinery api POST /diffinery/admin/status/incidents/<id>/updates --input - | |
| 128 | +``` | |
| 129 | + | |
| 130 | +## Monitoring from outside | |
| 131 | + | |
| 132 | +The status page runs in the same Worker as everything else. When the whole Worker is down, because of a bad deploy, a Cloudflare incident or an exceeded plan limit, `/status` is down too and can't tell anyone. Watch the instance from outside as well: | |
| 133 | + | |
| 134 | +- Point an external uptime monitor (any service that checks a URL every minute or so) at `https://<your host>/healthz`, which answers `{"ok": true}` without touching the database, and alert yourself when it fails. | |
| 135 | +- Optionally also watch `/api/v3/diffinery/status` and alert when `status.indicator` isn't `none`, to hear about incidents and kill switches. A database failure shows there as a `critical` indicator while `/healthz` still answers. | |
| 136 | + | |
| 137 | +Tell people where to look when the instance doesn't answer at all, for example a status page hosted elsewhere or a chat channel. |
addedapps/diffinery/migrations/0033_status_incidents.sql+27−0View file
| @@ -1,0 +1,27 @@ | ||
| 1 | +-- The status page (lib/status.ts): incidents site administrators open at /status, each with the | |
| 2 | +-- components it affects and its updates. An incident is investigating, identified, monitoring or | |
| 3 | +-- resolved; while it isn't resolved, its impact sets the status of each component it names. | |
| 4 | +CREATE TABLE IF NOT EXISTS status_incidents ( | |
| 5 | + id TEXT PRIMARY KEY, | |
| 6 | + title TEXT NOT NULL, | |
| 7 | + impact TEXT NOT NULL CHECK (impact IN ('minor', 'major', 'critical', 'maintenance')), | |
| 8 | + components TEXT NOT NULL, -- JSON array of component ids, e.g. ["web","git"] | |
| 9 | + state TEXT NOT NULL CHECK (state IN ('investigating', 'identified', 'monitoring', 'resolved')), | |
| 10 | + created_by TEXT, -- the administrator's login | |
| 11 | + created_at TEXT NOT NULL, | |
| 12 | + updated_at TEXT NOT NULL, | |
| 13 | + resolved_at TEXT -- set while the state is resolved | |
| 14 | +); | |
| 15 | +CREATE INDEX IF NOT EXISTS status_incidents_at ON status_incidents(created_at); | |
| 16 | +CREATE INDEX IF NOT EXISTS status_incidents_state ON status_incidents(state, resolved_at); | |
| 17 | + | |
| 18 | +-- What was said about an incident, newest last; the first is written with the incident. | |
| 19 | +CREATE TABLE IF NOT EXISTS status_incident_updates ( | |
| 20 | + id TEXT PRIMARY KEY, | |
| 21 | + incident_id TEXT NOT NULL, | |
| 22 | + state TEXT NOT NULL CHECK (state IN ('investigating', 'identified', 'monitoring', 'resolved')), | |
| 23 | + body TEXT NOT NULL, | |
| 24 | + author_login TEXT, | |
| 25 | + created_at TEXT NOT NULL | |
| 26 | +); | |
| 27 | +CREATE INDEX IF NOT EXISTS status_incident_updates_incident ON status_incident_updates(incident_id, created_at); |
modifiedapps/diffinery/src/app.ts+2−0View file
| @@ -30,6 +30,7 @@ | ||
| 30 | 30 | import { mailEvents, mailEventsQueue } from "./mail/events.js"; |
| 31 | 31 | import { securityTxtRoutes } from "./http/security-txt.js"; |
| 32 | 32 | import { securityReportsApi } from "./http/security-reports.js"; |
| 33 | +import { statusApi } from "./http/status.js"; | |
| 33 | 34 | import { onReviewEvent } from "./agentic/reviews.js"; |
| 34 | 35 | import { onVerifyEvent } from "./agentic/tasks.js"; |
| 35 | 36 | import { recordActivity } from "./agentic/today.js"; |
| @@ -97,6 +98,7 @@ | ||
| 97 | 98 | app.route("/", runner); |
| 98 | 99 | app.route("/", mailApi); |
| 99 | 100 | app.route("/", securityReportsApi); |
| 101 | +app.route("/", statusApi); | |
| 100 | 102 | app.route("/", mailEvents); |
| 101 | 103 | app.route("/", registry); |
| 102 | 104 | app.route("/", packagesApi); |
modifiedapps/diffinery/src/docs/api-meta.ts+35−0View file
| @@ -106,6 +106,9 @@ | ||
| 106 | 106 | "GET /diffinery/admin/security-reports": { scope: "read" }, |
| 107 | 107 | "GET /diffinery/admin/security-reports/{id}": { scope: "read" }, |
| 108 | 108 | "PATCH /diffinery/admin/security-reports/{id}": { scope: "admin" }, |
| 109 | + "GET /diffinery/admin/status/incidents": { scope: "read" }, | |
| 110 | + "POST /diffinery/admin/status/incidents": { scope: "admin" }, | |
| 111 | + "POST /diffinery/admin/status/incidents/{id}/updates": { scope: "admin" }, | |
| 109 | 112 | "PUT /user/avatar": { scope: "write" }, |
| 110 | 113 | "DELETE /user/avatar": { scope: "write" }, |
| 111 | 114 | "PUT /orgs/{org}/avatar": { scope: "write" }, |
| @@ -1572,6 +1575,38 @@ | ||
| 1572 | 1575 | access: "site-admin", |
| 1573 | 1576 | body: { state: ["string", "`triaged`, `resolved`, or `new` to reopen it.", true], note: ["string", "For the other site administrators, up to 2,000 characters."] }, |
| 1574 | 1577 | }, |
| 1578 | + "GET /diffinery/status": { | |
| 1579 | + summary: "Get the instance's status", | |
| 1580 | + description: | |
| 1581 | + "The status page as JSON, shaped like Statuspage's summary: `status` (`indicator`: `none`, `minor`, `major`, `critical` or `maintenance`, and a `description`), `components` (`id`, `name`, `status`: `operational`, `under_maintenance`, `degraded_performance`, `partial_outage` or `major_outage`, and a `note` when a kill switch or a database failure set it), `incidents` (the unresolved ones, newest first, each with its `updates`, newest first) and `updated_at`. Cached for 30 seconds. See [Status page](/docs/self-hosting/status-page).", | |
| 1582 | + access: "public", | |
| 1583 | + }, | |
| 1584 | + "GET /diffinery/admin/status/incidents": { | |
| 1585 | + summary: "List status incidents", | |
| 1586 | + description: "Incidents on the status page, newest first, resolved or not: `id`, `title`, `impact`, `state`, `components`, `created_by`, `created_at`, `updated_at`, `resolved_at`, `updates` (each with `id`, `state`, `body`, `author` and `created_at`, newest first) and `html_url`. 404 for anyone but a site administrator.", | |
| 1587 | + access: "site-admin", | |
| 1588 | + query: { per_page: ["integer", "At most this many (default 50, up to 200)."] }, | |
| 1589 | + }, | |
| 1590 | + "POST /diffinery/admin/status/incidents": { | |
| 1591 | + summary: "Open a status incident", | |
| 1592 | + description: "Opens an incident with its first update and answers with it. While it isn't resolved, its impact sets the status of each component it names: `critical` is a major outage, `major` a partial outage, `minor` degraded performance and `maintenance` under maintenance. Recorded in the audit log as `status_incident.create`. 404 for anyone but a site administrator.", | |
| 1593 | + access: "site-admin", | |
| 1594 | + status: 201, | |
| 1595 | + body: { | |
| 1596 | + title: ["string", "One line, up to 200 characters.", true], | |
| 1597 | + impact: ["string", "`minor`, `major`, `critical` or `maintenance`.", true], | |
| 1598 | + components: ["array", "At least one of `web`, `git`, `actions`, `agents`, `packages` and `email`.", true], | |
| 1599 | + state: ["string", "`investigating`, `identified`, `monitoring` or `resolved`.", true], | |
| 1600 | + body: ["string", "The first update, plain text up to 5,000 characters. It is public.", true], | |
| 1601 | + }, | |
| 1602 | + }, | |
| 1603 | + "POST /diffinery/admin/status/incidents/{id}/updates": { | |
| 1604 | + summary: "Post an update to a status incident", | |
| 1605 | + description: "Adds an update and moves the incident to its state; `resolved` resolves it, and any other state reopens a resolved incident. Answers with the incident. Recorded in the audit log as `status_incident.resolve` when it resolves the incident and `status_incident.update` otherwise. 404 when there is no such incident, or for anyone but a site administrator.", | |
| 1606 | + access: "site-admin", | |
| 1607 | + status: 201, | |
| 1608 | + body: { state: ["string", "`investigating`, `identified`, `monitoring` or `resolved`.", true], body: ["string", "Plain text up to 5,000 characters. It is public.", true] }, | |
| 1609 | + }, | |
| 1575 | 1610 | "GET /diffinery/admin/scheduler": { |
| 1576 | 1611 | summary: "Workflow scheduler state", |
| 1577 | 1612 | description: "`tick_minutes`, `last_tick_at` (when the last check finished), `covered_until` (the last minute whose schedules were started) and `next_tick_at` (null when the timer isn't set). Tells a scheduler that stopped ticking from a run that failed to start.", |
modifiedapps/diffinery/src/docs/reference.ts+3−1View file
| @@ -59,6 +59,7 @@ | ||
| 59 | 59 | ["Webhooks", /\/hooks(\/|$)/], |
| 60 | 60 | ["Packages", /\/packages(\/|$)/], |
| 61 | 61 | ["Imports", /^\/diffinery\/(imports|user\/linked-accounts|admin\/linked-accounts)(\/|$)/], |
| 62 | + ["Status page", /^\/diffinery\/(admin\/)?status(\/|$)/], | |
| 62 | 63 | ["Notifications and email", /^\/notifications$|\/subscription$|\/diffinery\/(notification-settings|emails|admin)/], |
| 63 | 64 | ["Tokens and agents", /\/diffinery\/(tokens|agents)/], |
| 64 | 65 | ["Sandboxes", /\/diffinery\/sandboxes/], |
| @@ -264,7 +265,8 @@ | ||
| 264 | 265 | ["GET, POST /security/report", "The private security report form: signed in, or anonymously after Cloudflare Turnstile. Only site administrators can read reports. See [Reporting a vulnerability](/docs/security/disclosure)."], |
| 265 | 266 | ["POST /-/email/events", "Signed bounce and complaint intake for the email suppression list; 404 until `MAIL_EVENTS_SECRET` is set. See [Site administration](/docs/admin/site-administration#bounces-complaints-and-the-suppression-list)."], |
| 266 | 267 | ["GET, POST /-/unsubscribe/{token}", "One-click unsubscribe from a notification email (RFC 8058): a POST with `List-Unsubscribe=One-Click` turns that kind of email off. See [Notifications](/docs/accounts/notifications)."], |
| 267 | - ["GET /healthz", "Health check: `{\"ok\": true}`."], | |
| 268 | + ["GET /status", "The status page: each component's status and the incidents site administrators posted, for anyone without signing in. As JSON at `GET /api/v3/diffinery/status`. See [Status page](/docs/self-hosting/status-page)."], | |
| 269 | + ["GET /healthz", "Health check: `{\"ok\": true}`. Point an external uptime monitor at it: the status page can't report the whole Worker being down."], | |
| 268 | 270 | ["GET /llms.txt", "Index of these docs for AI agents. See [Docs for AI agents](/docs/agents/docs-for-agents)."], |
| 269 | 271 | ["GET /llms-full.txt", "All of these docs in one Markdown file."], |
| 270 | 272 | ["GET /docs/{page}.md", "Any docs page as Markdown."], |
addedapps/diffinery/src/http/status.ts+59−0View file
| @@ -1,0 +1,59 @@ | ||
| 1 | +import { Hono } from "hono"; | |
| 2 | +import type { Env } from "../env.js"; | |
| 3 | +import { requireAccount, requireAccountScope, type AppContext, type AppVars } from "../auth/principal.js"; | |
| 4 | +import type { Account } from "../lib/directory.js"; | |
| 5 | +import { StatusIncidents, incidentJson, openIncident, postUpdate, statusSummary, summaryJson, validateIncident, validateUpdate } from "../lib/status.js"; | |
| 6 | +import { HttpError } from "../lib/util.js"; | |
| 7 | + | |
| 8 | +/** | |
| 9 | + * The status page over the REST API (lib/status.ts): the public summary, and incidents for site | |
| 10 | + * administrators. To anyone else the admin endpoints don't exist (404), like security reports'. | |
| 11 | + */ | |
| 12 | +export const statusApi = new Hono<{ Bindings: Env; Variables: AppVars }>().basePath("/api/v3"); | |
| 13 | + | |
| 14 | +const siteAdmin = (c: AppContext, scope: "read" | "admin"): Account => { | |
| 15 | + const a = requireAccount(c); | |
| 16 | + if (!a.site_admin || a.kind !== "user") throw new HttpError(404, "not found"); | |
| 17 | + requireAccountScope(c, scope); | |
| 18 | + return a; | |
| 19 | +}; | |
| 20 | + | |
| 21 | +async function body(c: AppContext): Promise<Record<string, unknown>> { | |
| 22 | + let b: unknown; | |
| 23 | + try { | |
| 24 | + b = await c.req.json(); | |
| 25 | + } catch { | |
| 26 | + throw new HttpError(400, "request body must be JSON"); | |
| 27 | + } | |
| 28 | + if (!b || typeof b !== "object" || Array.isArray(b)) throw new HttpError(422, "Send a JSON object."); | |
| 29 | + return b as Record<string, unknown>; | |
| 30 | +} | |
| 31 | + | |
| 32 | +/** Public, for anyone and any monitor; shared caches may keep it for 30 seconds. */ | |
| 33 | +statusApi.get("/diffinery/status", async (c) => { | |
| 34 | + const s = await statusSummary(c.env); | |
| 35 | + return c.json(summaryJson(c.env, s), 200, { "cache-control": "public, max-age=30" }); | |
| 36 | +}); | |
| 37 | + | |
| 38 | +statusApi.get("/diffinery/admin/status/incidents", async (c) => { | |
| 39 | + siteAdmin(c, "read"); | |
| 40 | + const n = Number(c.req.query("per_page") ?? 50); | |
| 41 | + const rows = await new StatusIncidents(c.env.DB).list({ limit: Number.isFinite(n) ? n : 50 }); | |
| 42 | + return c.json(rows.map((i) => incidentJson(c.env, i, { admin: true }))); | |
| 43 | +}); | |
| 44 | + | |
| 45 | +/** Open an incident with its first update. Audited. */ | |
| 46 | +statusApi.post("/diffinery/admin/status/incidents", async (c) => { | |
| 47 | + const me = siteAdmin(c, "admin"); | |
| 48 | + const input = validateIncident(await body(c)); | |
| 49 | + const i = await openIncident(c.env.DB, (e) => c.get("dir").audit(e), me, input, "api"); | |
| 50 | + return c.json(incidentJson(c.env, i, { admin: true }), 201); | |
| 51 | +}); | |
| 52 | + | |
| 53 | +/** Post an update, which moves the incident to its state (resolved resolves it). Audited. */ | |
| 54 | +statusApi.post("/diffinery/admin/status/incidents/:id/updates", async (c) => { | |
| 55 | + const me = siteAdmin(c, "admin"); | |
| 56 | + const input = validateUpdate(await body(c)); | |
| 57 | + const i = await postUpdate(c.env.DB, (e) => c.get("dir").audit(e), me, c.req.param("id"), input, "api"); | |
| 58 | + return c.json(incidentJson(c.env, i, { admin: true }), 201); | |
| 59 | +}); |
addedapps/diffinery/src/lib/status.ts+356−0View file
| @@ -1,0 +1,356 @@ | ||
| 1 | +import type { Env } from "../env.js"; | |
| 2 | +import { computePause } from "./killswitch.js"; | |
| 3 | +import { HttpError, newId, now } from "./util.js"; | |
| 4 | + | |
| 5 | +/** | |
| 6 | + * The status page: /status and GET /api/v3/diffinery/status, public, like githubstatus.com. Site | |
| 7 | + * administrators open incidents that name the components they affect and post updates until they | |
| 8 | + * are resolved. Each component's status is the worst impact of the unresolved incidents naming it, | |
| 9 | + * in Atlassian Statuspage's terms (which GitHub's page uses too). Two signals need no incident: an | |
| 10 | + * instance-wide kill switch (lib/killswitch.ts) puts what it stops under maintenance, and a summary | |
| 11 | + * that can't read D1 shows the website as down. The page is served by this Worker, so it is down | |
| 12 | + * whenever the whole Worker is: an external monitor of /healthz covers that. | |
| 13 | + */ | |
| 14 | + | |
| 15 | +export const COMPONENTS = [ | |
| 16 | + { id: "web", name: "Website and API" }, | |
| 17 | + { id: "git", name: "Git operations" }, | |
| 18 | + { id: "actions", name: "Actions" }, | |
| 19 | + { id: "agents", name: "Agents and sandboxes" }, | |
| 20 | + { id: "packages", name: "Packages" }, | |
| 21 | + { id: "email", name: "Email" }, | |
| 22 | +] as const; | |
| 23 | +export type ComponentId = (typeof COMPONENTS)[number]["id"]; | |
| 24 | +export const COMPONENT_IDS: readonly ComponentId[] = COMPONENTS.map((c) => c.id); | |
| 25 | + | |
| 26 | +export type Impact = "minor" | "major" | "critical" | "maintenance"; | |
| 27 | +export const IMPACTS: readonly Impact[] = ["minor", "major", "critical", "maintenance"]; | |
| 28 | +export type IncidentState = "investigating" | "identified" | "monitoring" | "resolved"; | |
| 29 | +export const INCIDENT_STATES: readonly IncidentState[] = ["investigating", "identified", "monitoring", "resolved"]; | |
| 30 | + | |
| 31 | +export type ComponentStatus = "operational" | "under_maintenance" | "degraded_performance" | "partial_outage" | "major_outage"; | |
| 32 | +export type Indicator = "none" | "minor" | "major" | "critical" | "maintenance"; | |
| 33 | + | |
| 34 | +/** Longest title and update, in characters. */ | |
| 35 | +export const MAX_TITLE = 200; | |
| 36 | +export const MAX_BODY = 5_000; | |
| 37 | +/** How long a resolved incident stays on the page, in days. */ | |
| 38 | +export const RECENT_DAYS = 14; | |
| 39 | + | |
| 40 | +/** What an unresolved incident of each impact does to the components it names. */ | |
| 41 | +export const IMPACT_STATUS: Record<Impact, ComponentStatus> = { critical: "major_outage", major: "partial_outage", minor: "degraded_performance", maintenance: "under_maintenance" }; | |
| 42 | +/** Worse is higher: maintenance is better than any outage, so an incident during one still shows. */ | |
| 43 | +const RANK: Record<ComponentStatus, number> = { operational: 0, under_maintenance: 1, degraded_performance: 2, partial_outage: 3, major_outage: 4 }; | |
| 44 | +/** The page's overall indicator and description, by its worst component. */ | |
| 45 | +const OVERALL: Record<ComponentStatus, { indicator: Indicator; description: string }> = { | |
| 46 | + operational: { indicator: "none", description: "All systems operational" }, | |
| 47 | + under_maintenance: { indicator: "maintenance", description: "Service under maintenance" }, | |
| 48 | + degraded_performance: { indicator: "minor", description: "Degraded performance" }, | |
| 49 | + partial_outage: { indicator: "major", description: "Partial outage" }, | |
| 50 | + major_outage: { indicator: "critical", description: "Major outage" }, | |
| 51 | +}; | |
| 52 | + | |
| 53 | +export interface IncidentUpdate { | |
| 54 | + id: string; | |
| 55 | + incident_id: string; | |
| 56 | + state: IncidentState; | |
| 57 | + body: string; | |
| 58 | + author_login: string | null; | |
| 59 | + created_at: string; | |
| 60 | +} | |
| 61 | + | |
| 62 | +export interface Incident { | |
| 63 | + id: string; | |
| 64 | + title: string; | |
| 65 | + impact: Impact; | |
| 66 | + components: ComponentId[]; | |
| 67 | + state: IncidentState; | |
| 68 | + created_by: string | null; | |
| 69 | + created_at: string; | |
| 70 | + updated_at: string; | |
| 71 | + resolved_at: string | null; | |
| 72 | + /** Newest first. */ | |
| 73 | + updates: IncidentUpdate[]; | |
| 74 | +} | |
| 75 | + | |
| 76 | +type IncidentRow = Omit<Incident, "components" | "updates"> & { components: string }; | |
| 77 | + | |
| 78 | +/** Something other than an incident that sets a component's status, with a note saying why. */ | |
| 79 | +export interface Signal { | |
| 80 | + component: ComponentId; | |
| 81 | + status: ComponentStatus; | |
| 82 | + note: string; | |
| 83 | +} | |
| 84 | + | |
| 85 | +export interface ComponentState { | |
| 86 | + id: ComponentId; | |
| 87 | + name: string; | |
| 88 | + status: ComponentStatus; | |
| 89 | + note?: string; | |
| 90 | +} | |
| 91 | + | |
| 92 | +export interface Summary { | |
| 93 | + status: { indicator: Indicator; description: string }; | |
| 94 | + components: ComponentState[]; | |
| 95 | + /** Unresolved incidents, newest first. */ | |
| 96 | + incidents: Incident[]; | |
| 97 | + /** Incidents resolved in the last RECENT_DAYS days, most recently resolved first. */ | |
| 98 | + resolved: Incident[]; | |
| 99 | + updated_at: string; | |
| 100 | +} | |
| 101 | + | |
| 102 | +// ---------------------------------------------------------------- deriving statuses | |
| 103 | + | |
| 104 | +/** | |
| 105 | + * Each component's status: the worst of the unresolved incidents naming it and of the signals for | |
| 106 | + * it, operational otherwise. A signal's note stays even when an incident is worse; it is still true. | |
| 107 | + */ | |
| 108 | +export function componentStates(incidents: readonly Incident[], signals: readonly Signal[]): ComponentState[] { | |
| 109 | + return COMPONENTS.map(({ id, name }) => { | |
| 110 | + let status: ComponentStatus = "operational"; | |
| 111 | + const worse = (s: ComponentStatus) => { | |
| 112 | + if (RANK[s] > RANK[status]) status = s; | |
| 113 | + }; | |
| 114 | + for (const i of incidents) if (i.state !== "resolved" && i.components.includes(id)) worse(IMPACT_STATUS[i.impact]); | |
| 115 | + const notes: string[] = []; | |
| 116 | + for (const s of signals) { | |
| 117 | + if (s.component !== id) continue; | |
| 118 | + worse(s.status); | |
| 119 | + notes.push(s.note); | |
| 120 | + } | |
| 121 | + return notes.length ? { id, name, status, note: notes.join(" ") } : { id, name, status }; | |
| 122 | + }); | |
| 123 | +} | |
| 124 | + | |
| 125 | +/** The overall indicator and description, from the worst component. */ | |
| 126 | +export function overall(components: readonly ComponentState[]): Summary["status"] { | |
| 127 | + const worst = components.reduce<ComponentStatus>((w, c) => (RANK[c.status] > RANK[w] ? c.status : w), "operational"); | |
| 128 | + return { ...OVERALL[worst] }; | |
| 129 | +} | |
| 130 | + | |
| 131 | +/** Which instance-wide kill switches are on. */ | |
| 132 | +export interface InstanceSwitches { | |
| 133 | + hostedJobs: boolean; | |
| 134 | + selfHostedJobs: boolean; | |
| 135 | + sandboxes: boolean; | |
| 136 | +} | |
| 137 | + | |
| 138 | +/** Kill switches on for the whole instance, as signals: what they stop is under maintenance. */ | |
| 139 | +export function killSwitchSignals(k: InstanceSwitches): Signal[] { | |
| 140 | + const out: Signal[] = []; | |
| 141 | + const jobs = | |
| 142 | + k.hostedJobs && k.selfHostedJobs | |
| 143 | + ? "Actions jobs are paused by the site administrators." | |
| 144 | + : k.hostedJobs | |
| 145 | + ? "Hosted jobs are paused by the site administrators. Jobs on self-hosted runners still run." | |
| 146 | + : k.selfHostedJobs | |
| 147 | + ? "Jobs on self-hosted runners are paused by the site administrators." | |
| 148 | + : null; | |
| 149 | + if (jobs) out.push({ component: "actions", status: "under_maintenance", note: jobs }); | |
| 150 | + if (k.sandboxes) out.push({ component: "agents", status: "under_maintenance", note: "Agent sandboxes are paused by the site administrators." }); | |
| 151 | + return out; | |
| 152 | +} | |
| 153 | + | |
| 154 | +/** The instance's kill switches. Pausing hosted compute stops hosted jobs and sandboxes; self-hosted runners have no instance-wide switch. */ | |
| 155 | +export async function instanceSwitches(env: Env): Promise<InstanceSwitches> { | |
| 156 | + const paused = !!(await computePause(env)); | |
| 157 | + return { hostedJobs: paused, selfHostedJobs: false, sandboxes: paused }; | |
| 158 | +} | |
| 159 | + | |
| 160 | +/** The note on the website when the summary couldn't read the database. */ | |
| 161 | +export const DB_DOWN = "This instance can't read its database right now, so most pages and API calls fail, and incidents can't be shown."; | |
| 162 | + | |
| 163 | +/** | |
| 164 | + * Everything the page shows. When D1 can't be read the summary still comes back, with the website | |
| 165 | + * as a major outage and no incidents, so the page says something useful while the rest is down. | |
| 166 | + */ | |
| 167 | +export async function statusSummary(env: Env, at = new Date()): Promise<Summary> { | |
| 168 | + const updated_at = at.toISOString(); | |
| 169 | + try { | |
| 170 | + const store = new StatusIncidents(env.DB); | |
| 171 | + const [incidents, resolved, switches] = await Promise.all([store.active(), store.resolvedSince(new Date(at.getTime() - RECENT_DAYS * 86400_000).toISOString()), instanceSwitches(env)]); | |
| 172 | + const components = componentStates(incidents, killSwitchSignals(switches)); | |
| 173 | + return { status: overall(components), components, incidents, resolved, updated_at }; | |
| 174 | + } catch (e) { | |
| 175 | + console.error("status page: reading D1 failed", e); | |
| 176 | + const components = componentStates([], [{ component: "web", status: "major_outage", note: DB_DOWN }]); | |
| 177 | + return { status: overall(components), components, incidents: [], resolved: [], updated_at }; | |
| 178 | + } | |
| 179 | +} | |
| 180 | + | |
| 181 | +// ---------------------------------------------------------------- validation | |
| 182 | + | |
| 183 | +export interface IncidentInput { | |
| 184 | + title: string; | |
| 185 | + impact: Impact; | |
| 186 | + components: ComponentId[]; | |
| 187 | + state: IncidentState; | |
| 188 | + body: string; | |
| 189 | +} | |
| 190 | + | |
| 191 | +export interface UpdateInput { | |
| 192 | + state: IncidentState; | |
| 193 | + body: string; | |
| 194 | +} | |
| 195 | + | |
| 196 | +const str = (v: unknown) => (typeof v === "string" ? v : ""); | |
| 197 | + | |
| 198 | +function checkState(v: unknown): IncidentState { | |
| 199 | + if (!INCIDENT_STATES.includes(v as IncidentState)) throw new HttpError(422, "Choose the incident's state: investigating, identified, monitoring or resolved."); | |
| 200 | + return v as IncidentState; | |
| 201 | +} | |
| 202 | + | |
| 203 | +function checkBody(v: unknown): string { | |
| 204 | + // Line endings as the browser sent them, but no leading or trailing blank lines. | |
| 205 | + const body = str(v).replace(/\r\n?/g, "\n").replace(/^\s*\n|\s+$/g, ""); | |
| 206 | + if (!body) throw new HttpError(422, "Write an update that says what is happening."); | |
| 207 | + if (body.length > MAX_BODY) throw new HttpError(422, `Keep the update to ${MAX_BODY.toLocaleString("en")} characters.`); | |
| 208 | + return body; | |
| 209 | +} | |
| 210 | + | |
| 211 | +/** An update as sent: a state and a message, both required. */ | |
| 212 | +export function validateUpdate(raw: { state?: unknown; body?: unknown }): UpdateInput { | |
| 213 | + return { state: checkState(raw.state), body: checkBody(raw.body) }; | |
| 214 | +} | |
| 215 | + | |
| 216 | +/** | |
| 217 | + * A new incident as sent: a one-line title, its impact, the components it affects (at least one, | |
| 218 | + * each a known id, kept in the page's order) and its first update. Too long is refused, never cut. | |
| 219 | + */ | |
| 220 | +export function validateIncident(raw: { title?: unknown; impact?: unknown; components?: unknown; state?: unknown; body?: unknown }): IncidentInput { | |
| 221 | + const title = str(raw.title).replace(/\s+/g, " ").trim(); | |
| 222 | + if (!title) throw new HttpError(422, "Give the incident a title."); | |
| 223 | + if (title.length > MAX_TITLE) throw new HttpError(422, `Keep the title to ${MAX_TITLE} characters; the details go in the update.`); | |
| 224 | + if (!IMPACTS.includes(raw.impact as Impact)) throw new HttpError(422, "Choose the incident's impact: minor, major, critical or maintenance."); | |
| 225 | + const list = Array.isArray(raw.components) ? raw.components : []; | |
| 226 | + if (!list.length) throw new HttpError(422, "Choose at least one affected component."); | |
| 227 | + for (const c of list) if (!COMPONENT_IDS.includes(c as ComponentId)) throw new HttpError(422, `There is no component called "${String(c).slice(0, 40)}". Choose from ${COMPONENT_IDS.join(", ")}.`); | |
| 228 | + const components = COMPONENT_IDS.filter((id) => list.includes(id)); | |
| 229 | + const { state, body } = validateUpdate(raw); | |
| 230 | + return { title, impact: raw.impact as Impact, components, state, body }; | |
| 231 | +} | |
| 232 | + | |
| 233 | +// ---------------------------------------------------------------- storage | |
| 234 | + | |
| 235 | +const parseComponents = (s: string): ComponentId[] => { | |
| 236 | + try { | |
| 237 | + const v = JSON.parse(s) as unknown; | |
| 238 | + return Array.isArray(v) ? COMPONENT_IDS.filter((id) => v.includes(id)) : []; | |
| 239 | + } catch { | |
| 240 | + return []; | |
| 241 | + } | |
| 242 | +}; | |
| 243 | + | |
| 244 | +/** Status incidents and their updates in D1. */ | |
| 245 | +export class StatusIncidents { | |
| 246 | + constructor(private db: D1Database) {} | |
| 247 | + | |
| 248 | + /** Incidents from `rows`, each with its updates, newest first. */ | |
| 249 | + private async withUpdates(rows: IncidentRow[]): Promise<Incident[]> { | |
| 250 | + if (!rows.length) return []; | |
| 251 | + const updates = ( | |
| 252 | + await this.db | |
| 253 | + .prepare("SELECT * FROM status_incident_updates WHERE incident_id IN (SELECT value FROM json_each(?)) ORDER BY created_at DESC, rowid DESC") | |
| 254 | + .bind(JSON.stringify(rows.map((r) => r.id))) | |
| 255 | + .all<IncidentUpdate>() | |
| 256 | + ).results; | |
| 257 | + return rows.map((r) => ({ ...r, components: parseComponents(r.components), updates: updates.filter((u) => u.incident_id === r.id) })); | |
| 258 | + } | |
| 259 | + | |
| 260 | + async get(id: string): Promise<Incident | null> { | |
| 261 | + if (!/^[0-9a-f]{32}$/.test(id)) return null; | |
| 262 | + const row = await this.db.prepare("SELECT * FROM status_incidents WHERE id = ?").bind(id).first<IncidentRow>(); | |
| 263 | + return row ? (await this.withUpdates([row]))[0]! : null; | |
| 264 | + } | |
| 265 | + | |
| 266 | + /** Every incident, newest first. */ | |
| 267 | + async list(o: { limit?: number } = {}): Promise<Incident[]> { | |
| 268 | + const limit = Math.min(Math.max(o.limit ?? 50, 1), 200); | |
| 269 | + return this.withUpdates((await this.db.prepare("SELECT * FROM status_incidents ORDER BY created_at DESC, rowid DESC LIMIT ?").bind(limit).all<IncidentRow>()).results); | |
| 270 | + } | |
| 271 | + | |
| 272 | + /** Unresolved incidents, newest first. */ | |
| 273 | + async active(): Promise<Incident[]> { | |
| 274 | + return this.withUpdates((await this.db.prepare("SELECT * FROM status_incidents WHERE state != 'resolved' ORDER BY created_at DESC, rowid DESC LIMIT 50").all<IncidentRow>()).results); | |
| 275 | + } | |
| 276 | + | |
| 277 | + /** Incidents resolved after `since`, most recently resolved first. */ | |
| 278 | + async resolvedSince(since: string): Promise<Incident[]> { | |
| 279 | + return this.withUpdates((await this.db.prepare("SELECT * FROM status_incidents WHERE state = 'resolved' AND resolved_at > ? ORDER BY resolved_at DESC, rowid DESC LIMIT 50").bind(since).all<IncidentRow>()).results); | |
| 280 | + } | |
| 281 | + | |
| 282 | + /** Store an incident with its first update, together. */ | |
| 283 | + async create(input: IncidentInput, by: string): Promise<Incident> { | |
| 284 | + const id = newId(); | |
| 285 | + const at = now(); | |
| 286 | + await this.db.batch([ | |
| 287 | + this.db | |
| 288 | + .prepare("INSERT INTO status_incidents (id, title, impact, components, state, created_by, created_at, updated_at, resolved_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)") | |
| 289 | + .bind(id, input.title, input.impact, JSON.stringify(input.components), input.state, by, at, at, input.state === "resolved" ? at : null), | |
| 290 | + this.db.prepare("INSERT INTO status_incident_updates (id, incident_id, state, body, author_login, created_at) VALUES (?, ?, ?, ?, ?, ?)").bind(newId(), id, input.state, input.body, by, at), | |
| 291 | + ]); | |
| 292 | + return (await this.get(id))!; | |
| 293 | + } | |
| 294 | + | |
| 295 | + /** | |
| 296 | + * Add an update and move the incident to its state. Resolving sets resolved_at (once); any other | |
| 297 | + * state clears it, so an incident resolved too soon can be reopened. | |
| 298 | + */ | |
| 299 | + async addUpdate(id: string, input: UpdateInput, by: string): Promise<void> { | |
| 300 | + const at = now(); | |
| 301 | + await this.db.batch([ | |
| 302 | + this.db.prepare("INSERT INTO status_incident_updates (id, incident_id, state, body, author_login, created_at) VALUES (?, ?, ?, ?, ?, ?)").bind(newId(), id, input.state, input.body, by, at), | |
| 303 | + this.db.prepare("UPDATE status_incidents SET state = ?1, updated_at = ?2, resolved_at = CASE WHEN ?1 = 'resolved' THEN coalesce(resolved_at, ?2) ELSE NULL END WHERE id = ?3").bind(input.state, at, id), | |
| 304 | + ]); | |
| 305 | + } | |
| 306 | +} | |
| 307 | + | |
| 308 | +// ---------------------------------------------------------------- changes (REST and web) | |
| 309 | + | |
| 310 | +type Actor = { id: string; login: string }; | |
| 311 | +type Audit = (e: { actorId?: string; actorLogin?: string; action: string; target?: string; detail?: unknown }) => Promise<void>; | |
| 312 | + | |
| 313 | +/** Open an incident with its first update, audited as `status_incident.create` (without the update's text). */ | |
| 314 | +export async function openIncident(db: D1Database, audit: Audit, me: Actor, input: IncidentInput, via: "web" | "api"): Promise<Incident> { | |
| 315 | + const i = await new StatusIncidents(db).create(input, me.login); | |
| 316 | + await audit({ actorId: me.id, actorLogin: me.login, action: "status_incident.create", target: i.id, detail: { title: i.title, impact: i.impact, components: i.components, state: i.state, via } }); | |
| 317 | + return i; | |
| 318 | +} | |
| 319 | + | |
| 320 | +/** | |
| 321 | + * Post an update to an incident, audited as `status_incident.resolve` when it resolves it and | |
| 322 | + * `status_incident.update` otherwise. 404 when there is no such incident. | |
| 323 | + */ | |
| 324 | +export async function postUpdate(db: D1Database, audit: Audit, me: Actor, id: string, input: UpdateInput, via: "web" | "api"): Promise<Incident> { | |
| 325 | + const store = new StatusIncidents(db); | |
| 326 | + const before = await store.get(id); | |
| 327 | + if (!before) throw new HttpError(404, "There is no incident with that id."); | |
| 328 | + await store.addUpdate(id, input, me.login); | |
| 329 | + await audit({ actorId: me.id, actorLogin: me.login, action: input.state === "resolved" ? "status_incident.resolve" : "status_incident.update", target: id, detail: { from: before.state, to: input.state, via } }); | |
| 330 | + return (await store.get(id))!; | |
| 331 | +} | |
| 332 | + | |
| 333 | +// ---------------------------------------------------------------- JSON | |
| 334 | + | |
| 335 | +/** The API's view of an incident. Who wrote it only for site administrators. */ | |
| 336 | +export const incidentJson = (env: Pick<Env, "PUBLIC_URL">, i: Incident, o: { admin?: boolean } = {}) => ({ | |
| 337 | + id: i.id, | |
| 338 | + title: i.title, | |
| 339 | + impact: i.impact, | |
| 340 | + state: i.state, | |
| 341 | + components: i.components, | |
| 342 | + ...(o.admin ? { created_by: i.created_by } : {}), | |
| 343 | + created_at: i.created_at, | |
| 344 | + updated_at: i.updated_at, | |
| 345 | + resolved_at: i.resolved_at, | |
| 346 | + updates: i.updates.map((u) => ({ id: u.id, state: u.state, body: u.body, ...(o.admin ? { author: u.author_login } : {}), created_at: u.created_at })), | |
| 347 | + html_url: `${env.PUBLIC_URL}/status#incident-${i.id}`, | |
| 348 | +}); | |
| 349 | + | |
| 350 | +/** The public summary, shaped like Statuspage's /api/v2/summary.json. */ | |
| 351 | +export const summaryJson = (env: Pick<Env, "PUBLIC_URL">, s: Summary) => ({ | |
| 352 | + status: s.status, | |
| 353 | + components: s.components.map((c) => ({ ...c })), | |
| 354 | + incidents: s.incidents.map((i) => incidentJson(env, i)), | |
| 355 | + updated_at: s.updated_at, | |
| 356 | +}); |
modifiedapps/diffinery/src/lib/util.ts+1−1View file
| @@ -79,7 +79,7 @@ | ||
| 79 | 79 | } |
| 80 | 80 | |
| 81 | 81 | /** Top-level paths the app serves; no account may use them as its login. */ |
| 82 | -export const RESERVED_LOGINS = new Set(["api", "mcp", "runner", "settings", "new", "notifications", "invite", "request-access", "healthz", "twirp", "login", "logout", "admin", "explore", "favicon.ico", "robots.txt", "about", "docs", "static", "assets", "www", "git", "diffinery", "auth", "signin", "signup", "security", "v2", "imports", "-", "diffinery-agent-review", "ghost", "github-actions", "diffinery-actions", "noreply", "no-reply", "support", "abuse", "postmaster", "root", "staff"]); | |
| 82 | +export const RESERVED_LOGINS = new Set(["api", "mcp", "runner", "settings", "new", "notifications", "invite", "request-access", "healthz", "twirp", "login", "logout", "admin", "explore", "favicon.ico", "robots.txt", "about", "docs", "static", "assets", "www", "git", "diffinery", "auth", "signin", "signup", "security", "status", "v2", "imports", "-", "diffinery-agent-review", "ghost", "github-actions", "diffinery-actions", "noreply", "no-reply", "support", "abuse", "postmaster", "root", "staff"]); | |
| 83 | 83 | |
| 84 | 84 | /** |
| 85 | 85 | * Who a deleted account's issues, pull requests and comments are attributed to, as on GitHub. Not an account: |
modifiedapps/diffinery/src/web/account.tsx+5−2View file
| @@ -646,9 +646,9 @@ | ||
| 646 | 646 | |
| 647 | 647 | // ---------------------------------------------------------------- /settings |
| 648 | 648 | |
| 649 | -export type SettingsTab = "profile" | "account" | "appearance" | "security" | "tokens" | "agents" | "model-access" | "orgs" | "invitations" | "notifications" | "email-address" | "blocked" | "linked" | "admin" | "reports" | "email" | "audit" | "kill-switches" | "errors" | "security-reports"; | |
| 649 | +export type SettingsTab = "profile" | "account" | "appearance" | "security" | "tokens" | "agents" | "model-access" | "orgs" | "invitations" | "notifications" | "email-address" | "blocked" | "linked" | "admin" | "reports" | "email" | "audit" | "kill-switches" | "errors" | "security-reports" | "status"; | |
| 650 | 650 | /** The site administration pages: they show when hosted compute is paused. */ |
| 651 | -export const ADMIN_TABS: readonly SettingsTab[] = ["admin", "reports", "email", "audit", "kill-switches", "errors", "security-reports"]; | |
| 651 | +export const ADMIN_TABS: readonly SettingsTab[] = ["admin", "reports", "email", "audit", "kill-switches", "errors", "security-reports", "status"]; | |
| 652 | 652 | |
| 653 | 653 | export const SettingsNav: FC<{ me: Account; tab: SettingsTab }> = ({ me, tab }) => ( |
| 654 | 654 | <aside class="side vt-side"> |
| @@ -719,6 +719,9 @@ | ||
| 719 | 719 | <a href="/settings/admin/security-reports" class={tab === "security-reports" ? "on" : ""}> |
| 720 | 720 | Security reports |
| 721 | 721 | </a> |
| 722 | + <a href="/settings/admin/status" class={tab === "status" ? "on" : ""}> | |
| 723 | + Status | |
| 724 | + </a> | |
| 722 | 725 | <a href="/settings/audit" class={tab === "audit" ? "on" : ""}> |
| 723 | 726 | Audit log |
| 724 | 727 | </a> |
modifiedapps/diffinery/src/web/assets.ts+2−0View file
| @@ -73,6 +73,8 @@ | ||
| 73 | 73 | .flash.error{border-color:color-mix(in srgb,var(--red) 40%,transparent);background:var(--red-soft);color:var(--red)} |
| 74 | 74 | .flash.notice{border-color:color-mix(in srgb,var(--green) 40%,transparent);background:var(--green-soft);color:var(--green)} |
| 75 | 75 | .flash.info{border-color:color-mix(in srgb,var(--blue) 40%,transparent);background:var(--blue-soft)} |
| 76 | +.flash.warn{border-color:color-mix(in srgb,var(--amber) 40%,transparent);background:var(--amber-soft);color:var(--amber)} | |
| 77 | +.site-foot{display:flex;justify-content:center;gap:20px;max-width:1280px;margin:0 auto;padding:16px 20px 32px;border-top:1px solid var(--border2);font-size:12px}.site-foot a{color:var(--fg2)}.site-foot a:hover{color:var(--accent)} | |
| 76 | 78 | /* Controls follow Kumo sizes: base 36px (buttons, inputs, selects), sm 26px. */ |
| 77 | 79 | .btn{display:inline-flex;align-items:center;justify-content:center;gap:6px;height:36px;padding:0 12px;border:1px solid var(--border);border-radius:var(--r);background:var(--bg);color:var(--fg);font:500 14px/1 var(--sans);cursor:pointer;white-space:nowrap;text-decoration:none!important;vertical-align:middle;flex:none} |
| 78 | 80 | .btn svg{flex:none} |
modifiedapps/diffinery/src/web/routes.tsx+3−0View file
| @@ -10,6 +10,7 @@ | ||
| 10 | 10 | import { registerTerminals } from "../agentic/terminal-web.js"; |
| 11 | 11 | import { registerMailWeb } from "../mail/web.js"; |
| 12 | 12 | import { registerSecurityReports } from "./security-reports.js"; |
| 13 | +import { registerStatus } from "./status.js"; | |
| 13 | 14 | import { registerAuth } from "./auth.js"; |
| 14 | 15 | import { registerActions } from "./actions.js"; |
| 15 | 16 | import { registerCode } from "./code.js"; |
| @@ -48,6 +49,8 @@ | ||
| 48 | 49 | registerPackages(web); |
| 49 | 50 | registerMailWeb(web); |
| 50 | 51 | registerSecurityReports(web); |
| 52 | +// /status: before the /:owner catch-all. | |
| 53 | +registerStatus(web); | |
| 51 | 54 | // Before registerAccount: its /:owner catch-all would otherwise swallow /imports, /new/import and /settings/linked-accounts. |
| 52 | 55 | registerImports(web); |
| 53 | 56 | // Settings → Model access and /:owner/:repo/tasks: before the /:owner and /:owner/:repo catch-alls. |
addedapps/diffinery/src/web/status.tsx+289−0View file
| @@ -1,0 +1,289 @@ | ||
| 1 | +/** @jsxImportSource hono/jsx */ | |
| 2 | +import type { Child, FC } from "hono/jsx"; | |
| 3 | +import { requireAccount, type AppContext } from "../auth/principal.js"; | |
| 4 | +import type { Account } from "../lib/directory.js"; | |
| 5 | +import { | |
| 6 | + COMPONENTS, | |
| 7 | + IMPACTS, | |
| 8 | + INCIDENT_STATES, | |
| 9 | + MAX_BODY, | |
| 10 | + MAX_TITLE, | |
| 11 | + RECENT_DAYS, | |
| 12 | + StatusIncidents, | |
| 13 | + openIncident, | |
| 14 | + postUpdate, | |
| 15 | + statusSummary, | |
| 16 | + validateIncident, | |
| 17 | + validateUpdate, | |
| 18 | + type ComponentStatus, | |
| 19 | + type Impact, | |
| 20 | + type Incident, | |
| 21 | + type IncidentState, | |
| 22 | + type Indicator, | |
| 23 | +} from "../lib/status.js"; | |
| 24 | +import { HttpError } from "../lib/util.js"; | |
| 25 | +import { SettingsNav } from "./account.js"; | |
| 26 | +import { csrfToken } from "./csrf.js"; | |
| 27 | +import { ComputePausedBanner } from "./kill-switches.js"; | |
| 28 | +import { KCheckbox, KSelect } from "./kumo.js"; | |
| 29 | +import { Form, Time, act, post, prm, render, type WebApp } from "./ui.js"; | |
| 30 | + | |
| 31 | +/** | |
| 32 | + * The status page in the web UI (lib/status.ts): the public page at /status, which anyone can open | |
| 33 | + * without signing in, and the **Status** page of site administration, where incidents are opened, | |
| 34 | + * updated and resolved. | |
| 35 | + */ | |
| 36 | + | |
| 37 | +const STATUS_TEXT: Record<ComponentStatus, string> = { | |
| 38 | + operational: "Operational", | |
| 39 | + under_maintenance: "Under maintenance", | |
| 40 | + degraded_performance: "Degraded performance", | |
| 41 | + partial_outage: "Partial outage", | |
| 42 | + major_outage: "Major outage", | |
| 43 | +}; | |
| 44 | +const STATUS_BADGE: Record<ComponentStatus, string> = { operational: "badge green", under_maintenance: "badge blue", degraded_performance: "badge amber", partial_outage: "badge red", major_outage: "badge red" }; | |
| 45 | +const BANNER: Record<Indicator, string> = { none: "flash notice", maintenance: "flash info", minor: "flash warn", major: "flash error", critical: "flash error" }; | |
| 46 | +const IMPACT_TEXT: Record<Impact, string> = { minor: "Minor", major: "Major", critical: "Critical", maintenance: "Maintenance" }; | |
| 47 | +const IMPACT_BADGE: Record<Impact, string> = { minor: "badge amber", major: "badge red", critical: "badge red", maintenance: "badge blue" }; | |
| 48 | +const STATE_TEXT: Record<IncidentState, string> = { investigating: "Investigating", identified: "Identified", monitoring: "Monitoring", resolved: "Resolved" }; | |
| 49 | +const NAMES = new Map<string, string>(COMPONENTS.map((c) => [c.id, c.name])); | |
| 50 | + | |
| 51 | +/** An incident with its updates, newest first. Site administrators also see who wrote each one. */ | |
| 52 | +const IncidentBox: FC<{ i: Incident; admin?: boolean; children?: Child }> = ({ i, admin, children }) => ( | |
| 53 | + <div class="box mb" id={`incident-${i.id}`} data-incident={i.id}> | |
| 54 | + <div class="body"> | |
| 55 | + <h3 class="m0"> | |
| 56 | + {i.title} <span class={IMPACT_BADGE[i.impact]}>{IMPACT_TEXT[i.impact]}</span> | |
| 57 | + </h3> | |
| 58 | + <div class="muted small"> | |
| 59 | + Affects {i.components.map((c) => NAMES.get(c) ?? c).join(", ")}. {i.resolved_at ? <>Resolved </> : <>Opened </>} | |
| 60 | + <Time at={i.resolved_at ?? i.created_at} />. | |
| 61 | + </div> | |
| 62 | + </div> | |
| 63 | + <ul class="list"> | |
| 64 | + {i.updates.map((u) => ( | |
| 65 | + <li> | |
| 66 | + <div class="grow"> | |
| 67 | + <b>{STATE_TEXT[u.state]}</b>{" "} | |
| 68 | + <span class="muted small"> | |
| 69 | + <Time at={u.created_at} /> | |
| 70 | + {admin && u.author_login ? <> by {u.author_login}</> : null} | |
| 71 | + </span> | |
| 72 | + {/* Plain text exactly as written: never rendered as Markdown or HTML. */} | |
| 73 | + <div style="white-space:pre-wrap;overflow-wrap:anywhere;margin-top:4px">{u.body}</div> | |
| 74 | + </div> | |
| 75 | + </li> | |
| 76 | + ))} | |
| 77 | + </ul> | |
| 78 | + {children} | |
| 79 | + </div> | |
| 80 | +); | |
| 81 | + | |
| 82 | +export function registerStatus(web: WebApp): void { | |
| 83 | + // ------------------------------------------------------------ the public page | |
| 84 | + | |
| 85 | + web.get("/status", async (c) => { | |
| 86 | + const s = await statusSummary(c.env); | |
| 87 | + return render( | |
| 88 | + c, | |
| 89 | + { title: "Status", description: "Whether this Diffinery instance is working: each component's status, incidents in progress and recent ones." }, | |
| 90 | + <div style="max-width:820px;margin:32px auto"> | |
| 91 | + <h1>Status</h1> | |
| 92 | + <div class={BANNER[s.status.indicator]} role="status" data-indicator={s.status.indicator}> | |
| 93 | + <b>{s.status.description}</b> | |
| 94 | + </div> | |
| 95 | + {s.incidents.length ? ( | |
| 96 | + <> | |
| 97 | + <h2>Active incidents</h2> | |
| 98 | + {s.incidents.map((i) => ( | |
| 99 | + <IncidentBox i={i} /> | |
| 100 | + ))} | |
| 101 | + </> | |
| 102 | + ) : null} | |
| 103 | + <h2>Components</h2> | |
| 104 | + <div class="box mb"> | |
| 105 | + <ul class="list"> | |
| 106 | + {s.components.map((comp) => ( | |
| 107 | + <li data-component={comp.id} data-status={comp.status}> | |
| 108 | + <div class="grow"> | |
| 109 | + <b>{comp.name}</b> | |
| 110 | + {comp.note ? <div class="meta">{comp.note}</div> : null} | |
| 111 | + </div> | |
| 112 | + <span class={STATUS_BADGE[comp.status]}>{STATUS_TEXT[comp.status]}</span> | |
| 113 | + </li> | |
| 114 | + ))} | |
| 115 | + </ul> | |
| 116 | + </div> | |
| 117 | + <h2>Past incidents</h2> | |
| 118 | + {s.resolved.length ? ( | |
| 119 | + s.resolved.map((i) => <IncidentBox i={i} />) | |
| 120 | + ) : ( | |
| 121 | + <div class="box mb"> | |
| 122 | + <div class="empty">No incidents in the last {RECENT_DAYS} days.</div> | |
| 123 | + </div> | |
| 124 | + )} | |
| 125 | + <p class="muted small"> | |
| 126 | + Updated <Time at={s.updated_at} />. Also as JSON at <a href="/api/v3/diffinery/status">/api/v3/diffinery/status</a>. This page is served by this instance itself, so it can't load when the whole instance is down. | |
| 127 | + </p> | |
| 128 | + </div>, | |
| 129 | + ); | |
| 130 | + }); | |
| 131 | + | |
| 132 | + // ------------------------------------------------------------ site administration: Status | |
| 133 | + | |
| 134 | + /** Incidents are managed by site administrators only: 404, not 403, for anyone else. */ | |
| 135 | + const requireSiteAdmin = (c: AppContext): Account => { | |
| 136 | + const me = requireAccount(c); | |
| 137 | + if (!me.site_admin) throw new HttpError(404, "not found"); | |
| 138 | + return me; | |
| 139 | + }; | |
| 140 | + | |
| 141 | + const BASE = "/settings/admin/status"; | |
| 142 | + const stateOptions = (states: readonly IncidentState[]) => states.map((s) => ({ value: s, label: STATE_TEXT[s] })); | |
| 143 | + | |
| 144 | + web.get(BASE, async (c) => { | |
| 145 | + const me = requireSiteAdmin(c); | |
| 146 | + const t = csrfToken(c); | |
| 147 | + const [s, all] = await Promise.all([statusSummary(c.env), new StatusIncidents(c.env.DB).list({ limit: 50 })]); | |
| 148 | + return render( | |
| 149 | + c, | |
| 150 | + { title: "Status" }, | |
| 151 | + <div class="grid left"> | |
| 152 | + <SettingsNav me={me} tab="status" /> | |
| 153 | + <div> | |
| 154 | + <ComputePausedBanner env={c.env} /> | |
| 155 | + <h1>Status</h1> | |
| 156 | + <p class="muted"> | |
| 157 | + Incidents on the public <a href="/status">status page</a>. While an incident isn't resolved, its impact sets the status of each component it affects. Pausing hosted compute under <a href="/settings/admin/kill-switches">Kill switches</a> puts Actions and agent sandboxes under maintenance there without an incident. Changes are in the <a href="/settings/audit">audit log</a>. | |
| 158 | + </p> | |
| 159 | + <div class={BANNER[s.status.indicator]} role="status"> | |
| 160 | + The status page says: <b>{s.status.description}</b>. | |
| 161 | + </div> | |
| 162 | + | |
| 163 | + <h2>Open an incident</h2> | |
| 164 | + <Form action={BASE} t={t} class="box mb"> | |
| 165 | + <div class="body"> | |
| 166 | + <div class="field"> | |
| 167 | + <label for="si-title">Title</label> | |
| 168 | + <input id="si-title" name="title" type="text" required maxlength={MAX_TITLE} placeholder="e.g. Pushes are slow or failing" autocomplete="off" /> | |
| 169 | + </div> | |
| 170 | + <div class="field"> | |
| 171 | + <label>Impact</label> | |
| 172 | + <KSelect | |
| 173 | + name="impact" | |
| 174 | + label="Impact" | |
| 175 | + value="minor" | |
| 176 | + options={IMPACTS.map((i) => ({ value: i, label: { minor: "Minor: degraded performance", major: "Major: partial outage", critical: "Critical: major outage", maintenance: "Maintenance: under maintenance" }[i] }))} | |
| 177 | + /> | |
| 178 | + </div> | |
| 179 | + <div class="field"> | |
| 180 | + <label>Affected components</label> | |
| 181 | + <div class="row" style="flex-wrap:wrap;gap:6px 16px"> | |
| 182 | + {COMPONENTS.map((comp) => ( | |
| 183 | + <KCheckbox name="components" value={comp.id} label={comp.name} /> | |
| 184 | + ))} | |
| 185 | + </div> | |
| 186 | + </div> | |
| 187 | + <div class="field"> | |
| 188 | + <label>State</label> | |
| 189 | + <KSelect name="state" label="State" value="investigating" options={stateOptions(INCIDENT_STATES.filter((s) => s !== "resolved"))} /> | |
| 190 | + </div> | |
| 191 | + <div class="field"> | |
| 192 | + <label for="si-body">First update</label> | |
| 193 | + <textarea id="si-body" name="body" required maxlength={MAX_BODY} style="min-height:90px" placeholder="e.g. We are investigating reports of failed pushes."></textarea> | |
| 194 | + <div class="hint">Plain text, shown to everyone on the status page.</div> | |
| 195 | + </div> | |
| 196 | + <button class="btn primary">Open the incident</button> | |
| 197 | + </div> | |
| 198 | + </Form> | |
| 199 | + | |
| 200 | + <h2>Open incidents</h2> | |
| 201 | + {s.incidents.length ? ( | |
| 202 | + s.incidents.map((i) => ( | |
| 203 | + <IncidentBox i={i} admin> | |
| 204 | + <Form action={`${BASE}/${i.id}`} t={t} class="body" style="border-top:1px solid var(--border2)"> | |
| 205 | + <div class="field"> | |
| 206 | + <label>State</label> | |
| 207 | + <KSelect name="state" label="State" value={i.state} options={stateOptions(INCIDENT_STATES)} /> | |
| 208 | + </div> | |
| 209 | + <div class="field"> | |
| 210 | + <label for={`si-body-${i.id}`}>Update</label> | |
| 211 | + <textarea id={`si-body-${i.id}`} name="body" required maxlength={MAX_BODY} style="min-height:70px" placeholder="e.g. We found the cause and are rolling out a fix."></textarea> | |
| 212 | + <div class="hint">Choose Resolved to close the incident: it then moves to past incidents.</div> | |
| 213 | + </div> | |
| 214 | + <button class="btn primary">Post the update</button> | |
| 215 | + </Form> | |
| 216 | + </IncidentBox> | |
| 217 | + )) | |
| 218 | + ) : ( | |
| 219 | + <div class="box mb"> | |
| 220 | + <div class="empty">No open incidents.</div> | |
| 221 | + </div> | |
| 222 | + )} | |
| 223 | + | |
| 224 | + <h2>Recent incidents</h2> | |
| 225 | + <div class="box" style="overflow:auto"> | |
| 226 | + {all.length ? ( | |
| 227 | + <table class="tbl"> | |
| 228 | + <thead> | |
| 229 | + <tr> | |
| 230 | + <th>Opened</th> | |
| 231 | + <th>Title</th> | |
| 232 | + <th>Impact</th> | |
| 233 | + <th>State</th> | |
| 234 | + </tr> | |
| 235 | + </thead> | |
| 236 | + <tbody> | |
| 237 | + {all.map((i) => ( | |
| 238 | + <tr> | |
| 239 | + <td class="small nowrap"> | |
| 240 | + <Time at={i.created_at} /> | |
| 241 | + </td> | |
| 242 | + <td> | |
| 243 | + {i.title} | |
| 244 | + <div class="muted small">{i.created_by ? <>by {i.created_by}</> : null}</div> | |
| 245 | + </td> | |
| 246 | + <td> | |
| 247 | + <span class={IMPACT_BADGE[i.impact]}>{IMPACT_TEXT[i.impact]}</span> | |
| 248 | + </td> | |
| 249 | + <td>{STATE_TEXT[i.state]}</td> | |
| 250 | + </tr> | |
| 251 | + ))} | |
| 252 | + </tbody> | |
| 253 | + </table> | |
| 254 | + ) : ( | |
| 255 | + <div class="empty">No incidents yet.</div> | |
| 256 | + )} | |
| 257 | + </div> | |
| 258 | + </div> | |
| 259 | + </div>, | |
| 260 | + ); | |
| 261 | + }); | |
| 262 | + | |
| 263 | + post(web, BASE, async (c, f) => { | |
| 264 | + const me = requireSiteAdmin(c); | |
| 265 | + return act( | |
| 266 | + c, | |
| 267 | + BASE, | |
| 268 | + async () => { | |
| 269 | + const input = validateIncident({ title: f.s("title"), impact: f.s("impact"), components: f.all("components"), state: f.s("state"), body: f.s("body") }); | |
| 270 | + await openIncident(c.env.DB, (e) => c.get("dir").audit(e), me, input, "web"); | |
| 271 | + }, | |
| 272 | + "Incident opened. It is on the status page now.", | |
| 273 | + ); | |
| 274 | + }); | |
| 275 | + | |
| 276 | + post(web, `${BASE}/:id`, async (c, f) => { | |
| 277 | + const me = requireSiteAdmin(c); | |
| 278 | + const id = prm(c, "id"); | |
| 279 | + return act( | |
| 280 | + c, | |
| 281 | + BASE, | |
| 282 | + async () => { | |
| 283 | + const input = validateUpdate({ state: f.s("state"), body: f.s("body") }); | |
| 284 | + await postUpdate(c.env.DB, (e) => c.get("dir").audit(e), me, id, input, "web"); | |
| 285 | + }, | |
| 286 | + f.s("state") === "resolved" ? "Incident resolved. It is under past incidents now." : "Update posted.", | |
| 287 | + ); | |
| 288 | + }); | |
| 289 | +} |
modifiedapps/diffinery/src/web/ui.tsx+5−0View file
| @@ -507,6 +507,11 @@ | ||
| 507 | 507 | {fl ? <div class={`flash ${fl.kind}`} role={fl.kind === "error" ? "alert" : "status"}>{fl.message}</div> : null} |
| 508 | 508 | {body} |
| 509 | 509 | </main> |
| 510 | + <footer class="site-foot"> | |
| 511 | + <a href="/docs">Docs</a> | |
| 512 | + <a href="/docs/security/disclosure">Security</a> | |
| 513 | + <a href="/status">Status</a> | |
| 514 | + </footer> | |
| 510 | 515 | {active ? ( |
| 511 | 516 | <dialog id="palette" aria-label="Command palette"> |
| 512 | 517 | <input type="text" placeholder="Jump to a repository, file, page or #number…" autocomplete="off" spellcheck={false} /> |
addedapps/diffinery/test/e2e/status.test.mjs+316−0View file
| @@ -1,0 +1,316 @@ | ||
| 1 | +// End-to-end status page: /status and GET /api/v3/diffinery/status for anyone, and incidents that a | |
| 2 | +// site administrator opens, updates and resolves on Settings → Status and through the admin API, on | |
| 3 | +// a real `wrangler dev` Worker with no fixed dev identity (so callers can be anonymous). Also covers | |
| 4 | +// 404 for everyone else, the audit log, and the hosted compute pause showing as maintenance. | |
| 5 | +// | |
| 6 | +// pnpm --filter @diffinery/app exec node --test test/e2e/status.test.mjs | |
| 7 | +import { spawn, execFileSync } from "node:child_process"; | |
| 8 | +import { createHash, randomBytes } from "node:crypto"; | |
| 9 | +import { mkdtempSync, rmSync } from "node:fs"; | |
| 10 | +import { tmpdir } from "node:os"; | |
| 11 | +import { join, dirname, resolve } from "node:path"; | |
| 12 | +import { fileURLToPath } from "node:url"; | |
| 13 | +import { createServer } from "node:net"; | |
| 14 | +import { test, after } from "node:test"; | |
| 15 | +import assert from "node:assert/strict"; | |
| 16 | + | |
| 17 | +const APP = resolve(dirname(fileURLToPath(import.meta.url)), "../.."); | |
| 18 | +const work = mkdtempSync(join(tmpdir(), "diffinery-status-e2e-")); | |
| 19 | +const running = []; | |
| 20 | + | |
| 21 | +const freePort = () => | |
| 22 | + new Promise((res) => { | |
| 23 | + const s = createServer(); | |
| 24 | + s.listen(0, "127.0.0.1", () => { | |
| 25 | + const { port } = s.address(); | |
| 26 | + s.close(() => res(port)); | |
| 27 | + }); | |
| 28 | + }); | |
| 29 | + | |
| 30 | +/** Start a Worker with extra `--var`s and seed SQL; resolves to its base URL once /healthz answers. */ | |
| 31 | +async function worker(name, vars, seed = []) { | |
| 32 | + const port = await freePort(); | |
| 33 | + const base = `http://127.0.0.1:${port}`; | |
| 34 | + const persist = join(work, name); | |
| 35 | + const wrangler = (...args) => execFileSync("pnpm", ["exec", "wrangler", ...args], { cwd: APP, stdio: "pipe", env: { ...process.env, CI: "1" } }); | |
| 36 | + wrangler("d1", "migrations", "apply", "diffinery-dev", "--local", "-e", "dev", "--persist-to", persist); | |
| 37 | + if (seed.length) wrangler("d1", "execute", "diffinery-dev", "--local", "-e", "dev", "--persist-to", persist, "--command", seed.join("; ")); | |
| 38 | + const args = ["exec", "wrangler", "dev", "-e", "dev", "--ip", "127.0.0.1", "--port", String(port), "--inspector-port", String(await freePort()), "--persist-to", persist, "--var", `PUBLIC_URL:${base}`, "--show-interactive-dev-session=false"]; | |
| 39 | + for (const [k, v] of Object.entries(vars)) args.push("--var", `${k}:${v}`); | |
| 40 | + const dev = spawn("pnpm", args, { cwd: APP, env: { ...process.env, CI: "1", WRANGLER_SEND_METRICS: "false" }, stdio: ["ignore", "pipe", "pipe"], detached: true }); | |
| 41 | + let log = ""; | |
| 42 | + dev.stdout.on("data", (d) => (log += d)); | |
| 43 | + dev.stderr.on("data", (d) => (log += d)); | |
| 44 | + running.push(dev); | |
| 45 | + const end = Date.now() + 120_000; | |
| 46 | + while (Date.now() < end) { | |
| 47 | + try { | |
| 48 | + if ((await fetch(`${base}/healthz`)).ok) return { base, stop: () => stopWorker(dev), log: () => log }; | |
| 49 | + } catch {} | |
| 50 | + await new Promise((r) => setTimeout(r, 500)); | |
| 51 | + } | |
| 52 | + assert.fail(`wrangler dev did not start\n${log.slice(-4000)}`); | |
| 53 | +} | |
| 54 | + | |
| 55 | +function stopWorker(dev) { | |
| 56 | + try { | |
| 57 | + process.kill(-dev.pid, "SIGTERM"); | |
| 58 | + } catch {} | |
| 59 | +} | |
| 60 | + | |
| 61 | +after(() => { | |
| 62 | + for (const dev of running) stopWorker(dev); | |
| 63 | + if (process.env.E2E_KEEP !== "1") rmSync(work, { recursive: true, force: true }); | |
| 64 | +}); | |
| 65 | + | |
| 66 | +const hash = (s) => createHash("sha256").update(s).digest("hex"); | |
| 67 | + | |
| 68 | +/** The Worker, with a site administrator (root) and a person (ann); started once. */ | |
| 69 | +let statusWorker; | |
| 70 | +const T = { root: `dfy_root_${randomBytes(12).toString("hex")}`, ann: `dfy_ann_${randomBytes(12).toString("hex")}` }; | |
| 71 | +const S = { root: `root-session-${randomBytes(12).toString("hex")}`, ann: `ann-session-${randomBytes(12).toString("hex")}` }; | |
| 72 | +const ids = {}; | |
| 73 | + | |
| 74 | +async function status() { | |
| 75 | + if (statusWorker) return statusWorker; | |
| 76 | + const at = new Date().toISOString(); | |
| 77 | + const later = new Date(Date.now() + 30 * 86400_000).toISOString(); | |
| 78 | + const person = (id, login, email, admin) => `INSERT INTO accounts (id, login, kind, name, email, status, site_admin, created_at) VALUES ('${id}', '${login}', 'user', '${login}', '${email}', 'active', ${admin}, '${at}')`; | |
| 79 | + const token = (id, account, t) => `INSERT INTO tokens (id, account_id, name, token_hash, prefix, scopes, expires_at, created_at) VALUES ('${id}', '${account}', 'e2e', '${hash(t)}', '${t.slice(0, 10)}', '["admin"]', '${later}', '${at}')`; | |
| 80 | + const session = (account, cookie) => `INSERT INTO sessions (id, account_id, method, created_at, last_seen_at, expires_at) VALUES ('${hash(cookie)}', '${account}', 'passkey', '${at}', '${at}', '${later}')`; | |
| 81 | + const w = await worker("status", { DEV_AUTH: "" }, [ | |
| 82 | + person("u-root", "root", "root@example.com", 1), | |
| 83 | + person("u-ann", "ann", "ann@example.com", 0), | |
| 84 | + token("t-root", "u-root", T.root), | |
| 85 | + token("t-ann", "u-ann", T.ann), | |
| 86 | + session("u-root", S.root), | |
| 87 | + session("u-ann", S.ann), | |
| 88 | + ]); | |
| 89 | + const cookieOf = (who) => (who ? `__Host-dfy_session=${S[who]}` : ""); | |
| 90 | + const api = async (method, path, body, tok) => { | |
| 91 | + const r = await fetch(`${w.base}/api/v3${path}`, { method, headers: { "content-type": "application/json", ...(tok ? { authorization: `Bearer ${tok}` } : {}) }, body: body === undefined ? undefined : JSON.stringify(body) }); | |
| 92 | + const text = await r.text(); | |
| 93 | + let json = null; | |
| 94 | + try { | |
| 95 | + json = text ? JSON.parse(text) : null; | |
| 96 | + } catch { | |
| 97 | + json = text; | |
| 98 | + } | |
| 99 | + return { status: r.status, json, headers: r.headers }; | |
| 100 | + }; | |
| 101 | + const page = (path, who) => fetch(`${w.base}${path}`, { headers: { cookie: cookieOf(who) }, redirect: "manual" }); | |
| 102 | + /** Submit a form on `pagePath` to `action` the way a browser does: the page's CSRF pair, same-origin. `fields` is a list of [name, value]. */ | |
| 103 | + const submit = async (pagePath, action, fields, who) => { | |
| 104 | + const res = await page(pagePath, who); | |
| 105 | + assert.equal(res.status, 200, `GET ${pagePath} -> ${res.status}`); | |
| 106 | + const csrf = /name="_csrf" value="([^"]+)"/.exec(await res.text())?.[1]; | |
| 107 | + const pair = /dfy_csrf=([^;]+)/.exec(res.headers.get("set-cookie") ?? "")?.[1]; | |
| 108 | + assert.ok(csrf && pair, `no CSRF pair on ${pagePath}`); | |
| 109 | + const r = await fetch(`${w.base}${action}`, { method: "POST", body: new URLSearchParams([["_csrf", csrf], ...fields]), redirect: "manual", headers: { cookie: [cookieOf(who), `dfy_csrf=${pair}`].filter(Boolean).join("; "), "sec-fetch-site": "same-origin" } }); | |
| 110 | + return { status: r.status, location: r.headers.get("location"), setCookie: r.headers.get("set-cookie") ?? "", html: await r.text() }; | |
| 111 | + }; | |
| 112 | + const summary = async () => { | |
| 113 | + const r = await api("GET", "/diffinery/status"); | |
| 114 | + assert.equal(r.status, 200, JSON.stringify(r.json)); | |
| 115 | + return r.json; | |
| 116 | + }; | |
| 117 | + statusWorker = { ...w, api, page, submit, summary }; | |
| 118 | + return statusWorker; | |
| 119 | +} | |
| 120 | + | |
| 121 | +/** The flash message a redirect sets, decoded. */ | |
| 122 | +function flashOf(setCookie) { | |
| 123 | + let v = /dfy_flash=([^;]+)/.exec(setCookie)?.[1] ?? ""; | |
| 124 | + for (let i = 0; i < 3 && /%[0-9A-F]{2}/i.test(v); i++) v = decodeURIComponent(v); | |
| 125 | + return v; | |
| 126 | +} | |
| 127 | +const componentOf = (s, id) => s.components.find((c) => c.id === id); | |
| 128 | +/** The part of the public page from its "Past incidents" heading on. */ | |
| 129 | +const pastPart = (html) => html.slice(html.indexOf("<h2>Past incidents</h2>")); | |
| 130 | +const activePart = (html) => html.slice(0, html.indexOf("<h2>Components</h2>")); | |
| 131 | + | |
| 132 | +const apiTitle = `Pushes are failing <b>${randomBytes(4).toString("hex")}</b>`; | |
| 133 | +const webTitle = `Emails are delayed ${randomBytes(4).toString("hex")}`; | |
| 134 | + | |
| 135 | +test("status page: anyone can read it, and everything is operational at first", async () => { | |
| 136 | + const { page, summary, api } = await status(); | |
| 137 | + const res = await page("/status"); | |
| 138 | + assert.equal(res.status, 200); | |
| 139 | + const html = await res.text(); | |
| 140 | + assert.match(html, /<h1>Status<\/h1>/); | |
| 141 | + assert.match(html, /All systems operational/); | |
| 142 | + for (const name of ["Website and API", "Git operations", "Actions", "Agents and sandboxes", "Packages", "Email"]) assert.ok(html.includes(name), `${name} is listed`); | |
| 143 | + assert.match(html, /No incidents in the last 14 days/); | |
| 144 | + // Every page links it from the footer. | |
| 145 | + assert.match(await (await page("/")).text(), /<footer class="site-foot">[\s\S]*href="\/status"/); | |
| 146 | + | |
| 147 | + const s = await summary(); | |
| 148 | + assert.deepEqual(s.status, { indicator: "none", description: "All systems operational" }); | |
| 149 | + assert.deepEqual( | |
| 150 | + s.components.map((c) => [c.id, c.status]), | |
| 151 | + ["web", "git", "actions", "agents", "packages", "email"].map((id) => [id, "operational"]), | |
| 152 | + ); | |
| 153 | + assert.deepEqual(s.incidents, []); | |
| 154 | + assert.ok(!Number.isNaN(Date.parse(s.updated_at))); | |
| 155 | + const r = await api("GET", "/diffinery/status"); | |
| 156 | + assert.equal(r.headers.get("cache-control"), "public, max-age=30"); | |
| 157 | +}); | |
| 158 | + | |
| 159 | +test("status page: only site administrators can manage incidents (404 for anyone else)", async () => { | |
| 160 | + const { api, page } = await status(); | |
| 161 | + const open = { title: "Nope", impact: "major", components: ["git"], state: "investigating", body: "x" }; | |
| 162 | + assert.equal((await api("GET", "/diffinery/admin/status/incidents", undefined, T.ann)).status, 404); | |
| 163 | + assert.equal((await api("POST", "/diffinery/admin/status/incidents", open, T.ann)).status, 404); | |
| 164 | + assert.equal((await api("POST", `/diffinery/admin/status/incidents/${"0".repeat(32)}/updates`, { state: "resolved", body: "x" }, T.ann)).status, 404); | |
| 165 | + assert.equal((await api("GET", "/diffinery/admin/status/incidents")).status, 401); | |
| 166 | + assert.equal((await api("POST", "/diffinery/admin/status/incidents", open)).status, 401); | |
| 167 | + assert.equal((await page("/settings/admin/status", "ann")).status, 404); | |
| 168 | + const anon = await page("/settings/admin/status"); | |
| 169 | + assert.equal(anon.status, 302, "a signed-out visitor is sent to sign in"); | |
| 170 | + assert.match(anon.headers.get("location") ?? "", /^\/login\?next=/); | |
| 171 | + assert.doesNotMatch(await (await page("/settings", "ann")).text(), /href="\/settings\/admin\/status"/); | |
| 172 | + // Nothing was opened. | |
| 173 | + assert.deepEqual((await api("GET", "/diffinery/admin/status/incidents", undefined, T.root)).json, []); | |
| 174 | +}); | |
| 175 | + | |
| 176 | +test("status page: a site administrator opens an incident through the API, and the page shows it", async () => { | |
| 177 | + const { api, page, summary } = await status(); | |
| 178 | + // Refused with sentences that say what to fix. | |
| 179 | + const bad = await api("POST", "/diffinery/admin/status/incidents", { title: "x", impact: "major", components: [], state: "investigating", body: "x" }, T.root); | |
| 180 | + assert.equal(bad.status, 422); | |
| 181 | + assert.equal(bad.json.message, "Choose at least one affected component."); | |
| 182 | + const unknown = await api("POST", "/diffinery/admin/status/incidents", { title: "x", impact: "major", components: ["database"], state: "investigating", body: "x" }, T.root); | |
| 183 | + assert.equal(unknown.status, 422); | |
| 184 | + assert.match(unknown.json.message, /There is no component called "database"/); | |
| 185 | + assert.equal((await api("POST", "/diffinery/admin/status/incidents", { title: "x", impact: "huge", components: ["git"], state: "investigating", body: "x" }, T.root)).status, 422); | |
| 186 | + | |
| 187 | + const r = await api("POST", "/diffinery/admin/status/incidents", { title: apiTitle, impact: "major", components: ["git"], state: "investigating", body: "We are investigating failed pushes.\n<script>alert(1)</script>" }, T.root); | |
| 188 | + assert.equal(r.status, 201, JSON.stringify(r.json)); | |
| 189 | + ids.api = r.json.id; | |
| 190 | + assert.match(ids.api, /^[0-9a-f]{32}$/); | |
| 191 | + assert.equal(r.json.created_by, "root"); | |
| 192 | + assert.equal(r.json.updates.length, 1); | |
| 193 | + assert.equal(r.json.updates[0].author, "root"); | |
| 194 | + | |
| 195 | + const s = await summary(); | |
| 196 | + assert.deepEqual(s.status, { indicator: "major", description: "Partial outage" }); | |
| 197 | + assert.equal(componentOf(s, "git").status, "partial_outage"); | |
| 198 | + assert.equal(componentOf(s, "web").status, "operational"); | |
| 199 | + assert.deepEqual( | |
| 200 | + s.incidents.map((i) => [i.id, i.title, i.state, i.components]), | |
| 201 | + [[ids.api, apiTitle, "investigating", ["git"]]], | |
| 202 | + ); | |
| 203 | + assert.equal(s.incidents[0].created_by, undefined, "the public JSON doesn't name administrators"); | |
| 204 | + assert.equal(s.incidents[0].updates[0].author, undefined); | |
| 205 | + | |
| 206 | + const html = await (await page("/status")).text(); | |
| 207 | + assert.match(html, /Partial outage/); | |
| 208 | + assert.match(html, /<li data-component="git" data-status="partial_outage">/); | |
| 209 | + assert.ok(activePart(html).includes(apiTitle.replace("<b>", "<b>").replace("</b>", "</b>")), "the title is escaped, under active incidents"); | |
| 210 | + assert.ok(!html.includes("<script>alert(1)</script>"), "the update is plain text"); | |
| 211 | + assert.match(html, /We are investigating failed pushes\./); | |
| 212 | +}); | |
| 213 | + | |
| 214 | +test("status page: the web form opens an incident with several components and posts updates", async () => { | |
| 215 | + const { submit, summary, api, page } = await status(); | |
| 216 | + const base = "/settings/admin/status"; | |
| 217 | + const missing = await submit(base, base, [["title", webTitle], ["impact", "minor"], ["state", "investigating"], ["body", "Looking."]], "root"); | |
| 218 | + assert.equal(missing.status, 303); | |
| 219 | + assert.match(flashOf(missing.setCookie), /Choose at least one affected component\./); | |
| 220 | + const opened = await submit( | |
| 221 | + base, | |
| 222 | + base, | |
| 223 | + [ | |
| 224 | + ["title", webTitle], | |
| 225 | + ["impact", "minor"], | |
| 226 | + ["components", "email"], | |
| 227 | + ["components", "web"], | |
| 228 | + ["state", "investigating"], | |
| 229 | + ["body", "Some emails are delayed by a few minutes."], | |
| 230 | + ], | |
| 231 | + "root", | |
| 232 | + ); | |
| 233 | + assert.equal(opened.status, 303); | |
| 234 | + assert.equal(opened.location, base); | |
| 235 | + let s = await summary(); | |
| 236 | + const web = s.incidents.find((i) => i.title === webTitle); | |
| 237 | + assert.ok(web, "the incident from the form is listed"); | |
| 238 | + ids.web = web.id; | |
| 239 | + assert.deepEqual(web.components, ["web", "email"]); | |
| 240 | + assert.equal(componentOf(s, "email").status, "degraded_performance"); | |
| 241 | + // Worst wins: git's partial outage outranks the email incident. | |
| 242 | + assert.deepEqual(s.status, { indicator: "major", description: "Partial outage" }); | |
| 243 | + | |
| 244 | + const adminHtml = await (await page(base, "root")).text(); | |
| 245 | + assert.ok(adminHtml.includes(webTitle)); | |
| 246 | + assert.ok(adminHtml.includes(`action="${base}/${ids.web}"`), "each open incident has an update form"); | |
| 247 | + assert.doesNotMatch(adminHtml, /<select|type="checkbox"/, "Kumo controls only"); | |
| 248 | + | |
| 249 | + const identified = await submit(base, `${base}/${ids.web}`, [["state", "identified"], ["body", "A mail queue was stuck; draining it."]], "root"); | |
| 250 | + assert.equal(identified.status, 303); | |
| 251 | + const resolved = await submit(base, `${base}/${ids.web}`, [["state", "resolved"], ["body", "Emails are on time again."]], "root"); | |
| 252 | + assert.equal(resolved.status, 303); | |
| 253 | + assert.match(flashOf(resolved.setCookie), /Incident resolved/); | |
| 254 | + const listed = (await api("GET", "/diffinery/admin/status/incidents", undefined, T.root)).json.find((i) => i.id === ids.web); | |
| 255 | + assert.equal(listed.state, "resolved"); | |
| 256 | + assert.ok(listed.resolved_at); | |
| 257 | + assert.deepEqual( | |
| 258 | + listed.updates.map((u) => u.state), | |
| 259 | + ["resolved", "identified", "investigating"], | |
| 260 | + ); | |
| 261 | + s = await summary(); | |
| 262 | + assert.equal(componentOf(s, "email").status, "operational"); | |
| 263 | + assert.ok(!s.incidents.some((i) => i.id === ids.web)); | |
| 264 | +}); | |
| 265 | + | |
| 266 | +test("status page: resolving through the API returns the component to operational and moves the incident to past incidents", async () => { | |
| 267 | + const { api, page, summary } = await status(); | |
| 268 | + assert.ok(ids.api, "the incident from the test above"); | |
| 269 | + const bad = await api("POST", `/diffinery/admin/status/incidents/${ids.api}/updates`, { state: "fixed", body: "x" }, T.root); | |
| 270 | + assert.equal(bad.status, 422); | |
| 271 | + assert.equal((await api("POST", `/diffinery/admin/status/incidents/${"0".repeat(32)}/updates`, { state: "resolved", body: "x" }, T.root)).status, 404); | |
| 272 | + const r = await api("POST", `/diffinery/admin/status/incidents/${ids.api}/updates`, { state: "resolved", body: "Pushes work again." }, T.root); | |
| 273 | + assert.equal(r.status, 201, JSON.stringify(r.json)); | |
| 274 | + assert.equal(r.json.state, "resolved"); | |
| 275 | + | |
| 276 | + const s = await summary(); | |
| 277 | + assert.deepEqual(s.status, { indicator: "none", description: "All systems operational" }); | |
| 278 | + assert.equal(componentOf(s, "git").status, "operational"); | |
| 279 | + assert.deepEqual(s.incidents, []); | |
| 280 | + | |
| 281 | + const html = await (await page("/status")).text(); | |
| 282 | + assert.match(html, /All systems operational/); | |
| 283 | + assert.doesNotMatch(html, /Active incidents/); | |
| 284 | + const past = pastPart(html); | |
| 285 | + assert.ok(past.includes("Pushes are failing"), "the API incident is under past incidents"); | |
| 286 | + assert.ok(past.includes(webTitle), "so is the one from the form"); | |
| 287 | + assert.match(past, /Pushes work again\./); | |
| 288 | +}); | |
| 289 | + | |
| 290 | +test("status page: changes are audited without the updates' text", async () => { | |
| 291 | + const { page } = await status(); | |
| 292 | + assert.ok(ids.api && ids.web, "the incidents from the tests above"); | |
| 293 | + const audit = await (await page("/settings/audit", "root")).text(); | |
| 294 | + for (const action of ["status_incident.create", "status_incident.update", "status_incident.resolve"]) assert.ok(audit.includes(action), `${action} is audited`); | |
| 295 | + assert.ok(audit.includes(ids.api) && audit.includes(ids.web), "the audit log names the incidents"); | |
| 296 | + assert.ok(!audit.includes("Pushes work again"), "the audit log holds no update text"); | |
| 297 | +}); | |
| 298 | + | |
| 299 | +test("status page: pausing hosted compute shows Actions and agent sandboxes under maintenance", async () => { | |
| 300 | + const { api, summary, page } = await status(); | |
| 301 | + const paused = await api("PUT", "/admin/hosted-compute-pause", { reason: "internal runner image problem" }, T.root); | |
| 302 | + assert.equal(paused.status, 200, JSON.stringify(paused.json)); | |
| 303 | + try { | |
| 304 | + const s = await summary(); | |
| 305 | + assert.deepEqual(s.status, { indicator: "maintenance", description: "Service under maintenance" }); | |
| 306 | + assert.deepEqual(componentOf(s, "actions"), { id: "actions", name: "Actions", status: "under_maintenance", note: "Hosted jobs are paused by the site administrators. Jobs on self-hosted runners still run." }); | |
| 307 | + assert.equal(componentOf(s, "agents").status, "under_maintenance"); | |
| 308 | + assert.equal(componentOf(s, "git").status, "operational"); | |
| 309 | + const html = await (await page("/status")).text(); | |
| 310 | + assert.match(html, /Hosted jobs are paused by the site administrators\./); | |
| 311 | + assert.ok(!html.includes("internal runner image problem"), "the pause's reason isn't public"); | |
| 312 | + } finally { | |
| 313 | + assert.equal((await api("DELETE", "/admin/hosted-compute-pause", undefined, T.root)).status, 200); | |
| 314 | + } | |
| 315 | + assert.equal((await summary()).status.indicator, "none"); | |
| 316 | +}); |
modifiedapps/diffinery/test/unit/docs.test.ts+1−1View file
| @@ -42,7 +42,7 @@ | ||
| 42 | 42 | } |
| 43 | 43 | |
| 44 | 44 | /** Paths outside /docs that pages may link to: real routes of the app. */ |
| 45 | -const SITE_PATHS = [/^\/$/, /^\/llms(-full)?\.txt$/, /^\/-\/cli\/diffinery\.mjs$/, /^\/settings(\/[\w/-]*)?$/, /^\/new$/, /^\/login$/, /^\/notifications$/, /^\/auth\/access$/, /^\/mcp$/, /^\/api\/v3(\/.*)?$/, /^\/\.well-known\/[\w.-]+$/, /^\/security\/report$/, /^\/healthz$/]; | |
| 45 | +const SITE_PATHS = [/^\/$/, /^\/llms(-full)?\.txt$/, /^\/-\/cli\/diffinery\.mjs$/, /^\/settings(\/[\w/-]*)?$/, /^\/new$/, /^\/login$/, /^\/notifications$/, /^\/auth\/access$/, /^\/mcp$/, /^\/api\/v3(\/.*)?$/, /^\/\.well-known\/[\w.-]+$/, /^\/security\/report$/, /^\/status$/, /^\/healthz$/]; | |
| 46 | 46 | |
| 47 | 47 | describe("docs pages", () => { |
| 48 | 48 | it("has every page in the navigation, in order", () => { |
addedapps/diffinery/test/unit/status.test.ts+240−0View file
| @@ -1,0 +1,240 @@ | ||
| 1 | +import { describe, expect, it } from "vitest"; | |
| 2 | +import type { Env } from "../../src/env.js"; | |
| 3 | +import { PAUSE_FLAG } from "../../src/lib/killswitch.js"; | |
| 4 | +import { | |
| 5 | + COMPONENT_IDS, | |
| 6 | + DB_DOWN, | |
| 7 | + MAX_BODY, | |
| 8 | + MAX_TITLE, | |
| 9 | + StatusIncidents, | |
| 10 | + componentStates, | |
| 11 | + incidentJson, | |
| 12 | + killSwitchSignals, | |
| 13 | + openIncident, | |
| 14 | + overall, | |
| 15 | + postUpdate, | |
| 16 | + statusSummary, | |
| 17 | + summaryJson, | |
| 18 | + validateIncident, | |
| 19 | + validateUpdate, | |
| 20 | + type Incident, | |
| 21 | +} from "../../src/lib/status.js"; | |
| 22 | +import { HttpError } from "../../src/lib/util.js"; | |
| 23 | +import { testD1 } from "./stubs/d1.js"; | |
| 24 | + | |
| 25 | +const PUB = "https://diffinery.example"; | |
| 26 | +const root = { id: "u-root", login: "root" }; | |
| 27 | + | |
| 28 | +function setup() { | |
| 29 | + const DB = testD1(); | |
| 30 | + const env = { DB, PUBLIC_URL: PUB } as unknown as Env; | |
| 31 | + const audits: Array<{ action: string; target?: string; actorLogin?: string; detail?: unknown }> = []; | |
| 32 | + const audit = async (e: (typeof audits)[number]) => void audits.push(e); | |
| 33 | + return { env, DB, audits, audit }; | |
| 34 | +} | |
| 35 | + | |
| 36 | +/** An incident as the store would return it. */ | |
| 37 | +const incident = (o: Partial<Incident>): Incident => ({ | |
| 38 | + id: "0".repeat(32), | |
| 39 | + title: "Something", | |
| 40 | + impact: "minor", | |
| 41 | + components: ["web"], | |
| 42 | + state: "investigating", | |
| 43 | + created_by: "root", | |
| 44 | + created_at: "2026-10-01T00:00:00.000Z", | |
| 45 | + updated_at: "2026-10-01T00:00:00.000Z", | |
| 46 | + resolved_at: null, | |
| 47 | + updates: [], | |
| 48 | + ...o, | |
| 49 | +}); | |
| 50 | + | |
| 51 | +const statusOf = (incidents: Incident[], signals: Parameters<typeof componentStates>[1] = []) => Object.fromEntries(componentStates(incidents, signals).map((c) => [c.id, c.status])); | |
| 52 | + | |
| 53 | +const refused = (f: () => unknown, msg: RegExp) => { | |
| 54 | + try { | |
| 55 | + f(); | |
| 56 | + } catch (e) { | |
| 57 | + expect(e).toBeInstanceOf(HttpError); | |
| 58 | + expect((e as HttpError).status).toBe(422); | |
| 59 | + expect((e as HttpError).detail).toMatch(msg); | |
| 60 | + return; | |
| 61 | + } | |
| 62 | + throw new Error("expected it to be refused"); | |
| 63 | +}; | |
| 64 | + | |
| 65 | +describe("component statuses", () => { | |
| 66 | + it("lists every component, operational when nothing is wrong", () => { | |
| 67 | + const c = componentStates([], []); | |
| 68 | + expect(c.map((x) => [x.id, x.name])).toEqual([ | |
| 69 | + ["web", "Website and API"], | |
| 70 | + ["git", "Git operations"], | |
| 71 | + ["actions", "Actions"], | |
| 72 | + ["agents", "Agents and sandboxes"], | |
| 73 | + ["packages", "Packages"], | |
| 74 | + ["email", "Email"], | |
| 75 | + ]); | |
| 76 | + expect(c.every((x) => x.status === "operational" && !("note" in x))).toBe(true); | |
| 77 | + expect(overall(c)).toEqual({ indicator: "none", description: "All systems operational" }); | |
| 78 | + }); | |
| 79 | + | |
| 80 | + it("maps each impact to Statuspage's component status, only for the components an incident names", () => { | |
| 81 | + expect(statusOf([incident({ impact: "critical", components: ["git"] })])).toMatchObject({ git: "major_outage", web: "operational" }); | |
| 82 | + expect(statusOf([incident({ impact: "major", components: ["actions", "agents"] })])).toMatchObject({ actions: "partial_outage", agents: "partial_outage", git: "operational" }); | |
| 83 | + expect(statusOf([incident({ impact: "minor", components: ["email"] })])).toMatchObject({ email: "degraded_performance" }); | |
| 84 | + expect(statusOf([incident({ impact: "maintenance", components: ["packages"] })])).toMatchObject({ packages: "under_maintenance" }); | |
| 85 | + }); | |
| 86 | + | |
| 87 | + it("takes the worst unresolved incident, and ignores resolved ones", () => { | |
| 88 | + const s = statusOf([ | |
| 89 | + incident({ impact: "minor", components: ["web", "git"] }), | |
| 90 | + incident({ impact: "critical", components: ["web"], state: "monitoring" }), | |
| 91 | + incident({ impact: "critical", components: ["git"], state: "resolved", resolved_at: "2026-10-02T00:00:00.000Z" }), | |
| 92 | + incident({ impact: "maintenance", components: ["git"] }), | |
| 93 | + ]); | |
| 94 | + expect(s).toMatchObject({ web: "major_outage", git: "degraded_performance" }); | |
| 95 | + }); | |
| 96 | + | |
| 97 | + it("gives the overall indicator and description of the worst component", () => { | |
| 98 | + const of = (impact: Incident["impact"]) => overall(componentStates([incident({ impact }), incident({ impact: "maintenance", components: ["email"] })], [])); | |
| 99 | + expect(of("critical")).toEqual({ indicator: "critical", description: "Major outage" }); | |
| 100 | + expect(of("major")).toEqual({ indicator: "major", description: "Partial outage" }); | |
| 101 | + expect(of("minor")).toEqual({ indicator: "minor", description: "Degraded performance" }); | |
| 102 | + expect(of("maintenance")).toEqual({ indicator: "maintenance", description: "Service under maintenance" }); | |
| 103 | + }); | |
| 104 | +}); | |
| 105 | + | |
| 106 | +describe("kill switches and failures", () => { | |
| 107 | + it("puts what an instance-wide switch stops under maintenance, with a note", () => { | |
| 108 | + expect(killSwitchSignals({ hostedJobs: false, selfHostedJobs: false, sandboxes: false })).toEqual([]); | |
| 109 | + const paused = killSwitchSignals({ hostedJobs: true, selfHostedJobs: false, sandboxes: true }); | |
| 110 | + const c = componentStates([], paused); | |
| 111 | + expect(c.find((x) => x.id === "actions")).toEqual({ id: "actions", name: "Actions", status: "under_maintenance", note: "Hosted jobs are paused by the site administrators. Jobs on self-hosted runners still run." }); | |
| 112 | + expect(c.find((x) => x.id === "agents")).toEqual({ id: "agents", name: "Agents and sandboxes", status: "under_maintenance", note: "Agent sandboxes are paused by the site administrators." }); | |
| 113 | + expect(overall(c).indicator).toBe("maintenance"); | |
| 114 | + expect(killSwitchSignals({ hostedJobs: true, selfHostedJobs: true, sandboxes: false })).toEqual([{ component: "actions", status: "under_maintenance", note: "Actions jobs are paused by the site administrators." }]); | |
| 115 | + expect(killSwitchSignals({ hostedJobs: false, selfHostedJobs: true, sandboxes: false })[0]!.note).toBe("Jobs on self-hosted runners are paused by the site administrators."); | |
| 116 | + }); | |
| 117 | + | |
| 118 | + it("lets a worse incident win over a switch, keeping the switch's note", () => { | |
| 119 | + const signals = killSwitchSignals({ hostedJobs: true, selfHostedJobs: false, sandboxes: true }); | |
| 120 | + const c = componentStates([incident({ impact: "major", components: ["actions"] }), incident({ impact: "maintenance", components: ["agents"] })], signals); | |
| 121 | + expect(c.find((x) => x.id === "actions")).toMatchObject({ status: "partial_outage", note: expect.stringMatching(/^Hosted jobs are paused/) }); | |
| 122 | + expect(c.find((x) => x.id === "agents")!.status).toBe("under_maintenance"); | |
| 123 | + }); | |
| 124 | + | |
| 125 | + it("reads the instance's hosted compute pause into the summary", async () => { | |
| 126 | + const { env, DB } = setup(); | |
| 127 | + expect((await statusSummary(env)).status).toEqual({ indicator: "none", description: "All systems operational" }); | |
| 128 | + DB.sqlite.prepare("INSERT INTO site_flags (key, value, updated_at, updated_by) VALUES (?, 'a secret reason', ?, 'root')").run(PAUSE_FLAG, new Date().toISOString()); | |
| 129 | + const s = await statusSummary(env); | |
| 130 | + expect(s.status.indicator).toBe("maintenance"); | |
| 131 | + expect(s.components.find((c) => c.id === "actions")).toMatchObject({ status: "under_maintenance", note: expect.stringMatching(/^Hosted jobs are paused by the site administrators/) }); | |
| 132 | + expect(s.components.find((c) => c.id === "agents")!.status).toBe("under_maintenance"); | |
| 133 | + expect(s.components.find((c) => c.id === "git")!.status).toBe("operational"); | |
| 134 | + // The administrators' reason isn't public. | |
| 135 | + expect(JSON.stringify(summaryJson(env, s))).not.toContain("secret"); | |
| 136 | + }); | |
| 137 | + | |
| 138 | + it("still answers when the database can't be read: the website is down and there are no incidents", async () => { | |
| 139 | + const DB = { prepare: () => ({ bind: () => ({ all: async () => { throw new Error("D1_ERROR: unavailable"); }, first: async () => { throw new Error("D1_ERROR: unavailable"); } }), all: async () => { throw new Error("D1_ERROR: unavailable"); }, first: async () => { throw new Error("D1_ERROR: unavailable"); } }) }; | |
| 140 | + const env = { DB, PUBLIC_URL: PUB } as unknown as Env; | |
| 141 | + const error = console.error; | |
| 142 | + console.error = () => {}; | |
| 143 | + try { | |
| 144 | + const s = await statusSummary(env, new Date("2026-10-14T12:00:00.000Z")); | |
| 145 | + expect(s.components.find((c) => c.id === "web")).toEqual({ id: "web", name: "Website and API", status: "major_outage", note: DB_DOWN }); | |
| 146 | + expect(s.components.filter((c) => c.status !== "operational").map((c) => c.id)).toEqual(["web"]); | |
| 147 | + expect(s.status).toEqual({ indicator: "critical", description: "Major outage" }); | |
| 148 | + expect(s).toMatchObject({ incidents: [], resolved: [], updated_at: "2026-10-14T12:00:00.000Z" }); | |
| 149 | + } finally { | |
| 150 | + console.error = error; | |
| 151 | + } | |
| 152 | + }); | |
| 153 | +}); | |
| 154 | + | |
| 155 | +describe("validation", () => { | |
| 156 | + const good = { title: " Pushes\n are failing ", impact: "major", components: ["git", "web", "git"], state: "investigating", body: "\n\nWe are looking into it.\r\nMore soon.\n\n " }; | |
| 157 | + | |
| 158 | + it("keeps a one-line title, the update's lines and the components in the page's order", () => { | |
| 159 | + expect(validateIncident(good)).toEqual({ title: "Pushes are failing", impact: "major", components: ["web", "git"], state: "investigating", body: "We are looking into it.\nMore soon." }); | |
| 160 | + expect(validateUpdate({ state: "resolved", body: "Fixed." })).toEqual({ state: "resolved", body: "Fixed." }); | |
| 161 | + }); | |
| 162 | + | |
| 163 | + it("refuses what is missing, too long or unknown, in full sentences", () => { | |
| 164 | + refused(() => validateIncident({ ...good, title: " \n " }), /^Give the incident a title\.$/); | |
| 165 | + refused(() => validateIncident({ ...good, title: "x".repeat(MAX_TITLE + 1) }), /200 characters/); | |
| 166 | + expect(validateIncident({ ...good, title: "x".repeat(MAX_TITLE) }).title).toHaveLength(MAX_TITLE); | |
| 167 | + refused(() => validateIncident({ ...good, impact: "huge" }), /^Choose the incident's impact: minor, major, critical or maintenance\.$/); | |
| 168 | + refused(() => validateIncident({ ...good, components: [] }), /^Choose at least one affected component\.$/); | |
| 169 | + refused(() => validateIncident({ ...good, components: "web" }), /at least one affected component/); | |
| 170 | + refused(() => validateIncident({ ...good, components: ["web", "database"] }), /^There is no component called "database"\. Choose from web, git, actions, agents, packages, email\.$/); | |
| 171 | + refused(() => validateIncident({ ...good, state: "closed" }), /^Choose the incident's state: investigating, identified, monitoring or resolved\.$/); | |
| 172 | + refused(() => validateIncident({ ...good, body: " \n " }), /^Write an update that says what is happening\.$/); | |
| 173 | + refused(() => validateUpdate({ state: "monitoring", body: "x".repeat(MAX_BODY + 1) }), /^Keep the update to 5,000 characters\.$/); | |
| 174 | + expect(validateUpdate({ state: "monitoring", body: "x".repeat(MAX_BODY) }).body).toHaveLength(MAX_BODY); | |
| 175 | + refused(() => validateUpdate({ state: undefined, body: "x" }), /state/); | |
| 176 | + refused(() => validateIncident({ ...good, title: 42 }), /title/); | |
| 177 | + expect(COMPONENT_IDS).toEqual(["web", "git", "actions", "agents", "packages", "email"]); | |
| 178 | + }); | |
| 179 | +}); | |
| 180 | + | |
| 181 | +describe("opening, updating and resolving incidents", () => { | |
| 182 | + it("opens an incident with its first update, audited, and the summary shows it", async () => { | |
| 183 | + const { env, audits, audit } = setup(); | |
| 184 | + const i = await openIncident(env.DB, audit, root, validateIncident({ title: "Pushes are failing", impact: "major", components: ["git"], state: "investigating", body: "Looking into it." }), "api"); | |
| 185 | + expect(i).toMatchObject({ title: "Pushes are failing", impact: "major", components: ["git"], state: "investigating", created_by: "root", resolved_at: null }); | |
| 186 | + expect(i.updates).toEqual([expect.objectContaining({ state: "investigating", body: "Looking into it.", author_login: "root" })]); | |
| 187 | + expect(audits).toEqual([{ actorId: "u-root", actorLogin: "root", action: "status_incident.create", target: i.id, detail: { title: "Pushes are failing", impact: "major", components: ["git"], state: "investigating", via: "api" } }]); | |
| 188 | + const s = await statusSummary(env); | |
| 189 | + expect(s.status).toEqual({ indicator: "major", description: "Partial outage" }); | |
| 190 | + expect(s.components.find((c) => c.id === "git")!.status).toBe("partial_outage"); | |
| 191 | + expect(s.incidents.map((x) => x.id)).toEqual([i.id]); | |
| 192 | + const json = summaryJson(env, s); | |
| 193 | + expect(json.incidents[0]).toEqual({ | |
| 194 | + id: i.id, | |
| 195 | + title: "Pushes are failing", | |
| 196 | + impact: "major", | |
| 197 | + state: "investigating", | |
| 198 | + components: ["git"], | |
| 199 | + created_at: i.created_at, | |
| 200 | + updated_at: i.updated_at, | |
| 201 | + resolved_at: null, | |
| 202 | + updates: [{ id: i.updates[0]!.id, state: "investigating", body: "Looking into it.", created_at: i.created_at }], | |
| 203 | + html_url: `${PUB}/status#incident-${i.id}`, | |
| 204 | + }); | |
| 205 | + expect(incidentJson(env, i, { admin: true })).toMatchObject({ created_by: "root", updates: [{ author: "root" }] }); | |
| 206 | + }); | |
| 207 | + | |
| 208 | + it("posts updates newest first, and resolving returns the components to operational", async () => { | |
| 209 | + const { env, audits, audit } = setup(); | |
| 210 | + const i = await openIncident(env.DB, audit, root, validateIncident({ title: "Slow pages", impact: "minor", components: ["web", "git"], state: "investigating", body: "Looking." }), "web"); | |
| 211 | + audits.length = 0; | |
| 212 | + const found = await postUpdate(env.DB, audit, root, i.id, validateUpdate({ state: "identified", body: "Found it." }), "web"); | |
| 213 | + expect(found.state).toBe("identified"); | |
| 214 | + const done = await postUpdate(env.DB, audit, root, i.id, validateUpdate({ state: "resolved", body: "Fixed." }), "api"); | |
| 215 | + expect(done.state).toBe("resolved"); | |
| 216 | + expect(done.resolved_at).toBeTruthy(); | |
| 217 | + expect(done.updates.map((u) => u.state)).toEqual(["resolved", "identified", "investigating"]); | |
| 218 | + expect(audits).toEqual([ | |
| 219 | + { actorId: "u-root", actorLogin: "root", action: "status_incident.update", target: i.id, detail: { from: "investigating", to: "identified", via: "web" } }, | |
| 220 | + { actorId: "u-root", actorLogin: "root", action: "status_incident.resolve", target: i.id, detail: { from: "identified", to: "resolved", via: "api" } }, | |
| 221 | + ]); | |
| 222 | + expect(JSON.stringify(audits)).not.toContain("Found it"); | |
| 223 | + const s = await statusSummary(env); | |
| 224 | + expect(s.components.every((c) => c.status === "operational")).toBe(true); | |
| 225 | + expect(s.incidents).toEqual([]); | |
| 226 | + expect(s.resolved.map((x) => x.id)).toEqual([i.id]); | |
| 227 | + // Fourteen days later it is off the page. | |
| 228 | + expect((await statusSummary(env, new Date(Date.now() + 15 * 86400_000))).resolved).toEqual([]); | |
| 229 | + // Reopening clears resolved_at. | |
| 230 | + const again = await postUpdate(env.DB, audit, root, i.id, validateUpdate({ state: "monitoring", body: "It came back." }), "api"); | |
| 231 | + expect(again).toMatchObject({ state: "monitoring", resolved_at: null }); | |
| 232 | + }); | |
| 233 | + | |
| 234 | + it("refuses an update to an incident that doesn't exist", async () => { | |
| 235 | + const { env, audit } = setup(); | |
| 236 | + await expect(postUpdate(env.DB, audit, root, "f".repeat(32), { state: "resolved", body: "x" }, "api")).rejects.toMatchObject({ status: 404 }); | |
| 237 | + await expect(postUpdate(env.DB, audit, root, "not-an-id", { state: "resolved", body: "x" }, "api")).rejects.toMatchObject({ status: 404 }); | |
| 238 | + expect(await new StatusIncidents(env.DB).list()).toEqual([]); | |
| 239 | + }); | |
| 240 | +}); |