Kill switches: suspend Actions for a repository or an account, pause hosted compute (#131)

* Kill switches: suspend Actions for a repository or an account, pause hosted compute (WIP)

Site administrators can suspend Actions for one repository or every
repository an account owns, and pause hosted compute on the instance.
State in D1 (migration 0020), checked where jobs launch, runs are created
and sandboxes start; turning a switch on stops what is running.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo

* Kill switches: tests and docs

A unit test of lib/killswitch.ts over the node:sqlite D1 shim, an e2e test
in security.test.mjs (repository and account suspension, the hosted compute
pause, 403 for anyone but site administrators, audit entries), the
Kill switches section of Site administration, the audit actions and the
REST reference entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo

* Kill switches: audit even when stopping work fails; guard a re-run that didn't start

The suspend and pause audit entries are written in a finally block, a
sleeping sandbox's kill switch check fails open like the job checks, and a
re-run whose run wasn't created answers 409 instead of throwing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo

* Kill switches e2e: claim as a registered runner

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo

* Kill switches: web terminals end, can't be attached and can't be opened while sandboxes are stopped

Migration renumbered to 0021 (0019 is web terminals, 0020 sign-up).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yhgAu6oKfqes812Yd4qjo

---------

Co-authored-by: Claude <noreply@anthropic.com>

Canberk Demirci authored 12 days ago· GitHub committed 12 days agosigned1 parent 0f99813commit 92fbc27b73cf975fde0c13a735cea9d11e761f71Browse files

20 files changed+1092−15

Files (20)

apps/diffinery/docs/admin/site-administration.md+74−2
apps/diffinery/docs/security/audit-log.md+5−1
apps/diffinery/migrations/0021_suspensions.sql+20−0
apps/diffinery/src/agentic/ProjectDO.ts+3−0
apps/diffinery/src/agentic/terminal-web.tsx+4−0
apps/diffinery/src/agentic/terminals.ts+6−1
apps/diffinery/src/docs/api-meta.ts+37−0
apps/diffinery/src/http/api.ts+84−0
apps/diffinery/src/lib/killswitch.ts+227−0
apps/diffinery/src/mail/web.tsx+6−2
apps/diffinery/src/repo/RepoDO.ts+6−0
apps/diffinery/src/repo/actions.ts+86−6
apps/diffinery/src/runner/containers.ts+7−0
apps/diffinery/src/web/account.tsx+76−2
apps/diffinery/src/web/actions.tsx+3−0
apps/diffinery/src/web/kill-switches.tsx+173−0
apps/diffinery/src/web/repo-settings.tsx+5−1
apps/diffinery/test/e2e/security.test.mjs+107−0
apps/diffinery/test/e2e/tasks.test.mjs+34−0
apps/diffinery/test/unit/killswitch.test.ts+129−0

modifiedapps/diffinery/docs/admin/site-administration.md+74−2View file

modifiedapps/diffinery/docs/security/audit-log.md+5−1View file

addedapps/diffinery/migrations/0021_suspensions.sql+20−0View file

modifiedapps/diffinery/src/agentic/ProjectDO.ts+3−0View file

modifiedapps/diffinery/src/agentic/terminal-web.tsx+4−0View file

modifiedapps/diffinery/src/agentic/terminals.ts+6−1View file

modifiedapps/diffinery/src/docs/api-meta.ts+37−0View file

modifiedapps/diffinery/src/http/api.ts+84−0View file

addedapps/diffinery/src/lib/killswitch.ts+227−0View file

modifiedapps/diffinery/src/mail/web.tsx+6−2View file

modifiedapps/diffinery/src/repo/RepoDO.ts+6−0View file

modifiedapps/diffinery/src/repo/actions.ts+86−6View file

modifiedapps/diffinery/src/runner/containers.ts+7−0View file

modifiedapps/diffinery/src/web/account.tsx+76−2View file

modifiedapps/diffinery/src/web/actions.tsx+3−0View file

addedapps/diffinery/src/web/kill-switches.tsx+173−0View file

modifiedapps/diffinery/src/web/repo-settings.tsx+5−1View file

modifiedapps/diffinery/test/e2e/security.test.mjs+107−0View file

modifiedapps/diffinery/test/e2e/tasks.test.mjs+34−0View file

addedapps/diffinery/test/unit/killswitch.test.ts+129−0View file

		@@ -1,18 +1,19 @@
1	1	---
2	2	title: Site administration
3		-description: What site administrators do: manage people and access, invite people, issue sign-in links, oversee GitHub imports, read the audit log and check email.
	3	+description: What site administrators do: manage people and access, invite people, issue sign-in links, stop Actions with kill switches, oversee imports, read the audit log and check email.
4	4	---
5	5
6	6	Site administrators run a Diffinery instance. They decide who gets in, help people who lost their passkeys, and check the audit log and email delivery. This page covers who the site administrators are and each page in the Site administration group of Settings.
7	7
8	8	## Site administrators
9	9
10		-Site administrators have admin on every repository, can act as an owner of every organization (including its Settings tab, where the organization's name, picture and base permission are changed, and the Add people form on its People tab), and see an extra Site administration group in the Settings sidebar with three pages:
	10	+Site administrators have admin on every repository, can act as an owner of every organization (including its Settings tab, where the organization's name, picture and base permission are changed, and the Add people form on its People tab), and see an extra Site administration group in the Settings sidebar with four pages:
11	11
12	12	\| Page \| Path \| What it is for \|
13	13	\| --- \| --- \| --- \|
14	14	\| People and invitations \| `/settings/admin` \| Access requests, instance invitations, the list of users, account status and sign-in links \|
15	15	\| Email \| `/settings/admin/email` \| Email delivery status, template previews and the log of every email \|
	16	+\| Kill switches \| `/settings/admin/kill-switches` \| Stop Actions and agent sandboxes for a repository, an account or the whole instance \|
16	17	\| Audit log \| `/settings/audit` \| The most recent security-relevant events \|
17	18
18	19	Their Settings → Profile page shows Site admin as yes, and they have an admin badge in the users list.
		@@ -106,6 +107,74 @@
106	107
107	108	Anyone with the link can add a passkey to the account, so send it only to that person. The link also works for a first sign-in, for an account that exists but has no passkey yet.
108	109
	110	+## Kill switches
	111	+
	112	+Kill switches stop Actions and agent sandboxes quickly, for abuse, a runaway workflow or a cost problem. There are three, from the narrowest:
	113	+
	114	+\| Switch \| What stops \| What keeps working \|
	115	+\| --- \| --- \| --- \|
	116	+\| Suspend Actions for a repository \| Every workflow run of the repository, on hosted and self-hosted runners, and its agent sandboxes (including agent tasks' and reviews') \| Everything else in the repository: Git, issues, pull requests \|
	117	+\| Suspend Actions for an account \| The same, for every repository a person or an organization owns, including repositories created or transferred in later \| The account's other repositories' Git and collaboration \|
	118	+\| Pause hosted compute \| Hosted Actions jobs and agent sandboxes in every repository on the instance \| Self-hosted runners, which cost the instance nothing, and workflow runs themselves \|
	119	+
	120	+Nobody is exempt, not even site administrators' own accounts and organizations (unlike [account quotas](/docs/reference/limits#account-quotas)). Every change is in the audit log: `actions.suspend` and `actions.unsuspend` with the repository (`owner/name`) or the account's login as the target, `compute.pause` and `compute.unpause`, each with the reason and what was stopped.
	121	+
	122	+### While a switch is on
	123	+
	124	+While a repository or its owner is suspended:
	125	+
	126	+- no workflow run starts: pushes, pull requests, schedules and other events start nothing, and no run is recorded;
	127	+- Run workflow (`workflow_dispatch`), `repository_dispatch` and Re-run are refused with `403 Actions are suspended for this repository by a site administrator: <reason>` (or `... for every repository of <account> ...`);
	128	+- a job that would start fails instead, with "The job was not started: Actions are suspended ...", and a self-hosted runner isn't given a job that was already waiting;
	129	+- a new sandbox is refused with the same `403`, so agent tasks and reviews fail when they try to start theirs, with that message.
	130	+- [web terminals](/docs/agents/terminal) can't be opened or attached to, and open ones end within a minute ("A site administrator suspended Actions and sandboxes here").
	131	+
	132	+While hosted compute is paused:
	133	+
	134	+- a job on a hosted runner fails without starting: "The job was not started: Hosted runners are paused on this instance by a site administrator: <reason>. Self-hosted runners still work." Jobs of the same run on self-hosted runners run as usual;
	135	+- a new sandbox is refused with `403 Agent sandboxes are paused on this instance by a site administrator: <reason>` (agent tasks and reviews fail to start theirs), and a sandbox that went to sleep doesn't wake up.
	136	+- web terminals can't be opened or attached to, and open ones end within a minute.
	137	+
	138	+The repository's Actions page, its run pages and its Settings pages show a red banner with the reason, so the people affected know why. While hosted compute is paused, every Site administration page shows "Hosted compute is paused on this instance" at the top, so nobody forgets to turn it back on.
	139	+
	140	+### Turning a switch on stops what is running
	141	+
	142	+- Suspending a repository or an account cancels each of its unfinished runs ("Cancelled: Actions are suspended ...", including jobs on self-hosted runners) and closes each of its open sandboxes.
	143	+- Pausing hosted compute fails each running hosted job ("The job was stopped: Hosted runners are paused ...") and closes the open sandboxes, in every repository with hosted compute running (Diffinery finds them from the hosted minutes it is counting). Jobs that wait in the queue fail when they would start.
	144	+
	145	+The answer says what was stopped: `stopped` with `repositories`, `runs`, `jobs`, `sandboxes`, and `failed` for repositories that didn't answer (their work still can't start).
	146	+
	147	+Turning a switch off only lets new work start again. Nothing that was stopped restarts: re-run the runs you want and start new sandboxes.
	148	+
	149	+### From the web
	150	+
	151	+Settings → Kill switches (`/settings/admin/kill-switches`):
	152	+
	153	+- Hosted compute shows running or paused. Enter a Reason and click Pause hosted compute (you confirm first), or click Resume hosted compute.
	154	+- Suspend Actions for an account: the person's or organization's login and a Reason, then Suspend.
	155	+- Suspend Actions for a repository: `owner/name` and a Reason, then Suspend.
	156	+- Suspended lists every suspended account and repository with who suspended it, when and why, and a Lift button.
	157	+
	158	+A reason is required (at most 500 characters): it is shown to the people affected and kept in the audit log.
	159	+
	160	+### From the API
	161	+
	162	+Site administrators only (`403 only site administrators can ...` for anyone else). Reading needs any token that isn't limited to repositories; changing needs an `admin`-scoped token that isn't limited to repositories, or a browser session.
	163	+
	164	+\| Method and path \| Does \|
	165	+\| --- \| --- \|
	166	+\| `GET /api/v3/admin/repos/{owner}/{repo}/actions-suspension` \| `suspended`, `reason`, `suspended_by`, `suspended_at`, and `owner_suspended` when the owner's Actions are suspended \|
	167	+\| `PUT /api/v3/admin/repos/{owner}/{repo}/actions-suspension` \| Suspend, with `{"reason": "..."}` \|
	168	+\| `DELETE /api/v3/admin/repos/{owner}/{repo}/actions-suspension` \| Lift \|
	169	+\| `GET`, `PUT`, `DELETE /api/v3/admin/accounts/{login}/actions-suspension` \| The same for a person or an organization \|
	170	+\| `GET`, `PUT`, `DELETE /api/v3/admin/hosted-compute-pause` \| `paused`, `reason`, `paused_by` and `paused_at`; pause with `{"reason": "..."}`, resume with `DELETE` \|
	171	+
	172	+```sh
	173	+echo '{"reason": "crypto mining, see ticket 123"}' \| diffinery api PUT /admin/repos/acme/website/actions-suspension --input -
	174	+diffinery api DELETE /admin/repos/acme/website/actions-suspension
	175	+echo '{"reason": "runner image incident"}' \| diffinery api PUT /admin/hosted-compute-pause --input -
	176	+```
	177	+
109	178	## GitHub imports and linked accounts
110	179
111	180	Site administrators have extra reach over [imports from GitHub](/docs/migrate-from-github):
		@@ -233,6 +302,9 @@
233	302	\| `403 site admin only` from `DELETE /api/v3/diffinery/admin/linked-accounts/{id}` \| Your account isn't a site administrator. \|
234	303	\| `404 link not found` \| No GitHub link has that id. Ask the account's owner for the id from `GET /api/v3/diffinery/user/linked-accounts`. \|
235	304	\| "you cannot change your own status" \| Ask another site administrator. \|
	305	+\| `403 Actions are suspended ... by a site administrator` \| A kill switch is on for the repository or its owner. Lift it under Settings → Kill switches. \|
	306	+\| "Hosted runners are paused on this instance" \| Hosted compute is paused. Resume it under Settings → Kill switches, or run the job on a self-hosted runner. \|
	307	+\| `422 give a reason` \| Kill switches need a reason. \|
236	308	\| "enable this account first" \| Sign-in links can't be created for disabled accounts. Set the status to `active`, then create the link. \|
237	309	\| "enter a valid email address" \| The invite form needs an address like `friend@example.com`. \|
238	310	\| `403 only site admins can invite people without an organization` \| Instance invites from the CLI or API need a site administrator; others must pass `--org`. \|

		@@ -85,7 +85,7 @@
85	85	\| `task.stop` \| Someone stops a task (its starter, its agent or a repository admin) \| `owner/repo#<task number>` \| none \|
86	86	\| `terminal.terms` \| Someone accepts the notice for [terminals](/docs/agents/terminal) \| `owner/repo` \| `version` \|
87	87	\| `terminal.open` \| Someone opens a [terminal](/docs/agents/terminal) \| `owner/repo` \| `terminal`, `agent`, `ref`, `runtimes` (the Claude Code and Codex versions its buttons install) \|
88		-\| `terminal.end` \| A terminal ends \| `owner/repo` \| `terminal`, `reason` (`person`, `idle`, `limit`, `access`, `stopped`, `failed` or `disabled`) \|
	88	+\| `terminal.end` \| A terminal ends \| `owner/repo` \| `terminal`, `reason` (`person`, `idle`, `limit`, `access`, `stopped`, `failed`, `disabled` or `suspended`) \|
89	89	\| `project.memory` \| Someone saves or restores the [project](/docs/agents/projects) memory \| `owner/repo` \| `version`, `length`, and `restored` (the version restored) for a restore \|
90	90	\| `project.settings` \| A repository admin changes Settings → Project \| `owner/repo` \| `max_running`, `instructions_length`, `plan_first` \|
91	91	\| `project.coordinator` \| Someone turns their [coordinator](/docs/agents/projects#coordinator) on, changes it or turns it off \| `owner/repo` \| `enabled`, and when on `agent`, `runtime`, `provider` and `model` (OpenRouter), `tokens_per_run`, `tokens_per_task` \|
		@@ -110,6 +110,10 @@
110	110	\| `invite.create` \| A site admin creates an invite from Site administration \| the email address \| none \|
111	111	\| `site.invite` \| A site admin invites someone with `POST /api/v3/invitations` \| none \| `email` \|
112	112	\| `account.quotas.set` \| A site administrator changes an account's quotas with `PATCH /api/v3/admin/accounts/{login}/quotas` \| the account's login \| the quotas sent, e.g. `hosted_minutes` \|
	113	+\| `actions.suspend` \| A site administrator suspends Actions for a repository or an account ([kill switches](/docs/admin/site-administration#kill-switches)), on Settings → Kill switches or with `PUT /api/v3/admin/repos/{owner}/{repo}/actions-suspension` or `PUT /api/v3/admin/accounts/{login}/actions-suspension` \| `owner/name`, or the account's login \| `kind` (`repo` or `account`), `reason`, `via` (`web` or `api`), `stopped` (runs, jobs and sandboxes stopped) \|
	114	+\| `actions.unsuspend` \| A site administrator lifts that suspension (Lift, or `DELETE` on the same path) \| `owner/name`, or the account's login \| `kind`, `via` \|
	115	+\| `compute.pause` \| A site administrator pauses hosted compute on the instance (Pause hosted compute, or `PUT /api/v3/admin/hosted-compute-pause`) \| none \| `reason`, `via`, `stopped` \|
	116	+\| `compute.unpause` \| A site administrator resumes it (Resume hosted compute, or `DELETE /api/v3/admin/hosted-compute-pause`) \| none \| `via` \|
113	117
114	118	### Repositories
115	119

		@@ -1,0 +1,20 @@
	1	+-- Kill switches (lib/killswitch.ts). A site administrator suspends Actions for one repository, or for
	2	+-- every repository an account owns: no new runs, queued jobs fail, no new agent sandboxes. NULL is not suspended.
	3	+ALTER TABLE repos ADD COLUMN actions_suspended_at TEXT;
	4	+ALTER TABLE repos ADD COLUMN actions_suspended_by TEXT;
	5	+ALTER TABLE repos ADD COLUMN actions_suspended_reason TEXT;
	6	+ALTER TABLE accounts ADD COLUMN actions_suspended_at TEXT;
	7	+ALTER TABLE accounts ADD COLUMN actions_suspended_by TEXT;
	8	+ALTER TABLE accounts ADD COLUMN actions_suspended_reason TEXT;
	9	+
	10	+-- Instance-wide switches; a row is a switch that is on. `hosted_compute_paused` stops hosted Actions
	11	+-- jobs and agent sandboxes everywhere (self-hosted runners keep working); its value is the reason.
	12	+CREATE TABLE IF NOT EXISTS site_flags (
	13	+ key TEXT PRIMARY KEY,
	14	+ value TEXT NOT NULL,
	15	+ updated_at TEXT NOT NULL,
	16	+ updated_by TEXT
	17	+);
	18	+
	19	+-- What is running now across the instance (pausing hosted compute stops it).
	20	+CREATE INDEX IF NOT EXISTS compute_usage_running ON compute_usage(ended_at, started_at);

		@@ -1,4 +1,5 @@
1	1	import { DurableObject } from "cloudflare:workers";
	2	+import { blockedBy, killSwitches } from "../lib/killswitch.js";
2	3	import type { Env } from "../env.js";
3	4	import { Directory } from "../lib/directory.js";
4	5	import { HttpError, now, randomToken, sha256 } from "../lib/util.js";
		@@ -1158,6 +1159,8 @@
1158	1159
1159	1160	private async tickTerminal(t: TerminalRow, terminals: Terminals, dir: Directory): Promise<void> {
1160	1161	if (!terminalsEnabled(this.env)) return endTerminal(this.env, dir, t, "disabled");
	1162	+ // A kill switch (lib/killswitch.ts) that stops sandboxes here ends the terminal too.
	1163	+ if (blockedBy(await killSwitches(this.env, t.repo_id), "sandbox")) return endTerminal(this.env, dir, t, "suspended");
1161	1164	// The same rule as tasks, checked again every time: the person drives the agent, and both may write.
1162	1165	const [person, agent, repo] = await Promise.all([dir.account(t.person_id), dir.account(t.agent_id), dir.repoById(t.repo_id)]);
1163	1166	let caller;

		@@ -1,4 +1,5 @@
1	1	/** @jsxImportSource hono/jsx */
	2	+import { blockedBy, killSwitches } from "../lib/killswitch.js";
2	3	import type { FC } from "hono/jsx";
3	4	import { requireAccount, type AppContext } from "../auth/principal.js";
4	5	import { loadRepo } from "../http/common.js";
		@@ -298,6 +299,9 @@
298	299	const l = await forTerminals(c);
299	300	const t = await new Terminals(c.env).mine(terminalId(c.req.param("id")), me, l.repo);
300	301	if (t.status !== "running" \|\| !t.sandbox_id \|\| !terminalsEnabled(c.env)) return c.text("this terminal isn't running", 409);
	302	+ // A kill switch (lib/killswitch.ts) stops sandboxes here: no attaching until it's lifted.
	303	+ const block = blockedBy(await killSwitches(c.env, l.repo.id), "sandbox");
	304	+ if (block) return c.text(block, 403);
301	305	const dir = c.get("dir");
302	306	const agent = await dir.account(t.agent_id);
303	307	try {

		@@ -1,4 +1,5 @@
1	1	import type { AppContext } from "../auth/principal.js";
	2	+import { blockedBy, killSwitches } from "../lib/killswitch.js";
2	3	import type { Env } from "../env.js";
3	4	import type { loadRepo } from "../http/common.js";
4	5	import type { Account, Directory, Repo } from "../lib/directory.js";
		@@ -31,7 +32,7 @@
31	32	export const TERMINAL_TERMS_VERSION = 1;
32	33
33	34	export type TerminalStatus = "starting" \| "running" \| "ended";
34		-export type TerminalEndReason = "person" \| "idle" \| "limit" \| "access" \| "stopped" \| "failed" \| "disabled";
	35	+export type TerminalEndReason = "person" \| "idle" \| "limit" \| "access" \| "stopped" \| "failed" \| "disabled" \| "suspended";
35	36
36	37	export interface TerminalRow {
37	38	id: string;
		@@ -55,6 +56,7 @@
55	56	stopped: "Its sandbox stopped.",
56	57	failed: "Its sandbox didn't start.",
57	58	disabled: "Terminals were switched off on this Diffinery.",
	59	+ suspended: "A site administrator suspended Actions and sandboxes here, or paused hosted compute on this Diffinery.",
58	60	};
59	61
60	62	/** Terminals can be switched off for the whole instance (`TERMINALS=off`). */
		@@ -163,6 +165,9 @@
163	165	if (open.total >= MAX_TERMINALS) throw new HttpError(429, `you have ${MAX_TERMINALS} terminals open, the most allowed; end one first`);
164	166	// The account that owns the repository pays for the sandbox (lib/quotas.ts); say so now rather than
165	167	// after it fails to start. RepoDO.createSandbox checks again when it starts.
	168	+ // Kill switches (lib/killswitch.ts) refuse new sandboxes, terminals included.
	169	+ const block = blockedBy(await killSwitches(c.env, l.repo.id), "sandbox");
	170	+ if (block) throw new HttpError(403, block);
166	171	const quota = await mayStart(c.env, l.repo.owner_id, "sandbox");
167	172	if (!quota.ok) throw new HttpError(429, quota.message);
168	173	const t = await terminals.insert({ repo_id: l.repo.id, person_id: me.id, agent_id: agent.id, ref });

		@@ -49,6 +49,15 @@
49	49	"GET /orgs/{org}/usage": { scope: "read", anyToken: true },
50	50	"GET /admin/accounts/{login}/quotas": { scope: "read", anyToken: true },
51	51	"PATCH /admin/accounts/{login}/quotas": { scope: "admin" },
	52	+ "GET /admin/repos/{owner}/{repo}/actions-suspension": { scope: "read" },
	53	+ "PUT /admin/repos/{owner}/{repo}/actions-suspension": { scope: "admin" },
	54	+ "DELETE /admin/repos/{owner}/{repo}/actions-suspension": { scope: "admin" },
	55	+ "GET /admin/accounts/{login}/actions-suspension": { scope: "read" },
	56	+ "PUT /admin/accounts/{login}/actions-suspension": { scope: "admin" },
	57	+ "DELETE /admin/accounts/{login}/actions-suspension": { scope: "admin" },
	58	+ "GET /admin/hosted-compute-pause": { scope: "read" },
	59	+ "PUT /admin/hosted-compute-pause": { scope: "admin" },
	60	+ "DELETE /admin/hosted-compute-pause": { scope: "admin" },
52	61	"DELETE /orgs/{org}/invitations/{id}": { scope: "admin" },
53	62	"GET /invitations": { scope: "read" },
54	63	"DELETE /invitations/{id}": { scope: "admin" },
		@@ -146,6 +155,34 @@
146	155	repositories: ["integer", "Repositories the account owns."],
147	156	},
148	157	},
	158	+ "GET /admin/repos/{owner}/{repo}/actions-suspension": {
	159	+ summary: "Get a repository's Actions suspension",
	160	+ description: "Site administrators only. `suspended`, `reason`, `suspended_by`, `suspended_at`, and `owner_suspended` when its owner's Actions are suspended.",
	161	+ access: "site-admin",
	162	+ },
	163	+ "PUT /admin/repos/{owner}/{repo}/actions-suspension": {
	164	+ summary: "Suspend Actions for a repository",
	165	+ description: "Site administrators only. No runs start, queued jobs fail and no sandboxes open; what runs now stops (`stopped`). Audited as `actions.suspend`.",
	166	+ access: "site-admin",
	167	+ body: { reason: ["string", "Why (up to 500 characters). Shown to the people affected and kept in the audit log.", true] },
	168	+ },
	169	+ "DELETE /admin/repos/{owner}/{repo}/actions-suspension": { summary: "Lift a repository's Actions suspension", description: "Site administrators only. New work can start again; nothing restarts. Audited as `actions.unsuspend`.", access: "site-admin" },
	170	+ "GET /admin/accounts/{login}/actions-suspension": { summary: "Get an account's Actions suspension", description: "Site administrators only. `suspended`, `reason`, `suspended_by` and `suspended_at`.", access: "site-admin" },
	171	+ "PUT /admin/accounts/{login}/actions-suspension": {
	172	+ summary: "Suspend Actions for an account",
	173	+ description: "Site administrators only. As for a repository, for every repository the person or organization owns; their running runs and sandboxes stop now. Audited as `actions.suspend`.",
	174	+ access: "site-admin",
	175	+ body: { reason: ["string", "Why (up to 500 characters). Shown to the people affected and kept in the audit log.", true] },
	176	+ },
	177	+ "DELETE /admin/accounts/{login}/actions-suspension": { summary: "Lift an account's Actions suspension", description: "Site administrators only. New work can start again; nothing restarts. Audited as `actions.unsuspend`.", access: "site-admin" },
	178	+ "GET /admin/hosted-compute-pause": { summary: "Get the hosted compute pause", description: "Site administrators only. `paused`, `reason`, `paused_by` and `paused_at`.", access: "site-admin" },
	179	+ "PUT /admin/hosted-compute-pause": {
	180	+ summary: "Pause hosted compute",
	181	+ description: "Site administrators only. Hosted jobs fail without starting and sandboxes are refused everywhere; running ones stop now. Self-hosted runners still work. Audited as `compute.pause`.",
	182	+ access: "site-admin",
	183	+ body: { reason: ["string", "Why (up to 500 characters). Shown to the people affected and kept in the audit log.", true] },
	184	+ },
	185	+ "DELETE /admin/hosted-compute-pause": { summary: "Resume hosted compute", description: "Site administrators only. New hosted jobs and sandboxes can start again; nothing restarts. Audited as `compute.unpause`.", access: "site-admin" },
149	186	"GET /user/orgs": { summary: "List your organizations", description: "Each organization carries your `role` in it (member, admin or owner).", access: "account" },
150	187	"GET /user/repository_invitations": { summary: "List your repository invitations", description: "Pending invitations to repositories, sent to your username or your email address.", access: "account" },
151	188	"PATCH /user/repository_invitations/{id}": { summary: "Accept a repository invitation", description: "You get the role it offers. Audited as `invite.accept`.", access: "account", status: 204 },

		@@ -5,6 +5,7 @@
5	5	import { baseRole, type Account, type Directory, type Repo, type TokenScope } from "../lib/directory.js";
6	6	import { encryptSecret } from "../lib/crypto.js";
7	7	import { monthStart, quotasFor, setQuotas, usageOf } from "../lib/quotas.js";
	8	+import { changePause, changeSuspension, cleanReason, computePause, killSwitches, suspensionOf, type Stopped } from "../lib/killswitch.js";
8	9	import { publicJwk } from "../lib/jwt.js";
9	10	import { HttpError, b64, fromB64 } from "../lib/util.js";
10	11	import { atLeast, ROLE_LEVEL, type ProtectionRules, type Role } from "../repo/types.js";
		@@ -131,6 +132,89 @@
131	132	c.executionCtx.waitUntil(c.get("dir").audit({ actorId: a.id, actorLogin: a.login, action: "account.quotas.set", target: account.login, detail: b }));
132	133	return c.json(await usageJson(c, account));
133	134	});
	135	+// Kill switches (lib/killswitch.ts): site administrators suspend Actions for a repository or an
	136	+// account, or pause hosted compute on the whole instance. Same checks as the quotas above.
	137	+const siteAdmin = (c: AppContext, what: string) => {
	138	+ const a = requireAccount(c);
	139	+ if (!a.site_admin) throw new HttpError(403, `only site administrators can ${what}`);
	140	+ return a;
	141	+};
	142	+const stoppedJson = (s: Stopped \| null) => (s ? { stopped: { repositories: s.repositories, runs: s.runs, jobs: s.jobs, sandboxes: s.sandboxes, failed: s.failed } } : {});
	143	+const repoSuspension = async (c: AppContext, write: boolean) => {
	144	+ if (write) {
	145	+ requireAccountScope(c, "admin");
	146	+ requireUnlimitedToken(c, "suspend Actions");
	147	+ } else requireAccountScope(c, "read");
	148	+ const a = siteAdmin(c, "see or change Actions suspensions");
	149	+ const repo = await c.get("dir").repo(c.req.param("owner") ?? "", c.req.param("repo") ?? "");
	150	+ if (!repo) throw new HttpError(404, "no repository with that name");
	151	+ return { a, repo, json: async (stopped: Stopped \| null = null) => {
	152	+ const k = await killSwitches(c.env, repo.id);
	153	+ return { repository: `${repo.owner_login}/${repo.name}`, suspended: !!k.repo, reason: k.repo?.reason ?? null, suspended_by: k.repo?.by ?? null, suspended_at: k.repo?.at ?? null, owner_suspended: !!k.account, ...stoppedJson(stopped) };
	154	+ } };
	155	+};
	156	+api.get("/admin/repos/:owner/:repo/actions-suspension", async (c) => c.json(await (await repoSuspension(c, false)).json()));
	157	+api.put("/admin/repos/:owner/:repo/actions-suspension", async (c) => {
	158	+ const { a, repo, json } = await repoSuspension(c, true);
	159	+ const reason = cleanReason((await body<{ reason?: unknown }>(c).catch(() => ({}) as { reason?: unknown }))?.reason);
	160	+ const stopped = await changeSuspension(c.env, (e) => c.get("dir").audit(e), a, { kind: "repo", id: repo.id, name: `${repo.owner_login}/${repo.name}` }, reason, "api");
	161	+ return c.json(await json(stopped));
	162	+});
	163	+api.delete("/admin/repos/:owner/:repo/actions-suspension", async (c) => {
	164	+ const { a, repo, json } = await repoSuspension(c, true);
	165	+ await changeSuspension(c.env, (e) => c.get("dir").audit(e), a, { kind: "repo", id: repo.id, name: `${repo.owner_login}/${repo.name}` }, null, "api");
	166	+ return c.json(await json());
	167	+});
	168	+const accountSuspension = async (c: AppContext, write: boolean) => {
	169	+ if (write) {
	170	+ requireAccountScope(c, "admin");
	171	+ requireUnlimitedToken(c, "suspend Actions");
	172	+ } else requireAccountScope(c, "read");
	173	+ const a = siteAdmin(c, "see or change Actions suspensions");
	174	+ const account = await c.get("dir").accountByLogin(c.req.param("login") ?? "");
	175	+ if (!account \|\| account.kind === "agent") throw new HttpError(404, "no person or organization with that name");
	176	+ return { a, account, json: async (stopped: Stopped \| null = null) => {
	177	+ const s = await suspensionOf(c.env, { accountId: account.id });
	178	+ return { account: account.login, suspended: !!s, reason: s?.reason ?? null, suspended_by: s?.by ?? null, suspended_at: s?.at ?? null, ...stoppedJson(stopped) };
	179	+ } };
	180	+};
	181	+api.get("/admin/accounts/:login/actions-suspension", async (c) => c.json(await (await accountSuspension(c, false)).json()));
	182	+api.put("/admin/accounts/:login/actions-suspension", async (c) => {
	183	+ const { a, account, json } = await accountSuspension(c, true);
	184	+ const reason = cleanReason((await body<{ reason?: unknown }>(c).catch(() => ({}) as { reason?: unknown }))?.reason);
	185	+ const stopped = await changeSuspension(c.env, (e) => c.get("dir").audit(e), a, { kind: "account", id: account.id, login: account.login }, reason, "api");
	186	+ return c.json(await json(stopped));
	187	+});
	188	+api.delete("/admin/accounts/:login/actions-suspension", async (c) => {
	189	+ const { a, account, json } = await accountSuspension(c, true);
	190	+ await changeSuspension(c.env, (e) => c.get("dir").audit(e), a, { kind: "account", id: account.id, login: account.login }, null, "api");
	191	+ return c.json(await json());
	192	+});
	193	+const pauseJson = async (c: AppContext, stopped: Stopped \| null = null) => {
	194	+ const p = await computePause(c.env);
	195	+ return { paused: !!p, reason: p?.reason ?? null, paused_by: p?.by ?? null, paused_at: p?.at ?? null, ...stoppedJson(stopped) };
	196	+};
	197	+const pauseAdmin = (c: AppContext, write: boolean) => {
	198	+ if (write) {
	199	+ requireAccountScope(c, "admin");
	200	+ requireUnlimitedToken(c, "pause hosted compute");
	201	+ } else requireAccountScope(c, "read");
	202	+ return siteAdmin(c, "see or change the hosted compute pause");
	203	+};
	204	+api.get("/admin/hosted-compute-pause", async (c) => {
	205	+ pauseAdmin(c, false);
	206	+ return c.json(await pauseJson(c));
	207	+});
	208	+api.put("/admin/hosted-compute-pause", async (c) => {
	209	+ const a = pauseAdmin(c, true);
	210	+ const reason = cleanReason((await body<{ reason?: unknown }>(c).catch(() => ({}) as { reason?: unknown }))?.reason);
	211	+ return c.json(await pauseJson(c, await changePause(c.env, (e) => c.get("dir").audit(e), a, reason, "api")));
	212	+});
	213	+api.delete("/admin/hosted-compute-pause", async (c) => {
	214	+ const a = pauseAdmin(c, true);
	215	+ await changePause(c.env, (e) => c.get("dir").audit(e), a, null, "api");
	216	+ return c.json(await pauseJson(c));
	217	+});
134	218	api.get("/user/orgs", async (c) => c.json((await c.get("dir").orgsFor(requireAccount(c).id)).map((o) => ({ ...userJson(c, o), role: o.role }))));
135	219	api.post("/user/orgs", async (c) => {
136	220	requireAccountScope(c, "admin");

		@@ -1,0 +1,227 @@
	1	+import type { Env } from "../env.js";
	2	+import { MAX_RUN_MS } from "./quotas.js";
	3	+import { HttpError, now } from "./util.js";
	4	+
	5	+/**
	6	+ * Kill switches: a site administrator stops Actions and agent sandboxes at three levels.
	7	+ *
	8	+ * - A repository: its Actions are suspended (repos.actions_suspended_*).
	9	+ * - An account (a person or an organization): Actions are suspended for every repository it owns
	10	+ * (accounts.actions_suspended_*).
	11	+ * - The instance: hosted compute (hosted Actions jobs and agent sandboxes) is paused everywhere
	12	+ * (site_flags `hosted_compute_paused`). Self-hosted runners keep working: they cost the instance nothing.
	13	+ *
	14	+ * While a repository or its owner is suspended no workflow run starts, queued jobs (hosted or
	15	+ * self-hosted) fail instead of starting, and no sandbox opens. While the instance is paused hosted
	16	+ * jobs fail instead of starting and no sandbox opens. Suspending or pausing also stops what is
	17	+ * running then (RepoDO.stopActions); lifting one only allows new work again. Nobody is exempt,
	18	+ * not even site administrators' own accounts.
	19	+ */
	20	+
	21	+export const PAUSE_FLAG = "hosted_compute_paused";
	22	+/** Longest reason kept, in characters. */
	23	+export const MAX_REASON = 500;
	24	+
	25	+export interface Suspension {
	26	+ at: string;
	27	+ by: string \| null;
	28	+ reason: string \| null;
	29	+}
	30	+
	31	+export interface KillSwitches {
	32	+ repo: Suspension \| null;
	33	+ /** The owning account's suspension, with its login. */
	34	+ account: (Suspension & { login: string }) \| null;
	35	+ paused: Suspension \| null;
	36	+}
	37	+
	38	+/** What wants to start: a workflow run, a hosted or self-hosted job, or an agent sandbox. */
	39	+export type Work = "run" \| "hosted-job" \| "self-hosted-job" \| "sandbox";
	40	+
	41	+const tail = (reason: string \| null) => (reason ? `: ${reason}` : "");
	42	+
	43	+/** Every switch that applies to a repository, in one query (this is on the path of every job and sandbox). */
	44	+export async function killSwitches(env: Env, repoId: string): Promise<KillSwitches> {
	45	+ const r = await env.DB.prepare(
	46	+ `SELECT r.actions_suspended_at AS r_at, r.actions_suspended_by AS r_by, r.actions_suspended_reason AS r_reason,
	47	+ a.login AS login, a.actions_suspended_at AS a_at, a.actions_suspended_by AS a_by, a.actions_suspended_reason AS a_reason,
	48	+ f.value AS p_reason, f.updated_at AS p_at, f.updated_by AS p_by
	49	+ FROM repos r JOIN accounts a ON a.id = r.owner_id LEFT JOIN site_flags f ON f.key = ?
	50	+ WHERE r.id = ?`,
	51	+ )
	52	+ .bind(PAUSE_FLAG, repoId)
	53	+ .first<Record<string, string \| null>>();
	54	+ if (!r) return { repo: null, account: null, paused: await computePause(env) };
	55	+ return {
	56	+ repo: r.r_at ? { at: r.r_at, by: r.r_by ?? null, reason: r.r_reason ?? null } : null,
	57	+ account: r.a_at ? { at: r.a_at, by: r.a_by ?? null, reason: r.a_reason ?? null, login: r.login! } : null,
	58	+ paused: r.p_at ? { at: r.p_at, by: r.p_by ?? null, reason: r.p_reason \|\| null } : null,
	59	+ };
	60	+}
	61	+
	62	+/** Why `work` may not start under these switches (a sentence without a final period), or null when it may. */
	63	+export function blockedBy(k: KillSwitches, work: Work): string \| null {
	64	+ if (k.repo) return `Actions are suspended for this repository by a site administrator${tail(k.repo.reason)}`;
	65	+ if (k.account) return `Actions are suspended for every repository of ${k.account.login} by a site administrator${tail(k.account.reason)}`;
	66	+ if (k.paused && work === "hosted-job") return `Hosted runners are paused on this instance by a site administrator${tail(k.paused.reason)}. Self-hosted runners still work`;
	67	+ if (k.paused && work === "sandbox") return `Agent sandboxes are paused on this instance by a site administrator${tail(k.paused.reason)}`;
	68	+ return null;
	69	+}
	70	+
	71	+/** The instance's hosted compute pause, or null. */
	72	+export async function computePause(env: Env): Promise<Suspension \| null> {
	73	+ const f = await env.DB.prepare("SELECT value, updated_at, updated_by FROM site_flags WHERE key = ?").bind(PAUSE_FLAG).first<{ value: string; updated_at: string; updated_by: string \| null }>();
	74	+ return f ? { at: f.updated_at, by: f.updated_by, reason: f.value \|\| null } : null;
	75	+}
	76	+
	77	+/** A reason as sent by an administrator: required, trimmed and bounded. */
	78	+export function cleanReason(v: unknown): string {
	79	+ const s = typeof v === "string" ? v.trim().replace(/\s+/g, " ") : "";
	80	+ if (!s) throw new HttpError(422, "give a reason (it is shown to the people affected and kept in the audit log)");
	81	+ if (s.length > MAX_REASON) throw new HttpError(422, `the reason can be at most ${MAX_REASON} characters`);
	82	+ return s;
	83	+}
	84	+
	85	+/** Suspend (with a reason) or lift (null) Actions for a repository or an account. */
	86	+export async function setSuspension(env: Env, target: { repoId: string } \| { accountId: string }, s: { reason: string; by: string } \| null): Promise<void> {
	87	+ const [table, id] = "repoId" in target ? ["repos", target.repoId] : ["accounts", target.accountId];
	88	+ await env.DB.prepare(`UPDATE ${table} SET actions_suspended_at = ?, actions_suspended_by = ?, actions_suspended_reason = ? WHERE id = ?`)
	89	+ .bind(s ? now() : null, s?.by ?? null, s?.reason ?? null, id)
	90	+ .run();
	91	+}
	92	+
	93	+/** A repository's or an account's own suspension (not its owner's), or null. */
	94	+export async function suspensionOf(env: Env, target: { repoId: string } \| { accountId: string }): Promise<Suspension \| null> {
	95	+ const [table, id] = "repoId" in target ? ["repos", target.repoId] : ["accounts", target.accountId];
	96	+ const r = await env.DB.prepare(`SELECT actions_suspended_at AS at, actions_suspended_by AS by, actions_suspended_reason AS reason FROM ${table} WHERE id = ?`).bind(id).first<{ at: string \| null; by: string \| null; reason: string \| null }>();
	97	+ return r?.at ? { at: r.at, by: r.by, reason: r.reason } : null;
	98	+}
	99	+
	100	+/** Pause (with a reason) or resume (null) hosted compute on the instance. */
	101	+export async function setComputePause(env: Env, s: { reason: string; by: string } \| null): Promise<void> {
	102	+ if (!s) await env.DB.prepare("DELETE FROM site_flags WHERE key = ?").bind(PAUSE_FLAG).run();
	103	+ else
	104	+ await env.DB.prepare("INSERT INTO site_flags (key, value, updated_at, updated_by) VALUES (?, ?, ?, ?) ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = excluded.updated_at, updated_by = excluded.updated_by")
	105	+ .bind(PAUSE_FLAG, s.reason, now(), s.by)
	106	+ .run();
	107	+}
	108	+
	109	+/**
	110	+ * Repositories with hosted compute running now (compute_usage rows still open), with the ids of
	111	+ * their jobs and sandboxes. A sandbox that slept and woke has segment ids `<sandbox id>:<n>`;
	112	+ * these are reported as the sandbox's own id. Rows older than the longest a job or sandbox can
	113	+ * run are counted as ended, as quotas count them.
	114	+ */
	115	+export async function runningCompute(env: Env, at = new Date()): Promise<Map<string, { jobs: string[]; sandboxes: string[] }>> {
	116	+ const since = new Date(at.getTime() - MAX_RUN_MS.sandbox).toISOString();
	117	+ const rows = await env.DB.prepare("SELECT id, repo_id, kind, started_at FROM compute_usage WHERE ended_at IS NULL AND started_at > ?").bind(since).all<{ id: string; repo_id: string; kind: "job" \| "sandbox"; started_at: string }>();
	118	+ const out = new Map<string, { jobs: string[]; sandboxes: string[] }>();
	119	+ for (const r of rows.results) {
	120	+ if (Date.parse(r.started_at) + MAX_RUN_MS[r.kind] <= at.getTime()) continue;
	121	+ const e = out.get(r.repo_id) ?? { jobs: [], sandboxes: [] };
	122	+ const id = r.kind === "sandbox" ? r.id.split(":")[0]! : r.id;
	123	+ const list = r.kind === "job" ? e.jobs : e.sandboxes;
	124	+ if (!list.includes(id)) list.push(id);
	125	+ out.set(r.repo_id, e);
	126	+ }
	127	+ return out;
	128	+}
	129	+
	130	+/** Suspended repositories and accounts, for the site administration page. */
	131	+export async function suspendedList(env: Env): Promise<{ accounts: Array<Suspension & { login: string; kind: string }>; repos: Array<Suspension & { owner: string; name: string }> }> {
	132	+ const [a, r] = await Promise.all([
	133	+ env.DB.prepare("SELECT login, kind, actions_suspended_at AS at, actions_suspended_by AS by, actions_suspended_reason AS reason FROM accounts WHERE actions_suspended_at IS NOT NULL ORDER BY actions_suspended_at DESC LIMIT 200").all<Suspension & { login: string; kind: string }>(),
	134	+ env.DB.prepare(
	135	+ "SELECT a.login AS owner, r.name AS name, r.actions_suspended_at AS at, r.actions_suspended_by AS by, r.actions_suspended_reason AS reason FROM repos r JOIN accounts a ON a.id = r.owner_id WHERE r.actions_suspended_at IS NOT NULL ORDER BY r.actions_suspended_at DESC LIMIT 200",
	136	+ ).all<Suspension & { owner: string; name: string }>(),
	137	+ ]);
	138	+ return { accounts: a.results, repos: r.results };
	139	+}
	140	+
	141	+// ---------------------------------------------------------------- turning switches on and off (REST and web)
	142	+
	143	+export type Stopped = { repositories: number; runs: number; jobs: number; sandboxes: number; failed: number };
	144	+
	145	+/** Ask each repository's RepoDO to stop its work (RepoDO.stopActions), a few at a time. */
	146	+export async function stopIn(env: Env, repoIds: string[], o: { hostedOnly: boolean; message: string }): Promise<Stopped> {
	147	+ const out: Stopped = { repositories: 0, runs: 0, jobs: 0, sandboxes: 0, failed: 0 };
	148	+ const queue = [...repoIds];
	149	+ const worker = async () => {
	150	+ for (let id = queue.shift(); id !== undefined; id = queue.shift()) {
	151	+ try {
	152	+ const r = await env.REPO.get(env.REPO.idFromName(id)).stopActions(o);
	153	+ out.repositories++;
	154	+ out.runs += r.runs;
	155	+ out.jobs += r.jobs;
	156	+ out.sandboxes += r.sandboxes;
	157	+ } catch (e) {
	158	+ console.error(`kill switch: stopping work in repository ${id} failed`, e);
	159	+ out.failed++;
	160	+ }
	161	+ }
	162	+ };
	163	+ await Promise.all(Array.from({ length: Math.min(8, queue.length) }, worker));
	164	+ return out;
	165	+}
	166	+
	167	+type Actor = { id: string; login: string };
	168	+type Audit = (entry: { actorId?: string; actorLogin?: string; action: string; target?: string; detail?: unknown }) => Promise<void>;
	169	+
	170	+/**
	171	+ * Suspend Actions for a repository or an account (`reason`), or lift the suspension (null), and
	172	+ * audit it as `actions.suspend` or `actions.unsuspend`. Suspending stops what runs now: every
	173	+ * unfinished run is cancelled and every open sandbox closed, in the repository or in each one
	174	+ * the account owns. Lifting what isn't suspended changes nothing and isn't audited.
	175	+ */
	176	+export async function changeSuspension(
	177	+ env: Env,
	178	+ audit: Audit,
	179	+ actor: Actor,
	180	+ target: { kind: "repo"; id: string; name: string } \| { kind: "account"; id: string; login: string },
	181	+ reason: string \| null,
	182	+ via: "api" \| "web",
	183	+): Promise<Stopped \| null> {
	184	+ const key = target.kind === "repo" ? { repoId: target.id } : { accountId: target.id };
	185	+ const label = target.kind === "repo" ? target.name : target.login;
	186	+ if (reason === null) {
	187	+ if (!(await suspensionOf(env, key))) return null;
	188	+ await setSuspension(env, key, null);
	189	+ await audit({ actorId: actor.id, actorLogin: actor.login, action: "actions.unsuspend", target: label, detail: { kind: target.kind, via } });
	190	+ return null;
	191	+ }
	192	+ await setSuspension(env, key, { reason, by: actor.login });
	193	+ const s = { at: now(), by: actor.login, reason };
	194	+ const message = blockedBy(target.kind === "repo" ? { repo: s, account: null, paused: null } : { repo: null, account: { ...s, login: target.login }, paused: null }, "run")!;
	195	+ let stopped: Stopped \| null = null;
	196	+ try {
	197	+ const ids = target.kind === "repo" ? [target.id] : (await env.DB.prepare("SELECT id FROM repos WHERE owner_id = ?").bind(target.id).all<{ id: string }>()).results.map((r) => r.id);
	198	+ stopped = await stopIn(env, ids, { hostedOnly: false, message });
	199	+ } finally {
	200	+ // The switch is on even if stopping what ran failed: it is recorded either way.
	201	+ await audit({ actorId: actor.id, actorLogin: actor.login, action: "actions.suspend", target: label, detail: { kind: target.kind, reason, via, stopped } });
	202	+ }
	203	+ return stopped;
	204	+}
	205	+
	206	+/**
	207	+ * Pause hosted compute on the instance (`reason`), or resume it (null), audited as
	208	+ * `compute.pause` or `compute.unpause`. Pausing stops the hosted jobs and sandboxes running now:
	209	+ * open compute_usage rows say which repositories to ask. Resuming what isn't paused changes nothing.
	210	+ */
	211	+export async function changePause(env: Env, audit: Audit, actor: Actor, reason: string \| null, via: "api" \| "web"): Promise<Stopped \| null> {
	212	+ if (reason === null) {
	213	+ if (!(await computePause(env))) return null;
	214	+ await setComputePause(env, null);
	215	+ await audit({ actorId: actor.id, actorLogin: actor.login, action: "compute.unpause", detail: { via } });
	216	+ return null;
	217	+ }
	218	+ await setComputePause(env, { reason, by: actor.login });
	219	+ const message = blockedBy({ repo: null, account: null, paused: { at: now(), by: actor.login, reason } }, "hosted-job")!;
	220	+ let stopped: Stopped \| null = null;
	221	+ try {
	222	+ stopped = await stopIn(env, [...(await runningCompute(env)).keys()], { hostedOnly: true, message });
	223	+ } finally {
	224	+ await audit({ actorId: actor.id, actorLogin: actor.login, action: "compute.pause", detail: { reason, via, stopped } });
	225	+ }
	226	+ return stopped;
	227	+}

		@@ -4,7 +4,8 @@
4	4	import type { Account } from "../lib/directory.js";
5	5	import { loadRepo } from "../http/common.js";
6	6	import { HttpError, now } from "../lib/util.js";
7		-import { SettingsNav, type SettingsTab } from "../web/account.js";
	7	+import { ADMIN_TABS, SettingsNav, type SettingsTab } from "../web/account.js";
	8	+import { ComputePausedBanner } from "../web/kill-switches.js";
8	9	import { csrfToken, formData } from "../web/csrf.js";
9	10	import { KCheckbox, KSelect } from "../web/kumo.js";
10	11	import { Form, Time, WATCH_OPTIONS, act, flash, post, prm, render, repoUrl, type WebApp } from "../web/ui.js";
		@@ -34,7 +35,10 @@
34	35	{ title },
35	36	<div class="grid left">
36	37	<SettingsNav me={me} tab={tab} />
37		- <div>{body}</div>
	38	+ <div>
	39	+ {ADMIN_TABS.includes(tab) ? <ComputePausedBanner env={c.env} /> : null}
	40	+ {body}
	41	+ </div>
38	42	</div>,
39	43	);
40	44

		@@ -1121,6 +1121,10 @@
1121	1121	closeTaskSandbox(...a: Parameters<Actions["closeTaskSandbox"]>) {
1122	1122	return this.actions.closeTaskSandbox(...a);
1123	1123	}
	1124	+ /** A kill switch was turned on (lib/killswitch.ts): stop what runs here now. */
	1125	+ stopActions(...a: Parameters<Actions["stopActions"]>) {
	1126	+ return this.actions.stopActions(...a);
	1127	+ }
1124	1128
1125	1129	plannedRunError(runId: string, message: string) {
1126	1130	return this.actions.plannedError(runId, message);
		@@ -1562,6 +1566,8 @@
1562	1566	if (!atLeast(caller.role, "write")) throw new HttpError(403, "write access is required to dispatch events");
1563	1567	const sha = this.getRef(this.headRef);
1564	1568	if (!sha) throw new HttpError(422, "the repository is empty");
	1569	+ // Suspended Actions refuse the dispatch with the reason (triggerWorkflows would swallow it).
	1570	+ await this.actions.refuseIfBlocked("run");
1565	1571	const m = this.meta;
1566	1572	const payload = { action: eventType, branch: m.defaultBranch, client_payload: clientPayload };
1567	1573	this.emit({ type: "repository_dispatch", action: eventType, repoId: m.repoId, owner: m.owner, repo: m.name, payload: JSON.parse(JSON.stringify(payload)), actor: { id: caller.id, login: caller.login, kind: caller.kind, onBehalfOf: caller.onBehalfOf }, at: now() });

		@@ -23,6 +23,7 @@
23	23	import { ghRepo, numericId } from "../http/gh.js";
24	24	import { HttpError, newId, now, randomToken, sha256 } from "../lib/util.js";
25	25	import { endCompute, mayStart, startCompute } from "../lib/quotas.js";
	26	+import { blockedBy, killSwitches, type Work } from "../lib/killswitch.js";
26	27	import type { Env } from "../env.js";
27	28	import type { RepoDO } from "./RepoDO.js";
28	29	import { completePayload } from "./event-payloads.js";
		@@ -419,6 +420,7 @@
419	420
420	421	async dispatch(workflowFile: string, ref: string, inputs: Record<string, unknown>, caller: Caller): Promise<RunInfo> {
421	422	if (!atLeast(caller.role, "write")) throw new HttpError(403, "write access is required to run workflows");
	423	+ await this.refuseIfBlocked("run");
422	424	const sha = await this.repo.resolve(ref);
423	425	if (!sha) throw new HttpError(422, `unknown ref ${ref}`);
424	426	const all = await this.loadWorkflows(sha);
		@@ -504,6 +506,8 @@
504	506	}
505	507
506	508	private async createFailedRun(path: string, event: string, ref: string, sha: string, caller: Actor, errors: string[]): Promise<void> {
	509	+ // While Actions are suspended no run is recorded, not even a failed one.
	510	+ if (await this.blocked("run")) return;
507	511	const id = newId();
508	512	const t = now();
509	513	this.sql.exec(
		@@ -556,6 +560,13 @@
556	560	// runs, so workflows cannot trigger themselves in a loop; workflow_dispatch and
557	561	// repository_dispatch are the exceptions.
558	562	if (isJobCaller(o.caller) && o.event !== "workflow_dispatch" && o.event !== "repository_dispatch") return null;
	563	+ // A site administrator suspended Actions here (lib/killswitch.ts): events start nothing, and a
	564	+ // dispatch, which someone asked for, is refused with the reason.
	565	+ const block = await this.blocked("run");
	566	+ if (block) {
	567	+ if (o.event === "workflow_dispatch" \|\| o.event === "repository_dispatch") throw new HttpError(403, block);
	568	+ return null;
	569	+ }
559	570	// A dispatch loop (or a flood of pushes) stops growing here.
560	571	const active = this.sql.exec<{ n: number }>("SELECT count(*) AS n FROM runs WHERE status != 'completed'").one().n;
561	572	if (active >= MAX_ACTIVE_RUNS) {
		@@ -975,7 +986,11 @@
975	986
976	987	private async launch(jobId: string): Promise<void> {
977	988	const job = this.jobRow(jobId);
978		- if (job.status === "completed" \|\| !(await this.gate(job)) \|\| !(await this.concurrencySlot(job))) return;
	989	+ if (job.status === "completed") return;
	990	+ // Kill switches (lib/killswitch.ts): a suspended repository or owner starts no job; a paused instance no hosted one.
	991	+ const block = await this.blocked(job.labels ? "self-hosted-job" : "hosted-job");
	992	+ if (block) return this.completeJob(jobId, { conclusion: "failure", outputs: {}, failure: { message: `The job was not started: ${block}.` } });
	993	+ if (!(await this.gate(job)) \|\| !(await this.concurrencySlot(job))) return;
979	994	const run = this.runRow(job.run_id);
980	995	const m = this.repo.meta;
981	996	if (job.labels) {
		@@ -1318,6 +1333,12 @@
1318	1333	.toArray();
1319	1334	const job = waiting.find((j) => (JSON.parse(j.labels!) as string[]).every((l) => offered.has(l)));
1320	1335	if (!job) return null;
	1336	+ // A job parked before Actions were suspended here isn't handed out.
	1337	+ const block = await this.blocked("self-hosted-job");
	1338	+ if (block) {
	1339	+ await this.completeJob(job.id, { conclusion: "failure", outputs: {}, failure: { message: `The job was not started: ${block}.` } });
	1340	+ return null;
	1341	+ }
1321	1342	const m = this.repo.meta;
1322	1343	const token = `djt_${m.repoId}_${job.id}_${randomToken(24)}`;
1323	1344	const container = `self-hosted:${runnerName.slice(0, 64)}`;
		@@ -1511,6 +1532,7 @@
1511	1532	async rerun(runId: string, caller: Caller): Promise<RunInfo> {
1512	1533	if (!atLeast(caller.role, "write")) throw new HttpError(403, "permission denied");
1513	1534	if (caller.id.startsWith("job:")) throw new HttpError(403, "a workflow's GITHUB_TOKEN cannot re-run workflows");
	1535	+ await this.refuseIfBlocked("run");
1514	1536	const old = this.runRow(runId);
1515	1537	const wf = (await this.loadWorkflows(old.sha)).find((w) => w.path === old.workflow_path)?.workflow;
1516	1538	if (!wf) throw new HttpError(422, "workflow file no longer valid at this commit");
		@@ -1527,10 +1549,12 @@
1527	1549	// Re-running code an agent wrote keeps it acting as the agent, whoever re-runs it.
1528	1550	agentOrigin: !!old.agent_origin,
1529	1551	});
	1552	+ // Not started after all (Actions suspended meanwhile, or too many unfinished runs).
	1553	+ if (!run) throw new HttpError(409, "the run was not started; try again later");
1530	1554	// Whoever started an earlier attempt still can't approve this one where an environment prevents self-review.
1531	1555	const prior = [...new Set([...((old.prior_actors ? JSON.parse(old.prior_actors) : []) as string[]), (JSON.parse(old.actor) as Actor).id])];
1532		- this.sql.exec("UPDATE runs SET attempt = ?, run_number = ?, prior_actors = ? WHERE id = ?", old.attempt + 1, old.run_number, JSON.stringify(prior), run!.id);
1533		- return toRun(this.runRow(run!.id));
	1556	+ this.sql.exec("UPDATE runs SET attempt = ?, run_number = ?, prior_actors = ? WHERE id = ?", old.attempt + 1, old.run_number, JSON.stringify(prior), run.id);
	1557	+ return toRun(this.runRow(run.id));
1534	1558	}
1535	1559
1536	1560	// ---------------------------------------------------------------- runner protocol
		@@ -2017,6 +2041,7 @@
2017	2041	.one().n;
2018	2042	if (open >= MAX_OPEN_SANDBOXES) throw new HttpError(429, `you have ${open} open sandboxes in this repository, the most allowed; close one first`);
2019	2043	}
	2044	+ await this.refuseIfBlocked("sandbox", false);
2020	2045	// The account that owns the repository pays for sandboxes too (lib/quotas.ts).
2021	2046	const quota = await mayStart(this.env, this.repo.meta.ownerId, "sandbox");
2022	2047	if (!quota.ok) throw new HttpError(429, quota.message);
		@@ -2072,6 +2097,11 @@
2072	2097
2073	2098	async closeSandbox(id: string, caller: Caller): Promise<void> {
2074	2099	this.getSandbox(id, caller);
	2100	+ await this.endSandbox(id);
	2101	+ }
	2102	+
	2103	+ /** Close a sandbox: its token stops working, its time stops counting and its container stops. */
	2104	+ private async endSandbox(id: string): Promise<void> {
2075	2105	this.sql.exec("UPDATE sandboxes SET status = 'closed' WHERE id = ?", id);
2076	2106	await endCompute(this.env, id);
2077	2107	await this.env.SANDBOX.get(this.env.SANDBOX.idFromName(id)).shutdown();
		@@ -2081,8 +2111,58 @@
2081	2111	async closeTaskSandbox(id: string): Promise<void> {
2082	2112	const s = this.sql.exec<{ kind: string }>("SELECT kind FROM sandboxes WHERE id = ?", id).toArray()[0];
2083	2113	if (s?.kind !== "task" && s?.kind !== "terminal") return;
2084		- this.sql.exec("UPDATE sandboxes SET status = 'closed' WHERE id = ?", id);
2085		- await endCompute(this.env, id);
2086		- await this.env.SANDBOX.get(this.env.SANDBOX.idFromName(id)).shutdown();
	2114	+ await this.endSandbox(id);
2087	2115	}
	2116	+
	2117	+ // ---------------------------------------------------------------- kill switches (lib/killswitch.ts)
	2118	+
	2119	+ /**
	2120	+ * Why `work` may not start here now, or null. One D1 read. A failed read lets work start
	2121	+ * (`failOpen`), as quotas do for jobs: a D1 outage shouldn't stop every repository's CI.
	2122	+ */
	2123	+ private async blocked(work: Work, failOpen = true): Promise<string \| null> {
	2124	+ try {
	2125	+ return blockedBy(await killSwitches(this.env, this.repo.meta.repoId), work);
	2126	+ } catch (e) {
	2127	+ if (!failOpen) throw e;
	2128	+ console.error("kill switch check failed", e);
	2129	+ return null;
	2130	+ }
	2131	+ }
	2132	+
	2133	+ /** Refuse (403, with the reason) what someone asked to start while a kill switch stops it. */
	2134	+ async refuseIfBlocked(work: Work, failOpen = true): Promise<void> {
	2135	+ const block = await this.blocked(work, failOpen);
	2136	+ if (block) throw new HttpError(403, block);
	2137	+ }
	2138	+
	2139	+ /**
	2140	+ * A site administrator just suspended Actions here or for the owner (`hostedOnly` false): every
	2141	+ * unfinished run is cancelled and every open sandbox closed. Or paused hosted compute on the
	2142	+ * instance (`hostedOnly`): running hosted jobs fail and open sandboxes close; self-hosted jobs go on.
	2143	+ * `message` says why. Jobs still queued fail when they would start (launch).
	2144	+ */
	2145	+ async stopActions(o: { hostedOnly: boolean; message: string }): Promise<{ runs: number; jobs: number; sandboxes: number }> {
	2146	+ let runs = 0;
	2147	+ let jobs = 0;
	2148	+ let sandboxes = 0;
	2149	+ if (o.hostedOnly) {
	2150	+ for (const j of this.sql.exec<JobRow>("SELECT * FROM jobs WHERE labels IS NULL AND container IS NOT NULL AND status IN ('waiting', 'in_progress') ORDER BY rowid").toArray()) {
	2151	+ await this.stopContainer(j);
	2152	+ await this.completeJob(j.id, { conclusion: "failure", outputs: {}, failure: { message: `The job was stopped: ${o.message}.` } });
	2153	+ jobs++;
	2154	+ }
	2155	+ } else {
	2156	+ for (const r of this.sql.exec<{ id: string }>("SELECT id FROM runs WHERE status != 'completed' ORDER BY rowid").toArray()) {
	2157	+ jobs += this.sql.exec<{ n: number }>("SELECT count(*) AS n FROM jobs WHERE run_id = ? AND status != 'completed'", r.id).one().n;
	2158	+ await this.cancelInternal(r.id, `Cancelled: ${o.message}.`);
	2159	+ runs++;
	2160	+ }
	2161	+ }
	2162	+ for (const s of this.sql.exec<{ id: string }>("SELECT id FROM sandboxes WHERE status = 'running'").toArray()) {
	2163	+ await this.endSandbox(s.id).catch((e: unknown) => console.error("closing a sandbox failed", e));
	2164	+ sandboxes++;
	2165	+ }
	2166	+ return { runs, jobs, sandboxes };
	2167	+ }
2088	2168	}

		@@ -3,6 +3,7 @@
3	3	import type { StopParams } from "@cloudflare/containers";
4	4	import type { Env } from "../env.js";
5	5	import { endCompute, startCompute } from "../lib/quotas.js";
	6	+import { blockedBy, killSwitches } from "../lib/killswitch.js";
6	7
7	8	/**
8	9	* One container per workflow job. The runner agent inside fetches its job spec
		@@ -71,6 +72,12 @@
71	72	// A sandbox that slept after 30 idle minutes starts again for this call: its time counts again.
72	73	const usage = await this.ctx.storage.get<{ id: string; accountId: string; repoId: string; part?: number }>("usage");
73	74	if (usage && !this.ctx.container?.running) {
	75	+ // It doesn't wake while a kill switch stops sandboxes (lib/killswitch.ts).
	76	+ const block = await killSwitches(this.env, usage.repoId).then(
	77	+ (k) => blockedBy(k, "sandbox"),
	78	+ () => null,
	79	+ );
	80	+ if (block) return { status: 403, body: JSON.stringify({ message: block }), json: true };
74	81	const part = (usage.part ?? 0) + 1;
75	82	await startCompute(this.env, { id: `${usage.id}:${part}`, accountId: usage.accountId, repoId: usage.repoId, kind: "sandbox" });
76	83	await this.ctx.storage.put("usage", { ...usage, part });

		@@ -20,6 +20,8 @@
20	20	import { MAX_AVATAR_BYTES } from "../lib/image.js";
21	21	import { gatherToday } from "../agentic/today.js";
22	22	import { TodayView } from "../agentic/web.js";
	23	+import { ComputePausedBanner, KillSwitchesPage } from "./kill-switches.js";
	24	+import { changePause, changeSuspension, cleanReason, type Stopped } from "../lib/killswitch.js";
23	25
24	26	// ---------------------------------------------------------------- shared bits
25	27
		@@ -641,7 +643,9 @@
641	643
642	644	// ---------------------------------------------------------------- /settings
643	645
644		-export type SettingsTab = "profile" \| "appearance" \| "security" \| "tokens" \| "agents" \| "model-access" \| "orgs" \| "invitations" \| "notifications" \| "linked" \| "admin" \| "email" \| "audit";
	646	+export type SettingsTab = "profile" \| "appearance" \| "security" \| "tokens" \| "agents" \| "model-access" \| "orgs" \| "invitations" \| "notifications" \| "linked" \| "admin" \| "email" \| "audit" \| "kill-switches";
	647	+/** The site administration pages: they show when hosted compute is paused. */
	648	+export const ADMIN_TABS: readonly SettingsTab[] = ["admin", "email", "audit", "kill-switches"];
645	649
646	650	export const SettingsNav: FC<{ me: Account; tab: SettingsTab }> = ({ me, tab }) => (
647	651	<aside class="side vt-side">
		@@ -694,6 +698,9 @@
694	698	<a href="/settings/admin/email" class={tab === "email" ? "on" : ""}>
695	699	Email
696	700	</a>
	701	+ <a href="/settings/admin/kill-switches" class={tab === "kill-switches" ? "on" : ""}>
	702	+ Kill switches
	703	+ </a>
697	704	<a href="/settings/audit" class={tab === "audit" ? "on" : ""}>
698	705	Audit log
699	706	</a>
		@@ -733,7 +740,10 @@
733	740	{ title, status },
734	741	<div class="grid left">
735	742	<SettingsNav me={me} tab={tab} />
736		- <div>{body}</div>
	743	+ <div>
	744	+ {ADMIN_TABS.includes(tab) ? <ComputePausedBanner env={c.env} /> : null}
	745	+ {body}
	746	+ </div>
737	747	</div>,
738	748	);
739	749
		@@ -1588,6 +1598,70 @@
1588	1598	);
1589	1599	});
1590	1600
	1601	+ // Kill switches (lib/killswitch.ts)
	1602	+ web.get("/settings/admin/kill-switches", async (c) => {
	1603	+ const me = requireSiteAdmin(c);
	1604	+ return page(c, me, "kill-switches", "Kill switches", <KillSwitchesPage env={c.env} t={csrfToken(c)} />);
	1605	+ });
	1606	+ const KS = "/settings/admin/kill-switches";
	1607	+ const auditFn = (c: AppContext) => (e: Parameters<Directory["audit"]>[0]) => c.get("dir").audit(e);
	1608	+ const stoppedNote = (s: Stopped \| null) => (s ? ` Stopped ${s.runs} runs, ${s.jobs} jobs and ${s.sandboxes} sandboxes${s.failed ? `; ${s.failed} repositories didn't answer, so their work stops when it next tries to start` : ""}.` : "");
	1609	+ post(web, `${KS}/pause`, async (c, f) => {
	1610	+ const me = requireSiteAdmin(c);
	1611	+ return act(c, KS, async () => {
	1612	+ const stopped = await changePause(c.env, auditFn(c), me, cleanReason(f.s("reason")), "web");
	1613	+ flash(c, "notice", `Hosted compute is paused.${stoppedNote(stopped)}`);
	1614	+ });
	1615	+ });
	1616	+ post(web, `${KS}/resume`, async (c) => {
	1617	+ const me = requireSiteAdmin(c);
	1618	+ return act(c, KS, async () => changePause(c.env, auditFn(c), me, null, "web").then(() => undefined), "Hosted compute is running again. Nothing that was stopped restarts.");
	1619	+ });
	1620	+ const accountTarget = async (c: AppContext, login: string) => {
	1621	+ const account = await c.get("dir").accountByLogin(login.trim());
	1622	+ if (!account \|\| account.kind === "agent") throw new HttpError(404, "no person or organization with that name");
	1623	+ return { kind: "account" as const, id: account.id, login: account.login };
	1624	+ };
	1625	+ const repoTarget = async (c: AppContext, owner: string, name: string) => {
	1626	+ const repo = await c.get("dir").repo(owner.trim(), name.trim());
	1627	+ if (!repo) throw new HttpError(404, "no repository with that name; use owner/name");
	1628	+ return { kind: "repo" as const, id: repo.id, name: `${repo.owner_login}/${repo.name}` };
	1629	+ };
	1630	+ post(web, `${KS}/accounts`, async (c, f) => {
	1631	+ const me = requireSiteAdmin(c);
	1632	+ return act(c, KS, async () => {
	1633	+ const target = await accountTarget(c, f.s("login"));
	1634	+ const stopped = await changeSuspension(c.env, auditFn(c), me, target, cleanReason(f.s("reason")), "web");
	1635	+ flash(c, "notice", `Actions are suspended for every repository of ${target.login}.${stoppedNote(stopped)}`);
	1636	+ });
	1637	+ });
	1638	+ post(web, `${KS}/accounts/:login/lift`, async (c) => {
	1639	+ const me = requireSiteAdmin(c);
	1640	+ return act(c, KS, async () => {
	1641	+ const target = await accountTarget(c, prm(c, "login"));
	1642	+ await changeSuspension(c.env, auditFn(c), me, target, null, "web");
	1643	+ flash(c, "notice", `Actions are no longer suspended for ${target.login}.`);
	1644	+ });
	1645	+ });
	1646	+ post(web, `${KS}/repos`, async (c, f) => {
	1647	+ const me = requireSiteAdmin(c);
	1648	+ return act(c, KS, async () => {
	1649	+ const [owner, name, extra] = f.s("repo").trim().replace(/^\/+\|\/+$/g, "").split("/");
	1650	+ if (!owner \|\| !name \|\| extra !== undefined) throw new HttpError(422, "enter the repository as owner/name");
	1651	+ const target = await repoTarget(c, owner, name);
	1652	+ const stopped = await changeSuspension(c.env, auditFn(c), me, target, cleanReason(f.s("reason")), "web");
	1653	+ flash(c, "notice", `Actions are suspended for ${target.name}.${stoppedNote(stopped)}`);
	1654	+ });
	1655	+ });
	1656	+ post(web, `${KS}/repos/:owner/:repo/lift`, async (c) => {
	1657	+ const me = requireSiteAdmin(c);
	1658	+ return act(c, KS, async () => {
	1659	+ const target = await repoTarget(c, prm(c, "owner"), prm(c, "repo"));
	1660	+ await changeSuspension(c.env, auditFn(c), me, target, null, "web");
	1661	+ flash(c, "notice", `Actions are no longer suspended for ${target.name}.`);
	1662	+ });
	1663	+ });
	1664	+
1591	1665	web.get("/settings/audit", async (c) => {
1592	1666	const me = requireSiteAdmin(c);
1593	1667	const rows = await c.get("dir").auditLog(200);

		@@ -11,6 +11,7 @@
11	11	import { ICONS } from "./assets.js";
12	12	import { KCheckbox, KSelect } from "./kumo.js";
13	13	import { csrfToken } from "./csrf.js";
	14	+import { ActionsBlockedBanner } from "./kill-switches.js";
14	15	import { Actor, Form, Markdown, Pager, Status, Time, act, duration, encPath, fmtBytes, pageNum, post, qs, refNames, render, repoUrl, shortSha, type WebApp, prm } from "./ui.js";
15	16
16	17	type Loaded = Awaited<ReturnType<typeof loadRepo>>;
		@@ -221,6 +222,7 @@
221	222	</section>
222	223	</aside>
223	224	<div>
	225	+ <ActionsBlockedBanner env={c.env} repoId={repo.id} />
224	226	<div class="row between mb">
225	227	<h1 class="m0">{selected ? selected.name : "All workflows"}</h1>
226	228	{selected ? <span class="mono muted small">{selected.path}</span> : null}
		@@ -362,6 +364,7 @@
362	364	c,
363	365	{ title: `${run.title \|\| run.workflowName} · Run #${run.number}`, repo: { repo, caller, tab: "actions" }, live: run.status !== "completed" ? `run:${run.id}` : undefined },
364	366	<div data-fragment="run">
	367	+ <ActionsBlockedBanner env={c.env} repoId={repo.id} />
365	368	<div class="row between mb" style="align-items:flex-start">
366	369	<div class="grow">
367	370	<div class="muted small">

		@@ -20,6 +20,8 @@
20	20	// images) that only readers can follow; emptied organizations can be deleted.
21	21	// Agents land changes on protected branches only through auto-merge (or the
22	22	// merge queue) where a person allowed it and a required check gates it, never by bypass.
	23	+// Site administrators' kill switches stop Actions and sandboxes for a repository, an
	24	+// account or (hosted compute) the whole instance.
23	25	import { spawn, spawnSync, execFileSync } from "node:child_process";
24	26	import { createHash, generateKeyPairSync, randomBytes } from "node:crypto";
25	27	import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
		@@ -2414,3 +2416,108 @@
2414	2416	async function setRitaRepos(n) {
2415	2417	await ok("PATCH", "/admin/accounts/rita/quotas", { repositories: n }, T.sully);
2416	2418	}
	2419	+
	2420	+test("security: kill switches suspend Actions for a repository or an account, and pause hosted compute on the instance", async () => {
	2421	+ const R = `/repos/${OWNER}/halted`;
	2422	+ const S = `/admin/repos/${OWNER}/halted/actions-suspension`;
	2423	+ const A = `/admin/accounts/${OWNER}/actions-suspension`;
	2424	+ const P = "/admin/hosted-compute-pause";
	2425	+ await repoWithReadme("halted");
	2426	+ const runs = async () => (await ok("GET", `${R}/actions/runs`)).workflow_runs;
	2427	+ const html = async (path, as = T.olive) => (await fetch(`${base}${path}`, { headers: { authorization: `Bearer ${as}` } })).text();
	2428	+ const push = async (path, text) => ok("PUT", `${R}/contents/${path}`, { message: path, content: b64(text) });
	2429	+ // A self-hosted job that no runner takes yet: the run stays in progress.
	2430	+ await push(".github/workflows/ci.yml", "name: ci\non:\n push:\n workflow_dispatch:\n repository_dispatch:\njobs:\n build:\n runs-on: [self-hosted, linux]\n steps:\n - run: echo hi\n");
	2431	+ const first = await until("a run waiting for a runner", async () => {
	2432	+ const [run] = await runs();
	2433	+ return run && (await ok("GET", `${R}/actions/runs/${run.id}/jobs`)).jobs.length ? run : null;
	2434	+ });
	2435	+
	2436	+ try {
	2437	+ // ------------------------------------------------------------ only site administrators
	2438	+ for (const path of [S, A, P]) {
	2439	+ for (const method of ["GET", "PUT", "DELETE"]) assert.match((await status(403, method, path, method === "PUT" ? { reason: "mine" } : undefined, T.olive)).message, /only site administrators/);
	2440	+ await status(401, "PUT", path, { reason: "no token" }, null);
	2441	+ }
	2442	+ await status(422, "PUT", S, {}, T.sully);
	2443	+ await status(404, "PUT", `/admin/repos/${OWNER}/nowhere/actions-suspension`, { reason: "x" }, T.sully);
	2444	+ await status(404, "PUT", "/admin/accounts/olive-bot/actions-suspension", { reason: "x" }, T.sully);
	2445	+ assert.equal((await ok("GET", S, undefined, T.sully)).suspended, false);

		@@ -1212,3 +1212,37 @@
1212	1212	assert.equal((await api("DELETE", "/diffinery/agents/term-bot")).status, 204);
1213	1213	await until("the second terminal to end", async () => /no longer have write access\|ended/i.test((await page(`terminals/${secondId}`)).text) && !(await page(`terminals/${secondId}`)).text.includes("data-socket"), 150_000);
1214	1214	});
	1215	+
	1216	+test("a kill switch ends web terminals and refuses new ones until it's lifted", async () => {
	1217	+ const owner = (await ok("GET", "/user")).login;
	1218	+ assert.equal((await asPerson("POST", "/diffinery/agents", { login: "term-bot-2", name: "Term Bot 2" })).status, 201);
	1219	+ const page = (path) => fetch(`${base}/${owner}/term-demo/${path}`).then(async (r) => ({ status: r.status, text: await r.text() }));
	1220	+ async function form(path, fields) {
	1221	+ const res = await fetch(`${base}/${owner}/term-demo/terminals`);
	1222	+ const csrf = /name="_csrf" value="([^"]+)"/.exec(await res.text())?.[1];
	1223	+ const cookie = /dfy_csrf=([^;]+)/.exec(res.headers.get("set-cookie") ?? "")?.[1];
	1224	+ return fetch(`${base}/${owner}/term-demo/${path}`, { method: "POST", redirect: "manual", headers: { cookie: `dfy_csrf=${cookie}`, "sec-fetch-site": "same-origin" }, body: new URLSearchParams({ _csrf: csrf ?? "", ...fields }) });
	1225	+ }
	1226	+ const opened = await form("terminals", { agent: "term-bot-2", ref: "main" });
	1227	+ assert.equal(opened.status, 303, await opened.text());
	1228	+ const id = new URL(opened.headers.get("location"), base).pathname.split("/").pop();
	1229	+ await until("the terminal to start", async () => (await page(`terminals/${id}`)).text.includes("data-socket"));
	1230	+ const url = `${base.replace("http", "ws")}/${owner}/term-demo/terminals/${id}/socket`;
	1231	+ const origin = new URL(base).origin;
	1232	+
	1233	+ // Suspending the repository's Actions ends the terminal, refuses attaching and opening another.
	1234	+ const S = `/admin/repos/${owner}/term-demo/actions-suspension`;
	1235	+ assert.equal((await api("PUT", S, { reason: "abuse report" })).status, 200);
	1236	+ const refused = await termSocket(url, { origin });
	1237	+ assert.ok([403, 409].includes(refused.status), JSON.stringify(refused));
	1238	+ const again = await form("terminals", { agent: "term-bot-2", ref: "main" });
	1239	+ assert.equal(again.status, 422);
	1240	+ assert.match(await again.text(), /Actions are suspended for this repository by a site administrator: abuse report/);
	1241	+ await until("the terminal to end", async () => /suspended Actions and sandboxes here\|Its sandbox stopped/.test((await page(`terminals/${id}`)).text), 150_000);
	1242	+
	1243	+ // Lifted, a terminal opens again.
	1244	+ assert.equal((await api("DELETE", S)).status, 200);
	1245	+ const after = await form("terminals", { agent: "term-bot-2", ref: "main" });
	1246	+ assert.equal(after.status, 303, await after.text());
	1247	+ assert.equal((await form(`${new URL(after.headers.get("location"), base).pathname.split("/").slice(3).join("/")}/end`, {})).status, 303);
	1248	+});

		@@ -1,0 +1,173 @@
	1	+/** @jsxImportSource hono/jsx */
	2	+import type { FC } from "hono/jsx";
	3	+import type { Env } from "../env.js";
	4	+import { computePause, killSwitches, suspendedList, type Suspension } from "../lib/killswitch.js";
	5	+import { Form, Time } from "./ui.js";
	6	+
	7	+/**
	8	+ * Kill switches in the web UI (lib/killswitch.ts): the banners people see while one is on, and
	9	+ * the Kill switches page of site administration (its routes are in account.tsx).
	10	+ */
	11	+
	12	+const By: FC<{ s: Suspension }> = ({ s }) => (
	13	+ <>
	14	+ {s.by ? <> by {s.by}</> : null} <Time at={s.at} />
	15	+ </>
	16	+);
	17	+
	18	+/** On the admin pages: hosted compute is paused, so nobody forgets it is. */
	19	+export const ComputePausedBanner: FC<{ env: Env }> = async ({ env }) => {
	20	+ const p = await computePause(env).catch(() => null);
	21	+ if (!p) return <></>;
	22	+ return (
	23	+ <div class="flash error" role="status" data-kill="paused">
	24	+ <b>Hosted compute is paused on this instance.</b> Hosted Actions jobs fail without starting and no agent sandbox can start; self-hosted runners still work. Paused
	25	+ <By s={p} />
	26	+ {p.reason ? <>: {p.reason}</> : null}. <a href="/settings/admin/kill-switches">Kill switches</a>
	27	+ </div>
	28	+ );
	29	+};
	30	+
	31	+/** On a repository's Actions pages and settings: its Actions are suspended, or hosted compute is paused. */
	32	+export const ActionsBlockedBanner: FC<{ env: Env; repoId: string }> = async ({ env, repoId }) => {
	33	+ const k = await killSwitches(env, repoId).catch(() => null);
	34	+ if (!k) return <></>;
	35	+ const s = k.repo ?? k.account;
	36	+ if (s)
	37	+ return (
	38	+ <div class="flash error" role="status" data-kill="suspended">
	39	+ <b>Actions are suspended {k.repo ? "for this repository" : <>for every repository of {k.account!.login}</>} by a site administrator</b>
	40	+ {s.reason ? <>: {s.reason}</> : null}. No workflow runs start, queued jobs fail, and agent sandboxes can't start until a site administrator lifts the suspension. Suspended
	41	+ <By s={s} />.
	42	+ </div>
	43	+ );
	44	+ if (k.paused)
	45	+ return (
	46	+ <div class="flash error" role="status" data-kill="paused">
	47	+ <b>Hosted runners are paused on this instance by a site administrator</b>
	48	+ {k.paused.reason ? <>: {k.paused.reason}</> : null}. Jobs on hosted runners fail without starting and agent sandboxes can't start; jobs on self-hosted runners still run.
	49	+ </div>
	50	+ );
	51	+ return <></>;
	52	+};
	53	+
	54	+/** The body of Settings → Kill switches. */
	55	+export const KillSwitchesPage: FC<{ env: Env; t: string }> = async ({ env, t }) => {
	56	+ const [paused, list] = await Promise.all([computePause(env), suspendedList(env)]);
	57	+ return (
	58	+ <>
	59	+ <h1>Kill switches</h1>
	60	+ <p class="muted">
	61	+ Stop Actions and agent sandboxes quickly: for one repository, for every repository of an account, or hosted compute on the whole instance. Turning one on stops what is running at once; turning it off only lets new work start. Nobody is exempt, site administrators included. Every change is in the{" "}
	62	+ <a href="/settings/audit">audit log</a>.
	63	+ </p>
	64	+ <div class="box">
	65	+ <div class="head">
	66	+ <h2>Hosted compute</h2>
	67	+ {paused ? <span class="badge red">paused</span> : <span class="badge green">running</span>}
	68	+ </div>
	69	+ <div class="body">
	70	+ {paused ? (
	71	+ <>
	72	+ <p class="m0">
	73	+ Paused
	74	+ <By s={paused} />
	75	+ {paused.reason ? <>: {paused.reason}</> : null}.
	76	+ </p>
	77	+ <Form action="/settings/admin/kill-switches/resume" t={t} class="mt">
	78	+ <button class="btn primary">Resume hosted compute</button>
	79	+ </Form>
	80	+ </>
	81	+ ) : (
	82	+ <Form action="/settings/admin/kill-switches/pause" t={t} confirm="Pause hosted compute on the whole instance? Running hosted jobs and open sandboxes stop now.">
	83	+ <div class="formrow">
	84	+ <div class="field grow">
	85	+ <label for="pause-reason">Reason</label>
	86	+ <input id="pause-reason" name="reason" required maxlength={500} placeholder="e.g. abuse investigation, cost overrun" />
	87	+ </div>
	88	+ <button class="btn danger">Pause hosted compute</button>
	89	+ </div>
	90	+ <div class="hint">Hosted Actions jobs fail without starting and no agent sandbox can start, in every repository. Self-hosted runners keep working.</div>
	91	+ </Form>
	92	+ )}
	93	+ </div>
	94	+ </div>
	95	+ <Form action="/settings/admin/kill-switches/accounts" t={t} class="box" confirm="Suspend Actions for every repository of this account? Its runs are cancelled and its sandboxes closed now.">
	96	+ <div class="head">
	97	+ <h2>Suspend Actions for an account</h2>
	98	+ </div>
	99	+ <div class="body">
	100	+ <div class="formrow">
	101	+ <div class="field">
	102	+ <label for="ks-login">Person or organization</label>
	103	+ <input id="ks-login" name="login" required placeholder="login" autocomplete="off" />
	104	+ </div>
	105	+ <div class="field grow">
	106	+ <label for="ks-areason">Reason</label>
	107	+ <input id="ks-areason" name="reason" required maxlength={500} />
	108	+ </div>
	109	+ <button class="btn danger">Suspend</button>
	110	+ </div>
	111	+ <div class="hint">Every repository the account owns: no workflow runs (hosted or self-hosted) and no agent sandboxes.</div>
	112	+ </div>
	113	+ </Form>
	114	+ <Form action="/settings/admin/kill-switches/repos" t={t} class="box" confirm="Suspend Actions for this repository? Its runs are cancelled and its sandboxes closed now.">
	115	+ <div class="head">
	116	+ <h2>Suspend Actions for a repository</h2>
	117	+ </div>
	118	+ <div class="body">
	119	+ <div class="formrow">
	120	+ <div class="field">
	121	+ <label for="ks-repo">Repository</label>
	122	+ <input id="ks-repo" name="repo" required placeholder="owner/name" autocomplete="off" />
	123	+ </div>
	124	+ <div class="field grow">
	125	+ <label for="ks-rreason">Reason</label>
	126	+ <input id="ks-rreason" name="reason" required maxlength={500} />
	127	+ </div>
	128	+ <button class="btn danger">Suspend</button>
	129	+ </div>
	130	+ </div>
	131	+ </Form>
	132	+ <div class="box">
	133	+ <div class="head">
	134	+ <h2>Suspended</h2>
	135	+ <span class="badge">{list.accounts.length + list.repos.length}</span>
	136	+ </div>
	137	+ {list.accounts.length + list.repos.length ? (
	138	+ <ul class="list">
	139	+ {list.accounts.map((a) => (
	140	+ <li>
	141	+ <div class="grow">
	142	+ <b>{a.login}</b> <span class="badge">{a.kind === "org" ? "organization" : "person"}</span> <span class="muted small">every repository · suspended<By s={a} /></span>
	143	+ {a.reason ? <div class="meta">{a.reason}</div> : null}
	144	+ </div>
	145	+ <Form action={`/settings/admin/kill-switches/accounts/${encodeURIComponent(a.login)}/lift`} t={t} class="inline">
	146	+ <button class="btn sm">Lift</button>
	147	+ </Form>
	148	+ </li>
	149	+ ))}
	150	+ {list.repos.map((r) => (
	151	+ <li>
	152	+ <div class="grow">
	153	+ <a href={`/${r.owner}/${r.name}/actions`}>
	154	+ <b>
	155	+ {r.owner}/{r.name}
	156	+ </b>
	157	+ </a>{" "}
	158	+ <span class="badge">repository</span> <span class="muted small">suspended<By s={r} /></span>
	159	+ {r.reason ? <div class="meta">{r.reason}</div> : null}
	160	+ </div>
	161	+ <Form action={`/settings/admin/kill-switches/repos/${encodeURIComponent(r.owner)}/${encodeURIComponent(r.name)}/lift`} t={t} class="inline">
	162	+ <button class="btn sm">Lift</button>
	163	+ </Form>
	164	+ </li>
	165	+ ))}
	166	+ </ul>
	167	+ ) : (
	168	+ <div class="empty">No account or repository has its Actions suspended.</div>
	169	+ )}
	170	+ </div>
	171	+ </>
	172	+ );
	173	+};

		@@ -10,6 +10,7 @@
10	10	import { ICONS } from "./assets.js";
11	11	import { KCheckbox, KRadio, KSelect, type KOption } from "./kumo.js";
12	12	import { csrfToken, type FormData } from "./csrf.js";
	13	+import { ActionsBlockedBanner } from "./kill-switches.js";
13	14	import { Actor, CopyButton, Form, PendingInvites, ROLES, Time, cap, act, ago, flash, post, refNames, render, repoUrl, statusOf, type WebApp, prm } from "./ui.js";
14	15
15	16	type Loaded = Awaited<ReturnType<typeof loadRepo>>;
		@@ -59,7 +60,10 @@
59	60	</div>
60	61	</section>
61	62	</aside>
62		- <div>{body}</div>
	63	+ <div>
	64	+ <ActionsBlockedBanner env={c.env} repoId={l.repo.id} />
	65	+ {body}
	66	+ </div>
63	67	</div>,
64	68	);
65	69	}

		@@ -1,0 +1,129 @@
	1	+import { describe, expect, it } from "vitest";
	2	+import { blockedBy, changePause, changeSuspension, cleanReason, computePause, killSwitches, runningCompute, suspendedList, suspensionOf } from "../../src/lib/killswitch.js";
	3	+import type { Env } from "../../src/env.js";
	4	+import { testD1 } from "./stubs/d1.js";
	5	+
	6	+const at = "2026-10-01T00:00:00.000Z";
	7	+
	8	+/** A D1 with accounts ann (a person, site administrator) and acme (an organization), and repositories r1, r2 (acme's) and r3 (ann's); REPO answers stopActions. */
	9	+function setup() {
	10	+ const DB = testD1();
	11	+ const add = (id: string, kind: string, siteAdmin = 0) => DB.sqlite.prepare("INSERT INTO accounts (id, login, kind, status, site_admin, created_at) VALUES (?, ?, ?, 'active', ?, ?)").run(id, id, kind, siteAdmin, at);
	12	+ add("ann", "user", 1);
	13	+ add("acme", "org");
	14	+ const repo = (id: string, owner: string) => DB.sqlite.prepare("INSERT INTO repos (id, owner_id, name, description, visibility, default_branch, created_at, updated_at) VALUES (?, ?, ?, '', 'private', 'main', ?, ?)").run(id, owner, id, at, at);
	15	+ repo("r1", "acme");
	16	+ repo("r2", "acme");
	17	+ repo("r3", "ann");
	18	+ const stops: Array<{ repo: string; hostedOnly: boolean; message: string }> = [];
	19	+ const REPO = {
	20	+ idFromName: (n: string) => n,
	21	+ get: (id: string) => ({
	22	+ stopActions: async (o: { hostedOnly: boolean; message: string }) => {
	23	+ stops.push({ repo: id, ...o });
	24	+ if (id === "broken") throw new Error("gone");
	25	+ return { runs: 1, jobs: 2, sandboxes: 1 };
	26	+ },
	27	+ }),
	28	+ };
	29	+ const audits: Array<{ action: string; target?: string; detail?: unknown }> = [];
	30	+ const audit = async (e: { action: string; target?: string; detail?: unknown }) => void audits.push(e);
	31	+ return { env: { DB, REPO } as unknown as Env, DB, stops, audits, audit };
	32	+}
	33	+
	34	+const ann = { id: "ann", login: "ann" };
	35	+
	36	+describe("kill switches", () => {
	37	+ it("block nothing until one is on", async () => {
	38	+ const { env } = setup();
	39	+ const k = await killSwitches(env, "r1");
	40	+ expect(k).toEqual({ repo: null, account: null, paused: null });
	41	+ for (const w of ["run", "hosted-job", "self-hosted-job", "sandbox"] as const) expect(blockedBy(k, w)).toBeNull();
	42	+ });
	43	+
	44	+ it("suspending a repository stops all its work, and lifting it only allows new work", async () => {
	45	+ const { env, stops, audits, audit } = setup();
	46	+ const stopped = await changeSuspension(env, audit, ann, { kind: "repo", id: "r1", name: "acme/r1" }, "crypto mining", "api");
	47	+ expect(stopped).toEqual({ repositories: 1, runs: 1, jobs: 2, sandboxes: 1, failed: 0 });
	48	+ expect(stops).toEqual([{ repo: "r1", hostedOnly: false, message: "Actions are suspended for this repository by a site administrator: crypto mining" }]);
	49	+ const k = await killSwitches(env, "r1");
	50	+ expect(k.repo).toMatchObject({ by: "ann", reason: "crypto mining" });
	51	+ for (const w of ["run", "hosted-job", "self-hosted-job", "sandbox"] as const) expect(blockedBy(k, w)).toBe("Actions are suspended for this repository by a site administrator: crypto mining");
	52	+ // Its neighbour isn't.
	53	+ expect(blockedBy(await killSwitches(env, "r2"), "run")).toBeNull();
	54	+ expect(audits).toEqual([{ actorId: "ann", actorLogin: "ann", action: "actions.suspend", target: "acme/r1", detail: { kind: "repo", reason: "crypto mining", via: "api", stopped } }]);
	55	+
	56	+ expect(await changeSuspension(env, audit, ann, { kind: "repo", id: "r1", name: "acme/r1" }, null, "web")).toBeNull();
	57	+ expect(blockedBy(await killSwitches(env, "r1"), "run")).toBeNull();
	58	+ expect(stops).toHaveLength(1);
	59	+ expect(audits[1]).toEqual({ actorId: "ann", actorLogin: "ann", action: "actions.unsuspend", target: "acme/r1", detail: { kind: "repo", via: "web" } });
	60	+ // Lifting what isn't suspended changes nothing and isn't recorded.
	61	+ await changeSuspension(env, audit, ann, { kind: "repo", id: "r1", name: "acme/r1" }, null, "web");
	62	+ expect(audits).toHaveLength(2);
	63	+ });
	64	+
	65	+ it("suspending an account covers every repository it owns, even a site administrator's", async () => {
	66	+ const { env, stops, audit } = setup();
	67	+ await changeSuspension(env, audit, ann, { kind: "account", id: "acme", login: "acme" }, "abuse", "api");
	68	+ expect(stops.map((s) => s.repo).sort()).toEqual(["r1", "r2"]);
	69	+ expect(stops[0]!.message).toBe("Actions are suspended for every repository of acme by a site administrator: abuse");
	70	+ for (const r of ["r1", "r2"]) expect(blockedBy(await killSwitches(env, r), "self-hosted-job")).toMatch(/every repository of acme/);
	71	+ expect(blockedBy(await killSwitches(env, "r3"), "run")).toBeNull();
	72	+ // Nobody is exempt: ann is a site administrator.
	73	+ await changeSuspension(env, audit, ann, { kind: "account", id: "ann", login: "ann" }, "testing", "web");
	74	+ expect(blockedBy(await killSwitches(env, "r3"), "sandbox")).toMatch(/every repository of ann/);
	75	+ expect(await suspensionOf(env, { accountId: "acme" })).toMatchObject({ by: "ann", reason: "abuse" });
	76	+ const list = await suspendedList(env);
	77	+ expect(list.accounts.map((a) => a.login).sort()).toEqual(["acme", "ann"]);
	78	+ expect(list.repos).toEqual([]);
	79	+ });
	80	+
	81	+ it("pausing hosted compute stops hosted jobs and sandboxes only, in the repositories running some", async () => {
	82	+ const { env, DB, stops, audits, audit } = setup();
	83	+ const usage = (id: string, repo: string, kind: "job" \| "sandbox", start: string, end: string \| null = null) =>
	84	+ DB.sqlite.prepare("INSERT INTO compute_usage (id, account_id, repo_id, kind, started_at, ended_at) VALUES (?, 'acme', ?, ?, ?, ?)").run(id, repo, kind, start, end);
	85	+ const recent = new Date(Date.now() - 60_000).toISOString();
	86	+ usage("j1", "r1", "job", recent);
	87	+ usage("s1", "r2", "sandbox", recent, recent);
	88	+ usage("s1:1", "r2", "sandbox", recent); // woke up after sleeping
	89	+ usage("j2", "r3", "job", recent, recent); // ended
	90	+ usage("j3", "r3", "job", new Date(Date.now() - 8 * 3600_000).toISOString()); // lost its end long ago
	91	+ const running = await runningCompute(env);
	92	+ expect([...running.entries()].sort()).toEqual([
	93	+ ["r1", { jobs: ["j1"], sandboxes: [] }],
	94	+ ["r2", { jobs: [], sandboxes: ["s1"] }],
	95	+ ]);
	96	+
	97	+ const stopped = await changePause(env, audit, ann, "cost overrun", "api");
	98	+ expect(stopped).toMatchObject({ repositories: 2, failed: 0 });
	99	+ expect(stops.map((s) => [s.repo, s.hostedOnly]).sort()).toEqual([
	100	+ ["r1", true],
	101	+ ["r2", true],
	102	+ ]);
	103	+ expect(stops[0]!.message).toBe("Hosted runners are paused on this instance by a site administrator: cost overrun. Self-hosted runners still work");
	104	+ const k = await killSwitches(env, "r3");
	105	+ expect(blockedBy(k, "hosted-job")).toMatch(/^Hosted runners are paused on this instance by a site administrator: cost overrun/);
	106	+ expect(blockedBy(k, "sandbox")).toBe("Agent sandboxes are paused on this instance by a site administrator: cost overrun");
	107	+ // Self-hosted runners still work, and runs still start (for them).
	108	+ expect(blockedBy(k, "self-hosted-job")).toBeNull();
	109	+ expect(blockedBy(k, "run")).toBeNull();
	110	+ expect(await computePause(env)).toMatchObject({ by: "ann", reason: "cost overrun" });
	111	+ expect(audits[0]).toMatchObject({ action: "compute.pause", detail: { reason: "cost overrun", via: "api" } });
	112	+
	113	+ expect(await changePause(env, audit, ann, null, "web")).toBeNull();
	114	+ expect(await computePause(env)).toBeNull();
	115	+ expect(blockedBy(await killSwitches(env, "r3"), "hosted-job")).toBeNull();
	116	+ expect(audits[1]).toMatchObject({ action: "compute.unpause", detail: { via: "web" } });
	117	+ expect(stops).toHaveLength(2);
	118	+ });
	119	+
	120	+ it("counts repositories that couldn't be stopped, and needs a reason", async () => {
	121	+ const { env, DB, audit } = setup();
	122	+ DB.sqlite.prepare("INSERT INTO compute_usage (id, account_id, repo_id, kind, started_at) VALUES ('j', 'acme', 'broken', 'job', ?)").run(new Date().toISOString());
	123	+ expect(await changePause(env, audit, ann, "x", "api")).toMatchObject({ repositories: 0, failed: 1 });
	124	+ expect(() => cleanReason(" ")).toThrow(/give a reason/);
	125	+ expect(() => cleanReason(undefined)).toThrow(/give a reason/);
	126	+ expect(() => cleanReason("x".repeat(501))).toThrow(/at most 500/);
	127	+ expect(cleanReason(" too many\nspaces ")).toBe("too many spaces");
	128	+ });
	129	+});

codician/diffinerypublic

Kill switches: suspend Actions for a repository or an account, pause hosted compute (#131)