Diffinery

codician/diffinerypublic

Open-source, agent-first GitHub and GitHub Actions alternative that runs on your own Cloudflare account: git hosting, pull requests, code review and drop-in GitHub Actions YAML on Workers, Durable Objects, R2 and Containers.

Agent tasks: run Claude Code, Codex or your own command with your own model key (#60)

* Agent tasks: run Claude Code, Codex or your own command with your own model key

A person gives one of their agents a goal on the repository's Tasks tab.
Diffinery starts a sandbox as the agent, installs the runtime, runs it
headless with the person's own model access, streams the transcript, then
pushes agent/<n>-<slug> and opens a pull request as the agent. Follow-ups,
retry and stop; tasks show on Today.

Settings -> Model access stores Anthropic and OpenAI keys per person,
encrypted and write-only, behind an opt-in to the provider's terms. The key
never enters a sandbox: each run gets a task-scoped token that only the model
proxy (/-/model/<provider>/v1/*) accepts; the proxy re-checks the person on
every request and adds the key server-side. The token ends with the run.

Starting, steering and retrying need the person's browser session; task
sandboxes are closed to the public sandbox API. wrangler dev binds SANDBOX to
SandboxPoolDO, backed by agent.mjs --sandbox-pool, so the e2e tests run tasks
end to end.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

* Model proxy: only the model endpoints; pin runtime versions; detect the push by ref

From the security review of #60. The proxy refuses every path but the model
endpoints the runtimes use (403 naming the path), so an agent can't reach a
person's stored files, batches or assistants with their key. Claude Code and
Codex install at pinned versions. Whether a run pushed comes from the
branch ref, not from output the agent could write to.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014VDQqT8oL18uLJ9fpsNxW2

---------

Co-authored-by: Claude <noreply@anthropic.com>
Canberk Demirci authored · GitHub committed signed1 parent b2caad3commit 50797a8d8e5812145460333fcfd46552031dbcbfBrowse files
43 files changed+3196−22
Files (43)
modifiedapps/diffinery/docs/agents/mcp.md+2−0View file
@@ -131,6 +131,8 @@
131131 | --- | --- | --- | --- |
132132 | `whoami` | Yes | The account the token acts as (`login`, `kind`) and, for an agent, `on_behalf_of` | Any token |
133133 | `today` | Yes | Your morning view: pull requests that need you, your agents' open pull requests and where each stands, and what merged, was released or failed in the last 24 hours. See [Today](/docs/agents/today) | Any token |
134+| `agent_tasks` | Yes | Agent tasks: with `owner` and `repo` that repository's, without them the ones you started (an agent: the ones it runs). See [Agent tasks](/docs/agents/tasks) | Write on the repository; any token for your own |
135+| `agent_task` | Yes | One agent task (`owner`, `repo`, `number`) with its runs, pull request and transcript after `after` | Write |
134136 | `list_repos` | Yes | Repositories visible to you, optionally filtered by `owner` | Any token |
135137 | `get_organization` | Yes | An organization's `login` and `name`; to its members, its agents, agents of its members and site administrators also `default_repository_permission` (the base permission members get on its repositories) | Any token |
136138 | `list_invitations` | Yes | Pending invitations to a repository (`owner` and `repo`), an organization (`org`) or the instance (neither), each with an `id` | Admin on the repository; organization owner or admin; site administrator |
modifiedapps/diffinery/docs/agents/sandboxes.md+1−0View file
@@ -12,6 +12,7 @@
1212 - **The hosted runner image.** Sandboxes run the same Ubuntu 24.04 image as hosted workflow jobs, so the same toolchains are there. See [What's inside](#whats-inside).
1313 - **An HTTP API, not a shell session.** Each command runs on its own in `bash -c`, so `cd` and `export` don't carry over to the next command; use `cwd` and `env` instead. Files stay between commands.
1414 - **Tied to its creator.** Only the account that created a sandbox (or a repository admin) can use it, and git inside it acts as that account.
15+- **Not the sandboxes of [agent tasks](/docs/agents/tasks).** Those run a person's model access, so no sandbox call reaches them, for anyone.
1516
1617 ## Lifecycle
1718
addedapps/diffinery/docs/agents/tasks.md+179−0View file
@@ -1,0 +1,179 @@
1+---
2+title: Agent tasks
3+description: Give one of your agents a goal; it runs Claude Code, Codex or your own command in a sandbox with your own model key, then pushes a branch and opens a pull request for review.
4+---
5+
6+An **agent task** is a goal you give one of your agents, such as "Fix the Contact sales link on the pricing page and add a test". Diffinery starts a sandbox as that agent with the repository checked out, runs a coding agent (Claude Code, Codex or a command of your own) with **your own model API key**, streams its transcript to the task page, then pushes the work to a branch and opens a pull request as the agent. The pull request needs a person's approval like any agent pull request.
7+
8+Tasks are optional. Nothing changes for a repository until someone with a model key starts one.
9+
10+## Before you start
11+
12+You need three things:
13+
14+1. **Write access** to the repository.
15+2. **An agent** you manage: one you own, or one of an organization where you are an owner or admin. The agent needs write access to the repository too (agents get at most their owner's access). Create one under **Settings → Agents**; see [Agents](/docs/agents/overview).
16+3. **Your own model key** under **Settings → Model access**: an Anthropic API key for Claude Code, an OpenAI API key for Codex. See [Model access](#model-access).
17+
18+## Start a task
19+
20+1. Open the repository's **Tasks** tab (people with write access see it) and click **New task**.
21+2. Write the **goal**, as you would ask a colleague. Markdown works.
22+3. Pick the **runtime**: Claude Code, Codex or **Custom command** (see [Runtimes](#runtimes)).
23+4. Pick the **agent** and the **base branch**.
24+5. Click **Start task**.
25+
26+The task page shows the transcript live: what the agent says, the commands it runs and their output.
27+
28+## What happens in a run
29+
30+| Step | What Diffinery does |
31+| --- | --- |
32+| Sandbox | Re-checks that you may run the task, then starts a fresh sandbox **as the agent**, on behalf of you, with the base branch (or the task's branch, for later runs) checked out. Its git credential is the agent's, at most write, and stops working when the run ends. |
33+| Install | Installs the runtime from its vendor's official npm package, at a version Diffinery pins (see [Runtimes](#runtimes)). No model credential exists in the sandbox yet. |
34+| Agent | Runs the runtime headless with the goal. Its process, and only its process, gets a model token for this run (see [How your key is protected](#how-your-key-is-protected)). |
35+| Push | Commits what the agent left uncommitted, as the agent, and pushes `agent/<number>-<slug>`. Branch protection, push protection and the agent's guardrails apply as to any push. |
36+| Pull request | Opens a pull request from that branch as the agent, with the goal as its `intent`, or updates the one the task already has. |
37+| Close | Closes the sandbox. The run's git credential and model token stop working. |
38+
39+A run stops after 6 hours. Tasks show on [Today](/docs/agents/today) under **Your agent tasks** while they run and for a day after they change.
40+
41+### Task states
42+
43+| State | Meaning |
44+| --- | --- |
45+| **Running** | A run is in progress. |
46+| **Ready** | The last run finished. Its work is on the task's pull request, if it changed anything. |
47+| **Failed** | The last run failed: the runtime exited with an error, the push was refused, or you lost access. The transcript says why. |
48+| **Stopped** | Someone stopped it. |
49+
50+### Steer, retry and stop
51+
52+- **Send a follow-up** from the task page to continue: a new run starts on the task's branch with the goal, how the previous run ended and your message, and its work goes to the same pull request. Sent while a run is active, the message waits and starts when that run ends.
53+- **Retry** runs the last run again when the task isn't running.
54+- **Stop** stops the agent and closes its sandbox. Work already pushed stays.
55+
56+Only the person who started a task can send it messages or retry it, because each run spends their model key. That person, the task's agent and repository admins can stop it.
57+
58+## Runtimes
59+
60+| Runtime | Needs | Runs |
61+| --- | --- | --- |
62+| **Claude Code** | Your Anthropic API key | `claude -p "$DIFFINERY_TASK_PROMPT" --output-format stream-json --verbose --permission-mode bypassPermissions` |
63+| **Codex** | Your OpenAI API key | `codex exec --json --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check "$DIFFINERY_TASK_PROMPT"` |
64+| **Custom command** | Nothing required | Your command, with the goal in `$DIFFINERY_TASK_PROMPT`, and `ANTHROPIC_BASE_URL` / `ANTHROPIC_API_KEY` and `OPENAI_BASE_URL` / `OPENAI_API_KEY` set for each provider you have a key for |
65+
66+The runtimes run unmodified, from the vendors' own packages, at pinned versions: Claude Code 2.1.296 (`@anthropic-ai/claude-code`) and Codex 0.162.1 (`@openai/codex`). Diffinery moves to newer versions deliberately rather than installing whatever was published last. The sandbox is its own container, so they run without permission prompts. The goal reaches the runtime in an environment variable, never pasted into a command line.
67+
68+The prompt tells the agent which task it is working on, that Diffinery pushes and opens the pull request, and not to push or change git credentials itself.
69+
70+## Model access
71+
72+**Settings → Model access** holds your own API keys, one per provider:
73+
74+- **Anthropic**, for Claude Code. Create a key at console.anthropic.com.
75+- **OpenAI**, for Codex. Create a key at platform.openai.com.
76+
77+Saving a key asks you to confirm that you use that provider through Diffinery **under your own account and the provider's terms**, with links to them (Anthropic's [Commercial Terms](https://www.anthropic.com/legal/commercial-terms) and [Usage Policy](https://www.anthropic.com/legal/aup); OpenAI's [Terms and Policies](https://openai.com/policies/)). The provider bills your account. Diffinery only provides the ability; whether and how you use it is your choice and your agreement with the provider. Your confirmation is stored with its date.
78+
79+- Keys are encrypted at rest and **write-only**: no page, API call or MCP tool returns one. You see the last four characters.
80+- Only you, signed in in a browser, can save, replace or delete your keys. No token, agent, job or sandbox can.
81+- Saving a key sends you a security email ("Your Anthropic API key was added to Diffinery"), which you can't turn off.
82+- Keys belong to people. Agents never have model keys of their own, and organization keys aren't available yet.
83+- Subscriptions (Claude Pro or Max, ChatGPT plans) can't be used for tasks: providers allow automated use with API keys.
84+
85+## How your key is protected
86+
87+Your key never enters a sandbox:
88+
89+1. When a run's agent starts, Diffinery creates a **model token for that run** (`dmt_...`) and gives it to the runtime's process only, as its API key, with the provider's base URL pointed at Diffinery's model proxy (`https://diffinery.com/-/model/<provider>`).
90+2. The proxy accepts the token only while that run's agent is running. On every request it re-checks that you can still run the task, then forwards the request to the provider with your key added.
91+3. The proxy forwards only the model endpoints the runtimes use, and refuses everything else with `403`, so an agent can't reach the rest of your provider account (stored files, batches, assistants):
92+
93+ | Provider | Endpoints |
94+ | --- | --- |
95+ | Anthropic | `POST /v1/messages`, `POST /v1/messages/count_tokens`, `GET /v1/models` |
96+ | OpenAI | `POST /v1/responses`, `POST /v1/responses/compact`, `GET /v1/responses/{id}`, `POST /v1/chat/completions`, `GET /v1/models` |
97+
98+4. When the agent exits, is stopped, or the run ends any other way, the token stops working. If you lose access to the repository or the agent, the task stops.
99+
100+Everything the sandbox returns, including the transcript, masks the model token and the git credential.
101+
102+**What this doesn't protect against.** Anything the agent runs in its sandbox (build scripts, tests, dependencies, code from the repository) can read the model token and the git credential while the run lasts, and repository content can try to trick the agent into misusing them ("prompt injection"). Sandboxes have internet access. So:
103+
104+- Use a **dedicated API key with a spending limit** for Diffinery, and revoke it with the provider if you suspect misuse.
105+- Run tasks on repositories whose content you trust.
106+- A transcript can show a credential in another form (for example encoded) that masking can't recognise. Transcripts are visible only to people with write access.
107+
108+Task sandboxes are closed to the [sandbox API](/docs/agents/sandboxes): no call reaches them, for anyone, including the agent's own tokens and repository admins.
109+
110+## Who sees what
111+
112+- Tasks and transcripts: people with **write** access to the repository (stricter than Actions logs, because transcripts show an agent's reasoning and command output).
113+- The task's pull request: everyone who can read the repository, like any pull request.
114+
115+## REST API
116+
117+| Endpoint | Does | Who |
118+| --- | --- | --- |
119+| `GET /repos/{owner}/{repo}/diffinery/tasks` | List tasks | Write access; any token |
120+| `GET /repos/{owner}/{repo}/diffinery/tasks/{number}` | A task with its runs | Write access; any token |
121+| `GET /repos/{owner}/{repo}/diffinery/tasks/{number}/transcript?after=0` | Transcript lines | Write access; any token |
122+| `POST /repos/{owner}/{repo}/diffinery/tasks` | Start a task | Browser session only |
123+| `POST .../tasks/{number}/messages` | Send a follow-up | Browser session only, the person who started it |
124+| `POST .../tasks/{number}/retry` | Retry | Browser session only, the person who started it |
125+| `POST .../tasks/{number}/stop` | Stop | The person who started it, its agent or a repository admin |
126+| `GET /diffinery/tasks` | Tasks you started, across repositories | Any token |
127+| `GET /diffinery/model-keys` | Your saved keys (provider, last four characters, dates) | Any token |
128+| `PUT` / `DELETE /diffinery/model-keys/{provider}` | Save or delete a key | Browser session only |
129+
130+```sh
131+curl https://diffinery.com/api/v3/repos/acme/website/diffinery/tasks/7 \
132+ -H "Authorization: Bearer dfy_..."
133+```
134+
135+```json
136+{
137+ "number": 7,
138+ "title": "Fix the Contact sales link on the pricing page",
139+ "goal": "Fix the Contact sales link on the pricing page so it goes to /contact, and add a test.",
140+ "state": "ready",
141+ "runtime": "claude-code",
142+ "agent": { "login": "ayse-bot", "type": "Bot" },
143+ "started_by": { "login": "ayse", "type": "User" },
144+ "base": "main",
145+ "branch": "agent/7-fix-the-contact-sales-link-on-the",
146+ "pull_request": { "number": 42, "html_url": "https://diffinery.com/acme/website/pull/42" },
147+ "queued_message": null,
148+ "error": null,
149+ "html_url": "https://diffinery.com/acme/website/tasks/7",
150+ "runs": [{ "number": 1, "kind": "start", "message": null, "state": "done", "exit_code": 0, "error": null, "started_at": "2026-10-14T07:02:11.000Z", "ended_at": "2026-10-14T07:09:40.000Z" }]
151+}
152+```
153+
154+Transcript lines have `seq`, `run`, `at`, `stream` (`out` and `err` for the runtime's output, `setup` for installing and pushing, `note` for Diffinery's own lines) and `text`. Claude Code writes `stream-json` lines and Codex writes `--json` lines to `out`.
155+
156+## MCP
157+
158+Two read-only tools, so your own agent can follow tasks:
159+
160+| Tool | What it does |
161+| --- | --- |
162+| `agent_tasks` | With `owner` and `repo`: that repository's tasks. Without: the tasks you started. |
163+| `agent_task` | One task with its runs and its transcript after `after`. |
164+
165+Starting, steering and retrying aren't available over MCP, because they spend a person's model key.
166+
167+## Troubleshooting
168+
169+| What you see | Cause and fix |
170+| --- | --- |
171+| `Claude Code needs your Anthropic API key` | Save a key under **Settings → Model access**. |
172+| `you can't run tasks as <agent>` | You must own the agent or be an owner or admin of the organization that owns it. |
173+| `<agent> needs write access to this repository` | Give the agent write access, or a role through its organization. Agents get at most their owner's access. |
174+| `starting or steering an agent task needs a browser session` | Use the web page; tokens can't start tasks. |
175+| The run failed at **Installing** | The npm registry couldn't be reached or the package failed to install. Retry. |
176+| `Pushing the work ... failed` | The push was refused, usually by branch protection, push protection (a secret in the changes) or the agent's guardrails. The transcript shows git's message. |
177+| The task stopped with "can no longer run it" | You lost write access to the repository, or can no longer manage the agent. |
178+| Model calls fail with `401 ... its task run has ended` | The run is over; that token no longer works. Start a new run. |
179+| A custom command gets `403 ... isn't available to agent tasks` | The model proxy forwards only the model endpoints listed under [How your key is protected](#how-your-key-is-protected). |
modifiedapps/diffinery/docs/agents/today.md+6−0View file
@@ -48,6 +48,10 @@
4848 - **Failed workflow runs** that you or your agents started, and failed runs on a repository's default branch (pull request runs only when they are yours or your agents').
4949 - **Merged pull requests**, with who opened and who merged them.
5050
51+### Your agent tasks
52+
53+[Agent tasks](/docs/agents/tasks) you started that are running or changed in the last 24 hours, with their state and pull request. The section appears only when you have some.
54+
5155 ## Which repositories it covers
5256
5357 Today looks at the repositories you can see that had activity in the last 30 days, such as a push, a pull request, a review, a workflow run or a release, most recent first, up to 100 of them. Archived repositories are left out. When some active repositories were left out, the page says so next to the repository count, and the API sets `incomplete_results`.
@@ -94,6 +98,7 @@
9498 "agents_at_work": [],
9599 "landed": [],
96100 "releases": [],
101+ "tasks": [],
97102 "failures": []
98103 }
99104 ```
@@ -107,6 +112,7 @@
107112 | `landed[]` | Pull requests merged since `since`, with `merged_by` and `merged_at` |
108113 | `releases[]` | Releases published since `since` |
109114 | `failures[]` | Failed runs since `since`, with `workflow`, `ref`, `event`, `actor` and `pull_request` |
115+| `tasks[]` | Your agent tasks that are running or changed since `since`, with `state`, `agent` and `pull_request` (an agent's token: the tasks it runs) |
110116
111117 Any token works (a `read` token is enough). A token limited to repositories sees only those repositories. When an agent's token calls it, `agents_at_work` lists that agent's own open pull requests, and `needs_you` never contains agent approvals, because an agent's approval doesn't count.
112118
modifiedapps/diffinery/docs/nav.json+1−0View file
@@ -70,6 +70,7 @@
7070 "pages": [
7171 "agents/overview",
7272 "agents/today",
73+ "agents/tasks",
7374 "agents/mcp",
7475 "agents/sandboxes",
7576 "agents/docs-for-agents"
addedapps/diffinery/migrations/0013_agent_tasks.sql+32−0View file
@@ -1,0 +1,32 @@
1+-- People's own model API keys for agent tasks (agentic/keys.ts), encrypted per
2+-- account and provider. Write-only: nothing ever returns `ciphertext` decrypted
3+-- except the model proxy, while a task of that person runs.
4+CREATE TABLE model_keys (
5+ account_id TEXT NOT NULL,
6+ provider TEXT NOT NULL,
7+ ciphertext TEXT NOT NULL,
8+ hint TEXT NOT NULL,
9+ opted_in_at TEXT NOT NULL,
10+ created_at TEXT NOT NULL,
11+ updated_at TEXT NOT NULL,
12+ PRIMARY KEY (account_id, provider)
13+);
14+
15+-- An index of agent tasks across repositories (each repository's ProjectDO holds
16+-- the tasks themselves), for Today and task lists.
17+CREATE TABLE agent_tasks (
18+ id TEXT PRIMARY KEY,
19+ repo_id TEXT NOT NULL,
20+ number INTEGER NOT NULL,
21+ title TEXT NOT NULL,
22+ runtime TEXT NOT NULL,
23+ agent_id TEXT NOT NULL,
24+ started_by TEXT NOT NULL,
25+ state TEXT NOT NULL,
26+ pr_number INTEGER,
27+ created_at TEXT NOT NULL,
28+ updated_at TEXT NOT NULL
29+);
30+CREATE INDEX agent_tasks_person ON agent_tasks(started_by, updated_at);
31+CREATE INDEX agent_tasks_agent ON agent_tasks(agent_id, updated_at);
32+CREATE UNIQUE INDEX agent_tasks_repo ON agent_tasks(repo_id, number);
modifiedapps/diffinery/package.json+1−1View file
@@ -9,7 +9,7 @@
99 "typecheck": "tsc --noEmit && tsc --noEmit -p islands",
1010 "test": "vitest run --passWithNoTests --exclude 'test/e2e/**'",
1111 "cf-typegen": "wrangler types",
12- "e2e": "node --test --test-concurrency=1 --test-timeout=600000 test/e2e/e2e.test.mjs test/e2e/passkeys.test.mjs test/e2e/registry.test.mjs test/e2e/email.test.mjs test/e2e/security.test.mjs test/e2e/agentic.test.mjs",
12+ "e2e": "node --test --test-concurrency=1 --test-timeout=600000 test/e2e/e2e.test.mjs test/e2e/passkeys.test.mjs test/e2e/registry.test.mjs test/e2e/email.test.mjs test/e2e/security.test.mjs test/e2e/agentic.test.mjs test/e2e/tasks.test.mjs",
1313 "db:migrate:local": "wrangler d1 migrations apply diffinery-dev --local -e dev",
1414 "build": "node islands/build.mjs && node scripts/build-docs.mjs",
1515 "predev": "node islands/build.mjs && node scripts/build-docs.mjs",
addedapps/diffinery/src/agentic/ProjectDO.ts+525−0View file
@@ -1,0 +1,525 @@
1+import { DurableObject } from "cloudflare:workers";
2+import type { Env } from "../env.js";
3+import { Directory } from "../lib/directory.js";
4+import { HttpError, now, randomToken, sha256 } from "../lib/util.js";
5+import { taskCaller, TaskAccessError } from "./access.js";
6+import type { Provider } from "./keys.js";
7+import { agentEnv, finalMessage, INSTALL_TIMEOUT_MS, RUN_TIMEOUT_MS, RUNTIMES, runPrompt, taskBranch, taskTitle, type RuntimeId, type TranscriptLine } from "./runtimes.js";
8+
9+/**
10+ * One per repository: its agent tasks. A task is a goal a person gives one of
11+ * their agents. Each run of it starts a fresh sandbox as the agent, installs
12+ * the runtime, runs it with the person's model access through the model proxy,
13+ * pushes the work to the task's branch as the agent and opens a pull request.
14+ *
15+ * Runs advance in the alarm, a step at a time: boot → install → agent → finish.
16+ * Nothing here checks who may do what: the Worker does (agentic/tasks.ts),
17+ * and `boot` re-checks the person and the agent before every run.
18+ */
19+
20+export type TaskState = "running" | "ready" | "failed" | "stopped";
21+export type RunStep = "boot" | "install" | "agent" | "finish" | "done" | "failed" | "stopped";
22+const ACTIVE: RunStep[] = ["boot", "install", "agent", "finish"];
23+
24+export type TaskRow = {
25+ number: number;
26+ id: string;
27+ repo_id: string;
28+ goal: string;
29+ title: string;
30+ runtime: RuntimeId;
31+ command: string | null;
32+ agent_id: string;
33+ agent_login: string;
34+ person_id: string;
35+ person_login: string;
36+ base: string;
37+ branch: string;
38+ state: TaskState;
39+ pr_number: number | null;
40+ queued: string | null;
41+ error: string | null;
42+ created_at: string;
43+ updated_at: string;
44+};
45+
46+export type RunRow = {
47+ task: number;
48+ seq: number;
49+ kind: "start" | "message" | "retry";
50+ message: string | null;
51+ prompt: string;
52+ providers: string;
53+ step: RunStep;
54+ sandbox_id: string | null;
55+ start_sha: string | null;
56+ proc: string | null;
57+ out_off: number;
58+ err_off: number;
59+ token_hash: string | null;
60+ exit_code: number | null;
61+ error: string | null;
62+ checked_at: string | null;
63+ started_at: string;
64+ ended_at: string | null;
65+};
66+
67+export interface TaskView extends TaskRow {
68+ runs: Array<Pick<RunRow, "seq" | "kind" | "message" | "step" | "exit_code" | "error" | "started_at" | "ended_at">>;
69+}
70+
71+const SCHEMA = `
72+CREATE TABLE IF NOT EXISTS tasks (
73+ number INTEGER PRIMARY KEY,
74+ id TEXT NOT NULL UNIQUE,
75+ repo_id TEXT NOT NULL,
76+ goal TEXT NOT NULL,
77+ title TEXT NOT NULL,
78+ runtime TEXT NOT NULL,
79+ command TEXT,
80+ agent_id TEXT NOT NULL,
81+ agent_login TEXT NOT NULL,
82+ person_id TEXT NOT NULL,
83+ person_login TEXT NOT NULL,
84+ base TEXT NOT NULL,
85+ branch TEXT NOT NULL,
86+ state TEXT NOT NULL,
87+ pr_number INTEGER,
88+ queued TEXT,
89+ error TEXT,
90+ created_at TEXT NOT NULL,
91+ updated_at TEXT NOT NULL
92+);
93+CREATE TABLE IF NOT EXISTS runs (
94+ task INTEGER NOT NULL,
95+ seq INTEGER NOT NULL,
96+ kind TEXT NOT NULL,
97+ message TEXT,
98+ prompt TEXT NOT NULL,
99+ providers TEXT NOT NULL,
100+ step TEXT NOT NULL,
101+ sandbox_id TEXT,
102+ start_sha TEXT,
103+ proc TEXT,
104+ out_off INTEGER NOT NULL DEFAULT 0,
105+ err_off INTEGER NOT NULL DEFAULT 0,
106+ token_hash TEXT,
107+ exit_code INTEGER,
108+ error TEXT,
109+ checked_at TEXT,
110+ started_at TEXT NOT NULL,
111+ ended_at TEXT,
112+ PRIMARY KEY (task, seq)
113+);
114+CREATE INDEX IF NOT EXISTS runs_token ON runs(token_hash);
115+CREATE TABLE IF NOT EXISTS lines (
116+ task INTEGER NOT NULL,
117+ seq INTEGER NOT NULL,
118+ run INTEGER NOT NULL,
119+ at TEXT NOT NULL,
120+ stream TEXT NOT NULL,
121+ text TEXT NOT NULL,
122+ PRIMARY KEY (task, seq)
123+);
124+`;
125+
126+/** How often a running task is polled. */
127+const POLL_MS = 3000;
128+/** How often the person's access is re-checked while a run is active. */
129+const RECHECK_MS = 60_000;
130+const MAX_LINE = 32 * 1024;
131+const MAX_LINES_PER_RUN = 20_000;
132+
133+export interface CreateTask {
134+ repoId: string;
135+ goal: string;
136+ runtime: RuntimeId;
137+ command: string | null;
138+ agent: { id: string; login: string };
139+ person: { id: string; login: string };
140+ base: string;
141+ providers: Provider[];
142+}
143+
144+export class ProjectDO extends DurableObject<Env> {
145+ private sql: SqlStorage;
146+
147+ constructor(ctx: DurableObjectState, env: Env) {
148+ super(ctx, env);
149+ this.sql = ctx.storage.sql;
150+ this.sql.exec(SCHEMA);
151+ }
152+
153+ // ---------------------------------------------------------------- reads
154+
155+ list(opts: { limit?: number; state?: TaskState | "active" } = {}): TaskRow[] {
156+ const limit = Math.min(Math.max(opts.limit ?? 50, 1), 100);
157+ if (opts.state === "active") return this.sql.exec<TaskRow>("SELECT * FROM tasks WHERE state = 'running' ORDER BY number DESC LIMIT ?", limit).toArray();
158+ if (opts.state) return this.sql.exec<TaskRow>("SELECT * FROM tasks WHERE state = ? ORDER BY number DESC LIMIT ?", opts.state, limit).toArray();
159+ return this.sql.exec<TaskRow>("SELECT * FROM tasks ORDER BY number DESC LIMIT ?", limit).toArray();
160+ }
161+
162+ get(number: number): TaskView | null {
163+ const t = this.task(number);
164+ if (!t) return null;
165+ const runs = this.sql.exec<RunRow>("SELECT seq, kind, message, step, exit_code, error, started_at, ended_at FROM runs WHERE task = ? ORDER BY seq", number).toArray();
166+ return { ...t, runs };
167+ }
168+
169+ lines(number: number, after = 0, limit = 2000): TranscriptLine[] {
170+ return this.sql.exec<TranscriptLine>("SELECT seq, run, at, stream, text FROM lines WHERE task = ? AND seq > ? ORDER BY seq LIMIT ?", number, after, Math.min(Math.max(limit, 1), 5000)).toArray();
171+ }
172+
173+ // ---------------------------------------------------------------- changes (authorised by the Worker)
174+
175+ async create(input: CreateTask): Promise<TaskView> {
176+ const number = (this.sql.exec<{ n: number | null }>("SELECT max(number) AS n FROM tasks").one().n ?? 0) + 1;
177+ const at = now();
178+ const branch = taskBranch(number, input.goal);
179+ this.sql.exec(
180+ "INSERT INTO tasks (number, id, repo_id, goal, title, runtime, command, agent_id, agent_login, person_id, person_login, base, branch, state, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'running', ?, ?)",
181+ number,
182+ crypto.randomUUID(),
183+ input.repoId,
184+ input.goal,
185+ taskTitle(input.goal),
186+ input.runtime,
187+ input.command,
188+ input.agent.id,
189+ input.agent.login,
190+ input.person.id,
191+ input.person.login,
192+ input.base,
193+ branch,
194+ at,
195+ at,
196+ );
197+ this.startRun(number, "start", null, input.providers);
198+ await this.index(number);
199+ return this.get(number)!;
200+ }
201+
202+ /**
203+ * A follow-up from the person who started the task. While a run is active it
204+ * waits, and starts as soon as that run ends.
205+ */
206+ async message(number: number, text: string, providers: Provider[]): Promise<TaskView> {
207+ const t = this.mustTask(number);
208+ if (this.activeRun(number)) {
209+ this.sql.exec("UPDATE tasks SET queued = ?, updated_at = ? WHERE number = ?", t.queued ? `${t.queued}\n\n${text}` : text, now(), number);
210+ this.note(number, this.lastRun(number)?.seq ?? 0, `${t.person_login} sent a message; it starts a new run when this one ends.`);
211+ } else {
212+ this.startRun(number, "message", text, providers);
213+ }
214+ await this.index(number);
215+ return this.get(number)!;
216+ }
217+
218+ async retry(number: number, providers: Provider[]): Promise<TaskView> {
219+ this.mustTask(number);
220+ if (this.activeRun(number)) throw new HttpError(409, "the task is running; stop it before retrying");
221+ const last = this.lastRun(number);
222+ this.startRun(number, "retry", last?.message ?? null, providers);
223+ await this.index(number);
224+ return this.get(number)!;
225+ }
226+
227+ /** Stop the active run: kill the agent, close the sandbox (revoking its git credential) and the model token. */
228+ async stop(number: number, by: string, reason?: string): Promise<TaskView> {
229+ this.mustTask(number);
230+ const run = this.activeRun(number);
231+ this.sql.exec("UPDATE tasks SET queued = NULL WHERE number = ?", number);
232+ if (run) await this.endRun(run, "stopped", reason ?? `Stopped by ${by}.`);
233+ this.sql.exec("UPDATE tasks SET state = 'stopped', updated_at = ? WHERE number = ?", now(), number);
234+ await this.index(number);
235+ return this.get(number)!;
236+ }
237+
238+ /** The model proxy's check: which run a proxy token belongs to, while that run's agent is running. */
239+ async modelAuth(token: string): Promise<{ task: number; run: number; personId: string; agentId: string; repoId: string; providers: Provider[] } | null> {
240+ const hash = await sha256(token);
241+ const r = this.sql.exec<RunRow>("SELECT * FROM runs WHERE token_hash = ? AND step = 'agent'", hash).toArray()[0];
242+ if (!r) return null;
243+ const t = this.task(r.task)!;
244+ return { task: t.number, run: r.seq, personId: t.person_id, agentId: t.agent_id, repoId: t.repo_id, providers: JSON.parse(r.providers) as Provider[] };
245+ }
246+
247+ // ---------------------------------------------------------------- runs
248+
249+ private startRun(number: number, kind: RunRow["kind"], message: string | null, providers: Provider[]): void {
250+ const t = this.mustTask(number);
251+ const seq = (this.lastRun(number)?.seq ?? 0) + 1;
252+ const previous = seq > 1 ? finalMessage(this.sql.exec<TranscriptLine>("SELECT seq, run, at, stream, text FROM lines WHERE task = ? AND run = ? ORDER BY seq DESC LIMIT 200", number, seq - 1).toArray().reverse()) : null;
253+ const prompt = runPrompt({ goal: t.goal, number, branch: t.branch, message: message ?? undefined, previous, by: t.person_login });
254+ const at = now();
255+ this.sql.exec("INSERT INTO runs (task, seq, kind, message, prompt, providers, step, started_at) VALUES (?, ?, ?, ?, ?, ?, 'boot', ?)", number, seq, kind, message, prompt, JSON.stringify(providers), at);
256+ this.sql.exec("UPDATE tasks SET state = 'running', error = NULL, updated_at = ? WHERE number = ?", at, number);
257+ if (message) this.line(number, seq, "note", `${t.person_login}: ${message}`);
258+ this.note(number, seq, kind === "retry" ? "Retrying." : kind === "message" ? "Starting a new run with the message." : `${t.person_login} started the task with ${RUNTIMES[t.runtime].name}.`);
259+ this.arm(0);
260+ }
261+
262+ override async alarm(): Promise<void> {
263+ const runs = this.sql.exec<RunRow>(`SELECT * FROM runs WHERE step IN (${ACTIVE.map(() => "?").join(", ")}) ORDER BY task, seq`, ...ACTIVE).toArray();
264+ for (const listed of runs) {
265+ const r = this.run(listed.task, listed.seq);
266+ if (!r || !ACTIVE.includes(r.step)) continue;
267+ try {
268+ await this.advance(r);
269+ } catch (e) {
270+ const run = this.run(r.task, r.seq);
271+ if (run && ACTIVE.includes(run.step)) await this.endRun(run, "failed", e instanceof TaskAccessError || e instanceof HttpError ? e.message : `Internal error: ${e instanceof Error ? e.message : String(e)}`);
272+ }
273+ }
274+ if (this.sql.exec<{ n: number }>(`SELECT count(*) AS n FROM runs WHERE step IN (${ACTIVE.map(() => "?").join(", ")})`, ...ACTIVE).one().n) this.arm(POLL_MS);
275+ }
276+
277+ private arm(ms: number): void {
278+ this.ctx.waitUntil(this.ctx.storage.setAlarm(Date.now() + ms));
279+ }
280+
281+ private async advance(r: RunRow): Promise<void> {
282+ const t = this.mustTask(r.task);
283+ if (Date.parse(r.started_at) + RUN_TIMEOUT_MS < Date.now()) return this.endRun(r, "failed", "The run took longer than 6 hours and was stopped.");
284+ switch (r.step) {
285+ case "boot":
286+ return this.boot(t, r);
287+ case "install": {
288+ const install = RUNTIMES[t.runtime].install;
289+ if (!install) return this.setStep(r, "agent");
290+ if (!r.proc) return this.startProc(r, "install", install, {}, {}, INSTALL_TIMEOUT_MS, `Installing ${RUNTIMES[t.runtime].name}.`);
291+ const done = await this.poll(r, "setup");
292+ if (done === null) return;
293+ if (done !== 0) return this.endRun(this.run(r.task, r.seq)!, "failed", `Installing ${RUNTIMES[t.runtime].name} failed (exit ${done}).`);
294+ return this.setStep(this.run(r.task, r.seq)!, "agent");
295+ }
296+ case "agent": {
297+ if (!r.proc) {
298+ // The key itself never enters the sandbox: the runtime gets a token for this run that only the model proxy accepts.
299+ const providers = JSON.parse(r.providers) as Provider[];
300+ const token = `dmt_${t.repo_id}_${randomToken(32)}`;
301+ this.sql.exec("UPDATE runs SET token_hash = ?, checked_at = ? WHERE task = ? AND seq = ?", await sha256(token), now(), r.task, r.seq);
302+ const { env, secret } = agentEnv(this.env.PUBLIC_URL, providers, token);
303+ return this.startProc(r, "agent", RUNTIMES[t.runtime].command(t.command ?? undefined), { ...env, DIFFINERY_TASK_PROMPT: r.prompt }, secret, RUN_TIMEOUT_MS - (Date.now() - Date.parse(r.started_at)), null);
304+ }
305+ if (!r.checked_at || Date.parse(r.checked_at) + RECHECK_MS < Date.now()) {
306+ await this.checkAccess(t);
307+ this.sql.exec("UPDATE runs SET checked_at = ? WHERE task = ? AND seq = ?", now(), r.task, r.seq);
308+ }
309+ const done = await this.poll(r, "out");
310+ if (done === null) return;
311+ const run = this.run(r.task, r.seq)!;
312+ // The agent is done: its model token stops working now, before anything else runs.
313+ this.sql.exec("UPDATE runs SET exit_code = ?, token_hash = NULL WHERE task = ? AND seq = ?", done, r.task, r.seq);
314+ this.note(r.task, r.seq, done === 0 ? `${RUNTIMES[t.runtime].name} finished.` : `${RUNTIMES[t.runtime].name} exited with code ${done}.`);
315+ return this.setStep(run, "finish");
316+ }
317+ case "finish": {
318+ if (!r.proc) {
319+ return this.startProc(r, "finish", FINISH_SCRIPT, { DIFFINERY_TASK_BRANCH: t.branch, DIFFINERY_TASK_START: r.start_sha ?? "", DIFFINERY_TASK_MESSAGE: commitMessage(t, r) }, {}, 10 * 60 * 1000, `Pushing to ${t.branch}.`);
320+ }
321+ const done = await this.poll(r, "setup");
322+ if (done === null) return;
323+ const run = this.run(r.task, r.seq)!;
324+ if (done !== 0) return this.endRun(run, "failed", `Pushing the work to ${t.branch} failed (exit ${done}).`);
325+ // Whether anything was pushed comes from the branch itself, not from output the agent could have written to.
326+ const head = await this.env.REPO.get(this.env.REPO.idFromName(t.repo_id)).getRef(`refs/heads/${t.branch}`);
327+ if (head && head !== r.start_sha) await this.openPull(t, run);
328+ else this.note(r.task, r.seq, "No changes to push.");
329+ const fresh = this.run(r.task, r.seq)!;
330+ return this.endRun(fresh, (fresh.exit_code ?? 0) === 0 ? "done" : "failed", (fresh.exit_code ?? 0) === 0 ? null : `${RUNTIMES[t.runtime].name} exited with code ${fresh.exit_code}.`);
331+ }
332+ }
333+ }
334+
335+ private async boot(t: TaskRow, r: RunRow): Promise<void> {
336+ const { caller, repo } = await this.checkAccess(t);
337+ const stub = this.env.REPO.get(this.env.REPO.idFromName(t.repo_id));
338+ const branchSha = await stub.getRef(`refs/heads/${t.branch}`);
339+ const ref = branchSha ? t.branch : t.base;
340+ const startSha = branchSha ?? (await stub.getRef(`refs/heads/${t.base}`));
341+ if (!startSha) throw new HttpError(422, `the branch ${t.base} doesn't exist in ${repo.owner_login}/${repo.name}`);
342+ this.note(t.number, r.seq, `Starting a sandbox on ${ref} as ${t.agent_login}.`);
343+ const sb = await stub.createSandbox(caller, ref, { personId: t.person_id });
344+ // Stopped while the sandbox started: close it again.
345+ const now_ = this.run(t.number, r.seq);
346+ if (!now_ || now_.step !== "boot") {
347+ await stub.closeTaskSandbox(sb.id).catch(() => {});
348+ return;
349+ }
350+ this.sql.exec("UPDATE runs SET sandbox_id = ?, start_sha = ?, step = 'install', proc = NULL WHERE task = ? AND seq = ?", sb.id, startSha, t.number, r.seq);
351+ }
352+
353+ /** The person and the agent may still run this task; otherwise stop it. */
354+ private async checkAccess(t: TaskRow) {
355+ const dir = new Directory(this.env);
356+ const [repo, person, agent] = await Promise.all([dir.repoById(t.repo_id), dir.account(t.person_id), dir.account(t.agent_id)]);
357+ if (!repo || !person || !agent) throw new TaskAccessError("the repository, the person or the agent no longer exists");
358+ if (repo.archived) throw new TaskAccessError("the repository is archived");
359+ return { caller: await taskCaller(dir, person, agent, repo), repo, person, agent };
360+ }
361+
362+ private async openPull(t: TaskRow, r: RunRow): Promise<void> {
363+ const stub = this.env.REPO.get(this.env.REPO.idFromName(t.repo_id));
364+ if (t.pr_number) {
365+ const pr = (await stub.issue(t.pr_number).catch(() => null)) as { state?: string } | null;
366+ if (pr?.state === "open") {
367+ this.note(t.number, r.seq, `Pushed to #${t.pr_number}.`);
368+ return;
369+ }
370+ }
371+ const { caller } = await this.checkAccess(t);
372+ const body = [`An agent task started by @${t.person_login}, run by @${t.agent_login} with ${RUNTIMES[t.runtime].name}.`, "", "**Goal**", "", t.goal, "", `[Task #${t.number}](/${await this.repoPath(t)}/tasks/${t.number})`].join("\n");
373+ try {
374+ const pr = await stub.createPull({ title: t.title, body, head: t.branch, base: t.base, intent: t.goal.slice(0, 1000) }, caller);
375+ this.sql.exec("UPDATE tasks SET pr_number = ? WHERE number = ?", pr.number, t.number);
376+ this.note(t.number, r.seq, `Opened pull request #${pr.number}.`);
377+ } catch (e) {
378+ const msg = e instanceof Error ? e.message : String(e);
379+ const dup = /already exists for .*#(\d+)/.exec(msg);
380+ if (dup) {
381+ this.sql.exec("UPDATE tasks SET pr_number = ? WHERE number = ?", Number(dup[1]), t.number);
382+ this.note(t.number, r.seq, `Pushed to #${dup[1]}.`);
383+ } else this.note(t.number, r.seq, `Couldn't open a pull request: ${msg}`);
384+ }
385+ }
386+
387+ private async repoPath(t: TaskRow): Promise<string> {
388+ const repo = await new Directory(this.env).repoById(t.repo_id);
389+ return repo ? `${repo.owner_login}/${repo.name}` : "";
390+ }
391+
392+ private async startProc(r: RunRow, id: string, cmd: string, env: Record<string, string>, secretEnv: Record<string, string>, timeoutMs: number, note: string | null): Promise<void> {
393+ if (note) this.note(r.task, r.seq, note);
394+ await this.sandbox(r, "/proc/start", { id, cmd, env, secretEnv, timeoutMs: Math.max(timeoutMs, 1000) });
395+ this.sql.exec("UPDATE runs SET proc = ?, out_off = 0, err_off = 0 WHERE task = ? AND seq = ?", id, r.task, r.seq);
396+ }
397+
398+ /** Read new output into the transcript; the exit code once the process has ended, else null. */
399+ private async poll(r: RunRow, out: "out" | "setup"): Promise<number | null> {
400+ const p = (await this.sandbox(r, "/proc/read", { id: r.proc, stdoutOffset: r.out_off, stderrOffset: r.err_off })) as { running: boolean; exitCode: number | null; timedOut: boolean; stdout: string; stderr: string; stdoutOffset: number; stderrOffset: number };
401+ for (const l of splitLines(p.stdout)) this.line(r.task, r.seq, out, l);
402+ for (const l of splitLines(p.stderr)) this.line(r.task, r.seq, out === "out" ? "err" : "setup", l);
403+ this.sql.exec("UPDATE runs SET out_off = ?, err_off = ? WHERE task = ? AND seq = ?", p.stdoutOffset, p.stderrOffset, r.task, r.seq);
404+ // Output read while it was running, or more than one read's worth after it ended: read again on the next tick.
405+ if (p.running || p.stdout.length || p.stderr.length) return null;
406+ if (p.timedOut) this.note(r.task, r.seq, "Timed out.");
407+ return p.exitCode ?? 1;
408+ }
409+
410+ private async sandbox(r: RunRow, path: string, body: unknown): Promise<unknown> {
411+ if (!r.sandbox_id) throw new Error("the run has no sandbox");
412+ const res = await this.env.SANDBOX.get(this.env.SANDBOX.idFromName(r.sandbox_id)).call(path, JSON.stringify(body));
413+ let out: unknown = res.body;
414+ try {
415+ out = JSON.parse(res.body);
416+ } catch {
417+ /* text */
418+ }
419+ if (res.status >= 400) throw new Error(`sandbox ${path}: ${out && typeof out === "object" && "error" in out ? String((out as { error: unknown }).error) : res.body}`);
420+ return out;
421+ }
422+
423+ private setStep(r: RunRow, step: RunStep): void {
424+ // Only while the run is still active: a stop that landed during an await wins.
425+ this.sql.exec(`UPDATE runs SET step = ?, proc = NULL, out_off = 0, err_off = 0 WHERE task = ? AND seq = ? AND step IN (${ACTIVE.map(() => "?").join(", ")})`, step, r.task, r.seq, ...ACTIVE);
426+ this.arm(0);
427+ }
428+
429+ /** End a run: revoke its model token, kill the agent, close the sandbox (which revokes its git credential), then start a queued message. */
430+ private async endRun(r: RunRow, step: "done" | "failed" | "stopped", error: string | null): Promise<void> {
431+ const at = now();
432+ const current = this.run(r.task, r.seq);
433+ if (!current || !ACTIVE.includes(current.step)) return;
434+ r = current;
435+ this.sql.exec("UPDATE runs SET step = ?, error = ?, token_hash = NULL, ended_at = ? WHERE task = ? AND seq = ?", step, error, at, r.task, r.seq);
436+ if (error) this.note(r.task, r.seq, error);
437+ if (r.sandbox_id) {
438+ if (r.proc) await this.sandbox(r, "/proc/kill", { id: r.proc, signal: "SIGKILL" }).catch(() => {});
439+ await this.env.REPO.get(this.env.REPO.idFromName(this.mustTask(r.task).repo_id))
440+ .closeTaskSandbox(r.sandbox_id)
441+ .catch((e) => console.error("closing a task sandbox failed", e));
442+ }
443+ const t = this.mustTask(r.task);
444+ if (t.queued && step !== "stopped") {
445+ this.sql.exec("UPDATE tasks SET queued = NULL WHERE number = ?", r.task);
446+ this.startRun(r.task, "message", t.queued, JSON.parse(r.providers) as Provider[]);
447+ } else {
448+ this.sql.exec("UPDATE tasks SET state = ?, error = ?, updated_at = ? WHERE number = ?", step === "done" ? "ready" : step, error, at, r.task);
449+ }
450+ await this.index(r.task);
451+ }
452+
453+ // ---------------------------------------------------------------- helpers
454+
455+ private task(number: number): TaskRow | null {
456+ return this.sql.exec<TaskRow>("SELECT * FROM tasks WHERE number = ?", number).toArray()[0] ?? null;
457+ }
458+
459+ private mustTask(number: number): TaskRow {
460+ const t = this.task(number);
461+ if (!t) throw new HttpError(404, "task not found");
462+ return t;
463+ }
464+
465+ private run(task: number, seq: number): RunRow | null {
466+ return this.sql.exec<RunRow>("SELECT * FROM runs WHERE task = ? AND seq = ?", task, seq).toArray()[0] ?? null;
467+ }
468+
469+ private lastRun(task: number): RunRow | null {
470+ return this.sql.exec<RunRow>("SELECT * FROM runs WHERE task = ? ORDER BY seq DESC LIMIT 1", task).toArray()[0] ?? null;
471+ }
472+
473+ private activeRun(task: number): RunRow | null {
474+ return this.sql.exec<RunRow>(`SELECT * FROM runs WHERE task = ? AND step IN (${ACTIVE.map(() => "?").join(", ")}) LIMIT 1`, task, ...ACTIVE).toArray()[0] ?? null;
475+ }
476+
477+ private note(task: number, run: number, text: string): void {
478+ this.line(task, run, "note", text);
479+ }
480+
481+ private line(task: number, run: number, stream: TranscriptLine["stream"], text: string): void {
482+ const n = this.sql.exec<{ n: number }>("SELECT count(*) AS n FROM lines WHERE task = ? AND run = ?", task, run).one().n;
483+ if (n > MAX_LINES_PER_RUN && stream !== "note") return;
484+ const seq = (this.sql.exec<{ s: number | null }>("SELECT max(seq) AS s FROM lines WHERE task = ?", task).one().s ?? 0) + 1;
485+ this.sql.exec("INSERT INTO lines (task, seq, run, at, stream, text) VALUES (?, ?, ?, ?, ?, ?)", task, seq, run, now(), stream, text.length > MAX_LINE ? `${text.slice(0, MAX_LINE)} … (line cut at ${MAX_LINE} characters)` : text);
486+ }
487+
488+ /** The D1 index of tasks across repositories, for Today and a person's task list. */
489+ private async index(number: number): Promise<void> {
490+ const t = this.task(number);
491+ if (!t) return;
492+ await this.env.DB.prepare(
493+ "INSERT INTO agent_tasks (id, repo_id, number, title, runtime, agent_id, started_by, state, pr_number, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (id) DO UPDATE SET title = excluded.title, state = excluded.state, pr_number = excluded.pr_number, updated_at = excluded.updated_at",
494+ )
495+ .bind(t.id, t.repo_id, t.number, t.title, t.runtime, t.agent_id, t.person_id, t.state, t.pr_number, t.created_at, t.updated_at)
496+ .run()
497+ .catch((e) => console.error("agent_tasks index failed", e));
498+ }
499+}
500+
501+function splitLines(s: string): string[] {
502+ if (!s) return [];
503+ const lines = s.split("\n");
504+ if (lines[lines.length - 1] === "") lines.pop();
505+ return lines;
506+}
507+
508+function commitMessage(t: TaskRow, r: RunRow): string {
509+ return `${t.title}\n\nAgent task #${t.number}, run ${r.seq}, started by @${t.person_login}.`;
510+}
511+
512+/**
513+ * Commit what the agent left uncommitted, then push to the task's branch if
514+ * anything changed since the run started. The sandbox's own git credential (the
515+ * agent's, through branch protection and push protection) does the push.
516+ */
517+const FINISH_SCRIPT = [
518+ "set -e",
519+ 'git add -A',
520+ 'if ! git diff --cached --quiet; then git commit -q -m "$DIFFINERY_TASK_MESSAGE"; fi',
521+ 'head=$(git rev-parse HEAD)',
522+ 'if [ "$head" = "$DIFFINERY_TASK_START" ]; then echo "diffinery: nothing to push"; exit 0; fi',
523+ 'git push -q origin "HEAD:refs/heads/$DIFFINERY_TASK_BRANCH" 2>&1',
524+ 'echo "diffinery: pushed $head"',
525+].join("\n");
addedapps/diffinery/src/agentic/access.ts+45−0View file
@@ -1,0 +1,45 @@
1+import type { Account, Directory, Repo } from "../lib/directory.js";
2+import { atLeast, type Caller } from "../repo/types.js";
3+
4+/**
5+ * Who may run an agent task: a person with write access to the repository,
6+ * driving an agent they manage (they own it, or they are an owner or admin of
7+ * the organization that owns it) that itself has write access there.
8+ */
9+
10+/** Whether `person` may drive `agent`: the same rule as managing it in Settings → Agents. */
11+export async function mayDriveAgent(dir: Directory, person: Account, agent: Account): Promise<boolean> {
12+ if (person.kind !== "user" || person.status !== "active") return false;
13+ if (agent.kind !== "agent" || agent.status !== "active" || !agent.owner_id) return false;
14+ if (agent.owner_id === person.id) return true;
15+ const role = await dir.orgRole(agent.owner_id, person.id);
16+ return role === "owner" || role === "admin";
17+}
18+
19+/**
20+ * The caller a task's sandbox acts as: the agent, on behalf of the person, at
21+ * most write. Throws when the person or the agent isn't allowed (any more).
22+ */
23+export async function taskCaller(dir: Directory, person: Account, agent: Account, repo: Repo): Promise<Caller> {
24+ const personRole = await dir.roleFor(person, repo);
25+ if (!atLeast(personRole, "write")) throw new TaskAccessError("you need write access to this repository to run agent tasks");
26+ if (!(await mayDriveAgent(dir, person, agent))) throw new TaskAccessError(`you can't run tasks as ${agent.login}: you must own it, or administer the organization that owns it`);
27+ const agentRole = await dir.roleFor(agent, repo);
28+ if (!atLeast(agentRole, "write")) throw new TaskAccessError(`${agent.login} needs write access to this repository`);
29+ // Like any sandbox, at most write, whatever the agent's own role.
30+ return { id: agent.id, login: agent.login, kind: "agent", onBehalfOf: person.login, role: "write" };
31+}
32+
33+/** Whether the person who started a task may still drive it (checked on every resume and every use of its git credential). */
34+export async function taskPersonAllowed(dir: Directory, personId: string, agent: Account, repo: Repo): Promise<boolean> {
35+ const person = await dir.account(personId);
36+ if (!person) return false;
37+ try {
38+ await taskCaller(dir, person, agent, repo);
39+ return true;
40+ } catch {
41+ return false;
42+ }
43+}
44+
45+export class TaskAccessError extends Error {}
modifiedapps/diffinery/src/agentic/api.ts+96−0View file
@@ -1,8 +1,20 @@
11 import { Hono } from "hono";
22 import type { Env } from "../env.js";
33 import { requireAccount, type AppContext, type AppVars } from "../auth/principal.js";
4+import { loadRepo } from "../http/common.js";
5+import { HttpError } from "../lib/util.js";
6+import { isProvider, ModelKeys, type ModelKeyInfo } from "./keys.js";
7+import { getTask, lineJson, messageTask, myTasks, projectStub, requirePersonSession, requireTaskReader, retryTask, startTask, stopTask, taskJson, taskNumber, type StartInput } from "./tasks.js";
48 import { gatherToday, todayJson } from "./today.js";
59
10+async function jsonBody<T>(c: AppContext): Promise<T> {
11+ try {
12+ return (await c.req.json()) as T;
13+ } catch {
14+ throw new HttpError(400, "request body must be JSON");
15+ }
16+}
17+
618 /** Diffinery-only REST endpoints for agentic work, mounted under /api/v3 (see http/api.ts). */
719 export const agenticApi = new Hono<{ Bindings: Env; Variables: AppVars }>();
820
@@ -17,3 +29,87 @@
1729 const today = await gatherToday(c.env, c.get("dir"), me, { repoIds: tokenRepoIds(c) });
1830 return c.json(todayJson(c.env.PUBLIC_URL.replace(/\/$/, ""), today));
1931 });
32+
33+// ---------------------------------------------------------------- model access (agentic/keys.ts)
34+
35+const keyJson = (k: ModelKeyInfo) => ({ provider: k.provider, hint: k.hint, opted_in_at: k.optedInAt, created_at: k.createdAt, updated_at: k.updatedAt });
36+
37+agenticApi.get("/diffinery/model-keys", async (c) => c.json((await new ModelKeys(c.env).list(requireAccount(c).id)).map(keyJson)));
38+
39+agenticApi.put("/diffinery/model-keys/:provider", async (c) => {
40+ // A key is spent by tasks this person starts; only they, in a browser, can set it.
41+ const me = requirePersonSession(c);
42+ const provider = c.req.param("provider");
43+ if (!isProvider(provider)) throw new HttpError(404, "unknown provider: use anthropic or openai");
44+ const b = await jsonBody<{ key?: unknown; accept_terms?: unknown }>(c);
45+ if (typeof b.key !== "string") throw new HttpError(422, "`key` is required");
46+ const k = await new ModelKeys(c.env).set(me, provider, b.key, b.accept_terms === true);
47+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "model_key.set", target: provider, detail: { hint: k.hint, opted_in_at: k.optedInAt } }));
48+ return c.json(keyJson(k));
49+});
50+
51+agenticApi.delete("/diffinery/model-keys/:provider", async (c) => {
52+ const me = requirePersonSession(c);
53+ const provider = c.req.param("provider");
54+ if (!isProvider(provider)) throw new HttpError(404, "unknown provider: use anthropic or openai");
55+ if (!(await new ModelKeys(c.env).delete(me.id, provider))) throw new HttpError(404, "no key for that provider");
56+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "model_key.delete", target: provider }));
57+ return c.body(null, 204);
58+});
59+
60+// ---------------------------------------------------------------- agent tasks (agentic/tasks.ts)
61+
62+const base = (c: AppContext) => c.env.PUBLIC_URL.replace(/\/$/, "");
63+const full = (l: { repo: { owner_login: string; name: string } }) => `${l.repo.owner_login}/${l.repo.name}`;
64+const repoOf = (c: AppContext, min: "read" | "write" = "read") => loadRepo(c, c.req.param("owner")!, c.req.param("repo")!, min);
65+
66+agenticApi.get("/repos/:owner/:repo/diffinery/tasks", async (c) => {
67+ const l = await repoOf(c);
68+ requireTaskReader(l);
69+ const state = c.req.query("state");
70+ const tasks = await projectStub(c.env, l.repo.id).list({ limit: Number(c.req.query("per_page")) || 50, state: state === "running" || state === "ready" || state === "failed" || state === "stopped" || state === "active" ? state : undefined });
71+ return c.json(tasks.map((t) => taskJson(base(c), full(l), t)));
72+});
73+
74+agenticApi.post("/repos/:owner/:repo/diffinery/tasks", async (c) => {
75+ requirePersonSession(c);
76+ const l = await repoOf(c, "write");
77+ const t = await startTask(c, l, await jsonBody<StartInput>(c));
78+ return c.json(taskJson(base(c), full(l), t), 201);
79+});
80+
81+agenticApi.get("/repos/:owner/:repo/diffinery/tasks/:number", async (c) => {
82+ const l = await repoOf(c);
83+ return c.json(taskJson(base(c), full(l), await getTask(c, l, taskNumber(c.req.param("number")))));
84+});
85+
86+agenticApi.get("/repos/:owner/:repo/diffinery/tasks/:number/transcript", async (c) => {
87+ const l = await repoOf(c);
88+ const n = taskNumber(c.req.param("number"));
89+ await getTask(c, l, n);
90+ const lines = await projectStub(c.env, l.repo.id).lines(n, Number(c.req.query("after")) || 0, Number(c.req.query("per_page")) || 2000);
91+ return c.json(lines.map(lineJson));
92+});
93+
94+agenticApi.post("/repos/:owner/:repo/diffinery/tasks/:number/messages", async (c) => {
95+ requirePersonSession(c);
96+ const l = await repoOf(c, "write");
97+ const b = await jsonBody<{ body?: unknown }>(c);
98+ return c.json(taskJson(base(c), full(l), await messageTask(c, l, taskNumber(c.req.param("number")), b.body)), 201);
99+});
100+
101+agenticApi.post("/repos/:owner/:repo/diffinery/tasks/:number/retry", async (c) => {
102+ requirePersonSession(c);
103+ const l = await repoOf(c, "write");
104+ return c.json(taskJson(base(c), full(l), await retryTask(c, l, taskNumber(c.req.param("number")))), 201);
105+});
106+
107+agenticApi.post("/repos/:owner/:repo/diffinery/tasks/:number/stop", async (c) => {
108+ const l = await repoOf(c, "write");
109+ return c.json(taskJson(base(c), full(l), await stopTask(c, l, taskNumber(c.req.param("number")))));
110+});
111+
112+agenticApi.get("/diffinery/tasks", async (c) => {
113+ const me = requireAccount(c);
114+ return c.json(await myTasks(c, me, tokenRepoIds(c)));
115+});
addedapps/diffinery/src/agentic/keys.ts+108−0View file
@@ -1,0 +1,108 @@
1+import type { Env } from "../env.js";
2+import { decryptSecret, encryptSecret } from "../lib/crypto.js";
3+import { HttpError, now } from "../lib/util.js";
4+import { enqueueMail } from "../mail/send.js";
5+
6+/**
7+ * People's own model API keys for agent tasks. Diffinery only provides the
8+ * ability: each person brings their own key, under their own account with the
9+ * provider and the provider's terms, after opting in per provider.
10+ *
11+ * Keys are write-only. Nothing returns one; the model proxy (proxy.ts)
12+ * decrypts a key only to forward a request of that person's running task, so a
13+ * key never reaches a sandbox.
14+ */
15+
16+export type Provider = "anthropic" | "openai";
17+
18+export interface ProviderInfo {
19+ id: Provider;
20+ name: string;
21+ /** Where people create a key. */
22+ keysUrl: string;
23+ terms: Array<{ label: string; url: string }>;
24+ /** A loose shape check, to catch pasting the wrong thing. */
25+ pattern: RegExp;
26+}
27+
28+export const PROVIDERS: Record<Provider, ProviderInfo> = {
29+ anthropic: {
30+ id: "anthropic",
31+ name: "Anthropic",
32+ keysUrl: "https://console.anthropic.com/settings/keys",
33+ terms: [
34+ { label: "Anthropic Commercial Terms of Service", url: "https://www.anthropic.com/legal/commercial-terms" },
35+ { label: "Anthropic Usage Policy", url: "https://www.anthropic.com/legal/aup" },
36+ ],
37+ pattern: /^sk-ant-[A-Za-z0-9_-]{20,400}$/,
38+ },
39+ openai: {
40+ id: "openai",
41+ name: "OpenAI",
42+ keysUrl: "https://platform.openai.com/api-keys",
43+ terms: [
44+ { label: "OpenAI Terms and Policies", url: "https://openai.com/policies/" },
45+ { label: "OpenAI Usage Policies", url: "https://openai.com/policies/usage-policies/" },
46+ ],
47+ pattern: /^sk-[A-Za-z0-9_-]{20,400}$/,
48+ },
49+};
50+
51+export const isProvider = (p: string): p is Provider => p === "anthropic" || p === "openai";
52+
53+export interface ModelKeyInfo {
54+ provider: Provider;
55+ /** The key's last four characters. */
56+ hint: string;
57+ optedInAt: string;
58+ createdAt: string;
59+ updatedAt: string;
60+}
61+
62+const scope = (accountId: string, provider: Provider) => `model-key:${accountId}:${provider}`;
63+
64+export class ModelKeys {
65+ constructor(private env: Env) {}
66+
67+ async list(accountId: string): Promise<ModelKeyInfo[]> {
68+ const r = await this.env.DB.prepare("SELECT provider, hint, opted_in_at, created_at, updated_at FROM model_keys WHERE account_id = ? ORDER BY provider")
69+ .bind(accountId)
70+ .all<{ provider: Provider; hint: string; opted_in_at: string; created_at: string; updated_at: string }>();
71+ return r.results.map((k) => ({ provider: k.provider, hint: k.hint, optedInAt: k.opted_in_at, createdAt: k.created_at, updatedAt: k.updated_at }));
72+ }
73+
74+ async providers(accountId: string): Promise<Set<Provider>> {
75+ return new Set((await this.list(accountId)).map((k) => k.provider));
76+ }
77+
78+ /**
79+ * Save (or replace) a key. The caller has checked that this is the person
80+ * themselves in a browser session; `optIn` is their acceptance of the notice.
81+ */
82+ async set(account: { id: string }, provider: Provider, key: string, optIn: boolean): Promise<ModelKeyInfo> {
83+ if (!optIn) throw new HttpError(422, `confirm that you use ${PROVIDERS[provider].name} under your own account and its terms`);
84+ const k = key.trim();
85+ if (!PROVIDERS[provider].pattern.test(k)) throw new HttpError(422, `that doesn't look like an ${PROVIDERS[provider].name} API key`);
86+ const at = now();
87+ const ciphertext = await encryptSecret(this.env.MASTER_KEY, scope(account.id, provider), k);
88+ const hint = k.slice(-4);
89+ await this.env.DB.prepare(
90+ "INSERT INTO model_keys (account_id, provider, ciphertext, hint, opted_in_at, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?) ON CONFLICT (account_id, provider) DO UPDATE SET ciphertext = excluded.ciphertext, hint = excluded.hint, opted_in_at = excluded.opted_in_at, updated_at = excluded.updated_at",
91+ )
92+ .bind(account.id, provider, ciphertext, hint, at, at, at)
93+ .run();
94+ await enqueueMail(this.env, [{ kind: "model_key_added", data: { provider: PROVIDERS[provider].name, hint }, to: { accountId: account.id }, reason: "security_alert", at }]);
95+ return { provider, hint, optedInAt: at, createdAt: at, updatedAt: at };
96+ }
97+
98+ async delete(accountId: string, provider: Provider): Promise<boolean> {
99+ const r = await this.env.DB.prepare("DELETE FROM model_keys WHERE account_id = ? AND provider = ?").bind(accountId, provider).run();
100+ return (r.meta.changes ?? 0) > 0;
101+ }
102+
103+ /** The key itself: only for the model proxy, while a task of this person runs. */
104+ async reveal(accountId: string, provider: Provider): Promise<string | null> {
105+ const row = await this.env.DB.prepare("SELECT ciphertext FROM model_keys WHERE account_id = ? AND provider = ?").bind(accountId, provider).first<{ ciphertext: string }>();
106+ return row ? decryptSecret(this.env.MASTER_KEY, scope(accountId, provider), row.ciphertext) : null;
107+ }
108+}
modifiedapps/diffinery/src/agentic/mcp.ts+45−0View file
@@ -1,8 +1,26 @@
11 import { requireAccount } from "../auth/principal.js";
22 import type { Tool } from "../mcp/server.js";
33 import { tokenRepoIds } from "./api.js";
4+import { loadRepo } from "../http/common.js";
5+import { HttpError } from "../lib/util.js";
6+import { getTask, lineJson, myTasks, projectStub, requireTaskReader, taskJson } from "./tasks.js";
47 import { gatherToday, todayJson } from "./today.js";
58
9+type Args = Record<string, unknown>;
10+const OWNER_REPO = {
11+ owner: { type: "string", description: "Repository owner (user or org login)" },
12+ repo: { type: "string", description: "Repository name" },
13+};
14+const need = (a: Args, k: string): string => {
15+ if (typeof a[k] !== "string" || !a[k]) throw new HttpError(422, `${k} is required`);
16+ return a[k] as string;
17+};
18+const num = (a: Args, k: string): number => {
19+ const n = Number(a[k]);
20+ if (!Number.isInteger(n) || n < 1) throw new HttpError(422, `${k} must be a positive integer`);
21+ return n;
22+};
23+
624 /** MCP tools for agentic work (merged into mcp/server.ts). */
725 export const AGENTIC_TOOLS: Record<string, Tool> = {
826 today: {
@@ -12,4 +30,31 @@
1230 readOnly: true,
1331 run: async (c) => todayJson(c.env.PUBLIC_URL.replace(/\/$/, ""), await gatherToday(c.env, c.get("dir"), requireAccount(c), { repoIds: tokenRepoIds(c) })),
1432 },
33+ // Agent tasks are read-only over MCP: starting, steering and retrying spend a person's own model key, so they need a browser session.
34+ agent_tasks: {
35+ description:
36+ "List agent tasks. With owner and repo: that repository's tasks (you need write access). Without: the tasks you started (for an agent: the tasks it runs). Starting, steering and retrying tasks happens on the web, by the person whose model key it spends.",
37+ input: { ...OWNER_REPO },
38+ readOnly: true,
39+ run: async (c, a) => {
40+ if (!a.owner && !a.repo) return myTasks(c, requireAccount(c), tokenRepoIds(c));
41+ const l = await loadRepo(c, need(a, "owner"), need(a, "repo"), "read", { follow: true });
42+ requireTaskReader(l);
43+ const base = c.env.PUBLIC_URL.replace(/\/$/, "");
44+ return (await projectStub(c.env, l.repo.id).list({ limit: 50 })).map((t) => taskJson(base, `${l.repo.owner_login}/${l.repo.name}`, t));
45+ },
46+ },
47+ agent_task: {
48+ description: "One agent task: its goal, state, runs, branch and pull request, and its transcript after `after` (a line number; default from the start, at most 500 lines).",
49+ input: { ...OWNER_REPO, number: { type: "integer", description: "Task number" }, after: { type: "integer", description: "Return transcript lines after this one" } },
50+ required: ["owner", "repo", "number"],
51+ readOnly: true,
52+ run: async (c, a) => {
53+ const l = await loadRepo(c, need(a, "owner"), need(a, "repo"), "read", { follow: true });
54+ const n = num(a, "number");
55+ const t = await getTask(c, l, n);
56+ const lines = await projectStub(c.env, l.repo.id).lines(n, Number(a.after) || 0, 500);
57+ return { ...taskJson(c.env.PUBLIC_URL.replace(/\/$/, ""), `${l.repo.owner_login}/${l.repo.name}`, t), transcript: lines.map(lineJson) };
58+ },
59+ },
1560 };
addedapps/diffinery/src/agentic/proxy.ts+87−0View file
@@ -1,0 +1,87 @@
1+import type { AppContext } from "../auth/principal.js";
2+import { Directory } from "../lib/directory.js";
3+import { taskPersonAllowed } from "./access.js";
4+import { isProvider, ModelKeys, PROVIDERS, type Provider } from "./keys.js";
5+
6+/**
7+ * The model proxy: `/-/model/<provider>/v1/...`. A task's runtime calls it
8+ * with the run's token (`dmt_...`, as its API key) and the proxy forwards the
9+ * request to the provider with the person's own key, added here. So the key
10+ * never enters the sandbox, and the token stops working when the run ends.
11+ *
12+ * It runs before the app's identity middleware: its token isn't an account
13+ * credential and works nowhere else.
14+ */
15+
16+const UPSTREAM: Record<Provider, string> = { anthropic: "https://api.anthropic.com", openai: "https://api.openai.com" };
17+
18+/**
19+ * Only the model endpoints the runtimes use. The person's key could reach the
20+ * rest of their provider account (stored files, batches, assistants), which an
21+ * agent misled by repository content must not.
22+ */
23+const ALLOWED: Record<Provider, Array<[method: string, path: RegExp]>> = {
24+ anthropic: [
25+ ["POST", /^\/v1\/messages$/],
26+ ["POST", /^\/v1\/messages\/count_tokens$/],
27+ ["GET", /^\/v1\/models(\/[A-Za-z0-9._:-]+)?$/],
28+ ],
29+ openai: [
30+ ["POST", /^\/v1\/responses$/],
31+ // Codex compacts long conversations, and reads a background response back by id.
32+ ["POST", /^\/v1\/responses\/compact$/],
33+ ["GET", /^\/v1\/responses\/[A-Za-z0-9_-]+$/],
34+ ["POST", /^\/v1\/chat\/completions$/],
35+ ["GET", /^\/v1\/models(\/[A-Za-z0-9._:-]+)?$/],
36+ ],
37+};
38+
39+export const proxyAllows = (provider: Provider, method: string, path: string): boolean => ALLOWED[provider].some(([m, re]) => m === method && re.test(path));
40+
41+/** Request headers that never go upstream: credentials, cookies and the hop's own. */
42+const DROP = new Set(["authorization", "x-api-key", "cookie", "host", "content-length", "connection", "cf-connecting-ip", "cf-ipcountry", "cf-ray", "cf-visitor", "x-forwarded-for", "x-forwarded-proto", "x-real-ip"]);
43+
44+const err = (status: number, message: string) =>
45+ new Response(JSON.stringify({ type: "error", error: { type: status === 401 ? "authentication_error" : status === 404 ? "not_found_error" : "permission_error", message } }), { status, headers: { "content-type": "application/json" } });
46+
47+export function proxyToken(req: Request): string | null {
48+ const key = req.headers.get("x-api-key");
49+ if (key?.startsWith("dmt_")) return key.trim();
50+ const m = /^Bearer\s+(dmt_\S+)$/i.exec(req.headers.get("authorization") ?? "");
51+ return m ? m[1]! : null;
52+}
53+
54+export async function modelProxy(c: AppContext): Promise<Response> {
55+ const provider = c.req.param("provider") ?? "";
56+ if (!isProvider(provider)) return err(404, "unknown model provider");
57+ const url = new URL(c.req.url);
58+ const rest = url.pathname.slice(`/-/model/${provider}`.length);
59+ if (!proxyAllows(provider, c.req.method, rest)) return err(403, `${c.req.method} ${rest} isn't available to agent tasks: only the ${PROVIDERS[provider].name} model endpoints are`);
60+ const token = proxyToken(c.req.raw);
61+ if (!token) return err(401, "a task's model token is required");
62+ const repoId = token.split("_")[1];
63+ if (!repoId || !/^[A-Za-z0-9-]{8,64}$/.test(repoId)) return err(401, "malformed model token");
64+ const env = c.env;
65+ const project = env.PROJECT.get(env.PROJECT.idFromName(repoId));
66+ const run = await project.modelAuth(token);
67+ if (!run) return err(401, "this model token isn't valid: its task run has ended");
68+ if (!run.providers.includes(provider)) return err(403, `this task doesn't use ${PROVIDERS[provider].name}`);
69+ // The person must still be allowed to run the task, on every request; otherwise the task stops.
70+ const dir = new Directory(env);
71+ const [repo, agent] = await Promise.all([dir.repoById(run.repoId), dir.account(run.agentId)]);
72+ if (!repo || !agent || !(await taskPersonAllowed(dir, run.personId, agent, repo))) {
73+ c.executionCtx.waitUntil(project.stop(run.task, "Diffinery", "Stopped: the person who started the task can no longer run it.").then(() => undefined, () => undefined));
74+ return err(403, "the person who started this task can no longer run it");
75+ }
76+ const key = await new ModelKeys(env).reveal(run.personId, provider);
77+ if (!key) return err(403, `no ${PROVIDERS[provider].name} key in Settings → Model access`);
78+ const headers = new Headers();
79+ for (const [k, v] of c.req.raw.headers) if (!DROP.has(k.toLowerCase())) headers.set(k, v);
80+ if (provider === "anthropic") headers.set("x-api-key", key);
81+ else headers.set("authorization", `Bearer ${key}`);
82+ const base = (provider === "anthropic" ? env.MODEL_UPSTREAM_ANTHROPIC : env.MODEL_UPSTREAM_OPENAI) || UPSTREAM[provider];
83+ const res = await fetch(`${base}${rest}${url.search}`, { method: c.req.method, headers, body: ["GET", "HEAD"].includes(c.req.method) ? undefined : c.req.raw.body, redirect: "manual" });
84+ const out = new Headers(res.headers);
85+ out.delete("set-cookie");
86+ return new Response(res.body, { status: res.status, headers: out });
87+}
addedapps/diffinery/src/agentic/runtimes.ts+212−0View file
@@ -1,0 +1,212 @@
1+import type { Provider } from "./keys.js";
2+
3+/**
4+ * The coding agents a task can run, installed from their vendors' official
5+ * npm packages and run unmodified, headless, with the person's own model
6+ * access through Diffinery's model proxy.
7+ */
8+
9+export type RuntimeId = "claude-code" | "codex" | "custom";
10+
11+/**
12+ * Pinned runtime versions: the install runs while the sandbox holds the agent's
13+ * git credential, so a new release is taken on deliberately, not automatically.
14+ */
15+export const RUNTIME_VERSIONS = { "claude-code": "2.1.296", codex: "0.162.1" } as const;
16+
17+export interface Runtime {
18+ id: RuntimeId;
19+ name: string;
20+ /** What it says on the new-task page. */
21+ description: string;
22+ /** Providers it needs a key for (custom: none required; it gets whichever the person has). */
23+ needs: Provider[];
24+ /** Run before any credential exists in the sandbox. */
25+ install: string | null;
26+ /** The run itself; the prompt is in $DIFFINERY_TASK_PROMPT, never on the command line we build. */
27+ command: (custom?: string) => string;
28+}
29+
30+export const RUNTIMES: Record<RuntimeId, Runtime> = {
31+ "claude-code": {
32+ id: "claude-code",
33+ name: "Claude Code",
34+ description: "Anthropic's coding agent, with your Anthropic API key.",
35+ needs: ["anthropic"],
36+ install: `npm install --global --no-fund --no-audit --loglevel=error @anthropic-ai/claude-code@${RUNTIME_VERSIONS["claude-code"]} && claude --version`,
37+ // The sandbox runs as root in its own container; IS_SANDBOX lets Claude Code skip permission prompts there.
38+ command: () => 'claude -p "$DIFFINERY_TASK_PROMPT" --output-format stream-json --verbose --permission-mode bypassPermissions',
39+ },
40+ codex: {
41+ id: "codex",
42+ name: "Codex",
43+ description: "OpenAI's coding agent, with your OpenAI API key.",
44+ needs: ["openai"],
45+ install: `npm install --global --no-fund --no-audit --loglevel=error @openai/codex@${RUNTIME_VERSIONS.codex} && codex --version`,
46+ command: () => 'codex exec --json --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check "$DIFFINERY_TASK_PROMPT"',
47+ },
48+ custom: {
49+ id: "custom",
50+ name: "Custom command",
51+ description: "Any shell command, such as another agent's CLI. It gets the model proxy for each provider you have a key for.",
52+ needs: [],
53+ install: null,
54+ command: (custom) => custom ?? "true",
55+ },
56+};
57+
58+export const isRuntime = (r: string): r is RuntimeId => r === "claude-code" || r === "codex" || r === "custom";
59+
60+/** The longest a run may take. A sandbox's git credential lasts 24 hours. */
61+export const RUN_TIMEOUT_MS = 6 * 60 * 60 * 1000;
62+export const INSTALL_TIMEOUT_MS = 10 * 60 * 1000;
63+export const MAX_GOAL = 20_000;
64+export const MAX_COMMAND = 2_000;
65+
66+/**
67+ * Environment for the agent process: where the model proxy is, and (in
68+ * `secret`) the task's proxy token under each provider's usual variable. The
69+ * real keys stay in Diffinery.
70+ */
71+export function agentEnv(publicUrl: string, providers: Provider[], proxyToken: string): { env: Record<string, string>; secret: Record<string, string> } {
72+ const env: Record<string, string> = { IS_SANDBOX: "1", CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: "1", DISABLE_AUTOUPDATER: "1" };
73+ const secret: Record<string, string> = {};
74+ for (const p of providers) {
75+ if (p === "anthropic") {
76+ env.ANTHROPIC_BASE_URL = `${publicUrl}/-/model/anthropic`;
77+ secret.ANTHROPIC_API_KEY = proxyToken;
78+ } else {
79+ env.OPENAI_BASE_URL = `${publicUrl}/-/model/openai/v1`;
80+ secret.OPENAI_API_KEY = proxyToken;
81+ secret.CODEX_API_KEY = proxyToken;
82+ }
83+ }
84+ return { env, secret };
85+}
86+
87+/** The prompt for a run: the goal, plus what came before for a follow-up or a retry. */
88+export function runPrompt(o: { goal: string; number: number; branch: string; message?: string; previous?: string | null; by: string }): string {
89+ const rules = [
90+ `You are working on task #${o.number} in this Git repository, on behalf of ${o.by}.`,
91+ "Work in the current directory. When you are done, leave your changes in the working tree or commit them; Diffinery pushes them to the branch " + o.branch + " and opens a pull request for review.",
92+ "Don't push, change Git remotes or credentials, or open pull requests yourself.",
93+ ].join(" ");
94+ if (!o.message) return `${rules}\n\nGoal:\n${o.goal}`;
95+ return [rules, "", "The original goal:", o.goal, ...(o.previous ? ["", "Where the previous run ended:", o.previous] : []), "", "The branch already has the previous runs' work checked out.", "", `New instructions from ${o.by}:`, o.message].join("\n");
96+}
97+
98+/** A transcript line, as stored. */
99+export type TranscriptLine = {
100+ seq: number;
101+ run: number;
102+ at: string;
103+ /** `out` and `err` are the agent's output; `setup` is installing and pushing; `note` is Diffinery's own. */
104+ stream: "out" | "err" | "setup" | "note";
105+ text: string;
106+};
107+
108+/** What the task page shows for a line. */
109+export interface TranscriptItem {
110+ kind: "text" | "tool" | "result" | "output" | "note" | "error";
111+ title?: string;
112+ body: string;
113+}
114+
115+const clip = (s: string, n = 4000) => (s.length > n ? `${s.slice(0, n)}\n… (${s.length - n} more characters)` : s);
116+
117+function str(v: unknown): string {
118+ if (typeof v === "string") return v;
119+ if (v === undefined || v === null) return "";
120+ return JSON.stringify(v, null, 2);
121+}
122+
123+/**
124+ * Read one line of Claude Code's `stream-json` or Codex's `--json` output into
125+ * something to show; other output is shown as is.
126+ */
127+export function transcriptItem(line: TranscriptLine): TranscriptItem[] {
128+ if (line.stream === "note") return [{ kind: "note", body: line.text }];
129+ if (line.stream === "setup") return [{ kind: "output", title: "setup", body: line.text }];
130+ if (line.stream === "err") return [{ kind: "error", body: line.text }];
131+ let j: Record<string, unknown>;
132+ try {
133+ const v = JSON.parse(line.text);
134+ if (!v || typeof v !== "object" || Array.isArray(v)) return [{ kind: "output", body: line.text }];
135+ j = v as Record<string, unknown>;
136+ } catch {
137+ return [{ kind: "output", body: line.text }];
138+ }
139+ const type = j.type;
140+ // Claude Code
141+ if (type === "system" && j.subtype === "init") return [{ kind: "note", body: `Claude Code started${j.model ? ` (${String(j.model)})` : ""}.` }];
142+ if (type === "assistant" || type === "user") {
143+ const content = ((j.message as { content?: unknown } | undefined)?.content ?? []) as Array<Record<string, unknown>>;
144+ if (!Array.isArray(content)) return [];
145+ const out: TranscriptItem[] = [];
146+ for (const b of content) {
147+ if (b.type === "text" && str(b.text).trim()) out.push({ kind: "text", body: str(b.text) });
148+ else if (b.type === "tool_use") out.push({ kind: "tool", title: str(b.name), body: clip(toolInput(b.input)) });
149+ else if (b.type === "tool_result") {
150+ const c = Array.isArray(b.content) ? (b.content as Array<Record<string, unknown>>).map((x) => str(x.text)).join("\n") : str(b.content);
151+ if (c.trim()) out.push({ kind: b.is_error ? "error" : "output", body: clip(c) });
152+ }
153+ }
154+ return out;
155+ }
156+ if (type === "result") {
157+ const cost = typeof j.total_cost_usd === "number" ? ` · $${j.total_cost_usd.toFixed(2)}` : "";
158+ const turns = typeof j.num_turns === "number" ? ` · ${j.num_turns} turns` : "";
159+ return [{ kind: "result", title: `${j.is_error ? "Ended with an error" : "Done"}${turns}${cost}`, body: str(j.result) }];
160+ }
161+ // Codex
162+ if (type === "thread.started") return [{ kind: "note", body: "Codex started." }];
163+ if (type === "item.completed" || type === "item.started") {
164+ const item = (j.item ?? {}) as Record<string, unknown>;
165+ if (type === "item.started") return [];
166+ if (item.type === "agent_message") return [{ kind: "text", body: str(item.text) }];
167+ if (item.type === "reasoning") return [{ kind: "note", body: str(item.text) }];
168+ if (item.type === "command_execution") return [{ kind: "tool", title: `$ ${str(item.command)}`, body: clip(str(item.aggregated_output)) + (typeof item.exit_code === "number" && item.exit_code !== 0 ? `\n(exit ${item.exit_code})` : "") }];
169+ if (item.type === "file_change") return [{ kind: "tool", title: "Edit files", body: ((item.changes ?? []) as Array<{ path?: string; kind?: string }>).map((c) => `${c.kind ?? "update"} ${c.path ?? ""}`).join("\n") }];
170+ if (item.type === "error") return [{ kind: "error", body: str(item.message) }];
171+ return [];
172+ }
173+ if (type === "turn.completed") return [];
174+ if (type === "turn.failed" || type === "error") return [{ kind: "error", body: str((j.error as { message?: string } | undefined)?.message ?? j.message) }];
175+ return [{ kind: "output", body: clip(line.text) }];
176+}
177+
178+function toolInput(input: unknown): string {
179+ if (!input || typeof input !== "object") return str(input);
180+ const i = input as Record<string, unknown>;
181+ if (typeof i.command === "string") return `$ ${i.command}`;
182+ if (typeof i.file_path === "string") return String(i.file_path);
183+ return str(input);
184+}
185+
186+/** What the runtime said last, to carry into a follow-up run. */
187+export function finalMessage(lines: TranscriptLine[]): string | null {
188+ for (let i = lines.length - 1; i >= 0; i--) {
189+ for (const item of transcriptItem(lines[i]!).reverse()) {
190+ if ((item.kind === "result" || item.kind === "text") && item.body.trim()) return clip(item.body, 6000);
191+ }
192+ }
193+ return null;
194+}
195+
196+/** A branch name for a task: `agent/<number>-<slug of the goal>`. */
197+export function taskBranch(number: number, goal: string): string {
198+ let slug = goal
199+ .toLowerCase()
200+ .replace(/[^a-z0-9]+/g, "-")
201+ .replace(/^-+|-+$/g, "");
202+ // At most 40 characters, cut between words.
203+ if (slug.length > 40) slug = slug[40] === "-" ? slug.slice(0, 40) : slug.slice(0, Math.max(slug.lastIndexOf("-", 40), 10));
204+ slug = slug.replace(/-+$/, "");
205+ return `agent/${number}${slug ? `-${slug}` : ""}`;
206+}
207+
208+/** A pull request title from the goal: its first line, shortened. */
209+export function taskTitle(goal: string): string {
210+ const first = goal.trim().split("\n")[0]!.trim();
211+ return first.length > 100 ? `${first.slice(0, 97)}…` : first || "Agent task";
212+}
addedapps/diffinery/src/agentic/tasks-web.tsx+425−0View file
@@ -1,0 +1,425 @@
1+/** @jsxImportSource hono/jsx */
2+import type { FC } from "hono/jsx";
3+import { requireAccount, type AppContext } from "../auth/principal.js";
4+import { loadRepo } from "../http/common.js";
5+import { HttpError } from "../lib/util.js";
6+import { SettingsNav } from "../web/account.js";
7+import { csrfToken } from "../web/csrf.js";
8+import { KCheckbox, KRadio, KSelect } from "../web/kumo.js";
9+import { Form, Markdown, Time, act, post, prm, render, repoUrl, statusOf, type WebApp } from "../web/ui.js";
10+import { isProvider, ModelKeys, PROVIDERS, type Provider } from "./keys.js";
11+import type { TaskRow, TaskView } from "./ProjectDO.js";
12+import { RUNTIMES, transcriptItem, type RuntimeId, type TranscriptItem } from "./runtimes.js";
13+import { drivableAgents, getTask, mayStop, messageTask, projectStub, requireTaskReader, retryTask, startTask, stopTask, taskNumber } from "./tasks.js";
14+
15+/**
16+ * Web pages for agent tasks: Settings → Model access, and a repository's
17+ * Tasks tab (list, new task, a task's live transcript with steer, stop and retry).
18+ */
19+
20+type Loaded = Awaited<ReturnType<typeof loadRepo>>;
21+
22+export const TASK_STATE: Record<TaskRow["state"], { label: string; tone: string }> = {
23+ running: { label: "Running", tone: "blue" },
24+ ready: { label: "Ready", tone: "green" },
25+ failed: { label: "Failed", tone: "red" },
26+ stopped: { label: "Stopped", tone: "" },
27+};
28+
29+export const TaskBadge: FC<{ state: TaskRow["state"] }> = ({ state }) => <span class={`badge ${TASK_STATE[state].tone}`}>{TASK_STATE[state].label}</span>;
30+
31+// ---------------------------------------------------------------- Settings → Model access
32+
33+const ModelAccessPage = async (c: AppContext) => {
34+ const me = requireAccount(c);
35+ const t = csrfToken(c);
36+ const keys = new Map((await new ModelKeys(c.env).list(me.id)).map((k) => [k.provider, k]));
37+ return render(
38+ c,
39+ { title: "Model access" },
40+ <div class="grid left">
41+ <SettingsNav me={me} tab="model-access" />
42+ <div>
43+ <h1>Model access</h1>
44+ <p class="muted">
45+ Agent tasks you start run Claude Code or Codex with your own API key. Diffinery keeps the key encrypted, never shows it again, and never puts it in a sandbox: the agent calls the provider through Diffinery, which adds your key. See <a href="/docs/agents/tasks">Agent tasks</a>.
46+ </p>
47+ <div class="flash info">
48+ Anything an agent runs in its sandbox, including code from the repository and its dependencies, can use your model access while the task runs, and could be tricked into misusing it. Use a <b>dedicated key with a spending limit</b> for Diffinery, and revoke it with the provider if you suspect misuse.
49+ </div>
50+ {(Object.keys(PROVIDERS) as Provider[]).map((id) => {
51+ const p = PROVIDERS[id];
52+ const k = keys.get(id);
53+ return (
54+ <div class="box">
55+ <div class="head">
56+ <h2>{p.name}</h2>
57+ {k ? <span class="badge green">Key saved</span> : <span class="badge">No key</span>}
58+ </div>
59+ <div class="body">
60+ {k ? (
61+ <div class="row mb">
62+ <div class="grow">
63+ <span class="mono">…{k.hint}</span>
64+ <span class="muted small">
65+ {" "}
66+ · added <Time at={k.createdAt} /> · updated <Time at={k.updatedAt} /> · terms accepted <Time at={k.optedInAt} />
67+ </span>
68+ </div>
69+ <Form action={`/settings/model-access/${id}/delete`} t={t} class="inline" confirm={`Delete your ${p.name} key? Running tasks that use it stop getting model access.`}>
70+ <button class="btn sm danger">Delete</button>
71+ </Form>
72+ </div>
73+ ) : null}
74+ <Form action={`/settings/model-access/${id}`} t={t}>
75+ <div class="field">
76+ <label for={`key-${id}`}>{k ? `Replace your ${p.name} API key` : `Your ${p.name} API key`}</label>
77+ <input id={`key-${id}`} name="key" type="password" autocomplete="off" required placeholder={id === "anthropic" ? "sk-ant-…" : "sk-…"} />
78+ <div class="hint">
79+ Create one at <a href={p.keysUrl} rel="noopener noreferrer" target="_blank">{p.keysUrl.replace("https://", "")}</a>.
80+ </div>
81+ </div>
82+ <div class="field">
83+ <KCheckbox
84+ name="accept"
85+ label={
86+ <>
87+ I use {p.name} through Diffinery under my own {p.name} account and its terms ({p.terms.map((x, i) => (
88+ <>
89+ {i ? ", " : ""}
90+ <a href={x.url} rel="noopener noreferrer" target="_blank">
91+ {x.label}
92+ </a>
93+ </>
94+ ))}
95+ ). Usage is billed to my account. Diffinery only provides the ability.
96+ </>
97+ }
98+ />
99+ </div>
100+ <button class="btn primary">{k ? "Replace key" : "Save key"}</button>
101+ </Form>
102+ </div>
103+ </div>
104+ );
105+ })}
106+ </div>
107+ </div>,
108+ );
109+};
110+
111+// ---------------------------------------------------------------- repository Tasks tab
112+
113+async function forTasks(c: AppContext): Promise<Loaded> {
114+ if (c.get("principal").type !== "account") throw new HttpError(401, "sign in required");
115+ const l = await loadRepo(c, prm(c, "owner"), prm(c, "repo"), "read");
116+ requireTaskReader(l);
117+ return l;
118+}
119+
120+const TaskList: FC<{ l: Loaded; tasks: TaskRow[] }> = ({ l, tasks }) => {
121+ const base = repoUrl(l.repo);
122+ return tasks.length ? (
123+ <ul class="list">
124+ {tasks.map((t) => (
125+ <li>
126+ <div class="grow">
127+ <a href={`${base}/tasks/${t.number}`}>
128+ <b>{t.title}</b>
129+ </a>{" "}
130+ <TaskBadge state={t.state} />
131+ <div class="muted small">
132+ #{t.number} · {RUNTIMES[t.runtime].name} as <a href={`/${t.agent_login}`}>{t.agent_login}</a> · started by <a href={`/${t.person_login}`}>{t.person_login}</a> <Time at={t.created_at} />
133+ {t.pr_number ? (
134+ <>
135+ {" "}
136+ · <a href={`${base}/pull/${t.pr_number}`}>#{t.pr_number}</a>
137+ </>
138+ ) : null}
139+ </div>
140+ </div>
141+ <span class="muted small">
142+ <Time at={t.updated_at} />
143+ </span>
144+ </li>
145+ ))}
146+ </ul>
147+ ) : (
148+ <div class="empty">
149+ No agent tasks yet. Give one of your agents a goal; it works in a sandbox and opens a pull request for review.
150+ </div>
151+ );
152+};
153+
154+const ItemView: FC<{ item: TranscriptItem; base: string }> = ({ item, base }) => {
155+ if (item.kind === "text") return <div class="tx-text"><Markdown text={item.body} /></div>;
156+ if (item.kind === "result")
157+ return (
158+ <div class="tx-result">
159+ <div class="small muted">{item.title}</div>
160+ {item.body ? <Markdown text={item.body} /> : null}
161+ </div>
162+ );
163+ if (item.kind === "note") return <div class="tx-note muted small">{item.body}</div>;
164+ if (item.kind === "tool")
165+ return (
166+ <details class="tx-tool">
167+ <summary class="mono small">{item.title}</summary>
168+ <pre class="tx-pre">{item.body}</pre>
169+ </details>
170+ );
171+ return <pre class={`tx-pre${item.kind === "error" ? " tx-err" : ""}`}>{item.body}</pre>;
172+};
173+
174+function taskPage(c: AppContext, l: Loaded, t: TaskView, items: Array<{ run: number; item: TranscriptItem }>, truncated: boolean) {
175+ const base = repoUrl(l.repo);
176+ const tk = csrfToken(c);
177+ const me = requireAccount(c);
178+ const starter = me.id === t.person_id;
179+ const running = t.state === "running";
180+ return render(
181+ c,
182+ { title: `${t.title} · Task #${t.number}`, repo: { repo: l.repo, caller: l.caller, tab: "tasks" }, refresh: running ? 5 : undefined },
183+ <div>
184+ <div class="row mb" style="align-items:flex-start">
185+ <div class="grow">
186+ <h1 class="m0">
187+ {t.title} <span class="faint">#{t.number}</span>
188+ </h1>
189+ <div class="muted small mt">
190+ <TaskBadge state={t.state} /> {RUNTIMES[t.runtime].name} as <a href={`/${t.agent_login}`}>{t.agent_login}</a>, started by <a href={`/${t.person_login}`}>{t.person_login}</a> <Time at={t.created_at} /> · branch <span class="mono">{t.branch}</span> from <span class="mono">{t.base}</span>
191+ {t.pr_number ? (
192+ <>
193+ {" "}
194+ · pull request <a href={`${base}/pull/${t.pr_number}`}>#{t.pr_number}</a>
195+ </>
196+ ) : null}
197+ </div>
198+ </div>
199+ {running && mayStop(l.caller, t) ? (
200+ <Form action={`${base}/tasks/${t.number}/stop`} t={tk} class="inline" confirm="Stop this task? The agent is stopped and its sandbox closed; work it already pushed stays.">
201+ <button class="btn sm danger">Stop</button>
202+ </Form>
203+ ) : null}
204+ {!running && starter ? (
205+ <Form action={`${base}/tasks/${t.number}/retry`} t={tk} class="inline">
206+ <button class="btn sm">Retry</button>
207+ </Form>
208+ ) : null}
209+ </div>
210+ <div class="box">
211+ <div class="head">
212+ <h2>Goal</h2>
213+ </div>
214+ <div class="body"><Markdown text={t.goal} /></div>
215+ </div>
216+ {t.error ? <div class="flash error">{t.error}</div> : null}
217+ {t.queued ? <div class="flash info">A message is waiting for the current run to end: {t.queued}</div> : null}
218+ <div class="box">
219+ <div class="head">
220+ <h2>Transcript</h2>
221+ {running ? <span class="badge blue">Live</span> : null}
222+ </div>
223+ <div class="body tx">
224+ {truncated ? (
225+ <p class="muted small">
226+ Showing the latest lines. The whole transcript is at <a href={`/api/v3/repos/${l.repo.owner_login}/${l.repo.name}/diffinery/tasks/${t.number}/transcript`}>the API</a>.
227+ </p>
228+ ) : null}
229+ {items.length ? items.map((x, i) => (
230+ <>
231+ {i === 0 || items[i - 1]!.run !== x.run ? <h4 class="tx-run">Run {x.run}{t.runs[x.run - 1]?.message ? " · follow-up" : ""}</h4> : null}
232+ <ItemView item={x.item} base={base} />
233+ </>
234+ )) : <div class="empty">Starting…</div>}
235+ </div>
236+ </div>
237+ {starter ? (
238+ <Form action={`${base}/tasks/${t.number}/messages`} t={tk} class="box">
239+ <div class="head">
240+ <h2>{running ? "Send a follow-up" : "Continue the task"}</h2>
241+ </div>
242+ <div class="body">
243+ <div class="field">
244+ <textarea name="body" rows={4} required maxlength={20000} placeholder={running ? "It starts a new run on the same branch when this run ends." : "What should the agent do next? It continues on the same branch and pull request."} />
245+ </div>
246+ <button class="btn primary">Send</button>
247+ </div>
248+ </Form>
249+ ) : (
250+ <p class="muted small">Only {t.person_login}, who started this task, can steer or retry it: each run spends their own model key.</p>
251+ )}
252+ </div>,
253+ );
254+}
255+
256+async function newTaskPage(c: AppContext, l: Loaded, error?: string, v: { goal?: string; runtime?: string; agent?: string; base?: string; command?: string } = {}) {
257+ const me = requireAccount(c);
258+ const base = repoUrl(l.repo);
259+ const tk = csrfToken(c);
260+ const [agents, keys, branches] = await Promise.all([drivableAgents(c, me, l), new ModelKeys(c.env).providers(me.id), l.stub.refs("refs/heads/")]);
261+ const runtimes = (Object.keys(RUNTIMES) as RuntimeId[]).map((id) => {
262+ const r = RUNTIMES[id];
263+ const missing = r.needs.filter((p) => !keys.has(p));
264+ return { value: id, label: r.name, description: missing.length ? `${r.description} Add your ${missing.map((p) => PROVIDERS[p].name).join(" and ")} key in Settings → Model access first.` : r.description, disabled: missing.length > 0 };
265+ });
266+ const firstUsable = runtimes.find((r) => !r.disabled)?.value ?? "custom";
267+ return render(
268+ c,
269+ { title: "New agent task", repo: { repo: l.repo, caller: l.caller, tab: "tasks" }, status: error ? 422 : undefined },
270+ <div style="max-width:780px">
271+ <h1>New agent task</h1>
272+ <p class="muted">
273+ The agent works in a fresh sandbox with this repository checked out, pushes to a new branch as itself and opens a pull request, which needs a person's approval like any agent pull request. The model calls use <a href="/settings/model-access">your own key</a>.
274+ </p>
275+ {error ? <div class="flash error">{error}</div> : null}
276+ {!agents.length ? (
277+ <div class="flash info">
278+ You don't have an agent with write access here yet. <a href="/settings/agents">Create an agent</a>, then give it write access to this repository (or let it inherit yours).
279+ </div>
280+ ) : null}
281+ <Form action={`${base}/tasks`} t={tk} class="box">
282+ <div class="body">
283+ <div class="field">
284+ <label for="goal">Goal</label>
285+ <textarea id="goal" name="goal" rows={8} required maxlength={20000} placeholder="Fix the Contact sales link on the pricing page so it goes to /contact, and add a test.">
286+ {v.goal ?? ""}
287+ </textarea>
288+ <div class="hint">What you want done, as you would ask a colleague. Markdown works.</div>
289+ </div>
290+ <div class="field">
291+ <label>Runtime</label>
292+ <KRadio name="runtime" label="Runtime" value={v.runtime ?? firstUsable} items={runtimes} />
293+ </div>
294+ <div class="field">
295+ <label for="command">
296+ Command <span class="faint">(custom runtime only)</span>
297+ </label>
298+ <input id="command" name="command" type="text" maxlength={2000} value={v.command ?? ""} placeholder='aider --yes --message "$DIFFINERY_TASK_PROMPT"' />
299+ <div class="hint">Runs in the repository with the goal in $DIFFINERY_TASK_PROMPT, and ANTHROPIC_BASE_URL / OPENAI_BASE_URL set for the providers you have keys for.</div>
300+ </div>
301+ <div class="row" style="align-items:flex-start">
302+ <div class="field grow">
303+ <label for="agent">Agent</label>
304+ <KSelect id="agent" name="agent" block label="Agent" value={v.agent ?? agents[0]?.login} options={agents.map((a) => ({ value: a.login, label: a.name ? `${a.login} (${a.name})` : a.login }))} placeholder="Choose an agent" />
305+ </div>
306+ <div class="field grow">
307+ <label for="base">Base branch</label>
308+ <KSelect id="base" name="base" block label="Base branch" value={v.base ?? l.repo.default_branch} options={branches.slice(0, 500).map((b) => ({ value: b.name.slice("refs/heads/".length), label: b.name.slice("refs/heads/".length) }))} />
309+ </div>
310+ </div>
311+ <button class="btn primary" disabled={!agents.length}>
312+ Start task
313+ </button>
314+ </div>
315+ </Form>
316+ </div>,
317+ );
318+}
319+
320+export function registerTasks(web: WebApp): void {
321+ web.get("/settings/model-access", (c) => ModelAccessPage(c));
322+
323+ post(web, "/settings/model-access/:provider", async (c, f) => {
324+ const me = requireAccount(c);
325+ const provider = prm(c, "provider");
326+ if (!isProvider(provider)) throw new HttpError(404, "not found");
327+ return act(
328+ c,
329+ "/settings/model-access",
330+ async () => {
331+ const k = await new ModelKeys(c.env).set(me, provider, f.s("key"), f.all("accept").length > 0);
332+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "model_key.set", target: provider, detail: { hint: k.hint, opted_in_at: k.optedInAt } }));
333+ },
334+ `${PROVIDERS[provider].name} key saved.`,
335+ );
336+ });
337+
338+ post(web, "/settings/model-access/:provider/delete", async (c) => {
339+ const me = requireAccount(c);
340+ const provider = prm(c, "provider");
341+ if (!isProvider(provider)) throw new HttpError(404, "not found");
342+ return act(
343+ c,
344+ "/settings/model-access",
345+ async () => {
346+ await new ModelKeys(c.env).delete(me.id, provider);
347+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: me.id, actorLogin: me.login, action: "model_key.delete", target: provider }));
348+ },
349+ `${PROVIDERS[provider].name} key deleted.`,
350+ );
351+ });
352+
353+ web.get("/:owner/:repo/tasks", async (c) => {
354+ const l = await forTasks(c);
355+ const tasks = await projectStub(c.env, l.repo.id).list({ limit: 100 });
356+ return render(
357+ c,
358+ { title: `Tasks · ${l.repo.owner_login}/${l.repo.name}`, repo: { repo: l.repo, caller: l.caller, tab: "tasks" }, refresh: tasks.some((t) => t.state === "running") ? 10 : undefined },
359+ <div>
360+ <div class="row mb">
361+ <h1 class="grow m0">Agent tasks</h1>
362+ {l.repo.archived ? null : (
363+ <a class="btn primary" href={`${repoUrl(l.repo)}/tasks/new`}>
364+ New task
365+ </a>
366+ )}
367+ </div>
368+ <div class="box">
369+ <TaskList l={l} tasks={tasks} />
370+ </div>
371+ </div>,
372+ );
373+ });
374+
375+ web.get("/:owner/:repo/tasks/new", async (c) => newTaskPage(c, await forTasks(c)));
376+
377+ post(web, "/:owner/:repo/tasks", async (c, f) => {
378+ const l = await forTasks(c);
379+ const v = { goal: f.s("goal"), runtime: f.s("runtime") || "claude-code", agent: f.s("agent"), base: f.s("base"), command: f.s("command") };
380+ try {
381+ const t = await startTask(c, l, v);
382+ return c.redirect(`${repoUrl(l.repo)}/tasks/${t.number}`, 303);
383+ } catch (e) {
384+ const st = statusOf(e);
385+ if (st.status >= 500) throw e;
386+ return newTaskPage(c, l, st.message, v);
387+ }
388+ });
389+
390+ web.get("/:owner/:repo/tasks/:number", async (c) => {
391+ const l = await forTasks(c);
392+ const n = taskNumber(c.req.param("number"));
393+ const t = await getTask(c, l, n);
394+ const stub = projectStub(c.env, l.repo.id);
395+ // The latest lines: page backwards from the end would need an index; fetch the head and keep the tail.
396+ const lines = await stub.lines(n, 0, 5000);
397+ const tail = lines.slice(-1500);
398+ const items = tail.flatMap((line) => transcriptItem(line).map((item) => ({ run: line.run, item })));
399+ return taskPage(c, l, t, items, lines.length > tail.length);
400+ });
401+
402+ post(web, "/:owner/:repo/tasks/:number/messages", async (c, f) => {
403+ const l = await forTasks(c);
404+ const n = taskNumber(c.req.param("number"));
405+ return act(c, `${repoUrl(l.repo)}/tasks/${n}`, async () => {
406+ await messageTask(c, l, n, f.s("body"));
407+ });
408+ });
409+
410+ post(web, "/:owner/:repo/tasks/:number/retry", async (c) => {
411+ const l = await forTasks(c);
412+ const n = taskNumber(c.req.param("number"));
413+ return act(c, `${repoUrl(l.repo)}/tasks/${n}`, async () => {
414+ await retryTask(c, l, n);
415+ }, "Retrying.");
416+ });
417+
418+ post(web, "/:owner/:repo/tasks/:number/stop", async (c) => {
419+ const l = await forTasks(c);
420+ const n = taskNumber(c.req.param("number"));
421+ return act(c, `${repoUrl(l.repo)}/tasks/${n}`, async () => {
422+ await stopTask(c, l, n);
423+ }, "Task stopped.");
424+ });
425+}
addedapps/diffinery/src/agentic/tasks.ts+238−0View file
@@ -1,0 +1,238 @@
1+import { requireAccount, type AppContext } from "../auth/principal.js";
2+import type { Env } from "../env.js";
3+import type { loadRepo } from "../http/common.js";
4+import type { Account, Directory } from "../lib/directory.js";
5+import { HttpError } from "../lib/util.js";
6+import { atLeast, type Caller } from "../repo/types.js";
7+import { mayDriveAgent, taskCaller, TaskAccessError } from "./access.js";
8+import { ModelKeys, PROVIDERS, type Provider } from "./keys.js";
9+import type { TaskRow, TaskView } from "./ProjectDO.js";
10+import { isRuntime, MAX_COMMAND, MAX_GOAL, RUNTIMES, type RuntimeId, type TranscriptLine } from "./runtimes.js";
11+
12+/**
13+ * Who may do what with agent tasks. Shared by the REST API, MCP and the web.
14+ *
15+ * - Reading tasks and transcripts: write access to the repository or more.
16+ * - Starting, sending a message and retrying: only a person, in a browser
17+ * session (each run spends their model key), and for an existing task only
18+ * the person who started it. No token, agent, job or sandbox can.
19+ * - Stopping: the person who started it, its agent, or a repository admin.
20+ */
21+
22+type Loaded = Awaited<ReturnType<typeof loadRepo>>;
23+
24+export const projectStub = (env: Env, repoId: string) => env.PROJECT.get(env.PROJECT.idFromName(repoId));
25+
26+/** Tasks are visible with write access or more: transcripts show an agent's reasoning and command output. */
27+export function requireTaskReader(l: Loaded): void {
28+ if (!atLeast(l.caller.role, "write")) throw new HttpError(404, "not found");
29+}
30+
31+/** A person signed in in a browser (or the local dev sign-in): the only one who can spend their model key. */
32+export function requirePersonSession(c: AppContext): Account {
33+ const p = c.get("principal");
34+ if (p.type === "anonymous") throw new HttpError(401, "sign in required");
35+ if (p.type !== "account" || (p.via !== "session" && p.via !== "dev")) throw new HttpError(403, "starting or steering an agent task needs a browser session: each run spends your own model key, so no token can do it");
36+ const me = requireAccount(c);
37+ if (me.kind !== "user") throw new HttpError(403, "only people can start agent tasks");
38+ return me;
39+}
40+
41+export function taskNumber(s: string | undefined): number {
42+ const n = Number(s);
43+ if (!Number.isInteger(n) || n <= 0) throw new HttpError(404, "task not found");
44+ return n;
45+}
46+
47+export async function getTask(c: AppContext, l: Loaded, number: number): Promise<TaskView> {
48+ requireTaskReader(l);
49+ const t = await projectStub(c.env, l.repo.id).get(number);
50+ if (!t) throw new HttpError(404, "task not found");
51+ return t;
52+}
53+
54+/** The providers a run gets, checked against the runtime's needs and the person's keys. */
55+async function providersFor(env: Env, person: Account, runtime: RuntimeId): Promise<Provider[]> {
56+ const have = await new ModelKeys(env).providers(person.id);
57+ for (const p of RUNTIMES[runtime].needs) {
58+ if (!have.has(p)) throw new HttpError(422, `${RUNTIMES[runtime].name} needs your ${PROVIDERS[p].name} API key: add it in Settings → Model access`);
59+ }
60+ return runtime === "custom" ? [...have] : RUNTIMES[runtime].needs;
61+}
62+
63+async function asTaskCaller(c: AppContext, person: Account, agent: Account, l: Loaded): Promise<Caller> {
64+ try {
65+ return await taskCaller(c.get("dir"), person, agent, l.repo);
66+ } catch (e) {
67+ if (e instanceof TaskAccessError) throw new HttpError(403, e.message);
68+ throw e;
69+ }
70+}
71+
72+export interface StartInput {
73+ goal?: unknown;
74+ runtime?: unknown;
75+ agent?: unknown;
76+ base?: unknown;
77+ command?: unknown;
78+}
79+
80+export async function startTask(c: AppContext, l: Loaded, input: StartInput): Promise<TaskView> {
81+ const person = requirePersonSession(c);
82+ if (l.repo.archived) throw new HttpError(403, "repository is archived (read-only)");
83+ const goal = typeof input.goal === "string" ? input.goal.trim() : "";
84+ if (!goal) throw new HttpError(422, "`goal` is required");
85+ if (goal.length > MAX_GOAL) throw new HttpError(422, `\`goal\` is longer than ${MAX_GOAL} characters`);
86+ const runtime = typeof input.runtime === "string" ? input.runtime : "claude-code";
87+ if (!isRuntime(runtime)) throw new HttpError(422, "`runtime` must be claude-code, codex or custom");
88+ const command = runtime === "custom" ? (typeof input.command === "string" ? input.command.trim() : "") : null;
89+ if (runtime === "custom" && !command) throw new HttpError(422, "`command` is required for the custom runtime");
90+ if (command && command.length > MAX_COMMAND) throw new HttpError(422, `\`command\` is longer than ${MAX_COMMAND} characters`);
91+ const agentLogin = typeof input.agent === "string" ? input.agent.trim() : "";
92+ const agent = agentLogin ? await c.get("dir").accountByLogin(agentLogin) : null;
93+ if (!agent || agent.kind !== "agent") throw new HttpError(422, "`agent` must be the login of one of your agents");
94+ await asTaskCaller(c, person, agent, l);
95+ const base = typeof input.base === "string" && input.base.trim() ? input.base.trim() : l.repo.default_branch;
96+ if (!(await l.stub.getRef(`refs/heads/${base}`))) throw new HttpError(422, `branch ${base} doesn't exist`);
97+ const providers = await providersFor(c.env, person, runtime);
98+ const t = await projectStub(c.env, l.repo.id).create({ repoId: l.repo.id, goal, runtime, command, agent: { id: agent.id, login: agent.login }, person: { id: person.id, login: person.login }, base, providers });
99+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.start", target: `${l.repo.owner_login}/${l.repo.name}#${t.number}`, detail: { agent: agent.login, runtime } }));
100+ return t;
101+}
102+
103+/** The person who started the task, in a browser session, still allowed to drive its agent here. */
104+async function starter(c: AppContext, l: Loaded, number: number): Promise<{ person: Account; task: TaskView }> {
105+ const person = requirePersonSession(c);
106+ const task = await getTask(c, l, number);
107+ if (task.person_id !== person.id) throw new HttpError(403, `only ${task.person_login}, who started this task, can steer or retry it: each run spends their model key`);
108+ if (l.repo.archived) throw new HttpError(403, "repository is archived (read-only)");
109+ const agent = await c.get("dir").account(task.agent_id);
110+ if (!agent) throw new HttpError(403, "the task's agent no longer exists");
111+ await asTaskCaller(c, person, agent, l);
112+ return { person, task };
113+}
114+
115+export async function messageTask(c: AppContext, l: Loaded, number: number, text: unknown): Promise<TaskView> {
116+ const body = typeof text === "string" ? text.trim() : "";
117+ if (!body) throw new HttpError(422, "`body` is required");
118+ if (body.length > MAX_GOAL) throw new HttpError(422, `\`body\` is longer than ${MAX_GOAL} characters`);
119+ const { person, task } = await starter(c, l, number);
120+ const t = await projectStub(c.env, l.repo.id).message(number, body, await providersFor(c.env, person, task.runtime));
121+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.message", target: `${l.repo.owner_login}/${l.repo.name}#${number}` }));
122+ return t;
123+}
124+
125+export async function retryTask(c: AppContext, l: Loaded, number: number): Promise<TaskView> {
126+ const { person, task } = await starter(c, l, number);
127+ const t = await projectStub(c.env, l.repo.id).retry(number, await providersFor(c.env, person, task.runtime));
128+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: person.id, actorLogin: person.login, action: "task.retry", target: `${l.repo.owner_login}/${l.repo.name}#${number}` }));
129+ return t;
130+}
131+
132+/** Whether `caller` can stop the task: its person, its agent, or a repository admin. */
133+export const mayStop = (caller: Caller, t: TaskRow) => caller.id === t.person_id || caller.id === t.agent_id || atLeast(caller.role, "admin");
134+
135+export async function stopTask(c: AppContext, l: Loaded, number: number): Promise<TaskView> {
136+ const task = await getTask(c, l, number);
137+ if (!mayStop(l.caller, task)) throw new HttpError(403, "only the person who started the task, its agent or a repository admin can stop it");
138+ const t = await projectStub(c.env, l.repo.id).stop(number, l.caller.login);
139+ c.executionCtx.waitUntil(c.get("dir").audit({ actorId: l.caller.id, actorLogin: l.caller.login, onBehalfOf: l.caller.onBehalfOf, action: "task.stop", target: `${l.repo.owner_login}/${l.repo.name}#${number}` }));
140+ return t;
141+}
142+
143+/** Agents a person can run tasks as, here: ones they manage that have write access to the repository. */
144+export async function drivableAgents(c: AppContext, person: Account, l: Loaded): Promise<Account[]> {
145+ const dir = c.get("dir");
146+ const orgs = (await dir.orgsFor(person.id)).filter((o) => o.role === "owner" || o.role === "admin").map((o) => o.id);
147+ const agents = await dir.agentsOwnedBy([person.id, ...orgs]);
148+ const out: Account[] = [];
149+ for (const a of agents) {
150+ if (!(await mayDriveAgent(dir, person, a))) continue;
151+ if (atLeast(await dir.roleFor(a, l.repo), "write")) out.push(a);
152+ }
153+ return out;
154+}
155+
156+// ---------------------------------------------------------------- JSON
157+
158+export function taskJson(base: string, repoFull: string, t: TaskRow | TaskView) {
159+ return {
160+ number: t.number,
161+ title: t.title,
162+ goal: t.goal,
163+ state: t.state,
164+ runtime: t.runtime,
165+ ...(t.command ? { command: t.command } : {}),
166+ agent: { login: t.agent_login, type: "Bot" },
167+ started_by: { login: t.person_login, type: "User" },
168+ base: t.base,
169+ branch: t.branch,
170+ pull_request: t.pr_number ? { number: t.pr_number, html_url: `${base}/${repoFull}/pull/${t.pr_number}` } : null,
171+ queued_message: t.queued ?? null,
172+ error: t.error,
173+ html_url: `${base}/${repoFull}/tasks/${t.number}`,
174+ created_at: t.created_at,
175+ updated_at: t.updated_at,
176+ ...("runs" in t ? { runs: t.runs.map((r) => ({ number: r.seq, kind: r.kind, message: r.message, state: r.step, exit_code: r.exit_code, error: r.error, started_at: r.started_at, ended_at: r.ended_at })) } : {}),
177+ };
178+}
179+
180+export const lineJson = (l: TranscriptLine) => ({ seq: l.seq, run: l.run, at: l.at, stream: l.stream, text: l.text });
181+
182+export interface MyTask {
183+ repository: string;
184+ repoId: string;
185+ number: number;
186+ title: string;
187+ runtime: RuntimeId;
188+ state: TaskRow["state"];
189+ agent: string;
190+ prNumber: number | null;
191+ createdAt: string;
192+ updatedAt: string;
193+}
194+
195+/**
196+ * Tasks a person started (or, for an agent, tasks it runs), newest activity
197+ * first, from the D1 index, in repositories where they still have write access.
198+ */
199+export async function tasksFor(env: Env, dir: Directory, me: Account, opts: { repoIds?: string[] | null; since?: string; activeOrSince?: boolean; limit?: number } = {}): Promise<MyTask[]> {
200+ const col = me.kind === "agent" ? "t.agent_id" : "t.started_by";
201+ const since = opts.since ?? "";
202+ const r = await env.DB.prepare(
203+ `SELECT t.repo_id, t.number, t.title, t.runtime, t.state, t.pr_number, t.created_at, t.updated_at, o.login AS owner_login, r.name AS repo_name, ag.login AS agent_login
204+ FROM agent_tasks t JOIN repos r ON r.id = t.repo_id JOIN accounts o ON o.id = r.owner_id JOIN accounts ag ON ag.id = t.agent_id
205+ WHERE ${col} = ? AND (t.updated_at >= ?${opts.activeOrSince ? " OR t.state = 'running'" : ""}) ORDER BY t.updated_at DESC LIMIT ?`,
206+ )
207+ .bind(me.id, since, Math.min(opts.limit ?? 50, 100))
208+ .all<{ repo_id: string; number: number; title: string; runtime: RuntimeId; state: TaskRow["state"]; pr_number: number | null; created_at: string; updated_at: string; owner_login: string; repo_name: string; agent_login: string }>();
209+ const allowed = new Map<string, boolean>();
210+ const out: MyTask[] = [];
211+ for (const t of r.results) {
212+ if (opts.repoIds && !opts.repoIds.includes(t.repo_id)) continue;
213+ if (!allowed.has(t.repo_id)) {
214+ const repo = await dir.repoById(t.repo_id);
215+ allowed.set(t.repo_id, !!repo && atLeast(await dir.roleFor(me, repo), "write"));
216+ }
217+ if (!allowed.get(t.repo_id)) continue;
218+ out.push({ repository: `${t.owner_login}/${t.repo_name}`, repoId: t.repo_id, number: t.number, title: t.title, runtime: t.runtime, state: t.state, agent: t.agent_login, prNumber: t.pr_number, createdAt: t.created_at, updatedAt: t.updated_at });
219+ }
220+ return out;
221+}
222+
223+export const myTaskJson = (base: string, t: MyTask) => ({
224+ repository: t.repository,
225+ number: t.number,
226+ title: t.title,
227+ state: t.state,
228+ runtime: t.runtime,
229+ agent: { login: t.agent, type: "Bot" },
230+ pull_request: t.prNumber ? { number: t.prNumber, html_url: `${base}/${t.repository}/pull/${t.prNumber}` } : null,
231+ html_url: `${base}/${t.repository}/tasks/${t.number}`,
232+ created_at: t.createdAt,
233+ updated_at: t.updatedAt,
234+});
235+
236+export async function myTasks(c: AppContext, me: Account, repoIds: string[] | null) {
237+ return (await tasksFor(c.env, c.get("dir"), me, { repoIds })).map((t) => myTaskJson(c.env.PUBLIC_URL.replace(/\/$/, ""), t));
238+}
modifiedapps/diffinery/src/agentic/today.ts+10−1View file
@@ -1,4 +1,5 @@
11 import type { DiffineryEvent } from "@diffinery/shared";
2+import { myTaskJson, tasksFor, type MyTask } from "./tasks.js";
23 import type { Env } from "../env.js";
34 import type { Account, Directory, Repo } from "../lib/directory.js";
45 import { repoStub } from "../http/common.js";
@@ -57,6 +58,8 @@
5758 landed: Array<{ repo: TodayRepo; pull: TodayMerged }>;
5859 releases: Array<{ repo: TodayRepo; release: TodayRelease }>;
5960 failures: Array<{ repo: TodayRepo; run: TodayRun }>;
61+ /** Agent tasks you started (an agent: the tasks it runs) that are running or changed since `since`. */
62+ tasks: MyTask[];
6063 }
6164
6265 export interface ClassifyInput {
@@ -89,7 +92,7 @@
8992
9093 const waitsForPerson = (p: TodayPull) => ["waiting_for_approval", "checks_running"].includes(agentPullState(p));
9194
92-export function classify(input: ClassifyInput): Omit<Today, "generatedAt" | "since" | "repositories" | "incomplete"> {
95+export function classify(input: ClassifyInput): Omit<Today, "generatedAt" | "since" | "repositories" | "incomplete" | "tasks"> {
9396 const me = input.me.login.toLowerCase();
9497 const needsYou: NeedsItem[] = [];
9598 const agentsAtWork: AgentItem[] = [];
@@ -158,6 +161,10 @@
158161 const agentIds = new Set((await dir.agentsOwnedBy(owners)).map((a) => a.id));
159162 if (me.kind === "agent") agentIds.add(me.id);
160163
164+ const tasks = tasksFor(env, dir, me, { repoIds: opts.repoIds, since, activeOrSince: true, limit: 20 }).catch((e) => {
165+ console.error("today: tasks failed", e);
166+ return [] as MyTask[];
167+ });
161168 const settled = await Promise.allSettled(repos.map(async (r) => ({ repo: repoRef(r), data: await repoStub(env, r.id).todaySummary(since) })));
162169 const answered = settled.flatMap((s, i) => {
163170 if (s.status === "fulfilled") return [s.value];
@@ -180,6 +187,7 @@
180187 repositories: answered.length,
181188 incomplete: skipped > 0 || answered.length < repos.length,
182189 ...classify({ me, agentIds, writable, repos: answered }),
190+ tasks: await tasks,
183191 };
184192 }
185193
@@ -233,6 +241,7 @@
233241 agents_at_work: t.agentsAtWork.map((x) => ({ state: x.state, ...pullJson(base, x.repo, x.pull) })),
234242 landed: t.landed.map((x) => ({ repository: x.repo.fullName, number: x.pull.number, title: x.pull.title, html_url: `${base}${x.repo.url}/pull/${x.pull.number}`, user: person(x.pull.author), merged_by: x.pull.mergedBy, merged_at: x.pull.mergedAt })),
235243 releases: t.releases.map((x) => ({ repository: x.repo.fullName, tag_name: x.release.tag, name: x.release.name, prerelease: x.release.prerelease, html_url: `${base}${x.repo.url}/releases/tag/${encodeURIComponent(x.release.tag)}`, author: person(x.release.author), published_at: x.release.publishedAt })),
244+ tasks: t.tasks.map((x) => myTaskJson(base, x)),
236245 failures: t.failures.map((x) => ({ repository: x.repo.fullName, run_id: x.run.id, run_number: x.run.runNumber, workflow: x.run.workflow, title: x.run.title, ref: x.run.ref, event: x.run.event, conclusion: x.run.conclusion, html_url: `${base}${x.repo.url}/actions/runs/${x.run.id}`, actor: person(x.run.actor), pull_request: x.run.prNumber, completed_at: x.run.at })),
237246 };
238247 }
modifiedapps/diffinery/src/agentic/web.tsx+42−0View file
@@ -5,6 +5,7 @@
55 import { Actor, Status, Time } from "../web/ui.js";
66 import type { AgentItem, AgentPullState, NeedsItem, NeedsKind, Today } from "./today.js";
77 import type { TodayPull } from "./today-repo.js";
8+import type { MyTask } from "./tasks.js";
89
910 /** The Today view's sections on the dashboard (web/account.tsx): what needs you, your agents, and the last day. */
1011
@@ -82,11 +83,52 @@
8283 </div>
8384 <NeedsYou items={today.needsYou} />
8485 <Agents items={today.agentsAtWork} />
86+ {today.tasks.length ? <Tasks items={today.tasks} /> : null}
8587 <SinceYesterday today={today} />
8688 </div>
8789 );
8890 };
8991
92+const TASK_TEXT: Record<MyTask["state"], { label: string; tone: string }> = {
93+ running: { label: "Running", tone: "blue" },
94+ ready: { label: "Ready", tone: "green" },
95+ failed: { label: "Failed", tone: "red" },
96+ stopped: { label: "Stopped", tone: "" },
97+};
98+
99+const Tasks: FC<{ items: MyTask[] }> = ({ items }) => (
100+ <div class="box" id="tasks">
101+ <div class="head">
102+ <h2>{raw(ICONS.zap)} Your agent tasks</h2>
103+ <span class="muted small">Running, or changed since yesterday</span>
104+ </div>
105+ <ul class="list">
106+ {items.map((t) => (
107+ <li>
108+ <div class="grow">
109+ <a href={`/${t.repository}/tasks/${t.number}`}>
110+ <b>{t.title}</b>
111+ </a>{" "}
112+ <span class={`badge ${TASK_TEXT[t.state].tone}`}>{TASK_TEXT[t.state].label}</span>
113+ <div class="muted small">
114+ {t.repository} · task #{t.number} · {t.agent}
115+ {t.prNumber ? (
116+ <>
117+ {" "}
118+ · <a href={`/${t.repository}/pull/${t.prNumber}`}>#{t.prNumber}</a>
119+ </>
120+ ) : null}
121+ </div>
122+ </div>
123+ <span class="muted small">
124+ <Time at={t.updatedAt} />
125+ </span>
126+ </li>
127+ ))}
128+ </ul>
129+ </div>
130+);
131+
90132 const NeedsYou: FC<{ items: NeedsItem[] }> = ({ items }) => (
91133 <div class="box" id="needs-you">
92134 <div class="head">
modifiedapps/diffinery/src/app.ts+4−0View file
@@ -27,6 +27,7 @@
2727 import { mailCron, notifyEvent } from "./mail/notify.js";
2828 import { mailQueue } from "./mail/send.js";
2929 import { recordActivity } from "./agentic/today.js";
30+import { modelProxy } from "./agentic/proxy.js";
3031 import type { MailJob } from "./mail/types.js";
3132
3233 export const app = new Hono<{ Bindings: Env; Variables: AppVars }>();
@@ -34,6 +35,9 @@
3435 app.onError((e) => errorResponse(e));
3536 app.notFound((c) => (c.req.path.startsWith("/api/") ? apiNotFound() : c.text("404 Not Found", 404)));
3637
38+// The model proxy for agent tasks: its own task-scoped token, not an account credential (agentic/proxy.ts).
39+app.all("/-/model/:provider/*", (c) => modelProxy(c));
40+
3741 // Security headers on every response, then identity: tokens, Access JWT, or anonymous.
3842 app.use("*", async (c, next) => {
3943 // Browser sessions ride on cookies, so refuse state-changing requests that come from another site.
modifiedapps/diffinery/src/auth/principal.ts+4−0View file
@@ -5,6 +5,7 @@
55 import { HttpError, fromB64 } from "../lib/util.js";
66 import type { Caller, Role } from "../repo/types.js";
77 import { ROLE_LEVEL } from "../repo/types.js";
8+import { taskPersonAllowed } from "../agentic/access.js";
89
910 export type Principal =
1011 | { type: "account"; account: Account; via: "session" | "token" | "dev"; scopes: TokenScope[]; repoIds: string[] | null; sessionId?: string; importId?: string; tokenId?: string }
@@ -164,6 +165,9 @@
164165 const account = await dir.account(auth.caller.id);
165166 const now = account?.status === "active" ? await dir.roleFor(account, repo) : null;
166167 if (!now) return null;
168+ // An agent task's sandbox (agentic/) also stops working once the person who started the task
169+ // is disabled, loses write access or can no longer drive the agent.
170+ if (auth.taskPerson && !(await taskPersonAllowed(dir, auth.taskPerson, account!, repo))) return null;
167171 let cap: Role = ROLE_LEVEL[auth.caller.role] > ROLE_LEVEL.write ? "write" : auth.caller.role;
168172 if (ROLE_LEVEL[now] < ROLE_LEVEL[cap]) cap = now;
169173 return { ...auth.caller, role: cap };
modifiedapps/diffinery/src/docs/api-meta.ts+64−0View file
@@ -50,6 +50,13 @@
5050 "POST /diffinery/tokens": { scope: "admin", sessionOnly: true },
5151 "DELETE /diffinery/tokens/{id}": { scope: "admin" },
5252 "POST /diffinery/agents": { scope: "admin", sessionOnly: true },
53+ "PUT /diffinery/model-keys/{provider}": { scope: "admin", sessionOnly: true },
54+ "DELETE /diffinery/model-keys/{provider}": { scope: "admin", sessionOnly: true },
55+ "GET /diffinery/model-keys": { scope: "read", anyToken: true },
56+ "GET /diffinery/tasks": { scope: "read", anyToken: true },
57+ "POST /repos/{owner}/{repo}/diffinery/tasks": { scope: "write", sessionOnly: true },
58+ "POST /repos/{owner}/{repo}/diffinery/tasks/{number}/messages": { scope: "write", sessionOnly: true },
59+ "POST /repos/{owner}/{repo}/diffinery/tasks/{number}/retry": { scope: "write", sessionOnly: true },
5360 "POST /diffinery/agents/{login}/tokens": { scope: "admin", sessionOnly: true },
5461 "DELETE /diffinery/agents/{login}": { scope: "admin" },
5562 "PUT /notifications": { scope: "write" },
@@ -1331,6 +1338,63 @@
13311338 status: 201,
13321339 body: { ref: ["string", "Branch, tag or commit to check out (default: the default branch)."] },
13331340 },
1341+ // ------------------------------------------------------------ model access and agent tasks (agentic/)
1342+ "GET /diffinery/model-keys": {
1343+ summary: "List your model keys",
1344+ description: "Your saved model API keys for agent tasks: `provider` (`anthropic` or `openai`), `hint` (the key's last four characters), `opted_in_at`, `created_at` and `updated_at`. The keys themselves are never returned. See /docs/agents/tasks#model-access.",
1345+ access: "account",
1346+ },
1347+ "PUT /diffinery/model-keys/{provider}": {
1348+ summary: "Save a model key",
1349+ description: "From a browser session only: no token, agent, job or sandbox can set a model key. `provider` is `anthropic` or `openai`. `accept_terms: true` records that you use that provider through Diffinery under your own account and its terms (422 without it). The key is encrypted, never shown again and never sent to a sandbox. Sends you a security email. Audited as `model_key.set`.",
1350+ access: "account",
1351+ body: { key: ["string", "The API key (`sk-ant-...` or `sk-...`)."], accept_terms: ["boolean", "Must be `true`."] },
1352+ },
1353+ "DELETE /diffinery/model-keys/{provider}": { summary: "Delete a model key", description: "From a browser session only. Running tasks that use it lose model access. Audited as `model_key.delete`.", access: "account", status: 204 },
1354+ "GET /diffinery/tasks": {
1355+ summary: "List your agent tasks",
1356+ description: "Agent tasks you started across repositories where you still have write access, most recently changed first (for an agent's token: the tasks it runs). A token limited to repositories sees only those.",
1357+ access: "account",
1358+ },
1359+ "GET /repos/{owner}/{repo}/diffinery/tasks": {
1360+ summary: "List agent tasks",
1361+ description: "The repository's agent tasks, newest first. Needs write access: transcripts show an agent's reasoning and command output. `state` filters: `running`, `ready`, `failed` or `stopped`.",
1362+ access: "write",
1363+ query: { state: ["string", "`running`, `ready`, `failed` or `stopped`."], per_page: ["integer", "At most 100 (default 50)."] },
1364+ },
1365+ "POST /repos/{owner}/{repo}/diffinery/tasks": {
1366+ summary: "Start an agent task",
1367+ description: "From a browser session only, by a person with write access, as one of their agents (they own it, or administer the organization that owns it) that has write access here: each run spends the person's own model key, so no token can start one. The agent works in a sandbox, then Diffinery pushes `agent/<number>-<slug>` and opens a pull request as the agent. 422 when the runtime needs a model key you haven't saved. Audited as `task.start`. See /docs/agents/tasks.",
1368+ access: "write",
1369+ status: 201,
1370+ body: {
1371+ goal: ["string", "What to do (required, at most 20,000 characters)."],
1372+ runtime: ["string", "`claude-code` (default), `codex` or `custom`."],
1373+ agent: ["string", "The agent's login (required)."],
1374+ base: ["string", "Branch to start from (default: the default branch)."],
1375+ command: ["string", "For `custom`: the shell command to run."],
1376+ },
1377+ },
1378+ "GET /repos/{owner}/{repo}/diffinery/tasks/{number}": { summary: "Get an agent task", description: "The task, its runs and its pull request. Needs write access.", access: "write" },
1379+ "GET /repos/{owner}/{repo}/diffinery/tasks/{number}/transcript": {
1380+ summary: "Read a task's transcript",
1381+ description: "Lines after `after` (a line number), oldest first: `stream` is `out` and `err` (the runtime's output, Claude Code's `stream-json` or Codex's `--json` lines), `setup` (installing and pushing) or `note` (Diffinery's own). Model keys and git credentials are masked.",
1382+ access: "write",
1383+ query: { after: ["integer", "Return lines after this one."], per_page: ["integer", "At most 5000 (default 2000)."] },
1384+ },
1385+ "POST /repos/{owner}/{repo}/diffinery/tasks/{number}/messages": {
1386+ summary: "Send a task a follow-up",
1387+ description: "From a browser session only, by the person who started the task. Starts a new run on the task's branch with the message (while a run is active, it waits and starts when that run ends). Audited as `task.message`.",
1388+ access: "write",
1389+ status: 201,
1390+ body: { body: ["string", "The message."] },
1391+ },
1392+ "POST /repos/{owner}/{repo}/diffinery/tasks/{number}/retry": { summary: "Retry a task", description: "From a browser session only, by the person who started the task, when no run is active (409 otherwise). Audited as `task.retry`.", access: "write", status: 201 },
1393+ "POST /repos/{owner}/{repo}/diffinery/tasks/{number}/stop": {
1394+ summary: "Stop a task",
1395+ description: "The person who started it, its agent, or a repository admin. Stops the agent, closes the sandbox (its git credential stops working) and ends the run's model access. Work already pushed stays. Audited as `task.stop`.",
1396+ access: "write",
1397+ },
13341398 "POST /repos/{owner}/{repo}/diffinery/sandboxes/{id}/{action}": {
13351399 summary: "Use a sandbox",
13361400 description: "`action` is `exec` (run a command), `read` or `write` (files), or `close`.",
modifiedapps/diffinery/src/env.ts+12−0View file
@@ -3,6 +3,7 @@
33 import type { ImportContainer } from "./imports/container.js";
44 import type { RunnerContainer, SandboxContainer } from "./runner/containers.js";
55 import type { MailJob } from "./mail/types.js";
6+import type { ProjectDO } from "./agentic/ProjectDO.js";
67
78 export interface Env {
89 DB: D1Database;
@@ -11,6 +12,17 @@
1112 REPO: DurableObjectNamespace<RepoDO>;
1213 RUNNER: DurableObjectNamespace<RunnerContainer>;
1314 SANDBOX: DurableObjectNamespace<SandboxContainer>;
15+ /** One per repository: its agent tasks (agentic/ProjectDO.ts). */
16+ PROJECT: DurableObjectNamespace<ProjectDO>;
17+ /**
18+ * Local development and end-to-end tests only: sandboxes run in one `agent.mjs --sandbox-pool`
19+ * process at this URL instead of Cloudflare Containers (which don't run under `wrangler dev`).
20+ */
21+ SANDBOX_POOL_URL?: string;
22+ SANDBOX_POOL_SECRET?: string;
23+ /** Local development and end-to-end tests only: where the model proxy forwards instead of the providers' APIs. */
24+ MODEL_UPSTREAM_ANTHROPIC?: string;
25+ MODEL_UPSTREAM_OPENAI?: string;
1426 RUN_WORKFLOW: Workflow<{ repoId: string; runId: string }>;
1527 /** Cloudflare Email Sending. Missing when sending isn't set up; emails are then logged as skipped. */
1628 EMAIL?: SendEmail;
modifiedapps/diffinery/src/index.ts+2−1View file
@@ -6,10 +6,11 @@
66 import { web } from "./web/routes.js";
77
88 export { RepoDO } from "./repo/RepoDO.js";
9-export { RunnerContainer, SandboxContainer } from "./runner/containers.js";
9+export { RunnerContainer, SandboxContainer, SandboxPoolDO } from "./runner/containers.js";
1010 export { RunWorkflow } from "./runner/workflow.js";
1111 export { ImportWorkflow } from "./imports/workflow.js";
1212 export { ImportContainer } from "./imports/container.js";
13+export { ProjectDO } from "./agentic/ProjectDO.js";
1314
1415 // The web UI is mounted last so API, git and runner routes take precedence;
1516 // unknown API paths answer in JSON rather than falling through to its HTML 404.
modifiedapps/diffinery/src/mail/samples.ts+1−0View file
@@ -64,6 +64,7 @@
6464 item("token-created", "Access token created", { kind: "token_created", reason: "security_alert", data: { name: "laptop", scopes: ["write"] } }),
6565 item("token-expiring", "Access token expiring", { kind: "token_expiring", reason: "security_alert", data: { name: "ci-deploy", prefix: "dfy_Ab12Cd", expiresAt: exp } }),
6666 item("agent-created", "Agent added", { kind: "agent_created", reason: "security_alert", data: { agent: "review-bot" } }),
67+ item("model-key-added", "Model key added", { kind: "model_key_added", reason: "security_alert", data: { provider: "Anthropic", hint: "x7Qa" } }),
6768 item("push-bypassed", "Push protection bypassed", { kind: "push_bypassed", reason: "security_alert", data: { repo, pusher: me, kinds: ["AWS access key"], reason: "Fake key used by a test fixture", refs: ["refs/heads/main"] } }),
6869 item("push-blocked", "Push blocked", { kind: "push_blocked", reason: "security_alert", data: { repo, kinds: ["GitHub token"], refs: ["refs/heads/feature"] } }),
6970 item("commit-blocked", "Commit blocked", { kind: "push_blocked", reason: "security_alert", data: { repo, kinds: ["AWS access key"], paths: ["config.env"] } }),
modifiedapps/diffinery/src/mail/templates.ts+16−0View file
@@ -437,6 +437,22 @@
437437 ],
438438 url: `${pub}/settings/agents`,
439439 };
440+ case "model_key_added": {
441+ const page = `${pub}/settings/model-access`;
442+ return {
443+ subject: `[Diffinery] Your ${job.data.provider} API key was added to Diffinery`,
444+ preheader: `Agent tasks you start can now use your ${job.data.provider} key.`,
445+ eyebrow: { text: "Security", tone: "red" },
446+ title: [`Your ${job.data.provider} API key was added to Diffinery`],
447+ blocks: [
448+ { p: ["Agent tasks you start will use this key, and the provider bills your account for them. Diffinery never shows the key again."] },
449+ { kv: [["Provider", job.data.provider], ["Ends with", [{ code: `…${job.data.hint}` }]], ["When", at(job.at)]] },
450+ { callout: ["If you didn't add this key, delete it here and revoke it with the provider."], tone: "warn" },
451+ { button: { label: "Review model access", url: page } },
452+ ],
453+ url: page,
454+ };
455+ }
440456 case "push_bypassed": {
441457 const d = job.data;
442458 return {
modifiedapps/diffinery/src/mail/types.ts+2−0View file
@@ -76,6 +76,7 @@
7676 | { kind: "token_created"; data: { name: string; scopes: string[]; agent?: string } }
7777 | { kind: "token_expiring"; data: { name: string; prefix: string; expiresAt: string; agent?: string } }
7878 | { kind: "agent_created"; data: { agent: string } }
79+ | { kind: "model_key_added"; data: { provider: string; hint: string } }
7980 | { kind: "push_bypassed"; data: { repo: string; pusher: string; kinds: string[]; reason: string; refs?: string[] } }
8081 | { kind: "push_blocked"; data: { repo: string; kinds: string[]; refs?: string[]; paths?: string[] } }
8182 | { kind: "account_status"; data: { status: "disabled" | "active" } }
@@ -113,6 +114,7 @@
113114 case "token_created":
114115 case "token_expiring":
115116 case "agent_created":
117+ case "model_key_added":
116118 case "push_bypassed":
117119 case "push_blocked":
118120 case "account_status":
modifiedapps/diffinery/src/repo/RepoDO.ts+3−0View file
@@ -923,6 +923,9 @@
923923 closeSandbox(...a: Parameters<Actions["closeSandbox"]>) {
924924 return this.actions.closeSandbox(...a);
925925 }
926+ closeTaskSandbox(...a: Parameters<Actions["closeTaskSandbox"]>) {
927+ return this.actions.closeTaskSandbox(...a);
928+ }
926929
927930 plannedRunError(runId: string, message: string) {
928931 return this.actions.plannedError(runId, message);
modifiedapps/diffinery/src/repo/actions.ts+30−8View file
@@ -1110,7 +1110,7 @@
11101110 /** Validate a job or sandbox token. Returns what it grants. */
11111111 async runnerAuth(
11121112 token: string,
1113- ): Promise<{ kind: "job"; jobId: string; runId: string; role: Caller["role"]; permissions: JobPermissions; actorKind: Actor["kind"] } | { kind: "sandbox"; sandboxId: string; caller: Caller } | null> {
1113+ ): Promise<{ kind: "job"; jobId: string; runId: string; role: Caller["role"]; permissions: JobPermissions; actorKind: Actor["kind"] } | { kind: "sandbox"; sandboxId: string; caller: Caller; taskPerson?: string } | null> {
11141114 const parts = token.split("_");
11151115 if (parts.length < 4) return null;
11161116 const hash = await sha256(token);
@@ -1124,11 +1124,11 @@
11241124 return { kind: "job", jobId: j.id, runId: j.run_id, role: grant.role, permissions: grant.permissions, actorKind };
11251125 }
11261126 if (parts[0] === "dst") {
1127- const s = this.sql.exec<{ id: string; secret: string; created_by: string; status: string; created_at: string }>("SELECT * FROM sandboxes WHERE id = ?", parts[2]!).toArray()[0];
1127+ const s = this.sql.exec<{ id: string; secret: string; created_by: string; status: string; created_at: string; task_person: string | null }>("SELECT * FROM sandboxes WHERE id = ?", parts[2]!).toArray()[0];
11281128 if (!s || s.secret !== hash || s.status !== "running") return null;
11291129 // A sandbox's token lasts a day at most, even if nobody closes the sandbox.
11301130 if (Date.now() - Date.parse(s.created_at) > SANDBOX_TOKEN_TTL_MS) return null;
1131- return { kind: "sandbox", sandboxId: s.id, caller: JSON.parse(s.created_by) as Caller };
1131+ return { kind: "sandbox", sandboxId: s.id, caller: JSON.parse(s.created_by) as Caller, ...(s.task_person ? { taskPerson: s.task_person } : {}) };
11321132 }
11331133 return null;
11341134 }
@@ -1526,7 +1526,11 @@
15261526
15271527 // ---------------------------------------------------------------- sandboxes for agents
15281528
1529- async createSandbox(caller: Caller, ref?: string): Promise<{ id: string; ref: string }> {
1529+ /**
1530+ * `task` is set only by agentic/ (never from a request): the sandbox of an agent task, started for
1531+ * the agent `caller` by the person `task.personId`. No public sandbox call reaches it.
1532+ */
1533+ async createSandbox(caller: Caller, ref?: string, task?: { personId: string }): Promise<{ id: string; ref: string }> {
15301534 if (!atLeast(caller.role, "write")) throw new HttpError(403, "write access is required to create a sandbox");
15311535 const sha = await this.repo.resolve(ref ?? "HEAD");
15321536 if (!sha) throw new HttpError(422, "unknown ref");
@@ -1534,7 +1538,16 @@
15341538 const m = this.repo.meta;
15351539 const token = `dst_${m.repoId}_${id}_${randomToken(24)}`;
15361540 const secret = randomToken(32);
1537- this.sql.exec("INSERT INTO sandboxes (id, secret, created_by, ref, created_at, status) VALUES (?, ?, ?, ?, ?, 'running')", id, await sha256(token), JSON.stringify(caller), ref ?? m.defaultBranch, now());
1541+ this.sql.exec(
1542+ "INSERT INTO sandboxes (id, secret, created_by, ref, created_at, status, kind, task_person) VALUES (?, ?, ?, ?, ?, 'running', ?, ?)",
1543+ id,
1544+ await sha256(token),
1545+ JSON.stringify(caller),
1546+ ref ?? m.defaultBranch,
1547+ now(),
1548+ task ? "task" : "api",
1549+ task?.personId ?? null,
1550+ );
15381551 const stub = this.env.SANDBOX.get(this.env.SANDBOX.idFromName(id));
15391552 try {
15401553 await stub.boot({
@@ -1547,7 +1560,7 @@
15471560 GIT_AUTHOR_EMAIL: `${caller.login}@users.noreply.diffinery.com`,
15481561 GIT_COMMITTER_NAME: caller.login,
15491562 GIT_COMMITTER_EMAIL: `${caller.login}@users.noreply.diffinery.com`,
1550- }, secret);
1563+ }, secret, id);
15511564 } catch (e) {
15521565 console.error("sandbox boot failed", e);
15531566 this.sql.exec("UPDATE sandboxes SET status = 'failed' WHERE id = ?", id);
@@ -1558,8 +1571,9 @@
15581571 }
15591572
15601573 getSandbox(id: string, caller: Caller): { id: string; ref: string; status: string; createdAt: string } {
1561- const s = this.sql.exec<{ id: string; ref: string; status: string; created_at: string; created_by: string }>("SELECT * FROM sandboxes WHERE id = ?", id).toArray()[0];
1562- if (!s) throw new HttpError(404, "sandbox not found");
1574+ const s = this.sql.exec<{ id: string; ref: string; status: string; created_at: string; created_by: string; kind: string }>("SELECT * FROM sandboxes WHERE id = ?", id).toArray()[0];
1575+ // A task's sandbox holds a credential for the person's model access: nobody reaches it through the sandbox API.
1576+ if (!s || s.kind === "task") throw new HttpError(404, "sandbox not found");
15631577 const owner = JSON.parse(s.created_by) as Caller;
15641578 if (owner.id !== caller.id && !atLeast(caller.role, "admin")) throw new HttpError(403, "not your sandbox");
15651579 return { id: s.id, ref: s.ref, status: s.status, createdAt: s.created_at };
@@ -1570,4 +1584,12 @@
15701584 this.sql.exec("UPDATE sandboxes SET status = 'closed' WHERE id = ?", id);
15711585 await this.env.SANDBOX.get(this.env.SANDBOX.idFromName(id)).shutdown();
15721586 }
1587+
1588+ /** Close an agent task's sandbox (agentic/), which revokes its git credential. */
1589+ async closeTaskSandbox(id: string): Promise<void> {
1590+ const s = this.sql.exec<{ kind: string }>("SELECT kind FROM sandboxes WHERE id = ?", id).toArray()[0];
1591+ if (s?.kind !== "task") return;
1592+ this.sql.exec("UPDATE sandboxes SET status = 'closed' WHERE id = ?", id);
1593+ await this.env.SANDBOX.get(this.env.SANDBOX.idFromName(id)).shutdown();
1594+ }
15731595 }
modifiedapps/diffinery/src/repo/schema.ts+4−0View file
@@ -186,6 +186,10 @@
186186 // Auto-merge while it is on (repo/auto-merge.ts AutoMergeState as JSON); null when off.
187187 ["pulls", "auto_merge", "TEXT"],
188188 ["releases", "imported", "TEXT"],
189+ // "task" for an agent task's sandbox (agentic/): no public sandbox call reaches it, and its git
190+ // credential also needs the person who started the task (task_person) to still be allowed.
191+ ["sandboxes", "kind", "TEXT NOT NULL DEFAULT 'api'"],
192+ ["sandboxes", "task_person", "TEXT"],
189193 ];
190194
191195 export function migrateRepoSchema(sql: SqlStorage): void {
modifiedapps/diffinery/src/runner/containers.ts+33−1View file
@@ -1,3 +1,4 @@
1+import { DurableObject } from "cloudflare:workers";
12 import { Container } from "@cloudflare/containers";
23 import type { StopParams } from "@cloudflare/containers";
34 import type { Env } from "../env.js";
@@ -48,7 +49,7 @@
4849 override sleepAfter = "30m";
4950 override enableInternet = true;
5051
51- async boot(envVars: Record<string, string>, secret: string): Promise<void> {
52+ async boot(envVars: Record<string, string>, secret: string, _name?: string): Promise<void> {
5253 await this.ctx.storage.put("secret", secret);
5354 // The first start on a host after a deploy pulls the runner image, which takes
5455 // well over the library's 8 s / 20 s defaults.
@@ -74,3 +75,34 @@
7475 if (this.ctx.container?.running) await this.stop();
7576 }
7677 }
78+
79+/**
80+ * Local development and end-to-end tests only: `env.dev` binds SANDBOX to this
81+ * instead of SandboxContainer, because containers don't run under `wrangler dev`.
82+ * The same calls, served by one `agent.mjs --sandbox-pool` process at
83+ * SANDBOX_POOL_URL that runs each sandbox's server.
84+ */
85+export class SandboxPoolDO extends DurableObject<Env> {
86+ private poolAuth() {
87+ return { authorization: `Bearer ${this.env.SANDBOX_POOL_SECRET ?? ""}` };
88+ }
89+
90+ async boot(envVars: Record<string, string>, secret: string, name?: string): Promise<void> {
91+ if (!this.env.SANDBOX_POOL_URL || !name) throw new Error("SANDBOX_POOL_URL is not set: start `agent.mjs --sandbox-pool` for sandboxes under wrangler dev");
92+ await this.ctx.storage.put({ secret, name });
93+ const r = await fetch(`${this.env.SANDBOX_POOL_URL}/boot`, { method: "POST", headers: this.poolAuth(), body: JSON.stringify({ id: name, secret, env: envVars }) });
94+ if (!r.ok) throw new Error(`sandbox pool boot failed: ${r.status} ${await r.text()}`);
95+ }
96+
97+ async call(path: string, body: string): Promise<{ status: number; body: string; json: boolean }> {
98+ const [secret, name] = await Promise.all([this.ctx.storage.get<string>("secret"), this.ctx.storage.get<string>("name")]);
99+ if (!name) return { status: 404, body: JSON.stringify({ error: "sandbox not found" }), json: true };
100+ const res = await fetch(`${this.env.SANDBOX_POOL_URL}/s/${name}${path}`, { method: "POST", headers: { "content-type": "application/json", authorization: `Bearer ${secret}` }, body: body || "{}" });
101+ return { status: res.status, body: await res.text(), json: (res.headers.get("content-type") ?? "").includes("json") };
102+ }
103+
104+ async shutdown(): Promise<void> {
105+ const name = await this.ctx.storage.get<string>("name");
106+ if (name) await fetch(`${this.env.SANDBOX_POOL_URL}/shutdown/${name}`, { method: "POST", headers: this.poolAuth() }).catch(() => {});
107+ }
108+}
modifiedapps/diffinery/src/web/account.tsx+4−1View file
@@ -600,7 +600,7 @@
600600
601601 // ---------------------------------------------------------------- /settings
602602
603-export type SettingsTab = "profile" | "appearance" | "security" | "tokens" | "agents" | "orgs" | "notifications" | "linked" | "admin" | "email" | "audit";
603+export type SettingsTab = "profile" | "appearance" | "security" | "tokens" | "agents" | "model-access" | "orgs" | "notifications" | "linked" | "admin" | "email" | "audit";
604604
605605 export const SettingsNav: FC<{ me: Account; tab: SettingsTab }> = ({ me, tab }) => (
606606 <aside class="side vt-side">
@@ -628,6 +628,9 @@
628628 <a href="/settings/agents" class={tab === "agents" ? "on" : ""}>
629629 Agents
630630 </a>
631+ <a href="/settings/model-access" class={tab === "model-access" ? "on" : ""}>
632+ Model access
633+ </a>
631634 <a href="/settings/orgs" class={tab === "orgs" ? "on" : ""}>
632635 Organizations
633636 </a>
modifiedapps/diffinery/src/web/assets.ts+7−0View file
@@ -256,6 +256,13 @@
256256 .features .box{padding:18px}.features h3{margin:0 0 6px}
257257 .stats{display:flex;gap:18px;flex-wrap:wrap;color:var(--fg2);font-size:13px}
258258 .stats b{color:var(--fg)}
259+.tx{display:flex;flex-direction:column;gap:10px}.tx .markdown{margin:0}
260+.tx-run{margin:8px 0 0;padding-top:8px;border-top:1px solid var(--border);font-size:13px;color:var(--muted)}.tx-run:first-child{border-top:0;padding-top:0;margin-top:0}
261+.tx-note{padding-left:10px;border-left:2px solid var(--border)}
262+.tx-result{padding:10px 12px;border:1px solid var(--border);border-radius:var(--r);background:var(--subtle,transparent)}
263+.tx-tool summary{cursor:pointer;color:var(--muted)}.tx-tool[open] summary{margin-bottom:6px}
264+.tx-pre{margin:0;padding:8px 10px;background:#0d1014;color:#d5d9de;border-radius:var(--r);font-size:12px;white-space:pre-wrap;word-break:break-word;max-height:360px;overflow:auto}
265+.tx-err{border-left:3px solid var(--red)}
259266 .tiles{display:grid;grid-template-columns:repeat(auto-fit,minmax(150px,1fr));gap:12px;margin:0 0 16px}
260267 .tile{display:flex;flex-direction:column;gap:2px;border:1px solid var(--border);border-radius:var(--r);padding:12px 14px;background:var(--bg);color:var(--fg);text-decoration:none}
261268 .tile:hover{border-color:var(--accent);text-decoration:none}
modifiedapps/diffinery/src/web/routes.tsx+3−0View file
@@ -3,6 +3,7 @@
33 import type { Env } from "../env.js";
44 import type { AppVars } from "../auth/principal.js";
55 import { registerAccount } from "./account.js";
6+import { registerTasks } from "../agentic/tasks-web.js";
67 import { registerMailWeb } from "../mail/web.js";
78 import { registerAuth } from "./auth.js";
89 import { registerActions } from "./actions.js";
@@ -42,6 +43,8 @@
4243 registerMailWeb(web);
4344 // Before registerAccount: its /:owner catch-all would otherwise swallow /imports, /new/import and /settings/linked-accounts.
4445 registerImports(web);
46+// Settings → Model access and /:owner/:repo/tasks: before the /:owner and /:owner/:repo catch-alls.
47+registerTasks(web);
4548 registerAccount(web);
4649 registerRepoSettings(web);
4750 registerActions(web);
modifiedapps/diffinery/src/web/ui.tsx+6−1View file
@@ -261,7 +261,7 @@
261261 export interface RepoCtx {
262262 repo: Repo;
263263 caller: Caller;
264- tab?: "code" | "issues" | "pulls" | "actions" | "packages" | "settings" | "";
264+ tab?: "code" | "issues" | "pulls" | "actions" | "tasks" | "packages" | "settings" | "";
265265 /** Current ref, used by the palette's "go to file". */
266266 ref?: string;
267267 counts?: { issues?: number; pulls?: number };
@@ -339,6 +339,11 @@
339339 <a href={`${base}/actions`} class={tab === "actions" ? "on" : ""}>
340340 {raw(ICONS.play)} Actions
341341 </a>
342+ {atLeast(r.caller.role, "write") ? (
343+ <a href={`${base}/tasks`} class={tab === "tasks" ? "on" : ""}>
344+ {raw(ICONS.bot)} Tasks
345+ </a>
346+ ) : null}
342347 <a href={`${base}/packages`} class={tab === "packages" ? "on" : ""}>
343348 {raw(ICONS.package)} Packages
344349 </a>
addedapps/diffinery/test/e2e/tasks.test.mjs+414−0View file
@@ -1,0 +1,414 @@
1+// End-to-end agent tasks: a real `wrangler dev` Worker, sandboxes in an
2+// `agent.mjs --sandbox-pool` process (containers don't run under wrangler dev),
3+// a fake model provider behind the model proxy, and the real git CLI.
4+//
5+// pnpm --filter @diffinery/app e2e
6+//
7+// Covers model keys (write-only, browser session only, opt-in), a task run end
8+// to end (sandbox as the agent, the model proxy adding the person's key, the
9+// push and the pull request as the agent), follow-ups, stopping, and the
10+// security rules: no token, agent or sandbox can start or steer a task, task
11+// sandboxes are closed to the sandbox API, and a run's model token stops
12+// working when the run ends.
13+import { spawn, execFileSync } from "node:child_process";
14+import { mkdtempSync, writeFileSync, mkdirSync, rmSync, readdirSync, statSync, readFileSync, existsSync } from "node:fs";
15+import { tmpdir } from "node:os";
16+import { randomBytes } from "node:crypto";
17+import { join, dirname, resolve } from "node:path";
18+import { fileURLToPath } from "node:url";
19+import { createServer as netServer } from "node:net";
20+import { createServer as httpServer } from "node:http";
21+import { test, before, after } from "node:test";
22+import assert from "node:assert/strict";
23+
24+const APP = resolve(dirname(fileURLToPath(import.meta.url)), "../..");
25+const AGENT = resolve(APP, "../../containers/runner/dist/agent.mjs");
26+const work = mkdtempSync(join(tmpdir(), "diffinery-tasks-e2e-"));
27+const poolRoot = join(work, "pool");
28+const ANTHROPIC_KEY = `sk-ant-api03-${randomBytes(30).toString("hex")}`;
29+let base;
30+let dev;
31+let pool;
32+let upstream;
33+let devLog = "";
34+let poolLog = "";
35+let token;
36+const modelRequests = [];
37+
38+const freePort = () =>
39+ new Promise((res) => {
40+ const s = netServer();
41+ s.listen(0, "127.0.0.1", () => {
42+ const { port } = s.address();
43+ s.close(() => res(port));
44+ });
45+ });
46+
47+async function api(method, path, body, tok = token) {
48+ const res = await fetch(`${base}/api/v3${path}`, {
49+ method,
50+ headers: { ...(tok === null ? {} : { authorization: `Bearer ${tok}` }), "content-type": "application/json" },
51+ body: body === undefined ? undefined : JSON.stringify(body),
52+ });
53+ const text = await res.text();
54+ let json = null;
55+ try {
56+ json = text ? JSON.parse(text) : null;
57+ } catch {
58+ json = text;
59+ }
60+ return { status: res.status, json };
61+}
62+async function ok(method, path, body, tok) {
63+ const r = await api(method, path, body, tok);
64+ assert.ok(r.status >= 200 && r.status < 300, `${method} ${path} -> ${r.status} ${JSON.stringify(r.json)}`);
65+ return r.json;
66+}
67+/** As the signed-in person in a browser (the dev sign-in: no credentials). */
68+const asPerson = (method, path, body) => api(method, path, body, null);
69+
70+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
71+async function until(what, fn, timeoutMs = 90_000) {
72+ const end = Date.now() + timeoutMs;
73+ let last;
74+ while (Date.now() < end) {
75+ last = await fn();
76+ if (last) return last;
77+ await sleep(500);
78+ }
79+ assert.fail(`timed out waiting for ${what}\n--- wrangler log tail ---\n${devLog.slice(-4000)}\n--- pool log tail ---\n${poolLog.slice(-3000)}`);
80+}
81+
82+function git(cwd, args, tok = token) {
83+ const auth = Buffer.from(`x-token:${tok}`).toString("base64");
84+ return execFileSync("git", ["-c", `http.extraHeader=Authorization: Basic ${auth}`, "-c", "protocol.version=2", ...args], {
85+ cwd,
86+ encoding: "utf8",
87+ stdio: ["ignore", "pipe", "pipe"],
88+ env: { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_AUTHOR_NAME: "E2E", GIT_AUTHOR_EMAIL: "e2e@example.com", GIT_COMMITTER_NAME: "E2E", GIT_COMMITTER_EMAIL: "e2e@example.com" },
89+ });
90+}
91+
92+async function mcp(name, args, tok = token) {
93+ const r = await fetch(`${base}/mcp`, { method: "POST", headers: { authorization: `Bearer ${tok}`, "content-type": "application/json", accept: "application/json, text/event-stream" }, body: JSON.stringify({ jsonrpc: "2.0", id: 1, method: "tools/call", params: { name, arguments: args } }) });
94+ const { result } = await r.json();
95+ return result;
96+}
97+
98+/** The sandboxes the pool has started, newest first. */
99+const poolSandboxes = () =>
100+ existsSync(poolRoot)
101+ ? readdirSync(poolRoot)
102+ .map((id) => ({ id, at: statSync(join(poolRoot, id)).mtimeMs }))
103+ .sort((a, b) => b.at - a.at)
104+ .map((x) => x.id)
105+ : [];
106+
107+before(async () => {
108+ if (!existsSync(AGENT)) execFileSync("pnpm", ["--filter", "@diffinery/runner", "build"], { cwd: APP, stdio: "pipe" });
109+ // A stand-in for the provider: records what reaches it and answers like the Messages API.
110+ upstream = httpServer((req, res) => {
111+ let body = "";
112+ req.on("data", (d) => (body += d));
113+ req.on("end", () => {
114+ modelRequests.push({ method: req.method, url: req.url, headers: req.headers, body });
115+ res.writeHead(200, { "content-type": "application/json", "set-cookie": "upstream=1" });
116+ res.end(JSON.stringify({ type: "message", content: [{ type: "text", text: "fake model says hi" }] }));
117+ });
118+ });
119+ await new Promise((r) => upstream.listen(0, "127.0.0.1", r));
120+ const upstreamUrl = `http://127.0.0.1:${upstream.address().port}`;
121+
122+ const poolPort = await freePort();
123+ const poolSecret = randomBytes(16).toString("hex");
124+ pool = spawn("node", [AGENT, "--sandbox-pool", "--work-root", poolRoot], { env: { ...process.env, DIFFINERY_SANDBOX_POOL_SECRET: poolSecret, DIFFINERY_SANDBOX_PORT: String(poolPort) }, stdio: ["ignore", "pipe", "pipe"], detached: true });
125+ pool.stdout.on("data", (d) => (poolLog += d));
126+ pool.stderr.on("data", (d) => (poolLog += d));
127+ await until("sandbox pool", async () => poolLog.includes("listening"), 30_000);
128+
129+ const port = await freePort();
130+ base = `http://127.0.0.1:${port}`;
131+ const persist = join(work, "state");
132+ execFileSync("pnpm", ["exec", "wrangler", "d1", "migrations", "apply", "diffinery-dev", "--local", "-e", "dev", "--persist-to", persist], { cwd: APP, stdio: "pipe", env: { ...process.env, CI: "1" } });
133+ const vars = { PUBLIC_URL: base, MASTER_KEY: randomBytes(32).toString("base64"), SANDBOX_POOL_URL: `http://127.0.0.1:${poolPort}`, SANDBOX_POOL_SECRET: poolSecret, MODEL_UPSTREAM_ANTHROPIC: upstreamUrl, MODEL_UPSTREAM_OPENAI: upstreamUrl };
134+ dev = spawn("pnpm", ["exec", "wrangler", "dev", "-e", "dev", "--ip", "127.0.0.1", "--port", String(port), "--inspector-port", String(await freePort()), "--persist-to", persist, ...Object.entries(vars).flatMap(([k, v]) => ["--var", `${k}:${v}`]), "--show-interactive-dev-session=false"], {
135+ cwd: APP,
136+ env: { ...process.env, CI: "1", WRANGLER_SEND_METRICS: "false" },
137+ stdio: ["ignore", "pipe", "pipe"],
138+ detached: true,
139+ });
140+ dev.stdout.on("data", (d) => (devLog += d));
141+ dev.stderr.on("data", (d) => (devLog += d));
142+ await until(
143+ "wrangler dev",
144+ async () => {
145+ try {
146+ return (await fetch(`${base}/healthz`)).ok;
147+ } catch {
148+ return false;
149+ }
150+ },
151+ 120_000,
152+ );
153+ const res = await fetch(`${base}/api/v3/diffinery/tokens`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ name: "e2e", scopes: ["admin"] }) });
154+ assert.equal(res.status, 201, await res.clone().text());
155+ token = (await res.json()).token;
156+});
157+
158+after(() => {
159+ for (const p of [dev, pool]) {
160+ if (p?.pid) {
161+ try {
162+ process.kill(-p.pid, "SIGTERM");
163+ } catch {}
164+ }
165+ }
166+ upstream?.close();
167+ if (process.env.E2E_KEEP !== "1") rmSync(work, { recursive: true, force: true });
168+});
169+
170+test("model keys: write-only, browser session only, with an opt-in", async () => {
171+ assert.deepEqual(await ok("GET", "/diffinery/model-keys"), []);
172+ // No token can set one, however wide.
173+ const viaToken = await api("PUT", "/diffinery/model-keys/anthropic", { key: ANTHROPIC_KEY, accept_terms: true });
174+ assert.equal(viaToken.status, 403, JSON.stringify(viaToken.json));
175+ assert.match(viaToken.json.message, /browser session/);
176+ // The opt-in is required, and the key's shape is checked.
177+ assert.equal((await asPerson("PUT", "/diffinery/model-keys/anthropic", { key: ANTHROPIC_KEY })).status, 422);
178+ assert.equal((await asPerson("PUT", "/diffinery/model-keys/anthropic", { key: "sk-proj-not-anthropic-1234567890abcdef", accept_terms: true })).status, 422);
179+ assert.equal((await asPerson("PUT", "/diffinery/model-keys/gemini", { key: ANTHROPIC_KEY, accept_terms: true })).status, 404);
180+ const saved = await asPerson("PUT", "/diffinery/model-keys/anthropic", { key: ANTHROPIC_KEY, accept_terms: true });
181+ assert.equal(saved.status, 200, JSON.stringify(saved.json));
182+ assert.equal(saved.json.hint, ANTHROPIC_KEY.slice(-4));
183+ assert.ok(saved.json.opted_in_at);
184+ // Never returned: not by the API, not on the page.
185+ const listed = await ok("GET", "/diffinery/model-keys");
186+ assert.deepEqual(listed.map((k) => k.provider), ["anthropic"]);
187+ assert.ok(!JSON.stringify(listed).includes(ANTHROPIC_KEY));
188+ const page = await (await fetch(`${base}/settings/model-access`)).text();
189+ assert.match(page, /Model access/);
190+ assert.match(page, /Commercial Terms/);
191+ assert.ok(page.includes(`…${ANTHROPIC_KEY.slice(-4)}`));
192+ assert.ok(!page.includes(ANTHROPIC_KEY));
193+ assert.equal((await api("DELETE", "/diffinery/model-keys/anthropic")).status, 403);
194+});
195+
196+test("an agent task runs in a sandbox as the agent, through the model proxy, and opens a pull request", async () => {
197+ const me = await ok("GET", "/user");
198+ const owner = me.login;
199+ await ok("POST", "/user/repos", { name: "tasks-demo" });
200+ const src = join(work, "src");
201+ mkdirSync(src, { recursive: true });
202+ writeFileSync(join(src, "README.md"), "# tasks\n");
203+ git(src, ["init", "-q", "-b", "main"]);
204+ git(src, ["add", "-A"]);
205+ git(src, ["commit", "-q", "-m", "initial"]);
206+ git(src, ["remote", "add", "origin", `${base}/${owner}/tasks-demo.git`]);
207+ git(src, ["push", "-q", "origin", "main"]);
208+ const R = `/repos/${owner}/tasks-demo/diffinery/tasks`;
209+ const bot = (await asPerson("POST", "/diffinery/agents", { login: "task-bot", name: "Task Bot" })).json.token.token;
210+
211+ const tokFile = join(work, "model-token");
212+ // The custom runtime: call the model through the proxy, show the key variable (masked), save the token for later, change a file.
213+ const command = [
214+ `curl -sS -X POST "$ANTHROPIC_BASE_URL/v1/messages" -H "x-api-key: $ANTHROPIC_API_KEY" -H "content-type: application/json" -d '{"model":"m","messages":[]}'`,
215+ "echo",
216+ 'echo "key is $ANTHROPIC_API_KEY"',
217+ `printf %s "$ANTHROPIC_API_KEY" > ${tokFile}`,
218+ 'echo "goal: $DIFFINERY_TASK_PROMPT" | tail -1',
219+ "echo hello > hello.txt",
220+ ].join("; ");
221+ const goal = "Add a hello file";
222+
223+ // Nobody but a person in a browser starts a task: not their token, not the agent's.
224+ for (const tok of [token, bot]) {
225+ const r = await api("POST", R, { goal, runtime: "custom", command, agent: "task-bot" }, tok);
226+ assert.equal(r.status, 403, JSON.stringify(r.json));
227+ assert.match(r.json.message, /browser session/);
228+ }
229+ // Codex needs an OpenAI key the person doesn't have.
230+ const codex = await asPerson("POST", R, { goal, runtime: "codex", agent: "task-bot" });
231+ assert.equal(codex.status, 422);
232+ assert.match(codex.json.message, /OpenAI API key/);
233+ assert.equal((await asPerson("POST", R, { goal, runtime: "custom", command, agent: owner })).status, 422, "a person isn't an agent");
234+
235+ const started = await asPerson("POST", R, { goal, runtime: "custom", command, agent: "task-bot" });
236+ assert.equal(started.status, 201, JSON.stringify(started.json));
237+ const n = started.json.number;
238+ assert.equal(started.json.state, "running");
239+ assert.equal(started.json.branch, "agent/1-add-a-hello-file");
240+ assert.equal(started.json.agent.login, "task-bot");
241+ assert.equal(started.json.started_by.login, owner);
242+
243+ const done = await until("the task to finish", async () => {
244+ const t = await ok("GET", `${R}/${n}`);
245+ return t.state !== "running" ? t : null;
246+ });
247+ const lines = await ok("GET", `${R}/${n}/transcript`);
248+ const text = lines.map((l) => `[${l.stream}] ${l.text}`).join("\n");
249+ assert.equal(done.state, "ready", text);
250+ assert.ok(done.pull_request, text);
251+
252+ // The provider got the person's key, never the run's token; the sandbox never saw the key.
253+ assert.equal(modelRequests.length, 1, text);
254+ assert.equal(modelRequests[0].headers["x-api-key"], ANTHROPIC_KEY);
255+ assert.equal(modelRequests[0].url, "/v1/messages");
256+ assert.ok(!JSON.stringify(modelRequests[0].headers).includes("dmt_"));
257+ assert.match(text, /fake model says hi/);
258+ assert.match(text, /key is \*\*\*/, "the run's token is masked in the transcript");
259+ assert.match(text, /\[out\] Add a hello file/, "the prompt reaches the runtime");
260+ assert.ok(!text.includes(ANTHROPIC_KEY));
261+ const runToken = readFileSync(tokFile, "utf8");
262+ assert.match(runToken, /^dmt_/);
263+ assert.ok(!text.includes(runToken));
264+
265+ // The run is over: its model token no longer works.
266+ const late = await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken, "content-type": "application/json" }, body: "{}" });
267+ assert.equal(late.status, 401);
268+ assert.equal(modelRequests.length, 1);
269+ // Nor does the proxy take an account token, or anything outside /v1/.
270+ assert.equal((await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { authorization: `Bearer ${token}` } })).status, 401);
271+ assert.equal((await fetch(`${base}/-/model/anthropic/admin`, { headers: { "x-api-key": runToken } })).status, 403);
272+
273+ // The pull request is the agent's, from the task's branch, with the work.
274+ const pr = await ok("GET", `/repos/${owner}/tasks-demo/pulls/${done.pull_request.number}`);
275+ assert.equal(pr.user.login, "task-bot");
276+ assert.equal(pr.head.ref, "agent/1-add-a-hello-file");
277+ assert.equal(pr.intent, goal);
278+ const files = await ok("GET", `/repos/${owner}/tasks-demo/pulls/${pr.number}/files`);
279+ assert.deepEqual(files.map((f) => f.filename), ["hello.txt"]);
280+ const commit = await ok("GET", `/repos/${owner}/tasks-demo/commits/${pr.head.sha}`);
281+ assert.equal(commit.commit.author.name, "task-bot");
282+
283+ // Read over MCP and on Today; steering is not on MCP.
284+ const viaMcp = await mcp("agent_task", { owner, repo: "tasks-demo", number: n });
285+ assert.equal(viaMcp.isError, false, viaMcp.content[0].text);
286+ assert.equal(viaMcp.structuredContent.state, "ready");
287+ assert.ok(viaMcp.structuredContent.transcript.length > 0);
288+ const mine = await mcp("agent_tasks", {});
289+ assert.deepEqual(mine.structuredContent.result.map((t) => t.number), [n]);
290+ const today = await ok("GET", "/diffinery/today");
291+ assert.equal(today.tasks[0].number, n);
292+ assert.equal(today.tasks[0].state, "ready");
293+
294+ // Pages.
295+ for (const path of ["tasks", `tasks/${n}`, "tasks/new"]) {
296+ const res = await fetch(`${base}/${owner}/tasks-demo/${path}`);
297+ assert.equal(res.status, 200, path);
298+ }
299+ const taskPage = await (await fetch(`${base}/${owner}/tasks-demo/tasks/${n}`)).text();
300+ assert.match(taskPage, /Add a hello file/);
301+ assert.match(taskPage, /fake model says hi/);
302+
303+ // Follow-ups and retries: the person, in a browser; never a token.
304+ assert.equal((await api("POST", `${R}/${n}/messages`, { body: "again" })).status, 403);
305+ assert.equal((await api("POST", `${R}/${n}/retry`, {})).status, 403);
306+ assert.equal((await api("POST", `${R}/${n}/messages`, { body: "again" }, bot)).status, 403);
307+ const followUp = await asPerson("POST", `${R}/${n}/messages`, { body: "Say hello again" });
308+ assert.equal(followUp.status, 201, JSON.stringify(followUp.json));
309+ const second = await until("the follow-up run", async () => {
310+ const t = await ok("GET", `${R}/${n}`);
311+ return t.state !== "running" ? t : null;
312+ });
313+ assert.equal(second.state, "ready");
314+ assert.deepEqual(second.runs.map((r) => [r.kind, r.state]), [
315+ ["start", "done"],
316+ ["message", "done"],
317+ ]);
318+ const text2 = (await ok("GET", `${R}/${n}/transcript?after=${lines.at(-1).seq}`)).map((l) => l.text).join("\n");
319+ assert.match(text2, /Say hello again/);
320+ assert.match(text2, /No changes to push/);
321+ assert.match(text2, /Starting a sandbox on agent\/1-add-a-hello-file/);
322+});
323+
324+test("task sandboxes are closed to the sandbox API, and stopping ends the run's access", async () => {
325+ const me = await ok("GET", "/user");
326+ const owner = me.login;
327+ const R = `/repos/${owner}/tasks-demo/diffinery/tasks`;
328+ const bot = (await asPerson("POST", "/diffinery/agents/task-bot/tokens", { name: "second" })).json.token;
329+ const tokFile = join(work, "model-token-2");
330+ const before_ = new Set(poolSandboxes());
331+ const started = await asPerson("POST", R, { goal: "Wait a while", runtime: "custom", command: `printf %s "$ANTHROPIC_API_KEY" > ${tokFile}; sleep 120`, agent: "task-bot" });
332+ assert.equal(started.status, 201, JSON.stringify(started.json));
333+ const n = started.json.number;
334+ const id = await until("the task's sandbox", async () => poolSandboxes().find((x) => !before_.has(x)) ?? null);
335+ await until("the agent to start", async () => existsSync(tokFile));
336+ const runToken = readFileSync(tokFile, "utf8");
337+
338+ // While it runs, its token works through the proxy.
339+ const live = await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken, "content-type": "application/json" }, body: "{}" });
340+ assert.equal(live.status, 200);
341+
342+ // Only the model endpoints: not the rest of the person's provider account.
343+ for (const [method, path] of [
344+ ["GET", "/v1/files"],
345+ ["POST", "/v1/messages/batches"],
346+ ["DELETE", "/v1/files/file_1"],
347+ ]) {
348+ const seen = modelRequests.length;
349+ const r = await fetch(`${base}/-/model/anthropic${path}`, { method, headers: { "x-api-key": runToken, "content-type": "application/json" }, body: method === "POST" ? "{}" : undefined });
350+ assert.equal(r.status, 403, `${method} ${path}`);
351+ assert.match((await r.json()).error.message, new RegExp(path.replace(/\//g, "\\/")));
352+ assert.equal(modelRequests.length, seen, `${method} ${path} must not reach the provider`);
353+ }
354+
355+ // No public sandbox call reaches it: the person (a repository admin here), their token, the agent.
356+ for (const [who, tok] of [
357+ ["the person in a browser", null],
358+ ["the person's admin token", token],
359+ ["the agent's token", bot],
360+ ]) {
361+ for (const action of ["exec", "read", "write", "close"]) {
362+ const r = await api("POST", `/repos/${owner}/tasks-demo/diffinery/sandboxes/${id}/${action}`, { cmd: "env", path: "README.md", content: "" }, tok);
363+ assert.equal(r.status, 404, `${who} ${action}: ${JSON.stringify(r.json)}`);
364+ }
365+ }
366+ for (const tool of ["sandbox_exec", "sandbox_read_file", "sandbox_write_file", "sandbox_close"]) {
367+ const r = await mcp(tool, { owner, repo: "tasks-demo", sandbox_id: id, command: "env", path: "README.md", content: "" }, bot);
368+ assert.equal(r.isError, true, tool);
369+ assert.match(r.content[0].text, /sandbox not found/, tool);
370+ }
371+
372+ // A plain sandbox's own credential can't start or steer a task either.
373+ const plain = await ok("POST", `/repos/${owner}/tasks-demo/diffinery/sandboxes`, {}, bot);
374+ const cfgPath = join(poolRoot, plain.id, "repo", ".git", "config");
375+ const config = await until("the plain sandbox's git credential", async () => (existsSync(cfgPath) && /extraheader/i.test(readFileSync(cfgPath, "utf8")) ? readFileSync(cfgPath, "utf8") : null));
376+ const basic = /extraheader = Authorization: Basic (\S+)/i.exec(config);
377+ assert.ok(basic, config);
378+ const dst = Buffer.from(basic[1], "base64").toString().split(":").pop();
379+ assert.match(dst, /^dst_/);
380+ assert.equal((await api("POST", R, { goal: "x", runtime: "custom", command: "true", agent: "task-bot" }, dst)).status, 403);
381+ assert.equal((await api("POST", `${R}/${n}/messages`, { body: "x" }, dst)).status, 403);
382+ await ok("POST", `/repos/${owner}/tasks-demo/diffinery/sandboxes/${plain.id}/close`, {}, bot);
383+
384+ // The agent can stop its own task; the run's token stops working.
385+ const stopped = await ok("POST", `${R}/${n}/stop`, {}, bot);
386+ assert.equal(stopped.state, "stopped");
387+ const after_ = await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken, "content-type": "application/json" }, body: "{}" });
388+ assert.equal(after_.status, 401);
389+ const t = await ok("GET", `${R}/${n}`);
390+ assert.equal(t.runs[0].state, "stopped");
391+ assert.match((await ok("GET", `${R}/${n}/transcript`)).map((l) => l.text).join("\n"), /Stopped by task-bot/);
392+});
393+
394+test("losing access to the agent stops the task", async () => {
395+ const me = await ok("GET", "/user");
396+ const owner = me.login;
397+ const R = `/repos/${owner}/tasks-demo/diffinery/tasks`;
398+ const tokFile = join(work, "model-token-3");
399+ const started = await asPerson("POST", R, { goal: "Lose access", runtime: "custom", command: `printf %s "$ANTHROPIC_API_KEY" > ${tokFile}; sleep 120`, agent: "task-bot" });
400+ assert.equal(started.status, 201, JSON.stringify(started.json));
401+ const n = started.json.number;
402+ await until("the agent to start", async () => existsSync(tokFile));
403+ const runToken = readFileSync(tokFile, "utf8");
404+ // Disabling the agent ends its access, and with it the task's.
405+ assert.equal((await api("DELETE", "/diffinery/agents/task-bot")).status, 204);
406+ const refused = await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken, "content-type": "application/json" }, body: "{}" });
407+ assert.equal(refused.status, 403);
408+ const t = await until("the task to stop", async () => {
409+ const x = await ok("GET", `${R}/${n}`);
410+ return x.state !== "running" ? x : null;
411+ });
412+ assert.equal(t.state, "stopped");
413+ assert.equal((await fetch(`${base}/-/model/anthropic/v1/messages`, { method: "POST", headers: { "x-api-key": runToken }, body: "{}" })).status, 401);
414+});
modifiedapps/diffinery/test/unit/email.test.ts+1−1View file
@@ -15,7 +15,7 @@
1515 describe("email templates", () => {
1616 it("has a sample for every kind of email", () => {
1717 const kinds = new Set(samples.map((s) => s.job.kind));
18- for (const k of ["signin_new", "passkey_added", "passkey_removed", "token_created", "token_expiring", "agent_created", "push_bypassed", "push_blocked", "account_status", "invite", "invite_accepted", "welcome", "signin_link", "access_request", "org_membership", "repo_access", "repo_created", "repo_deleted", "repo_transferred", "thread", "run", "test"]) {
18+ for (const k of ["signin_new", "passkey_added", "passkey_removed", "token_created", "token_expiring", "agent_created", "model_key_added", "push_bypassed", "push_blocked", "account_status", "invite", "invite_accepted", "welcome", "signin_link", "access_request", "org_membership", "repo_access", "repo_created", "repo_deleted", "repo_transferred", "thread", "run", "test"]) {
1919 expect(kinds.has(k as MailJob["kind"]), k).toBe(true);
2020 }
2121 });
addedapps/diffinery/test/unit/tasks.test.ts+145−0View file
@@ -1,0 +1,145 @@
1+import { describe, expect, it } from "vitest";
2+import { PROVIDERS } from "../../src/agentic/keys.js";
3+import { proxyAllows, proxyToken } from "../../src/agentic/proxy.js";
4+import { agentEnv, finalMessage, runPrompt, RUNTIMES, taskBranch, taskTitle, transcriptItem, type TranscriptLine } from "../../src/agentic/runtimes.js";
5+import { mayStop } from "../../src/agentic/tasks.js";
6+import type { TaskRow } from "../../src/agentic/ProjectDO.js";
7+
8+const line = (text: string, stream: TranscriptLine["stream"] = "out", seq = 1): TranscriptLine => ({ seq, run: 1, at: "2026-10-13T08:00:00Z", stream, text });
9+
10+describe("task names", () => {
11+ it("makes a branch and a title from the goal", () => {
12+ expect(taskBranch(7, "Fix the Contact sales link on the pricing page, and add a test!")).toBe("agent/7-fix-the-contact-sales-link-on-the");
13+ expect(taskBranch(3, " *** ")).toBe("agent/3");
14+ expect(taskBranch(12, "Düzelt: ödeme")).toBe("agent/12-d-zelt-deme");
15+ expect(taskTitle("First line\nmore detail")).toBe("First line");
16+ expect(taskTitle("x".repeat(150))).toHaveLength(98);
17+ expect(taskTitle("")).toBe("Agent task");
18+ });
19+});
20+
21+describe("runPrompt", () => {
22+ it("carries the goal, and for a follow-up the previous ending and the message", () => {
23+ const first = runPrompt({ goal: "Add a test", number: 4, branch: "agent/4-add-a-test", by: "ayse" });
24+ expect(first).toMatch(/task #4/);
25+ expect(first).toMatch(/Don't push/);
26+ expect(first.endsWith("Goal:\nAdd a test")).toBe(true);
27+ const next = runPrompt({ goal: "Add a test", number: 4, branch: "agent/4-add-a-test", by: "ayse", message: "Also cover the error path", previous: "Added test/foo.test.ts" });
28+ expect(next).toMatch(/The original goal:\nAdd a test/);
29+ expect(next).toMatch(/Where the previous run ended:\nAdded test\/foo.test.ts/);
30+ expect(next.endsWith("New instructions from ayse:\nAlso cover the error path")).toBe(true);
31+ });
32+});
33+
34+describe("agentEnv", () => {
35+ it("points each provider at the proxy and puts only the run's token in the secret environment", () => {
36+ const { env, secret } = agentEnv("https://diffinery.com", ["anthropic", "openai"], "dmt_r_tok");
37+ expect(env.ANTHROPIC_BASE_URL).toBe("https://diffinery.com/-/model/anthropic");
38+ expect(env.OPENAI_BASE_URL).toBe("https://diffinery.com/-/model/openai/v1");
39+ expect(secret).toEqual({ ANTHROPIC_API_KEY: "dmt_r_tok", OPENAI_API_KEY: "dmt_r_tok", CODEX_API_KEY: "dmt_r_tok" });
40+ expect(Object.values(env)).not.toContain("dmt_r_tok");
41+ expect(agentEnv("https://x", ["anthropic"], "t").secret).toEqual({ ANTHROPIC_API_KEY: "t" });
42+ });
43+
44+ it("installs runtimes without any credential, and passes the prompt by environment", () => {
45+ for (const r of Object.values(RUNTIMES)) {
46+ expect(r.install ?? "").not.toMatch(/KEY|TOKEN/);
47+ if (r.id !== "custom") expect(r.command()).toContain('"$DIFFINERY_TASK_PROMPT"');
48+ }
49+ expect(RUNTIMES["claude-code"].needs).toEqual(["anthropic"]);
50+ expect(RUNTIMES.codex.needs).toEqual(["openai"]);
51+ expect(RUNTIMES.custom.command("echo hi")).toBe("echo hi");
52+ });
53+});
54+
55+describe("transcriptItem", () => {
56+ it("reads Claude Code's stream-json", () => {
57+ expect(transcriptItem(line(JSON.stringify({ type: "system", subtype: "init", model: "claude-x", session_id: "s" })))).toEqual([{ kind: "note", body: "Claude Code started (claude-x)." }]);
58+ const msg = { type: "assistant", message: { content: [{ type: "text", text: "Looking at the link." }, { type: "tool_use", name: "Bash", input: { command: "grep -rn contact" } }] } };
59+ expect(transcriptItem(line(JSON.stringify(msg)))).toEqual([
60+ { kind: "text", body: "Looking at the link." },
61+ { kind: "tool", title: "Bash", body: "$ grep -rn contact" },
62+ ]);
63+ const result = { type: "user", message: { content: [{ type: "tool_result", content: [{ type: "text", text: "src/a.ts:1" }], is_error: false }] } };
64+ expect(transcriptItem(line(JSON.stringify(result)))).toEqual([{ kind: "output", body: "src/a.ts:1" }]);
65+ const done = { type: "result", is_error: false, num_turns: 4, total_cost_usd: 0.1234, result: "Fixed it." };
66+ expect(transcriptItem(line(JSON.stringify(done)))).toEqual([{ kind: "result", title: "Done · 4 turns · $0.12", body: "Fixed it." }]);
67+ });
68+
69+ it("reads Codex's --json", () => {
70+ expect(transcriptItem(line(JSON.stringify({ type: "item.completed", item: { type: "agent_message", text: "All done." } })))).toEqual([{ kind: "text", body: "All done." }]);
71+ expect(transcriptItem(line(JSON.stringify({ type: "item.completed", item: { type: "command_execution", command: "npm test", aggregated_output: "ok", exit_code: 1 } })))).toEqual([{ kind: "tool", title: "$ npm test", body: "ok\n(exit 1)" }]);
72+ expect(transcriptItem(line(JSON.stringify({ type: "item.started", item: { type: "command_execution" } })))).toEqual([]);
73+ expect(transcriptItem(line(JSON.stringify({ type: "turn.failed", error: { message: "quota" } })))).toEqual([{ kind: "error", body: "quota" }]);
74+ });
75+
76+ it("shows other lines as they are", () => {
77+ expect(transcriptItem(line("plain output"))).toEqual([{ kind: "output", body: "plain output" }]);
78+ expect(transcriptItem(line("[1,2]"))).toEqual([{ kind: "output", body: "[1,2]" }]);
79+ expect(transcriptItem(line("boom", "err"))).toEqual([{ kind: "error", body: "boom" }]);
80+ expect(transcriptItem(line("Installing.", "note"))).toEqual([{ kind: "note", body: "Installing." }]);
81+ });
82+
83+ it("finds how a run ended", () => {
84+ const lines = [line(JSON.stringify({ type: "assistant", message: { content: [{ type: "text", text: "Working" }] } }), "out", 1), line(JSON.stringify({ type: "result", result: "Changed 2 files." }), "out", 2), line("Pushed.", "note", 3)];
85+ expect(finalMessage(lines)).toBe("Changed 2 files.");
86+ expect(finalMessage([line("nothing")])).toBeNull();
87+ });
88+});
89+
90+describe("model proxy token", () => {
91+ const req = (h: Record<string, string>) => new Request("https://d/-/model/anthropic/v1/messages", { headers: h });
92+ it("takes the run's token from x-api-key or a bearer header, and nothing else", () => {
93+ expect(proxyToken(req({ "x-api-key": "dmt_r_abc" }))).toBe("dmt_r_abc");
94+ expect(proxyToken(req({ authorization: "Bearer dmt_r_abc" }))).toBe("dmt_r_abc");
95+ expect(proxyToken(req({ "x-api-key": "sk-ant-real" }))).toBeNull();
96+ expect(proxyToken(req({ authorization: "Bearer dfy_personal" }))).toBeNull();
97+ expect(proxyToken(req({}))).toBeNull();
98+ });
99+});
100+
101+describe("model proxy endpoints", () => {
102+ it("allows only the model endpoints the runtimes use", () => {
103+ expect(proxyAllows("anthropic", "POST", "/v1/messages")).toBe(true);
104+ expect(proxyAllows("anthropic", "POST", "/v1/messages/count_tokens")).toBe(true);
105+ expect(proxyAllows("anthropic", "GET", "/v1/models")).toBe(true);
106+ expect(proxyAllows("anthropic", "GET", "/v1/models/claude-x")).toBe(true);
107+ expect(proxyAllows("openai", "POST", "/v1/responses")).toBe(true);
108+ expect(proxyAllows("openai", "GET", "/v1/responses/resp_123")).toBe(true);
109+ expect(proxyAllows("openai", "POST", "/v1/chat/completions")).toBe(true);
110+ for (const [p, m, path] of [
111+ ["anthropic", "GET", "/v1/files"],
112+ ["anthropic", "DELETE", "/v1/files/file_1"],
113+ ["anthropic", "POST", "/v1/messages/batches"],
114+ ["anthropic", "GET", "/v1/messages"],
115+ ["anthropic", "POST", "/v1/messages/../files"],
116+ ["openai", "GET", "/v1/files"],
117+ ["openai", "POST", "/v1/batches"],
118+ ["openai", "GET", "/v1/vector_stores"],
119+ ["openai", "POST", "/v1/assistants"],
120+ ["openai", "DELETE", "/v1/responses/resp_123"],
121+ ["openai", "GET", "/v1/responses/resp_1/input_items"],
122+ ["openai", "POST", "/v1/messages"],
123+ ] as const) expect(proxyAllows(p, m, path), `${p} ${m} ${path}`).toBe(false);
124+ });
125+});
126+
127+describe("model keys", () => {
128+ it("checks the key's shape per provider", () => {
129+ expect(PROVIDERS.anthropic.pattern.test(`sk-ant-api03-${"a".repeat(40)}`)).toBe(true);
130+ expect(PROVIDERS.anthropic.pattern.test(`sk-proj-${"a".repeat(40)}`)).toBe(false);
131+ expect(PROVIDERS.openai.pattern.test(`sk-proj-${"a".repeat(40)}`)).toBe(true);
132+ expect(PROVIDERS.openai.pattern.test("sk-short")).toBe(false);
133+ expect(PROVIDERS.openai.pattern.test(`sk-${"a".repeat(30)} extra`)).toBe(false);
134+ });
135+});
136+
137+describe("mayStop", () => {
138+ const t = { person_id: "u-ayse", agent_id: "a-bot" } as TaskRow;
139+ it("is the person, the agent or an admin", () => {
140+ expect(mayStop({ id: "u-ayse", login: "ayse", kind: "user", role: "write" }, t)).toBe(true);
141+ expect(mayStop({ id: "a-bot", login: "bot", kind: "agent", role: "write" }, t)).toBe(true);
142+ expect(mayStop({ id: "u-bob", login: "bob", kind: "user", role: "admin" }, t)).toBe(true);
143+ expect(mayStop({ id: "u-bob", login: "bob", kind: "user", role: "write" }, t)).toBe(false);
144+ });
145+});
modifiedapps/diffinery/wrangler.jsonc+9−4View file
@@ -45,12 +45,15 @@
4545 { "name": "REPO", "class_name": "RepoDO" },
4646 { "name": "RUNNER", "class_name": "RunnerContainer" },
4747 { "name": "SANDBOX", "class_name": "SandboxContainer" },
48- { "name": "IMPORTER", "class_name": "ImportContainer" }
48+ { "name": "IMPORTER", "class_name": "ImportContainer" },
49+ { "name": "PROJECT", "class_name": "ProjectDO" }
4950 ]
5051 },
5152 "migrations": [
5253 { "tag": "v1", "new_sqlite_classes": ["RepoDO", "RunnerContainer", "SandboxContainer"] },
53- { "tag": "v2", "new_sqlite_classes": ["ImportContainer"] }
54+ { "tag": "v2", "new_sqlite_classes": ["ImportContainer"] },
55+ // Agent tasks, one Durable Object per repository (agentic/ProjectDO.ts).
56+ { "tag": "v3", "new_sqlite_classes": ["ProjectDO"] }
5457 ],
5558 "workflows": [
5659 { "name": "diffinery-run", "binding": "RUN_WORKFLOW", "class_name": "RunWorkflow" },
@@ -97,8 +100,10 @@
97100 "bindings": [
98101 { "name": "REPO", "class_name": "RepoDO" },
99102 { "name": "RUNNER", "class_name": "RunnerContainer" },
100- { "name": "SANDBOX", "class_name": "SandboxContainer" },
101- { "name": "IMPORTER", "class_name": "ImportContainer" }
103+ // Sandboxes run in an `agent.mjs --sandbox-pool` process (SANDBOX_POOL_URL), not containers.
104+ { "name": "SANDBOX", "class_name": "SandboxPoolDO" },
105+ { "name": "IMPORTER", "class_name": "ImportContainer" },
106+ { "name": "PROJECT", "class_name": "ProjectDO" }
102107 ]
103108 },
104109 "workflows": [
modifiedcontainers/runner/src/main.ts+17−2View file
@@ -5,6 +5,8 @@
55 * agent.mjs --spec job.json run a JobSpec from a file (reports to spec.apiUrl,
66 * or DIFFINERY_API_URL; `--offline` logs to stdout only)
77 * agent.mjs --sandbox AI-agent sandbox HTTP server on :8080
8+ * agent.mjs --sandbox-pool many sandboxes in one process, for local development and
9+ * end-to-end tests (see sandbox-pool.ts)
810 * agent.mjs --registry-relay HTTPS proxy for the Docker daemon that splits big
911 * image layer uploads to the Diffinery registry (see relay.ts)
1012 *
@@ -17,28 +19,31 @@
1719 import { RunnerApi } from "./api.js";
1820 import { runImportAgent } from "./importer.js";
1921 import { startSandbox } from "./sandbox.js";
22+import { startSandboxPool } from "./sandbox-pool.js";
2023 import { startRelay } from "./relay.js";
2124
2225 interface Args {
2326 spec?: string;
2427 sandbox: boolean;
28+ pool: boolean;
2529 relay: boolean;
2630 offline: boolean;
2731 workRoot?: string;
2832 }
2933
3034 function parseArgs(argv: string[]): Args {
31- const a: Args = { sandbox: false, relay: false, offline: false };
35+ const a: Args = { sandbox: false, pool: false, relay: false, offline: false };
3236 for (let i = 0; i < argv.length; i++) {
3337 const v = argv[i]!;
3438 if (v === "--spec") a.spec = argv[++i];
3539 else if (v.startsWith("--spec=")) a.spec = v.slice(7);
3640 else if (v === "--sandbox") a.sandbox = true;
41+ else if (v === "--sandbox-pool") a.pool = true;
3742 else if (v === "--registry-relay") a.relay = true;
3843 else if (v === "--offline") a.offline = true;
3944 else if (v === "--work-root") a.workRoot = argv[++i];
4045 else if (v === "--help" || v === "-h") {
41- process.stdout.write("usage: agent.mjs [--spec job.json [--offline]] [--sandbox] [--work-root dir]\n");
46+ process.stdout.write("usage: agent.mjs [--spec job.json [--offline]] [--sandbox | --sandbox-pool] [--work-root dir]\n");
4247 process.exit(0);
4348 }
4449 }
@@ -62,6 +67,16 @@
6267 return;
6368 }
6469
70+ if (args.pool) {
71+ const secret = env.DIFFINERY_SANDBOX_POOL_SECRET;
72+ if (!secret) fail("DIFFINERY_SANDBOX_POOL_SECRET is required in --sandbox-pool mode");
73+ const pool = await startSandboxPool({ port: Number(env.DIFFINERY_SANDBOX_PORT ?? env.PORT ?? 8080), secret, workRoot });
74+ process.stdout.write(`[sandbox-pool] listening on :${pool.port}\n`);
75+ const stop = () => void pool.close().then(() => process.exit(0));
76+ process.on("SIGTERM", stop);
77+ process.on("SIGINT", stop);
78+ return;
79+ }
6580 if (args.sandbox) {
6681 const secret = env.DIFFINERY_SANDBOX_SECRET;
6782 if (!secret) fail("DIFFINERY_SANDBOX_SECRET is required in --sandbox mode");
addedcontainers/runner/src/procs.ts+154−0View file
@@ -1,0 +1,154 @@
1+import { spawn, type ChildProcess } from "node:child_process";
2+import { createWriteStream, existsSync, type WriteStream } from "node:fs";
3+import { mkdir, open, stat } from "node:fs/promises";
4+import path from "node:path";
5+import type { Masker } from "./mask.js";
6+import { killTree } from "./util.js";
7+
8+/**
9+ * Background processes in a sandbox: a coding agent's run can last hours, far
10+ * longer than one /exec request. Each process writes stdout and stderr to its
11+ * own files; callers read them from an offset while it runs, and after.
12+ *
13+ * Values passed in `secretEnv` (a person's model API key) reach only that
14+ * process's environment, and are added to the sandbox's masker so no output
15+ * the sandbox returns ever contains them.
16+ */
17+
18+export interface ProcStart {
19+ id: string;
20+ cmd: string;
21+ cwd: string;
22+ env: NodeJS.ProcessEnv;
23+ secretEnv?: Record<string, string>;
24+ timeoutMs?: number;
25+}
26+
27+export interface ProcRead {
28+ id: string;
29+ running: boolean;
30+ exitCode: number | null;
31+ timedOut: boolean;
32+ stdout: string;
33+ stderr: string;
34+ /** Offsets to pass to the next read. */
35+ stdoutOffset: number;
36+ stderrOffset: number;
37+}
38+
39+interface Proc {
40+ child: ChildProcess;
41+ out: string;
42+ err: string;
43+ files: WriteStream[];
44+ exitCode: number | null;
45+ running: boolean;
46+ timedOut: boolean;
47+ timer?: NodeJS.Timeout;
48+ done: Promise<void>;
49+}
50+
51+const ID = /^[A-Za-z0-9_-]{1,64}$/;
52+/** Each read returns at most this much of each stream. */
53+export const MAX_READ = 512 * 1024;
54+
55+export class ProcTable {
56+ private procs = new Map<string, Proc>();
57+
58+ constructor(
59+ private dir: string,
60+ private masker: Masker,
61+ ) {}
62+
63+ async start(o: ProcStart): Promise<{ id: string; pid: number | null }> {
64+ if (!ID.test(o.id)) throw new Error("`id` must be 1-64 letters, digits, - or _");
65+ const prev = this.procs.get(o.id);
66+ if (prev?.running) throw new Error(`process ${o.id} is already running`);
67+ await mkdir(this.dir, { recursive: true });
68+ for (const v of Object.values(o.secretEnv ?? {})) this.masker.add(v);
69+ const out = path.join(this.dir, `${o.id}.out`);
70+ const err = path.join(this.dir, `${o.id}.err`);
71+ const files = [createWriteStream(out, { flags: "w" }), createWriteStream(err, { flags: "w" })];
72+ const child = spawn("bash", ["-c", o.cmd], { cwd: o.cwd, env: { ...o.env, ...o.secretEnv }, detached: true, stdio: ["ignore", "pipe", "pipe"] });
73+ child.stdout!.pipe(files[0]!);
74+ child.stderr!.pipe(files[1]!);
75+ let finish!: () => void;
76+ const p: Proc = { child, out, err, files, exitCode: null, running: true, timedOut: false, done: new Promise((r) => (finish = r)) };
77+ const end = (code: number) => {
78+ if (!p.running) return;
79+ p.running = false;
80+ p.exitCode = p.timedOut ? 124 : code;
81+ if (p.timer) clearTimeout(p.timer);
82+ // Let the pipes drain into the files before readers see the exit.
83+ let open = files.length;
84+ for (const f of files) f.end(() => --open === 0 && finish());
85+ };
86+ child.on("error", (e) => {
87+ files[1]!.write(`${e.message}\n`);
88+ end(127);
89+ });
90+ child.on("close", (code, signal) => end(code ?? (signal ? 128 + 15 : 1)));
91+ if (o.timeoutMs && o.timeoutMs > 0) {
92+ p.timer = setTimeout(() => {
93+ p.timedOut = true;
94+ killTree(child);
95+ }, o.timeoutMs);
96+ }
97+ this.procs.set(o.id, p);
98+ return { id: o.id, pid: child.pid ?? null };
99+ }
100+
101+ async read(id: string, stdoutOffset = 0, stderrOffset = 0): Promise<ProcRead> {
102+ const p = this.procs.get(id);
103+ if (!p) throw new Error(`no process ${id}`);
104+ // Once it has exited, wait for the last output to reach the files.
105+ if (!p.running) await p.done;
106+ const [so, se] = await Promise.all([this.chunk(p.out, stdoutOffset, p.running), this.chunk(p.err, stderrOffset, p.running)]);
107+ return { id, running: p.running, exitCode: p.exitCode, timedOut: p.timedOut, stdout: this.masker.mask(so.text), stderr: this.masker.mask(se.text), stdoutOffset: so.next, stderrOffset: se.next };
108+ }
109+
110+ kill(id: string, signal: "SIGTERM" | "SIGKILL" | "SIGINT" = "SIGTERM"): boolean {
111+ const p = this.procs.get(id);
112+ if (!p?.running) return false;
113+ if (signal === "SIGTERM") killTree(p.child);
114+ else {
115+ try {
116+ process.kill(-p.child.pid!, signal);
117+ } catch {
118+ p.child.kill(signal);
119+ }
120+ }
121+ return true;
122+ }
123+
124+ killAll(): void {
125+ for (const id of this.procs.keys()) this.kill(id);
126+ }
127+
128+ /**
129+ * Bytes from `offset`, ending at a line break while the process runs so a
130+ * masked value is never split between two reads (a single line longer than
131+ * MAX_READ is the exception).
132+ */
133+ private async chunk(file: string, offset: number, running: boolean): Promise<{ text: string; next: number }> {
134+ if (!existsSync(file)) return { text: "", next: offset };
135+ const size = (await stat(file)).size;
136+ if (offset >= size) return { text: "", next: offset };
137+ const len = Math.min(size - offset, MAX_READ);
138+ const fh = await open(file, "r");
139+ try {
140+ const buf = Buffer.alloc(len);
141+ await fh.read(buf, 0, len, offset);
142+ let end = len;
143+ const partial = running || offset + len < size;
144+ if (partial) {
145+ const nl = buf.lastIndexOf(0x0a);
146+ if (nl >= 0) end = nl + 1;
147+ else if (len < MAX_READ) end = 0;
148+ }
149+ return { text: buf.subarray(0, end).toString("utf8"), next: offset + end };
150+ } finally {
151+ await fh.close();
152+ }
153+ }
154+}
addedcontainers/runner/src/sandbox-pool.ts+97−0View file
@@ -1,0 +1,97 @@
1+import { timingSafeEqual } from "node:crypto";
2+import { createServer, request, type IncomingMessage, type Server, type ServerResponse } from "node:http";
3+import type { AddressInfo } from "node:net";
4+import path from "node:path";
5+import { startSandbox, type SandboxHandle } from "./sandbox.js";
6+
7+/**
8+ * Local development and end-to-end tests only: many sandboxes in one process,
9+ * standing in for one Cloudflare Container per sandbox (containers don't run
10+ * under `wrangler dev`). The Worker boots a sandbox with the same environment a
11+ * container would get (`POST /boot`), then reaches it under `/s/<id>/...`.
12+ */
13+
14+export interface PoolOptions {
15+ port: number;
16+ host?: string;
17+ /** Guards /boot; each sandbox still checks its own secret. */
18+ secret: string;
19+ workRoot: string;
20+}
21+
22+const same = (a: string, b: string) => a.length === b.length && timingSafeEqual(Buffer.from(a), Buffer.from(b));
23+
24+function json(res: ServerResponse, status: number, body: unknown): void {
25+ const data = JSON.stringify(body);
26+ res.writeHead(status, { "content-type": "application/json", "content-length": Buffer.byteLength(data) });
27+ res.end(data);
28+}
29+
30+async function readJson(req: IncomingMessage): Promise<Record<string, unknown>> {
31+ const chunks: Buffer[] = [];
32+ for await (const c of req) chunks.push(c as Buffer);
33+ return JSON.parse(Buffer.concat(chunks).toString("utf8") || "{}") as Record<string, unknown>;
34+}
35+
36+export async function startSandboxPool(o: PoolOptions): Promise<{ server: Server; port: number; close(): Promise<void> }> {
37+ const sandboxes = new Map<string, SandboxHandle>();
38+ const server = createServer((req, res) => {
39+ void (async () => {
40+ const url = new URL(req.url ?? "/", "http://pool");
41+ try {
42+ const poolAuth = () => same(/^Bearer\s+(.+)$/i.exec(req.headers.authorization ?? "")?.[1]?.trim() ?? "", o.secret);
43+ if (req.method === "POST" && url.pathname === "/boot") {
44+ if (!poolAuth()) return json(res, 401, { error: "unauthorized" });
45+ const b = await readJson(req);
46+ const id = String(b.id ?? "");
47+ if (!/^[A-Za-z0-9_-]{1,64}$/.test(id) || typeof b.secret !== "string") return json(res, 400, { error: "`id` and `secret` are required" });
48+ const env = (b.env ?? {}) as Record<string, string>;
49+ await sandboxes.get(id)?.close();
50+ const sb = await startSandbox({
51+ port: 0,
52+ host: "127.0.0.1",
53+ secret: b.secret,
54+ workRoot: path.join(o.workRoot, id),
55+ cloneUrl: env.DIFFINERY_CLONE_URL,
56+ ref: env.DIFFINERY_REF,
57+ token: env.DIFFINERY_JOB_TOKEN,
58+ baseEnv: { ...process.env, ...env },
59+ });
60+ sandboxes.set(id, sb);
61+ return json(res, 200, { ok: true });
62+ }
63+ if (req.method === "POST" && url.pathname.startsWith("/shutdown/")) {
64+ if (!poolAuth()) return json(res, 401, { error: "unauthorized" });
65+ const id = url.pathname.slice("/shutdown/".length);
66+ await sandboxes.get(id)?.close();
67+ sandboxes.delete(id);
68+ return json(res, 200, { ok: true });
69+ }
70+ const m = /^\/s\/([A-Za-z0-9_-]+)(\/.*)$/.exec(url.pathname);
71+ const sb = m ? sandboxes.get(m[1]!) : undefined;
72+ if (!m || !sb) return json(res, 404, { error: "sandbox not found" });
73+ // Forward as is; the sandbox checks its own secret.
74+ const up = request({ host: "127.0.0.1", port: sb.port, method: req.method, path: m[2]! + url.search, headers: req.headers }, (r) => {
75+ res.writeHead(r.statusCode ?? 502, r.headers);
76+ r.pipe(res);
77+ });
78+ up.on("error", (e) => json(res, 502, { error: e.message }));
79+ req.pipe(up);
80+ } catch (e) {
81+ json(res, 500, { error: e instanceof Error ? e.message : String(e) });
82+ }
83+ })();
84+ });
85+ await new Promise<void>((resolve, reject) => {
86+ server.once("error", reject);
87+ server.listen(o.port, o.host ?? "127.0.0.1", () => resolve());
88+ });
89+ return {
90+ server,
91+ port: (server.address() as AddressInfo).port,
92+ close: async () => {
93+ await Promise.all([...sandboxes.values()].map((s) => s.close()));
94+ await new Promise<void>((r) => server.close(() => r()));
95+ },
96+ };
97+}
modifiedcontainers/runner/src/sandbox.ts+47−1View file
@@ -6,6 +6,7 @@
66 import path from "node:path";
77 import { basicAuthHeader } from "./builtins/checkout.js";
88 import { Masker } from "./mask.js";
9+import { ProcTable } from "./procs.js";
910 import { exec } from "./util.js";
1011
1112 export interface SandboxOptions {
@@ -128,6 +129,18 @@
128129 process.stderr.write(`[sandbox] ${cloneError}\n`);
129130 });
130131
132+ const procs = new ProcTable(path.join(o.workRoot, ".procs"), masker);
133+ const baseEnv = (): NodeJS.ProcessEnv => {
134+ const env: NodeJS.ProcessEnv = { ...(o.baseEnv ?? process.env), CI: "true", DIFFINERY: "true", DIFFINERY_SANDBOX: "true" };
135+ for (const k of SENSITIVE) delete env[k];
136+ return env;
137+ };
138+ const strings = (v: unknown, field: string): Record<string, string> => {
139+ if (v === undefined || v === null) return {};
140+ if (typeof v !== "object" || Array.isArray(v)) throw new HttpErr(400, `\`${field}\` must be an object of strings`);
141+ return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, String(x)]));
142+ };
143+
131144 const resolvePath = (p: unknown): string => {
132145 if (typeof p !== "string" || !p) throw new HttpErr(400, "`path` is required");
133146 return path.isAbsolute(p) ? p : path.join(repoDir, p);
@@ -192,6 +205,36 @@
192205 send(res, 200, { ok: true, path: p, size: data.length });
193206 return;
194207 }
208+ // Background processes (procs.ts): long agent runs that outlive one request.
209+ case "/proc/start": {
210+ if (typeof body.id !== "string" || typeof body.cmd !== "string" || !body.cmd) throw new HttpErr(400, "`id` and `cmd` are required");
211+ const cwd = body.cwd ? resolvePath(body.cwd) : repoDir;
212+ if (!existsSync(cwd)) throw new HttpErr(400, `cwd does not exist: ${cwd}`);
213+ const env = { ...baseEnv(), ...strings(body.env, "env") };
214+ const timeoutMs = typeof body.timeoutMs === "number" && body.timeoutMs > 0 ? body.timeoutMs : undefined;
215+ try {
216+ send(res, 200, await procs.start({ id: body.id, cmd: body.cmd, cwd, env, secretEnv: strings(body.secretEnv, "secretEnv"), timeoutMs }));
217+ } catch (e) {
218+ throw new HttpErr(409, masker.mask(e instanceof Error ? e.message : String(e)));
219+ }
220+ return;
221+ }
222+ case "/proc/read": {
223+ if (typeof body.id !== "string") throw new HttpErr(400, "`id` is required");
224+ const off = (v: unknown) => (typeof v === "number" && v >= 0 ? v : 0);
225+ try {
226+ send(res, 200, await procs.read(body.id, off(body.stdoutOffset), off(body.stderrOffset)));
227+ } catch (e) {
228+ throw new HttpErr(404, e instanceof Error ? e.message : String(e));
229+ }
230+ return;
231+ }
232+ case "/proc/kill": {
233+ if (typeof body.id !== "string") throw new HttpErr(400, "`id` is required");
234+ const sig = body.signal === "SIGKILL" || body.signal === "SIGINT" ? body.signal : "SIGTERM";
235+ send(res, 200, { killed: procs.kill(body.id, sig) });
236+ return;
237+ }
195238 default:
196239 throw new HttpErr(404, "not found");
197240 }
@@ -212,6 +255,9 @@
212255 port,
213256 repoDir,
214257 ready,
215- close: () => new Promise<void>((r) => server.close(() => r())),
258+ close: () => {
259+ procs.killAll();
260+ return new Promise<void>((r) => server.close(() => r()));
261+ },
216262 };
217263 }
modifiedcontainers/runner/test/sandbox.test.ts+62−0View file
@@ -3,6 +3,7 @@
33 import path from "node:path";
44 import { afterAll, beforeAll, describe, expect, it } from "vitest";
55 import { startSandbox, type SandboxHandle } from "../src/sandbox.js";
6+import { startSandboxPool } from "../src/sandbox-pool.js";
67 import { tmp } from "./helpers.js";
78
89 const SECRET = "sandbox-secret-xyz";
@@ -88,4 +89,65 @@
8889 expect((await call("/files/read", {})).status).toBe(400);
8990 expect((await call("/nope", {})).status).toBe(404);
9091 });
92+
93+ it("runs background processes: env, secret env masked, output from offsets, exit codes", async () => {
94+ // Built at run time so push protection never sees a key-shaped literal in this file.
95+ const KEY = ["sk", "ant", "api03", "secret-key-value"].join("-");
96+ const start = await call("/proc/start", { id: "run1", cmd: 'echo "a=$A"; echo "key=$MODEL_KEY"; echo oops >&2; sleep 0.3; echo done; exit 4', env: { A: "1" }, secretEnv: { MODEL_KEY: KEY } });
97+ expect(start.status).toBe(200);
98+ expect((await call("/proc/start", { id: "run1", cmd: "true" })).status).toBe(409);
99+ let out = "";
100+ let err = "";
101+ let so = 0;
102+ let se = 0;
103+ let r;
104+ for (;;) {
105+ r = await call("/proc/read", { id: "run1", stdoutOffset: so, stderrOffset: se });
106+ out += r.json.stdout;
107+ err += r.json.stderr;
108+ so = r.json.stdoutOffset;
109+ se = r.json.stderrOffset;
110+ if (!r.json.running) break;
111+ await new Promise((x) => setTimeout(x, 50));
112+ }
113+ expect(out).toBe("a=1\nkey=***\ndone\n");
114+ expect(err).toBe("oops\n");
115+ expect(r.json.exitCode).toBe(4);
116+ // The secret is masked everywhere from now on, and other commands don't get it in their environment.
117+ const ex = await call("/exec", { cmd: `echo ${KEY}; echo "env=$MODEL_KEY"` });
118+ expect(ex.json.stdout).toBe("***\nenv=\n");
119+ // A finished process can start again under the same id.
120+ expect((await call("/proc/start", { id: "run1", cmd: "echo again" })).status).toBe(200);
121+ expect((await call("/proc/read", { id: "missing" })).status).toBe(404);
122+ expect((await call("/proc/start", { id: "bad id!", cmd: "true" })).status).toBe(409);
123+ });
124+
125+ it("kills a background process tree and times one out", async () => {
126+ await call("/proc/start", { id: "long", cmd: "sleep 30 & sleep 30; echo never" });
127+ expect((await call("/proc/kill", { id: "long" })).json.killed).toBe(true);
128+ let r;
129+ do r = await call("/proc/read", { id: "long" });
130+ while (r.json.running);
131+ expect(r.json.stdout).toBe("");
132+ await call("/proc/start", { id: "slow", cmd: "sleep 30", timeoutMs: 200 });
133+ do r = await call("/proc/read", { id: "slow" });
134+ while (r.json.running);
135+ expect(r.json).toMatchObject({ exitCode: 124, timedOut: true });
136+ });
91137 });
138+
139+describe("sandbox pool", () => {
140+ it("boots sandboxes on demand and routes calls to each by id", async () => {
141+ const pool = await startSandboxPool({ port: 0, secret: "pool-secret", workRoot: tmp("pool-") });
142+ const p = `http://127.0.0.1:${pool.port}`;
143+ const boot = (auth: string) => fetch(`${p}/boot`, { method: "POST", headers: { authorization: `Bearer ${auth}` }, body: JSON.stringify({ id: "sb1", secret: "s1", env: { DIFFINERY_CLONE_URL: `file://${origin}`, DIFFINERY_REF: "main", FOO: "bar" } }) });
144+ expect((await boot("wrong")).status).toBe(401);
145+ expect((await boot("pool-secret")).status).toBe(200);
146+ const exec = (auth: string) => fetch(`${p}/s/sb1/exec`, { method: "POST", headers: { authorization: `Bearer ${auth}`, "content-type": "application/json" }, body: JSON.stringify({ cmd: 'cat hello.txt; echo "$FOO"' }) });
147+ expect((await exec("wrong")).status).toBe(401);
148+ expect(await (await exec("s1")).json()).toMatchObject({ exitCode: 0, stdout: "hello sandbox\nbar\n" });
149+ expect((await fetch(`${p}/s/nope/exec`, { method: "POST" })).status).toBe(404);
150+ await pool.close();
151+ });
152+});
153+